使用 LangChain 开发应用程序

Posted on In Views: Word count in article: 144k Reading time ≈ 3:59

引言

在这一部分，我们将对 LangChain 展开深入介绍，帮助学习者了解如何使用 LangChain，并基于 LangChain 开发完整的、具备强大能力的应用程序。通过学习本部分，您能够掌握如何使用 LangChain，打通大模型开发的快速通道，结合前面部分学习的基础能力，快速成为一名 LLM 开发者。

本部分的主要内容包括：一些重要概念介绍；存储；模型链；基于文档的问答；评估与代理等。

简介 Introduction

欢迎来到《第三部分：基于 LangChain 开发应用程序》！

本教程由 LangChain 创始人 Harrison Chase 与 DeepLearning.AI 合作推出，旨在帮助大家掌握这个强大的大语言模型应用开发框架。

LangChain的诞生和发展

通过对LLM或大型语言模型给出提示(prompt)，现在可以比以往更快地开发AI应用程序，但是一个应用程序可能需要进行多轮提示以及解析输出。

在此过程有很多重复代码需要编写，基于此需求，哈里森·蔡斯 (Harrison Chase) 创建了LangChain，使开发过程变得更加丝滑。

LangChain开源社区快速发展，贡献者已达数百人，正以惊人的速度更新代码和功能。

课程基本内容

LangChain 是用于构建大模型应用程序的开源框架，有Python和JavaScript两个不同版本的包。LangChain 也是一个开源项目，社区活跃，新增功能快速迭代。LangChain基于模块化组合，有许多单独的组件，可以一起使用或单独使用。

本模块将重点介绍 LangChain 的常用组件：

  • 模型(Models)：集成各种语言模型与向量模型。
  • 提示(Prompts)：向模型提供指令的途径。
  • 索引(Indexes)：提供数据检索功能。
  • 链(Chains)：将组件组合实现端到端应用。
  • 代理(Agents)：扩展模型的推理能力。

通过学习使用这些组件构建链式应用，你将可以快速上手 LangChain，开发出功能强大的语言模型程序。让我们开始探索LangChain的魅力吧！

模型、提示和解析器 Models, Prompts and Output Parsers

本章我们将简要介绍关于 LLM 开发的一些重要概念：模型、提示与解释器。如果您已完整学习过前面两个部分的内容，对这三个概念不会陌生。但是，在 LangChain 的定义中，对这三个概念的定义与使用又与之前有着细微的差别。我们仍然推荐您认真阅读本章，以进一步深入了解 LLM 开发。同时，如果您直接学习本部分的话，本章内容更是重要的基础。

我们首先向您演示直接调用 OpenAI 的场景，以充分说明为什么我们需要使用 LangChain。

直接调用OpenAI

计算1+1

我们来看一个简单的例子，直接使用通过 OpenAI 接口封装的函数get_completion来让模型告诉我们：1+1是什么？

1
2
3
from tool import get_completion

get_completion("1+1是什么？")
1
'1+1等于2。'
用普通话表达海盗邮件

在上述简单示例中，模型gpt-3.5-turbo为我们提供了关于1+1是什么的答案。而现在，我们进入一个更为丰富和复杂的场景。

设想一下，你是一家电商公司的员工。你们的客户中有一位名为海盗A的特殊顾客。他在你们的平台上购买了一个榨汁机，目的是为了制作美味的奶昔。但在制作过程中，由于某种原因，奶昔的盖子突然弹开，导致厨房的墙上洒满了奶昔。想象一下这名海盗的愤怒和挫败之情。他用充满海盗特色的英语方言，给你们的客服中心写了一封邮件：customer_email。

1
2
3
4
5
customer_email = """
嗯呐，我现在可是火冒三丈，我那个搅拌机盖子竟然飞了出去，把我厨房的墙壁都溅上了果汁！
更糟糕的是，保修条款可不包括清理我厨房的费用。
伙计，赶紧给我过来！
"""

在处理来自多元文化背景的顾客时，我们的客服团队可能会遇到某些特殊的语言障碍。如上，我们收到了一名海盗客户的邮件，而他的表达方式对于我们的客服团队来说略显生涩。

为了解决这一挑战，我们设定了以下两个目标：

  • 首先，我们希望模型能够将这封充满海盗方言的邮件翻译成普通话，这样客服团队就能更容易地理解其内容。
  • 其次，在进行翻译时，我们期望模型能采用平和和尊重的语气，这不仅能确保信息准确传达，还能保持与顾客之间的和谐关系。

为了指导模型的输出，我们定义了一个文本表达风格标签，简称为style。

1
2
3
4
# 普通话 + 平静、尊敬的语调
style = """正式普通话 \
用一个平静、尊敬、有礼貌的语调
"""

下一步我们需要做的是将customer_email和style结合起来构造我们的提示:prompt

1
2
3
4
5
6
7
# 要求模型根据给出的语调进行转化
prompt = f"""把由三个反引号分隔的文本\
翻译成一种{style}风格。
文本: ```{customer_email}```
"""

print("提示：", prompt)
1
2
3
4
5
6
7
提示： 
 把由三个反引号分隔的文本翻译成一种正式普通话 用一个平静、尊敬、有礼貌的语调
风格。
文本: ```
嗯呐，我现在可是火冒三丈，我那个搅拌机盖子竟然飞了出去，把我厨房的墙壁都溅上了果汁！
更糟糕的是，保修条款可不包括清理我厨房的费用。
伙计，赶紧给我过来！
1
2
3
4
5
6

经过精心设计的`prompt`已经准备就绪。接下来，只需调用`get_completion`方法，我们就可以获得期望的输出——那封原汁原味的海盗方言邮件，将被翻译成既平和又尊重的正式普通话表达。

```python
response = get_completion(prompt)
print(response)
1
非常抱歉，我现在感到非常愤怒和不满。我的搅拌机盖子竟然飞了出去，导致我厨房的墙壁上都溅满了果汁！更糟糕的是，保修条款并不包括清理我厨房的费用。先生/女士，请您尽快过来处理这个问题！

在进行语言风格转换之后，我们可以观察到明显的变化：原本的用词变得更为正式，那些带有极端情绪的表达得到了替代，并且文本中还加入了表示感激的词汇。

小建议：你可以调整并尝试不同的提示，来探索模型能为你带来怎样的创新性输出。每次尝试都可能为你带来意想不到的惊喜！

通过LangChain使用OpenAI

在前面的小节中，我们使用了封装好的函数get_completion，利用 OpenAI 接口成功地对那封充满方言特色的邮件进行了翻译。得到一封采用平和且尊重的语气、并用标准普通话所写的邮件。接下来，我们将尝试使用 LangChain 解决该问题。

模型

现在让我们尝试使用LangChain来实现相同的功能。从langchain.chat_models导入OpenAI的对话模型ChatOpenAI。 除去OpenAI以外，langchain.chat_models还集成了其他对话模型，更多细节可以查看 Langchain 官方文档(https://python.langchain.com/en/latest/modules/models/chat/integrations.html)。

1
2
3
4
5
6
from langchain.chat_models import ChatOpenAI

# 这里我们将参数temperature设置为0.0，从而减少生成答案的随机性。
# 如果你想要每次得到不一样的有新意的答案，可以尝试调整该参数。
chat = ChatOpenAI(temperature=0.0)
chat
1
ChatOpenAI(cache=None, verbose=False, callbacks=None, callback_manager=None, tags=None, metadata=None, client=<class 'openai.api_resources.chat_completion.ChatCompletion'>, model_name='gpt-3.5-turbo', temperature=0.0, model_kwargs={}, openai_api_key='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', openai_api_base='', openai_organization='', openai_proxy='', request_timeout=None, max_retries=6, streaming=False, n=1, max_tokens=None, tiktoken_model_name=None)

上面的输出显示ChatOpenAI的默认模型为gpt-3.5-turbo

使用提示模版

在前面的例子中，我们通过f字符串把Python表达式的值style和customer_email添加到prompt字符串内。

langchain提供了接口方便快速的构造和使用提示。

用普通话表达海盗邮件

现在我们来看看如何使用langchain来构造提示吧！

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
from langchain.prompts import ChatPromptTemplate

# 首先，构造一个提示模版字符串：`template_string`
template_string = """把由三个反引号分隔的文本\
翻译成一种{style}风格。\
文本: ```{text}```
"""

# 然后，我们调用`ChatPromptTemplatee.from_template()`函数将
# 上面的提示模版字符`template_string`转换为提示模版`prompt_template`

prompt_template = ChatPromptTemplate.from_template(template_string)


print("\n", prompt_template.messages[0].prompt)
1
input_variables=['style', 'text'] output_parser=None partial_variables={} template='把由三个反引号分隔的文本翻译成一种{style}风格。文本: ```{text}```\n' template_format='f-string' validate_template=True

对于给定的customer_style和customer_email, 我们可以使用提示模版prompt_template的format_messages方法生成想要的客户消息customer_messages。

提示模版prompt_template需要两个输入变量： style 和 text。 这里分别对应

  • customer_style: 我们想要的顾客邮件风格
  • customer_email: 顾客的原始邮件文本。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
customer_style = """正式普通话 \
用一个平静、尊敬的语气
"""

customer_email = """
嗯呐，我现在可是火冒三丈，我那个搅拌机盖子竟然飞了出去，把我厨房的墙壁都溅上了果汁！
更糟糕的是，保修条款可不包括清理我厨房的费用。
伙计，赶紧给我过来！
"""

# 使用提示模版
customer_messages = prompt_template.format_messages(
                    style=customer_style,
                    text=customer_email)
# 打印客户消息类型
print("客户消息类型:",type(customer_messages),"\n")

# 打印第一个客户消息类型
print("第一个客户消息的类型:", type(customer_messages[0]),"\n")

# 打印第一个元素
print("第一个客户消息: ", customer_messages[0],"\n")

1
2
3
4
5
6
7
8
客户消息类型:
 <class 'list'> 

第一个客户消息的类型:
 <class 'langchain.schema.messages.HumanMessage'> 

第一个客户消息: 
 content='把由三个反引号分隔的文本翻译成一种正式普通话 用一个平静、尊敬的语气\n风格。文本: ```\n嗯呐，我现在可是火冒三丈，我那个搅拌机盖子竟然飞了出去，把我厨房的墙壁都溅上了果汁！\n更糟糕的是，保修条款可不包括清理我厨房的费用。\n伙计，赶紧给我过来！\n```\n' additional_kwargs={} example=False

可以看出

  • customer_messages变量类型为列表(list)
  • 列表里的元素变量类型为langchain自定义消息(langchain.schema.HumanMessage)。

现在我们可以调用模型部分定义的chat模型来实现转换客户消息风格。

1
2
customer_response = chat(customer_messages)
print(customer_response.content)
1
非常抱歉，我现在感到非常愤怒。我的搅拌机盖子竟然飞了出去，导致我厨房的墙壁上都溅满了果汁！更糟糕的是，保修条款并不包括清理我厨房的费用。伙计，请你尽快过来帮我解决这个问题！
用海盗方言回复邮件

到目前为止，我们已经实现了在前一部分的任务。接下来，我们更进一步，将客服人员回复的消息，转换为海盗风格英语，并确保消息比较有礼貌。 这里，我们可以继续使用起前面构造的的langchain提示模版，来获得我们回复消息提示。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
service_reply = """嘿，顾客， \
保修不包括厨房的清洁费用， \
因为您在启动搅拌机之前 \
忘记盖上盖子而误用搅拌机, \
这是您的错。 \
倒霉！ 再见！
"""

service_style_pirate = """\
一个有礼貌的语气 \
使用海盗风格\
"""
service_messages = prompt_template.format_messages(
    style=service_style_pirate,
    text=service_reply)

print("\n", service_messages[0].content)
1
把由三个反引号分隔的文本翻译成一种一个有礼貌的语气 使用海盗风格风格。文本: ```嘿，顾客， 保修不包括厨房的清洁费用， 因为您在启动搅拌机之前 忘记盖上盖子而误用搅拌机, 这是您的错。 倒霉！ 再见！
1
2
3
4
5

```python
# 调用模型部分定义的chat模型来转换回复消息风格
service_response = chat(service_messages)
print(service_response.content)
1
嘿，尊贵的客户啊，保修可不包括厨房的清洁费用，因为您在启动搅拌机之前竟然忘记盖上盖子而误用了搅拌机，这可是您的疏忽之过啊。真是倒霉透顶啊！祝您一路顺风！
为什么需要提示模版

在应用于比较复杂的场景时，提示可能会非常长并且包含涉及许多细节。使用提示模版，可以让我们更为方便地重复使用设计好的提示。英文版提示2.2.3 给出了作业的提示模版案例：学生们线上学习并提交作业，通过提示来实现对学生的提交的作业的评分。

此外，LangChain还提供了提示模版用于一些常用场景。比如自动摘要、问答、连接到SQL数据库、连接到不同的API。通过使用LangChain内置的提示模版，你可以快速建立自己的大模型应用，而不需要花时间去设计和构造提示。

最后，我们在建立大模型应用时，通常希望模型的输出为给定的格式，比如在输出使用特定的关键词来让输出结构化。英文版提示2.2.3 给出了使用大模型进行链式思考推理结果示例 – 对于问题：What is the elevation range for the area that the eastern sector of the Colorado orogeny extends into? 通过使用LangChain库函数，输出采用”Thought”（思考）、”Action”（行动）、”Observation”（观察）作为链式思考推理的关键词，让输出结构化。

输出解析器
不使用输出解释器提取客户评价中的信息

对于给定的评价customer_review, 我们希望提取信息，并按以下格式输出：

1
2
3
4
5
{
  "gift": False,
  "delivery_days": 5,
  "price_value": "pretty affordable!"
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
from langchain.prompts import ChatPromptTemplate

customer_review = """\
这款吹叶机非常神奇。 它有四个设置：\
吹蜡烛、微风、风城、龙卷风。 \
两天后就到了，正好赶上我妻子的\
周年纪念礼物。 \
我想我的妻子会喜欢它到说不出话来。 \
到目前为止，我是唯一一个使用它的人，而且我一直\
每隔一天早上用它来清理草坪上的叶子。 \
它比其他吹叶机稍微贵一点，\
但我认为它的额外功能是值得的。
"""

review_template = """\
对于以下文本，请从中提取以下信息：

礼物：该商品是作为礼物送给别人的吗？ \
如果是，则回答 是的；如果否或未知，则回答 不是。

交货天数：产品需要多少天\
到达？ 如果没有找到该信息，则输出-1。

价钱：提取有关价值或价格的任何句子，\
并将它们输出为逗号分隔的 Python 列表。

使用以下键将输出格式化为 JSON：
礼物
交货天数
价钱

文本: {text}
"""

prompt_template = ChatPromptTemplate.from_template(review_template)
print("提示模版：", prompt_template)


messages = prompt_template.format_messages(text=customer_review)


chat = ChatOpenAI(temperature=0.0)
response = chat(messages)

print("结果类型:", type(response.content))
print("结果:", response.content)
1
2
3
4
5
6
7
8
9
10
11
12
提示模版： 
 input_variables=['text'] output_parser=None partial_variables={} messages=[HumanMessagePromptTemplate(prompt=PromptTemplate(input_variables=['text'], output_parser=None, partial_variables={}, template='对于以下文本，请从中提取以下信息：\n\n礼物：该商品是作为礼物送给别人的吗？ 如果是，则回答 是的；如果否或未知，则回答 不是。\n\n交货天数：产品需要多少天到达？ 如果没有找到该信息，则输出-1。\n\n价钱：提取有关价值或价格的任何句子，并将它们输出为逗号分隔的 Python 列表。\n\n使用以下键将输出格式化为 JSON：\n礼物\n交货天数\n价钱\n\n文本: {text}\n', template_format='f-string', validate_template=True), additional_kwargs={})]

结果类型:
 <class 'str'>

结果:
 {
  "礼物": "是的",
  "交货天数": 2,
  "价钱": ["它比其他吹叶机稍微贵一点"]
}

可以看出 response.content类型为字符串（str），而并非字典(dict), 如果想要从中更方便的提取信息，我们需要使用Langchain中的输出解释器。

使用输出解析器提取客户评价中的信息

接下来，我们将展示如何使用输出解释器。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
review_template_2 = """\
对于以下文本，请从中提取以下信息：：

礼物：该商品是作为礼物送给别人的吗？
如果是，则回答 是的；如果否或未知，则回答 不是。

交货天数：产品到达需要多少天？ 如果没有找到该信息，则输出-1。

价钱：提取有关价值或价格的任何句子，并将它们输出为逗号分隔的 Python 列表。

文本: {text}

{format_instructions}
"""

prompt = ChatPromptTemplate.from_template(template=review_template_2)

from langchain.output_parsers import ResponseSchema
from langchain.output_parsers import StructuredOutputParser

gift_schema = ResponseSchema(name="礼物",
                             description="这件物品是作为礼物送给别人的吗？\
                            如果是，则回答 是的，\
                            如果否或未知，则回答 不是。")

delivery_days_schema = ResponseSchema(name="交货天数",
                                      description="产品需要多少天才能到达？\
                                      如果没有找到该信息，则输出-1。")

price_value_schema = ResponseSchema(name="价钱",
                                    description="提取有关价值或价格的任何句子，\
                                    并将它们输出为逗号分隔的 Python 列表")


response_schemas = [gift_schema, 
                    delivery_days_schema,
                    price_value_schema]
output_parser = StructuredOutputParser.from_response_schemas(response_schemas)
format_instructions = output_parser.get_format_instructions()
print("输出格式规定：",format_instructions)
1
2
3
4
5
6
7
8
9
输出格式规定： 
 The output should be a markdown code snippet formatted in the following schema, including the leading and trailing "```json" and "```":

```json
{
	"礼物": string  // 这件物品是作为礼物送给别人的吗？                            如果是，则回答 是的，                            如果否或未知，则回答 不是。
	"交货天数": string  // 产品需要多少天才能到达？                                      如果没有找到该信息，则输出-1。
	"价钱": string  // 提取有关价值或价格的任何句子，                                    并将它们输出为逗号分隔的 Python 列表
}
1
2
3
4

```python
messages = prompt.format_messages(text=customer_review, format_instructions=format_instructions)
print("第一条客户消息:",messages[0].content)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
第一条客户消息:
 对于以下文本，请从中提取以下信息：：

礼物：该商品是作为礼物送给别人的吗？
如果是，则回答 是的；如果否或未知，则回答 不是。

交货天数：产品到达需要多少天？ 如果没有找到该信息，则输出-1。

价钱：提取有关价值或价格的任何句子，并将它们输出为逗号分隔的 Python 列表。

文本: 这款吹叶机非常神奇。 它有四个设置：吹蜡烛、微风、风城、龙卷风。 两天后就到了，正好赶上我妻子的周年纪念礼物。 我想我的妻子会喜欢它到说不出话来。 到目前为止，我是唯一一个使用它的人，而且我一直每隔一天早上用它来清理草坪上的叶子。 它比其他吹叶机稍微贵一点，但我认为它的额外功能是值得的。


The output should be a markdown code snippet formatted in the following schema, including the leading and trailing "```json" and "```":

```json
{
	"礼物": string  // 这件物品是作为礼物送给别人的吗？                            如果是，则回答 是的，                            如果否或未知，则回答 不是。
	"交货天数": string  // 产品需要多少天才能到达？                                      如果没有找到该信息，则输出-1。
	"价钱": string  // 提取有关价值或价格的任何句子，                                    并将它们输出为逗号分隔的 Python 列表
}
1
2
3
4
5
6

```python
response = chat(messages)

print("结果类型:", type(response.content))
print("结果:", response.content)
1
2
3
4
5
6
7
8
9
10
结果类型:
 <class 'str'>

结果:
 ```json
{
	"礼物": "不是",
	"交货天数": "两天后就到了",
	"价钱": "它比其他吹叶机稍微贵一点"
}
1
2
3
4
5
6

```python
output_dict = output_parser.parse(response.content)

print("解析后的结果类型:", type(output_dict))
print("解析后的结果:", output_dict)
1
2
3
4
5
解析后的结果类型:
 <class 'dict'>

解析后的结果:
 {'礼物': '不是', '交货天数': '两天后就到了', '价钱': '它比其他吹叶机稍微贵一点'}

output_dict类型为字典(dict), 可直接使用get方法。这样的输出更方便下游任务的处理。

英文版提示

用美式英语表达海盗邮件
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
customer_email = """
Arrr, I be fuming that me blender lid \
flew off and splattered me kitchen walls \
with smoothie! And to make matters worse,\
the warranty don't cover the cost of \
cleaning up me kitchen. I need yer help \
right now, matey!
"""

# 美式英语 + 平静、尊敬的语调
style = """American English \
in a calm and respectful tone
"""


# 要求模型根据给出的语调进行转化
prompt = f"""Translate the text \
that is delimited by triple backticks 
into a style that is {style}.
text: ```{customer_email}```
"""

print("提示：", prompt)

response = get_completion(prompt)

print("美式英语表达的海盗邮件: ", response)
1
2
3
4
5
6
提示： 
 Translate the text that is delimited by triple backticks 
into a style that is American English in a calm and respectful tone
.
text: ```
Arrr, I be fuming that me blender lid flew off and splattered me kitchen walls with smoothie! And to make matters worse,the warranty don't cover the cost of cleaning up me kitchen. I need yer help right now, matey!
美式英语表达的海盗邮件:  
 I am quite frustrated that my blender lid flew off and made a mess of my kitchen walls with smoothie! To add to my frustration, the warranty does not cover the cost of cleaning up my kitchen. I kindly request your assistance at this moment, my friend.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

###### 用标准美式英语表达海盗邮件

```python
from langchain.prompts import ChatPromptTemplate

template_string = """Translate the text \
that is delimited by triple backticks \
into a style that is {style}. \
text: ```{text}```
"""


prompt_template = ChatPromptTemplate.from_template(template_string)

print("提示模版中的第一个提示：", prompt_template.messages[0].prompt)

customer_style = """American English \
in a calm and respectful tone
"""


customer_email = """
Arrr, I be fuming that me blender lid \
flew off and splattered me kitchen walls \
with smoothie! And to make matters worse, \
the warranty don't cover the cost of \
cleaning up me kitchen. I need yer help \
right now, matey!
"""

customer_messages = prompt_template.format_messages(
                    style=customer_style,
                    text=customer_email)


print("用提示模版中生成的第一条客户消息：", customer_messages[0])
1
2
3
4
5
提示模版中的第一个提示： 
 input_variables=['style', 'text'] output_parser=None partial_variables={} template='Translate the text that is delimited by triple backticks into a style that is {style}. text: ```{text}```\n' template_format='f-string' validate_template=True

用提示模版中生成的第一条客户消息： 
 content="Translate the text that is delimited by triple backticks into a style that is American English in a calm and respectful tone\n. text: ```\nArrr, I be fuming that me blender lid flew off and splattered me kitchen walls with smoothie! And to make matters worse, the warranty don't cover the cost of cleaning up me kitchen. I need yer help right now, matey!\n```\n" additional_kwargs={} example=False
用海盗方言回复邮件
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
service_reply = """Hey there customer, \
the warranty does not cover \
cleaning expenses for your kitchen \
because it's your fault that \
you misused your blender \
by forgetting to put the lid on before \
starting the blender. \
Tough luck! See ya!
"""
service_style_pirate = """\
a polite tone \
that speaks in English Pirate\
"""

service_messages = prompt_template.format_messages(
    style=service_style_pirate,
    text=service_reply)

print("提示模版中的第一条客户消息内容：", service_messages[0].content)

service_response = chat(service_messages)
print("模型得到的回复邮件：", service_response.content)
1
2
提示模版中的第一条客户消息内容： 
 Translate the text that is delimited by triple backticks into a style that is a polite tone that speaks in English Pirate. text: ```Hey there customer, the warranty does not cover cleaning expenses for your kitchen because it's your fault that you misused your blender by forgetting to put the lid on before starting the blender. Tough luck! See ya!
模型得到的回复内容： 
 Ahoy there, matey! I regret to inform ye that the warranty be not coverin' the costs o' cleanin' yer galley, as 'tis yer own fault fer misusin' yer blender by forgettin' to secure the lid afore startin' it. Aye, tough luck, me heartie! Fare thee well!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

###### 不使用输出解释器提取客户评价中的信息

```python
customer_review = """\
This leaf blower is pretty amazing.  It has four settings:\
candle blower, gentle breeze, windy city, and tornado. \
It arrived in two days, just in time for my wife's \
anniversary present. \
I think my wife liked it so much she was speechless. \
So far I've been the only one using it, and I've been \
using it every other morning to clear the leaves on our lawn. \
It's slightly more expensive than the other leaf blowers \
out there, but I think it's worth it for the extra features.
"""

review_template = """\
For the following text, extract the following information:

gift: Was the item purchased as a gift for someone else? \
Answer True if yes, False if not or unknown.

delivery_days: How many days did it take for the product \
to arrive? If this information is not found, output -1.

price_value: Extract any sentences about the value or price,\
and output them as a comma separated Python list.

Format the output as JSON with the following keys:
gift
delivery_days
price_value

text: {text}
"""

from langchain.prompts import ChatPromptTemplate
prompt_template = ChatPromptTemplate.from_template(review_template)

print("提示模版：",prompt_template)

messages = prompt_template.format_messages(text=customer_review)

chat = ChatOpenAI(temperature=0.0)
response = chat(messages)
print("回复内容：",response.content)
1
2
3
4
5
6
7
8
9
提示模版： 
 input_variables=['text'] output_parser=None partial_variables={} messages=[HumanMessagePromptTemplate(prompt=PromptTemplate(input_variables=['text'], output_parser=None, partial_variables={}, template='For the following text, extract the following information:\n\ngift: Was the item purchased as a gift for someone else? Answer True if yes, False if not or unknown.\n\ndelivery_days: How many days did it take for the product to arrive? If this information is not found, output -1.\n\nprice_value: Extract any sentences about the value or price,and output them as a comma separated Python list.\n\nFormat the output as JSON with the following keys:\ngift\ndelivery_days\nprice_value\n\ntext: {text}\n', template_format='f-string', validate_template=True), additional_kwargs={})]

回复内容： 
 {
  "gift": false,
  "delivery_days": 2,
  "price_value": ["It's slightly more expensive than the other leaf blowers out there, but I think it's worth it for the extra features."]
}
使用输出解析器提取客户评价中的信息
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
review_template_2 = """\
For the following text, extract the following information:

gift: Was the item purchased as a gift for someone else? \
Answer True if yes, False if not or unknown.

delivery_days: How many days did it take for the product\
to arrive? If this information is not found, output -1.

price_value: Extract any sentences about the value or price,\
and output them as a comma separated Python list.

text: {text}

{format_instructions}
"""

prompt = ChatPromptTemplate.from_template(template=review_template_2)

from langchain.output_parsers import ResponseSchema
from langchain.output_parsers import StructuredOutputParser

gift_schema = ResponseSchema(name="gift",
                             description="Was the item purchased\
                             as a gift for someone else? \
                             Answer True if yes,\
                             False if not or unknown.")

delivery_days_schema = ResponseSchema(name="delivery_days",
                                      description="How many days\
                                      did it take for the product\
                                      to arrive? If this \
                                      information is not found,\
                                      output -1.")

price_value_schema = ResponseSchema(name="price_value",
                                    description="Extract any\
                                    sentences about the value or \
                                    price, and output them as a \
                                    comma separated Python list.")


response_schemas = [gift_schema, 
                    delivery_days_schema,
                    price_value_schema]
output_parser = StructuredOutputParser.from_response_schemas(response_schemas)
format_instructions = output_parser.get_format_instructions()
print(format_instructions)

messages = prompt.format_messages(text=customer_review, format_instructions=format_instructions)

print("提示消息：", messages[0].content)

response = chat(messages)
print("回复内容：",response.content)

output_dict = output_parser.parse(response.content)
print("解析后的结果类型:", type(output_dict))
print("解析后的结果:", output_dict)
1
2
3
4
5
6
7
8
The output should be a markdown code snippet formatted in the following schema, including the leading and trailing "```json" and "```":

```json
{
	"gift": string  // Was the item purchased                             as a gift for someone else?                              Answer True if yes,                             False if not or unknown.
	"delivery_days": string  // How many days                                      did it take for the product                                      to arrive? If this                                       information is not found,                                      output -1.
	"price_value": string  // Extract any                                    sentences about the value or                                     price, and output them as a                                     comma separated Python list.
}
提示消息： 
 For the following text, extract the following information:

gift: Was the item purchased as a gift for someone else? Answer True if yes, False if not or unknown.

delivery_days: How many days did it take for the productto arrive? If this information is not found, output -1.

price_value: Extract any sentences about the value or price,and output them as a comma separated Python list.

text: This leaf blower is pretty amazing.  It has four settings:candle blower, gentle breeze, windy city, and tornado. It arrived in two days, just in time for my wife's anniversary present. I think my wife liked it so much she was speechless. So far I've been the only one using it, and I've been using it every other morning to clear the leaves on our lawn. It's slightly more expensive than the other leaf blowers out there, but I think it's worth it for the extra features.


The output should be a markdown code snippet formatted in the following schema, including the leading and trailing "```json" and "```":


1
2
3
4
5
{
	"gift": string  // Was the item purchased                             as a gift for someone else?                              Answer True if yes,                             False if not or unknown.
	"delivery_days": string  // How many days                                      did it take for the product                                      to arrive? If this                                       information is not found,                                      output -1.
	"price_value": string  // Extract any                                    sentences about the value or                                     price, and output them as a                                     comma separated Python list.
}


1

回复内容： 
 
1
2
3
4
5
{
	"gift": false,
	"delivery_days": "2",
	"price_value": "It's slightly more expensive than the other leaf blowers out there, but I think it's worth it for the extra features."
}


解析后的结果类型:
 <class 'dict'>

解析后的结果:
 {'gift': False, 'delivery_days': '2', 'price_value': "It's slightly more expensive than the other leaf blowers out there, but I think it's worth it for the extra features."}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

### 储存 Memory

在与语言模型交互时，你可能已经注意到一个关键问题：它们并不记忆你之前的交流内容，这在我们构建一些应用程序（如聊天机器人）的时候，带来了很大的挑战，使得对话似乎缺乏真正的连续性。因此，在本节中我们将介绍 LangChain 中的储存模块，即如何将先前的对话嵌入到语言模型中的，使其具有连续对话的能力。

当使用 LangChain 中的储存(Memory)模块时，它旨在保存、组织和跟踪整个对话的历史，从而为用户和模型之间的交互提供连续的上下文。

LangChain 提供了多种储存类型。其中，缓冲区储存允许保留最近的聊天消息，摘要储存则提供了对整个对话的摘要。实体储存则允许在多轮对话中保留有关特定实体的信息。这些记忆组件都是模块化的，可与其他组件组合使用，从而增强机器人的对话管理能力。储存模块可以通过简单的 API 调用来访问和更新，允许开发人员更轻松地实现对话历史记录的管理和维护。

此次课程主要介绍其中四种储存模块，其他模块可查看文档学习。
- 对话缓存储存 (ConversationBufferMemory）
- 对话缓存窗口储存 (ConversationBufferWindowMemory）
- 对话令牌缓存储存 (ConversationTokenBufferMemory）
- 对话摘要缓存储存 (ConversationSummaryBufferMemory）

在 LangChain 中，储存指的是大语言模型（LLM）的短期记忆。为什么是短期记忆？那是因为LLM训练好之后 (获得了一些长期记忆)，它的参数便不会因为用户的输入而发生改变。当用户与训练好的LLM进行对话时，LLM 会暂时记住用户的输入和它已经生成的输出，以便预测之后的输出，而模型输出完毕后，它便会“遗忘”之前用户的输入和它的输出。因此，之前的这些信息只能称作为 LLM 的短期记忆。

为了延长 LLM 短期记忆的保留时间，则需要借助一些外部储存方式来进行记忆，以便在用户与 LLM 对话中，LLM 能够尽可能的知道用户与它所进行的历史对话信息。

#### 对话缓存储存

##### 初始化对话模型

让我们先来初始化对话模型。

```python
from langchain.chains import ConversationChain
from langchain.chat_models import ChatOpenAI
from langchain.memory import ConversationBufferMemory

# 这里我们将参数temperature设置为0.0，从而减少生成答案的随机性。
# 如果你想要每次得到不一样的有新意的答案，可以尝试增大该参数。
llm = ChatOpenAI(temperature=0.0)  
memory = ConversationBufferMemory()


# 新建一个 ConversationChain Class 实例
# verbose参数设置为True时，程序会输出更详细的信息，以提供更多的调试或运行时信息。
# 相反，当将verbose参数设置为False时，程序会以更简洁的方式运行，只输出关键的信息。
conversation = ConversationChain(llm=llm, memory = memory, verbose=True )
第一轮对话

当我们运行预测(predict)时，生成了一些提示，如下所见，他说“以下是人类和 AI 之间友好的对话，AI 健谈“等等，这实际上是 LangChain 生成的提示，以使系统进行希望和友好的对话，并且必须保存对话，并提示了当前已完成的模型链。

1
conversation.predict(input="你好, 我叫皮皮鲁")
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
> Entering new  chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.

Current conversation:

Human: 你好, 我叫皮皮鲁
AI:

> Finished chain.





'你好，皮皮鲁！很高兴认识你。我是一个AI助手，可以回答你的问题和提供帮助。有什么我可以帮你的吗？'
第二轮对话

当我们进行第二轮对话时，它会保留上面的提示

1
conversation.predict(input="1+1等于多少？")
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
> Entering new ConversationChain chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.

Current conversation:
Human: 你好, 我叫皮皮鲁
AI: 你好，皮皮鲁！很高兴认识你。我是一个AI助手，可以回答你的问题和提供帮助。有什么我可以帮你的吗？
Human: 1+1等于多少？
AI:

> Finished chain.





'1+1等于2。'
第三轮对话

为了验证他是否记忆了前面的对话内容，我们让他回答前面已经说过的内容（我的名字），可以看到他确实输出了正确的名字，因此这个对话链随着往下进行会越来越长。

1
conversation.predict(input="我叫什么名字？")
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
> Entering new ConversationChain chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.

Current conversation:
Human: 你好, 我叫皮皮鲁
AI: 你好，皮皮鲁！很高兴认识你。我是一个AI助手，可以回答你的问题和提供帮助。有什么我可以帮你的吗？
Human: 1+1等于多少？
AI: 1+1等于2。
Human: 我叫什么名字？
AI:

> Finished chain.





'你叫皮皮鲁。'
查看储存缓存

储存缓存(buffer)，即储存了当前为止所有的对话信息

1
print(memory.buffer) 
1
2
3
4
5
6
Human: 你好, 我叫皮皮鲁
AI: 你好，皮皮鲁！很高兴认识你。我是一个AI助手，可以回答你的问题和提供帮助。有什么我可以帮你的吗？
Human: 1+1等于多少？
AI: 1+1等于2。
Human: 我叫什么名字？
AI: 你叫皮皮鲁。

也可以通过load_memory_variables({})打印缓存中的历史消息。这里的{}是一个空字典，有一些更高级的功能，使用户可以使用更复杂的输入，具体可以通过 LangChain 的官方文档查询更高级的用法。

1
print(memory.load_memory_variables({}))
1
{'history': 'Human: 你好, 我叫皮皮鲁\nAI: 你好，皮皮鲁！很高兴认识你。我是一个AI助手，可以回答你的问题和提供帮助。有什么我可以帮你的吗？\nHuman: 1+1等于多少？\nAI: 1+1等于2。\nHuman: 我叫什么名字？\nAI: 你叫皮皮鲁。'}
直接添加内容到储存缓存

我们可以使用save_context来直接添加内容到buffer中。

1
2
3
memory = ConversationBufferMemory()
memory.save_context({"input": "你好，我叫皮皮鲁"}, {"output": "你好啊，我叫鲁西西"})
memory.load_memory_variables({})
1
{'history': 'Human: 你好，我叫皮皮鲁\nAI: 你好啊，我叫鲁西西'}

继续添加新的内容

1
2
memory.save_context({"input": "很高兴和你成为朋友！"}, {"output": "是的，让我们一起去冒险吧！"})
memory.load_memory_variables({})
1
{'history': 'Human: 你好，我叫皮皮鲁\nAI: 你好啊，我叫鲁西西\nHuman: 很高兴和你成为朋友！\nAI: 是的，让我们一起去冒险吧！'}

可以看到对话历史都保存下来了！

当我们在使用大型语言模型进行聊天对话时，大型语言模型本身实际上是无状态的。语言模型本身并不记得到目前为止的历史对话。每次调用API结点都是独立的。储存(Memory)可以储存到目前为止的所有术语或对话，并将其输入或附加上下文到LLM中用于生成输出。如此看起来就好像它在进行下一轮对话的时候，记得之前说过什么。

对话缓存窗口储存

随着对话变得越来越长，所需的内存量也变得非常长。将大量的tokens发送到LLM的成本，也会变得更加昂贵，这也就是为什么API的调用费用，通常是基于它需要处理的tokens数量而收费的。

针对以上问题，LangChain也提供了几种方便的储存方式来保存历史对话。其中，对话缓存窗口储存只保留一个窗口大小的对话。它只使用最近的n次交互。这可以用于保持最近交互的滑动窗口，以便缓冲区不会过大。

添加两轮对话到窗口储存

我们先来尝试一下使用ConversationBufferWindowMemory来实现交互的滑动窗口，并设置k=1，表示只保留一个对话记忆。接下来我们手动添加两轮对话到窗口储存中，然后查看储存的对话。

1
2
3
4
5
6
7
from langchain.memory import ConversationBufferWindowMemory

# k=1表明只保留一个对话记忆
memory = ConversationBufferWindowMemory(k=1)  
memory.save_context({"input": "你好，我叫皮皮鲁"}, {"output": "你好啊，我叫鲁西西"})
memory.save_context({"input": "很高兴和你成为朋友！"}, {"output": "是的，让我们一起去冒险吧！"})
memory.load_memory_variables({})
1
{'history': 'Human: 很高兴和你成为朋友！\nAI: 是的，让我们一起去冒险吧！'}

通过结果，我们可以看到窗口储存中只有最后一轮的聊天记录。

在对话链中应用窗口储存

接下来，让我们来看看如何在ConversationChain中运用ConversationBufferWindowMemory吧！

1
2
3
4
5
6
7
8
9
10
11
12
llm = ChatOpenAI(temperature=0.0)
memory = ConversationBufferWindowMemory(k=1)
conversation = ConversationChain(llm=llm, memory=memory, verbose=False  )

print("第一轮对话：")
print(conversation.predict(input="你好, 我叫皮皮鲁"))

print("第二轮对话：")
print(conversation.predict(input="1+1等于多少？"))

print("第三轮对话：")
print(conversation.predict(input="我叫什么名字？"))
1
2
3
4
5
6
第一轮对话：
你好，皮皮鲁！很高兴认识你。我是一个AI助手，可以回答你的问题和提供帮助。有什么我可以帮你的吗？
第二轮对话：
1+1等于2。
第三轮对话：
很抱歉，我无法知道您的名字。

注意此处！由于这里用的是一个窗口的记忆，因此只能保存一轮的历史消息，因此AI并不能知道你第一轮对话中提到的名字，他最多只能记住上一轮（第二轮）的对话信息

对话字符缓存储存

使用对话字符缓存记忆，内存将限制保存的token数量。如果字符数量超出指定数目，它会切掉这个对话的早期部分
以保留与最近的交流相对应的字符数量，但不超过字符限制。

添加对话到Token缓存储存,限制token数量，进行测试

1
2
3
4
5
6
from langchain.llms import OpenAI
from langchain.memory import ConversationTokenBufferMemory
memory = ConversationTokenBufferMemory(llm=llm, max_token_limit=30)
memory.save_context({"input": "朝辞白帝彩云间，"}, {"output": "千里江陵一日还。"})
memory.save_context({"input": "两岸猿声啼不住，"}, {"output": "轻舟已过万重山。"})
memory.load_memory_variables({})
1
{'history': 'AI: 轻舟已过万重山。'}

ChatGPT 使用一种基于字节对编码（Byte Pair Encoding，BPE）的方法来进行 tokenization （将输入文本拆分为token）。BPE 是一种常见的 tokenization 技术，它将输入文本分割成较小的子词单元。 OpenAI 在其官方 GitHub 上公开了一个最新的开源 Python 库 tiktoken(https://github.com/openai/tiktoken)，这个库主要是用来计算 tokens 数量的。相比较 HuggingFace 的 tokenizer ，其速度提升了好几倍。
具体 token 计算方式,特别是汉字和英文单词的 token 区别，具体可参考知乎文章(https://www.zhihu.com/question/594159910)。

对话摘要缓存储存

对话摘要缓存储存，使用 LLM 对到目前为止历史对话自动总结摘要，并将其保存下来。

使用对话摘要缓存储存

我们创建了一个长字符串，其中包含某人的日程安排。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
from langchain.chains import ConversationChain
from langchain.chat_models import ChatOpenAI
from langchain.memory import ConversationSummaryBufferMemory

# 创建一个长字符串
schedule = "在八点你和你的产品团队有一个会议。 \
你需要做一个PPT。 \
上午9点到12点你需要忙于LangChain。\
Langchain是一个有用的工具，因此你的项目进展的非常快。\
中午，在意大利餐厅与一位开车来的顾客共进午餐 \
走了一个多小时的路程与你见面，只为了解最新的 AI。 \
确保你带了笔记本电脑可以展示最新的 LLM 样例."

llm = ChatOpenAI(temperature=0.0)
memory = ConversationSummaryBufferMemory(llm=llm, max_token_limit=100)
memory.save_context({"input": "你好，我叫皮皮鲁"}, {"output": "你好啊，我叫鲁西西"})
memory.save_context({"input": "很高兴和你成为朋友！"}, {"output": "是的，让我们一起去冒险吧！"})
memory.save_context({"input": "今天的日程安排是什么？"}, {"output": f"{schedule}"})

print(memory.load_memory_variables({})['history'])
1
System: The human introduces themselves as Pipilu and the AI introduces themselves as Luxixi. They express happiness at becoming friends and decide to go on an adventure together. The human asks about the schedule for the day. The AI informs them that they have a meeting with their product team at 8 o'clock and need to prepare a PowerPoint presentation. From 9 am to 12 pm, they will be busy with LangChain, a useful tool that helps their project progress quickly. At noon, they will have lunch with a customer who has driven for over an hour just to learn about the latest AI. The AI advises the human to bring their laptop to showcase the latest LLM samples.
基于对话摘要缓存储存的对话链

基于上面的对话摘要缓存储存，我们新建一个对话链。

1
2
conversation = ConversationChain(llm=llm, memory=memory, verbose=True)
conversation.predict(input="展示什么样的样例最好呢？")
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
> Entering new ConversationChain chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.

Current conversation:
System: The human introduces themselves as Pipilu and the AI introduces themselves as Luxixi. They express happiness at becoming friends and decide to go on an adventure together. The human asks about the schedule for the day. The AI informs them that they have a meeting with their product team at 8 o'clock and need to prepare a PowerPoint presentation. From 9 am to 12 pm, they will be busy with LangChain, a useful tool that helps their project progress quickly. At noon, they will have lunch with a customer who has driven for over an hour just to learn about the latest AI. The AI advises the human to bring their laptop to showcase the latest LLM samples.
Human: 展示什么样的样例最好呢？
AI:

> Finished chain.





'展示一些具有多样性和创新性的样例可能是最好的选择。你可以展示一些不同领域的应用，比如自然语言处理、图像识别、语音合成等。另外，你也可以展示一些具有实际应用价值的样例，比如智能客服、智能推荐等。总之，选择那些能够展示出我们AI技术的强大和多样性的样例会给客户留下深刻的印象。'
1
print(memory.load_memory_variables({}))  # 摘要记录更新了
1
{'history': "System: The human introduces themselves as Pipilu and the AI introduces themselves as Luxixi. They express happiness at becoming friends and decide to go on an adventure together. The human asks about the schedule for the day. The AI informs them that they have a meeting with their product team at 8 o'clock and need to prepare a PowerPoint presentation. From 9 am to 12 pm, they will be busy with LangChain, a useful tool that helps their project progress quickly. At noon, they will have lunch with a customer who has driven for over an hour just to learn about the latest AI. The AI advises the human to bring their laptop to showcase the latest LLM samples. The human asks what kind of samples would be best to showcase. The AI suggests that showcasing diverse and innovative samples would be the best choice. They recommend demonstrating applications in different fields such as natural language processing, image recognition, and speech synthesis. Additionally, they suggest showcasing practical examples like intelligent customer service and personalized recommendations to impress the customer with the power and versatility of their AI technology."}

通过对比上一次输出，发现摘要记录更新了，添加了最新一次对话的内容总结。

英文版提示

对话缓存储存
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
from langchain.chains import ConversationChain
from langchain.chat_models import ChatOpenAI
from langchain.memory import ConversationBufferMemory


llm = ChatOpenAI(temperature=0.0)  
memory = ConversationBufferMemory()
conversation = ConversationChain(llm=llm, memory = memory, verbose=True )

print("第一轮对话：")
conversation.predict(input="Hi, my name is Andrew")

print("第二轮对话：")
conversation.predict(input="What is 1+1?")

print("第三轮对话：")
conversation.predict(input="What is my name?")
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
第一轮对话：


> Entering new  chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.

Current conversation:

Human: Hi, my name is Andrew
AI:

> Finished chain.
第二轮对话：


> Entering new  chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.

Current conversation:
Human: Hi, my name is Andrew
AI: Hello Andrew! It's nice to meet you. How can I assist you today?
Human: What is 1+1?
AI:

> Finished chain.
第三轮对话：


> Entering new  chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.

Current conversation:
Human: Hi, my name is Andrew
AI: Hello Andrew! It's nice to meet you. How can I assist you today?
Human: What is 1+1?
AI: 1+1 is equal to 2.
Human: What is my name?
AI:

> Finished chain.





'Your name is Andrew.'
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
print("查看储存缓存方式一：")
print(memory.buffer) 

print("查看储存缓存方式二：")
print(memory.load_memory_variables({}))

print("向缓存区添加指定对话的输入输出, 并查看")
memory = ConversationBufferMemory()  # 新建一个空的对话缓存记忆
memory.save_context({"input": "Hi"}, {"output": "What's up"})  # 向缓存区添加指定对话的输入输出
print(memory.buffer)  # 查看缓存区结果
print(memory.load_memory_variables({}))# 再次加载记忆变量

print("继续向向缓存区添加指定对话的输入输出, 并查看")
memory.save_context({"input": "Not much, just hanging"}, {"output": "Cool"})
print(memory.buffer)  # 查看缓存区结果
print(memory.load_memory_variables({}))# 再次加载记忆变量

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
查看储存缓存方式一：
Human: Hi, my name is Andrew
AI: Hello Andrew! It's nice to meet you. How can I assist you today?
Human: What is 1+1?
AI: 1+1 is equal to 2.
Human: What is my name?
AI: Your name is Andrew.
查看储存缓存方式二：
{'history': "Human: Hi, my name is Andrew\nAI: Hello Andrew! It's nice to meet you. How can I assist you today?\nHuman: What is 1+1?\nAI: 1+1 is equal to 2.\nHuman: What is my name?\nAI: Your name is Andrew."}
向缓存区添加指定对话的输入输出, 并查看
Human: Hi
AI: What's up
{'history': "Human: Hi\nAI: What's up"}
继续向向缓存区添加指定对话的输入输出, 并查看
Human: Hi
AI: What's up
Human: Not much, just hanging
AI: Cool
{'history': "Human: Hi\nAI: What's up\nHuman: Not much, just hanging\nAI: Cool"}
对话缓存窗口储存
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
from langchain.memory import ConversationBufferWindowMemory

# k 为窗口参数，k=1表明只保留一个对话记忆
memory = ConversationBufferWindowMemory(k=1)  

# 向memory添加两轮对话
memory.save_context({"input": "Hi"}, {"output": "What's up"})
memory.save_context({"input": "Not much, just hanging"}, {"output": "Cool"})

# 并查看记忆变量当前的记录
memory.load_memory_variables({})


llm = ChatOpenAI(temperature=0.0)
memory = ConversationBufferWindowMemory(k=1)
conversation = ConversationChain(llm=llm, memory=memory, verbose=False  )


print("第一轮对话：")
print(conversation.predict(input="Hi, my name is Andrew"))

print("第二轮对话：")
print(conversation.predict(input="What is 1+1?"))

print("第三轮对话：")
print(conversation.predict(input="What is my name?"))
1
2
3
4
5
6
第一轮对话：
Hello Andrew! It's nice to meet you. How can I assist you today?
第二轮对话：
1+1 is equal to 2.
第三轮对话：
I'm sorry, but I don't have access to personal information.
对话字符缓存储存
1
2
3
4
5
6
7
from langchain.llms import OpenAI
from langchain.memory import ConversationTokenBufferMemory
memory = ConversationTokenBufferMemory(llm=llm, max_token_limit=30)
memory.save_context({"input": "AI is what?!"}, {"output": "Amazing!"})
memory.save_context({"input": "Backpropagation is what?"}, {"output": "Beautiful!"})
memory.save_context({"input": "Chatbots are what?"}, {"output": "Charming!"})
print(memory.load_memory_variables({}))
1
{'history': 'AI: Beautiful!\nHuman: Chatbots are what?\nAI: Charming!'}
对话摘要缓存储存
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
from langchain.chains import ConversationChain
from langchain.chat_models import ChatOpenAI
from langchain.memory import ConversationSummaryBufferMemory

# 创建一个长字符串
schedule = "There is a meeting at 8am with your product team. \
You will need your powerpoint presentation prepared. \
9am-12pm have time to work on your LangChain \
project which will go quickly because Langchain is such a powerful tool. \
At Noon, lunch at the italian resturant with a customer who is driving \
from over an hour away to meet you to understand the latest in AI. \
Be sure to bring your laptop to show the latest LLM demo."

# 使用对话摘要缓存
llm = ChatOpenAI(temperature=0.0)
memory = ConversationSummaryBufferMemory(llm=llm, max_token_limit=100) 
memory.save_context({"input": "Hello"}, {"output": "What's up"})
memory.save_context({"input": "Not much, just hanging"}, {"output": "Cool"})
memory.save_context({"input": "What is on the schedule today?"}, {"output": f"{schedule}"})

print("查看对话摘要缓存储存")
print(memory.load_memory_variables({})['history'])

conversation = ConversationChain(llm=llm, memory=memory, verbose=True)

print("基于对话摘要缓存储存的对话链")
conversation.predict(input="What would be a good demo to show?")

print("再次查看对话摘要缓存储存")
print(memory.load_memory_variables({})['history'])
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
查看对话摘要缓存储存
System: The human and AI exchange greetings. The human asks about the schedule for the day. The AI provides a detailed schedule, including a meeting with the product team, work on the LangChain project, and a lunch meeting with a customer interested in AI. The AI emphasizes the importance of bringing a laptop to showcase the latest LLM demo during the lunch meeting.
基于对话摘要缓存储存的对话链


> Entering new  chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.

Current conversation:
System: The human and AI exchange greetings. The human asks about the schedule for the day. The AI provides a detailed schedule, including a meeting with the product team, work on the LangChain project, and a lunch meeting with a customer interested in AI. The AI emphasizes the importance of bringing a laptop to showcase the latest LLM demo during the lunch meeting.
Human: What would be a good demo to show?
AI:

> Finished chain.
再次查看对话摘要缓存储存
System: The human and AI exchange greetings and discuss the schedule for the day. The AI provides a detailed schedule, including a meeting with the product team, work on the LangChain project, and a lunch meeting with a customer interested in AI. The AI emphasizes the importance of bringing a laptop to showcase the latest LLM demo during the lunch meeting. The human asks what would be a good demo to show, and the AI suggests showcasing the latest LLM (Language Model) demo. The LLM is a cutting-edge AI model that can generate human-like text based on a given prompt. It has been trained on a vast amount of data and can generate coherent and contextually relevant responses. By showcasing the LLM demo, the AI can demonstrate the capabilities of their AI technology and how it can be applied to various industries and use cases.

模型链 Chains

链（Chains）通常将大语言模型（LLM）与提示（Prompt）结合在一起，基于此，我们可以对文本或数据进行一系列操作。链（Chains）可以一次性接受多个输入。例如，我们可以创建一个链，该链接受用户输入，使用提示模板对其进行格式化，然后将格式化的响应传递给 LLM 。我们可以通过将多个链组合在一起，或者通过将链与其他组件组合在一起来构建更复杂的链。

大语言模型链

大语言模型链（LLMChain）是一个简单但非常强大的链，也是后面我们将要介绍的许多链的基础。

初始化语言模型
1
2
3
4
5
6
7
8
9
10
import warnings
warnings.filterwarnings('ignore')

from langchain.chat_models import ChatOpenAI 
from langchain.prompts import ChatPromptTemplate  
from langchain.chains import LLMChain   

# 这里我们将参数temperature设置为0.0，从而减少生成答案的随机性。
# 如果你想要每次得到不一样的有新意的答案，可以尝试调整该参数。
llm = ChatOpenAI(temperature=0.0)
初始化提示模版

初始化提示，这个提示将接受一个名为product的变量。该prompt将要求LLM生成一个描述制造该产品的公司的最佳名称

1
prompt = ChatPromptTemplate.from_template("描述制造{product}的一个公司的最佳名称是什么?")
构建大语言模型链

将大语言模型(LLM)和提示（Prompt）组合成链。这个大语言模型链非常简单，可以让我们以一种顺序的方式去通过运行提示并且结合到大语言模型中。

1
chain = LLMChain(llm=llm, prompt=prompt)
运行大语言模型链

因此，如果我们有一个名为”Queen Size Sheet Set”的产品，我们可以通过使用chain.run将其通过这个链运行

1
2
product = "大号床单套装"
chain.run(product)
1
'"豪华床纺"'

您可以输入任何产品描述，然后查看链将输出什么结果。

简单顺序链

顺序链（SequentialChains）是按预定义顺序执行其链接的链。具体来说，我们将使用简单顺序链（SimpleSequentialChain），这是顺序链的最简单类型，其中每个步骤都有一个输入/输出，一个步骤的输出是下一个步骤的输入。

1
2
from langchain.chains import SimpleSequentialChain
llm = ChatOpenAI(temperature=0.9)
创建两个子链
1
2
3
4
5
6
7
8
9
10
11
12
# 提示模板 1 ：这个提示将接受产品并返回最佳名称来描述该公司
first_prompt = ChatPromptTemplate.from_template(   
    "描述制造{product}的一个公司的最好的名称是什么"
)
chain_one = LLMChain(llm=llm, prompt=first_prompt)

# 提示模板 2 ：接受公司名称，然后输出该公司的长为20个单词的描述
second_prompt = ChatPromptTemplate.from_template(   
    "写一个20字的描述对于下面这个\
    公司：{company_name}的"
)
chain_two = LLMChain(llm=llm, prompt=second_prompt)
构建简单顺序链

现在我们可以组合两个LLMChain，以便我们可以在一个步骤中创建公司名称和描述

1
2
overall_simple_chain = SimpleSequentialChain(chains=[chain_one, chain_two],
                                             verbose=True)

给一个输入，然后运行上面的链

运行简单顺序链
1
2
product = "大号床单套装"
overall_simple_chain.run(product)
1
2
3
4
5
> Entering new SimpleSequentialChain chain...
优床制造公司
优床制造公司是一家专注于生产高品质床具的公司。

> Finished chain.
1
'优床制造公司是一家专注于生产高品质床具的公司。'

顺序链

当只有一个输入和一个输出时，简单顺序链（SimpleSequentialChain）即可实现。当有多个输入或多个输出时，我们则需要使用顺序链（SequentialChain）来实现。

1
2
3
4
5
6
7
import pandas as pd
from langchain.chains import SequentialChain
from langchain.chat_models import ChatOpenAI    #导入OpenAI模型
from langchain.prompts import ChatPromptTemplate   #导入聊天提示模板
from langchain.chains import LLMChain    #导入LLM链。

llm = ChatOpenAI(temperature=0.9)

接下来我们将创建一系列的链，然后一个接一个使用他们

创建四个子链
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
#子链1
# prompt模板 1: 翻译成英语（把下面的review翻译成英语）
first_prompt = ChatPromptTemplate.from_template(
    "把下面的评论review翻译成英文:"
    "\n\n{Review}"
)
# chain 1: 输入：Review    输出：英文的 Review
chain_one = LLMChain(llm=llm, prompt=first_prompt, output_key="English_Review")

#子链2
# prompt模板 2: 用一句话总结下面的 review
second_prompt = ChatPromptTemplate.from_template(
    "请你用一句话来总结下面的评论review:"
    "\n\n{English_Review}"
)
# chain 2: 输入：英文的Review   输出：总结
chain_two = LLMChain(llm=llm, prompt=second_prompt, output_key="summary")


#子链3
# prompt模板 3: 下面review使用的什么语言
third_prompt = ChatPromptTemplate.from_template(
    "下面的评论review使用的什么语言:\n\n{Review}"
)
# chain 3: 输入：Review  输出：语言
chain_three = LLMChain(llm=llm, prompt=third_prompt, output_key="language")


#子链4
# prompt模板 4: 使用特定的语言对下面的总结写一个后续回复
fourth_prompt = ChatPromptTemplate.from_template(
    "使用特定的语言对下面的总结写一个后续回复:"
    "\n\n总结: {summary}\n\n语言: {language}"
)
# chain 4: 输入： 总结, 语言    输出： 后续回复
chain_four = LLMChain(llm=llm, prompt=fourth_prompt, output_key="followup_message")
对四个子链进行组合
1
2
3
4
5
6
7
8
#输入：review    
#输出：英文review，总结，后续回复 
overall_chain = SequentialChain(
    chains=[chain_one, chain_two, chain_three, chain_four],
    input_variables=["Review"],
    output_variables=["English_Review", "summary","followup_message"],
    verbose=True
)

让我们选择一篇评论并通过整个链传递它，可以发现，原始review是法语，可以把英文review看做是一种翻译，接下来是根据英文review得到的总结，最后输出的是用法语原文进行的续写信息。

1
2
3
df = pd.read_csv('../data/Data.csv')
review = df.Review[5]
overall_chain(review)
1
2
3
4
5
6
7
8
9
10
11
12
> Entering new SequentialChain chain...

> Finished chain.





{'Review': "Je trouve le goût médiocre. La mousse ne tient pas, c'est bizarre. J'achète les mêmes dans le commerce et le goût est bien meilleur...\nVieux lot ou contrefaçon !?",
 'English_Review': "I find the taste mediocre. The foam doesn't hold, it's weird. I buy the same ones in stores and the taste is much better...\nOld batch or counterfeit!?",
 'summary': "The reviewer finds the taste mediocre, the foam doesn't hold well, and suspects the product may be either an old batch or a counterfeit.",
 'followup_message': "后续回复（法语）：Merci beaucoup pour votre avis. Nous sommes désolés d'apprendre que vous avez trouvé le goût médiocre et que la mousse ne tient pas bien. Nous prenons ces problèmes très au sérieux et nous enquêterons sur la possibilité que le produit soit soit un ancien lot, soit une contrefaçon. Nous vous prions de nous excuser pour cette expérience décevante et nous ferons tout notre possible pour résoudre ce problème. Votre satisfaction est notre priorité et nous apprécions vos commentaires précieux."}

路由链

到目前为止，我们已经学习了大语言模型链和顺序链。但是，如果我们想做一些更复杂的事情怎么办？一个相当常见但基本的操作是根据输入将其路由到一条链，具体取决于该输入到底是什么。如果你有多个子链，每个子链都专门用于特定类型的输入，那么可以组成一个路由链，它首先决定将它传递给哪个子链，然后将它传递给那个链。

路由器由两个组件组成：

  • 路由链（Router Chain）：路由器链本身，负责选择要调用的下一个链
  • destination_chains：路由器链可以路由到的链

举一个具体的例子，让我们看一下我们在不同类型的链之间路由的地方，我们在这里有不同的prompt:

1
2
3
4
from langchain.chains.router import MultiPromptChain  #导入多提示链
from langchain.chains.router.llm_router import LLMRouterChain,RouterOutputParser
from langchain.prompts import PromptTemplate
llm = ChatOpenAI(temperature=0)
定义提示模板

首先，我们定义提示适用于不同场景下的提示模板。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
# 中文
#第一个提示适合回答物理问题
physics_template = """你是一个非常聪明的物理专家。 \
你擅长用一种简洁并且易于理解的方式去回答问题。\
当你不知道问题的答案时，你承认\
你不知道.

这是一个问题:
{input}"""


#第二个提示适合回答数学问题
math_template = """你是一个非常优秀的数学家。 \
你擅长回答数学问题。 \
你之所以如此优秀， \
是因为你能够将棘手的问题分解为组成部分，\
回答组成部分，然后将它们组合在一起，回答更广泛的问题。

这是一个问题：
{input}"""


#第三个适合回答历史问题
history_template = """你是以为非常优秀的历史学家。 \
你对一系列历史时期的人物、事件和背景有着极好的学识和理解\
你有能力思考、反思、辩证、讨论和评估过去。\
你尊重历史证据，并有能力利用它来支持你的解释和判断。

这是一个问题:
{input}"""


#第四个适合回答计算机问题
computerscience_template = """ 你是一个成功的计算机科学专家。\
你有创造力、协作精神、\
前瞻性思维、自信、解决问题的能力、\
对理论和算法的理解以及出色的沟通技巧。\
你非常擅长回答编程问题。\
你之所以如此优秀，是因为你知道  \
如何通过以机器可以轻松解释的命令式步骤描述解决方案来解决问题，\
并且你知道如何选择在时间复杂性和空间复杂性之间取得良好平衡的解决方案。

这还是一个输入：
{input}"""
对提示模版进行命名和描述

在定义了这些提示模板后，我们可以为每个模板命名，并给出具体描述。例如，第一个物理学的描述适合回答关于物理学的问题，这些信息将传递给路由链，然后由路由链决定何时使用此子链。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
# 中文
prompt_infos = [
    {
        "名字": "物理学", 
        "描述": "擅长回答关于物理学的问题", 
        "提示模板": physics_template
    },
    {
        "名字": "数学", 
        "描述": "擅长回答数学问题", 
        "提示模板": math_template
    },
    {
        "名字": "历史", 
        "描述": "擅长回答历史问题", 
        "提示模板": history_template
    },
    {
        "名字": "计算机科学", 
        "描述": "擅长回答计算机科学问题", 
        "提示模板": computerscience_template
    }
]

LLMRouterChain（此链使用 LLM 来确定如何路由事物）

在这里，我们需要一个多提示链。这是一种特定类型的链，用于在多个不同的提示模板之间进行路由。 但是这只是路由的一种类型，我们也可以在任何类型的链之间进行路由。

这里我们要实现的几个类是大模型路由器链。这个类本身使用语言模型来在不同的子链之间进行路由。这就是上面提供的描述和名称将被使用的地方。

基于提示模版信息创建相应目标链

目标链是由路由链调用的链，每个目标链都是一个语言模型链

1
2
3
4
5
6
7
8
9
10
destination_chains = {}
for p_info in prompt_infos:
    name = p_info["名字"]
    prompt_template = p_info["提示模板"]
    prompt = ChatPromptTemplate.from_template(template=prompt_template)
    chain = LLMChain(llm=llm, prompt=prompt)
    destination_chains[name] = chain  
    
destinations = [f"{p['名字']}: {p['描述']}" for p in prompt_infos]
destinations_str = "\n".join(destinations)
创建默认目标链

除了目标链之外，我们还需要一个默认目标链。这是一个当路由器无法决定使用哪个子链时调用的链。在上面的示例中，当输入问题与物理、数学、历史或计算机科学无关时，可能会调用它。

1
2
default_prompt = ChatPromptTemplate.from_template("{input}")
default_chain = LLMChain(llm=llm, prompt=default_prompt)
定义不同链之间的路由模板

这包括要完成的任务的说明以及输出应该采用的特定格式。

注意：此处在原教程的基础上添加了一个示例，主要是因为”gpt-3.5-turbo”模型不能很好适应理解模板的意思，使用 “text-davinci-003” 或者”gpt-4-0613”可以很好的工作，因此在这里多加了示例提示让其更好的学习。

例如:

<< INPUT >>

“What is black body radiation?”

<< OUTPUT >>

1
2
3
4
{{{{
    "destination": string \ name of the prompt to use or "DEFAULT"
    "next_inputs": string \ a potentially modified version of the original input
}}}}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
# 多提示路由模板
MULTI_PROMPT_ROUTER_TEMPLATE = """给语言模型一个原始文本输入，\
让其选择最适合输入的模型提示。\
系统将为您提供可用提示的名称以及最适合改提示的描述。\
如果你认为修改原始输入最终会导致语言模型做出更好的响应，\
你也可以修改原始输入。


<< 格式 >>
返回一个带有JSON对象的markdown代码片段，该JSON对象的格式如下：
```json
{{{{
    "destination": 字符串 \ 使用的提示名字或者使用 "DEFAULT"
    "next_inputs": 字符串 \ 原始输入的改进版本
}}}}



记住：“destination”必须是下面指定的候选提示名称之一，\
或者如果输入不太适合任何候选提示，\
则可以是 “DEFAULT” 。
记住：如果您认为不需要任何修改，\
则 “next_inputs” 可以只是原始输入。

<< 候选提示 >>
{destinations}

<< 输入 >>
{{input}}

<< 输出 (记得要包含 ```json)>>

样例:
<< 输入 >>
"什么是黑体辐射?"
<< 输出 >>
```json
{{{{
    "destination": 字符串 \ 使用的提示名字或者使用 "DEFAULT"
    "next_inputs": 字符串 \ 原始输入的改进版本
}}}}

"""
构建路由链

首先，我们通过格式化上面定义的目标创建完整的路由器模板。这个模板可以适用许多不同类型的目标。
因此，在这里，您可以添加一个不同的学科，如英语或拉丁语，而不仅仅是物理、数学、历史和计算机科学。

接下来，我们从这个模板创建提示模板。

最后，通过传入llm和整个路由提示来创建路由链。需要注意的是这里有路由输出解析，这很重要，因为它将帮助这个链路决定在哪些子链路之间进行路由。

1
2
3
4
5
6
7
8
9
10
11
12
router_template = MULTI_PROMPT_ROUTER_TEMPLATE.format(
    destinations=destinations_str
)
router_prompt = PromptTemplate(
    template=router_template,
    input_variables=["input"],
    output_parser=RouterOutputParser(),
)

router_chain = LLMRouterChain.from_llm(llm, router_prompt)


创建整体链路
1
2
3
4
5
6
#多提示链
chain = MultiPromptChain(router_chain=router_chain,    #l路由链路
                         destination_chains=destination_chains,   #目标链路
                         default_chain=default_chain,      #默认链路
                         verbose=True   
                        )
进行提问

如果我们问一个物理问题，我们希望看到他被路由到物理链路。

1
chain.run("什么是黑体辐射？")
1
2
3
4
5
6
7
8
9
> Entering new MultiPromptChain chain...
物理学: {'input': '什么是黑体辐射？'}
> Finished chain.





'黑体辐射是指一个理想化的物体，它能够完全吸收并且以最高效率地辐射出所有入射到它上面的电磁辐射。这种辐射的特点是它的辐射强度与波长有关，且在不同波长下的辐射强度符合普朗克辐射定律。黑体辐射在物理学中有广泛的应用，例如研究热力学、量子力学和宇宙学等领域。'

如果我们问一个数学问题，我们希望看到他被路由到数学链路。

1
chain.run("2+2等于多少？")
1
2
3
4
5
6
7
8
9
> Entering new MultiPromptChain chain...
数学: {'input': '2+2等于多少？'}
> Finished chain.





'2+2等于4。'

如果我们传递一个与任何子链路都无关的问题时，会发生什么呢？

这里，我们问了一个关于生物学的问题，我们可以看到它选择的链路是无。这意味着它将被传递到默认链路，它本身只是对语言模型的通用调用。语言模型幸运地对生物学知道很多，所以它可以帮助我们。

1
chain.run("为什么我们身体里的每个细胞都包含DNA？")
1
2
3
4
5
6
7
8
9
> Entering new MultiPromptChain chain...
物理学: {'input': '为什么我们身体里的每个细胞都包含DNA？'}
> Finished chain.





'我们身体里的每个细胞都包含DNA，因为DNA是遗传信息的载体。DNA是由四种碱基（腺嘌呤、胸腺嘧啶、鸟嘌呤和胞嘧啶）组成的长链状分子，它存储了我们的遗传信息，包括我们的基因和遗传特征。每个细胞都需要这些遗传信息来执行其特定的功能和任务。所以，DNA存在于每个细胞中，以确保我们的身体正常运作。'

英文版提示

大语言模型链
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
from langchain.chat_models import ChatOpenAI 
from langchain.prompts import ChatPromptTemplate  
from langchain.chains import LLMChain   


llm = ChatOpenAI(temperature=0.0)  

prompt = ChatPromptTemplate.from_template(   
    "What is the best name to describe \
    a company that makes {product}?"
)

chain = LLMChain(llm=llm, prompt=prompt)

product = "Queen Size Sheet Set"
chain.run(product)
1
'Royal Comfort Linens'
简单顺序链
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
from langchain.chains import SimpleSequentialChain
llm = ChatOpenAI(temperature=0.9)


first_prompt = ChatPromptTemplate.from_template(
    "What is the best name to describe \
    a company that makes {product}?"
)

# Chain 1
chain_one = LLMChain(llm=llm, prompt=first_prompt)

second_prompt = ChatPromptTemplate.from_template(
    "Write a 20 words description for the following \
    company:{company_name}"
)
# chain 2
chain_two = LLMChain(llm=llm, prompt=second_prompt)
                     
                     
overall_simple_chain = SimpleSequentialChain(chains=[chain_one, chain_two], verbose=True)
product = "Queen Size Sheet Set"
overall_simple_chain.run(product)
1
2
3
4
5
6
7
8
9
10
11
> Entering new SimpleSequentialChain chain...
"Royal Comfort Beddings"
Royal Comfort Beddings is a reputable company that offers luxurious and comfortable bedding options fit for royalty.

> Finished chain.





'Royal Comfort Beddings is a reputable company that offers luxurious and comfortable bedding options fit for royalty.'
顺序链
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
from langchain.chains import SequentialChain
from langchain.chat_models import ChatOpenAI    
from langchain.prompts import ChatPromptTemplate   
from langchain.chains import LLMChain    

llm = ChatOpenAI(temperature=0.9)

first_prompt = ChatPromptTemplate.from_template(
    "Translate the following review to english:"
    "\n\n{Review}"
)

chain_one = LLMChain(llm=llm, prompt=first_prompt, output_key="English_Review")


second_prompt = ChatPromptTemplate.from_template(
    "Can you summarize the following review in 1 sentence:"
    "\n\n{English_Review}"
)

chain_two = LLMChain(llm=llm, prompt=second_prompt, output_key="summary")

third_prompt = ChatPromptTemplate.from_template(
    "What language is the following review:\n\n{Review}"
)

chain_three = LLMChain(llm=llm, prompt=third_prompt, output_key="language")

fourth_prompt = ChatPromptTemplate.from_template(
    "Write a follow up response to the following "
    "summary in the specified language:"
    "\n\nSummary: {summary}\n\nLanguage: {language}"
)

chain_four = LLMChain(llm=llm, prompt=fourth_prompt, output_key="followup_message")


overall_chain = SequentialChain(
    chains=[chain_one, chain_two, chain_three, chain_four],
    input_variables=["Review"],
    output_variables=["English_Review", "summary","followup_message"],
    verbose=True
)

review = df.Review[5]
overall_chain(review)
1
2
3
4
5
6
7
8
9
10
11
12
> Entering new SequentialChain chain...

> Finished chain.





{'Review': "Je trouve le goût médiocre. La mousse ne tient pas, c'est bizarre. J'achète les mêmes dans le commerce et le goût est bien meilleur...\nVieux lot ou contrefaçon !?",
 'English_Review': "I find the taste poor. The foam doesn't hold, it's weird. I buy the same ones from the store and the taste is much better...\nOld batch or counterfeit!?",
 'summary': 'The reviewer is disappointed with the poor taste and lack of foam in the product, suspecting it to be either an old batch or a counterfeit.',
 'followup_message': "Réponse de suivi:\n\nCher(e) critique,\n\nNous sommes vraiment désolés d'apprendre que vous avez été déçu(e) par le mauvais goût et l'absence de mousse de notre produit. Nous comprenons votre frustration et souhaitons rectifier cette situation.\n\nTout d'abord, nous tenons à vous assurer que nous ne vendons que des produits authentiques et de haute qualité. Chaque lot de notre produit est soigneusement contrôlé avant d'être mis sur le marché.\n\nCependant, il est possible qu'un incident se soit produit dans votre cas. Nous vous invitons donc à nous contacter directement afin que nous puissions mieux comprendre la situation et résoudre ce problème. Nous accorderons une attention particulière à la fraîcheur et à la mousse de notre produit pour garantir votre satisfaction.\n\nNous tenons à vous remercier pour vos commentaires, car ils nous aident à améliorer continuellement notre produit. Votre satisfaction est notre priorité absolue et nous ferons tout notre possible pour rectifier cette situation.\n\nNous espérons avoir l'opportunité de vous fournir une expérience agréable avec notre produit à l'avenir.\n\nCordialement,\nL'équipe du service client"}

基于文档的问答 Question and Answer

使用大语言模型构建一个能够回答关于给定文档和文档集合的问答系统是一种非常实用和有效的应用场景。与仅依赖模型预训练知识不同，这种方法可以进一步整合用户自有数据，实现更加个性化和专业的问答服务。例如,我们可以收集某公司的内部文档、产品说明书等文字资料，导入问答系统中。然后用户针对这些文档提出问题时，系统可以先在文档中检索相关信息，再提供给语言模型生成答案。

这样，语言模型不仅利用了自己的通用知识，还可以充分运用外部输入文档的专业信息来回答用户问题，显著提升答案的质量和适用性。构建这类基于外部文档的问答系统，可以让语言模型更好地服务于具体场景，而不是停留在通用层面。这种灵活应用语言模型的方法值得在实际使用中推广。

基于文档问答的这个过程，我们会涉及 LangChain 中的其他组件，比如：嵌入模型（Embedding Models)和向量储存(Vector Stores)，本章让我们一起来学习这部分的内容。

直接使用向量储存查询

导入数据
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
from langchain.chains import RetrievalQA  #检索QA链，在文档上进行检索
from langchain.chat_models import ChatOpenAI  #openai模型
from langchain.document_loaders import CSVLoader #文档加载器，采用csv格式存储
from langchain.vectorstores import DocArrayInMemorySearch  #向量存储
from IPython.display import display, Markdown #在jupyter显示信息的工具
import pandas as pd

file = '../data/OutdoorClothingCatalog_1000.csv'

# 使用langchain文档加载器对数据进行导入
loader = CSVLoader(file_path=file)

# 使用pandas导入数据，用以查看
data = pd.read_csv(file,usecols=[1, 2])
data.head()

name description
0 Women's Campside Oxfords This ultracomfortable lace-to-toe Oxford boast...
1 Recycled Waterhog Dog Mat, Chevron Weave Protect your floors from spills and splashing ...
2 Infant and Toddler Girls' Coastal Chill Swimsu... She'll love the bright colors, ruffles and exc...
3 Refresh Swimwear, V-Neck Tankini Contrasts Whether you're going for a swim or heading out...
4 EcoFlex 3L Storm Pants Our new TEK O2 technology makes our four-seaso...

数据是字段为 name 和 description 的文本数据：

可以看到，导入的数据集为一个户外服装的 CSV 文件，接下来我们将在语言模型中使用它。

基本文档加载器创建向量存储
1
2
3
4
5
#导入向量存储索引创建器
from langchain.indexes import VectorstoreIndexCreator 

# 创建指定向量存储类, 创建完成后，从加载器中调用, 通过文档加载器列表加载
index = VectorstoreIndexCreator(vectorstore_cls=DocArrayInMemorySearch).from_loaders([loader])
查询创建的向量存储
1
2
3
4
5
6
7
query ="请用markdown表格的方式列出所有具有防晒功能的衬衫，对每件衬衫描述进行总结"

#使用索引查询创建一个响应，并传入这个查询
response = index.query(query)

#查看查询返回的内容
display(Markdown(response))
Name Description
Men’s Tropical Plaid Short-Sleeve Shirt UPF 50+ rated sun protection, 100% polyester fabric, wrinkle-resistant, front and back cape venting, two front bellows pockets
Men’s Plaid Tropic Shirt, Short-Sleeve UPF 50+ rated sun protection, 52% polyester and 48% nylon fabric, wrinkle-free, quickly evaporates perspiration, front and back cape venting, two front bellows pockets
Girls’ Ocean Breeze Long-Sleeve Stripe Shirt UPF 50+ rated sun protection, Nylon Lycra®-elastane blend fabric, quick-drying and fade-resistant, holds shape well, durable seawater-resistant fabric retains its color

在上面我们得到了一个 Markdown 表格，其中包含所有带有防晒衣的衬衫的 名称(Name) 和 描述(Description) ，其中描述是语言模型总结过的结果。

结合表征模型和向量存储

由于语言模型的上下文长度限制，直接处理长文档具有困难。为实现对长文档的问答，我们可以引入向量嵌入(Embeddings)和向量存储(Vector Store)等技术：

首先，使用文本嵌入(Embeddings)算法对文档进行向量化，使语义相似的文本片段具有接近的向量表示。其次，将向量化的文档切分为小块，存入向量数据库，这个流程正是创建索引(index)的过程。向量数据库对各文档片段进行索引，支持快速检索。这样，当用户提出问题时，可以先将问题转换为向量，在数据库中快速找到语义最相关的文档片段。然后将这些文档片段与问题一起传递给语言模型，生成回答。

通过嵌入向量化和索引技术，我们实现了对长文档的切片检索和问答。这种流程克服了语言模型的上下文限制，可以构建处理大规模文档的问答系统。

导入数据
1
2
3
4
5
6
7
#创建一个文档加载器，通过csv格式加载
file = '../data/OutdoorClothingCatalog_1000.csv'
loader = CSVLoader(file_path=file)
docs = loader.load()

#查看单个文档，每个文档对应于CSV中的一行数据
docs[0]
1
Document(page_content=": 0\nname: Women's Campside Oxfords\ndescription: This ultracomfortable lace-to-toe Oxford boasts a super-soft canvas, thick cushioning, and quality construction for a broken-in feel from the first time you put them on. \n\nSize & Fit: Order regular shoe size. For half sizes not offered, order up to next whole size. \n\nSpecs: Approx. weight: 1 lb.1 oz. per pair. \n\nConstruction: Soft canvas material for a broken-in feel and look. Comfortable EVA innersole with Cleansport NXT® antimicrobial odor control. Vintage hunt, fish and camping motif on innersole. Moderate arch contour of innersole. EVA foam midsole for cushioning and support. Chain-tread-inspired molded rubber outsole with modified chain-tread pattern. Imported. \n\nQuestions? Please contact us for any inquiries.", metadata={'source': '../data/OutdoorClothingCatalog_1000.csv', 'row': 0})
文本向量表征模型
1
2
3
4
5
6
7
8
9
10
11
12
13
14
#使用OpenAIEmbedding类
from langchain.embeddings import OpenAIEmbeddings 

embeddings = OpenAIEmbeddings() 

#因为文档比较短了，所以这里不需要进行任何分块,可以直接进行向量表征
#使用初始化OpenAIEmbedding实例上的查询方法embed_query为文本创建向量表征
embed = embeddings.embed_query("你好呀，我的名字叫小可爱")

#查看得到向量表征的长度
print("\n\033[32m向量表征的长度: \033[0m \n", len(embed))

#每个元素都是不同的数字值，组合起来就是文本的向量表征
print("\n\033[32m向量表征前5个元素: \033[0m \n", embed[:5])
1
2
3
4
5
向量表征的长度:  
 1536

向量表征前5个元素:  
 [-0.019283676849006164, -0.006842594710511029, -0.007344046732916966, -0.024501312942119265, -0.026608679897592472]
基于向量表征创建并查询向量存储
1
2
3
4
5
6
7
8
9
10
# 将刚才创建文本向量表征(embeddings)存储在向量存储(vector store)中
# 使用DocArrayInMemorySearch类的from_documents方法来实现
# 该方法接受文档列表以及向量表征模型作为输入
db = DocArrayInMemorySearch.from_documents(docs, embeddings)

query = "请推荐一件具有防晒功能的衬衫"
#使用上面的向量存储来查找与传入查询类似的文本，得到一个相似文档列表
docs = db.similarity_search(query)
print("\n\033[32m返回文档的个数: \033[0m \n", len(docs))
print("\n\033[32m第一个文档: \033[0m \n", docs[0])
1
2
3
4
5
返回文档的个数:  
 4

第一个文档:  
 page_content=": 535\nname: Men's TropicVibe Shirt, Short-Sleeve\ndescription: This Men’s sun-protection shirt with built-in UPF 50+ has the lightweight feel you want and the coverage you need when the air is hot and the UV rays are strong. Size & Fit: Traditional Fit: Relaxed through the chest, sleeve and waist. Fabric & Care: Shell: 71% Nylon, 29% Polyester. Lining: 100% Polyester knit mesh. UPF 50+ rated – the highest rated sun protection possible. Machine wash and dry. Additional Features: Wrinkle resistant. Front and back cape venting lets in cool breezes. Two front bellows pockets. Imported.\n\nSun Protection That Won't Wear Off: Our high-performance fabric provides SPF 50+ sun protection, blocking 98% of the sun's harmful rays." metadata={'source': '../data/OutdoorClothingCatalog_1000.csv', 'row': 535}

我们可以看到一个返回了四个结果。输出的第一结果是一件关于防晒的衬衫，满足我们查询的要求：请推荐一件具有防晒功能的衬衫

使用查询结果构造提示来回答问题
1
2
3
4
5
6
7
8
9
10
11
12
13
#导入大语言模型, 这里使用默认模型gpt-3.5-turbo会出现504服务器超时，
#因此使用gpt-3.5-turbo-0301
llm = ChatOpenAI(model_name="gpt-3.5-turbo-0301",temperature = 0.0) 

#合并获得的相似文档内容
qdocs = "".join([docs[i].page_content for i in range(len(docs))])  


#将合并的相似文档内容后加上问题（question）输入到 `llm.call_as_llm`中
#这里问题是：以Markdown表格的方式列出所有具有防晒功能的衬衫并总结 
response = llm.call_as_llm(f"{qdocs}问题：请用markdown表格的方式列出所有具有防晒功能的衬衫，对每件衬衫描述进行总结") 

display(Markdown(response))
衣服名称 描述总结
Men’s TropicVibe Shirt, Short-Sleeve 男士短袖衬衫，内置UPF 50+防晒功能，轻盈舒适，前后通风口，两个前口袋，防皱，最高级别的防晒保护。
Men’s Tropical Plaid Short-Sleeve Shirt 男士短袖衬衫，UPF 50+防晒，100%聚酯纤维，防皱，前后通风口，两个前口袋，最高级别的防晒保护。
Men’s Plaid Tropic Shirt, Short-Sleeve 男士短袖衬衫，UPF 50+防晒，52%聚酯纤维和48%尼龙，防皱，前后通风口，两个前口袋，最高级别的防晒保护。
Girls’ Ocean Breeze Long-Sleeve Stripe Shirt 女孩长袖衬衫，UPF 50+防晒，尼龙Lycra®-弹性纤维混纺，快干，耐褪色，防水，最高级别的防晒保护，适合与我们的泳衣系列搭配。
使用检索问答链来回答问题

通过LangChain创建一个检索问答链，对检索到的文档进行问题回答。检索问答链的输入包含以下

  • llm: 语言模型，进行文本生成
  • chain_type: 传入链类型，这里使用stuff，将所有查询得到的文档组合成一个文档传入下一步。其他的方式包括：
    • Map Reduce： 将所有块与问题一起传递给语言模型，获取回复，使用另一个语言模型调用将所有单独的回复总结成最终答案，它可以在任意数量的文档上运行。可以并行处理单个问题，同时也需要更多的调用。它将所有文档视为独立的
    • Refine： 用于循环许多文档，实际上它是用迭代实现的，它建立在先前文档的答案之上，非常适合用于合并信息并随时间逐步构建答案，由于依赖于先前调用的结果，因此它通常需要更长的时间，并且基本上需要与Map Reduce一样多的调用
    • Map Re-rank： 对每个文档进行单个语言模型调用，要求它返回一个分数，选择最高分，这依赖于语言模型知道分数应该是什么，需要告诉它，如果它与文档相关，则应该是高分，并在那里精细调整说明，可以批量处理它们相对较快，但是更加昂贵

  • retriever:检索器
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
#基于向量储存，创建检索器
retriever = db.as_retriever() 

qa_stuff = RetrievalQA.from_chain_type(
    llm=llm, 
    chain_type="stuff", 
    retriever=retriever, 
    verbose=True
)

#创建一个查询并在此查询上运行链
query =  "请用markdown表格的方式列出所有具有防晒功能的衬衫，对每件衬衫描述进行总结"

response = qa_stuff.run(query)

display(Markdown(response)) 
1
2
3
> Entering new RetrievalQA chain...

> Finished chain.
编号 名称 描述
618 Men’s Tropical Plaid Short-Sleeve Shirt 100%聚酯纤维制成，轻便，防皱，前后背部有通风口，两个前面的褶皱口袋，UPF 50+防晒等级，可阻挡98%的紫外线
374 Men’s Plaid Tropic Shirt, Short-Sleeve 52%聚酯纤维和48%尼龙制成，轻便，防皱，前后背部有通风口，两个前面的褶皱口袋，UPF 50+防晒等级，可阻挡98%的紫外线
535 Men’s TropicVibe Shirt, Short-Sleeve 71%尼龙和29%聚酯纤维制成，轻便，防皱，前后背部有通风口，两个前面的褶皱口袋，UPF 50+防晒等级，可阻挡98%的紫外线
293 Girls’ Ocean Breeze Long-Sleeve Stripe Shirt 尼龙Lycra®-弹性纤维混纺，长袖，UPF 50+防晒等级，可阻挡98%的紫外线，快干，耐褪色，可与我们的泳衣系列轻松搭配

总结：这些衬衫都具有防晒功能，防晒等级为UPF 50+，可阻挡98%的紫外线。它们都是轻便的，防皱的，有前后背部通风口和前面的褶皱口袋。其中女孩的长袖条纹衬衫是由尼龙Lycra®-弹性纤维混纺制成，快干，耐褪色，可与泳衣系列轻松搭配。

可以看到 2.5 和 2.6 部分的这两个方式返回相同的结果。

英文版提示

直接使用向量储存查询
1
2
3
4
5
6
7
8
9
10
11
12
13
14
from langchain.document_loaders import CSVLoader 
from langchain.indexes import VectorstoreIndexCreator

file = '../data/OutdoorClothingCatalog_1000.csv'
loader = CSVLoader(file_path=file)

index = VectorstoreIndexCreator(vectorstore_cls=DocArrayInMemorySearch).from_loaders([loader])

query ="Please list all your shirts with sun protection \
in a table in markdown and summarize each one."

response = index.query(query)

display(Markdown(response))
Name Description
Men’s Tropical Plaid Short-Sleeve Shirt UPF 50+ rated, 100% polyester, wrinkle-resistant, front and back cape venting, two front bellows pockets
Men’s Plaid Tropic Shirt, Short-Sleeve UPF 50+ rated, 52% polyester and 48% nylon, machine washable and dryable, front and back cape venting, two front bellows pockets
Men’s TropicVibe Shirt, Short-Sleeve UPF 50+ rated, 71% Nylon, 29% Polyester, 100% Polyester knit mesh, machine wash and dry, front and back cape venting, two front bellows pockets
Sun Shield Shirt by UPF 50+ rated, 78% nylon, 22% Lycra Xtra Life fiber, handwash, line dry, wicks moisture, fits comfortably over swimsuit, abrasion resistant

All four shirts provide UPF 50+ sun protection, blocking 98% of the sun’s harmful rays. The Men’s Tropical Plaid Short-Sleeve Shirt is made of 100% polyester and is wrinkle-resistant

结合表征模型和向量存储
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54

from langchain.document_loaders import CSVLoader 
from langchain.embeddings import OpenAIEmbeddings 
from langchain.vectorstores import DocArrayInMemorySearch  


embeddings = OpenAIEmbeddings() 
embed = embeddings.embed_query("Hi my name is Harrison")

print("\n\033[32m向量表征的长度: \033[0m \n", len(embed))
print("\n\033[32m向量表征前5个元素: \033[0m \n", embed[:5])

file = '../data/OutdoorClothingCatalog_1000.csv'
loader = CSVLoader(file_path=file)
docs = loader.load()
embeddings = OpenAIEmbeddings() 
db = DocArrayInMemorySearch.from_documents(docs, embeddings)

query = "Please suggest a shirt with sunblocking"
docs = db.similarity_search(query)
print("\n\033[32m返回文档的个数: \033[0m \n", len(docs))
print("\n\033[32m第一个文档: \033[0m \n", docs[0])
      
      
# 使用查询结果构造提示来回答问题
llm = ChatOpenAI(model_name="gpt-3.5-turbo-0301",temperature = 0.0) 

qdocs = "".join([docs[i].page_content for i in range(len(docs))])  

response = llm.call_as_llm(f"{qdocs} Question: Please list all your \
shirts with sun protection in a table in markdown and summarize each one.") 

print("\n\033[32m使用查询结果构造提示来回答问题: \033[0m \n", docs[0])
display(Markdown(response))


# 使用检索问答链来回答问题
retriever = db.as_retriever() 

qa_stuff = RetrievalQA.from_chain_type(
    llm=llm, 
    chain_type="stuff", 
    retriever=retriever, 
    verbose=True
)


query =  "Please list all your shirts with sun protection in a table \
in markdown and summarize each one."

response = qa_stuff.run(query)

print("\n\033[32m 使用检索问答链来回答问题： \033[0m \n")
display(Markdown(response)) 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
向量表征的长度:  
 1536

向量表征前5个元素:  
 [-0.021913960932078383, 0.006774206755842609, -0.018190348816400977, -0.039148249368104494, -0.014089343366938917]

返回文档的个数:  
 4

第一个文档:  
 page_content=': 255\nname: Sun Shield Shirt by\ndescription: "Block the sun, not the fun – our high-performance sun shirt is guaranteed to protect from harmful UV rays. \n\nSize & Fit: Slightly Fitted: Softly shapes the body. Falls at hip.\n\nFabric & Care: 78% nylon, 22% Lycra Xtra Life fiber. UPF 50+ rated – the highest rated sun protection possible. Handwash, line dry.\n\nAdditional Features: Wicks moisture for quick-drying comfort. Fits comfortably over your favorite swimsuit. Abrasion resistant for season after season of wear. Imported.\n\nSun Protection That Won\'t Wear Off\nOur high-performance fabric provides SPF 50+ sun protection, blocking 98% of the sun\'s harmful rays. This fabric is recommended by The Skin Cancer Foundation as an effective UV protectant.' metadata={'source': '../data/OutdoorClothingCatalog_1000.csv', 'row': 255}

使用查询结果构造提示来回答问题:  
 page_content=': 255\nname: Sun Shield Shirt by\ndescription: "Block the sun, not the fun – our high-performance sun shirt is guaranteed to protect from harmful UV rays. \n\nSize & Fit: Slightly Fitted: Softly shapes the body. Falls at hip.\n\nFabric & Care: 78% nylon, 22% Lycra Xtra Life fiber. UPF 50+ rated – the highest rated sun protection possible. Handwash, line dry.\n\nAdditional Features: Wicks moisture for quick-drying comfort. Fits comfortably over your favorite swimsuit. Abrasion resistant for season after season of wear. Imported.\n\nSun Protection That Won\'t Wear Off\nOur high-performance fabric provides SPF 50+ sun protection, blocking 98% of the sun\'s harmful rays. This fabric is recommended by The Skin Cancer Foundation as an effective UV protectant.' metadata={'source': '../data/OutdoorClothingCatalog_1000.csv', 'row': 255}
Name Description
Sun Shield Shirt High-performance sun shirt with UPF 50+ sun protection, moisture-wicking, and abrasion-resistant fabric. Recommended by The Skin Cancer Foundation.
Men’s Plaid Tropic Shirt Ultracomfortable shirt with UPF 50+ sun protection, wrinkle-free fabric, and front/back cape venting. Made with 52% polyester and 48% nylon.
Men’s TropicVibe Shirt Men’s sun-protection shirt with built-in UPF 50+ and front/back cape venting. Made with 71% nylon and 29% polyester.
Men’s Tropical Plaid Short-Sleeve Shirt Lightest hot-weather shirt with UPF 50+ sun protection, front/back cape venting, and two front bellows pockets. Made with 100% polyester.

All of these shirts provide UPF 50+ sun protection, blocking 98% of the sun’s harmful rays. They also have additional features such as moisture-wicking, wrinkle-free fabric, and front/back cape venting for added comfort.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
> Entering new RetrievalQA chain...

> Finished chain.

 使用检索问答链来回答问题：  

| Shirt Number | Name | Description |
| --- | --- | --- |
| 618 | Men's Tropical Plaid Short-Sleeve Shirt | Rated UPF 50+ for superior protection from the sun's UV rays. Made of 100% polyester and is wrinkle-resistant. With front and back cape venting that lets in cool breezes and two front bellows pockets. |
| 374 | Men's Plaid Tropic Shirt, Short-Sleeve | Rated to UPF 50+ and offers sun protection. Made with 52% polyester and 48% nylon, this shirt is machine washable and dryable. Additional features include front and back cape venting, two front bellows pockets. |
| 535 | Men's TropicVibe Shirt, Short-Sleeve | Built-in UPF 50+ has the lightweight feel you want and the coverage you need when the air is hot and the UV rays are strong. Made with 71% Nylon, 29% Polyester. Wrinkle resistant. Front and back cape venting lets in cool breezes. Two front bellows pockets. |
| 255 | Sun Shield Shirt | High-performance sun shirt is guaranteed to protect from harmful UV rays. Made with 78% nylon, 22% Lycra Xtra Life fiber. Wicks moisture for quick-drying comfort. Fits comfortably over your favorite swimsuit. Abrasion-resistant. |

All of the shirts listed above provide sun protection with a UPF rating of 50+ and block 98% of the sun's harmful rays. The Men's Tropical Plaid Short-Sleeve Shirt is made of 100% polyester and has front and back cape venting and two front bellows pockets. The Men's Plaid Tropic Shirt, Short-Sleeve is made with 52% polyester and 48% nylon and has front and back cape venting and two front bellows pockets. The Men's TropicVibe Shirt, Short-Sleeve is made with 71% Nylon, 29% Polyester and has front and back cape venting and two front bellows pockets. The Sun Shield Shirt is made with 78% nylon, 22% Lycra Xtra Life fiber and is abrasion-resistant. It fits comfortably over your favorite swimsuit.

评估 Evaluation

评估是检验语言模型问答质量的关键环节。评估可以检验语言模型在不同文档上的问答效果，找出其弱点。还可以通过比较不同模型，选择最佳系统。此外，定期评估也可以检查模型质量的衰减。评估通常有两个目的：

  • 检验LLM应用是否达到了验收标准
  • 分析改动对于LLM应用性能的影响

基本的思路就是利用语言模型本身和链本身，来辅助评估其他的语言模型、链和应用程序。我们还是以上一章节的文档问答应用为例，在本章节中讨论如何在 LangChain 中处理和考虑评估的内容。

创建LLM应用

首先，按照 langchain 链的方式构建一个 LLM 的文档问答应用

1
2
3
4
5
6
7
8
9
10
11
12
13
14
from langchain.chains import RetrievalQA #检索QA链，在文档上进行检索
from langchain.chat_models import ChatOpenAI #openai模型
from langchain.document_loaders import CSVLoader #文档加载器，采用csv格式存储
from langchain.indexes import VectorstoreIndexCreator #导入向量存储索引创建器
from langchain.vectorstores import DocArrayInMemorySearch #向量存储
#加载中文数据
file = '../data/product_data.csv'
loader = CSVLoader(file_path=file)
data = loader.load()

#查看数据
import pandas as pd
test_data = pd.read_csv(file,skiprows=0)
display(test_data.head())

product_name description
0 全自动咖啡机 规格:\n大型 - 尺寸：13.8'' x 17.3''。\n中型 - 尺寸：11.5'' ...
1 电动牙刷 规格:\n一般大小 - 高度：9.5''，宽度：1''。\n\n为什么我们热爱它:\n我们的...
2 橙味维生素C泡腾片 规格:\n每盒含有20片。\n\n为什么我们热爱它:\n我们的橙味维生素C泡腾片是快速补充维...
3 无线蓝牙耳机 规格:\n单个耳机尺寸：1.5'' x 1.3''。\n\n为什么我们热爱它:\n这款无线蓝...
4 瑜伽垫 规格:\n尺寸：24'' x 68''。\n\n为什么我们热爱它:\n我们的瑜伽垫拥有出色的... 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# 将指定向量存储类,创建完成后，我们将从加载器中调用,通过文档记载器列表加载

index = VectorstoreIndexCreator(
    vectorstore_cls=DocArrayInMemorySearch
).from_loaders([loader])


#通过指定语言模型、链类型、检索器和我们要打印的详细程度来创建检索QA链
llm = ChatOpenAI(temperature = 0.0)
qa = RetrievalQA.from_chain_type(
    llm=llm, 
    chain_type="stuff", 
    retriever=index.vectorstore.as_retriever(), 
    verbose=True,
    chain_type_kwargs = {
        "document_separator": "<<<<>>>>>"
    }
)

上述代码的主要功能及作用在上一章节中都已说明，这里不再赘述

设置测试的数据

我们查看一下经过文档加载器 CSVLoad 加载后生成的 data 内的信息，这里我们抽取 data 中的第九条和第十条数据，看看它们的主要内容：

第十条数据：

1
data[10]
1
Document(page_content="product_name: 高清电视机\ndescription: 规格:\n尺寸：50''。\n\n为什么我们热爱它:\n我们的高清电视机拥有出色的画质和强大的音效，带来沉浸式的观看体验。\n\n材质与护理:\n使用干布清洁。\n\n构造:\n由塑料、金属和电子元件制成。\n\n其他特性:\n支持网络连接，可以在线观看视频。\n配备遥控器。\n在韩国制造。\n\n有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。", metadata={'source': '../data/product_data.csv', 'row': 10})

第十一条数据：

1
data[11]
1
Document(page_content="product_name: 旅行背包\ndescription: 规格:\n尺寸：18'' x 12'' x 6''。\n\n为什么我们热爱它:\n我们的旅行背包拥有多个实用的内外袋，轻松装下您的必需品，是短途旅行的理想选择。\n\n材质与护理:\n可以手洗，自然晾干。\n\n构造:\n由防水尼龙制成。\n\n其他特性:\n附带可调节背带和安全锁。\n在中国制造。\n\n有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。", metadata={'source': '../data/product_data.csv', 'row': 11})

看上面的第一个文档中有高清电视机，第二个文档中有旅行背包，从这些细节中，我们可以创建一些例子查询和答案

手动创建测试数据

需要说明的是这里我们的文档是 csv 文件，所以我们使用的是文档加载器是 CSVLoader ，CSVLoader 会对 csv 文件中的每一行数据进行分割，所以这里看到的 data[10], data[11]的内容则是 csv 文件中的第10条，第11条数据的内容。下面我们根据这两条数据手动设置两条“问答对”，每一个“问答对”中包含一个 query ，一个 answer ：

1
2
3
4
5
6
7
8
9
10
examples = [
    {
        "query": "高清电视机怎么进行护理？",
        "answer": "使用干布清洁。"
    },
    {
        "query": "旅行背包有内外袋吗？",
        "answer": "有。"
    }
]
通过LLM生成测试用例

在前面的内容中，我们使用的方法都是通过手动的方法来构建测试数据集，比如说我们手动创建10个问题和10个答案，然后让 LLM 回答这10个问题，再将 LLM 给出的答案与我们准备好的答案做比较，最后再给 LLM 打分，评估的流程大概就是这样。但是这里有一个问题，就是我们需要手动去创建所有的问题集和答案集，那会是一个非常耗费时间和人力的成本。那有没有一种可以自动创建大量问答测试集的方法呢？那当然是有的，今天我们就来介绍 Langchain 提供的方法：QAGenerateChain，我们可以通过QAGenerateChain来为我们的文档自动创建问答集：

由于QAGenerateChain类中使用的PROMPT是英文，故我们继承QAGenerateChain类，将PROMPT加上“请使用中文输出”。下面是generate_chain.py文件中的QAGenerateChain类的源码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
from langchain.evaluation.qa import QAGenerateChain #导入QA生成链，它将接收文档，并从每个文档中创建一个问题答案对

# 下面是langchain.evaluation.qa.generate_prompt中的源码，在template的最后加上“请使用中文输出”
from langchain.output_parsers.regex import RegexParser
from langchain.prompts import PromptTemplate
from langchain.base_language import BaseLanguageModel
from typing import Any

template = """You are a teacher coming up with questions to ask on a quiz. 
Given the following document, please generate a question and answer based on that document.

Example Format:
<Begin Document>
...
<End Document>
QUESTION: question here
ANSWER: answer here

These questions should be detailed and be based explicitly on information in the document. Begin!

<Begin Document>
{doc}
<End Document>
请使用中文输出
"""
output_parser = RegexParser(
    regex=r"QUESTION: (.*?)\nANSWER: (.*)", output_keys=["query", "answer"]
)
PROMPT = PromptTemplate(
    input_variables=["doc"], template=template, output_parser=output_parser
)

# 继承QAGenerateChain
class ChineseQAGenerateChain(QAGenerateChain):
    """LLM Chain specifically for generating examples for question answering."""

    @classmethod
    def from_llm(cls, llm: BaseLanguageModel, **kwargs: Any) -> QAGenerateChain:
        """Load QA Generate Chain from LLM."""
        return cls(llm=llm, prompt=PROMPT, **kwargs)



example_gen_chain = ChineseQAGenerateChain.from_llm(ChatOpenAI())#通过传递chat open AI语言模型来创建这个链
new_examples = example_gen_chain.apply([{"doc": t} for t in data[:5]]) 

#查看用例数据
new_examples 
1
2
3
4
5
6
7
[{'qa_pairs': {'query': '这款全自动咖啡机的尺寸是多少？',
   'answer': "大型尺寸为13.8'' x 17.3''，中型尺寸为11.5'' x 15.2''。"}},
 {'qa_pairs': {'query': '这款电动牙刷的规格是什么？', 'answer': "一般大小 - 高度：9.5''，宽度：1''。"}},
 {'qa_pairs': {'query': '这种产品的名称是什么？', 'answer': '这种产品的名称是橙味维生素C泡腾片。'}},
 {'qa_pairs': {'query': '这款无线蓝牙耳机的尺寸是多少？',
   'answer': "该无线蓝牙耳机的尺寸为1.5'' x 1.3''。"}},
 {'qa_pairs': {'query': '这款瑜伽垫的尺寸是多少？', 'answer': "这款瑜伽垫的尺寸是24'' x 68''。"}}]

在上面的代码中，我们创建了一个QAGenerateChain，然后我们应用了QAGenerateChain的 apply 方法对 data 中的前5条数据创建了5个“问答对”，由于创建问答集是由 LLM 来自动完成的，因此会涉及到 token 成本的问题，所以我们这里出于演示的目的，只对 data 中的前5条数据创建问答集。

1
new_examples[0]
1
2
{'qa_pairs': {'query': '这款全自动咖啡机的尺寸是多少？',
  'answer': "大型尺寸为13.8'' x 17.3''，中型尺寸为11.5'' x 15.2''。"}}

源数据：

1
data[0]
1
Document(page_content="product_name: 全自动咖啡机\ndescription: 规格:\n大型 - 尺寸：13.8'' x 17.3''。\n中型 - 尺寸：11.5'' x 15.2''。\n\n为什么我们热爱它:\n这款全自动咖啡机是爱好者的理想选择。 一键操作，即可研磨豆子并沏制出您喜爱的咖啡。它的耐用性和一致性使它成为家庭和办公室的理想选择。\n\n材质与护理:\n清洁时只需轻擦。\n\n构造:\n由高品质不锈钢制成。\n\n其他特性:\n内置研磨器和滤网。\n预设多种咖啡模式。\n在中国制造。\n\n有问题？ 请随时联系我们的客户服务团队，他们会解答您的所有问题。", metadata={'source': '../data/product_data.csv', 'row': 0})
整合测试集

还记得我们前面手动创建的两个问答集吗？现在我们需要将之前手动创建的问答集合并到QAGenerateChain创建的问答集中，这样在答集中既有手动创建的例子又有 llm 自动创建的例子，这会使我们的测试集更加完善。

接下来我们就需要让之前创建的文档问答链qa来回答这个测试集里的问题，来看看 LLM 是怎么回答的吧：

1
2
examples += [ v for item in new_examples for k,v in item.items()]
qa.run(examples[0]["query"])
1
2
3
4
5
6
7
8
9
> Entering new RetrievalQA chain...

> Finished chain.





'高清电视机的护理非常简单。您只需要使用干布清洁即可。避免使用湿布或化学清洁剂，以免损坏电视机的表面。'

这里我们看到qa回答了第0个问题：”高清电视机怎么进行护理？” ，这里的第0个问题就是先前我们手动创建的第一个问题，并且我们手动创建的 answer 是：”使用干布清洁。” 这里我们发现问答链qa回答的也是“您只需要使用干布清洁即可”，只是它比我们的答案还多了一段说明：“高清电视机的护理非常简单。您只需要使用干布清洁即可。避免使用湿布或化学清洁剂，以免损坏电视机的表面。”。

人工评估

你想知道qa是怎么找到问题的答案的吗？下面让我们打开debug，看看qa是如何找到问题的答案！

1
2
3
4
5
import langchain
langchain.debug = True

#重新运行与上面相同的示例，可以看到它开始打印出更多的信息
qa.run(examples[0]["query"])
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
[chain/start] [1:chain:RetrievalQA] Entering Chain run with input:
{
  "query": "高清电视机怎么进行护理？"
}
[chain/start] [1:chain:RetrievalQA > 3:chain:StuffDocumentsChain] Entering Chain run with input:
[inputs]
[chain/start] [1:chain:RetrievalQA > 3:chain:StuffDocumentsChain > 4:chain:LLMChain] Entering Chain run with input:
{
  "question": "高清电视机怎么进行护理？",
  "context": "product_name: 高清电视机\ndescription: 规格:\n尺寸：50''。\n\n为什么我们热爱它:\n我们的高清电视机拥有出色的画质和强大的音效，带来沉浸式的观看体验。\n\n材质与护理:\n使用干布清洁。\n\n构造:\n由塑料、金属和电子元件制成。\n\n其他特性:\n支持网络连接，可以在线观看视频。\n配备遥控器。\n在韩国制造。\n\n有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。<<<<>>>>>product_name: 空气净化器\ndescription: 规格:\n尺寸：15'' x 15'' x 20''。\n\n为什么我们热爱它:\n我们的空气净化器采用了先进的HEPA过滤技术，能有效去除空气中的微粒和异味，为您提供清新的室内环境。\n\n材质与护理:\n清洁时使用干布擦拭。\n\n构造:\n由塑料和电子元件制成。\n\n其他特性:\n三档风速，附带定时功能。\n在德国制造。\n\n有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。<<<<>>>>>product_name: 宠物自动喂食器\ndescription: 规格:\n尺寸：14'' x 9'' x 15''。\n\n为什么我们热爱它:\n我们的宠物自动喂食器可以定时定量投放食物，让您无论在家或外出都能确保宠物的饮食。\n\n材质与护理:\n可用湿布清洁。\n\n构造:\n由塑料和电子元件制成。\n\n其他特性:\n配备LCD屏幕，操作简单。\n可以设置多次投食。\n在美国制造。\n\n有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。<<<<>>>>>product_name: 玻璃保护膜\ndescription: 规格:\n适用于各种尺寸的手机屏幕。\n\n为什么我们热爱它:\n我们的玻璃保护膜可以有效防止手机屏幕刮伤和破裂，而且不影响触控的灵敏度。\n\n材质与护理:\n使用干布擦拭。\n\n构造:\n由高强度的玻璃材料制成。\n\n其他特性:\n安装简单，适合自行安装。\n在日本制造。\n\n有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。"
}
[llm/start] [1:chain:RetrievalQA > 3:chain:StuffDocumentsChain > 4:chain:LLMChain > 5:llm:ChatOpenAI] Entering LLM run with input:
{
  "prompts": [
    "System: Use the following pieces of context to answer the users question. \nIf you don't know the answer, just say that you don't know, don't try to make up an answer.\n----------------\nproduct_name: 高清电视机\ndescription: 规格:\n尺寸：50''。\n\n为什么我们热爱它:\n我们的高清电视机拥有出色的画质和强大的音效，带来沉浸式的观看体验。\n\n材质与护理:\n使用干布清洁。\n\n构造:\n由塑料、金属和电子元件制成。\n\n其他特性:\n支持网络连接，可以在线观看视频。\n配备遥控器。\n在韩国制造。\n\n有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。<<<<>>>>>product_name: 空气净化器\ndescription: 规格:\n尺寸：15'' x 15'' x 20''。\n\n为什么我们热爱它:\n我们的空气净化器采用了先进的HEPA过滤技术，能有效去除空气中的微粒和异味，为您提供清新的室内环境。\n\n材质与护理:\n清洁时使用干布擦拭。\n\n构造:\n由塑料和电子元件制成。\n\n其他特性:\n三档风速，附带定时功能。\n在德国制造。\n\n有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。<<<<>>>>>product_name: 宠物自动喂食器\ndescription: 规格:\n尺寸：14'' x 9'' x 15''。\n\n为什么我们热爱它:\n我们的宠物自动喂食器可以定时定量投放食物，让您无论在家或外出都能确保宠物的饮食。\n\n材质与护理:\n可用湿布清洁。\n\n构造:\n由塑料和电子元件制成。\n\n其他特性:\n配备LCD屏幕，操作简单。\n可以设置多次投食。\n在美国制造。\n\n有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。<<<<>>>>>product_name: 玻璃保护膜\ndescription: 规格:\n适用于各种尺寸的手机屏幕。\n\n为什么我们热爱它:\n我们的玻璃保护膜可以有效防止手机屏幕刮伤和破裂，而且不影响触控的灵敏度。\n\n材质与护理:\n使用干布擦拭。\n\n构造:\n由高强度的玻璃材料制成。\n\n其他特性:\n安装简单，适合自行安装。\n在日本制造。\n\n有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。\nHuman: 高清电视机怎么进行护理？"
  ]
}
[llm/end] [1:chain:RetrievalQA > 3:chain:StuffDocumentsChain > 4:chain:LLMChain > 5:llm:ChatOpenAI] [2.86s] Exiting LLM run with output:
{
  "generations": [
    [
      {
        "text": "高清电视机的护理非常简单。您只需要使用干布清洁即可。避免使用湿布或化学清洁剂，以免损坏电视机的表面。",
        "generation_info": {
          "finish_reason": "stop"
        },
        "message": {
          "lc": 1,
          "type": "constructor",
          "id": [
            "langchain",
            "schema",
            "messages",
            "AIMessage"
          ],
          "kwargs": {
            "content": "高清电视机的护理非常简单。您只需要使用干布清洁即可。避免使用湿布或化学清洁剂，以免损坏电视机的表面。",
            "additional_kwargs": {}
          }
        }
      }
    ]
  ],
  "llm_output": {
    "token_usage": {
      "prompt_tokens": 823,
      "completion_tokens": 58,
      "total_tokens": 881
    },
    "model_name": "gpt-3.5-turbo"
  },
  "run": null
}
[chain/end] [1:chain:RetrievalQA > 3:chain:StuffDocumentsChain > 4:chain:LLMChain] [2.86s] Exiting Chain run with output:
{
  "text": "高清电视机的护理非常简单。您只需要使用干布清洁即可。避免使用湿布或化学清洁剂，以免损坏电视机的表面。"
}
[chain/end] [1:chain:RetrievalQA > 3:chain:StuffDocumentsChain] [2.87s] Exiting Chain run with output:
{
  "output_text": "高清电视机的护理非常简单。您只需要使用干布清洁即可。避免使用湿布或化学清洁剂，以免损坏电视机的表面。"
}
[chain/end] [1:chain:RetrievalQA] [3.26s] Exiting Chain run with output:
{
  "result": "高清电视机的护理非常简单。您只需要使用干布清洁即可。避免使用湿布或化学清洁剂，以免损坏电视机的表面。"
}





'高清电视机的护理非常简单。您只需要使用干布清洁即可。避免使用湿布或化学清洁剂，以免损坏电视机的表面。'

我们可以看到它首先深入到检索 QA 链中，然后它进入了一些文档链。如上所述，我们正在使用 stuff 方法，现在我们正在传递这个上下文，可以看到，这个上下文是由我们检索到的不同文档创建的。因此，在进行问答时，当返回错误结果时，通常不是语言模型本身出错了，实际上是检索步骤出错了，仔细查看问题的确切内容和上下文可以帮助调试出错的原因。

然后，我们可以再向下一级，看看进入语言模型的确切内容，以及 OpenAI 自身，在这里，我们可以看到传递的完整提示，我们有一个系统消息，有所使用的提示的描述，这是问题回答链使用的提示，我们可以看到提示打印出来，使用以下上下文片段回答用户的问题。

如果您不知道答案，只需说您不知道即可，不要试图编造答案。然后我们看到一堆之前插入的上下文，我们还可以看到有关实际返回类型的更多信息。我们不仅仅返回一个答案，还有 token 的使用情况，可以了解到 token 数的使用情况

由于这是一个相对简单的链，我们现在可以看到最终的响应，通过链返回给用户。这部分我们主要讲解了如何查看和调试单个输入到该链的情况。

通过LLM进行评估实例

来简要梳理一下问答评估的流程:

  • 首先，我们使用 LLM 自动构建了问答测试集，包含问题及标准答案。

  • 然后，同一 LLM 试图回答测试集中的所有问题，得到响应。

  • 下一步，需要评估语言模型的回答是否正确。这里奇妙的是，我们再使用另一个 LLM 链进行判断，所以 LLM 既是“球员”，又是“裁判”。

具体来说，第一个语言模型负责回答问题。第二个语言模型链用来进行答案判定。最后我们可以收集判断结果，得到语言模型在这一任务上的效果分数。需要注意的是，回答问题的语言模型链和答案判断链是分开的，职责不同。这避免了同一个模型对自己结果的主观判断。

总之,语言模型可以自动完成构建测试集、回答问题和判定答案等全流程，使评估过程更加智能化和自动化。我们只需要提供文档并解析最终结果即可。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
langchain.debug = False

#为所有不同的示例创建预测
predictions = qa.apply(examples) 

# 对预测的结果进行评估，导入QA问题回答，评估链，通过语言模型创建此链
from langchain.evaluation.qa import QAEvalChain #导入QA问题回答，评估链

#通过调用chatGPT进行评估
llm = ChatOpenAI(temperature=0)
eval_chain = QAEvalChain.from_llm(llm)

#在此链上调用evaluate，进行评估
graded_outputs = eval_chain.evaluate(examples, predictions)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
> Entering new RetrievalQA chain...

> Finished chain.


> Entering new RetrievalQA chain...

> Finished chain.


> Entering new RetrievalQA chain...

> Finished chain.


> Entering new RetrievalQA chain...

> Finished chain.


> Entering new RetrievalQA chain...

> Finished chain.


> Entering new RetrievalQA chain...

> Finished chain.


> Entering new RetrievalQA chain...

> Finished chain.
1
2
3
4
5
6
7
8
#我们将传入示例和预测，得到一堆分级输出，循环遍历它们打印答案
for i, eg in enumerate(examples):
    print(f"Example {i}:")
    print("Question: " + predictions[i]['query'])
    print("Real Answer: " + predictions[i]['answer'])
    print("Predicted Answer: " + predictions[i]['result'])
    print("Predicted Grade: " + graded_outputs[i]['results'])
    print()
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
Example 0:
Question: 高清电视机怎么进行护理？
Real Answer: 使用干布清洁。
Predicted Answer: 高清电视机的护理非常简单。您只需要使用干布清洁即可。避免使用湿布或化学清洁剂，以免损坏电视机的表面。
Predicted Grade: CORRECT

Example 1:
Question: 旅行背包有内外袋吗？
Real Answer: 有。
Predicted Answer: 是的，旅行背包有多个实用的内外袋，可以轻松装下您的必需品。
Predicted Grade: CORRECT

Example 2:
Question: 这款全自动咖啡机的尺寸是多少？
Real Answer: 大型尺寸为13.8'' x 17.3''，中型尺寸为11.5'' x 15.2''。
Predicted Answer: 这款全自动咖啡机有两种尺寸可选：
- 大型尺寸为13.8'' x 17.3''。
- 中型尺寸为11.5'' x 15.2''。
Predicted Grade: CORRECT

Example 3:
Question: 这款电动牙刷的规格是什么？
Real Answer: 一般大小 - 高度：9.5''，宽度：1''。
Predicted Answer: 这款电动牙刷的规格是：高度为9.5英寸，宽度为1英寸。
Predicted Grade: CORRECT

Example 4:
Question: 这种产品的名称是什么？
Real Answer: 这种产品的名称是橙味维生素C泡腾片。
Predicted Answer: 这种产品的名称是儿童益智玩具。
Predicted Grade: INCORRECT

Example 5:
Question: 这款无线蓝牙耳机的尺寸是多少？
Real Answer: 该无线蓝牙耳机的尺寸为1.5'' x 1.3''。
Predicted Answer: 这款无线蓝牙耳机的尺寸是1.5'' x 1.3''。
Predicted Grade: CORRECT

Example 6:
Question: 这款瑜伽垫的尺寸是多少？
Real Answer: 这款瑜伽垫的尺寸是24'' x 68''。
Predicted Answer: 这款瑜伽垫的尺寸是24'' x 68''。
Predicted Grade: CORRECT

从上面的返回结果中我们可以看到，在评估结果中每一个问题中都包含了Question，Real Answer，Predicted Answer和Predicted Grade 四组内容，其中Real Answer是有先前的QAGenerateChain创建的问答测试集中的答案，而Predicted Answer则是由我们的qa链给出的答案，最后的Predicted Grade则是由上面代码中的QAEvalChain回答的。

在本章中，我们学习了使用 LangChain 框架实现 LLM 问答效果自动化评估的方法。与传统手工准备评估集、逐题判断等方式不同，LangChain 使整个评估流程自动化。它可以自动构建包含问答样本的测试集，然后使用语言模型对测试集自动产生回复，最后通过另一个模型链自动判断每个回答的准确性。这种全自动的评估方式极大地简化了问答系统的评估和优化过程，开发者无需手动准备测试用例，也无需逐一判断正确性，大大提升了工作效率。

借助LangChain的自动评估功能，我们可以快速评估语言模型在不同文档集上的问答效果，并可以持续地进行模型调优，无需人工干预。这种自动化的评估方法解放了双手，使我们可以更高效地迭代优化问答系统的性能。

总之，自动评估是 LangChain 框架的一大优势，它将极大地降低问答系统开发的门槛，使任何人都可以轻松训练出性能强大的问答模型。

英文版提示

创建LLM应用
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
from langchain.chains import RetrievalQA 
from langchain.chat_models import ChatOpenAI 
from langchain.document_loaders import CSVLoader 
from langchain.indexes import VectorstoreIndexCreator 
from langchain.vectorstores import DocArrayInMemorySearch 
from langchain.evaluation.qa import QAGenerateChain 
import pandas as pd

file = '../data/OutdoorClothingCatalog_1000.csv'
loader = CSVLoader(file_path=file)
data = loader.load()



test_data = pd.read_csv(file,skiprows=0,usecols=[1,2])
display(test_data.head())

llm = ChatOpenAI(temperature = 0.0)

index = VectorstoreIndexCreator(
    vectorstore_cls=DocArrayInMemorySearch
).from_loaders([loader])

qa = RetrievalQA.from_chain_type(
    llm=llm, 
    chain_type="stuff", 
    retriever=index.vectorstore.as_retriever(), 
    verbose=True,
    chain_type_kwargs = {
        "document_separator": "<<<<>>>>>"
    }
)

print(data[10],"\n")
print(data[11],"\n")

examples = [
    {
        "query": "Do the Cozy Comfort Pullover Set have side pockets?",
        "answer": "Yes"
    },
    {
        "query": "What collection is the Ultra-Lofty 850 Stretch Down Hooded Jacket from?",
        "answer": "The DownTek collection"
    }
]


example_gen_chain = QAGenerateChain.from_llm(ChatOpenAI())


from langchain.evaluation.qa import QAGenerateChain #导入QA生成链，它将接收文档，并从每个文档中创建一个问题答案对
example_gen_chain = QAGenerateChain.from_llm(ChatOpenAI())#通过传递chat open AI语言模型来创建这个链
new_examples = example_gen_chain.apply([{"doc": t} for t in data[:5]]) 

#查看用例数据
print(new_examples) 

examples += [ v for item in new_examples for k,v in item.items()]
qa.run(examples[0]["query"])

name description
0 Women's Campside Oxfords This ultracomfortable lace-to-toe Oxford boast...
1 Recycled Waterhog Dog Mat, Chevron Weave Protect your floors from spills and splashing ...
2 Infant and Toddler Girls' Coastal Chill Swimsu... She'll love the bright colors, ruffles and exc...
3 Refresh Swimwear, V-Neck Tankini Contrasts Whether you're going for a swim or heading out...
4 EcoFlex 3L Storm Pants Our new TEK O2 technology makes our four-seaso... 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
page_content=": 10\nname: Cozy Comfort Pullover Set, Stripe\ndescription: Perfect for lounging, this striped knit set lives up to its name. We used ultrasoft fabric and an easy design that's as comfortable at bedtime as it is when we have to make a quick run out.\n\nSize & Fit\n- Pants are Favorite Fit: Sits lower on the waist.\n- Relaxed Fit: Our most generous fit sits farthest from the body.\n\nFabric & Care\n- In the softest blend of 63% polyester, 35% rayon and 2% spandex.\n\nAdditional Features\n- Relaxed fit top with raglan sleeves and rounded hem.\n- Pull-on pants have a wide elastic waistband and drawstring, side pockets and a modern slim leg.\n\nImported." metadata={'source': '../data/OutdoorClothingCatalog_1000.csv', 'row': 10} 

page_content=': 11\nname: Ultra-Lofty 850 Stretch Down Hooded Jacket\ndescription: This technical stretch down jacket from our DownTek collection is sure to keep you warm and comfortable with its full-stretch construction providing exceptional range of motion. With a slightly fitted style that falls at the hip and best with a midweight layer, this jacket is suitable for light activity up to 20° and moderate activity up to -30°. The soft and durable 100% polyester shell offers complete windproof protection and is insulated with warm, lofty goose down. Other features include welded baffles for a no-stitch construction and excellent stretch, an adjustable hood, an interior media port and mesh stash pocket and a hem drawcord. Machine wash and dry. Imported.' metadata={'source': '../data/OutdoorClothingCatalog_1000.csv', 'row': 11} 

[{'qa_pairs': {'query': "What is the description of the Women's Campside Oxfords?", 'answer': "The description of the Women's Campside Oxfords is that they are an ultracomfortable lace-to-toe Oxford made of super-soft canvas. They have thick cushioning and quality construction, providing a broken-in feel from the first time they are worn."}}, {'qa_pairs': {'query': 'What are the dimensions of the small and medium sizes of the Recycled Waterhog Dog Mat, Chevron Weave?', 'answer': 'The dimensions of the small size of the Recycled Waterhog Dog Mat, Chevron Weave are 18" x 28". The dimensions of the medium size are 22.5" x 34.5".'}}, {'qa_pairs': {'query': "What are the features of the Infant and Toddler Girls' Coastal Chill Swimsuit, Two-Piece?", 'answer': "The swimsuit has bright colors, ruffles, and exclusive whimsical prints. It is made of four-way-stretch and chlorine-resistant fabric, which keeps its shape and resists snags. The fabric is UPF 50+ rated, providing the highest rated sun protection possible by blocking 98% of the sun's harmful rays. The swimsuit also has crossover no-slip straps and a fully lined bottom for a secure fit and maximum coverage."}}, {'qa_pairs': {'query': 'What is the fabric composition of the Refresh Swimwear, V-Neck Tankini Contrasts?', 'answer': 'The Refresh Swimwear, V-Neck Tankini Contrasts is made of 82% recycled nylon and 18% Lycra® spandex for the body, and 90% recycled nylon with 10% Lycra® spandex for the lining.'}}, {'qa_pairs': {'query': 'What is the fabric composition of the EcoFlex 3L Storm Pants?', 'answer': 'The EcoFlex 3L Storm Pants are made of 100% nylon, exclusive of trim.'}}]


> Entering new RetrievalQA chain...

> Finished chain.





'Yes, the Cozy Comfort Pullover Set does have side pockets.'
人工评估
1
2
3
4
5
6
7
import langchain
langchain.debug = True

#重新运行与上面相同的示例，可以看到它开始打印出更多的信息
qa.run(examples[0]["query"])

langchain.debug = False
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
[chain/start] [1:chain:RetrievalQA] Entering Chain run with input:
{
  "query": "Do the Cozy Comfort Pullover Set have side pockets?"
}
[chain/start] [1:chain:RetrievalQA > 3:chain:StuffDocumentsChain] Entering Chain run with input:
[inputs]
[chain/start] [1:chain:RetrievalQA > 3:chain:StuffDocumentsChain > 4:chain:LLMChain] Entering Chain run with input:
{
  "question": "Do the Cozy Comfort Pullover Set have side pockets?",
  "context": ": 10\nname: Cozy Comfort Pullover Set, Stripe\ndescription: Perfect for lounging, this striped knit set lives up to its name. We used ultrasoft fabric and an easy design that's as comfortable at bedtime as it is when we have to make a quick run out.\n\nSize & Fit\n- Pants are Favorite Fit: Sits lower on the waist.\n- Relaxed Fit: Our most generous fit sits farthest from the body.\n\nFabric & Care\n- In the softest blend of 63% polyester, 35% rayon and 2% spandex.\n\nAdditional Features\n- Relaxed fit top with raglan sleeves and rounded hem.\n- Pull-on pants have a wide elastic waistband and drawstring, side pockets and a modern slim leg.\n\nImported.<<<<>>>>>: 73\nname: Cozy Cuddles Knit Pullover Set\ndescription: Perfect for lounging, this knit set lives up to its name. We used ultrasoft fabric and an easy design that's as comfortable at bedtime as it is when we have to make a quick run out. \n\nSize & Fit \nPants are Favorite Fit: Sits lower on the waist. \nRelaxed Fit: Our most generous fit sits farthest from the body. \n\nFabric & Care \nIn the softest blend of 63% polyester, 35% rayon and 2% spandex.\n\nAdditional Features \nRelaxed fit top with raglan sleeves and rounded hem. \nPull-on pants have a wide elastic waistband and drawstring, side pockets and a modern slim leg. \nImported.<<<<>>>>>: 151\nname: Cozy Quilted Sweatshirt\ndescription: Our sweatshirt is an instant classic with its great quilted texture and versatile weight that easily transitions between seasons. With a traditional fit that is relaxed through the chest, sleeve, and waist, this pullover is lightweight enough to be worn most months of the year. The cotton blend fabric is super soft and comfortable, making it the perfect casual layer. To make dressing easy, this sweatshirt also features a snap placket and a heritage-inspired Mt. Katahdin logo patch. For care, machine wash and dry. Imported.<<<<>>>>>: 265\nname: Cozy Workout Vest\ndescription: For serious warmth that won't weigh you down, reach for this fleece-lined vest, which provides you with layering options whether you're inside or outdoors.\nSize & Fit\nRelaxed Fit. Falls at hip.\nFabric & Care\nSoft, textured fleece lining. Nylon shell. Machine wash and dry. \nAdditional Features \nTwo handwarmer pockets. Knit side panels stretch for a more flattering fit. Shell fabric is treated to resist water and stains. Imported."
}
[llm/start] [1:chain:RetrievalQA > 3:chain:StuffDocumentsChain > 4:chain:LLMChain > 5:llm:ChatOpenAI] Entering LLM run with input:
{
  "prompts": [
    "System: Use the following pieces of context to answer the users question. \nIf you don't know the answer, just say that you don't know, don't try to make up an answer.\n----------------\n: 10\nname: Cozy Comfort Pullover Set, Stripe\ndescription: Perfect for lounging, this striped knit set lives up to its name. We used ultrasoft fabric and an easy design that's as comfortable at bedtime as it is when we have to make a quick run out.\n\nSize & Fit\n- Pants are Favorite Fit: Sits lower on the waist.\n- Relaxed Fit: Our most generous fit sits farthest from the body.\n\nFabric & Care\n- In the softest blend of 63% polyester, 35% rayon and 2% spandex.\n\nAdditional Features\n- Relaxed fit top with raglan sleeves and rounded hem.\n- Pull-on pants have a wide elastic waistband and drawstring, side pockets and a modern slim leg.\n\nImported.<<<<>>>>>: 73\nname: Cozy Cuddles Knit Pullover Set\ndescription: Perfect for lounging, this knit set lives up to its name. We used ultrasoft fabric and an easy design that's as comfortable at bedtime as it is when we have to make a quick run out. \n\nSize & Fit \nPants are Favorite Fit: Sits lower on the waist. \nRelaxed Fit: Our most generous fit sits farthest from the body. \n\nFabric & Care \nIn the softest blend of 63% polyester, 35% rayon and 2% spandex.\n\nAdditional Features \nRelaxed fit top with raglan sleeves and rounded hem. \nPull-on pants have a wide elastic waistband and drawstring, side pockets and a modern slim leg. \nImported.<<<<>>>>>: 151\nname: Cozy Quilted Sweatshirt\ndescription: Our sweatshirt is an instant classic with its great quilted texture and versatile weight that easily transitions between seasons. With a traditional fit that is relaxed through the chest, sleeve, and waist, this pullover is lightweight enough to be worn most months of the year. The cotton blend fabric is super soft and comfortable, making it the perfect casual layer. To make dressing easy, this sweatshirt also features a snap placket and a heritage-inspired Mt. Katahdin logo patch. For care, machine wash and dry. Imported.<<<<>>>>>: 265\nname: Cozy Workout Vest\ndescription: For serious warmth that won't weigh you down, reach for this fleece-lined vest, which provides you with layering options whether you're inside or outdoors.\nSize & Fit\nRelaxed Fit. Falls at hip.\nFabric & Care\nSoft, textured fleece lining. Nylon shell. Machine wash and dry. \nAdditional Features \nTwo handwarmer pockets. Knit side panels stretch for a more flattering fit. Shell fabric is treated to resist water and stains. Imported.\nHuman: Do the Cozy Comfort Pullover Set have side pockets?"
  ]
}
[llm/end] [1:chain:RetrievalQA > 3:chain:StuffDocumentsChain > 4:chain:LLMChain > 5:llm:ChatOpenAI] [879.746ms] Exiting LLM run with output:
{
  "generations": [
    [
      {
        "text": "Yes, the Cozy Comfort Pullover Set does have side pockets.",
        "generation_info": {
          "finish_reason": "stop"
        },
        "message": {
          "lc": 1,
          "type": "constructor",
          "id": [
            "langchain",
            "schema",
            "messages",
            "AIMessage"
          ],
          "kwargs": {
            "content": "Yes, the Cozy Comfort Pullover Set does have side pockets.",
            "additional_kwargs": {}
          }
        }
      }
    ]
  ],
  "llm_output": {
    "token_usage": {
      "prompt_tokens": 626,
      "completion_tokens": 14,
      "total_tokens": 640
    },
    "model_name": "gpt-3.5-turbo"
  },
  "run": null
}
[chain/end] [1:chain:RetrievalQA > 3:chain:StuffDocumentsChain > 4:chain:LLMChain] [880.5269999999999ms] Exiting Chain run with output:
{
  "text": "Yes, the Cozy Comfort Pullover Set does have side pockets."
}
[chain/end] [1:chain:RetrievalQA > 3:chain:StuffDocumentsChain] [881.4499999999999ms] Exiting Chain run with output:
{
  "output_text": "Yes, the Cozy Comfort Pullover Set does have side pockets."
}
[chain/end] [1:chain:RetrievalQA] [1.21s] Exiting Chain run with output:
{
  "result": "Yes, the Cozy Comfort Pullover Set does have side pockets."
}
通过LLM进行评估实例
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
langchain.debug = False

#为所有不同的示例创建预测
predictions = qa.apply(examples) 

# 对预测的结果进行评估，导入QA问题回答，评估链，通过语言模型创建此链
from langchain.evaluation.qa import QAEvalChain #导入QA问题回答，评估链

#通过调用chatGPT进行评估
llm = ChatOpenAI(temperature=0)
eval_chain = QAEvalChain.from_llm(llm)

#在此链上调用evaluate，进行评估
graded_outputs = eval_chain.evaluate(examples, predictions)

#我们将传入示例和预测，得到一堆分级输出，循环遍历它们打印答案
for i, eg in enumerate(examples):
    print(f"Example {i}:")
    print("Question: " + predictions[i]['query'])
    print("Real Answer: " + predictions[i]['answer'])
    print("Predicted Answer: " + predictions[i]['result'])
    print("Predicted Grade: " + graded_outputs[i]['results'])
    print()
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
> Entering new RetrievalQA chain...

> Finished chain.


> Entering new RetrievalQA chain...

> Finished chain.


> Entering new RetrievalQA chain...

> Finished chain.


> Entering new RetrievalQA chain...

> Finished chain.


> Entering new RetrievalQA chain...

> Finished chain.


> Entering new RetrievalQA chain...

> Finished chain.


> Entering new RetrievalQA chain...

> Finished chain.
Example 0:
Question: Do the Cozy Comfort Pullover Set have side pockets?
Real Answer: Yes
Predicted Answer: Yes, the Cozy Comfort Pullover Set does have side pockets.
Predicted Grade: CORRECT

Example 1:
Question: What collection is the Ultra-Lofty 850 Stretch Down Hooded Jacket from?
Real Answer: The DownTek collection
Predicted Answer: The Ultra-Lofty 850 Stretch Down Hooded Jacket is from the DownTek collection.
Predicted Grade: CORRECT

Example 2:
Question: What is the description of the Women's Campside Oxfords?
Real Answer: The description of the Women's Campside Oxfords is that they are an ultracomfortable lace-to-toe Oxford made of super-soft canvas. They have thick cushioning and quality construction, providing a broken-in feel from the first time they are worn.
Predicted Answer: The description of the Women's Campside Oxfords is: "This ultracomfortable lace-to-toe Oxford boasts a super-soft canvas, thick cushioning, and quality construction for a broken-in feel from the first time you put them on."
Predicted Grade: CORRECT

Example 3:
Question: What are the dimensions of the small and medium sizes of the Recycled Waterhog Dog Mat, Chevron Weave?
Real Answer: The dimensions of the small size of the Recycled Waterhog Dog Mat, Chevron Weave are 18" x 28". The dimensions of the medium size are 22.5" x 34.5".
Predicted Answer: The dimensions of the small size of the Recycled Waterhog Dog Mat, Chevron Weave are 18" x 28". The dimensions of the medium size are 22.5" x 34.5".
Predicted Grade: CORRECT

Example 4:
Question: What are the features of the Infant and Toddler Girls' Coastal Chill Swimsuit, Two-Piece?
Real Answer: The swimsuit has bright colors, ruffles, and exclusive whimsical prints. It is made of four-way-stretch and chlorine-resistant fabric, which keeps its shape and resists snags. The fabric is UPF 50+ rated, providing the highest rated sun protection possible by blocking 98% of the sun's harmful rays. The swimsuit also has crossover no-slip straps and a fully lined bottom for a secure fit and maximum coverage.
Predicted Answer: The features of the Infant and Toddler Girls' Coastal Chill Swimsuit, Two-Piece are:

- Bright colors and ruffles
- Exclusive whimsical prints
- Four-way-stretch and chlorine-resistant fabric
- UPF 50+ rated fabric for sun protection
- Crossover no-slip straps
- Fully lined bottom for a secure fit and maximum coverage
- Machine washable and line dry for best results
- Imported
Predicted Grade: CORRECT

Example 5:
Question: What is the fabric composition of the Refresh Swimwear, V-Neck Tankini Contrasts?
Real Answer: The Refresh Swimwear, V-Neck Tankini Contrasts is made of 82% recycled nylon and 18% Lycra® spandex for the body, and 90% recycled nylon with 10% Lycra® spandex for the lining.
Predicted Answer: The fabric composition of the Refresh Swimwear, V-Neck Tankini Contrasts is 82% recycled nylon with 18% Lycra® spandex for the body, and 90% recycled nylon with 10% Lycra® spandex for the lining.
Predicted Grade: CORRECT

Example 6:
Question: What is the fabric composition of the EcoFlex 3L Storm Pants?
Real Answer: The EcoFlex 3L Storm Pants are made of 100% nylon, exclusive of trim.
Predicted Answer: The fabric composition of the EcoFlex 3L Storm Pants is 100% nylon, exclusive of trim.
Predicted Grade: CORRECT
```    


### 代理 Agent

大型语言模型（LLMs）非常强大，但它们缺乏“最笨”的计算机程序可以轻松处理的特定能力。LLM 对逻辑推理、计算和检索外部信息的能力较弱，这与最简单的计算机程序形成对比。例如，语言模型无法准确回答简单的计算问题，还有当询问最近发生的事件时，其回答也可能过时或错误，因为无法主动获取最新信息。这是由于当前语言模型仅依赖预训练数据，与外界“断开”。要克服这一缺陷，`LangChain`框架提出了`“代理”(Agent)`的解决方案。

**代理作为语言模型的外部模块，可提供计算、逻辑、检索等功能的支持，使语言模型获得异常强大的推理和获取信息的超能力**。

在本章中，我们将详细介绍代理的工作机制、种类、以及如何在`LangChain`中将其与语言模型配合，构建功能更全面、智能程度更高的应用程序。代理机制极大扩展了语言模型的边界，是当前提升其智能的重要途径之一。让我们开始学习如何通过代理释放语言模型的最大潜力。

#### 使用LangChain内置工具llm-math和wikipedia

要使用代理 (Agents) ，我们需要三样东西：

- 一个基本的 LLM
- 我们将要进行交互的工具 Tools
- 一个控制交互的代理 (Agents) 。

```python
from langchain.agents import load_tools, initialize_agent
from langchain.agents import AgentType
from langchain.python import PythonREPL
from langchain.chat_models import ChatOpenAI

首先，让我们新建一个基本的 LLM

1
2
# 参数temperature设置为0.0，从而减少生成答案的随机性。
llm = ChatOpenAI(temperature=0)

接下来，初始化工具 Tool ，我们可以创建自定义工具 Tool 或加载预构建工具 Tool。无论哪种情况，工具 Tool 都是一个给定工具 名称 name 和 描述 description 的 实用链。

  • llm-math 工具结合语言模型和计算器用以进行数学计算
  • wikipedia工具通过API连接到wikipedia进行搜索查询。
1
2
3
4
5

tools = load_tools(
    ["llm-math","wikipedia"], 
    llm=llm #第一步初始化的模型
)

现在我们有了 LLM 和工具，最后让我们初始化一个简单的代理 (Agents) ：

1
2
3
4
5
6
7
8
# 初始化代理
agent= initialize_agent(
    tools, #第二步加载的工具
    llm, #第一步初始化的模型
    agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION,  #代理类型
    handle_parsing_errors=True, #处理解析错误
    verbose = True #输出中间步骤
)
  • agent: 代理类型。这里使用的是AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION。其中CHAT代表代理模型为针对对话优化的模型；Zero-shot 意味着代理 (Agents) 仅在当前操作上起作用，即它没有记忆；REACT代表针对REACT设计的提示模版。DESCRIPTION根据工具的描述 description 来决定使用哪个工具。(我们不会在本章中讨论 * REACT 框架 ，但您可以将其视为 LLM 可以循环进行 Reasoning 和 Action 步骤的过程。它启用了一个多步骤的过程来识别答案。)
  • handle_parsing_errors: 是否处理解析错误。当发生解析错误时，将错误信息返回给大模型，让其进行纠正。
  • verbose: 是否输出中间步骤结果。

使用代理回答数学问题

1
agent("计算300的25%") 
1
2
3
4
5
6
7
8
9
> Entering new AgentExecutor chain...
Question: 计算300的25%
Thought: I can use the calculator tool to calculate 25% of 300.
Action:
```json
{
  "action": "Calculator",
  "action_input": "300 * 0.25"
}
Observation: Answer: 75.0
Thought:The calculator tool returned the answer 75.0, which is 25% of 300.
Final Answer: 25% of 300 is 75.0.

> Finished chain.





{'input': '计算300的25%', 'output': '25% of 300 is 75.0.'}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

**上面的过程可以总结为下**

1. 模型对于接下来需要做什么，给出思考

   <p style="font-family:verdana; font-size:12px;color:green"> <strong>思考</strong>：我可以使用计算工具来计算300的25%</p>

2. 模型基于思考采取行动
    <p style="font-family:verdana; font-size:12px;color:green"> <strong>行动</strong>: 使用计算器（calculator），输入（action_input）300*0.25</p>
3. 模型得到观察
    <p style="font-family:verdana; font-size:12px;color:green"><strong>观察</strong>：答案: 75.0</p>

4. 基于观察，模型对于接下来需要做什么，给出思考
    <p style="font-family:verdana; font-size:12px;color:green"> <strong>思考</strong>: 计算工具返回了300的25%，答案为75 </p>

5. 给出最终答案（Final Answer）
     <p style="font-family:verdana; font-size:12px;color:green"> <strong>最终答案</strong>: 300的25%等于75。 </p>
5. 以字典的形式给出最终答案。

Tom M. Mitchell的书

```python
question = "Tom M. Mitchell是一位美国计算机科学家，\
也是卡内基梅隆大学（CMU）的创始人大学教授。\
他写了哪本书呢？"

agent(question) 
1
2
3
4
5
6
7
8
> Entering new AgentExecutor chain...
Thought: I can use Wikipedia to find information about Tom M. Mitchell and his books.
Action:
```json
{
  "action": "Wikipedia",
  "action_input": "Tom M. Mitchell"
}
Observation: Page: Tom M. Mitchell
Summary: Tom Michael Mitchell (born August 9, 1951) is an American computer scientist and the Founders University Professor at Carnegie Mellon University (CMU). He is a founder and former Chair of the Machine Learning Department at CMU. Mitchell is known for his contributions to the advancement of machine learning, artificial intelligence, and cognitive neuroscience and is the author of the textbook Machine Learning. He is a member of the United States National Academy of Engineering since 2010. He is also a Fellow of the American Academy of Arts and Sciences, the American Association for the Advancement of Science and a Fellow and past President of the Association for the Advancement of Artificial Intelligence. In October 2018, Mitchell was appointed as the Interim Dean of the School of Computer Science at Carnegie Mellon.

Page: Tom Mitchell (Australian footballer)
Summary: Thomas Mitchell (born 31 May 1993) is a professional Australian rules footballer playing for the Collingwood Football Club in the Australian Football League (AFL). He previously played for the Adelaide Crows, Sydney Swans from 2012 to 2016, and the Hawthorn Football Club between 2017 and 2022. Mitchell won the Brownlow Medal as the league's best and fairest player in 2018 and set the record for the most disposals in a VFL/AFL match, accruing 54 in a game against Collingwood during that season.
Thought:The book written by Tom M. Mitchell is "Machine Learning".
Thought: I have found the answer.
Final Answer: The book written by Tom M. Mitchell is "Machine Learning".

> Finished chain.





{'input': 'Tom M. Mitchell是一位美国计算机科学家，也是卡内基梅隆大学（CMU）的创始人大学教授。他写了哪本书呢？',
 'output': 'The book written by Tom M. Mitchell is "Machine Learning".'}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38


✅ **总结**

1. 模型对于接下来需要做什么，给出思考（Thought）
   <p style="font-family:verdana; font-size:12px;color:green"> <strong>思考</strong>：我应该使用维基百科去搜索。</p>

2. 模型基于思考采取行动（Action）
    <p style="font-family:verdana; font-size:12px;color:green"> <strong>行动</strong>: 使用维基百科，输入Tom M. Mitchell</p>
3. 模型得到观察（Observation）
    <p style="font-family:verdana; font-size:12px;color:green"><strong>观测</strong>: 页面: Tom M. Mitchell，页面: Tom Mitchell (澳大利亚足球运动员)</p>

4. 基于观察，模型对于接下来需要做什么，给出思考（Thought）
    <p style="font-family:verdana; font-size:12px;color:green"> <strong>思考</strong>: Tom M. Mitchell写的书是Machine Learning </p>

5. 给出最终答案（Final Answer）
     <p style="font-family:verdana; font-size:12px;color:green"> <strong>最终答案</strong>: Machine Learning </p>
5. 以字典的形式给出最终答案。


值得注意的是，模型每次运行推理的过程可能存在差异，但最终的结果一致。

#### 使用LangChain内置工具PythonREPLTool

我们创建一个能将顾客名字转换为拼音的 python 代理，步骤与上一部分的一样：

```python
from langchain.agents.agent_toolkits import create_python_agent
from langchain.tools.python.tool import PythonREPLTool

agent = create_python_agent(
    llm,  #使用前面一节已经加载的大语言模型
    tool=PythonREPLTool(), #使用Python交互式环境工具 REPLTool
    verbose=True #输出中间步骤
)
customer_list = ["小明","小黄","小红","小蓝","小橘","小绿",]

agent.run(f"将使用pinyin拼音库这些客户名字转换为拼音，并打印输出列表: {customer_list}。") 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
> Entering new AgentExecutor chain...


Python REPL can execute arbitrary code. Use with caution.


I need to use the pinyin library to convert the names to pinyin. I can then print out the list of converted names.
Action: Python_REPL
Action Input: import pinyin
Observation: 
Thought:I have imported the pinyin library. Now I can use it to convert the names to pinyin.
Action: Python_REPL
Action Input: names = ['小明', '小黄', '小红', '小蓝', '小橘', '小绿']
pinyin_names = [pinyin.get(i, format='strip') for i in names]
print(pinyin_names)
Observation: ['xiaoming', 'xiaohuang', 'xiaohong', 'xiaolan', 'xiaoju', 'xiaolv']

Thought:I have successfully converted the names to pinyin and printed out the list of converted names.
Final Answer: ['xiaoming', 'xiaohuang', 'xiaohong', 'xiaolan', 'xiaoju', 'xiaolv']

> Finished chain.





"['xiaoming', 'xiaohuang', 'xiaohong', 'xiaolan', 'xiaoju', 'xiaolv']"

在调试（debug）模式下再次运行，我们可以把上面的6步分别对应到下面的具体流程

  1. 模型对于接下来需要做什么，给出思考（Thought）

    • [chain/start] [1:chain:AgentExecutor] Entering Chain run with input

    • [chain/start] [1:chain:AgentExecutor > 2:chain:LLMChain] Entering Chain run with input

    • [llm/start] [1:chain:AgentExecutor > 2:chain:LLMChain > 3:llm:ChatOpenAI] Entering LLM run with input

    • [llm/end] [1:chain:AgentExecutor > 2:chain:LLMChain > 3:llm:ChatOpenAI] [1.91s] Exiting LLM run with output

    • [chain/end] [1:chain:AgentExecutor > 2:chain:LLMChain] [1.91s] Exiting Chain run with output

  2. 模型基于思考采取行动（Action), 因为使用的工具不同，Action的输出也和之前有所不同，这里输出的为python代码 import pinyin

    • [tool/start] [1:chain:AgentExecutor > 4:tool:Python REPL] Entering Tool run with input

    • [tool/end] [1:chain:AgentExecutor > 4:tool:Python_REPL] [1.28ms] Exiting Tool run with output

  3. 模型得到观察（Observation）

    • [chain/start] [1:chain:AgentExecutor > 5:chain:LLMChain] Entering Chain run with input

  4. 基于观察，模型对于接下来需要做什么，给出思考（Thought）

    • [llm/start] [1:chain:AgentExecutor > 5:chain:LLMChain > 6:llm:ChatOpenAI] Entering LLM run with input

    • [llm/end] [1:chain:AgentExecutor > 5:chain:LLMChain > 6:llm:ChatOpenAI] [3.48s] Exiting LLM run with output

  5. 给出最终答案（Final Answer）

    • [chain/end] [1:chain:AgentExecutor > 5:chain:LLMChain] [3.48s] Exiting Chain run with output

  6. 返回最终答案。

    • [chain/end] [1:chain:AgentExecutor] [19.20s] Exiting Chain run with output

1
2
3
4
import langchain
langchain.debug=True
agent.run(f"使用pinyin拼音库将这些客户名字转换为拼音，并打印输出列表: {customer_list}") 
langchain.debug=False
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
[chain/start] [1:chain:AgentExecutor] Entering Chain run with input:
{
  "input": "使用pinyin拼音库将这些客户名字转换为拼音，并打印输出列表: ['小明', '小黄', '小红', '小蓝', '小橘', '小绿']"
}
[chain/start] [1:chain:AgentExecutor > 2:chain:LLMChain] Entering Chain run with input:
{
  "input": "使用pinyin拼音库将这些客户名字转换为拼音，并打印输出列表: ['小明', '小黄', '小红', '小蓝', '小橘', '小绿']",
  "agent_scratchpad": "",
  "stop": [
    "\nObservation:",
    "\n\tObservation:"
  ]
}
[llm/start] [1:chain:AgentExecutor > 2:chain:LLMChain > 3:llm:ChatOpenAI] Entering LLM run with input:
{
  "prompts": [
    "Human: You are an agent designed to write and execute python code to answer questions.\nYou have access to a python REPL, which you can use to execute python code.\nIf you get an error, debug your code and try again.\nOnly use the output of your code to answer the question. \nYou might know the answer without running any code, but you should still run the code to get the answer.\nIf it does not seem like you can write code to answer the question, just return \"I don't know\" as the answer.\n\n\nPython_REPL: A Python shell. Use this to execute python commands. Input should be a valid python command. If you want to see the output of a value, you should print it out with `print(...)`.\n\nUse the following format:\n\nQuestion: the input question you must answer\nThought: you should always think about what to do\nAction: the action to take, should be one of [Python_REPL]\nAction Input: the input to the action\nObservation: the result of the action\n... (this Thought/Action/Action Input/Observation can repeat N times)\nThought: I now know the final answer\nFinal Answer: the final answer to the original input question\n\nBegin!\n\nQuestion: 使用pinyin拼音库将这些客户名字转换为拼音，并打印输出列表: ['小明', '小黄', '小红', '小蓝', '小橘', '小绿']\nThought:"
  ]
}
[llm/end] [1:chain:AgentExecutor > 2:chain:LLMChain > 3:llm:ChatOpenAI] [2.32s] Exiting LLM run with output:
{
  "generations": [
    [
      {
        "text": "I need to use the pinyin library to convert the names to pinyin. I can then print out the list of converted names.\nAction: Python_REPL\nAction Input: import pinyin",
        "generation_info": {
          "finish_reason": "stop"
        },
        "message": {
          "lc": 1,
          "type": "constructor",
          "id": [
            "langchain",
            "schema",
            "messages",
            "AIMessage"
          ],
          "kwargs": {
            "content": "I need to use the pinyin library to convert the names to pinyin. I can then print out the list of converted names.\nAction: Python_REPL\nAction Input: import pinyin",
            "additional_kwargs": {}
          }
        }
      }
    ]
  ],
  "llm_output": {
    "token_usage": {
      "prompt_tokens": 320,
      "completion_tokens": 39,
      "total_tokens": 359
    },
    "model_name": "gpt-3.5-turbo"
  },
  "run": null
}
[chain/end] [1:chain:AgentExecutor > 2:chain:LLMChain] [2.33s] Exiting Chain run with output:
{
  "text": "I need to use the pinyin library to convert the names to pinyin. I can then print out the list of converted names.\nAction: Python_REPL\nAction Input: import pinyin"
}
[tool/start] [1:chain:AgentExecutor > 4:tool:Python_REPL] Entering Tool run with input:
"import pinyin"
[tool/end] [1:chain:AgentExecutor > 4:tool:Python_REPL] [1.5659999999999998ms] Exiting Tool run with output:
""
[chain/start] [1:chain:AgentExecutor > 5:chain:LLMChain] Entering Chain run with input:
{
  "input": "使用pinyin拼音库将这些客户名字转换为拼音，并打印输出列表: ['小明', '小黄', '小红', '小蓝', '小橘', '小绿']",
  "agent_scratchpad": "I need to use the pinyin library to convert the names to pinyin. I can then print out the list of converted names.\nAction: Python_REPL\nAction Input: import pinyin\nObservation: \nThought:",
  "stop": [
    "\nObservation:",
    "\n\tObservation:"
  ]
}
[llm/start] [1:chain:AgentExecutor > 5:chain:LLMChain > 6:llm:ChatOpenAI] Entering LLM run with input:
{
  "prompts": [
    "Human: You are an agent designed to write and execute python code to answer questions.\nYou have access to a python REPL, which you can use to execute python code.\nIf you get an error, debug your code and try again.\nOnly use the output of your code to answer the question. \nYou might know the answer without running any code, but you should still run the code to get the answer.\nIf it does not seem like you can write code to answer the question, just return \"I don't know\" as the answer.\n\n\nPython_REPL: A Python shell. Use this to execute python commands. Input should be a valid python command. If you want to see the output of a value, you should print it out with `print(...)`.\n\nUse the following format:\n\nQuestion: the input question you must answer\nThought: you should always think about what to do\nAction: the action to take, should be one of [Python_REPL]\nAction Input: the input to the action\nObservation: the result of the action\n... (this Thought/Action/Action Input/Observation can repeat N times)\nThought: I now know the final answer\nFinal Answer: the final answer to the original input question\n\nBegin!\n\nQuestion: 使用pinyin拼音库将这些客户名字转换为拼音，并打印输出列表: ['小明', '小黄', '小红', '小蓝', '小橘', '小绿']\nThought:I need to use the pinyin library to convert the names to pinyin. I can then print out the list of converted names.\nAction: Python_REPL\nAction Input: import pinyin\nObservation: \nThought:"
  ]
}
[llm/end] [1:chain:AgentExecutor > 5:chain:LLMChain > 6:llm:ChatOpenAI] [4.09s] Exiting LLM run with output:
{
  "generations": [
    [
      {
        "text": "I have imported the pinyin library. Now I can use it to convert the names to pinyin.\nAction: Python_REPL\nAction Input: names = ['小明', '小黄', '小红', '小蓝', '小橘', '小绿']\npinyin_names = [pinyin.get(i, format='strip') for i in names]\nprint(pinyin_names)",
        "generation_info": {
          "finish_reason": "stop"
        },
        "message": {
          "lc": 1,
          "type": "constructor",
          "id": [
            "langchain",
            "schema",
            "messages",
            "AIMessage"
          ],
          "kwargs": {
            "content": "I have imported the pinyin library. Now I can use it to convert the names to pinyin.\nAction: Python_REPL\nAction Input: names = ['小明', '小黄', '小红', '小蓝', '小橘', '小绿']\npinyin_names = [pinyin.get(i, format='strip') for i in names]\nprint(pinyin_names)",
            "additional_kwargs": {}
          }
        }
      }
    ]
  ],
  "llm_output": {
    "token_usage": {
      "prompt_tokens": 365,
      "completion_tokens": 87,
      "total_tokens": 452
    },
    "model_name": "gpt-3.5-turbo"
  },
  "run": null
}
[chain/end] [1:chain:AgentExecutor > 5:chain:LLMChain] [4.09s] Exiting Chain run with output:
{
  "text": "I have imported the pinyin library. Now I can use it to convert the names to pinyin.\nAction: Python_REPL\nAction Input: names = ['小明', '小黄', '小红', '小蓝', '小橘', '小绿']\npinyin_names = [pinyin.get(i, format='strip') for i in names]\nprint(pinyin_names)"
}
[tool/start] [1:chain:AgentExecutor > 7:tool:Python_REPL] Entering Tool run with input:
"names = ['小明', '小黄', '小红', '小蓝', '小橘', '小绿']
pinyin_names = [pinyin.get(i, format='strip') for i in names]
print(pinyin_names)"
[tool/end] [1:chain:AgentExecutor > 7:tool:Python_REPL] [0.8809999999999999ms] Exiting Tool run with output:
"['xiaoming', 'xiaohuang', 'xiaohong', 'xiaolan', 'xiaoju', 'xiaolv']"
[chain/start] [1:chain:AgentExecutor > 8:chain:LLMChain] Entering Chain run with input:
{
  "input": "使用pinyin拼音库将这些客户名字转换为拼音，并打印输出列表: ['小明', '小黄', '小红', '小蓝', '小橘', '小绿']",
  "agent_scratchpad": "I need to use the pinyin library to convert the names to pinyin. I can then print out the list of converted names.\nAction: Python_REPL\nAction Input: import pinyin\nObservation: \nThought:I have imported the pinyin library. Now I can use it to convert the names to pinyin.\nAction: Python_REPL\nAction Input: names = ['小明', '小黄', '小红', '小蓝', '小橘', '小绿']\npinyin_names = [pinyin.get(i, format='strip') for i in names]\nprint(pinyin_names)\nObservation: ['xiaoming', 'xiaohuang', 'xiaohong', 'xiaolan', 'xiaoju', 'xiaolv']\n\nThought:",
  "stop": [
    "\nObservation:",
    "\n\tObservation:"
  ]
}
[llm/start] [1:chain:AgentExecutor > 8:chain:LLMChain > 9:llm:ChatOpenAI] Entering LLM run with input:
{
  "prompts": [
    "Human: You are an agent designed to write and execute python code to answer questions.\nYou have access to a python REPL, which you can use to execute python code.\nIf you get an error, debug your code and try again.\nOnly use the output of your code to answer the question. \nYou might know the answer without running any code, but you should still run the code to get the answer.\nIf it does not seem like you can write code to answer the question, just return \"I don't know\" as the answer.\n\n\nPython_REPL: A Python shell. Use this to execute python commands. Input should be a valid python command. If you want to see the output of a value, you should print it out with `print(...)`.\n\nUse the following format:\n\nQuestion: the input question you must answer\nThought: you should always think about what to do\nAction: the action to take, should be one of [Python_REPL]\nAction Input: the input to the action\nObservation: the result of the action\n... (this Thought/Action/Action Input/Observation can repeat N times)\nThought: I now know the final answer\nFinal Answer: the final answer to the original input question\n\nBegin!\n\nQuestion: 使用pinyin拼音库将这些客户名字转换为拼音，并打印输出列表: ['小明', '小黄', '小红', '小蓝', '小橘', '小绿']\nThought:I need to use the pinyin library to convert the names to pinyin. I can then print out the list of converted names.\nAction: Python_REPL\nAction Input: import pinyin\nObservation: \nThought:I have imported the pinyin library. Now I can use it to convert the names to pinyin.\nAction: Python_REPL\nAction Input: names = ['小明', '小黄', '小红', '小蓝', '小橘', '小绿']\npinyin_names = [pinyin.get(i, format='strip') for i in names]\nprint(pinyin_names)\nObservation: ['xiaoming', 'xiaohuang', 'xiaohong', 'xiaolan', 'xiaoju', 'xiaolv']\n\nThought:"
  ]
}
[llm/end] [1:chain:AgentExecutor > 8:chain:LLMChain > 9:llm:ChatOpenAI] [2.05s] Exiting LLM run with output:
{
  "generations": [
    [
      {
        "text": "I have successfully converted the names to pinyin and printed out the list of converted names.\nFinal Answer: ['xiaoming', 'xiaohuang', 'xiaohong', 'xiaolan', 'xiaoju', 'xiaolv']",
        "generation_info": {
          "finish_reason": "stop"
        },
        "message": {
          "lc": 1,
          "type": "constructor",
          "id": [
            "langchain",
            "schema",
            "messages",
            "AIMessage"
          ],
          "kwargs": {
            "content": "I have successfully converted the names to pinyin and printed out the list of converted names.\nFinal Answer: ['xiaoming', 'xiaohuang', 'xiaohong', 'xiaolan', 'xiaoju', 'xiaolv']",
            "additional_kwargs": {}
          }
        }
      }
    ]
  ],
  "llm_output": {
    "token_usage": {
      "prompt_tokens": 483,
      "completion_tokens": 48,
      "total_tokens": 531
    },
    "model_name": "gpt-3.5-turbo"
  },
  "run": null
}
[chain/end] [1:chain:AgentExecutor > 8:chain:LLMChain] [2.05s] Exiting Chain run with output:
{
  "text": "I have successfully converted the names to pinyin and printed out the list of converted names.\nFinal Answer: ['xiaoming', 'xiaohuang', 'xiaohong', 'xiaolan', 'xiaoju', 'xiaolv']"
}
[chain/end] [1:chain:AgentExecutor] [8.47s] Exiting Chain run with output:
{
  "output": "['xiaoming', 'xiaohuang', 'xiaohong', 'xiaolan', 'xiaoju', 'xiaolv']"
}

定义自己的工具并在代理中使用

在本节，我们将创建和使用自定义时间工具。LangChian tool 函数装饰器可以应用用于任何函数，将函数转化为LangChain 工具，使其成为代理可调用的工具。我们需要给函数加上非常详细的文档字符串, 使得代理知道在什么情况下、如何使用该函数/工具。比如下面的函数time,我们加上了详细的文档字符串。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
# 导入tool函数装饰器
from langchain.agents import tool
from datetime import date

@tool
def time(text: str) -> str:
    """
    返回今天的日期，用于任何需要知道今天日期的问题。\
    输入应该总是一个空字符串，\
    这个函数将总是返回今天的日期，任何日期计算应该在这个函数之外进行。
    """
    return str(date.today())

# 初始化代理
agent= initialize_agent(
    tools=[time], #将刚刚创建的时间工具加入代理
    llm=llm, #初始化的模型
    agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION,  #代理类型
    handle_parsing_errors=True, #处理解析错误
    verbose = True #输出中间步骤
)

# 使用代理询问今天的日期. 
# 注: 代理有时候可能会出错（该功能正在开发中）。如果出现错误，请尝试再次运行它。
agent("今天的日期是？") 
1
2
3
4
5
6
> Entering new AgentExecutor chain...
根据提供的工具，我们可以使用`time`函数来获取今天的日期。

Thought: 使用`time`函数来获取今天的日期。

Action:

{
“action”: “time”,
“action_input”: “”
}

1
2
3
4
5
6
7
8
9
10
11
12
13


Observation: 2023-08-09
Thought:我现在知道了最终答案。
Final Answer: 今天的日期是2023-08-09。

> Finished chain.





{'input': '今天的日期是？', 'output': '今天的日期是2023-08-09。'}

上面的过程可以总结为下

  1. 模型对于接下来需要做什么，给出思考（Thought）

    思考：我需要使用 time 工具来获取今天的日期

  2. 模型基于思考采取行动（Action), 因为使用的工具不同，Action的输出也和之前有所不同，这里输出的为python代码

    行动: 使用time工具，输入为空字符串

  3. 模型得到观察（Observation）

    观测: 2023-07-04

  4. 基于观察，模型对于接下来需要做什么，给出思考（Thought）

    思考: 我已成功使用 time 工具检索到了今天的日期

  5. 给出最终答案（Final Answer）

    最终答案: 今天的日期是2023-08-09.

  6. 返回最终答案。

英文版

使用LangChain内置工具llm-math和wikipedia
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
from langchain.agents import load_tools, initialize_agent
from langchain.agents import AgentType
from langchain.python import PythonREPL
from langchain.chat_models import ChatOpenAI

llm = ChatOpenAI(temperature=0)
tools = load_tools(
    ["llm-math","wikipedia"], 
    llm=llm 
)


agent= initialize_agent(
    tools,  
    llm,
    agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION,
    handle_parsing_errors=True,
    verbose = True
)

agent("What is the 25% of 300?")
> Entering new AgentExecutor chain...
I can use the calculator tool to find the answer to this question.

Action:

1
2
3
4
{
  "action": "Calculator",
  "action_input": "25% of 300"
}

Observation: Answer: 75.0
Thought:The answer is 75.0.
Final Answer: 75.0

> Finished chain.





{'input': 'What is the 25% of 300?', 'output': '75.0'}

Tom M. Mitchell的书

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
from langchain.agents import load_tools, initialize_agent
from langchain.agents import AgentType
from langchain.python import PythonREPL
from langchain.chat_models import ChatOpenAI

llm = ChatOpenAI(temperature=0)
tools = load_tools(
    ["llm-math","wikipedia"], 
    llm=llm 
)


agent= initialize_agent(
    tools,  
    llm,
    agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION,
    handle_parsing_errors=True,
    verbose = True
)

question = "Tom M. Mitchell is an American computer scientist \
and the Founders University Professor at Carnegie Mellon University (CMU)\
what book did he write?"
agent(question) 
> Entering new AgentExecutor chain...
Thought: I can use Wikipedia to find out what book Tom M. Mitchell wrote.
Action:

1
2
3
4
{
  "action": "Wikipedia",
  "action_input": "Tom M. Mitchell"
}

Observation: Page: Tom M. Mitchell
Summary: Tom Michael Mitchell (born August 9, 1951) is an American computer scientist and the Founders University Professor at Carnegie Mellon University (CMU). He is a founder and former Chair of the Machine Learning Department at CMU. Mitchell is known for his contributions to the advancement of machine learning, artificial intelligence, and cognitive neuroscience and is the author of the textbook Machine Learning. He is a member of the United States National Academy of Engineering since 2010. He is also a Fellow of the American Academy of Arts and Sciences, the American Association for the Advancement of Science and a Fellow and past President of the Association for the Advancement of Artificial Intelligence. In October 2018, Mitchell was appointed as the Interim Dean of the School of Computer Science at Carnegie Mellon.

Page: Tom Mitchell (Australian footballer)
Summary: Thomas Mitchell (born 31 May 1993) is a professional Australian rules footballer playing for the Collingwood Football Club in the Australian Football League (AFL). He previously played for the Adelaide Crows, Sydney Swans from 2012 to 2016, and the Hawthorn Football Club between 2017 and 2022. Mitchell won the Brownlow Medal as the league's best and fairest player in 2018 and set the record for the most disposals in a VFL/AFL match, accruing 54 in a game against Collingwood during that season.
Thought:The book that Tom M. Mitchell wrote is "Machine Learning".
Final Answer: Machine Learning

> Finished chain.





{'input': 'Tom M. Mitchell is an American computer scientist and the Founders University Professor at Carnegie Mellon University (CMU)what book did he write?',
 'output': 'Machine Learning'}
使用LangChain内置工具PythonREPLTool
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
from langchain.agents.agent_toolkits import create_python_agent
from langchain.tools.python.tool import PythonREPLTool

agent = create_python_agent(
    llm,  #使用前面一节已经加载的大语言模型
    tool=PythonREPLTool(), #使用Python交互式环境工具（REPLTool）
    verbose=True #输出中间步骤
)

customer_list = [["Harrison", "Chase"], 
                 ["Lang", "Chain"],
                 ["Dolly", "Too"],
                 ["Elle", "Elem"], 
                 ["Geoff","Fusion"], 
                 ["Trance","Former"],
                 ["Jen","Ayai"]
                ]
agent.run(f"""Sort these customers by \
last name and then first name \
and print the output: {customer_list}""") 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
> Entering new AgentExecutor chain...
I can use the `sorted()` function to sort the list of customers. I will need to provide a key function that specifies the sorting order based on last name and then first name.
Action: Python_REPL
Action Input: sorted([['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']], key=lambda x: (x[1], x[0]))
Observation: 
Thought:The customers have been sorted by last name and then first name.
Final Answer: [['Jen', 'Ayai'], ['Harrison', 'Chase'], ['Lang', 'Chain'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Dolly', 'Too']]

> Finished chain.





"[['Jen', 'Ayai'], ['Harrison', 'Chase'], ['Lang', 'Chain'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Dolly', 'Too']]"
1
2
3
4
5
6
import langchain
langchain.debug=True
agent.run(f"""Sort these customers by \
last name and then first name \
and print the output: {customer_list}""") 
langchain.debug=False
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
[chain/start] [1:chain:AgentExecutor] Entering Chain run with input:
{
  "input": "Sort these customers by last name and then first name and print the output: [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]"
}
[chain/start] [1:chain:AgentExecutor > 2:chain:LLMChain] Entering Chain run with input:
{
  "input": "Sort these customers by last name and then first name and print the output: [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]",
  "agent_scratchpad": "",
  "stop": [
    "\nObservation:",
    "\n\tObservation:"
  ]
}
[llm/start] [1:chain:AgentExecutor > 2:chain:LLMChain > 3:llm:ChatOpenAI] Entering LLM run with input:
{
  "prompts": [
    "Human: You are an agent designed to write and execute python code to answer questions.\nYou have access to a python REPL, which you can use to execute python code.\nIf you get an error, debug your code and try again.\nOnly use the output of your code to answer the question. \nYou might know the answer without running any code, but you should still run the code to get the answer.\nIf it does not seem like you can write code to answer the question, just return \"I don't know\" as the answer.\n\n\nPython_REPL: A Python shell. Use this to execute python commands. Input should be a valid python command. If you want to see the output of a value, you should print it out with `print(...)`.\n\nUse the following format:\n\nQuestion: the input question you must answer\nThought: you should always think about what to do\nAction: the action to take, should be one of [Python_REPL]\nAction Input: the input to the action\nObservation: the result of the action\n... (this Thought/Action/Action Input/Observation can repeat N times)\nThought: I now know the final answer\nFinal Answer: the final answer to the original input question\n\nBegin!\n\nQuestion: Sort these customers by last name and then first name and print the output: [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]\nThought:"
  ]
}
[llm/end] [1:chain:AgentExecutor > 2:chain:LLMChain > 3:llm:ChatOpenAI] [4.59s] Exiting LLM run with output:
{
  "generations": [
    [
      {
        "text": "I can use the `sorted()` function to sort the list of customers. I will need to provide a key function that specifies the sorting order based on last name and then first name.\nAction: Python_REPL\nAction Input: sorted([['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']], key=lambda x: (x[1], x[0]))",
        "generation_info": {
          "finish_reason": "stop"
        },
        "message": {
          "lc": 1,
          "type": "constructor",
          "id": [
            "langchain",
            "schema",
            "messages",
            "AIMessage"
          ],
          "kwargs": {
            "content": "I can use the `sorted()` function to sort the list of customers. I will need to provide a key function that specifies the sorting order based on last name and then first name.\nAction: Python_REPL\nAction Input: sorted([['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']], key=lambda x: (x[1], x[0]))",
            "additional_kwargs": {}
          }
        }
      }
    ]
  ],
  "llm_output": {
    "token_usage": {
      "prompt_tokens": 328,
      "completion_tokens": 112,
      "total_tokens": 440
    },
    "model_name": "gpt-3.5-turbo"
  },
  "run": null
}
[chain/end] [1:chain:AgentExecutor > 2:chain:LLMChain] [4.59s] Exiting Chain run with output:
{
  "text": "I can use the `sorted()` function to sort the list of customers. I will need to provide a key function that specifies the sorting order based on last name and then first name.\nAction: Python_REPL\nAction Input: sorted([['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']], key=lambda x: (x[1], x[0]))"
}
[tool/start] [1:chain:AgentExecutor > 4:tool:Python_REPL] Entering Tool run with input:
"sorted([['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']], key=lambda x: (x[1], x[0]))"
[tool/end] [1:chain:AgentExecutor > 4:tool:Python_REPL] [1.35ms] Exiting Tool run with output:
""
[chain/start] [1:chain:AgentExecutor > 5:chain:LLMChain] Entering Chain run with input:
{
  "input": "Sort these customers by last name and then first name and print the output: [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]",
  "agent_scratchpad": "I can use the `sorted()` function to sort the list of customers. I will need to provide a key function that specifies the sorting order based on last name and then first name.\nAction: Python_REPL\nAction Input: sorted([['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']], key=lambda x: (x[1], x[0]))\nObservation: \nThought:",
  "stop": [
    "\nObservation:",
    "\n\tObservation:"
  ]
}
[llm/start] [1:chain:AgentExecutor > 5:chain:LLMChain > 6:llm:ChatOpenAI] Entering LLM run with input:
{
  "prompts": [
    "Human: You are an agent designed to write and execute python code to answer questions.\nYou have access to a python REPL, which you can use to execute python code.\nIf you get an error, debug your code and try again.\nOnly use the output of your code to answer the question. \nYou might know the answer without running any code, but you should still run the code to get the answer.\nIf it does not seem like you can write code to answer the question, just return \"I don't know\" as the answer.\n\n\nPython_REPL: A Python shell. Use this to execute python commands. Input should be a valid python command. If you want to see the output of a value, you should print it out with `print(...)`.\n\nUse the following format:\n\nQuestion: the input question you must answer\nThought: you should always think about what to do\nAction: the action to take, should be one of [Python_REPL]\nAction Input: the input to the action\nObservation: the result of the action\n... (this Thought/Action/Action Input/Observation can repeat N times)\nThought: I now know the final answer\nFinal Answer: the final answer to the original input question\n\nBegin!\n\nQuestion: Sort these customers by last name and then first name and print the output: [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]\nThought:I can use the `sorted()` function to sort the list of customers. I will need to provide a key function that specifies the sorting order based on last name and then first name.\nAction: Python_REPL\nAction Input: sorted([['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']], key=lambda x: (x[1], x[0]))\nObservation: \nThought:"
  ]
}
[llm/end] [1:chain:AgentExecutor > 5:chain:LLMChain > 6:llm:ChatOpenAI] [3.89s] Exiting LLM run with output:
{
  "generations": [
    [
      {
        "text": "The customers have been sorted by last name and then first name.\nFinal Answer: [['Jen', 'Ayai'], ['Harrison', 'Chase'], ['Lang', 'Chain'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Dolly', 'Too']]",
        "generation_info": {
          "finish_reason": "stop"
        },
        "message": {
          "lc": 1,
          "type": "constructor",
          "id": [
            "langchain",
            "schema",
            "messages",
            "AIMessage"
          ],
          "kwargs": {
            "content": "The customers have been sorted by last name and then first name.\nFinal Answer: [['Jen', 'Ayai'], ['Harrison', 'Chase'], ['Lang', 'Chain'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Dolly', 'Too']]",
            "additional_kwargs": {}
          }
        }
      }
    ]
  ],
  "llm_output": {
    "token_usage": {
      "prompt_tokens": 445,
      "completion_tokens": 67,
      "total_tokens": 512
    },
    "model_name": "gpt-3.5-turbo"
  },
  "run": null
}
[chain/end] [1:chain:AgentExecutor > 5:chain:LLMChain] [3.89s] Exiting Chain run with output:
{
  "text": "The customers have been sorted by last name and then first name.\nFinal Answer: [['Jen', 'Ayai'], ['Harrison', 'Chase'], ['Lang', 'Chain'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Dolly', 'Too']]"
}
[chain/end] [1:chain:AgentExecutor] [8.49s] Exiting Chain run with output:
{
  "output": "[['Jen', 'Ayai'], ['Harrison', 'Chase'], ['Lang', 'Chain'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Dolly', 'Too']]"
}
定义自己的工具并在代理中使用
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# 导入tool函数装饰器
from langchain.agents import tool
from datetime import date

@tool
def time(text: str) -> str:
    """Returns todays date, use this for any \
    questions related to knowing todays date. \
    The input should always be an empty string, \
    and this function will always return todays \
    date - any date mathmatics should occur \
    outside this function."""
    return str(date.today())

agent= initialize_agent(
    tools + [time], #将刚刚创建的时间工具加入到已有的工具中
    llm, #初始化的模型
    agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION,  #代理类型
    handle_parsing_errors=True, #处理解析错误
    verbose = True #输出中间步骤
)

agent("whats the date today?") 
1
2
3
4
> Entering new AgentExecutor chain...
Question: What's the date today?
Thought: I can use the `time` tool to get the current date.
Action:

{
“action”: “time”,
“action_input”: “”
}

1
2
3
4
5
6
7
8
9
10
11
Observation: 2023-08-09
Thought:I now know the final answer.
Final Answer: The date today is 2023-08-09.

> Finished chain.





{'input': 'whats the date today?', 'output': 'The date today is 2023-08-09.'}

总结 Conclusion

本单元教程涵盖了一系列使用 LangChain 构建语言模型应用的实践，包括处理用户评论、基于文档问答、寻求外部知识等。

  1. 强大的 LangChain

通过这一系列案例，我们可以深刻体会到 LangChain 极大简化并加速了语言模型应用开发。过去需要数周才能实现的功能，现在只需极少量的代码即可通过 LangChain 快速构建。LangChain已成为开发大模型应用的有力范式，希望大家拥抱这个强大工具，积极探索更多更广泛的应用场景。

  1. 不同组合，更多可能性

LangChain 还可以协助我们做什么呢：基于 CSV 文件回答问题、查询 SQL 数据库、与 API 交互，有很多例子通过 Chain 以及不同的提示（Prompts）和输出解析器（output parsers）组合得以实现。

  1. 出发，去探索新世界吧

感谢 LangChain 的贡献者们，你们不断丰富文档和案例，让这一框架更易学易用。如果你还未开始使用 LangChain，现在就打开 Python ，运行pip install LangChain吧，一探这一魔法般工具的无限魅力!