LangChain Core Components on On My Way

9 月 20 日学习报告：LangChain Agent 的 Tool、Memory 与 Streaming

Mon, 01 Jan 0001 00:00:00 +0000

9 月 20 日学习报告：LangChain Agent 的 Tool、Memory 与 Streaming

仅整理 2026-09-20 修改的 4 个文件。文件按最后修改时间排列；各文件内部保持原文标题顺序与层级，只压缩重复表述和示例，不省略知识点。

14:08:52 — Tools.md

Tools

Tools extend what agents can do。

Create tools

Basic tool definition

定义 Tool 最简单的方式是使用 @tool decorator。函数 docstring 会成为 Tool description；参数 type hints 是必须的，因为它们定义 Tool 的输入 Schema。docstring 应内容丰富且简明扼要。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


from langchain.tools import tool

@tool
def search_database(query: str, limit: int = 10) -> str:
    """Search the customer database for records matching the query.

    Args:
        query: Search terms to look for
        limit: Maximum number of results to return
    """
    return f"Found {limit} results for '{query}'"

Customize tool properties

Custom tool name

Tool name 默认等于函数名，可通过 @tool("web_search") 覆盖。

Custom tool description

默认 description 来自 docstring。需要更明确地指导 model 时，使用 description= 覆盖自动生成的 description。

Advanced schema definition

复杂输入使用 Pydantic model 或 JSON Schema。Field(description=...) 描述字段，Literal 限定可选值，@tool(args_schema=...) 绑定输入 Schema。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


from typing import Literal
from pydantic import BaseModel, Field

class WeatherInput(BaseModel):
    location: str = Field(description="City name or coordinates")
    units: Literal["celsius", "fahrenheit"] = Field(
        default="celsius",
        description="Temperature unit preference",
    )
    include_forecast: bool = Field(default=False)

@tool(args_schema=WeatherInput)
def get_weather(
    location: str,
    units: str = "celsius",
    include_forecast: bool = False,
) -> str:
    """Get current weather and optional forecast."""
    ...

Reserved argument names

config 保留给 RunnableConfig；
runtime 保留给 ToolRuntime。

二者不能用作普通业务参数，否则会产生 runtime error。

Access context

Tool 通过 ToolRuntime 访问 conversation history、user data、persistent memory 和执行信息。

Component	访问方式	含义与用途
State	`runtime.state`	当前 conversation 的可变 short-term memory；messages、计数器、自定义字段
Context	`runtime.context`	invocation 时传入的不可变配置；user ID、session info
Store	`runtime.store`	跨 conversation 持久化的 long-term memory
Stream Writer	`runtime.stream_writer`	Tool 执行时发送实时 custom updates
Execution Info	`runtime.execution_info`	thread ID、run ID、attempt number
Server Info	`runtime.server_info`	assistant ID、graph ID、authenticated user
Config	`runtime.config`	`RunnableConfig` 的 callbacks、tags、metadata
Tool Call ID	`runtime.tool_call_id`	当前 Tool invocation ID，用于匹配 `ToolMessage`

Short-term memory(State)

State 包含 message history 和 graph state 中定义的自定义字段。

Access state

在 Tool 签名中声明 runtime: ToolRuntime。ToolNode 会自动注入它，但它不会出现在发送给 model 的 Tool Schema 中。Tool 通过 runtime.state 读取 State。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


from langchain.messages import HumanMessage
from langchain.tools import ToolRuntime, tool

@tool
def get_last_user_message(runtime: ToolRuntime) -> str:
    """Get the most recent message from the user."""
    for message in reversed(runtime.state["messages"]):
        if isinstance(message, HumanMessage):
            return message.content
    return "No user messages found"

@tool
def get_user_preference(pref_name: str, runtime: ToolRuntime) -> str:
    """Get a user preference value."""
    return runtime.state.get("user_preferences", {}).get(pref_name, "Not set")

第二个 Tool 的 model-visible Schema 只有 pref_name，没有 runtime。

Update state

Tool 更新 State 时返回 Command。当目标是当前 graph 时：

在 Command.update 写入 State 字段；
在 Command.update["messages"] 中加入 ToolMessage；
结果写入 ToolMessage.content；
ToolMessage.tool_call_id 必须等于 runtime.tool_call_id。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25


from langchain.agents import AgentState
from langchain.messages import ToolMessage
from langchain.tools import ToolRuntime, tool
from langgraph.types import Command

class CustomState(AgentState):
    user_name: str

@tool
def set_user_name(
    new_name: str,
    runtime: ToolRuntime[None, CustomState],
) -> Command:
    """Set the user's name in the conversation state."""
    return Command(
        update={
            "user_name": new_name,
            "messages": [
                ToolMessage(
                    content=f"User name set to {new_name}.",
                    tool_call_id=runtime.tool_call_id,
                )
            ],
        }
    )

每个 AIMessage tool call 都必须有对应的 ToolMessage。消息缺失或 ID 不匹配时，ToolNode 抛出 ValueError。多个 Tool 并行更新同一 State 字段时，应定义 reducer，明确覆盖、追加或其他合并规则。

Context

Context 是 invocation 时传入的不可变配置，包括 user ID、session details 和 application settings。使用 context_schema 声明结构，通过 context= 传入，Tool 从 runtime.context 读取。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


from dataclasses import dataclass

@dataclass
class UserContext:
    user_id: str

@tool
def get_account_info(runtime: ToolRuntime[UserContext]) -> str:
    """Get the current user's account information."""
    return str(USER_DATABASE.get(runtime.context.user_id, "User not found"))

agent = create_agent(
    model="openai:gpt-5.5",
    tools=[get_account_info],
    context_schema=UserContext,
)

result = agent.invoke(
    {"messages": [{"role": "user", "content": "What's my balance?"}]},
    config={"configurable": {"thread_id": "thread-1"}},
    context=UserContext(user_id="user123"),
)

thread_id 标识 conversation；user_id 标识业务用户，二者不同。

Long-term memory(Store)

BaseStore 提供跨 conversation 持久化的数据。使用 runtime.store 访问，以 namespace/key 组织记录。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


@tool
def get_user_info(user_id: str, runtime: ToolRuntime) -> str:
    """Look up user info."""
    item = runtime.store.get(("users",), user_id)
    return str(item.value) if item else "Unknown user"

@tool
def save_user_info(
    user_id: str,
    name: str,
    age: int,
    email: str,
    runtime: ToolRuntime,
) -> str:
    """Save user info."""
    runtime.store.put(
        ("users",), user_id, {"name": name, "age": age, "email": email}
    )
    return "Successfully saved user info."

把同一 Store 传给 create_agent(store=store) 后，可在一个 session 保存、另一个 session 读取。

Streaming writer

runtime.stream_writer 可在 Tool 执行中发送实时 custom updates，适合长任务进度；Tool 最终仍用 return 返回完成结果。

1
2
3
4
5
6


@tool
def get_weather(city: str, runtime: ToolRuntime) -> str:
    """Get weather for a given city."""
    runtime.stream_writer(f"Looking up data for city: {city}")
    runtime.stream_writer(f"Acquired data for city: {city}")
    return f"It's always sunny in {city}!"

Execution info

runtime.execution_info 提供 thread_id、run_id 和 node_attempt，用于日志关联和重试判断。

Server info

在 LangGraph Server 上，runtime.server_info 提供 assistant_id、graph_id 和 authenticated user；非 Server 环境下为 None。

Tool execution

Tool 由 Agent 使用；Tool error handling 由 middleware 配置。

Tool return values

Return a string

返回 human-readable str，供 model 下一步读取；框架自动将其转换为 ToolMessage。

Return an object

返回结构化 object，供 model 读取字段并推理。对象会序列化为 Tool output，但不会直接更新 graph State。

Return multimodal content

多模态结果使用 standard content blocks，例如 text block 和 image block。结果转换成带 multimodal content 的 ToolMessage；可用 message.content_blocks 读取归一化 blocks。model 必须支持所返回的 image、audio 或 video modality。

Return a Command

需要更新 graph State 时返回 Command。若目标是当前 graph，必须显式提供与当前 tool call ID 匹配的 ToolMessage。普通 str/object 会自动包装为 ToolMessage，Command 不会。

Return directly from a tool

@tool(return_direct=True) 使 agent loop 短路：Tool 正常执行，输出包装成 ToolMessage 后直接返回 caller，不再追加 model call。

适用于：Tool output 已是完整答案；希望避免额外 model call；需要 deterministic、unmodified output。

同一步包含多个并行 tool calls 时，会先执行整个批次；仅当每个 Tool 都设置 return_direct=True 时才路由到 END，最终响应包含全部 Tool outputs。

Return a Command with return_direct

return_direct=True 的 Tool 可返回 Command，在退出前更新 State。Command 仍不会自动生成 ToolMessage，必须手动加入 tool_call_id=runtime.tool_call_id 的消息。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


@tool(return_direct=True)
def fetch_and_store_order(order_id: str, runtime: ToolRuntime) -> Command:
    """Fetch order status and store it in state."""
    status = f"Order {order_id} is shipped."
    return Command(
        update={
            "last_order_status": status,
            "messages": [
                ToolMessage(content=status, tool_call_id=runtime.tool_call_id)
            ],
        }
    )

Error handing

使用 middleware 重试失败 Tool，或把异常转换为 model 可处理的自定义 ToolMessage。异常结果的 tool_call_id 必须取原调用的 request.tool_call["id"]。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


from langchain.agents.middleware import wrap_tool_call
from langchain.messages import ToolMessage

@wrap_tool_call
def handle_tool_errors(request, handler) -> ToolMessage:
    try:
        return handler(request)
    except Exception as error:
        return ToolMessage(
            content=f"Tool error: Please check your input. ({error})",
            tool_call_id=request.tool_call["id"],
        )

State injection

Tool 通过框架注入且对 model 隐藏的 ToolRuntime 访问 State：读取用 runtime.state，更新用带匹配 ToolMessage 的 Command。

Dynamic tool section

Tool 太多会占用 context，太少会限制能力。动态 Tool 根据启动时是否已知分为两类。

Filtering pre-registered tools

全部 Tool 在创建 Agent 时已知：先预注册，再依据 State、permissions 或 Context 动态过滤暴露给 model 的集合。适合 Tool 静态、availability 动态的场景。

Runtime tool register

Tool 在 runtime 才发现或创建时，需要同时处理：

wrap_model_call：用 request.override(tools=[...]) 加入本次 model request；
wrap_tool_call：识别对应 name，用 request.override(tool=...) 提供执行对象。

只让 model 看见而不提供执行处理器，runtime 无法完成调用。此方式适合 MCP server、database、user configuration 和 external registry。

Headless tools

服务端只注册 name、description 和 parameter Schema，实现只在客户端。model 可正常调用，但执行通过 interrupt/resume 在进程外完成。

when to use

用于依赖客户端 environment、device 或 UI 的工作：Browser APIs、privacy/locality、latency、structured safe effects。

How it work

定义 Tool Schema，不提供 in-process implementation；
将定义注册给 create_agent；
调用时 graph 暂停，返回 {"type":"tool","tool_call":{"id","name","args"}}；
客户端执行，再 resume graph。

onTool callback 可观察 start、success、error，用于 spinner 或 toast。

Prebuilt tools

LangChain 提供 web search、code interpretation、database access 等 prebuilt tools/toolkits，可直接集成，无需自行实现。

Tools from MCP

MCP 统一应用向 model 暴露 Tools 的方式。MCPAdapter 发现 MCP server 的 Tools，将 list_tools() 结果转换为 LangChain Tools，再传给 create_agent。

Server-side tool use

部分 chat models 提供由 provider 执行的 built-in Tools，例如 web search 和 code interpreter，无需自行定义或托管 Tool logic；具体配置取决于 model integration。

15:38:37 — Short-term_memory.md

Conversation history 是最常见的 short-term memory；Message 包含 instructions 和 inputs。

Usage

为 Agent 添加 short-term memory，需要在 create_agent 时指定 checkpointer。Short-term memory 是 Agent State 的一部分；checkpointer 按 thread 持久化 State，使 conversation 可恢复。

State 在 Agent invocation 或 tool call 等 step 完成时更新，在每个 step 开始时读取。相同 thread_id 恢复相同 conversation；不同 thread_id 使用独立 State。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18


from langchain.agents import create_agent
from langgraph.checkpoint.memory import InMemorySaver

agent = create_agent(
    model="openai:gpt-5.5",
    tools=[get_user_info],
    checkpointer=InMemorySaver(),
)

config = {"configurable": {"thread_id": "1"}}
agent.invoke(
    {"messages": [{"role": "user", "content": "Hi! My name is Bob."}]},
    config,
)
result = agent.invoke(
    {"messages": [{"role": "user", "content": "What's my name?"}]},
    config,
)

In production

生产环境使用 database-backed checkpointer；InMemorySaver 只适合本地示例和测试。

Customizing agent memory

Agent 使用 AgentState 管理 short-term memory，其中 messages key 保存 conversation history。需要额外字段时继承 AgentState，通过 state_schema= 传给 create_agent。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21


from langchain.agents import AgentState

class CustomAgentState(AgentState):
    user_id: str
    preferences: dict

agent = create_agent(
    model="openai:gpt-5.5",
    tools=[get_user_info],
    state_schema=CustomAgentState,
    checkpointer=InMemorySaver(),
)

result = agent.invoke(
    {
        "messages": [{"role": "user", "content": "Hello"}],
        "user_id": "user_123",
        "preferences": {"theme": "dark"},
    },
    {"configurable": {"thread_id": "1"}},
)

state_schema 定义 State 结构；checkpointer 负责按 thread 保存它。

Common patterns

以下模式用于处理 conversation history 超过 LLM context window。

Trim message(截断消息)

接近 token limit 时，根据 token count 截断，并指定 boundary strategy。也可在 @before_model 中清空旧 messages，再写回首条与最近 messages。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20


from langchain.agents.middleware import before_model
from langchain.messages import RemoveMessage
from langgraph.graph.message import REMOVE_ALL_MESSAGES

@before_model
def trim_messages(state, runtime) -> dict | None:
    """Keep only the first and most recent messages."""
    messages = state["messages"]
    if len(messages) <= 3:
        return None

    first_message = messages[0]
    recent_messages = messages[-3:] if len(messages) % 2 == 0 else messages[-4:]
    return {
        "messages": [
            RemoveMessage(id=REMOVE_ALL_MESSAGES),
            first_message,
            *recent_messages,
        ]
    }

None 表示不更新 State。选择最近 3 或 4 条是为了尽量维持有效的 request/response 边界。

Delete message

使用 RemoveMessage 删除指定 message 或清空全部 history。Messages State key 必须使用 add_messages reducer；默认 AgentState 已提供。

To remove specific messages:

1
2
3
4
5
6
7
8


from langchain.messages import RemoveMessage

def delete_messages(state):
    messages = state["messages"]
    if len(messages) > 2:
        return {
            "messages": [RemoveMessage(id=message.id) for message in messages[:2]]
        }

To remove all messages:

1
2
3
4
5


from langchain.messages import RemoveMessage
from langgraph.graph.message import REMOVE_ALL_MESSAGES

def delete_messages(state):
    return {"messages": [RemoveMessage(id=REMOVE_ALL_MESSAGES)]}

删除后必须保持有效消息历史：部分 provider 要求以 user message 开始；多数 provider 要求带 tool call 的 assistant message 后紧跟对应的 tool result，不能只删除配对的一侧。

Summarize messages

Trim 和 Delete 会丢失信息。需要保留旧语义时，使用内置 SummarizationMiddleware。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


from langchain.agents.middleware import SummarizationMiddleware

agent = create_agent(
    model="openai:gpt-5.5",
    tools=[...],
    middleware=[
        SummarizationMiddleware(
            model="openai:gpt-5.4-mini",
            trigger=("tokens", 4000),
            keep=("messages", 20),
        )
    ],
    checkpointer=checkpointer,
)

示例在 4000 tokens 时触发摘要，保留最近 20 条 messages。

Access memory

tools

Read short-term memory in a tool

Tool 使用对 model 隐藏的 runtime: ToolRuntime，通过 runtime.state 读取 short-term memory。

1
2
3
4
5


@tool
def get_user_info(runtime: ToolRuntime) -> str:
    """Look up user info."""
    user_id = runtime.state["user_id"]
    return "User is John Smith" if user_id == "user_123" else "Unknown user"

write short-term memory form tools

Tool 写 short-term memory 时返回 Command，并在 Command.update["messages"] 中加入 tool_call_id=runtime.tool_call_id 的 ToolMessage。

Prompt

使用 middleware 根据 conversation history 或 custom State fields 创建 dynamic prompt。@dynamic_prompt 可通过 request.runtime.context 读取 invocation Context。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


from typing import TypedDict
from langchain.agents.middleware import ModelRequest, dynamic_prompt

class CustomContext(TypedDict):
    user_name: str

@dynamic_prompt
def dynamic_system_prompt(request: ModelRequest) -> str:
    user_name = request.runtime.context["user_name"]
    return f"You are a helpful assistant. Address the user as {user_name}."

创建 Agent 时传入 context_schema=CustomContext，调用时通过 context=CustomContext(...) 提供值。

Before model

@before_model 在每次 model call 前处理 State，适合 trim、删除、注入或重排 messages。返回 None 表示无修改，返回 dict 表示要合并进 State 的更新。

After model

@after_model 在 model call 后检查 State。可读取最后一条 response，用 RemoveMessage(id=last_message.id) 删除不符合要求的输出。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


from langchain.agents.middleware import after_model
from langchain.messages import RemoveMessage

@after_model
def validate_response(state, runtime) -> dict | None:
    """Remove messages containing sensitive words."""
    stop_words = ["password", "secret"]
    last_message = state["messages"][-1]
    if any(word in last_message.content for word in stop_words):
        return {"messages": [RemoveMessage(id=last_message.id)]}
    return None

该 hook 修改最终 State；对于实时 Stream，只在 after_model 删除可能太晚，流式输出需在离开 run 前过滤。

16:27:01 — Event_Streaming.md

LangChain Agents 构建在 LangGraph 上，因此支持相同的 streaming stack，并提供 messages、tool calls、State 和 custom updates 等 Agent-focused projections。

多数场景使用 stream_events(..., version="v3")。它返回带 typed projections 的 run object，各 projection 可独立消费，无需解析 stream-mode tuples。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


stream = agent.stream_events(
    {"messages": [{"role": "user", "content": "What is the weather in SF?"}]},
    version="v3",
)

for message in stream.messages:
    for delta in message.text:
        print(delta, end="", flush=True)

final_state = stream.output

What you can stream

Projection	内容
`for event in stream`	带完整 envelope、可访问全部 channel 的 raw protocol events
`stream.messages`	每次 LLM call 的 model message streams
`message.text`	Text deltas 与 final text
`message.reasoning`	Model 支持时的 reasoning deltas
`message.tool_calls`	Tool-call argument chunks 与 finalized tool calls
`message.output`	本次 model call 的 final message object
`stream.values`	Agent State snapshots
`stream.output`	Final Agent State
`stream.subgraphs`	Nested graph runs，包括 sub-agents 和普通 subgraphs
`stream.extensions`	Custom transformer projections
`stream.tool_calls`	Tool lifecycle、inputs、output deltas、final output、errors

stream.messages 产生 ChatModelStream。每个 message stream 暴露 .text、.reasoning、.tool_calls、.output。同步 projection 可实时迭代，也可排空后读取最终值：str(message.text) 得到最终文本，message.tool_calls.get() 得到完整 Tool calls。

Agent messages

需要每次 LLM call 的 model output 时使用 stream.messages。message.output 是 finalized AI message，包含 provider-specific content blocks；token usage 位于 message.output.usage_metadata。

1
2
3
4
5
6


for message in stream.messages:
    print(f"[{message.node}] ", end="")
    for delta in message.text:
        print(delta, end="", flush=True)
    if message.output.usage_metadata:
        print(message.output.usage_metadata)

Reasoning content

message.reasoning 与 .text 用法相同，但只有 model 发出 reasoning blocks 时才有内容。

1
2
3
4
5


for message in stream.messages:
    for delta in message.reasoning:
        print(f"[thinking] {delta}", end="", flush=True)
    for delta in message.text:
        print(delta, end="", flush=True)

Tool calls

Tool calls 有两种 projection：

message.tool_calls：model 生成 Tool call 时，流式输出参数 chunks 和 finalized call；
stream.tool_calls：Tool 开始执行后，流式输出 lifecycle、input、output deltas、final output、error。

前者观察 model 正在构造什么调用，后者观察 Tool 实际如何执行。

Streaming sub-agent

当一个 create_agent 调用另一个已命名的 create_agent 时，内部 Agent 事件位于嵌套 namespace。传给 create_agent 的 name= 用于标识内部 Agent。

命名 sub-agent 出现在 stream.subagents projection；每个 handle 暴露 .messages、.values、.tool_calls、.output，以及 .name、.cause。

1
2
3
4
5


for subagent in stream.subagents:
    print(f"{subagent.name}: ", end="")
    for message in subagent.messages:
        for token in message.text:
            print(token, end="", flush=True)

state and final output

stream.values 获取运行过程中的 State snapshots；stream.output 获取最终 Agent State。

1
2
3
4


for snapshot in stream.values:
    print(snapshot)

final_state = stream.output

message.output 是单次 model call 的最终 AI message；stream.output 是整个 Agent run 的最终 State。

Multiple projections

Async code 使用 astream_events 与 asyncio.gather 并发消费多个 projections。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


import asyncio

stream = await agent.astream_events(input, version="v3")

async def consume_messages():
    async for message in stream.messages:
        print(await message.text)

async def consume_tool_calls():
    async for call in stream.tool_calls:
        print(call.tool_name, call.input)

await asyncio.gather(consume_messages(), consume_tool_calls())

Synchronous code 使用 stream.interleave(...)，返回 (name, item) 区分 projection。

1
2
3
4
5
6
7


for name, item in stream.interleave("messages", "tool_calls", "values"):
    if name == "messages":
        print(item.text)
    elif name == "tool_calls":
        print(item.tool_name, item.input)
    elif name == "values":
        print(item)

Custom updates

内置 projection 不够时，使用 custom stream transformer，并从 stream.extensions 读取扩展 projection。

1
2
3
4
5
6
7
8


stream = agent.stream_events(
    input,
    version="v3",
    transformers=[ToolActivityTransformer],
)

for activity in stream.extensions["tool_activity"]:
    print(activity)

Register transformers on middleware

Middleware 可在 hooks 和 Tools 旁声明 transformer factories。将 AgentMiddleware.transformers 设置为 factories 序列；每个 factory 形如 Callable[[tuple[str, ...]], StreamTransformer]，框架调用 factory(scope)。

Root mux 的 scope 为 ()，subgraph 为非空 tuple。每次返回新的 transformer，使各 subgraph 隔离。

1
2
3
4


from langchain.agents.middleware import AgentMiddleware

class ToolActivityMiddleware(AgentMiddleware):
    transformers = (ToolActivityTransformer,)

create_agent 编译时合并 middleware factories 与自身 transformers=，顺序为：

内置 ToolCallTransformer；
Middleware factories，按 middleware 顺序；
调用方传给 create_agent(transformers=...) 的 factories。

这让内置 Tool call projection 位于消费者 transformer 之前，并让调用方条目拥有最终处理权。

PIIMiddleware(..., apply_to_output=True) 会在 text deltas、tool call arguments、tool outputs、State snapshots 离开 run 前清除 PII。

1
2
3
4
5
6
7


agent = create_agent(
    model="openai:gpt-5-nano",
    tools=[],
    middleware=[
        PIIMiddleware("email", strategy="redact", apply_to_output=True),
    ],
)

若只依赖 after_model 做 State-level filtering，原始 PII 可能已通过 stream_events(version="v3") 实时发送给 reader。

16:52:27 — Streaming.md

Overview

Streaming 支持：

Stream Agent progress；
Stream LLM tokens；
Streaming thinking/reasoning tokens；
Stream custom updates；
从 updates、messages、custom 中选择一个或多个 modes。

Supported stream modes

将一个 mode 或多个 modes 的 list 传给 stream / astream。

Mode	Description
`updates`	每个 Agent step 后发送 State updates；同一步运行多个 nodes 时分别发送
`messages`	从所有调用 LLM 的 graph nodes 发送 `(token, metadata)` tuples
`custom`	发送 graph nodes 内通过 stream writer 写出的 custom data

Agent progress

使用 stream_mode="updates" 的 stream 或 astream，在每个 Agent step 后接收事件。一次 Tool call 的典型顺序是：

LLM node：输出带 Tool call requests 的 AIMessage；
Tool node：输出带 execution result 的 ToolMessage；
LLM node：输出 final AI response。

Event Streaming

Mon, 01 Jan 0001 00:00:00 +0000

LangChain agents are built on LangGraph, so they support the same streaming stack with agent-focused projections for messages, tool calls, state, and custom updates.

For most situation,use Event Streaming throughstream_event(...,version="v3") Event Streaming returns a run object with typed projections, so each projection can be consumed independently instead of parsing stream-mode tuples

1
2
3
4
5
6
7
8
9


stream = agent.stream_events({
    "messages": [{"role": "user", "content": "What is the weather in SF?"}],
}, version="v3")

for message in stream.messages:
    for delta in message.text:
        print(delta, end="", flush=True)

final_state = stream.output

What you can stream

Projection	Use
`for event in stream`	Raw protocol events with full envelope and access to every channel.
`stream.messages`	Model message streams, one per LLM call.
`message.text`	Text deltas and final text for a message.
`message.reasoning`	Reasoning deltas for models that expose reasoning content.
`message.tool_calls`	Tool-call argument chunks and finalized tool calls.
`message.output`	Final message object after the model call completes.
`stream.values`	Agent state snapshots.
`stream.output`	Final agent state.
`stream.subgraphs`	Nested graph runs (sub-agents and plain subgraphs).
`stream.extensions`	Custom transformer projections.
`stream.tool_calls`	Tool execution lifecycle, inputs, output deltas, final output, and errors.

stream.messages 会生成 ChatModelStream 对象。每个消息流都暴露了 .text、.reasoning、.tool_calls 和 .output。同步投影可迭代以获取实时增量，也可排空以获取最终值：使用 str(message.text) 获取最终文本，使用 message.tool_calls.get() 获取已完成的工具调用

Agent messages

Use stream.message when you want model output from each LLM call

message.output gives you the finalized AI message,including previder-specific content blocks read usage from message.output.usage_metadata.(If you want token counts)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


stream = agent.stream_events(input, version="v3")

for message in stream.messages:
    print(f"[{message.node}] ", end="")
    for delta in message.text:
        print(delta, end="", flush=True)

    full_message = message.output
    usage = full_message.usage_metadata
    if usage:
        print(usage)

Reasoning content

the same shape as text ,but only available when the model emits reasoning blocks

1
2
3
4
5
6
7
8


stream = agent.stream_events(input, version="v3")

for message in stream.messages:
    for delta in message.reasoning:
        print(f"[thinking] {delta}", end="", flush=True)

    for delta in message.text:
        print(delta, end="", flush=True)

将上面的message.text更换为message.reasoning

Tool calls

有两种投影

message.tool_calls在模型生成工具调用的时候,会流式传输工具调用参数块
stream.tool_calls会在工具调用开始后流式传输工具执行的生命周期

1
2
3
4
5
6
7
8


stream = agent.stream_events(input, version="v3")

for message in stream.messages:
    for delta in message.reasoning:
        print(f"[thinking] {delta}", end="", flush=True)

    for delta in message.text:
        print(delta, end="", flush=True)

Streaming sub-agent

当一个create_agent调用另一个已命名的create_agent时,内部智能体的时间会在一个内嵌的namespace中流动,你传递给create_agent的name= 会在stream中标识该内部智能体

命名的子智能体会在主智能体的stream.subagents投影上呈现,而每个句柄都暴露出该内部智能体的.message,.value,.tool_calls和.output加上你传递的name和.cause

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


upervisor = create_agent(
    model=init_chat_model("openai:gpt-5.5"),
    tools=[call_weather],
    name="supervisor",
)

stream = supervisor.stream_events(
    {"messages": [{"role": "user", "content": "What's the weather in Boston?"}]},
    version="v3",
)

for subagent in stream.subagents:
    print(f"{subagent.name}: ", end="")
    for message in subagent.messages:
        for token in message.text:
            print(token, end="", flush=True)
    print()

state and final output

用stream.value来获取state的快照用stream.output来获取最终的agent state

1
2
3
4
5
6


stream = agent.stream_events(input, version="v3")

for snapshot in stream.values:
    print(snapshot)

final_state = stream.output

Multiple projections

对于 async code中的并发消费,请使用 astream_events with asyncio.gather 前者创建异步事件流,后者并发执行多个事件并且根据返回顺序聚合他们的结果

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


import asyncio

stream = await agent.astream_events(input, version="v3")

async def consume_messages():
    async for message in stream.messages:
        print(await message.text)

async def consume_tool_calls():
    async for call in stream.tool_calls:
        print(call.tool_name, call.input)

await asyncio.gather(consume_messages(), consume_tool_calls())

对于synchronous code,使用stream.interleave(…)

1
2
3
4
5
6
7
8
9


stream = agent.stream_events(input, version="v3")

for name, item in stream.interleave("messages", "tool_calls", "values"):
    if name == "messages":
        print(item.text)
    elif name == "tool_calls":
        print(item.tool_name, item.input)
    elif name == "values":
        print(item)

Custom updates

使用custom stream transformer当你需要内置未提供的投影

1
2
3
4
5
6
7
8


stream = agent.stream_events(
    input,
    version="v3",
    transformers=[ToolActivityTransformer],
)

for activity in stream.extensions["tool_activity"]:
    print(activity)

Register transformers on middleware

中间件可以在其钩子和工具旁边声明流转换器工厂。工厂的形态在不同语言中有所不同：将 AgentMiddleware 子类上的 transformers 属性设置为一系列工厂。每个工厂的形态为 Callable[[tuple[str, ...]], StreamTransformer]，调用方式为 factory(scope)，其中 scope 是 mini-mux 作用域元组（根 mux 为 ()，子图则为非空）。每次调用返回一个新的转换器可使每个子图保持隔离。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


from langchain.agents import create_agent
from langchain.agents.middleware import AgentMiddleware


class ToolActivityMiddleware(AgentMiddleware):
    transformers = (ToolActivityTransformer,)


agent = create_agent(
    model="gpt-5-nano",
    tools=[get_weather],
    middleware=[ToolActivityMiddleware()],
)

在编译时，create_agent 会将中间件注册的工厂与传递给其自身的 transformers= 参数的内容进行合并。编译后图上的最终顺序为：

内置的 ToolCallTransformer。
中间件注册的工厂，按中间件顺序排列。
由调用方通过 create_agent 提供的 transformers=。

这使得内置的工具调用投影排在消费者转换器之前，并让调用方提供的条目拥有最终决定权。内置的 PIIMiddleware 使用此钩子从流式传输的线路输出中过滤 PII。通过设置 apply_to_output=True，其注册的转换器会在文本增量、工具调用参数、工具输出和状态快照离开运行之前，清除其中检测到的 PII，从而关闭了一个窗口——在该窗口中，after_model 状态级别的过滤原本会让原始 PII 透传给 stream_events(version="v3") 的实时读取者。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


from langchain.agents import create_agent
from langchain.agents.middleware import PIIMiddleware

agent = create_agent(
    model="gpt-5-nano",
    tools=[],
    middleware=[
        PIIMiddleware("email", strategy="redact", apply_to_output=True),
    ],
)

LangChain A Quick Start

Mon, 01 Jan 0001 00:00:00 +0000

一个简单的示例

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


from langchain.agents import create_agent

def get_weather(city: str) -> str:
    """Get weather for a given city."""
    return f"It's always sunny in {city}!"

agent = create_agent(
    model="claude-sonnet-4-6",
    tools=[get_weather],
    system_prompt="You are a helpful assistant",
)

result = agent.invoke(
    {"messages": [{"role": "user", "content": "What's the weather in San Francisco?"}]}
)
print(result["messages"][-1].content_blocks)

Build a real-world agent:

Detailed System Prompt
Create Tools
Model Configuration
Conversational memory
Deep agents
Testing

1.Define the system prompt

defines your agent’s role and behavior. Keep it specific and actionable.

2.Craete tools

Tools let a model interact with external systems by calling functions you define. Tools can depend on runtime context and also interact with agent memory.

Tools should be well-documented: their name,description,parameters will become part of the model’s prompt. LangChain’s tool decorator adds metadata and enables runtime injection with the ToolRunTime paramete.

3.Configure your model

Set up your language model with the right parameters foe your use case

4.Add memory

Add memory to your agent to maintain state across interactions. This allows the agent to remember previous conversations and context.

5.Create and run the agent

Tow different Frameworks for creating agents: LangChain agents and deep agents Use deep agents when you want maximum with minimal setup; Choose LangChain agents when you need fine-grained contorl.

6.Review the results

相较于LangChain, deep agent拥有更多定义好的功能:

使用内置的 write_todos 工具规划其方法，以分解研究任务。
通过调用 fetch_text_from_url 工具加载文件以收集信息。
通过使用文件系统工具（grep 和 read_file）来管理上下文。
生成子智能体：根据需要将复杂的子任务委派给专门的子智能体。

For LangChain agents, you must implement more capabilities to get a similar lever of service and can customize them along the way as needed

In short,deep agent is well-organized and provides many integrated tools and LangChain can fit custom needs better, but requires implementing more functionalities.

Trace agent calls

Use LangSmith to trace LLM calls and inspect what is going on inside your agent.

Emmm, you should sign up for a LangSmith account first. Besides,you can also set up LangSmith Engine which monitors your traces,detects issues ,and preposes fixes.

LangChain Philosophy

Mon, 01 Jan 0001 00:00:00 +0000

LangChain is driven by a few core beliefs:

Large Language Models (LLMs) are great, powerful new technology.
LLMs are even better when you combine them with external sources of data.
LLMs will transform what the applications of the future look like. Specifically, the applications of the future will look more and more agentic.
It is still very early on in that transformation.
While it’s easy to build a prototype of those agentic applications, it’s still really hard to build agents that are reliable enough to put into production.

Message

Mon, 01 Jan 0001 00:00:00 +0000

Message

Carrying both the content and metadata which represent the state

Messages contain:

Role: identifies the message type(system,user)
Content: the actual content
Metadata: optional fields: such as information,message IDs,and token usage

This is the standard message type

Basic usage

The easiest way to use message is to create message objects and pass them to a model when invoking

Text prompts

String Ideal for generation tasks where you don’t need to retain.

1

response = model.invoke("Write a haiku about spring")

When to use:

a single ,standalone request
no need for conversation history
minimal code complexity

Message prompts

Providing a list of messages

1
2
3
4
5
6
7
8


from langchain.messages import SystemMessage, HumanMessage, AIMessage

messages = [
    SystemMessage("You are a poetry expert"),
    HumanMessage("Write a haiku about spring"),
    AIMessage("Cherry blossoms bloom...")
]
response = model.invoke(messages)

When:

Managing multi-turn conversations
Working with multimodal content (images, audio, files)
Including system instructions

Dictionary format

Specify message in OpenAI chat completions format:

1
2
3
4
5
6


messages = [
    {"role": "system", "content": "You are a poetry expert"},
    {"role": "user", "content": "Write a haiku about spring"},
    {"role": "assistant", "content": "Cherry blossoms bloom..."}
]
response = model.invoke(messages)

Message types

System messages
Human messages
Al messages:
Tool messages:

System message

An initial set of instructions that primes the model’s behavior

Human message

user input and interactions

AI message

The LLM sometimes treat the message differently, so sometimes we can make up a AI message manually.

Tool calls

are included in AI messages

Token usage

in AI message in usage_metadata field

Streaming and chunks

When streaming, u can receive AIMessageChunk objects and it can be combined into a full message by summing

Tool message

tool result in ToolMessage object

Message content

An loosely-typed attribute,supporting strings and lists if untyped objects

Beside, LangChain provides dedicated content types for text, reasoning, citations, multi-modal data, server-side tool calls, and other message content

This may contain either:

A string
A list of content blocks in a provider-native format
A list of LangChain’s standard content blocks

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18


from langchain.messages import HumanMessage

# String content
human_message = HumanMessage("Hello, how are you?")

# Provider-native format (e.g., OpenAI)
human_message = HumanMessage(content=[
    {"type": "text", 
    "text": "Hello, how are you?"},
    {"type": "image_url", 
    "image_url": {"url": "https://example.com/image.jpg"}}
])

# List of standard content blocks
human_message = HumanMessage(content_blocks=[
    {"type": "text", "text": "Hello, how are you?"},
    {"type": "image", "url": "https://example.com/image.jpg"},
])

Standard content blocks

LangChain provides a standard representation for message content that works across providers.

消息对象实现了一个 content_blocks 属性，该属性会延迟将 content 属性解析为标准的、类型安全的表示形式。例如，从 ChatAnthropic 或 ChatOpenAI 生成的消息将包含各自提供商格式的 thinking 或 reasoning 块，但可以延迟解析为一致的 ReasoningContentBlock 表示形式

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


from langchain.messages import AIMessage

message = AIMessage(
    content=[
        {
            "type": "reasoning",
            "id": "rs_abc123",
            "summary": [
                {"type": "summary_text", "text": "summary 1"},
                {"type": "summary_text", "text": "summary 2"},
            ],
        },
        {"type": "text", "text": "...", "id": "msg_abc123"},
    ],
    response_metadata={"model_provider": "openai"}
)
message.content_blocks

1
2
3


[{'type': 'reasoning', 'id': 'rs_abc123', 'reasoning': 'summary 1'},
 {'type': 'reasoning', 'id': 'rs_abc123', 'reasoning': 'summary 2'},
 {'type': 'text', 'text': '...', 'id': 'msg_abc123'}]

将不同的provider的统一接口,是通过content_blocks属性来实现的

Multimodal

Multimodality refers to the ability to work with data that comes in different forms, such as text, audio, images, and video

Chat models can accept multimodal data as input and generate it as output. Below we show short examples of input messages featuring multimodal data.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30


# From URL
message = {
    "role": "user",
    "content": [
        {"type": "text", "text": "Describe the content of this image."},
        {"type": "image", "url": "https://example.com/path/to/image.jpg"},
    ]
}

# From base64 data
message = {
    "role": "user",
    "content": [
        {"type": "text", "text": "Describe the content of this image."},
        {
            "type": "image",
            "base64": "AAAAIGZ0eXBtcDQyAAAAAGlzb21tcDQyAAACAGlzb2...",
            "mime_type": "image/jpeg",
        },
    ]
}

# From provider-managed File ID
message = {
    "role": "user",
    "content": [
        {"type": "text", "text": "Describe the content of this image."},
        {"type": "image", "file_id": "file-abc123"},
    ]
}

Content blocks are represented (either when creating a message or accessing the content_blocks property) as a list of typed dictionaries

Serialization

You can serialize messages to plain objects for storage and deserialize back to message instances. This is useful for persisting conversation history and resuming sessions.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


from langchain.messages import HumanMessage
from langchain_core.load import dumpd, load

message = HumanMessage("What is the capital of France?")

# Serialize to a plain dict
serialized = dumpd(message)

# Deserialize back to a message object
restored = load(serialized)

Models

Mon, 01 Jan 0001 00:00:00 +0000

In addition to text generation,many LLMs provide :tool calling,Structured output,Multimodality,Reasoning different model excel at different tasks

LangChain has Standard model interface,which allow you to experiment with and switch models

Basic usage

1.with agents 2.Standalone(outside the agent loop)

Initialize a model

the easiest way: use init_chat_model

1
2
3
4
5
6


import os
from langchain.chat_models import init_chat_model

os.environ["OPENAI_API_KEY"] = "sk-..."

model = init_chat_model("gpt-5.5")

key method: invoke,streaming.patch(批量处理)

Parameters

model: string (required) api_key: string temperature: number max_tokens: number timeout: number max_retries: number (default: 6)

1
2
3
4
5
6
7
8


model = init_chat_model(
    "claude-sonnet-4-6",
    # Kwargs passed to the model:
    temperature=0.7,
    timeout=30,
    max_tokens=1000,
    max_retries=6,  # Default; increase for unreliable networks
)

Connection resilience(连接韧性?)

LangChain chat models automatically retry failed API requests with exponential backoff

By default, models retry up to 6 times for network errors, rate limits (429), and server errors (5xx). Client errors like 401 (unauthorized) or 404 are not retried.

Use max_retries and timeout

Invocation

invoke:

with a message along with a thread_id role of a message: models use it to verify who sent the message

Streaming:

Calling stream() returns an iterator that yields output chunks as they are produced. You can use a loop to process each chunk in real-time:

Return AIMessageChunkobjects ,each chunk in a stream is designed to be gathered into a full message via summation

1
2
3
4


full = None  # None | AIMessageChunk
for chunk in model.stream("What color is the sky?"):
    full = chunk if full is None else full + chunk
    print(full.text)

Batch

Batching a collection of independent requests to a model can significantly improve performance and reduce costs, as the processing can be done in parallel:

Use batch_as_completed() to stream result,but the response is out of order

Tool calling

A schema, including the name of the tool, a description, and/or argument definitions (often a JSON schema)
A function or coroutine to execute.

Basic tool calling flow:

Use bind_tools to make tools you have defined available

1
2
3
4
5
6
7
8
9


from langchain.tools import tool

@tool
def get_weather(location: str) -> str:
    """Get the weather at a location."""
    return f"It's sunny in {location}."


model_with_tools = model.bind_tools([get_weather])

Common way to use tool calling

Tool execution loop
Forcing tool calls
Parallel tool calls
Streaming tool calls: tool calls are progressively built through ToolCallChunk You can accumulate chunks to build complete tool calls

Structured output

Provide their response in a format matching a given schema LangChain supports multiple schema types and methods for enforcing structured output.

Pydantic models provide the richest feature set with field validation, descriptions, and nested structures.

Python’s TypedDict provides a simpler alternative to Pydantic models, ideal when you don’t need runtime validation.

Provide a JSON Schema for maximum control and interoperability.

1.raw message和parsed message并存 2.nested structure

Multimodal

You can use content block to provide ono-text data to the model If the response contains multimodal data,the resulting AIMessage

will have content blocks with multimodal types

Reasoning

Many models are capable of performing multi-step reasoning to arrive at a conclusion. This involves breaking down complex problems into smaller, more

manageable steps.If supported by the underlying model, you can surface this reasoning process to better understand how the model arrived at its final answer.

Depending on the model,you can sometimes specify the level of effort it should put into reasoning. Using reasoning_effort parameter when invoking.

Local models

Prompt caching

ou can engage caching at three levels:

Implicit provider caching:
Provider-level explicit controls(显式): providers let you manually indicate cache points for greater control or to guarantee cost savings. These mirror the underlying provider/API behavior. Examples:
- ChatOpenAI (via prompt_cache_key)
- Anthropic content-block cache_control
- Gemini.
- AWS Bedrock cachePoint blocks
LangChain middleware:

for agents, middleware lets LangChain optimize caching of stable system prompt and tool content. Examples:
- Anthropic’s AnthropicPromptCachingMiddleware
- AWS Bedrock’s BedrockPromptCachingMiddleware

Cache usage will be reflected in the usage metadata of the model response.

Server-side tool use

Some providers support server-side tool-calling loops: models can interact with web search, code interpreters, and other tools and analyze the results in a single conversational turn.

If a model invokes a tool server-side, the content of the response message will include content representing the invocation and result of the tool. Accessing the content blocks of the response will return the server-side tool calls and results in a provider-agnostic format:

1
2
3
4
5
6
7
8
9


from langchain.chat_models import init_chat_model

model = init_chat_model("gpt-5.4-mini")

tool = {"type": "web_search"}
model_with_tools = model.bind_tools([tool])

response = model_with_tools.invoke("What was a positive news story from today?")
print(response.content_blocks)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29


[
    {
        "type": "server_tool_call",
        "name": "web_search",
        "args": {
            "query": "positive news stories today",
            "type": "search"
        },
        "id": "ws_abc123"
    },
    {
        "type": "server_tool_result",
        "tool_call_id": "ws_abc123",
        "status": "success"
    },
    {
        "type": "text",
        "text": "Here are some positive news stories from today...",
        "annotations": [
            {
                "end_index": 410,
                "start_index": 337,
                "title": "article title",
                "type": "citation",
                "url": "..."
            }
        ]
    }
]

This represents a single conversational turn; there are no associated ToolMessage objects that need to be passed in as in client-side tool-calling.

Model exceptions

Major integration packages raise standard exception types from langchain_core.exceptions for common model failures such as authentication errors, rate limits, and timeouts. These exceptions inherit from both the LangChain base type and the provider SDK’s own exception type, so you can catch either one:

Rate limiting

To control your call frequency ,and i you hit it, you will receive a rate limit error

To help manage rate limits, chat model integrations accept a rate_limiter parameter that can be provided during initialization to control the rate at which requests are made.

Base URL and proxy settings

You can configure a custom base URL for providers that implement the OpenAI Chat Completions API.

Custom base URL:

1
2
3
4
5
6


model = init_chat_model(
    model="MODEL_NAME",
    model_provider="openai",
    base_url="BASE_URL",
    api_key="YOUR_API_KEY",
)

HTTP proxy configuration:

1
2
3
4
5
6


from langchain_openai import ChatOpenAI

model = ChatOpenAI(
    model="gpt-5.5",
    openai_proxy="http://proxy.example.com:8080"
)

Log probabilities

Token usage

A number of model providers return token usage information as part of the invocation response.

You can track aggregate token counts across models in an application using either a callback or context manager, as shown below:

Callback:

1

You can track aggregate token counts across models in an application using either a callback or context manager, as shown below:

Context manager:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


from langchain.chat_models import init_chat_model
from langchain_core.callbacks import get_usage_metadata_callback

model_1 = init_chat_model(model="gpt-5.4-mini")
model_2 = init_chat_model(model="claude-haiku-4-5-20251001")

with get_usage_metadata_callback() as cb:
    model_1.invoke("Hello")
    model_2.invoke("Hello")
    print(cb.usage_metadata)

Invocation config

When invoking a model, you can pass additional configuration through the config parameter using a RunnableConfig dictionary. This provides run-time control over execution behavior, callbacks, and metadata tracking.

1
2
3
4
5
6
7
8
9


response = model.invoke(
    "Tell me a joke",
    config={
        "run_name": "joke_generation",      # Custom name for this run
        "tags": ["humor", "demo"],          # Tags for categorization
        "metadata": {"user_id": "123"},     # Custom metadata
        "callbacks": [my_callback_handler], # Callback handlers
    }
)

Key configuration attributes run_name: string : identifier tags: string[] metadata: object max_concurrency: number callbacks: array recursion_limit: number

Configurable models: You can also create a runtime-configurable model by specifying configurable_fields. If you don’t specify a model value, then 'model' and 'model_provider' will be configurable by default.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


from langchain.chat_models import init_chat_model

configurable_model = init_chat_model(temperature=0)

configurable_model.invoke(
    "what's your name",
    config={"configurable": {"model": "gpt-5-nano"}},  # Run with GPT-5-Nano
)
configurable_model.invoke(
    "what's your name",
    config={"configurable": {"model": "claude-sonnet-4-6"}},  # Run with Claude
)

Dynamic model selection

Dynamic models are selected at runtime based on the current state and context. This enables sophisticated routing logic and cost optimization.

To use a dynamic model, create middleware using the @wrap_model_call decorator that modifies the model in the request:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26


from langchain_openai import ChatOpenAI
from langchain.agents import create_agent
from langchain.agents.middleware import wrap_model_call, ModelRequest, ModelResponse


basic_model = ChatOpenAI(model="gpt-5.4-mini")
advanced_model = ChatOpenAI(model="gpt-5.5")

@wrap_model_call
def dynamic_model_selection(request: ModelRequest, handler) -> ModelResponse:
    """Choose model based on conversation complexity."""
    message_count = len(request.state["messages"])

    if message_count > 10:
        # Use an advanced model for longer conversations
        model = advanced_model
    else:
        model = basic_model

    return handler(request.override(model=model))

agent = create_agent(
    model=basic_model,  # Default model
    tools=[],
    middleware=[dynamic_model_selection]
)

Pre-bound models (models with bind_tools already called) are not supported when using structured output. If you need dynamic model selection with structured output, ensure the models passed to the middleware are not pre-bound.

Short-term Memory

Mon, 01 Jan 0001 00:00:00 +0000

Conversation history is the most common form of short-term memory

Fact1: message includes instructions and inpus

Usage

To add short-term memory to an agent ,you need to specify a checkpointer when creating an agent note: short-term memory is part of your agent’s state. State is persisted in a database using a checkpointer so thread can resume it an any time. Short-term memory updates when the agent is invoked or a step (like a tool call) is completed, and the state is read at the start of each step.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29


from langchain.agents import create_agent
from langgraph.checkpoint.memory import InMemorySaver  


def get_user_info() -> str:
    """Look up information about the current user."""
    return "No user profile on file."


agent = create_agent(
    model="openai:gpt-5.5",
    tools=[get_user_info],
    checkpointer=InMemorySaver(),
)

thread_config = {"configurable": {"thread_id": "1"}}
response = agent.invoke(
    {"messages": [{"role": "user", "content": "Hi! My name is Bob."}]},
    thread_config,
)["messages"][-1].content

print(response)  # "Hi Bob! Nice to see you here. How are you doing?"

response = agent.invoke(
    {"messages": [{"role": "user", "content": "What's my name?"}]},
    thread_config,
)["messages"][-1].content

print(response)  # "You are Bob!"

In production

use a checkpointer backed by a database

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29


from langchain.agents import create_agent
from langgraph.checkpoint.memory import InMemorySaver  


def get_user_info() -> str:
    """Look up information about the current user."""
    return "No user profile on file."


agent = create_agent(
    model="openai:gpt-5.5",
    tools=[get_user_info],
    checkpointer=InMemorySaver(),
)

thread_config = {"configurable": {"thread_id": "1"}}
response = agent.invoke(
    {"messages": [{"role": "user", "content": "Hi! My name is Bob."}]},
    thread_config,
)["messages"][-1].content

print(response)  # "Hi Bob! Nice to see you here. How are you doing?"

response = agent.invoke(
    {"messages": [{"role": "user", "content": "What's my name?"}]},
    thread_config,
)["messages"][-1].content

print(response)  # "You are Bob!"

Customizing agent memory

Agents use AgentState to manage short-term memory, specifically the conversation history via a message key

You can extend AgentState to ad additional fields Custom state schema are passed to create_agent using the state_scheam parameter

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24


from langchain.agents import create_agent, AgentState
from langgraph.checkpoint.memory import InMemorySaver


class CustomAgentState(AgentState):
    user_id: str
    preferences: dict

agent = create_agent(
    "gpt-5.5",
    tools=[get_user_info],
    state_schema=CustomAgentState,
    checkpointer=InMemorySaver(),
)

# Custom state can be passed in invoke
result = agent.invoke(
    {
        "messages": [{"role": "user", "content": "Hello"}],
        "user_id": "user_123",
        "preferences": {"theme": "dark"}
    },
    {"configurable": {"thread_id": "1"}})
    

Common patterns

Common solutions to exceed the LLM’s context window

Trim message(截断消息)

A way: keep track of the token count and truncate when it approaches that limit

Using trim messages utility and specify the number if tokens to keep ang the strategy to use for handing the boundary

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51



@before_mfrom langchain.messages import RemoveMessage
from langgraph.graph.message import REMOVE_ALL_MESSAGES
from langgraph.checkpoint.memory import InMemorySaver
from langchain.agents import create_agent, AgentState
from langchain.agents.middleware import before_model
from langgraph.runtime import Runtime
from langchain_core.runnables import RunnableConfig
from typing import Any


@before_model
def trim_messages(state: AgentState, runtime: Runtime) -> dict[str, Any] | None:
    """Keep only the last few messages to fit context window."""
    messages = state["messages"]

    if len(messages) <= 3:
        return None  # No changes needed

    first_msg = messages[0]
    recent_messages = messages[-3:] if len(messages) % 2 == 0 else messages[-4:]
    new_messages = [first_msg] + recent_messages

    return {
        "messages": [
            RemoveMessage(id=REMOVE_ALL_MESSAGES),
            *new_messages
        ]
    }

agent = create_agent(
    "gpt-5.5",
    tools=[...],
    middleware=[trim_messages],
    checkpointer=InMemorySaver(),
)

config: RunnableConfig = {"configurable": {"thread_id": "1"}}

agent.invoke({"messages": "hi, my name is bob"}, config)
agent.invoke({"messages": "write a short poem about cats"}, config)
agent.invoke({"messages": "now do the same but for dogs"}, config)
final_response = agent.invoke({"messages": "what's my name?"}, config)

final_response["messages"][-1].pretty_print()
"""
================================== Ai Message ==================================

Your name is Bob. You told me that earlier.
If you'd like me to call you a nickname or use a different name, just say the word.
"""

Delete message

Useful when you want to remove specific message or clear the entire message history Use RemoveMessage to delete message from the graph state and you need to use a state key with add_messages reducer and default theAgentState provides it

To remove specific messages:

1
2
3
4
5
6
7


from langchain.messages import RemoveMessage  

def delete_messages(state):
    messages = state["messages"]
    if len(messages) > 2:
        # remove the earliest two messages
        return {"messages": [RemoveMessage(id=m.id) for m in messages[:2]]}

To remove all messages:

1
2
3
4


from langgraph.graph.message import REMOVE_ALL_MESSAGES  

def delete_messages(state):
    return {"messages": [RemoveMessage(id=REMOVE_ALL_MESSAGES)]}

ps: 删除消息时，请确保生成的消息历史有效。检查你正在使用的 LLM 提供商的限制。例如：

某些提供程序期望消息历史记录以 user 消息开头
大多数提供程序要求带有工具调用的 assistant 消息后跟相应的 tool 结果消息。

Summarize messages

above pattern will lose information Summarizing message with a LLM and use the built-in SUmmarizationMiddleware

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


agent = create_agent(
    model="gpt-5.5",
    tools=[...],
    middleware=[
        SummarizationMiddleware(
            model="gpt-5.4-mini",
            trigger=("tokens", 4000),
            keep=("messages", 20)
        )
    ],
    checkpointer=checkpointer,
)

Access memory

tools

Read short-term memory in a tool

Using runtime parameter(typed as ToolRuntime) runtime parameter can not be seen by model but the tool can access the state through it

1
2
3
4
5
6
7


@tool
def get_user_info(
    runtime: ToolRuntime
) -> str:
    """Look up user info."""
    user_id = runtime.state["user_id"]
    return "User is John Smith" if user_id == "user_123" else "Unknown user"

write short-term memory form tools

Using return Command with a ToolMessage

Prompt

Using middleware to access to ~ to create dynamic prompts on conversation history or custom state fields

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34


from langchain.agents import create_agent
from typing import TypedDict
from langchain.agents.middleware import dynamic_prompt, ModelRequest


class CustomContext(TypedDict):
    user_name: str


def get_weather(city: str) -> str:
    """Get the weather in a city."""
    return f"The weather in {city} is always sunny!"


@dynamic_prompt
def dynamic_system_prompt(request: ModelRequest) -> str:
    user_name = request.runtime.context["user_name"]
    system_prompt = f"You are a helpful assistant. Address the user as {user_name}."
    return system_prompt


agent = create_agent(
    model="gpt-5-nano",
    tools=[get_weather],
    middleware=[dynamic_system_prompt],
    context_schema=CustomContext,
)

result = agent.invoke(
    {"messages": [{"role": "user", "content": "What is the weather in SF?"}]},
    context=CustomContext(user_name="John Smith"),
)
for msg in result["messages"]:
    msg.pretty_print()

Before model

Using @before_model middleware to process message before model calls

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51


from langchain.messages import RemoveMessage
from langgraph.graph.message import REMOVE_ALL_MESSAGES
from langgraph.checkpoint.memory import InMemorySaver
from langchain.agents import create_agent, AgentState
from langchain.agents.middleware import before_model
from langchain_core.runnables import RunnableConfig
from langgraph.runtime import Runtime
from typing import Any


@before_model
def trim_messages(state: AgentState, runtime: Runtime) -> dict[str, Any] | None:
    """Keep only the last few messages to fit context window."""
    messages = state["messages"]

    if len(messages) <= 3:
        return None  # No changes needed

    first_msg = messages[0]
    recent_messages = messages[-3:] if len(messages) % 2 == 0 else messages[-4:]
    new_messages = [first_msg] + recent_messages

    return {
        "messages": [
            RemoveMessage(id=REMOVE_ALL_MESSAGES),
            *new_messages
        ]
    }


agent = create_agent(
    "gpt-5-nano",
    tools=[],
    middleware=[trim_messages],
    checkpointer=InMemorySaver()
)

config: RunnableConfig = {"configurable": {"thread_id": "1"}}

agent.invoke({"messages": "hi, my name is bob"}, config)
agent.invoke({"messages": "write a short poem about cats"}, config)
agent.invoke({"messages": "now do the same but for dogs"}, config)
final_response = agent.invoke({"messages": "what's my name?"}, config)

final_response["messages"][-1].pretty_print()
"""
================================== Ai Message ==================================

Your name is Bob. You told me that earlier.
If you'd like me to call you a nickname or use a different name, just say the word.
"""

After model Using @after_model

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


from langchain.messages import RemoveMessage
from langgraph.checkpoint.memory import InMemorySaver
from langchain.agents import create_agent, AgentState
from langchain.agents.middleware import after_model
from langgraph.runtime import Runtime


@after_model
def validate_response(state: AgentState, runtime: Runtime) -> dict | None:
    """Remove messages containing sensitive words."""
    STOP_WORDS = ["password", "secret"]
    last_message = state["messages"][-1]
    if any(word in last_message.content for word in STOP_WORDS):
        return {"messages": [RemoveMessage(id=last_message.id)]}
    return None

agent = create_agent(
    model="gpt-5-nano",
    tools=[],
    middleware=[validate_response],
    checkpointer=InMemorySaver(),
)

Streaming

Mon, 01 Jan 0001 00:00:00 +0000

Overview

实现的功能:

Stream agent progress
Stream LLM tokens
Streaming thinking/reasoning tokens
Stream custom updates
Stream multiple modes:从updates,messages,custom中进行选择

Supported stream modes

将下列的一个或者多个流式传输模式作为列表传输给stream或者astream方法

Mode	Description
`updates`	Streams state updates after each agent step. If multiple updates are made in the same step (e.g., multiple nodes are run), those updates are streamed separately.
`messages`	Streams tuples of `(token, metadata)` from any graph nodes where an LLM is invoked.
`custom`	Streams custom data from inside your graph nodes using the stream writer.

Agent progress

使用带有stream_mode =“updates"的stream或者astream方法将在每个agent step之后发送一个事件

例子: 在一次tool call之后,你应该看到下列更新

LLM node: AIMessage with tool call requests
Tool node: ToolMessage with execution result
LLM node: Final AI response

通过config传递thread_id以便checkpointer进行保存和后续的resume 也可以传递context,后续可以使用runtime.context进行访问

LLM tokens

使用stream_mode = "message"来访问token usage 在输出中可以看到agent的工具调用和最终的response

Custom updates

想要流式传输工具执行的过程(或结果),使用``get_stream_writer `inside the tool

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24


from langchain.agents import create_agent
from langgraph.config import get_stream_writer  


def get_weather(city: str) -> str:
    """Get weather for a given city."""
    writer = get_stream_writer()
    # stream any arbitrary data
    writer(f"Looking up data for city: {city}")
    writer(f"Acquired data for city: {city}")
    return f"It's always sunny in {city}!"

agent = create_agent(
    model="claude-sonnet-4-6",
    tools=[get_weather],
)

for chunk in agent.stream(
    {"messages": [{"role": "user", "content": "What is the weather in SF?"}]},
    stream_mode="custom",
    version="v2",
):
    if chunk["type"] == "custom":
        print(chunk["data"])

Stream multiple modes

通过将它们以mode列表的形式传输:stream_mode=[“updates”,“custom”]

每一段streamed chunk都是StreamPart的dict with type,ns and data 的keys 通过chunk[“type”]来决定流式模式,通过chunk[“data”]来访问负载

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24


from langchain.agents import create_agent
from langgraph.config import get_stream_writer


def get_weather(city: str) -> str:
    """Get weather for a given city."""
    writer = get_stream_writer()
    writer(f"Looking up data for city: {city}")
    writer(f"Acquired data for city: {city}")
    return f"It's always sunny in {city}!"

agent = create_agent(
    model="gpt-5-nano",
    tools=[get_weather],
)

for chunk in agent.stream(
    {"messages": [{"role": "user", "content": "What is the weather in SF?"}]},
    stream_mode=["updates", "custom"],
    version="v2",
):
    print(f"stream_mode: {chunk['type']}")
    print(f"content: {chunk['data']}")
    print("\n")

Common patterns

常见的几种应用场景

Stream thinking/reasoning tokens

一些模型会在产生最终的answer之前在内部进行思考可以通过过滤standard contnent blocks中的type = “reasoning"的部分来这些thinging/reasoning tokens

需要使用的mode为message即,stream_mode =“message”(与上文相符合)

1
2
3
4
5
6
7
8
9


stream = agent.stream_events(
    {"messages": [{"role": "user", "content": "What is the weather in SF?"}]},
    version="v3",
)
for message in stream.messages:
    for token in message.reasoning:
        print(f"[thinking] {token}", end="")
    for token in message.text:
        print(token, end="", flush=True)

Streaming tool calls

if you want to stream both:

Partial JSON as tool calls are generated
The completed, parsed tool calls that are executed

前者可以通过指定stream_mode= “message” 后者,根据是否在state中被tracked: True: 指定stream_mode = [“messages”,“updates”],并通过state updates来访问(如下所示) False: 使用custom updates 或者 aggregate the chunks during the streaming loop (next section).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


for chunk in agent.stream(
    {"messages": [input_message]},
    stream_mode=["messages", "updates"],
    version="v2",
):
    if chunk["type"] == "messages":
        token, metadata = chunk["data"]
        if isinstance(token, AIMessageChunk):
            _render_message_chunk(token)
    elif chunk["type"] == "updates":
        for source, update in chunk["data"].items():
            if source in ("model", "tools"):  # `source` captures node name
                _render_completed_message(update["messages"][-1])

Accessing completed messages

在message不通过state updates反映时采取的途径之一通过aggrrgate message chunks in the streaming loop

1.使用stream_writer inside the tool 2.如果你不能添加自定义的时间到stream中时,你可以流式传输中聚合消息块

Streaming with human-in-the-loop

解决human-in-the-loop interrupts:

为agent配置hunman-in-the-loop middleare 和 checkpointer
搜集在"update"流式传输模式下的interrupts
通过command来对这些interrupts进行响应

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72


from typing import Any

from langchain.agents import create_agent
from langchain.agents.middleware import HumanInTheLoopMiddleware
from langchain.messages import AIMessage, AIMessageChunk, AnyMessage, ToolMessage
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.types import Command, Interrupt


def get_weather(city: str) -> str:
    """Get weather for a given city."""

    return f"It's always sunny in {city}!"


checkpointer = InMemorySaver()

agent = create_agent(
    "openai:gpt-5.5",
    tools=[get_weather],
    middleware=[
        HumanInTheLoopMiddleware(interrupt_on={"get_weather": True}),
    ],
    checkpointer=checkpointer,
)


def _render_message_chunk(token: AIMessageChunk) -> None:
    if token.text:
        print(token.text, end="|")
    if token.tool_call_chunks:
        print(token.tool_call_chunks)


def _render_completed_message(message: AnyMessage) -> None:
    if isinstance(message, AIMessage) and message.tool_calls:
        print(f"Tool calls: {message.tool_calls}")
    if isinstance(message, ToolMessage):
        print(f"Tool response: {message.content_blocks}")


def _render_interrupt(interrupt: Interrupt) -> None:
    interrupts = interrupt.value  
    for request in interrupts["action_requests"]:
        print(request["description"])


input_message = {
    "role": "user",
    "content": (
        "Can you look up the weather in Boston and San Francisco?"
    ),
}
config = {"configurable": {"thread_id": "some_id"}}
interrupts = []
for chunk in agent.stream(
    {"messages": [input_message]},
    config=config,
    stream_mode=["messages", "updates"],
    version="v2",
):
    if chunk["type"] == "messages":
        token, metadata = chunk["data"]
        if isinstance(token, AIMessageChunk):
            _render_message_chunk(token)
    elif chunk["type"] == "updates":
        for source, update in chunk["data"].items():
            if source in ("model", "tools"):
                _render_completed_message(update["messages"][-1])
            if source == "__interrupt__":
                interrupts.extend(update)
                _render_interrupt(update[0])

接下来,为每一个interrupt收集对应的decision,并且decision的顺序与我们收集的action对应

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21


def _get_interrupt_decisions(interrupt: Interrupt) -> list[dict]:
    return [
        {
            "type": "edit",
            "edited_action": {
                "name": "get_weather",
                "args": {"city": "Boston, U.K."},
            },
        }
        if "boston" in request["description"].lower()
        else {"type": "approve"}
        for request in interrupt.value["action_requests"]
    ]

decisions = {}
for interrupt in interrupts:
    decisions[interrupt.id] = {
        "decisions": _get_interrupt_decisions(interrupt)
    }

decisions

我们可以通过向流式传输的循环传入一个command来resume

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


interrupts = []
for chunk in agent.stream(
    Command(resume=decisions),
    config=config,
    stream_mode=["messages", "updates"],
    version="v2",
):
    # Streaming loop is unchanged
    if chunk["type"] == "messages":
        token, metadata = chunk["data"]
        if isinstance(token, AIMessageChunk):
            _render_message_chunk(token)
    elif chunk["type"] == "updates":
        for source, update in chunk["data"].items():
            if source in ("model", "tools"):
                _render_completed_message(update["messages"][-1])
            if source == "__interrupt__":
                interrupts.extend(update)
                _render_interrupt(update[0])

Streaming from sub-agents

当agent中有多个LLMs时,需要对其来源进行消歧在创建agent时使用name参数,该参数在"message"的stream_mode中可以通过lc_agent_name键中的metadata访问 ps: 在创建stream时specify subgraph是= True

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35


from typing import Any

from langchain.agents import create_agent
from langchain.chat_models import init_chat_model
from langchain.messages import AIMessage, AnyMessage


def get_weather(city: str) -> str:
    """Get weather for a given city."""

    return f"It's always sunny in {city}!"


weather_model = init_chat_model("openai:gpt-5.5")
weather_agent = create_agent(
    model=weather_model,
    tools=[get_weather],
    name="weather_agent",
)


def call_weather_agent(query: str) -> str:
    """Query the weather agent."""
    result = weather_agent.invoke({
        "messages": [{"role": "user", "content": query}]
    })
    return result["messages"][-1].text


supervisor_model = init_chat_model("openai:gpt-5.5")
agent = create_agent(
    model=supervisor_model,
    tools=[call_weather_agent],
    name="supervisor",
)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34


def _render_message_chunk(token: AIMessageChunk) -> None:
    if token.text:
        print(token.text, end="|")
    if token.tool_call_chunks:
        print(token.tool_call_chunks)


def _render_completed_message(message: AnyMessage) -> None:
    if isinstance(message, AIMessage) and message.tool_calls:
        print(f"Tool calls: {message.tool_calls}")
    if isinstance(message, ToolMessage):
        print(f"Tool response: {message.content_blocks}")


input_message = {"role": "user", "content": "What is the weather in Boston?"}
current_agent = None
for chunk in agent.stream(
    {"messages": [input_message]},
    stream_mode=["messages", "updates"],
    subgraphs=True,
    version="v2",
):
    if chunk["type"] == "messages":
        token, metadata = chunk["data"]
        if agent_name := metadata.get("lc_agent_name"):
            if agent_name != current_agent:
                print(f"🤖 {agent_name}: ")
                current_agent = agent_name  
        if isinstance(token, AIMessageChunk):
            _render_message_chunk(token)
    elif chunk["type"] == "updates":
        for source, update in chunk["data"].items():
            if source in ("model", "tools"):
                _render_completed_message(update["messages"][-1])

Disable streaming

初始化时: set streaming = False

v2 streaming format

Pass version="v2" to stream() or astream() to get a unified output format. Every chunk is a StreamPart dict with type, ns, and data keys — the same shape regardless of stream mode or number of modes:

The v2 format also improves invoke() — it returns a GraphOutput object with .value and .interrupts attributes, cleanly separating state from interrupt metadata:

Structured Output

Mon, 01 Jan 0001 00:00:00 +0000

返回的形式: JSON objects, Pydantic models,or dataclasses that your application can use directly

LangChain’s create_agent 会自动处理结构化输出用户设置desired 结构化输出的schema 返回在 in the key strucured_responese key of the agent’s state

Field函数的作用: 结构化输出时,是创建元数据的重要途径,且其内部的description可以作为Prompt的一部分指导LLM的工作

1
2
3
4
5
6
7
8
9


def create_agent(
    ...
    response_format: Union[
        ToolStrategy[StructuredResponseT],
        ProviderStrategy[StructuredResponseT],
        type[StructuredResponseT],
        None,
    ]
)

Response format

使用response_format来控制agent返回结构化数据的方式

ToolStrategy[StructuredResponseT]: Uses tool calling for structured output
ProviderStrategy[StructuredResponseT]: Uses provider-native structured output
type[StructuredResponseT]: Schema type - automatically selects best strategy based on model capabilities
None: Structured output not explicitly requested

When a schema type is provided directly, LangChain automatically chooses:

ProviderStrategy if the model and provider chosen supports native structured output (e.g. OpenAI, Anthropic (Claude), or xAI (Grok)).
ToolStrategy for all other models.

Provider strategy

一些provider支持原生的结构化输出这是最可靠的方式要使用这种方式,请配置 Providerstrategy

1
2
3


class ProviderStrategy(Generic[SchemaT]):
    schema: type[SchemaT]
    strict: bool | None = None

参数 schema(required): 规定了结构化输出的format,取值有:

Pydantic models: BaseModel subclasses with field validation. Returns validated Pydantic instance.
Dataclasses: Python dataclasses with type annotations. Returns dict.
TypedDict: Typed dictionary classes. Returns dict.
JSON Schema: Dictionary with JSON schema specification. Must include top-level title and description keys. Returns dict.

strict: 可选;默认None;控制是否严格遵守schema

当你直接把schema type传给create_agent.response_format 时且模型支持原生的结构化输出时,LangChain会自动选择ProviderStrategy

Tool calling strategy

对于模型不支持原生的结构化输出,LangChain使用tool calling,这对于所有的支持tool calling的模型都适用使用:配置 ToolStrategy

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


class ToolStrategy(Generic[SchemaT]):
    schema: type[SchemaT]
    tool_message_content: str | None
    handle_errors: Union[
        bool,
        str,
        type[Exception],
        tuple[type[Exception], ...],
        Callable[[Exception], str],
    ]

schema: required 同上

tool_message_content: 生成结构化输出时的tool message的自定义内容如果没有提供,为显示结构化响应的响应数据

handle_errors: 默认为True 当结构化输出的验证失败的时候的错误处理策略,参数如下

True: Catch all errors with default error template
str: Catch all errors with this custom message
type[Exception]: Only catch this exception type with default message
tuple[type[Exception], ...]: Only catch these exception types with default message
Callable[[Exception], str]: Custom function that returns error message
False: No retry, let exceptions propagate

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29


from pydantic import BaseModel, Field
from typing import Literal, Union
from langchain.agents import create_agent
from langchain.agents.structured_output import ToolStrategy


class ProductReview(BaseModel):
    """Analysis of a product review."""
    rating: int | None = Field(description="The rating of the product", ge=1, le=5)
    sentiment: Literal["positive", "negative"] = Field(description="The sentiment of the review")
    key_points: list[str] = Field(description="The key points of the review. Lowercase, 1-3 words each.")

class CustomerComplaint(BaseModel):
    """A customer complaint about a product or service."""
    issue_type: Literal["product", "service", "shipping", "billing"] = Field(description="The type of issue")
    severity: Literal["low", "medium", "high"] = Field(description="The severity of the complaint")
    description: str = Field(description="Brief description of the complaint")

agent = create_agent(
    model="gpt-5.5",
    tools=[],
    response_format=ToolStrategy(Union[ProductReview, CustomerComplaint])
)

result = agent.invoke({
    "messages": [{"role": "user", "content": "Analyze this review: 'Great product: 5 out of 5 stars. Fast shipping, but expensive'"}]
})
result["structured_response"]
# ProductReview(rating=5, sentiment='positive', key_points=['fast shipping', 'expensive'])

Custom tool message content

the tool_message_content 参数允许你自定义结构化输出产生时呈现在会话历史中的message

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29


from pydantic import BaseModel, Field
from typing import Literal, Union
from langchain.agents import create_agent
from langchain.agents.structured_output import ToolStrategy


class ProductReview(BaseModel):
    """Analysis of a product review."""
    rating: int | None = Field(description="The rating of the product", ge=1, le=5)
    sentiment: Literal["positive", "negative"] = Field(description="The sentiment of the review")
    key_points: list[str] = Field(description="The key points of the review. Lowercase, 1-3 words each.")

class CustomerComplaint(BaseModel):
    """A customer complaint about a product or service."""
    issue_type: Literal["product", "service", "shipping", "billing"] = Field(description="The type of issue")
    severity: Literal["low", "medium", "high"] = Field(description="The severity of the complaint")
    description: str = Field(description="Brief description of the complaint")

agent = create_agent(
    model="gpt-5.5",
    tools=[],
    response_format=ToolStrategy(Union[ProductReview, CustomerComplaint])
)

result = agent.invoke({
    "messages": [{"role": "user", "content": "Analyze this review: 'Great product: 5 out of 5 stars. Fast shipping, but expensive'"}]
})
result["structured_response"]
# ProductReview(rating=5, sentiment='positive', key_points=['fast shipping', 'expensive'])

Error handing

LangChain提供了智能重试机制

Multiple structured outputs error

当模型错误地调用了多个结构化工具输出,会在ToolMessage中生成相应的反馈和提示词以便模型进行重试

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23


from pydantic import BaseModel, Field
from typing import Union
from langchain.agents import create_agent
from langchain.agents.structured_output import ToolStrategy


class ContactInfo(BaseModel):
    name: str = Field(description="Person's name")
    email: str = Field(description="Email address")

class EventDetails(BaseModel):
    event_name: str = Field(description="Name of the event")
    date: str = Field(description="Event date")

agent = create_agent(
    model="gpt-5.5",
    tools=[],
    response_format=ToolStrategy(Union[ContactInfo, EventDetails])  # Default: handle_errors=True
)

agent.invoke({
    "messages": [{"role": "user", "content": "Extract info: John Doe (john@email.com) is organizing Tech Conference on March 15th"}]
})

Schema validation error

当结构化输出和schema不匹配时,agent会返回特定的错误反馈

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


from pydantic import BaseModel, Field
from langchain.agents import create_agent
from langchain.agents.structured_output import ToolStrategy


class ProductRating(BaseModel):
    rating: int | None = Field(description="Rating from 1-5", ge=1, le=5)
    comment: str = Field(description="Review comment")

agent = create_agent(
    model="gpt-5.5",
    tools=[],
    response_format=ToolStrategy(ProductRating),  # Default: handle_errors=True
    system_prompt="You are a helpful assistant that parses product reviews. Do not make any field or value up."
)

agent.invoke({
    "messages": [{"role": "user", "content": "Parse this: Amazing product, 10/10!"}]
})

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27


================================ Human Message =================================

Parse this: Amazing product, 10/10!
================================== Ai Message ==================================
Tool Calls:
  ProductRating (call_1)
 Call ID: call_1
  Args:
    rating: 10
    comment: Amazing product
================================= Tool Message =================================
Name: ProductRating

Error: Failed to parse structured output for tool 'ProductRating': 1 validation error for ProductRating.rating
  Input should be less than or equal to 5 [type=less_than_equal, input_value=10, input_type=int].
 Please fix your mistakes.
================================== Ai Message ==================================
Tool Calls:
  ProductRating (call_2)
 Call ID: call_2
  Args:
    rating: 5
    comment: Amazing product
================================= Tool Message =================================
Name: ProductRating

Returning structured response: {'rating': 5, 'comment': 'Amazing product'}

Error handling strategies

你可以通过handle_errors参数自定义错误处理的方式

Custom error message:

1
2
3
4


ToolStrategy(
    schema=ProductRating,
    handle_errors="Please provide a valid rating between 1-5 and include a comment."
)

If handle_errors is a string, the agent will always prompt the model to re-try with a fixed tool message:

1
2
3
4


================================= Tool Message =================================
Name: ProductRating

Please provide a valid rating between 1-5 and include a comment.

Handle specific exceptions only:

1
2
3
4


ToolStrategy(
    schema=ProductRating,
    handle_errors=ValueError  # Only retry on ValueError, raise others
)

If handle_errors is an exception type, the agent will only retry (using the default error message) if the exception raised is the specified type. In all other cases, the exception will be raised.Handle multiple exception types:

1
2
3
4


ToolStrategy(
    schema=ProductRating,
    handle_errors=(ValueError, TypeError)  # Retry on ValueError and TypeError
)

If handle_errors is a tuple of exceptions, the agent will only retry (using the default error message) if the exception raised is one of the specified types. In all other cases, the exception will be raised.Custom error handler function:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32


from langchain.agents.structured_output import StructuredOutputValidationError
from langchain.agents.structured_output import MultipleStructuredOutputsError

def custom_error_handler(error: Exception) -> str:
    if isinstance(error, StructuredOutputValidationError):
        return "There was an issue with the format. Try again."
    elif isinstance(error, MultipleStructuredOutputsError):
        return "Multiple structured outputs were returned. Pick the most relevant one."
    else:
        return f"Error: {str(error)}"


agent = create_agent(
    model="gpt-5.5",
    tools=[],
    response_format=ToolStrategy(
                        schema=Union[ContactInfo, EventDetails],
                        handle_errors=custom_error_handler
                    )  # Default: handle_errors=True
)

result = agent.invoke({
    "messages": [{"role": "user", "content": "Extract info: John Doe (john@email.com) is organizing Tech Conference on March 15th"}]
})

for msg in result['messages']:
    # If message is actually a ToolMessage object (not a dict), check its class name
    if type(msg).__name__ == "ToolMessage":
        print(msg.content)
    # If message is a dictionary or you want a fallback
    elif isinstance(msg, dict) and msg.get('tool_call_id'):
        print(msg['content'])

On StructuredOutputValidationError:

1
2
3
4


================================= Tool Message =================================
Name: ToolStrategy

There was an issue with the format. Try again.

On MultipleStructuredOutputsError:

1
2
3
4


================================= Tool Message =================================
Name: ToolStrategy

Multiple structured outputs were returned. Pick the most relevant one.

On other errors:

1
2
3
4


================================= Tool Message =================================
Name: ToolStrategy

Error: 

No error handling:

1
2
3
4


response_format = ToolStrategy(
    schema=ProductRating,
    handle_errors=False  # All errors raised
)

Tools

Mon, 01 Jan 0001 00:00:00 +0000

Tools

Extend what agents can do!

Create tools

Basic tool definition

The easiest way to create a tool is to use the @tool decorator In this way the docstring will become the description of the tool

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


from langchain.tools import tool

@tool
def search_database(query: str, limit: int = 10) -> str:
    """Search the customer database for records matching the query.

    Args:
        query: Search terms to look for
        limit: Maximum number of results to return
    """
    return f"Found {limit} results for '{query}'"

docstring:’’' ada ’’' 的部分 docstring应该内容丰富且简明扼要参数的type hints是必须的,因为他们定义了工具的输入参数

Customize tool properties

Custom tool name

by default,the tool name is the function name. But you can override it.

1
2
3
4
5
6


@tool("web_search")  # Custom name
def search(query: str) -> str:
    """Search the web for information."""
    return f"Results for: {query}"

print(search.name)  # web_search

Custom tool description

Override the auto-generated tool description(refer to the docstring) for clearer model guidance

1
2
3
4


@tool("calculator", description="Performs arithmetic calculations. Use this for any math problems.")
def calc(expression: str) -> str:
    """Evaluate mathematical expressions."""
    return str(eval(expression))

Advanced schema definition

Define complex inputs with Pydantic models or JSON schemas

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23


from pydantic import BaseModel, Field
from typing import Literal

class WeatherInput(BaseModel):
    """Input for weather queries."""
    location: str = Field(description="City name or coordinates")
    units: Literal["celsius", "fahrenheit"] = Field(
        default="celsius",
        description="Temperature unit preference"
    )
    include_forecast: bool = Field(
        default=False,
        description="Include 5-day forecast"
    )

@tool(args_schema=WeatherInput)
def get_weather(location: str, units: str = "celsius", include_forecast: bool = False) -> str:
    """Get current weather and optional forecast."""
    temp = 22 if units == "celsius" else 72
    result = f"Current weather in {location}: {temp} degrees {units[0].upper()}"
    if include_forecast:
        result += "\nNext 5 days: Sunny"
    return result

Reserved argument names

Those can not be used as parameters’ name which will cause runtime errors config: reserved for RunnableConfig runtime: reserved for ToolRuntime

Access context

How to access and update this information(conversation history,user data,persistent memory)

How! By the ToolRuntime parameter

Component	Description	Use case
State	Short-term memory - mutable data that exists for the current conversation (messages, counters, custom fields)	Access conversation history, track tool call counts
Context	Immutable configuration passed at invocation time (user IDs, session info)	Personalize responses based on user identity
Store	Long-term memory - persistent data that survives across conversations	Save user preferences, maintain knowledge base
Stream Writer	Emit real-time updates during tool execution	Show progress for long-running operations
Execution Info	Identity and retry information for the current execution (thread ID, run ID, attempt number)	Access thread/run IDs, adjust behavior based on retry state
Server Info	Server-specific metadata when running on LangGraph Server (assistant ID, graph ID, authenticated user)	Access assistant ID, graph ID, or authenticated user info
Config	`RunnableConfig` for the execution	Access callbacks, tags, and metadata
Tool Call ID	Unique identifier for the current tool invocation	Correlate tool calls for logs and model invocations

Short-term memory(State)

Include the message history and any custom field you define in your graph state.

Access state

Use runtime: ToolRuntime to your signature to access state 在调用的时候,ToolNode会自动地注入value,但是这个参数不会出现在发送给model地schema上你需要通过runtime.state才能访问到当前对话的state

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24


from langchain.tools import tool, ToolRuntime
from langchain.messages import HumanMessage

@tool
def get_last_user_message(runtime: ToolRuntime) -> str:
    """Get the most recent message from the user."""
    messages = runtime.state["messages"]

    # Find the last human message
    for message in reversed(messages):
        if isinstance(message, HumanMessage):
            return message.content

    return "No user messages found"

# Access custom state fields
@tool
def get_user_preference(
    pref_name: str,
    runtime: ToolRuntime
) -> str:
    """Get a user preference value."""
    preferences = runtime.state.get("user_preferences", {})
    return preferences.get(pref_name, "Not set")

The runtime parameter is hidden from the model. For the example above, the model only sees pref_name in the tool schema.(上面说过了,结合例子更好理解)

Update state

Use Command to update the agent’s state. Useful to update custom state field. Include a ToolMessage in the update so the model can see the result of this tool call

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24


from langchain.agents import AgentState
from langchain.messages import ToolMessage
from langchain.tools import ToolRuntime, tool
from langgraph.types import Command


class CustomState(AgentState):
    user_name: str


@tool
def set_user_name(new_name: str, runtime: ToolRuntime[None, CustomState]) -> Command:
    """Set the user's name in the conversation state."""
    return Command(
        update={
            "user_name": new_name,
            "messages": [
                ToolMessage(
                    content=f"User name set to {new_name}.",
                    tool_call_id=runtime.tool_call_id,
                )
            ],
        }
    )

When tools update state variables, consider defining a reducer for those fields. Since LLMs can call multiple tools in parallel, a reducer determines how to resolve conflicts when the same state field is updated by concurrent tool calls. (创建reducer来应对并发修改同一个state值)

Context

Provides immutable configuration at invocation time Including user IDs,session details,application-specific settings

Access context through runtime.context. Pass it alongside a thread_id so the conversation is persisted across turns.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57


from dataclasses import dataclass

from langchain.agents import create_agent
from langchain.tools import tool, ToolRuntime
from langchain_core.utils.uuid import uuid7
from langchain_openai import ChatOpenAI


USER_DATABASE = {
    "user123": {
        "name": "Alice Johnson",
        "account_type": "Premium",
        "balance": 5000,
        "email": "alice@example.com",
    },
    "user456": {
        "name": "Bob Smith",
        "account_type": "Standard",
        "balance": 1200,
        "email": "bob@example.com",
    },
}


@dataclass
class UserContext:
    user_id: str


@tool
def get_account_info(runtime: ToolRuntime[UserContext]) -> str:
    """Get the current user's account information."""
    user_id = runtime.context.user_id

    if user_id in USER_DATABASE:
        user = USER_DATABASE[user_id]
        return (
            f"Account holder: {user['name']}\n"
            f"Type: {user['account_type']}\n"
            f"Balance: ${user['balance']}"
        )
    return "User not found"


model = ChatOpenAI(model="gpt-5.5")
agent = create_agent(
    model,
    tools=[get_account_info],
    context_schema=UserContext,
    system_prompt="You are a financial assistant.",
)

result = agent.invoke(
    {"messages": [{"role": "user", "content": "What's my current balance?"}]},
    config={"configurable": {"thread_id": str(uuid7())}},
    context=UserContext(user_id="user123"),
)

Long-term memory(Store)

The BaseStore provides persistent storage that survives across conversations. Access store through runtime.store. The store uses a namespace/key pattern to organize data

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43


from langgraph.store.memory import InMemoryStore
from langchain.agents import create_agent
from langchain.tools import tool, ToolRuntime
from langchain_openai import ChatOpenAI

# Access memory
@tool
def get_user_info(user_id: str, runtime: ToolRuntime) -> str:
    """Look up user info."""
    store = runtime.store
    user_info = store.get(("users",), user_id)
    return str(user_info.value) if user_info else "Unknown user"

# Update memory
@tool
def save_user_info(user_id: str, name: str, age: int, email: str, runtime: ToolRuntime) -> str:
    """Save user info."""
    store = runtime.store
    store.put(("users",), user_id, {"name": name, "age": age, "email": email})
    return "Successfully saved user info."

model = ChatOpenAI(model="gpt-5.5")

store = InMemoryStore()
agent = create_agent(
    model,
    tools=[get_user_info, save_user_info],
    store=store
)

# First session: save user info
agent.invoke({
    "messages": [{"role": "user", "content": "Save the following user: userid: abc123, name: Foo, age: 25, email: foo@langchain.dev"}]
})

# Second session: get user info
agent.invoke({
    "messages": [{"role": "user", "content": "Get user info for user with id 'abc123'"}]
})
# Here is the user info for user with ID "abc123":
# - Name: Foo
# - Age: 25
# - Email: foo@langchain.dev

Streaming writer

Stream real-time updates from tools during execution. Useful for providing progress feedback Use runtime.stream_Writer to emit custom updates

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


from langchain.tools import tool, ToolRuntime

@tool
def get_weather(city: str, runtime: ToolRuntime) -> str:
    """Get weather for a given city."""
    writer = runtime.stream_writer

    # Stream custom updates as the tool executes
    writer(f"Looking up data for city: {city}")
    writer(f"Acquired data for city: {city}")

    return f"It's always sunny in {city}!"

Execution info

Access thread ID,run ID,and retry state from within a tool via runtime.execution_info

1
2
3
4
5
6
7
8
9


from langchain.tools import tool, ToolRuntime

@tool
def log_execution_context(runtime: ToolRuntime) -> str:
    """Log execution identity information."""
    info = runtime.execution_info
    print(f"Thread: {info.thread_id}, Run: {info.run_id}")
    print(f"Attempt: {info.node_attempt}")
    return "done"

Server info

When your tool runs on LangGraph Server , access assistant ID,graph ID,and authenticated user via runtime.server_info

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


from langchain.tools import tool, ToolRuntime

@tool
def get_assistant_scoped_data(runtime: ToolRuntime) -> str:
    """Fetch data scoped to the current assistant."""
    server = runtime.server_info
    if server is not None:
        print(f"Assistant: {server.assistant_id}, Graph: {server.graph_id}")
        if server.user is not None:
            print(f"User: {server.user.identity}")
    return "done"

server_info is None when the tool is not running on LangGraph Server

Tool execution

Tool: used by agent Tool error handing is configured by middleware

Tool return values

a string for human-readable results;an object for structured results;a Command with optional message when you need to update the state

Return a string

when tool should provide plain text for the model to read and use in its next response

1
2
3
4
5
6
7


from langchain.tools import tool


@tool
def get_weather(city: str) -> str:
    """Get weather for a city."""
    return f"It is currently sunny in {city}."

The return value is converted to a ToolMessage

Return an object

when your tool produces structured data that the model should inspect

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


from langchain.tools import tool


@tool
def get_weather_data(city: str) -> dict:
    """Get structured weather data for a city."""
    return {
        "city": city,
        "temperature_c": 22,
        "conditions": "sunny",
    }

The object is serialized and sent back as tool output.
The model can read specific fields and reason over them.
Like string returns, this does not directly update graph state.

Return multimodal content

For multimodal content ,use standard content blocks so the model receives multimodal resources

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


from langchain.tools import tool


@tool
def capture_screenshot() -> list[dict]:
    """Capture a screenshot of the current page."""
    return [
        {"type": "text", "text": "Screenshot of the current page:"},
        {"type": "image", "url": "https://example.com/page.png"},
    ]

The return value is converted to a ToolMessage with multimodal content.
Use message.content_blocks to read the normalized block list after the tool runs.
The model must support the modalities you return. Check your model’s capabilities before returning images, audio, or video.

Return a Command

when the tool need to update graph state(or example, setting user preferences or app state)

When the Command targets the current graph, include a ToolMessage in the update whose tool call ID matches the current tool call. Every tool call in the message history must have a corresponding ToolMessage. 返回的一个Command的Command.update中必须包含一个对应的ToolMessage.对应是指tool_call_id对应

Use runtime.tool_call_id for the tool_call_id parameter. ToolNode enforces this requirement: if the update has no ToolMessage matching the tool call, it raises a ValueError. 否则会返回一个ValueError

Return directly from a tool

设置return_direct来使得agent-loop短路,这样工具输出不会传回给模型而是直接传给caller

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


from langchain.agents import create_agent
from langchain.tools import tool
from langchain_openai import ChatOpenAI


@tool(return_direct=True)
def fetch_order_status(order_id: str) -> str:
    """Fetch the current status of a customer order."""
    # In production, query your order management system here
    return f"Order {order_id} is shipped and will arrive in 2 days."


agent = create_agent(
    ChatOpenAI(model="gpt-4o-mini"),
    tools=[fetch_order_status],
)

result = agent.invoke({
    "messages": [{"role": "user", "content": "What is the status of order #12345?"}]
})
# The agent returns the tool output directly without another LLM call:
# "Order 12345 is shipped and will arrive in 2 days."

Behavior:

The tool executes normally and its output is wrapped in a ToolMessage.
The agent stops looping and returns the tool’s output as the final response, bypassing any additional model call.
Multiple parallel tool calls: When the model calls several tools in one step, all of them execute first. After all tools finish, the agent routes to END only if every tool in that batch has return_direct=True. The final response includes the ToolMessage outputs of every tool that was called in that step.

Use this when:

The tool’s output is the complete, user-ready answer (for example, a lookup that returns a ready-to-display result).
You want to avoid an extra model call when no additional reasoning is needed.
You need deterministic, unmodified output: the model cannot rephrase, summarize, or act on the tool result.

Return a Command with return_direct

A tool with return_direct=True can also return a Command to update graph state before the agent exits. Unlike a plain return value, a Command is not automatically converted into a ToolMessage. When the Command targets the current graph (graph is not set or is None), include a ToolMessage in Command.update matching the tool call’s tool_call_id. Omitting it causes ToolNode to raise a ValueError, because every AIMessage tool call must have a corresponding ToolMessage in the message history.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21


from langchain.messages import ToolMessage
from langchain.tools import ToolRuntime, tool
from langgraph.types import Command


@tool(return_direct=True)
def fetch_and_store_order(order_id: str, runtime: ToolRuntime) -> Command:
    """Fetch order status and store it in state."""
    status = f"Order {order_id} is shipped and will arrive in 2 days."
    return Command(
        update={
            "last_order_status": status,
            # Must include a ToolMessage so the message history stays valid
            "messages": [
                ToolMessage(
                    content=status,
                    tool_call_id=runtime.tool_call_id,
                )
            ],
        }
    )

Error handing

Using LangChain middleware to retry failed tool calls or return custom error message

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28


from collections.abc import Callable

from langchain.agents import create_agent
from langchain.agents.middleware import wrap_tool_call
from langchain.messages import ToolMessage
from langchain.tools.tool_node import ToolCallRequest


@wrap_tool_call
def handle_tool_errors(
    request: ToolCallRequest,
    handler: Callable[[ToolCallRequest], ToolMessage],
) -> ToolMessage:
    """Convert tool exceptions into ToolMessages the model can handle."""
    try:
        return handler(request)
    except Exception as e:
        return ToolMessage(
            content=f"Tool error: Please check your input and try again. ({e})",
            tool_call_id=request.tool_call["id"],
        )


agent = create_agent(
    model="openai:gpt-5.5",
    tools=[],
    middleware=[handle_tool_errors],
)

State injection

Tool access graph state through ToolRUntime

Dynamic tool section

太多tool会占用过多的context,太少则会限制能力动态工具允许在runtime时修改主要有两种方式实现,取决于工具是否事先已知

Filtering pre-registered tools

When all tools are known at agent creation time,you can pre-register them and dynamically filter which ones are exposed to the model based on state,permissions,or context

1

  代码有三个,自己去官网看

All possible tools are known at compile/startup time
You want to filter based on permissions, feature flags, or conversation state
Tools are static but their availability is dynamic

Runtime tool register

when tools are not discovered or created at runtime

wrap_model_call - Add the dynamic tools to the request
wrap_tool_call - Handle execution of the dynamically added tools

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36


from langchain.tools import tool
from langchain.agents import create_agent
from langchain.agents.middleware import AgentMiddleware, ModelRequest, ToolCallRequest

# A tool that will be added dynamically at runtime
@tool
def calculate_tip(bill_amount: float, tip_percentage: float = 20.0) -> str:
    """Calculate the tip amount for a bill."""
    tip = bill_amount * (tip_percentage / 100)
    return f"Tip: ${tip:.2f}, Total: ${bill_amount + tip:.2f}"

class DynamicToolMiddleware(AgentMiddleware):
    """Middleware that registers and handles dynamic tools."""

    def wrap_model_call(self, request: ModelRequest, handler):
        # Add dynamic tool to the request
        # This could be loaded from an MCP server, database, etc.
        updated = request.override(tools=[*request.tools, calculate_tip])
        return handler(updated)

    def wrap_tool_call(self, request: ToolCallRequest, handler):
        # Handle execution of the dynamic tool
        if request.tool_call["name"] == "calculate_tip":
            return handler(request.override(tool=calculate_tip))
        return handler(request)

agent = create_agent(
    model="gpt-5.5",
    tools=[get_weather],  # Only static tools registered here
    middleware=[DynamicToolMiddleware()],
)

# The agent can now use both get_weather AND calculate_tip
result = agent.invoke({
    "messages": [{"role": "user", "content": "Calculate a 20% tip on $85"}]
})

This approach is best when:

Tools are discovered at runtime (e.g., from an MCP server)
Tools are generated dynamically based on user data or configuration
You’re integrating with external tool registries

Headless tools

某些工具应该运行在用户应用运行的地方（通常是浏览器），而不是在进程内部。无头工具 (Headless tools)是工具定义（包括名称、描述和参数 schema），您在服务端向智能体进行注册。而实现仅在客户端注册，并在简短的中断/恢复（interrupt/resume）握手后执行

when to use

when the work depends on the environment, device, or UI that only exists on the client. For example:

Browser APIs
Privacy and locality
Latency(延迟)
Structured,safe effects

How it work

model can see it and call it but it actually happens outside the process

Define with (name, description,parameter) without in-process implementation
Register it with create agent so the model can call it normally
Handle the interrupt payload when the tool is invoked. Instead of running locally, the graph pauses with a payload shaped like {"type": "tool", "tool_call": {"id", "name", "args"}}.
Resume the graph

Use the optional onTool callback to observe lifecycle events (start, success, error) for UI feedback such as spinners or toasts.

Prebuilt tools

LangChain provides a large collection of prebuilt tools and toolkits for common tasks like web search, code interpretation, database access, and more. These ready-to-use tools can be directly integrated into your agents without writing custom code.See the tools and toolkits integration page for a complete list of available tools organized by category.

Tools from MCP

Model Context Protocol (MCP) is an open protocol that standardizes how applications expose tools to language models. Rather than hand-authoring a tool, you connect to an MCP server and adapt the tools it advertises into LangChain tools, ready to pass to an agent like any other tool.MCPAdapter discovers a server’s tools and converts them into LangChain tools. Open the adapter, call list_tools(), and hand the result to create_agent:

1
2
3
4
5
6


from langchain.agents import create_agent
from langchain.mcp import MCPAdapter

async with MCPAdapter("https://example.com/mcp") as adapter:
    tools = await adapter.list_tools()
    agent = create_agent("claude-sonnet-4-6", tools)

Server-side tool use

Some chat models feature built-in tools that are executed server-side by the model provider. These include capabilities like web search and code interpreters that don’t require you to define or host the tool logic.Refer to the individual chat model integration pages and the tool calling documentation for details on enabling and using these built-in tools.