LangChain on On My Way

9 月 20 日学习报告：LangChain Agent 的 Tool、Memory 与 Streaming

Mon, 01 Jan 0001 00:00:00 +0000

9 月 20 日学习报告：LangChain Agent 的 Tool、Memory 与 Streaming

仅整理 2026-09-20 修改的 4 个文件。文件按最后修改时间排列；各文件内部保持原文标题顺序与层级，只压缩重复表述和示例，不省略知识点。

14:08:52 — Tools.md

Tools

Tools extend what agents can do。

Create tools

Basic tool definition

定义 Tool 最简单的方式是使用 @tool decorator。函数 docstring 会成为 Tool description；参数 type hints 是必须的，因为它们定义 Tool 的输入 Schema。docstring 应内容丰富且简明扼要。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


from langchain.tools import tool

@tool
def search_database(query: str, limit: int = 10) -> str:
    """Search the customer database for records matching the query.

    Args:
        query: Search terms to look for
        limit: Maximum number of results to return
    """
    return f"Found {limit} results for '{query}'"

Customize tool properties

Custom tool name

Tool name 默认等于函数名，可通过 @tool("web_search") 覆盖。

Custom tool description

默认 description 来自 docstring。需要更明确地指导 model 时，使用 description= 覆盖自动生成的 description。

Advanced schema definition

复杂输入使用 Pydantic model 或 JSON Schema。Field(description=...) 描述字段，Literal 限定可选值，@tool(args_schema=...) 绑定输入 Schema。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


from typing import Literal
from pydantic import BaseModel, Field

class WeatherInput(BaseModel):
    location: str = Field(description="City name or coordinates")
    units: Literal["celsius", "fahrenheit"] = Field(
        default="celsius",
        description="Temperature unit preference",
    )
    include_forecast: bool = Field(default=False)

@tool(args_schema=WeatherInput)
def get_weather(
    location: str,
    units: str = "celsius",
    include_forecast: bool = False,
) -> str:
    """Get current weather and optional forecast."""
    ...

Reserved argument names

config 保留给 RunnableConfig；
runtime 保留给 ToolRuntime。

二者不能用作普通业务参数，否则会产生 runtime error。

Access context

Tool 通过 ToolRuntime 访问 conversation history、user data、persistent memory 和执行信息。

Component	访问方式	含义与用途
State	`runtime.state`	当前 conversation 的可变 short-term memory；messages、计数器、自定义字段
Context	`runtime.context`	invocation 时传入的不可变配置；user ID、session info
Store	`runtime.store`	跨 conversation 持久化的 long-term memory
Stream Writer	`runtime.stream_writer`	Tool 执行时发送实时 custom updates
Execution Info	`runtime.execution_info`	thread ID、run ID、attempt number
Server Info	`runtime.server_info`	assistant ID、graph ID、authenticated user
Config	`runtime.config`	`RunnableConfig` 的 callbacks、tags、metadata
Tool Call ID	`runtime.tool_call_id`	当前 Tool invocation ID，用于匹配 `ToolMessage`

Short-term memory(State)

State 包含 message history 和 graph state 中定义的自定义字段。

Access state

在 Tool 签名中声明 runtime: ToolRuntime。ToolNode 会自动注入它，但它不会出现在发送给 model 的 Tool Schema 中。Tool 通过 runtime.state 读取 State。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


from langchain.messages import HumanMessage
from langchain.tools import ToolRuntime, tool

@tool
def get_last_user_message(runtime: ToolRuntime) -> str:
    """Get the most recent message from the user."""
    for message in reversed(runtime.state["messages"]):
        if isinstance(message, HumanMessage):
            return message.content
    return "No user messages found"

@tool
def get_user_preference(pref_name: str, runtime: ToolRuntime) -> str:
    """Get a user preference value."""
    return runtime.state.get("user_preferences", {}).get(pref_name, "Not set")

第二个 Tool 的 model-visible Schema 只有 pref_name，没有 runtime。

Update state

Tool 更新 State 时返回 Command。当目标是当前 graph 时：

在 Command.update 写入 State 字段；
在 Command.update["messages"] 中加入 ToolMessage；
结果写入 ToolMessage.content；
ToolMessage.tool_call_id 必须等于 runtime.tool_call_id。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25


from langchain.agents import AgentState
from langchain.messages import ToolMessage
from langchain.tools import ToolRuntime, tool
from langgraph.types import Command

class CustomState(AgentState):
    user_name: str

@tool
def set_user_name(
    new_name: str,
    runtime: ToolRuntime[None, CustomState],
) -> Command:
    """Set the user's name in the conversation state."""
    return Command(
        update={
            "user_name": new_name,
            "messages": [
                ToolMessage(
                    content=f"User name set to {new_name}.",
                    tool_call_id=runtime.tool_call_id,
                )
            ],
        }
    )

每个 AIMessage tool call 都必须有对应的 ToolMessage。消息缺失或 ID 不匹配时，ToolNode 抛出 ValueError。多个 Tool 并行更新同一 State 字段时，应定义 reducer，明确覆盖、追加或其他合并规则。

Context

Context 是 invocation 时传入的不可变配置，包括 user ID、session details 和 application settings。使用 context_schema 声明结构，通过 context= 传入，Tool 从 runtime.context 读取。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


from dataclasses import dataclass

@dataclass
class UserContext:
    user_id: str

@tool
def get_account_info(runtime: ToolRuntime[UserContext]) -> str:
    """Get the current user's account information."""
    return str(USER_DATABASE.get(runtime.context.user_id, "User not found"))

agent = create_agent(
    model="openai:gpt-5.5",
    tools=[get_account_info],
    context_schema=UserContext,
)

result = agent.invoke(
    {"messages": [{"role": "user", "content": "What's my balance?"}]},
    config={"configurable": {"thread_id": "thread-1"}},
    context=UserContext(user_id="user123"),
)

thread_id 标识 conversation；user_id 标识业务用户，二者不同。

Long-term memory(Store)

BaseStore 提供跨 conversation 持久化的数据。使用 runtime.store 访问，以 namespace/key 组织记录。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


@tool
def get_user_info(user_id: str, runtime: ToolRuntime) -> str:
    """Look up user info."""
    item = runtime.store.get(("users",), user_id)
    return str(item.value) if item else "Unknown user"

@tool
def save_user_info(
    user_id: str,
    name: str,
    age: int,
    email: str,
    runtime: ToolRuntime,
) -> str:
    """Save user info."""
    runtime.store.put(
        ("users",), user_id, {"name": name, "age": age, "email": email}
    )
    return "Successfully saved user info."

把同一 Store 传给 create_agent(store=store) 后，可在一个 session 保存、另一个 session 读取。

Streaming writer

runtime.stream_writer 可在 Tool 执行中发送实时 custom updates，适合长任务进度；Tool 最终仍用 return 返回完成结果。

1
2
3
4
5
6


@tool
def get_weather(city: str, runtime: ToolRuntime) -> str:
    """Get weather for a given city."""
    runtime.stream_writer(f"Looking up data for city: {city}")
    runtime.stream_writer(f"Acquired data for city: {city}")
    return f"It's always sunny in {city}!"

Execution info

runtime.execution_info 提供 thread_id、run_id 和 node_attempt，用于日志关联和重试判断。

Server info

在 LangGraph Server 上，runtime.server_info 提供 assistant_id、graph_id 和 authenticated user；非 Server 环境下为 None。

Tool execution

Tool 由 Agent 使用；Tool error handling 由 middleware 配置。

Tool return values

Return a string

返回 human-readable str，供 model 下一步读取；框架自动将其转换为 ToolMessage。

Return an object

返回结构化 object，供 model 读取字段并推理。对象会序列化为 Tool output，但不会直接更新 graph State。

Return multimodal content

多模态结果使用 standard content blocks，例如 text block 和 image block。结果转换成带 multimodal content 的 ToolMessage；可用 message.content_blocks 读取归一化 blocks。model 必须支持所返回的 image、audio 或 video modality。

Return a Command

需要更新 graph State 时返回 Command。若目标是当前 graph，必须显式提供与当前 tool call ID 匹配的 ToolMessage。普通 str/object 会自动包装为 ToolMessage，Command 不会。

Return directly from a tool

@tool(return_direct=True) 使 agent loop 短路：Tool 正常执行，输出包装成 ToolMessage 后直接返回 caller，不再追加 model call。

适用于：Tool output 已是完整答案；希望避免额外 model call；需要 deterministic、unmodified output。

同一步包含多个并行 tool calls 时，会先执行整个批次；仅当每个 Tool 都设置 return_direct=True 时才路由到 END，最终响应包含全部 Tool outputs。

Return a Command with return_direct

return_direct=True 的 Tool 可返回 Command，在退出前更新 State。Command 仍不会自动生成 ToolMessage，必须手动加入 tool_call_id=runtime.tool_call_id 的消息。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


@tool(return_direct=True)
def fetch_and_store_order(order_id: str, runtime: ToolRuntime) -> Command:
    """Fetch order status and store it in state."""
    status = f"Order {order_id} is shipped."
    return Command(
        update={
            "last_order_status": status,
            "messages": [
                ToolMessage(content=status, tool_call_id=runtime.tool_call_id)
            ],
        }
    )

Error handing

使用 middleware 重试失败 Tool，或把异常转换为 model 可处理的自定义 ToolMessage。异常结果的 tool_call_id 必须取原调用的 request.tool_call["id"]。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


from langchain.agents.middleware import wrap_tool_call
from langchain.messages import ToolMessage

@wrap_tool_call
def handle_tool_errors(request, handler) -> ToolMessage:
    try:
        return handler(request)
    except Exception as error:
        return ToolMessage(
            content=f"Tool error: Please check your input. ({error})",
            tool_call_id=request.tool_call["id"],
        )

State injection

Tool 通过框架注入且对 model 隐藏的 ToolRuntime 访问 State：读取用 runtime.state，更新用带匹配 ToolMessage 的 Command。

Dynamic tool section

Tool 太多会占用 context，太少会限制能力。动态 Tool 根据启动时是否已知分为两类。

Filtering pre-registered tools

全部 Tool 在创建 Agent 时已知：先预注册，再依据 State、permissions 或 Context 动态过滤暴露给 model 的集合。适合 Tool 静态、availability 动态的场景。

Runtime tool register

Tool 在 runtime 才发现或创建时，需要同时处理：

wrap_model_call：用 request.override(tools=[...]) 加入本次 model request；
wrap_tool_call：识别对应 name，用 request.override(tool=...) 提供执行对象。

只让 model 看见而不提供执行处理器，runtime 无法完成调用。此方式适合 MCP server、database、user configuration 和 external registry。

Headless tools

服务端只注册 name、description 和 parameter Schema，实现只在客户端。model 可正常调用，但执行通过 interrupt/resume 在进程外完成。

when to use

用于依赖客户端 environment、device 或 UI 的工作：Browser APIs、privacy/locality、latency、structured safe effects。

How it work

定义 Tool Schema，不提供 in-process implementation；
将定义注册给 create_agent；
调用时 graph 暂停，返回 {"type":"tool","tool_call":{"id","name","args"}}；
客户端执行，再 resume graph。

onTool callback 可观察 start、success、error，用于 spinner 或 toast。

Prebuilt tools

LangChain 提供 web search、code interpretation、database access 等 prebuilt tools/toolkits，可直接集成，无需自行实现。

Tools from MCP

MCP 统一应用向 model 暴露 Tools 的方式。MCPAdapter 发现 MCP server 的 Tools，将 list_tools() 结果转换为 LangChain Tools，再传给 create_agent。

Server-side tool use

部分 chat models 提供由 provider 执行的 built-in Tools，例如 web search 和 code interpreter，无需自行定义或托管 Tool logic；具体配置取决于 model integration。

15:38:37 — Short-term_memory.md

Conversation history 是最常见的 short-term memory；Message 包含 instructions 和 inputs。

Usage

为 Agent 添加 short-term memory，需要在 create_agent 时指定 checkpointer。Short-term memory 是 Agent State 的一部分；checkpointer 按 thread 持久化 State，使 conversation 可恢复。

State 在 Agent invocation 或 tool call 等 step 完成时更新，在每个 step 开始时读取。相同 thread_id 恢复相同 conversation；不同 thread_id 使用独立 State。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18


from langchain.agents import create_agent
from langgraph.checkpoint.memory import InMemorySaver

agent = create_agent(
    model="openai:gpt-5.5",
    tools=[get_user_info],
    checkpointer=InMemorySaver(),
)

config = {"configurable": {"thread_id": "1"}}
agent.invoke(
    {"messages": [{"role": "user", "content": "Hi! My name is Bob."}]},
    config,
)
result = agent.invoke(
    {"messages": [{"role": "user", "content": "What's my name?"}]},
    config,
)

In production

生产环境使用 database-backed checkpointer；InMemorySaver 只适合本地示例和测试。

Customizing agent memory

Agent 使用 AgentState 管理 short-term memory，其中 messages key 保存 conversation history。需要额外字段时继承 AgentState，通过 state_schema= 传给 create_agent。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21


from langchain.agents import AgentState

class CustomAgentState(AgentState):
    user_id: str
    preferences: dict

agent = create_agent(
    model="openai:gpt-5.5",
    tools=[get_user_info],
    state_schema=CustomAgentState,
    checkpointer=InMemorySaver(),
)

result = agent.invoke(
    {
        "messages": [{"role": "user", "content": "Hello"}],
        "user_id": "user_123",
        "preferences": {"theme": "dark"},
    },
    {"configurable": {"thread_id": "1"}},
)

state_schema 定义 State 结构；checkpointer 负责按 thread 保存它。

Common patterns

以下模式用于处理 conversation history 超过 LLM context window。

Trim message(截断消息)

接近 token limit 时，根据 token count 截断，并指定 boundary strategy。也可在 @before_model 中清空旧 messages，再写回首条与最近 messages。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20


from langchain.agents.middleware import before_model
from langchain.messages import RemoveMessage
from langgraph.graph.message import REMOVE_ALL_MESSAGES

@before_model
def trim_messages(state, runtime) -> dict | None:
    """Keep only the first and most recent messages."""
    messages = state["messages"]
    if len(messages) <= 3:
        return None

    first_message = messages[0]
    recent_messages = messages[-3:] if len(messages) % 2 == 0 else messages[-4:]
    return {
        "messages": [
            RemoveMessage(id=REMOVE_ALL_MESSAGES),
            first_message,
            *recent_messages,
        ]
    }

None 表示不更新 State。选择最近 3 或 4 条是为了尽量维持有效的 request/response 边界。

Delete message

使用 RemoveMessage 删除指定 message 或清空全部 history。Messages State key 必须使用 add_messages reducer；默认 AgentState 已提供。

To remove specific messages:

1
2
3
4
5
6
7
8


from langchain.messages import RemoveMessage

def delete_messages(state):
    messages = state["messages"]
    if len(messages) > 2:
        return {
            "messages": [RemoveMessage(id=message.id) for message in messages[:2]]
        }

To remove all messages:

1
2
3
4
5


from langchain.messages import RemoveMessage
from langgraph.graph.message import REMOVE_ALL_MESSAGES

def delete_messages(state):
    return {"messages": [RemoveMessage(id=REMOVE_ALL_MESSAGES)]}

删除后必须保持有效消息历史：部分 provider 要求以 user message 开始；多数 provider 要求带 tool call 的 assistant message 后紧跟对应的 tool result，不能只删除配对的一侧。

Summarize messages

Trim 和 Delete 会丢失信息。需要保留旧语义时，使用内置 SummarizationMiddleware。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


from langchain.agents.middleware import SummarizationMiddleware

agent = create_agent(
    model="openai:gpt-5.5",
    tools=[...],
    middleware=[
        SummarizationMiddleware(
            model="openai:gpt-5.4-mini",
            trigger=("tokens", 4000),
            keep=("messages", 20),
        )
    ],
    checkpointer=checkpointer,
)

示例在 4000 tokens 时触发摘要，保留最近 20 条 messages。

Access memory

tools

Read short-term memory in a tool

Tool 使用对 model 隐藏的 runtime: ToolRuntime，通过 runtime.state 读取 short-term memory。

1
2
3
4
5


@tool
def get_user_info(runtime: ToolRuntime) -> str:
    """Look up user info."""
    user_id = runtime.state["user_id"]
    return "User is John Smith" if user_id == "user_123" else "Unknown user"

write short-term memory form tools

Tool 写 short-term memory 时返回 Command，并在 Command.update["messages"] 中加入 tool_call_id=runtime.tool_call_id 的 ToolMessage。

Prompt

使用 middleware 根据 conversation history 或 custom State fields 创建 dynamic prompt。@dynamic_prompt 可通过 request.runtime.context 读取 invocation Context。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


from typing import TypedDict
from langchain.agents.middleware import ModelRequest, dynamic_prompt

class CustomContext(TypedDict):
    user_name: str

@dynamic_prompt
def dynamic_system_prompt(request: ModelRequest) -> str:
    user_name = request.runtime.context["user_name"]
    return f"You are a helpful assistant. Address the user as {user_name}."

创建 Agent 时传入 context_schema=CustomContext，调用时通过 context=CustomContext(...) 提供值。

Before model

@before_model 在每次 model call 前处理 State，适合 trim、删除、注入或重排 messages。返回 None 表示无修改，返回 dict 表示要合并进 State 的更新。

After model

@after_model 在 model call 后检查 State。可读取最后一条 response，用 RemoveMessage(id=last_message.id) 删除不符合要求的输出。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


from langchain.agents.middleware import after_model
from langchain.messages import RemoveMessage

@after_model
def validate_response(state, runtime) -> dict | None:
    """Remove messages containing sensitive words."""
    stop_words = ["password", "secret"]
    last_message = state["messages"][-1]
    if any(word in last_message.content for word in stop_words):
        return {"messages": [RemoveMessage(id=last_message.id)]}
    return None

该 hook 修改最终 State；对于实时 Stream，只在 after_model 删除可能太晚，流式输出需在离开 run 前过滤。

16:27:01 — Event_Streaming.md

LangChain Agents 构建在 LangGraph 上，因此支持相同的 streaming stack，并提供 messages、tool calls、State 和 custom updates 等 Agent-focused projections。

多数场景使用 stream_events(..., version="v3")。它返回带 typed projections 的 run object，各 projection 可独立消费，无需解析 stream-mode tuples。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


stream = agent.stream_events(
    {"messages": [{"role": "user", "content": "What is the weather in SF?"}]},
    version="v3",
)

for message in stream.messages:
    for delta in message.text:
        print(delta, end="", flush=True)

final_state = stream.output

What you can stream

Projection	内容
`for event in stream`	带完整 envelope、可访问全部 channel 的 raw protocol events
`stream.messages`	每次 LLM call 的 model message streams
`message.text`	Text deltas 与 final text
`message.reasoning`	Model 支持时的 reasoning deltas
`message.tool_calls`	Tool-call argument chunks 与 finalized tool calls
`message.output`	本次 model call 的 final message object
`stream.values`	Agent State snapshots
`stream.output`	Final Agent State
`stream.subgraphs`	Nested graph runs，包括 sub-agents 和普通 subgraphs
`stream.extensions`	Custom transformer projections
`stream.tool_calls`	Tool lifecycle、inputs、output deltas、final output、errors

stream.messages 产生 ChatModelStream。每个 message stream 暴露 .text、.reasoning、.tool_calls、.output。同步 projection 可实时迭代，也可排空后读取最终值：str(message.text) 得到最终文本，message.tool_calls.get() 得到完整 Tool calls。

Agent messages

需要每次 LLM call 的 model output 时使用 stream.messages。message.output 是 finalized AI message，包含 provider-specific content blocks；token usage 位于 message.output.usage_metadata。

1
2
3
4
5
6


for message in stream.messages:
    print(f"[{message.node}] ", end="")
    for delta in message.text:
        print(delta, end="", flush=True)
    if message.output.usage_metadata:
        print(message.output.usage_metadata)

Reasoning content

message.reasoning 与 .text 用法相同，但只有 model 发出 reasoning blocks 时才有内容。

1
2
3
4
5


for message in stream.messages:
    for delta in message.reasoning:
        print(f"[thinking] {delta}", end="", flush=True)
    for delta in message.text:
        print(delta, end="", flush=True)

Tool calls

Tool calls 有两种 projection：

message.tool_calls：model 生成 Tool call 时，流式输出参数 chunks 和 finalized call；
stream.tool_calls：Tool 开始执行后，流式输出 lifecycle、input、output deltas、final output、error。

前者观察 model 正在构造什么调用，后者观察 Tool 实际如何执行。

Streaming sub-agent

当一个 create_agent 调用另一个已命名的 create_agent 时，内部 Agent 事件位于嵌套 namespace。传给 create_agent 的 name= 用于标识内部 Agent。

命名 sub-agent 出现在 stream.subagents projection；每个 handle 暴露 .messages、.values、.tool_calls、.output，以及 .name、.cause。

1
2
3
4
5


for subagent in stream.subagents:
    print(f"{subagent.name}: ", end="")
    for message in subagent.messages:
        for token in message.text:
            print(token, end="", flush=True)

state and final output

stream.values 获取运行过程中的 State snapshots；stream.output 获取最终 Agent State。

1
2
3
4


for snapshot in stream.values:
    print(snapshot)

final_state = stream.output

message.output 是单次 model call 的最终 AI message；stream.output 是整个 Agent run 的最终 State。

Multiple projections

Async code 使用 astream_events 与 asyncio.gather 并发消费多个 projections。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


import asyncio

stream = await agent.astream_events(input, version="v3")

async def consume_messages():
    async for message in stream.messages:
        print(await message.text)

async def consume_tool_calls():
    async for call in stream.tool_calls:
        print(call.tool_name, call.input)

await asyncio.gather(consume_messages(), consume_tool_calls())

Synchronous code 使用 stream.interleave(...)，返回 (name, item) 区分 projection。

1
2
3
4
5
6
7


for name, item in stream.interleave("messages", "tool_calls", "values"):
    if name == "messages":
        print(item.text)
    elif name == "tool_calls":
        print(item.tool_name, item.input)
    elif name == "values":
        print(item)

Custom updates

内置 projection 不够时，使用 custom stream transformer，并从 stream.extensions 读取扩展 projection。

1
2
3
4
5
6
7
8


stream = agent.stream_events(
    input,
    version="v3",
    transformers=[ToolActivityTransformer],
)

for activity in stream.extensions["tool_activity"]:
    print(activity)

Register transformers on middleware

Middleware 可在 hooks 和 Tools 旁声明 transformer factories。将 AgentMiddleware.transformers 设置为 factories 序列；每个 factory 形如 Callable[[tuple[str, ...]], StreamTransformer]，框架调用 factory(scope)。

Root mux 的 scope 为 ()，subgraph 为非空 tuple。每次返回新的 transformer，使各 subgraph 隔离。

1
2
3
4


from langchain.agents.middleware import AgentMiddleware

class ToolActivityMiddleware(AgentMiddleware):
    transformers = (ToolActivityTransformer,)

create_agent 编译时合并 middleware factories 与自身 transformers=，顺序为：

内置 ToolCallTransformer；
Middleware factories，按 middleware 顺序；
调用方传给 create_agent(transformers=...) 的 factories。

这让内置 Tool call projection 位于消费者 transformer 之前，并让调用方条目拥有最终处理权。

PIIMiddleware(..., apply_to_output=True) 会在 text deltas、tool call arguments、tool outputs、State snapshots 离开 run 前清除 PII。

1
2
3
4
5
6
7


agent = create_agent(
    model="openai:gpt-5-nano",
    tools=[],
    middleware=[
        PIIMiddleware("email", strategy="redact", apply_to_output=True),
    ],
)

若只依赖 after_model 做 State-level filtering，原始 PII 可能已通过 stream_events(version="v3") 实时发送给 reader。

16:52:27 — Streaming.md

Overview

Streaming 支持：

Stream Agent progress；
Stream LLM tokens；
Streaming thinking/reasoning tokens；
Stream custom updates；
从 updates、messages、custom 中选择一个或多个 modes。

Supported stream modes

将一个 mode 或多个 modes 的 list 传给 stream / astream。

Mode	Description
`updates`	每个 Agent step 后发送 State updates；同一步运行多个 nodes 时分别发送
`messages`	从所有调用 LLM 的 graph nodes 发送 `(token, metadata)` tuples
`custom`	发送 graph nodes 内通过 stream writer 写出的 custom data

Agent progress

使用 stream_mode="updates" 的 stream 或 astream，在每个 Agent step 后接收事件。一次 Tool call 的典型顺序是：

LLM node：输出带 Tool call requests 的 AIMessage；
Tool node：输出带 execution result 的 ToolMessage；
LLM node：输出 final AI response。

LangChain MCP Overview

Mon, 01 Jan 0001 00:00:00 +0000

Model Context Protocol (MCP) is an open protocol that standardizes how applications provide tools and context to language models.

Event Streaming

Mon, 01 Jan 0001 00:00:00 +0000

LangChain agents are built on LangGraph, so they support the same streaming stack with agent-focused projections for messages, tool calls, state, and custom updates.

For most situation,use Event Streaming throughstream_event(...,version="v3") Event Streaming returns a run object with typed projections, so each projection can be consumed independently instead of parsing stream-mode tuples

1
2
3
4
5
6
7
8
9


stream = agent.stream_events({
    "messages": [{"role": "user", "content": "What is the weather in SF?"}],
}, version="v3")

for message in stream.messages:
    for delta in message.text:
        print(delta, end="", flush=True)

final_state = stream.output

What you can stream

Projection	Use
`for event in stream`	Raw protocol events with full envelope and access to every channel.
`stream.messages`	Model message streams, one per LLM call.
`message.text`	Text deltas and final text for a message.
`message.reasoning`	Reasoning deltas for models that expose reasoning content.
`message.tool_calls`	Tool-call argument chunks and finalized tool calls.
`message.output`	Final message object after the model call completes.
`stream.values`	Agent state snapshots.
`stream.output`	Final agent state.
`stream.subgraphs`	Nested graph runs (sub-agents and plain subgraphs).
`stream.extensions`	Custom transformer projections.
`stream.tool_calls`	Tool execution lifecycle, inputs, output deltas, final output, and errors.

stream.messages 会生成 ChatModelStream 对象。每个消息流都暴露了 .text、.reasoning、.tool_calls 和 .output。同步投影可迭代以获取实时增量，也可排空以获取最终值：使用 str(message.text) 获取最终文本，使用 message.tool_calls.get() 获取已完成的工具调用

Agent messages

Use stream.message when you want model output from each LLM call

message.output gives you the finalized AI message,including previder-specific content blocks read usage from message.output.usage_metadata.(If you want token counts)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


stream = agent.stream_events(input, version="v3")

for message in stream.messages:
    print(f"[{message.node}] ", end="")
    for delta in message.text:
        print(delta, end="", flush=True)

    full_message = message.output
    usage = full_message.usage_metadata
    if usage:
        print(usage)

Reasoning content

the same shape as text ,but only available when the model emits reasoning blocks

1
2
3
4
5
6
7
8


stream = agent.stream_events(input, version="v3")

for message in stream.messages:
    for delta in message.reasoning:
        print(f"[thinking] {delta}", end="", flush=True)

    for delta in message.text:
        print(delta, end="", flush=True)

将上面的message.text更换为message.reasoning

Tool calls

有两种投影

message.tool_calls在模型生成工具调用的时候,会流式传输工具调用参数块
stream.tool_calls会在工具调用开始后流式传输工具执行的生命周期

1
2
3
4
5
6
7
8


stream = agent.stream_events(input, version="v3")

for message in stream.messages:
    for delta in message.reasoning:
        print(f"[thinking] {delta}", end="", flush=True)

    for delta in message.text:
        print(delta, end="", flush=True)

Streaming sub-agent

当一个create_agent调用另一个已命名的create_agent时,内部智能体的时间会在一个内嵌的namespace中流动,你传递给create_agent的name= 会在stream中标识该内部智能体

命名的子智能体会在主智能体的stream.subagents投影上呈现,而每个句柄都暴露出该内部智能体的.message,.value,.tool_calls和.output加上你传递的name和.cause

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


upervisor = create_agent(
    model=init_chat_model("openai:gpt-5.5"),
    tools=[call_weather],
    name="supervisor",
)

stream = supervisor.stream_events(
    {"messages": [{"role": "user", "content": "What's the weather in Boston?"}]},
    version="v3",
)

for subagent in stream.subagents:
    print(f"{subagent.name}: ", end="")
    for message in subagent.messages:
        for token in message.text:
            print(token, end="", flush=True)
    print()

state and final output

用stream.value来获取state的快照用stream.output来获取最终的agent state

1
2
3
4
5
6


stream = agent.stream_events(input, version="v3")

for snapshot in stream.values:
    print(snapshot)

final_state = stream.output

Multiple projections

对于 async code中的并发消费,请使用 astream_events with asyncio.gather 前者创建异步事件流,后者并发执行多个事件并且根据返回顺序聚合他们的结果

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


import asyncio

stream = await agent.astream_events(input, version="v3")

async def consume_messages():
    async for message in stream.messages:
        print(await message.text)

async def consume_tool_calls():
    async for call in stream.tool_calls:
        print(call.tool_name, call.input)

await asyncio.gather(consume_messages(), consume_tool_calls())

对于synchronous code,使用stream.interleave(…)

1
2
3
4
5
6
7
8
9


stream = agent.stream_events(input, version="v3")

for name, item in stream.interleave("messages", "tool_calls", "values"):
    if name == "messages":
        print(item.text)
    elif name == "tool_calls":
        print(item.tool_name, item.input)
    elif name == "values":
        print(item)

Custom updates

使用custom stream transformer当你需要内置未提供的投影

1
2
3
4
5
6
7
8


stream = agent.stream_events(
    input,
    version="v3",
    transformers=[ToolActivityTransformer],
)

for activity in stream.extensions["tool_activity"]:
    print(activity)

Register transformers on middleware

中间件可以在其钩子和工具旁边声明流转换器工厂。工厂的形态在不同语言中有所不同：将 AgentMiddleware 子类上的 transformers 属性设置为一系列工厂。每个工厂的形态为 Callable[[tuple[str, ...]], StreamTransformer]，调用方式为 factory(scope)，其中 scope 是 mini-mux 作用域元组（根 mux 为 ()，子图则为非空）。每次调用返回一个新的转换器可使每个子图保持隔离。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


from langchain.agents import create_agent
from langchain.agents.middleware import AgentMiddleware


class ToolActivityMiddleware(AgentMiddleware):
    transformers = (ToolActivityTransformer,)


agent = create_agent(
    model="gpt-5-nano",
    tools=[get_weather],
    middleware=[ToolActivityMiddleware()],
)

在编译时，create_agent 会将中间件注册的工厂与传递给其自身的 transformers= 参数的内容进行合并。编译后图上的最终顺序为：

内置的 ToolCallTransformer。
中间件注册的工厂，按中间件顺序排列。
由调用方通过 create_agent 提供的 transformers=。

这使得内置的工具调用投影排在消费者转换器之前，并让调用方提供的条目拥有最终决定权。内置的 PIIMiddleware 使用此钩子从流式传输的线路输出中过滤 PII。通过设置 apply_to_output=True，其注册的转换器会在文本增量、工具调用参数、工具输出和状态快照离开运行之前，清除其中检测到的 PII，从而关闭了一个窗口——在该窗口中，after_model 状态级别的过滤原本会让原始 PII 透传给 stream_events(version="v3") 的实时读取者。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


from langchain.agents import create_agent
from langchain.agents.middleware import PIIMiddleware

agent = create_agent(
    model="gpt-5-nano",
    tools=[],
    middleware=[
        PIIMiddleware("email", strategy="redact", apply_to_output=True),
    ],
)

Context Engineering

Mon, 01 Jan 0001 00:00:00 +0000

Overview

构建agent的难点在于使他们足够可靠即时他们在原型阶段往往会表现得很好

Why do agent fails

LLM调用采取了错误的措施或者没有按照预期执行,原因:

LLM能力不够
正确的context没有传递给LLM

更多情况是后者

The agent loop

two main step2:

Model call: calls the LLM with a prompt and available tools, returns either a response or a request to execute tools
Tool execution: execute the tools that LLM requested,returns tool results

What can you control:

Context Type	What You Control	Transient or Persistent
Model Context	What goes into model calls (instructions, message history, tools, response format)	Transient
Tool Context	What tools can access and produce (reads/writes to state, store, runtime context)	Persistent
Life-cycle Context	What happens between model and tool calls (summarization, guardrails, logging, etc.)	Persistent

Data sources

Data Source	Also Known As	Scope	Examples
Runtime Context	Static configuration	Conversation-scoped	User ID, API keys, database connections, permissions, environment settings
State	Short-term memory	Conversation-scoped	Current messages, uploaded files, authentication status, tool results
Store	Long-term memory	Cross-conversation	User preferences, extracted insights, memories, historical data

How it works

Using middleware:

update context
Jump to a different step in the agent lifecycle

Model context

Instructions,available tools,which model to use,output format.

System prompt
Messages:conversation history
Tools
Model
Response format

System prompt

Successful agents draw on memories, preferences, and configuration to provide the right instructions for the current state of the conversation.

Access message count or conversation context from state
Access user preferences from long-term memory
Access user ID or configuration from Runtime Context

Message

Make up the prompt that is sent to the LLM

分为state,store,Runtime context

Tools

Defining tools

name,description(docstring),argument name, argument description

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20


from langchain.tools import tool

@tool(parse_docstring=True)
def search_orders(
    user_id: str,
    status: str,
    limit: int = 10
) -> str:
    """Search for user orders by status.

    Use this when the user asks about order history or wants to check
    order status. Always filter by the provided status.

    Args:
        user_id: Unique identifier for the user
        status: Order status: 'pending', 'shipped', or 'delivered'
        limit: Maximum number of results to return
    """
    # Implementation here
    pass

Selecting tools

从分为从三个数据源来组织上下文

Model

根据三种数据源中的各种数据来选择合适的模型

Response format

defining formats

Schema definitions guide the model. Field names, types, and descriptions specify exactly what format the output should adhere to.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


from pydantic import BaseModel, Field

class CustomerSupportTicket(BaseModel):
    """Structured ticket information extracted from customer message."""

    category: str = Field(
        description="Issue category: 'billing', 'technical', 'account', or 'product'"
    )
    priority: str = Field(
        description="Urgency level: 'low', 'medium', 'high', or 'critical'"
    )
    summary: str = Field(
        description="One-sentence summary of the customer's issue"
    )
    customer_sentiment: str = Field(
        description="Customer's emotional tone: 'frustrated', 'neutral', or 'satisfied'"
    )

selecting format

Tool context

Both read and write context

Reads

大多数的工具不只是需要LLM传入的参数,需要从其他的数据源获得,主要是state,store,Runtime context

Writes

update the memory

Write to State to track session-specific information using Command:
Write to Store to persist data across sessions:

Life-cycle context

控制核心agent步骤之间发生的情况:拦截数据流以实现总结,安全护栏,日志记录等横切关注点

使用middleware:

update context:修改state和存储以持久化更改,更新对话历史或者保存insights
Jump in the lifecycle:根据上下文移动到Agent loop中的不同步骤(例如如果满足条件就跳过工具执行,或使用修改后的上下文重复model call)

Example: Summarization

automatically condensing conversation history

LangChain offers built-in middleware for this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


from langchain.agents import create_agent
from langchain.agents.middleware import SummarizationMiddleware

agent = create_agent(
    model="gpt-5.5",
    tools=[...],
    middleware=[
        SummarizationMiddleware(
            model="gpt-5.4-mini",
            trigger={"tokens": 4000},
            keep=("messages", 20),
        ),
    ],
)

When the conversation exceeds the token limit, SummarizationMiddleware automatically:

Summarizes older messages using a separate LLM call
Replaces them with a summary message in State (permanently)
Keeps recent messages intact for context

LangChain A Quick Start

Mon, 01 Jan 0001 00:00:00 +0000

一个简单的示例

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


from langchain.agents import create_agent

def get_weather(city: str) -> str:
    """Get weather for a given city."""
    return f"It's always sunny in {city}!"

agent = create_agent(
    model="claude-sonnet-4-6",
    tools=[get_weather],
    system_prompt="You are a helpful assistant",
)

result = agent.invoke(
    {"messages": [{"role": "user", "content": "What's the weather in San Francisco?"}]}
)
print(result["messages"][-1].content_blocks)

Build a real-world agent:

Detailed System Prompt
Create Tools
Model Configuration
Conversational memory
Deep agents
Testing

1.Define the system prompt

defines your agent’s role and behavior. Keep it specific and actionable.

2.Craete tools

Tools let a model interact with external systems by calling functions you define. Tools can depend on runtime context and also interact with agent memory.

Tools should be well-documented: their name,description,parameters will become part of the model’s prompt. LangChain’s tool decorator adds metadata and enables runtime injection with the ToolRunTime paramete.

3.Configure your model

Set up your language model with the right parameters foe your use case

4.Add memory

Add memory to your agent to maintain state across interactions. This allows the agent to remember previous conversations and context.

5.Create and run the agent

Tow different Frameworks for creating agents: LangChain agents and deep agents Use deep agents when you want maximum with minimal setup; Choose LangChain agents when you need fine-grained contorl.

6.Review the results

相较于LangChain, deep agent拥有更多定义好的功能:

使用内置的 write_todos 工具规划其方法，以分解研究任务。
通过调用 fetch_text_from_url 工具加载文件以收集信息。
通过使用文件系统工具（grep 和 read_file）来管理上下文。
生成子智能体：根据需要将复杂的子任务委派给专门的子智能体。

For LangChain agents, you must implement more capabilities to get a similar lever of service and can customize them along the way as needed

In short,deep agent is well-organized and provides many integrated tools and LangChain can fit custom needs better, but requires implementing more functionalities.

Trace agent calls

Use LangSmith to trace LLM calls and inspect what is going on inside your agent.

Emmm, you should sign up for a LangSmith account first. Besides,you can also set up LangSmith Engine which monitors your traces,detects issues ,and preposes fixes.

Custom Middleware

Mon, 01 Jan 0001 00:00:00 +0000

通过在agent工作流中实现hook来构建自定义的midddleware

Hooks

two types:

Node-style hook

在特定的执行点顺序执行.Use for validation,logging and state updates.

Hook	When it runs
`before_agent`	Before agent starts (once per invocation)
`before_model`	Before each model call
`after_model`	After each model response
`after_agent`	After agent completes (once per invocation)

Wrap-style hooks run around each call, giving you control over execution:

Hook	When it runs
`wrap_model_call`	Around each model call
`wrap_tool_call`	Around each tool call

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


from langchain.agents.middleware import before_model, after_model, AgentState
from langchain.messages import AIMessage
from langgraph.runtime import Runtime
from typing import Any


@before_model(can_jump_to=["end"])
def check_message_limit(state: AgentState, runtime: Runtime) -> dict[str, Any] | None:
    if len(state["messages"]) >= 50:
        return {
            "messages": [AIMessage("Conversation limit reached.")],
            "jump_to": "end"
        }
    return None

@after_model
def log_response(state: AgentState, runtime: Runtime) -> dict[str, Any] | None:
    print(f"Model returned: {state['messages'][-1].content}")
    return None

Warp-style hooks

拦截执行并且控制出处理机制的调用时机.Use for retries,caching,and information. 由你决定处理机制是被调用zero times(short-circuit短路),once(normal flow), or multiple times(retry logic)

Available hooks:

wrap_model_call - Around each model call
wrap_tool_call - Around each tool call

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


from langchain.agents.middleware import wrap_model_call, ModelRequest, ModelResponse
from typing import Callable


@wrap_model_call
def retry_model(
    request: ModelRequest,
    handler: Callable[[ModelRequest], ModelResponse],
) -> ModelResponse:
    for attempt in range(3):
        try:
            return handler(request)
        except Exception as e:
            if attempt == 2:
                raise
            print(f"Retry {attempt + 1}/3 after error: {e}")

State updates

Both node-style and warp-style hooks can update agent state. But the mechanism differs.

Node-style hooks: 直接返回一个dict, 由graph的reducers将该dict应用到agent’s state

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


from langchain.agents.middleware import after_model, AgentState
from langgraph.runtime import Runtime
from typing import Any
from typing_extensions import NotRequired


class TrackingState(AgentState):
    model_call_count: NotRequired[int]


@after_model(state_schema=TrackingState)
def increment_after_model(state: TrackingState, runtime: Runtime) -> dict[str, Any] | None:
    return {"model_call_count": state.get("model_call_count", 0) + 1}

Warp-style hooks: 对于model calls,返回ExtendedModeResponse with a Command来通过model response插入state updates

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27


from typing import Callable
from langchain.agents.middleware import (
    wrap_model_call,
    ModelRequest,
    ModelResponse,
    AgentState,
    ExtendedModelResponse
)
from langgraph.types import Command
from typing_extensions import NotRequired

class UsageTrackingState(AgentState):
    """Agent state with token usage tracking."""

    last_model_call_tokens: NotRequired[int]


@wrap_model_call(state_schema=UsageTrackingState)
def track_usage(
    request: ModelRequest,
    handler: Callable[[ModelRequest], ModelResponse],
) -> ExtendedModelResponse:
    response = handler(request)
    return ExtendedModelResponse(
        model_response=response,
        command=Command(update={"last_model_call_tokens": 150}),
    )

Command 会流经图的 reducer，因此更新会被正确应用，并且消息是累加的，而不是替换现有状态

对于tool calls,直接返回一个Command

Composition with multiple middleware

当多个Middleware layer返回ExtendedModelResponse时,他们的命令会进行组合

命令通过reducer应用:每个Command都会成为一个独立的状态更新,对于消息而言,他们是additive
冲突时外层优先
重试安全:如果外层的Middleware会导致handler()的多次调用,会抛弃早期的调用命令

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58


from typing import Annotated, Callable

from langchain.agents.middleware import (
    AgentMiddleware,
    AgentState,
    ExtendedModelResponse,
    ModelRequest,
    ModelResponse,
)
from langchain.messages import SystemMessage
from langgraph.types import Command
from typing_extensions import NotRequired


def _last_wins(_a: str, b: str) -> str:
    """Reducer: last writer wins (outer overwrites inner)."""
    return b


class CustomMiddlewareState(AgentState):
    """Agent state: trace_layer uses last-wins (outer wins), messages use additive reducer."""

    # Non-reducer field with last-wins: both middleware write; outermost value wins
    trace_layer: NotRequired[Annotated[str, _last_wins]]


class OuterMiddleware(AgentMiddleware):
    def wrap_model_call(
        self,
        request: ModelRequest,
        handler: Callable[[ModelRequest], ModelResponse],
    ) -> ExtendedModelResponse:
        response = handler(request)
        return ExtendedModelResponse(
            model_response=response,
            command=Command(update={
                "trace_layer": "outer",
                "messages": [SystemMessage(content="[Outer ran]")],
            }),
        )


class InnerMiddleware(AgentMiddleware):
    """Adds trace_layer and message. Outer adds to same keys; trace_layer: outer wins, messages: additive."""

    def wrap_model_call(
        self,
        request: ModelRequest,
        handler: Callable[[ModelRequest], ModelResponse],
    ):
        response = handler(request)
        return ExtendedModelResponse(
            model_response=response,
            command=Command(update={
                "trace_layer": "inner",
                "messages": [SystemMessage(content="[Inner ran]")],
            }),
        )

Create middleware

In two ways:

Decorator-base middleware

for single-hook middleware. To warp individual functions.

Node-style:

@before_agent - Runs before agent starts (once per invocation)
@before_model - Runs before each model call
@after_model - Runs after each model response
@after_agent - Runs after agent completes (once per invocation)

Wrap-style:

@wrap_model_call - Wraps each model call with custom logic
@wrap_tool_call - Wraps each tool call with custom logic

Convenience:

@dynamic_prompt - Generates dynamic system prompts

When to use decorators:

Single hook needed
No complex configuration
Quick prototyping

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35


from langchain.agents.middleware import (
    before_model,
    wrap_model_call,
    AgentState,
    ModelRequest,
    ModelResponse,
)
from langchain.agents import create_agent
from langgraph.runtime import Runtime
from typing import Any, Callable


@before_model
def log_before_model(state: AgentState, runtime: Runtime) -> dict[str, Any] | None:
    print(f"About to call model with {len(state['messages'])} messages")
    return None

@wrap_model_call
def retry_model(
    request: ModelRequest,
    handler: Callable[[ModelRequest], ModelResponse],
) -> ModelResponse:
    for attempt in range(3):
        try:
            return handler(request)
        except Exception as e:
            if attempt == 2:
                raise
            print(f"Retry {attempt + 1}/3 after error: {e}")

agent = create_agent(
    model="gpt-5.5",
    middleware=[log_before_model, retry_model],
    tools=[...],
)

Class-based middleware

适用于multiple hooks和更复杂的configuration;适用于你需要同时实现同步和异步for the same hook,或者你想要在一个组件内部组合多个hooks

py 的 AgentMiddleware子类会声明三个属性,agent factory会在编译时会自动提取

state_schema:适用自定义的字段拓展agent state
tools:注册middleware承载的工具
transformer:注册范围感知的流转化工厂

下面这个例子体现了多个hooks和同时实现同步与异步

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37


from langchain.agents.middleware import (
    AgentMiddleware,
    AgentState,
    ModelRequest,
    ModelResponse,
)
from langgraph.runtime import Runtime
from typing import Any, Callable

class LoggingMiddleware(AgentMiddleware):
    def before_model(self, state: AgentState, runtime: Runtime) -> dict[str, Any] | None:
        print(f"About to call model with {len(state['messages'])} messages")
        return None

    def after_model(self, state: AgentState, runtime: Runtime) -> dict[str, Any] | None:
        print(f"Model returned: {state['messages'][-1].content}")
        return None

    async def abefore_model(
        self, state: AgentState, runtime: Runtime
    ) -> dict[str, Any] | None:
        # Async version of before_model
        return None

    async def aafter_model(
        self, state: AgentState, runtime: Runtime
    ) -> dict[str, Any] | None:
        # Async version of after_model
        print(f"Model returned: {state['messages'][-1].content}")
        return None


agent = create_agent(
    model="gpt-5.5",
    middleware=[LoggingMiddleware()],
    tools=[...],
)

适用场景:

同时定义同步与异步实现for same hook
multiple hooks needed in a single middleware
复杂的定义需要
有跨项目的复用需求

Custom state schema

when you need to track state across hooks,middleware can extend the agent’s state with custom properties.

跨执行跟踪状态
在hooks之间共享数据
实现横切聚焦
执行条件性决策

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38


from langchain.agents import create_agent
from langchain.messages import HumanMessage
from langchain.agents.middleware import AgentState, before_model, after_model
from typing_extensions import NotRequired
from typing import Any
from langgraph.runtime import Runtime


class CustomState(AgentState):
    model_call_count: NotRequired[int]
    user_id: NotRequired[str]


@before_model(state_schema=CustomState, can_jump_to=["end"])
def check_call_limit(state: CustomState, runtime: Runtime) -> dict[str, Any] | None:
    count = state.get("model_call_count", 0)
    if count > 10:
        return {"jump_to": "end"}
    return None


@after_model(state_schema=CustomState)
def increment_counter(state: CustomState, runtime: Runtime) -> dict[str, Any] | None:
    return {"model_call_count": state.get("model_call_count", 0) + 1}


agent = create_agent(
    model="gpt-5.5",
    middleware=[check_call_limit, increment_counter],
    tools=[],
)

# Invoke with custom state
result = agent.invoke({
    "messages": [HumanMessage("Hello")],
    "model_call_count": 0,
    "user_id": "user-123",
})

Custom stream transformer(目前不知道有什么用处)

中间件可以注册流转换器工厂，将来自活动智能体流的事件投影到类型化的扩展通道上。这对于显现计数器、旁路伪影（side-channel artifacts）、部分输出或网络层脱敏（wire-level redaction）非常有用，而无需与框架的内置投影耦合。在编译时，中间件注册的工厂将与调用者直接传递给智能体工厂的任何内容合并。最终排序规则使内置的 ToolCallTransformer 处于最前面，并让调用者提供的条目排在最后。将 transformers 类属性设置为一个由工厂可调用对象组成的元组。每个工厂的形式为 Callable[[tuple[str, ...]], StreamTransformer] 并通过 factory(scope) 进行调用，其中 scope 是 mini-mux 作用域元组（() 表示根目录，非空表示子图）；每次调用都返回一个新的转换器，以保持每个子图的隔离。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


from langchain.agents import create_agent
from langchain.agents.middleware import AgentMiddleware


class ToolActivityMiddleware(AgentMiddleware):
    transformers = (ToolActivityTransformer,)


agent = create_agent(
    model="gpt-5-nano",
    tools=[...],
    middleware=[ToolActivityMiddleware()],
)

Execution order

key rules:

before_* hooks: First to last
after_* hooks: Last to first (reverse)
wrap_* hooks: Nested (first middleware wraps all others)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18


Before 钩子按顺序运行
middleware1.before_agent()
middleware2.before_agent()
middleware3.before_agent()
智能体循环开始
middleware1.before_model()
middleware2.before_model()
middleware3.before_model()
Wrap 钩子像函数调用一样嵌套
middleware1.wrap_model_call() → middleware2.wrap_model_call() → middleware3.wrap_model_call() → 模型
After 钩子按相反顺序运行
middleware3.after_model()
middleware2.after_model()
middleware1.after_model()
智能体循环结束
middleware3.after_agent()
middleware2.after_agent()
middleware1.after_agent()

Agent jumps

为了提前结束middleware,发挥一个 dictionary with jump_to

可选的 targets:

'end':跳转到agent执行的end(或者first after_agent hook)
'tools':跳转到tool节点
'model'跳转到model节点(或者first before_model hook)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


from langchain.agents.middleware import after_model, hook_config, AgentState
from langchain.messages import AIMessage
from langgraph.runtime import Runtime
from typing import Any


@after_model
@hook_config(can_jump_to=["end"])
def check_for_blocked(state: AgentState, runtime: Runtime) -> dict[str, Any] | None:
    last_message = state["messages"][-1]
    if "BLOCKED" in last_message.content:
        return {
            "messages": [AIMessage("I cannot respond to that request.")],
            "jump_to": "end"
        }
    return None

Configure tracing

自己看

Best practices

单一职责
优雅地处理错误
使用正确地hook类型
清晰地记录每一个自定义的state属性
在集成之前对于每一个中间件进行单元测试
考虑执行的顺序:将关键的middleware放在第一位(前面)
使用内置的中间件

Examples

Dynamic prompt

在运行时动态的调整提示词,以注入上下文,用户特定的指令和其他的信息这是最常用的middleware之一使用ModelRequest中的system_message字段来阅读和修改系统提示词.他包含了一个SystemMessage对象

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


from collections.abc import Callable

from langchain.agents.middleware import AgentMiddleware, ModelRequest, ModelResponse


class ContextMiddleware(AgentMiddleware):
    def wrap_model_call(
        self,
        request: ModelRequest,
        handler: Callable[[ModelRequest], ModelResponse],
    ) -> ModelResponse:
        new_content = list(request.system_message.content_blocks) + [
            {"type": "text", "text": "Additional context."}
        ]
        new_system_message = SystemMessage(content=new_content)
        return handler(request.override(system_message=new_system_message))

1
2
3
4


ModelRequest.system_message 始终是 SystemMessage 对象，即使智能体是使用 system_prompt="string" 创建的
使用 SystemMessage.content_blocks 将内容作为块列表进行访问，无论原始内容是字符串还是列表
修改系统消息时，请使用 content_blocks 并追加新块，以保留现有结构
对于缓存控制等高级用例，您可以将 SystemMessage 对象直接传递给 create_agent 的 system_prompt 参数

Dynamic model selection

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20


from collections.abc import Callable

from langchain.agents.middleware import AgentMiddleware, ModelRequest, ModelResponse
from langchain.chat_models import init_chat_model

complex_model = init_chat_model("claude-sonnet-4-6")
simple_model = init_chat_model("claude-haiku-4-5-20251001")


class DynamicModelMiddleware(AgentMiddleware):
    def wrap_model_call(
        self,
        request: ModelRequest,
        handler: Callable[[ModelRequest], ModelResponse],
    ) -> ModelResponse:
        if len(request.messages) > 10:
            model = complex_model
        else:
            model = simple_model
        return handler(request.override(model=model))

Dynamically selecting tools

动态选择工具适用于:需要减短prompts;更好的准确性;访问控制

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21


from langchain.agents import create_agent
from langchain.agents.middleware import AgentMiddleware, ModelRequest, ModelResponse
from typing import Callable


class ToolSelectorMiddleware(AgentMiddleware):
    def wrap_model_call(
        self,
        request: ModelRequest,
        handler: Callable[[ModelRequest], ModelResponse],
    ) -> ModelResponse:
        """Middleware to select relevant tools based on state/context."""
        # Select a small, relevant subset of tools based on state/context
        relevant_tools = select_relevant_tools(request.state, request.runtime)
        return handler(request.override(tools=relevant_tools))

agent = create_agent(
    model="gpt-5.5",
    tools=all_tools,  # All available tools need to be registered upfront
    middleware=[ToolSelectorMiddleware()],
)

Tool calling monitoring

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23


from collections.abc import Callable

from langchain.agents.middleware import AgentMiddleware
from langchain.messages import ToolMessage
from langchain.tools.tool_node import ToolCallRequest
from langgraph.types import Command


class ToolMonitoringMiddleware(AgentMiddleware):
    def wrap_tool_call(
        self,
        request: ToolCallRequest,
        handler: Callable[[ToolCallRequest], ToolMessage | Command],
    ) -> ToolMessage | Command:
        print(f"Executing tool: {request.tool_call['name']}")
        print(f"Arguments: {request.tool_call['args']}")
        try:
            result = handler(request)
            print("Tool completed successfully")
            return result
        except Exception as e:
            print(f"Tool failed: {e}")
            raise

Prompt caching(Anthropic)

ModelRequest.system_message 始终是 SystemMessage 对象，即使智能体是使用 system_prompt="string" 创建的
使用 SystemMessage.content_blocks 将内容作为块列表进行访问，无论原始内容是字符串还是列表
修改系统消息时，请使用 content_blocks 并追加新块，以保留现有结构
对于缓存控制等高级用例，您可以将 SystemMessage 对象直接传递给 create_agent 的 system_prompt 参数

Guardrails

Mon, 01 Jan 0001 00:00:00 +0000

GuardRails通过validating和flitering content来帮助你构建安全合规的agent

常见的应用场景:

预防个人信息的泄露
检测和阻止提示词注入攻击
阻止不恰当和有害的攻击
执行业务规则和合规性要求
验证输出重量和正确性

你可以在使用middleware在关键节点拦截执行 before the agent starts, after it completes, or around model and tool calls.

两种互补的办法实现 Guardrails: 1.Deterministic guardrails: 使用规则的逻辑比如正则表达式,关键词匹配或者显式检查优点:快且省缺点:可能漏掉细微的违规情况

2.Model-based guradrails 使用LLM或者classifiers进行语义分析优点:效果好缺点:延迟高,成本更高

Built-in guardrail

PII detection

可以检测emails,credit cards,IP addresses和其他的PII 提供了几种解决PII检测的策略:

Strategy	Description	Example
`redact`	Replace with `[REDACTED_{PII_TYPE}]`	`[REDACTED_EMAIL]`
`mask`	Partially obscure (e.g., last 4 digits)	`**--**-1234`
`hash`	Replace with deterministic hash	`a8f5f167...`
`block`	Raise exception when detected	Error thrown

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34


from langchain.agents import create_agent
from langchain.agents.middleware import PIIMiddleware


agent = create_agent(
    model="gpt-5.5",
    tools=[customer_service_tool, email_tool],
    middleware=[
        # Redact emails in user input before sending to model
        PIIMiddleware(
            "email",
            strategy="redact",
            apply_to_input=True,
        ),
        # Mask credit cards in user input
        PIIMiddleware(
            "credit_card",
            strategy="mask",
            apply_to_input=True,
        ),
        # Block API keys - raise error if detected
        PIIMiddleware(
            "api_key",
            detector=r"sk-[a-zA-Z0-9]{32}",
            strategy="block",
            apply_to_input=True,
        ),
    ],
)

# When user provides PII, it will be handled according to the strategy
result = agent.invoke({
    "messages": [{"role": "user", "content": "My email is john.doe@example.com and card is 5105-1051-0510-5100"}]
})

Human-in-the-loop

执行敏感操作之前征求人工的approval

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37


from langchain.agents import create_agent
from langchain.agents.middleware import HumanInTheLoopMiddleware
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.types import Command


agent = create_agent(
    model="gpt-5.5",
    tools=[search_tool, send_email_tool, delete_database_tool],
    middleware=[
        HumanInTheLoopMiddleware(
            interrupt_on={
                # Require approval for sensitive operations
                "send_email": True,
                "delete_database": True,
                # Auto-approve safe operations
                "search": False,
            }
        ),
    ],
    # Persist the state across interrupts
    checkpointer=InMemorySaver(),
)

# Human-in-the-loop requires a thread ID for persistence
config = {"configurable": {"thread_id": "some_id"}}

# Agent will pause and wait for approval before executing sensitive tools
result = agent.invoke(
    {"messages": [{"role": "user", "content": "Send an email to the team"}]},
    config=config
)

result = agent.invoke(
    Command(resume={"decisions": [{"type": "approve"}]}),
    config=config  # Same thread ID to resume the paused conversation
)

Custom guardrail

对于更加复杂的需求,可以自定义在agent执行之前和之后的middleware

Before agent guardrail

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55


from typing import Any

from langchain.agents.middleware import AgentMiddleware, AgentState, hook_config
from langgraph.runtime import Runtime

class ContentFilterMiddleware(AgentMiddleware):
    """Deterministic guardrail: Block requests containing banned keywords."""

    def __init__(self, banned_keywords: list[str]):
        super().__init__()
        self.banned_keywords = [kw.lower() for kw in banned_keywords]

    @hook_config(can_jump_to=["end"])
    def before_agent(self, state: AgentState, runtime: Runtime) -> dict[str, Any] | None:
        # Get the first user message
        if not state["messages"]:
            return None

        first_message = state["messages"][0]
        if first_message.type != "human":
            return None

        content = first_message.content.lower()

        # Check for banned keywords
        for keyword in self.banned_keywords:
            if keyword in content:
                # Block execution before any processing
                return {
                    "messages": [{
                        "role": "assistant",
                        "content": "I cannot process requests containing inappropriate content. Please rephrase your request."
                    }],
                    "jump_to": "end"
                }

        return None

# Use the custom guardrail
from langchain.agents import create_agent

agent = create_agent(
    model="gpt-5.5",
    tools=[search_tool, calculator_tool],
    middleware=[
        ContentFilterMiddleware(
            banned_keywords=["hack", "exploit", "malware"]
        ),
    ],
)

# This request will be blocked before any processing
result = agent.invoke({
    "messages": [{"role": "user", "content": "How do I hack into a database?"}]
})

Use “before agent” hook 来验证适用于会话级别的检查(例如身份验证,速率限制,拦截不安全的请求)

After agent guardrail

使用"after agent" hooks来验证输出使用于居于模型的安全检查,质量验证和最终合规性扫描

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48


from langchain.agents.middleware import AgentMiddleware, AgentState, hook_config
from langgraph.runtime import Runtime
from langchain.messages import AIMessage
from langchain.chat_models import init_chat_model
from typing import Any

class SafetyGuardrailMiddleware(AgentMiddleware):
    """Model-based guardrail: Use an LLM to evaluate response safety."""

    def __init__(self):
        super().__init__()
        self.safety_model = init_chat_model("gpt-5.4-mini")

    @hook_config(can_jump_to=["end"])
    def after_agent(self, state: AgentState, runtime: Runtime) -> dict[str, Any] | None:
        # Get the final AI response
        if not state["messages"]:
            return None

        last_message = state["messages"][-1]
        if not isinstance(last_message, AIMessage):
            return None

        # Use a model to evaluate safety
        safety_prompt = f"""Evaluate if this response is safe and appropriate.
        Respond with only 'SAFE' or 'UNSAFE'.

        Response: {last_message.content}"""

        result = self.safety_model.invoke([{"role": "user", "content": safety_prompt}])

        if "UNSAFE" in result.content:
            last_message.content = "I cannot provide that response. Please rephrase your request."

        return None

# Use the safety guardrail
from langchain.agents import create_agent

agent = create_agent(
    model="gpt-5.5",
    tools=[search_tool, calculator_tool],
    middleware=[SafetyGuardrailMiddleware()],
)

result = agent.invoke({
    "messages": [{"role": "user", "content": "How do I make explosives?"}]
})

Combine multiple guardrails

你可以将多个安全护栏加入middleware数组来堆叠他们,他们将按顺序执行,并且允许你构建层级保护

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21


from langchain.agents import create_agent
from langchain.agents.middleware import PIIMiddleware, HumanInTheLoopMiddleware

agent = create_agent(
    model="gpt-5.5",
    tools=[search_tool, send_email_tool],
    middleware=[
        # Layer 1: Deterministic input filter (before agent)
        ContentFilterMiddleware(banned_keywords=["hack", "exploit"]),

        # Layer 2: PII protection (before and after model)
        PIIMiddleware("email", strategy="redact", apply_to_input=True),
        PIIMiddleware("email", strategy="redact", apply_to_output=True),

        # Layer 3: Human approval for sensitive tools
        HumanInTheLoopMiddleware(interrupt_on={"send_email": True}),

        # Layer 4: Model-based safety check (after agent)
        SafetyGuardrailMiddleware(),
    ],
)

LangChain Philosophy

Mon, 01 Jan 0001 00:00:00 +0000

LangChain is driven by a few core beliefs:

Large Language Models (LLMs) are great, powerful new technology.
LLMs are even better when you combine them with external sources of data.
LLMs will transform what the applications of the future look like. Specifically, the applications of the future will look more and more agentic.
It is still very early on in that transformation.
While it’s easy to build a prototype of those agentic applications, it’s still really hard to build agents that are reliable enough to put into production.

DeepFlow

Mon, 01 Jan 0001 00:00:00 +0000

1.langgraph.json文件的作用

when:

使用Langgraph cli部署本地API服务
部署到langgraph cloud/langsmith deployment
使用langgraph studio及逆行调试

不需要: 使用传统的后端API服务;纯脚本/嵌入式开发

作用: 指定graph入口;生命项目依赖;配置环境变量等等

2.Sandbox的作用

提供隔离的安全执行环境

安全隔离(防止恶意代码和误操作):主要是限制一些危险操作,比如rm等删除操作,防止访问敏感数据,防止向外发送未经授权的请求等等
环境独立和资源限制:
1. 资源配额:CPU,内存,磁盘限额
2. 隔离环境:每一个session都有干净的上下文
支持复杂工具运行:
1. 提供工具:文件系统操作工具
2. 提供无头(Headless)浏览器环境

3.技能加载的递归实现:

递归发现skills下嵌套的SKILL.md文件

4.Subagent的SSE事件

SSE(Server sent evnet):一种基于HTTP协议的单向实时通信技术,创建HTTP连接后,主动,持续地向客户端推送数据

subagent的SSE事件

subagent持续向用户端暴露该agent的执行情况可视化的多agent协作;支持中途打断和交互

5.常见的工具生态系统

沙箱;内置工具(自己编写的);社区工具;MCP;Skill

Human-in-the-loop

Mon, 01 Jan 0001 00:00:00 +0000

By checking each tool call against a configurable policy 如果需要暂停,middleware将会issue an interrupt来暂停执行图状态将会使用LangGraph的persistence layer来保存,因此可以pause and resume later

Interrupt decision types

Decision Type	Description	Example Use Case
✅ `approve`	Execute the tool with the original arguments as proposed by the agent.	Send an email draft exactly as written
✏️ `edit`	Modify the tool arguments before execution.	Change the recipient before sending an email
❌ `reject`	Skip executing this tool call entirely and return rejection feedback to the agent.	Deny file deletion and explain why
💬 `respond`	Return the human’s message directly as a synthetic tool result, skipping execution, for “ask user” style tools.	Answer an `"ask_user"` prompt with a direct reply

每个工具可用的决策类型取决于你在interrupt_on中配置的策略使用场景: Use reject when the human is denying the requested action. Use respond only when the human is acting as the tool, such as answering an ask_user prompt. Do not use respond to deny side-effecting tools, because its message is treated as a successful tool result.

Configurable interrupt

要使用HITL,在create_agent时把中间件添加到agent的middleware列表你可以使用tool actions到允许的决策类型的映射来配置他

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25


from langchain.agents import create_agent
from langchain.agents.middleware import HumanInTheLoopMiddleware 
from langgraph.checkpoint.memory import InMemorySaver 


agent = create_agent(
    model="gpt-5.5",
    tools=[write_file, execute_sql, read_data],
    middleware=[
        HumanInTheLoopMiddleware(
            interrupt_on={
                "write_file": True,  # All decisions (approve, edit, reject, respond) allowed
                "execute_sql": {"allowed_decisions": ["approve", "reject"]},  # No editing allowed
                "read_data": False, # Safe operation, no approval needed
            },
            # Prefix for interrupt messages - combined with tool name and args to form the full message
            # e.g., "Tool execution pending approval: execute_sql with query='DELETE FROM...'"
            # Individual tools can override this by specifying a "description" in their interrupt config
            description_prefix="Tool execution pending approval",
        ),
    ],
    # Human-in-the-loop requires checkpointing to handle interrupts.
    # In production, use a persistent checkpointer like AsyncPostgresSaver or MongoDBSaver.
    checkpointer=InMemorySaver(),
)

ps:你必须配置一个checkpointer来持久化graph state, 在生产环境中,使用持久性的checkpointerSaver,如AsyncPostgresSaver 或 MongoDBSaver,对于测试和原型设计,使用InMemorySaver。

Conditional interrupts

默认情况下,interrupt_on中列出的tools都会暂停以供审查,如果要仅暂停某些调用,在tool的InterruptConfig中添加when谓词,该谓词接受一个ToolCallRequest来返回True以中断或者False以自动批准

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36


from langchain.agents import create_agent
from langchain.agents.middleware import HumanInTheLoopMiddleware, ToolCallRequest
from langgraph.checkpoint.memory import InMemorySaver


def writes_outside_workspace(request: ToolCallRequest) -> bool:
    """Pause writes to paths outside the workspace directory."""
    path = request.tool_call["args"].get("path", "")
    return not path.startswith("/workspace/")


def is_write_query(request: ToolCallRequest) -> bool:
    """Pause SQL that isn't a read-only SELECT."""
    query = request.tool_call["args"].get("query", "")
    return not query.lstrip().upper().startswith("SELECT")


agent = create_agent(
    model="gpt-5.5",
    tools=[write_file, execute_sql, read_data],
    middleware=[
        HumanInTheLoopMiddleware(
            interrupt_on={
                "write_file": {
                    "allowed_decisions": ["approve", "edit", "reject"],
                    "when": writes_outside_workspace,
                },
                "execute_sql": {
                    "allowed_decisions": ["approve", "reject"],
                    "when": is_write_query,
                },
            },
        ),
    ],
    checkpointer=InMemorySaver(),
)

评估为False的tool不会pause

Responding to interrupts

在version=v2中,结果时一个带有interrupts属性的GraphOutput,该属性包含需要审查的动作,你可以将这些动作提交给审查者,并在提供决策后恢复执行

Decision types

同上

Multiple types

当出现多个actions时,根据中断触发的顺序为每个动作提供一个决策

Streaming with human-in-the-loop

使用stream_event()在agent运行和中断时流式传输实时更新; stream.message流式传输LLM令牌,使用stream.value检查agent state快照以获取中断

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27


from langgraph.types import Command

config = {"configurable": {"thread_id": "some_id"}}

# Stream agent progress and LLM tokens until interrupt
stream = agent.stream_events(
    {"messages": [{"role": "user", "content": "Delete old records from the database"}]},
    config=config,
    version="v3",
)
for message in stream.messages:
    for token in message.text:
        print(token, end="", flush=True)

# Check whether the run paused for human input
if stream.interrupted:
    print(f"\n\nInterrupt: {stream.interrupts}")

# Resume with streaming after human decision
stream = agent.stream_events(
    Command(resume={"decisions": [{"type": "approve"}]}),
    config=config,
    version="v3",
)
for message in stream.messages:
    for token in message.text:
        print(token, end="", flush=True)

Execution lifecycle

middleware定义了一个在模型产生response之后但是在任何的tool call之前的hook

The agent invokes the model to generate a response.
The middleware inspects the response for tool calls.
If any calls require human input, the middleware builds a HITLRequest with action_requests and review_configs and calls interrupt.
The agent waits for human decisions.
Based on the HITLResponse decisions, the middleware executes approved or edited calls, synthesizes ToolMessage’s for rejected calls, returns human replies directly as ToolMessage’s for respond decisions, and resumes execution.

Custom HITL logic 对于更加复杂的工作流,你可以构建自定义的中断原语哥中间件来实现

Message

Mon, 01 Jan 0001 00:00:00 +0000

Message

Carrying both the content and metadata which represent the state

Messages contain:

Role: identifies the message type(system,user)
Content: the actual content
Metadata: optional fields: such as information,message IDs,and token usage

This is the standard message type

Basic usage

The easiest way to use message is to create message objects and pass them to a model when invoking

Text prompts

String Ideal for generation tasks where you don’t need to retain.

1

response = model.invoke("Write a haiku about spring")

When to use:

a single ,standalone request
no need for conversation history
minimal code complexity

Message prompts

Providing a list of messages

1
2
3
4
5
6
7
8


from langchain.messages import SystemMessage, HumanMessage, AIMessage

messages = [
    SystemMessage("You are a poetry expert"),
    HumanMessage("Write a haiku about spring"),
    AIMessage("Cherry blossoms bloom...")
]
response = model.invoke(messages)

When:

Managing multi-turn conversations
Working with multimodal content (images, audio, files)
Including system instructions

Dictionary format

Specify message in OpenAI chat completions format:

1
2
3
4
5
6


messages = [
    {"role": "system", "content": "You are a poetry expert"},
    {"role": "user", "content": "Write a haiku about spring"},
    {"role": "assistant", "content": "Cherry blossoms bloom..."}
]
response = model.invoke(messages)

Message types

System messages
Human messages
Al messages:
Tool messages:

System message

An initial set of instructions that primes the model’s behavior

Human message

user input and interactions

AI message

The LLM sometimes treat the message differently, so sometimes we can make up a AI message manually.

Tool calls

are included in AI messages

Token usage

in AI message in usage_metadata field

Streaming and chunks

When streaming, u can receive AIMessageChunk objects and it can be combined into a full message by summing

Tool message

tool result in ToolMessage object

Message content

An loosely-typed attribute,supporting strings and lists if untyped objects

Beside, LangChain provides dedicated content types for text, reasoning, citations, multi-modal data, server-side tool calls, and other message content

This may contain either:

A string
A list of content blocks in a provider-native format
A list of LangChain’s standard content blocks

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18


from langchain.messages import HumanMessage

# String content
human_message = HumanMessage("Hello, how are you?")

# Provider-native format (e.g., OpenAI)
human_message = HumanMessage(content=[
    {"type": "text", 
    "text": "Hello, how are you?"},
    {"type": "image_url", 
    "image_url": {"url": "https://example.com/image.jpg"}}
])

# List of standard content blocks
human_message = HumanMessage(content_blocks=[
    {"type": "text", "text": "Hello, how are you?"},
    {"type": "image", "url": "https://example.com/image.jpg"},
])

Standard content blocks

LangChain provides a standard representation for message content that works across providers.

消息对象实现了一个 content_blocks 属性，该属性会延迟将 content 属性解析为标准的、类型安全的表示形式。例如，从 ChatAnthropic 或 ChatOpenAI 生成的消息将包含各自提供商格式的 thinking 或 reasoning 块，但可以延迟解析为一致的 ReasoningContentBlock 表示形式

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


from langchain.messages import AIMessage

message = AIMessage(
    content=[
        {
            "type": "reasoning",
            "id": "rs_abc123",
            "summary": [
                {"type": "summary_text", "text": "summary 1"},
                {"type": "summary_text", "text": "summary 2"},
            ],
        },
        {"type": "text", "text": "...", "id": "msg_abc123"},
    ],
    response_metadata={"model_provider": "openai"}
)
message.content_blocks

1
2
3


[{'type': 'reasoning', 'id': 'rs_abc123', 'reasoning': 'summary 1'},
 {'type': 'reasoning', 'id': 'rs_abc123', 'reasoning': 'summary 2'},
 {'type': 'text', 'text': '...', 'id': 'msg_abc123'}]

将不同的provider的统一接口,是通过content_blocks属性来实现的

Multimodal

Multimodality refers to the ability to work with data that comes in different forms, such as text, audio, images, and video

Chat models can accept multimodal data as input and generate it as output. Below we show short examples of input messages featuring multimodal data.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30


# From URL
message = {
    "role": "user",
    "content": [
        {"type": "text", "text": "Describe the content of this image."},
        {"type": "image", "url": "https://example.com/path/to/image.jpg"},
    ]
}

# From base64 data
message = {
    "role": "user",
    "content": [
        {"type": "text", "text": "Describe the content of this image."},
        {
            "type": "image",
            "base64": "AAAAIGZ0eXBtcDQyAAAAAGlzb21tcDQyAAACAGlzb2...",
            "mime_type": "image/jpeg",
        },
    ]
}

# From provider-managed File ID
message = {
    "role": "user",
    "content": [
        {"type": "text", "text": "Describe the content of this image."},
        {"type": "image", "file_id": "file-abc123"},
    ]
}

Content blocks are represented (either when creating a message or accessing the content_blocks property) as a list of typed dictionaries

Serialization

You can serialize messages to plain objects for storage and deserialize back to message instances. This is useful for persisting conversation history and resuming sessions.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


from langchain.messages import HumanMessage
from langchain_core.load import dumpd, load

message = HumanMessage("What is the capital of France?")

# Serialize to a plain dict
serialized = dumpd(message)

# Deserialize back to a message object
restored = load(serialized)

Overview

Mon, 01 Jan 0001 00:00:00 +0000

The situation that fits middleware

Tracking agent behavior with logging, analytics, and debugging.
Transforming prompts, tool selection, and output formatting.
Adding retries, fallbacks, and early termination logic.
Applying rate limits, guardrails, and PII detection.

The agent loop

the core agent loop involves calling a model ,letting it choose tools to execute,and then finishing when it calls no more tools

Middleware在其中的作用: exposes hooks before and after each of these steps

Use middleware inside a LangGraph workflow

将整个agent作为更大的StateGraph的子图或者节点适用场景: 当拓扑结构不再是简单的循环直到结束时例如:路由到多个agents之一并且对输入进行分类,并发工作,或者通过确定性步骤将agent调用串联起来

HumanInTheLoopMiddleware通过 tool’s name匹配 @tool修饰符从函数名获取器名称

Long-term-memory

Mon, 01 Jan 0001 00:00:00 +0000

Across sessions and conversations

Long-term memory 建立在Lang Graph stores,存储的形式:JSON documents organized by namespace and key

Usage

为了给一个agent添加memory功能,在使用create_agent时create a store and pass it to agent

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


from langchain.agents import create_agent
from langchain_core.runnables import Runnable
from langgraph.store.memory import InMemoryStore

# InMemoryStore saves data to an in-memory dictionary. Use a DB-backed store in production use.
store = InMemoryStore()

agent: Runnable = create_agent(
    "claude-sonnet-4-6",
    tools=[],
    store=store,
)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


from langchain.agents import create_agent
from langchain_core.runnables import Runnable
from langgraph.store.postgres import PostgresStore  # type: ignore[import-not-found]

DB_URI = "postgresql://postgres:postgres@localhost:5432/postgres?sslmode=disable"

with PostgresStore.from_conn_string(DB_URI) as store:
    store.setup()
    agent: Runnable = create_agent(
        "claude-sonnet-4-6",
        tools=[],
        store=store,
    )

tool可以通过runtime.store来访问并且写入store

Memory storage

每个记忆都被组织在自定义的namespace(类似于文件夹)和一个独特的key(类似于文件名)之下随后可以通过内容过滤器实现跨命名空间的搜索 InmemoryStore

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33


from collections.abc import Sequence

from langgraph.store.base import IndexConfig
from langgraph.store.memory import InMemoryStore


def embed(texts: Sequence[str]) -> list[list[float]]:
    # Replace with an actual embedding function or LangChain embeddings object
    return [[1.0, 2.0] for _ in texts]


# InMemoryStore saves data to an in-memory dictionary. Use a DB-backed store in production use.
store = InMemoryStore(index=IndexConfig(embed=embed, dims=2))
user_id = "my-user"
application_context = "chitchat"
namespace = (user_id, application_context)
store.put(
    namespace,
    "a-memory",
    {
        "rules": [
            "User likes short, direct language",
            "User only speaks English & python",
        ],
        "my-key": "my-value",
    },
)
# get the "memory" by ID
item = store.get(namespace, "a-memory")
# search for "memories" within this namespace, filtering on content equivalence, sorted by vector similarity
items = store.search(
    namespace, filter={"my-key": "my-value"}, query="language preferences"
)

PostgreSQL

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36


from collections.abc import Sequence

from langgraph.store.base import IndexConfig
from langgraph.store.postgres import PostgresStore  # type: ignore[import-not-found]


def embed(texts: Sequence[str]) -> list[list[float]]:
    # Replace with an actual embedding function or LangChain embeddings object
    return [[1.0, 2.0] for _ in texts]


DB_URI = "postgresql://postgres:postgres@localhost:5432/postgres?sslmode=disable"

with PostgresStore.from_conn_string(
    DB_URI,
    index=IndexConfig(embed=embed, dims=2),  # type: ignore[arg-type]
) as store:
    store.setup()
    user_id = "my-user"
    application_context = "chitchat"
    namespace = (user_id, application_context)
    store.put(
        namespace,
        "a-memory",
        {
            "rules": [
                "User likes short, direct language",
                "User only speaks English & python",
            ],
            "my-key": "my-value",
        },
    )
    item = store.get(namespace, "a-memory")
    items = store.search(
        namespace, filter={"my-key": "my-value"}, query="language preferences"
    )

Read long-term memory in tools

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37


from dataclasses import dataclass

from langchain.agents import create_agent
from langchain.tools import ToolRuntime, tool
from langchain_core.runnables import Runnable
from langgraph.store.postgres import PostgresStore  # type: ignore[import-not-found]


@dataclass
class Context:
    user_id: str


DB_URI = "postgresql://postgres:postgres@localhost:5432/postgres?sslmode=disable"

with PostgresStore.from_conn_string(DB_URI) as store:
    store.setup()
    store.put(("users",), "user_123", {"name": "John Smith", "language": "English"})

    @tool
    def get_user_info(runtime: ToolRuntime[Context]) -> str:
        """Look up user info."""
        assert runtime.store is not None
        user_info = runtime.store.get(("users",), runtime.context.user_id)
        return str(user_info.value) if user_info else "Unknown user"

    agent: Runnable = create_agent(
        "claude-sonnet-4-6",
        tools=[get_user_info],
        store=store,
        context_schema=Context,
    )

    result = agent.invoke(
        {"messages": [{"role": "user", "content": "look up user information"}]},
        context=Context(user_id="user_123"),
    )

Write long-term memory from tools

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41


from dataclasses import dataclass

from langchain.agents import create_agent
from langchain.tools import ToolRuntime, tool
from langchain_core.runnables import Runnable
from langgraph.store.postgres import PostgresStore  # type: ignore[import-not-found]
from typing_extensions import TypedDict


@dataclass
class Context:
    user_id: str


class UserInfo(TypedDict):
    name: str


@tool
def save_user_info(user_info: UserInfo, runtime: ToolRuntime[Context]) -> str:
    """Save user info."""
    assert runtime.store is not None
    runtime.store.put(("users",), runtime.context.user_id, dict(user_info))
    return "Successfully saved user info."


DB_URI = "postgresql://postgres:postgres@localhost:5432/postgres?sslmode=disable"

with PostgresStore.from_conn_string(DB_URI) as store:
    store.setup()
    agent: Runnable = create_agent(
        "claude-sonnet-4-6",
        tools=[save_user_info],
        store=store,
        context_schema=Context,
    )

    agent.invoke(
        {"messages": [{"role": "user", "content": "My name is John Smith"}]},
        context=Context(user_id="user_123"),
    )

Models

Mon, 01 Jan 0001 00:00:00 +0000

In addition to text generation,many LLMs provide :tool calling,Structured output,Multimodality,Reasoning different model excel at different tasks

LangChain has Standard model interface,which allow you to experiment with and switch models

Basic usage

1.with agents 2.Standalone(outside the agent loop)

Initialize a model

the easiest way: use init_chat_model

1
2
3
4
5
6


import os
from langchain.chat_models import init_chat_model

os.environ["OPENAI_API_KEY"] = "sk-..."

model = init_chat_model("gpt-5.5")

key method: invoke,streaming.patch(批量处理)

Parameters

model: string (required) api_key: string temperature: number max_tokens: number timeout: number max_retries: number (default: 6)

1
2
3
4
5
6
7
8


model = init_chat_model(
    "claude-sonnet-4-6",
    # Kwargs passed to the model:
    temperature=0.7,
    timeout=30,
    max_tokens=1000,
    max_retries=6,  # Default; increase for unreliable networks
)

Connection resilience(连接韧性?)

LangChain chat models automatically retry failed API requests with exponential backoff

By default, models retry up to 6 times for network errors, rate limits (429), and server errors (5xx). Client errors like 401 (unauthorized) or 404 are not retried.

Use max_retries and timeout

Invocation

invoke:

with a message along with a thread_id role of a message: models use it to verify who sent the message

Streaming:

Calling stream() returns an iterator that yields output chunks as they are produced. You can use a loop to process each chunk in real-time:

Return AIMessageChunkobjects ,each chunk in a stream is designed to be gathered into a full message via summation

1
2
3
4


full = None  # None | AIMessageChunk
for chunk in model.stream("What color is the sky?"):
    full = chunk if full is None else full + chunk
    print(full.text)

Batch

Batching a collection of independent requests to a model can significantly improve performance and reduce costs, as the processing can be done in parallel:

Use batch_as_completed() to stream result,but the response is out of order

Tool calling

A schema, including the name of the tool, a description, and/or argument definitions (often a JSON schema)
A function or coroutine to execute.

Basic tool calling flow:

Use bind_tools to make tools you have defined available

1
2
3
4
5
6
7
8
9


from langchain.tools import tool

@tool
def get_weather(location: str) -> str:
    """Get the weather at a location."""
    return f"It's sunny in {location}."


model_with_tools = model.bind_tools([get_weather])

Common way to use tool calling

Tool execution loop
Forcing tool calls
Parallel tool calls
Streaming tool calls: tool calls are progressively built through ToolCallChunk You can accumulate chunks to build complete tool calls

Structured output

Provide their response in a format matching a given schema LangChain supports multiple schema types and methods for enforcing structured output.

Pydantic models provide the richest feature set with field validation, descriptions, and nested structures.

Python’s TypedDict provides a simpler alternative to Pydantic models, ideal when you don’t need runtime validation.

Provide a JSON Schema for maximum control and interoperability.

1.raw message和parsed message并存 2.nested structure

Multimodal

You can use content block to provide ono-text data to the model If the response contains multimodal data,the resulting AIMessage

will have content blocks with multimodal types

Reasoning

Many models are capable of performing multi-step reasoning to arrive at a conclusion. This involves breaking down complex problems into smaller, more

manageable steps.If supported by the underlying model, you can surface this reasoning process to better understand how the model arrived at its final answer.

Depending on the model,you can sometimes specify the level of effort it should put into reasoning. Using reasoning_effort parameter when invoking.

Local models

Prompt caching

ou can engage caching at three levels:

Implicit provider caching:
Provider-level explicit controls(显式): providers let you manually indicate cache points for greater control or to guarantee cost savings. These mirror the underlying provider/API behavior. Examples:
- ChatOpenAI (via prompt_cache_key)
- Anthropic content-block cache_control
- Gemini.
- AWS Bedrock cachePoint blocks
LangChain middleware:

for agents, middleware lets LangChain optimize caching of stable system prompt and tool content. Examples:
- Anthropic’s AnthropicPromptCachingMiddleware
- AWS Bedrock’s BedrockPromptCachingMiddleware

Cache usage will be reflected in the usage metadata of the model response.

Server-side tool use

Some providers support server-side tool-calling loops: models can interact with web search, code interpreters, and other tools and analyze the results in a single conversational turn.

If a model invokes a tool server-side, the content of the response message will include content representing the invocation and result of the tool. Accessing the content blocks of the response will return the server-side tool calls and results in a provider-agnostic format:

1
2
3
4
5
6
7
8
9


from langchain.chat_models import init_chat_model

model = init_chat_model("gpt-5.4-mini")

tool = {"type": "web_search"}
model_with_tools = model.bind_tools([tool])

response = model_with_tools.invoke("What was a positive news story from today?")
print(response.content_blocks)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29


[
    {
        "type": "server_tool_call",
        "name": "web_search",
        "args": {
            "query": "positive news stories today",
            "type": "search"
        },
        "id": "ws_abc123"
    },
    {
        "type": "server_tool_result",
        "tool_call_id": "ws_abc123",
        "status": "success"
    },
    {
        "type": "text",
        "text": "Here are some positive news stories from today...",
        "annotations": [
            {
                "end_index": 410,
                "start_index": 337,
                "title": "article title",
                "type": "citation",
                "url": "..."
            }
        ]
    }
]

This represents a single conversational turn; there are no associated ToolMessage objects that need to be passed in as in client-side tool-calling.

Model exceptions

Major integration packages raise standard exception types from langchain_core.exceptions for common model failures such as authentication errors, rate limits, and timeouts. These exceptions inherit from both the LangChain base type and the provider SDK’s own exception type, so you can catch either one:

Rate limiting

To control your call frequency ,and i you hit it, you will receive a rate limit error

To help manage rate limits, chat model integrations accept a rate_limiter parameter that can be provided during initialization to control the rate at which requests are made.

Base URL and proxy settings

You can configure a custom base URL for providers that implement the OpenAI Chat Completions API.

Custom base URL:

1
2
3
4
5
6


model = init_chat_model(
    model="MODEL_NAME",
    model_provider="openai",
    base_url="BASE_URL",
    api_key="YOUR_API_KEY",
)

HTTP proxy configuration:

1
2
3
4
5
6


from langchain_openai import ChatOpenAI

model = ChatOpenAI(
    model="gpt-5.5",
    openai_proxy="http://proxy.example.com:8080"
)

Log probabilities

Token usage

A number of model providers return token usage information as part of the invocation response.

You can track aggregate token counts across models in an application using either a callback or context manager, as shown below:

Callback:

1

You can track aggregate token counts across models in an application using either a callback or context manager, as shown below:

Context manager:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


from langchain.chat_models import init_chat_model
from langchain_core.callbacks import get_usage_metadata_callback

model_1 = init_chat_model(model="gpt-5.4-mini")
model_2 = init_chat_model(model="claude-haiku-4-5-20251001")

with get_usage_metadata_callback() as cb:
    model_1.invoke("Hello")
    model_2.invoke("Hello")
    print(cb.usage_metadata)

Invocation config

When invoking a model, you can pass additional configuration through the config parameter using a RunnableConfig dictionary. This provides run-time control over execution behavior, callbacks, and metadata tracking.

1
2
3
4
5
6
7
8
9


response = model.invoke(
    "Tell me a joke",
    config={
        "run_name": "joke_generation",      # Custom name for this run
        "tags": ["humor", "demo"],          # Tags for categorization
        "metadata": {"user_id": "123"},     # Custom metadata
        "callbacks": [my_callback_handler], # Callback handlers
    }
)

Key configuration attributes run_name: string : identifier tags: string[] metadata: object max_concurrency: number callbacks: array recursion_limit: number

Configurable models: You can also create a runtime-configurable model by specifying configurable_fields. If you don’t specify a model value, then 'model' and 'model_provider' will be configurable by default.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


from langchain.chat_models import init_chat_model

configurable_model = init_chat_model(temperature=0)

configurable_model.invoke(
    "what's your name",
    config={"configurable": {"model": "gpt-5-nano"}},  # Run with GPT-5-Nano
)
configurable_model.invoke(
    "what's your name",
    config={"configurable": {"model": "claude-sonnet-4-6"}},  # Run with Claude
)

Dynamic model selection

Dynamic models are selected at runtime based on the current state and context. This enables sophisticated routing logic and cost optimization.

To use a dynamic model, create middleware using the @wrap_model_call decorator that modifies the model in the request:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26


from langchain_openai import ChatOpenAI
from langchain.agents import create_agent
from langchain.agents.middleware import wrap_model_call, ModelRequest, ModelResponse


basic_model = ChatOpenAI(model="gpt-5.4-mini")
advanced_model = ChatOpenAI(model="gpt-5.5")

@wrap_model_call
def dynamic_model_selection(request: ModelRequest, handler) -> ModelResponse:
    """Choose model based on conversation complexity."""
    message_count = len(request.state["messages"])

    if message_count > 10:
        # Use an advanced model for longer conversations
        model = advanced_model
    else:
        model = basic_model

    return handler(request.override(model=model))

agent = create_agent(
    model=basic_model,  # Default model
    tools=[],
    middleware=[dynamic_model_selection]
)

Pre-bound models (models with bind_tools already called) are not supported when using structured output. If you need dynamic model selection with structured output, ensure the models passed to the middleware are not pre-bound.

Open-SWE

Mon, 01 Jan 0001 00:00:00 +0000

Convention part

1.使用async方法,只在必须的时候才使用sync方法(不是通用的设计范式):

约束之一为:Use async-only implementations. Add a sync method only when an interface requires it, and then raise NotImplementedError.

1.使用async方法优化并发性能 2.使用NotImplementedError显式告知为提供async实现,采用同步方法

2.Use strong types everywhere

类型安全:防止隐式转换带来的诸多问题
代码即文档(Self-Documenting Code)(可读性)与可维护性:该可读性不只是对于人,也可以是LLM,agent等等

常见的情况是: Never use Any

3.Use absolute imports across packages; same-package imports may start with one dot. Never use parent-relative imports.

解释:1.使用绝对路径而非相对路径2.同一个package内部的导入可以使用. 3.但是禁止使用多.来向上级或者祖先目录寻找模块

作用: 1.编码"跨层级重构"引入的导入崩溃 2.增强代码的可读性 3.允许单点.的折中考虑:既保留了同模块的内聚性,又省略了编码的繁琐性

4.Use structured logging with a static message and values in extra; never interpolate values into log messages. Avoid standard LogRecord field names in extra.

使用结构化日志,保留固定的静态信息,将动态变量放入extra属性中 1.优化了日志聚合和分析系统 2.防止破坏日志结构的内部状态 3.增加日志的安全性,防止个人私密信息的泄露

5.Never discard an error. Every `except` either propagates (re-raise, or raise a more useful error) or logs what it swallowed — a bare `except ...: return None` / `pass` hides the failure from everyone debugging it later.

1.向上抛出更具业务语义的异常 2.记录日志后重新抛出 3.吞掉异常,但必须记录日志

Test Part

运行测试: 切勿在本地运行完整测试套件；仅运行与本次变更相关的测试。添加测试: 仅当测试能有意义地保护可观察行为时才添加测试。不要添加仅简单重复常量、映射、提示词文本、源码结构或内部调用顺序等非本质交互的“变更检测（change-detector）”测试。覆盖有意义的边缘情况，并保持测试的确定性（deterministic）。

Codes

1.显式导出接口:

1
2
3
4
5
6
7


# 1. 显式引入（Re-export / Forwarding）：
# 从子模块/深层模块 agent.server 中导入 get_agent 和 traced_agent
from agent.server import get_agent, traced_agent

# 2. 导出控制（Public API Definition）：
# 定义该模块对外的公开 API 白名单
__all__ = ["get_agent", "traced_agent"]

1.精简包的导出路径 2.控制from module import * 只有在__all__中的对象才会被导 3.静态类型检查:将–明确识别为公开给外部调用的Pubilc API

2._function_name(),function_name()的关系

OOP中的封装和面向接口编程的思想在Python中,函数前带有_下划线的通常被认为是私有函数(认为,这是大家的一种约定俗称,但是并没有真正意义上的private函数), 而通过这个方法,将function_name()暴露为接口,将__function_name()封装为内部实现

3.日志的冒泡传递机制

当当前目录的logger是一个没有绑定任何输出目的地(Handler)的节点时,他会把日志信息抛给父级logger

Server.py中关于subagent的创建部分

1.显式沙盒隔离

对于敏感工具的调用,不是使用prompt劝导不要调用敏感工具,而是直接使用_SubagentToolGuard 和 is_subagent_excluded_tool的MIddleware进行了切面阻断且拦截后不抛出异常导致崩溃,而是优雅的返回结构化的ToolMessage进行告知

2.纵深防御的中间件分层:

代码将中间件按职责严格分层装配：

模型层 (_subagent_model_middleware)：拦截 LLM 调用的基础异常（超时、格式清洗、报错捕捉）。
运行期层 (_subagent_middleware)：工作流推送保护、动态工具注入。
环境守卫层 (_subagent_guard_middleware)：区分 local_run 环境（桌面端自由执行 vs 云端环境 PR 拦截）。这种模块化的 list 解包组合（* 语法），极大地提升了中间件复用率。

3.显式继承覆盖机制

通过 _DisableInheritedMiddleware 解决了子图继承父图中间件时的冲突问题。子 Agent 作为独立图编译，有时继承父级的某些中间件（如重复检查消息队列）会导致死锁或逻辑冗余，显式声明 inherited_middleware_exclusions 实现了精细的取消重写。

4.提示词动态增强

SubAgent 的 description 决定了 Parent Agent 能否准确路由任务。源码不仅使用基础描述，还动态追加了 load_prompt('system/general-purpose-subagent-suffix.md') 这个技巧在返回敏感工具调用的ToolMessage也有使用

5.创建Subagent的标准通用流程

1.确定运行模式和分叉策略

确定Subagent的隔离级别

2.裁剪工具集和构建安全工具防护网

列出Subagent可用的工具,并使用切面Middleware横切阻断违规操作

3.组装中间件栈

按顺序: 权限拦截 -> 日志/轨迹记录 -> 动态工具 -> 模型超时/容错控制 -> 上下文卸载

4.根据运行环境注入安全守卫

根据部署目标（如本地 CLI、Docker 容器或 LangGraph Cloud）注入环境特定的守卫，如禁用高危 Shell 命令或 PR 自动提交

5.拼接路由 Prompt 与定义能力边界

为 SubAgent 编写明确的 name 和 description，并附加标准化的系统后缀说明（Suffix），帮助主 Agent 准确判断何种子任务需要委派（Delegate）。

6.打包工厂函数并编译子图

将上述组件封装工厂函数，返回符合框架规范的 SubAgent 配置结构供图引擎（Graph Engine）统一编译。

6.海象运算符

:= 含义为赋值并返回赋值之后的值

7.创建FastAPI应用总结(api/app.py文件)

配置校验前置：配置、数据库、云端沙箱和本地 LLM 的校验（validate_*）全部在 lifespan 启动阶段完成，遵循 Fail-Fast（快速失败） 原则[cite: 1]。

数据迁移幂等设计：版本升级时的存储迁移代码应当做成 no-op 驱动的防重入函数，并容忍局部失败[cite: 1]。

资源清理彻底：在 lifespan 的 finally 块中集中释放 DB 连接、HTTP Client 和 Worker 句柄，防止 Docker 容器优雅停机（SIGTERM）时出现资源泄漏或数据损坏[cite: 1]。

8.版本向下兼容和容错包的导入技巧

try-except Import Error组合优先尝试符合新规范的导入方式;导入失败自动降级(fallback)到旧版的导入路径

9.dispatch.py的思想

对于不同的data source,将其进行归一化然后在传入进行处理,提高了代码的可阅读性和可复用性,而不是建立对于每个data source都进行独特的处理

Prebuilt Middleware

Mon, 01 Jan 0001 00:00:00 +0000

Tool error

捕获工具异常,并且转化为ToolMessage,模型收到信息并且从中修复 API reference: ToolErrorMiddleware

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35


from langchain.agents import create_agent
from langchain.agents.middleware import ToolErrorMiddleware, ToolRetryMiddleware


def on_error(exc: Exception, request: ToolCallRequest) -> str | None:
    # Surface ValueError to the model so it can correct the input
    if isinstance(exc, ValueError):
        return f"`{request.tool_call['name']}` failed: {type(exc).__name__}. Fix the input and retry."
    # Let all other exceptions propagate (halts the run)
    return None


# Async-only usage
async def aon_error(exc: Exception, request: ToolCallRequest) -> str | None:
    if isinstance(exc, ConnectionError):
        return f"Tool `{request.tool_call['name']}` encountered a connection error."
    return None


agent = create_agent(
    model="gpt-5.5",
    tools=[search_tool, database_tool],
    middleware=[
        # Place retry inner so exceptions reach ToolErrorMiddleware after retries are exhausted
        ToolRetryMiddleware(max_retries=3, on_failure="error"),
        ToolErrorMiddleware(on_error=on_error, tools=["search_tool"]),
    ],
)

# Async-only: pass aon_error alone (do not pass on_error)
async_agent = create_agent(
    model="gpt-5.5",
    tools=[api_tool],
    middleware=[ToolErrorMiddleware(aon_error=aon_error)],
)

Tool retry

通过可配置的指数避退机制来自动重试失败tool calls

API reference: ToolRetryMiddleware

Model retry

通过可配置的指数避退机制来自动重试失败的model calls 常见:API错误,网络问题

API reference: ModelRetryMiddleware

Model fallback

当主模型失败时,回退到备用模型 API reference: ModelFallbackMiddleware

Summarization

当接近token limit自动触发对话历史的压缩,保留最近几轮对话,摘要旧的上下文

API reference: SummarizationMiddleware

摘要是针对文本,对于多模态的信息会仅有生成的文本摘要来表示

Human-in-the-loop

在工具运行之前,需要人工的同意,编辑或者拒绝时

API reference: HumanInTheLoopMiddleware

Model call limit

限制model call的数量来防止无限的loop和过度的费用 API reference: ModelCallLimitMiddleware

Tool call limit

限制工具调用的次数(可以全局控制,也可以对于单一的工具进行限制) API reference: ToolCallLimitMiddleware

PII detection

使用可配置的策略来检测并处理个人认证信息

API reference: PIIMiddleware

Custom PII types

使用detector参数来自定义PII types

Three ways to create custom detectors:

Regex pattern string - Simple pattern matching
Custom function - Complex detection logic with validation
Custom detector function signature:

To-do list

使得模型有对于复杂任务的planning和track能力

API reference: TodoListMiddleware

LLM tool selector

顾名思义

API reference: LLMToolSelectorMiddleware 向模型提供结构化的结果输出

Provider tool search

适用于按需暴露tool的场景,通过将selected的tool放置在server-side tool search之后

API reference: ProviderToolSearchMiddleware

Shell tool

向agent提供持久的shell会话来执行命令,目前的shell会话暂时不支持中断(人机交互) 选择恰当的策略来匹配你的安全策略(HostExecutionPolicy, DockerExecutionPolicy, or CodexSandboxExecutionPolicy)

FileSystem middleware

提供agent同时与长期记忆和短期记忆互动的能力

ls: List the files in the filesystem
read_file: Read an entire file or a certain number of lines from a file
write_file: Write a new file to the filesystem
edit_file: Edit an existing file in the filesystem

短期与长期文件系统

默认情况下，这些工具写入您图状态（graph state）中的本地“文件系统”。要启用跨线程的持久存储，请配置一个 CompositeBackend，将特定路径（如 /memories/）路由到 StoreBackend。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23


from langchain.agents import create_agent
from deepagents.middleware import FilesystemMiddleware
from deepagents.backends import CompositeBackend, StateBackend, StoreBackend
from langgraph.store.memory import InMemoryStore

store = InMemoryStore()

agent = create_agent(
    model="claude-sonnet-4-6",
    store=store,
    middleware=[
        FilesystemMiddleware(
            backend=CompositeBackend(
                default=StateBackend(),
                routes={"/memories/": StoreBackend()}
            ),
            custom_tool_descriptions={
                "ls": "Use the ls tool when...",
                "read_file": "Use the read_file tool to..."
            }  # Optional: Custom descriptions for filesystem tools
        ),
    ],
)

当您为 /memories/ 配置了带有 StoreBackend 的 CompositeBackend 时，任何以 /memories/ 为前缀的文件都将保存到持久存储中，并在不同的线程之间保持存在。没有此前缀的文件将保留在暂存的状态存储中。

Subagent

上下文独立,并且保持主agent的上下文干净允许你通过task来创建subagents

A subagent is defined with a name, description, system prompt, and tools. You can also provide a subagent with a custom model, or with additional middleware.

对于更复杂的情景,你需要提供预编译的LangGraph graph作为subagent

Rubric grading

某些任务有明确的“完成”定义，智能体在第一次尝试时可能无法可靠地达到。RubricMiddleware 允许您将*“完成”的具体标准*声明为量规（rubric），让智能体自我评估并不断迭代，直到满足该标准或达到最大迭代次数限制。

API reference: RubricMiddleware

File search

提供Glob和Grep能力来访问文件系统

API reference: FilesystemFileSearchMiddleware

Context editing

上下文管理

API reference: ContextEditingMiddleware, ClearToolUsesEdit

LLM tool emulator

将实际的工具相应更换为AI生成的response,通常是为了测试目的

Retrieval

Mon, 01 Jan 0001 00:00:00 +0000

Two limitations of LLM:

finite context
Static Knowledge

RAG(检索增强生成)

Building a knowledge base

A repository of documents of structured data 使用LangChain’s document loaders and vector stores 来建立你自己的知识库如果你已经有知识库,你不需要重建,你只需要

Connect it as a tool for agent in agentic RAG
Query it and supply the retrieved content as context to the LLM (2-Step RAG)

From retrieval to RAG

Retrieval pipeline

Document loaders,text splitters,Embedding models,vector stores,retrievers

RAG architectures

Architecture	Description	Control	Flexibility	Latency	Example Use Case
2-Step RAG	Retrieval always happens before generation. Simple and predictable	✅ High	❌ Low	⚡ Fast	FAQs, documentation bots
Agentic RAG	An LLM-powered agent decides when and how to retrieve during reasoning	❌ Low	✅ High	⏳ Variable	Research assistants with access to multiple tools
Hybrid	Combines characteristics of both approaches with validation steps	⚖️ Medium	⚖️ Medium	⏳ Variable	Domain-specific Q&A with quality validation

2-step RAG

Agentic RAG

由LLM来决定是否需要检索和如何检索信息

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


import requests
from langchain.tools import tool
from langchain.chat_models import init_chat_model
from langchain.agents import create_agent


@tool
def fetch_url(url: str) -> str:
    """Fetch text content from a URL"""
    response = requests.get(url, timeout=10.0)
    response.raise_for_status()
    return response.text

system_prompt = """\
Use fetch_url when you need to fetch information from a web-page; quote relevant snippets.
"""

agent = create_agent(
    model="claude-sonnet-4-6",
    tools=[fetch_url], # A tool for retrieval
    system_prompt=system_prompt,
)

Hybrid RAG

结合了前两者的优点,核心组件包括:

查询增强:修改question来提高检索质量,这可能包括重写不清晰的查询、生成多个变体，或使用其他上下文扩展查询。

检索验证:评估检索到的资料是否充足

答案验证:检查生成的答案准确性,完整性和于源文件的一致性

此架构适用于

存在模糊或描述不详的查询的应用场景
需要验证或质量控制步骤的系统
涉及多个数据源或迭代优化的工作流

Short-term Memory

Mon, 01 Jan 0001 00:00:00 +0000

Conversation history is the most common form of short-term memory

Fact1: message includes instructions and inpus

Usage

To add short-term memory to an agent ,you need to specify a checkpointer when creating an agent note: short-term memory is part of your agent’s state. State is persisted in a database using a checkpointer so thread can resume it an any time. Short-term memory updates when the agent is invoked or a step (like a tool call) is completed, and the state is read at the start of each step.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29


from langchain.agents import create_agent
from langgraph.checkpoint.memory import InMemorySaver  


def get_user_info() -> str:
    """Look up information about the current user."""
    return "No user profile on file."


agent = create_agent(
    model="openai:gpt-5.5",
    tools=[get_user_info],
    checkpointer=InMemorySaver(),
)

thread_config = {"configurable": {"thread_id": "1"}}
response = agent.invoke(
    {"messages": [{"role": "user", "content": "Hi! My name is Bob."}]},
    thread_config,
)["messages"][-1].content

print(response)  # "Hi Bob! Nice to see you here. How are you doing?"

response = agent.invoke(
    {"messages": [{"role": "user", "content": "What's my name?"}]},
    thread_config,
)["messages"][-1].content

print(response)  # "You are Bob!"

In production

use a checkpointer backed by a database

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29


from langchain.agents import create_agent
from langgraph.checkpoint.memory import InMemorySaver  


def get_user_info() -> str:
    """Look up information about the current user."""
    return "No user profile on file."


agent = create_agent(
    model="openai:gpt-5.5",
    tools=[get_user_info],
    checkpointer=InMemorySaver(),
)

thread_config = {"configurable": {"thread_id": "1"}}
response = agent.invoke(
    {"messages": [{"role": "user", "content": "Hi! My name is Bob."}]},
    thread_config,
)["messages"][-1].content

print(response)  # "Hi Bob! Nice to see you here. How are you doing?"

response = agent.invoke(
    {"messages": [{"role": "user", "content": "What's my name?"}]},
    thread_config,
)["messages"][-1].content

print(response)  # "You are Bob!"

Customizing agent memory

Agents use AgentState to manage short-term memory, specifically the conversation history via a message key

You can extend AgentState to ad additional fields Custom state schema are passed to create_agent using the state_scheam parameter

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24


from langchain.agents import create_agent, AgentState
from langgraph.checkpoint.memory import InMemorySaver


class CustomAgentState(AgentState):
    user_id: str
    preferences: dict

agent = create_agent(
    "gpt-5.5",
    tools=[get_user_info],
    state_schema=CustomAgentState,
    checkpointer=InMemorySaver(),
)

# Custom state can be passed in invoke
result = agent.invoke(
    {
        "messages": [{"role": "user", "content": "Hello"}],
        "user_id": "user_123",
        "preferences": {"theme": "dark"}
    },
    {"configurable": {"thread_id": "1"}})
    

Common patterns

Common solutions to exceed the LLM’s context window

Trim message(截断消息)

A way: keep track of the token count and truncate when it approaches that limit

Using trim messages utility and specify the number if tokens to keep ang the strategy to use for handing the boundary

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51



@before_mfrom langchain.messages import RemoveMessage
from langgraph.graph.message import REMOVE_ALL_MESSAGES
from langgraph.checkpoint.memory import InMemorySaver
from langchain.agents import create_agent, AgentState
from langchain.agents.middleware import before_model
from langgraph.runtime import Runtime
from langchain_core.runnables import RunnableConfig
from typing import Any


@before_model
def trim_messages(state: AgentState, runtime: Runtime) -> dict[str, Any] | None:
    """Keep only the last few messages to fit context window."""
    messages = state["messages"]

    if len(messages) <= 3:
        return None  # No changes needed

    first_msg = messages[0]
    recent_messages = messages[-3:] if len(messages) % 2 == 0 else messages[-4:]
    new_messages = [first_msg] + recent_messages

    return {
        "messages": [
            RemoveMessage(id=REMOVE_ALL_MESSAGES),
            *new_messages
        ]
    }

agent = create_agent(
    "gpt-5.5",
    tools=[...],
    middleware=[trim_messages],
    checkpointer=InMemorySaver(),
)

config: RunnableConfig = {"configurable": {"thread_id": "1"}}

agent.invoke({"messages": "hi, my name is bob"}, config)
agent.invoke({"messages": "write a short poem about cats"}, config)
agent.invoke({"messages": "now do the same but for dogs"}, config)
final_response = agent.invoke({"messages": "what's my name?"}, config)

final_response["messages"][-1].pretty_print()
"""
================================== Ai Message ==================================

Your name is Bob. You told me that earlier.
If you'd like me to call you a nickname or use a different name, just say the word.
"""

Delete message

Useful when you want to remove specific message or clear the entire message history Use RemoveMessage to delete message from the graph state and you need to use a state key with add_messages reducer and default theAgentState provides it

To remove specific messages:

1
2
3
4
5
6
7


from langchain.messages import RemoveMessage  

def delete_messages(state):
    messages = state["messages"]
    if len(messages) > 2:
        # remove the earliest two messages
        return {"messages": [RemoveMessage(id=m.id) for m in messages[:2]]}

To remove all messages:

1
2
3
4


from langgraph.graph.message import REMOVE_ALL_MESSAGES  

def delete_messages(state):
    return {"messages": [RemoveMessage(id=REMOVE_ALL_MESSAGES)]}

ps: 删除消息时，请确保生成的消息历史有效。检查你正在使用的 LLM 提供商的限制。例如：

某些提供程序期望消息历史记录以 user 消息开头
大多数提供程序要求带有工具调用的 assistant 消息后跟相应的 tool 结果消息。

Summarize messages

above pattern will lose information Summarizing message with a LLM and use the built-in SUmmarizationMiddleware

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


agent = create_agent(
    model="gpt-5.5",
    tools=[...],
    middleware=[
        SummarizationMiddleware(
            model="gpt-5.4-mini",
            trigger=("tokens", 4000),
            keep=("messages", 20)
        )
    ],
    checkpointer=checkpointer,
)

Access memory

tools

Read short-term memory in a tool

Using runtime parameter(typed as ToolRuntime) runtime parameter can not be seen by model but the tool can access the state through it

1
2
3
4
5
6
7


@tool
def get_user_info(
    runtime: ToolRuntime
) -> str:
    """Look up user info."""
    user_id = runtime.state["user_id"]
    return "User is John Smith" if user_id == "user_123" else "Unknown user"

write short-term memory form tools

Using return Command with a ToolMessage

Prompt

Using middleware to access to ~ to create dynamic prompts on conversation history or custom state fields

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34


from langchain.agents import create_agent
from typing import TypedDict
from langchain.agents.middleware import dynamic_prompt, ModelRequest


class CustomContext(TypedDict):
    user_name: str


def get_weather(city: str) -> str:
    """Get the weather in a city."""
    return f"The weather in {city} is always sunny!"


@dynamic_prompt
def dynamic_system_prompt(request: ModelRequest) -> str:
    user_name = request.runtime.context["user_name"]
    system_prompt = f"You are a helpful assistant. Address the user as {user_name}."
    return system_prompt


agent = create_agent(
    model="gpt-5-nano",
    tools=[get_weather],
    middleware=[dynamic_system_prompt],
    context_schema=CustomContext,
)

result = agent.invoke(
    {"messages": [{"role": "user", "content": "What is the weather in SF?"}]},
    context=CustomContext(user_name="John Smith"),
)
for msg in result["messages"]:
    msg.pretty_print()

Before model

Using @before_model middleware to process message before model calls

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51


from langchain.messages import RemoveMessage
from langgraph.graph.message import REMOVE_ALL_MESSAGES
from langgraph.checkpoint.memory import InMemorySaver
from langchain.agents import create_agent, AgentState
from langchain.agents.middleware import before_model
from langchain_core.runnables import RunnableConfig
from langgraph.runtime import Runtime
from typing import Any


@before_model
def trim_messages(state: AgentState, runtime: Runtime) -> dict[str, Any] | None:
    """Keep only the last few messages to fit context window."""
    messages = state["messages"]

    if len(messages) <= 3:
        return None  # No changes needed

    first_msg = messages[0]
    recent_messages = messages[-3:] if len(messages) % 2 == 0 else messages[-4:]
    new_messages = [first_msg] + recent_messages

    return {
        "messages": [
            RemoveMessage(id=REMOVE_ALL_MESSAGES),
            *new_messages
        ]
    }


agent = create_agent(
    "gpt-5-nano",
    tools=[],
    middleware=[trim_messages],
    checkpointer=InMemorySaver()
)

config: RunnableConfig = {"configurable": {"thread_id": "1"}}

agent.invoke({"messages": "hi, my name is bob"}, config)
agent.invoke({"messages": "write a short poem about cats"}, config)
agent.invoke({"messages": "now do the same but for dogs"}, config)
final_response = agent.invoke({"messages": "what's my name?"}, config)

final_response["messages"][-1].pretty_print()
"""
================================== Ai Message ==================================

Your name is Bob. You told me that earlier.
If you'd like me to call you a nickname or use a different name, just say the word.
"""

After model Using @after_model

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


from langchain.messages import RemoveMessage
from langgraph.checkpoint.memory import InMemorySaver
from langchain.agents import create_agent, AgentState
from langchain.agents.middleware import after_model
from langgraph.runtime import Runtime


@after_model
def validate_response(state: AgentState, runtime: Runtime) -> dict | None:
    """Remove messages containing sensitive words."""
    STOP_WORDS = ["password", "secret"]
    last_message = state["messages"][-1]
    if any(word in last_message.content for word in STOP_WORDS):
        return {"messages": [RemoveMessage(id=last_message.id)]}
    return None

agent = create_agent(
    model="gpt-5-nano",
    tools=[],
    middleware=[validate_response],
    checkpointer=InMemorySaver(),
)

Runtime

Mon, 01 Jan 0001 00:00:00 +0000

LangChain’s create_agent的底层运行在LangGraph’s runtime LangGraph暴露一个Runtime object,提供以下的信息: 1.context:静态信息:user id,数据库连接和其他的依赖 2.Store:用于长期记忆的BaseStore实例 3.Stream writer:通过"custom"流式传输模式来硫化信息的object 4.Execution info: identity and retry information for the current execution(thread ID, run ID ,attmpt number) 5.Server info:在 LangGraph Server 上运行时的特定于服务器的元数据(assistant ID,graph ID, authenticated user ) You can access the runtime information within tools and middleware

Access

当创建agent with create_agent时,可以传入一个context_schema来定义存储在agent的Runtime中的context的结构

当调用agent是,则可以传入context with 相关的参数

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


from dataclasses import dataclass

from langchain.agents import create_agent


@dataclass
class Context:
    user_name: str

agent = create_agent(
    model="gpt-5-nano",
    tools=[...],
    context_schema=Context  
)

agent.invoke(
    {"messages": [{"role": "user", "content": "What's my name?"}]},
    context=Context(user_name="John Smith")
)

Inside tools

你可以在tool中访问runtime来:

Access the context
Read or write long-term memory
Wtrie tp the custom stream

通过ToolRuntime参数来访问Runtime对象:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18


from dataclasses import dataclass
from langchain.tools import tool, ToolRuntime  

@dataclass
class Context:
    user_id: str

@tool
def fetch_user_email_preferences(runtime: ToolRuntime[Context]) -> str:
    """Fetch the user's email preferences from the store."""
    user_id = runtime.context.user_id  

    preferences: str = "The user prefers you to write a brief and polite email."
    if runtime.store:
        if memory := runtime.store.get(("users",), user_id):
            preferences = memory.value["preferences"]

    return preferences

Execution info and server info inside tools

Access execution identity (thread ID, run ID) via runtime.execution_info, and server-specific metadata (assistant ID, authenticated user) via runtime.server_info when running on LangGraph Server

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


from langchain.tools import tool, ToolRuntime

@tool
def context_aware_tool(runtime: ToolRuntime) -> str:
    """A tool that uses execution and server info."""
    # Access thread and run IDs
    info = runtime.execution_info
    print(f"Thread: {info.thread_id}, Run: {info.run_id}")

    # Access server info (only available on LangGraph Server)
    server = runtime.server_info
    if server is not None:
        print(f"Assistant: {server.assistant_id}")
        if server.user is not None:
            print(f"User: {server.user.identity}")

    return "done"

Inside middleware

通过middleware访问runtime来:创建动态prompt,修改messages,根据上下文控制agent的行为

Use the Runtime parameter to access the Runtime object inside node-style hooks. For wrap-style hooks, the Runtime object is available inside the ModelRequest parameter.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42


from dataclasses import dataclass

from langchain.messages import AnyMessage
from langchain.agents import create_agent, AgentState
from langchain.agents.middleware import dynamic_prompt, ModelRequest, before_model, after_model
from langgraph.runtime import Runtime


@dataclass
class Context:
    user_name: str

# Dynamic prompts
@dynamic_prompt
def dynamic_system_prompt(request: ModelRequest) -> str:
    user_name = request.runtime.context.user_name  
    system_prompt = f"You are a helpful assistant. Address the user as {user_name}."
    return system_prompt

# Before model hook
@before_model
def log_before_model(state: AgentState, runtime: Runtime[Context]) -> dict | None:
    print(f"Processing request for user: {runtime.context.user_name}")
    return None

# After model hook
@after_model
def log_after_model(state: AgentState, runtime: Runtime[Context]) -> dict | None:
    print(f"Completed request for user: {runtime.context.user_name}")
    return None

agent = create_agent(
    model="gpt-5-nano",
    tools=[...],
    middleware=[dynamic_system_prompt, log_before_model, log_after_model],
    context_schema=Context
)

agent.invoke(
    {"messages": [{"role": "user", "content": "What's my name?"}]},
    context=Context(user_name="John Smith")
)

Execution info and server info inside middleware

Middleware hooks can also access runtime.execution_info and runtime.server_info:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


from langchain.agents import AgentState
from langchain.agents.middleware import before_model
from langgraph.runtime import Runtime


@before_model
def auth_gate(state: AgentState, runtime: Runtime) -> dict | None:
    """Block unauthenticated users when running on LangGraph Server."""
    server = runtime.server_info
    if server is not None and server.user is None:
        raise ValueError("Authentication required")
    print(f"Thread: {runtime.execution_info.thread_id}")
    return None

Streaming

Mon, 01 Jan 0001 00:00:00 +0000

Overview

实现的功能:

Stream agent progress
Stream LLM tokens
Streaming thinking/reasoning tokens
Stream custom updates
Stream multiple modes:从updates,messages,custom中进行选择

Supported stream modes

将下列的一个或者多个流式传输模式作为列表传输给stream或者astream方法

Mode	Description
`updates`	Streams state updates after each agent step. If multiple updates are made in the same step (e.g., multiple nodes are run), those updates are streamed separately.
`messages`	Streams tuples of `(token, metadata)` from any graph nodes where an LLM is invoked.
`custom`	Streams custom data from inside your graph nodes using the stream writer.

Agent progress

使用带有stream_mode =“updates"的stream或者astream方法将在每个agent step之后发送一个事件

例子: 在一次tool call之后,你应该看到下列更新

LLM node: AIMessage with tool call requests
Tool node: ToolMessage with execution result
LLM node: Final AI response

通过config传递thread_id以便checkpointer进行保存和后续的resume 也可以传递context,后续可以使用runtime.context进行访问

LLM tokens

使用stream_mode = "message"来访问token usage 在输出中可以看到agent的工具调用和最终的response

Custom updates

想要流式传输工具执行的过程(或结果),使用``get_stream_writer `inside the tool

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24


from langchain.agents import create_agent
from langgraph.config import get_stream_writer  


def get_weather(city: str) -> str:
    """Get weather for a given city."""
    writer = get_stream_writer()
    # stream any arbitrary data
    writer(f"Looking up data for city: {city}")
    writer(f"Acquired data for city: {city}")
    return f"It's always sunny in {city}!"

agent = create_agent(
    model="claude-sonnet-4-6",
    tools=[get_weather],
)

for chunk in agent.stream(
    {"messages": [{"role": "user", "content": "What is the weather in SF?"}]},
    stream_mode="custom",
    version="v2",
):
    if chunk["type"] == "custom":
        print(chunk["data"])

Stream multiple modes

通过将它们以mode列表的形式传输:stream_mode=[“updates”,“custom”]

每一段streamed chunk都是StreamPart的dict with type,ns and data 的keys 通过chunk[“type”]来决定流式模式,通过chunk[“data”]来访问负载

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24


from langchain.agents import create_agent
from langgraph.config import get_stream_writer


def get_weather(city: str) -> str:
    """Get weather for a given city."""
    writer = get_stream_writer()
    writer(f"Looking up data for city: {city}")
    writer(f"Acquired data for city: {city}")
    return f"It's always sunny in {city}!"

agent = create_agent(
    model="gpt-5-nano",
    tools=[get_weather],
)

for chunk in agent.stream(
    {"messages": [{"role": "user", "content": "What is the weather in SF?"}]},
    stream_mode=["updates", "custom"],
    version="v2",
):
    print(f"stream_mode: {chunk['type']}")
    print(f"content: {chunk['data']}")
    print("\n")

Common patterns

常见的几种应用场景

Stream thinking/reasoning tokens

一些模型会在产生最终的answer之前在内部进行思考可以通过过滤standard contnent blocks中的type = “reasoning"的部分来这些thinging/reasoning tokens

需要使用的mode为message即,stream_mode =“message”(与上文相符合)

1
2
3
4
5
6
7
8
9


stream = agent.stream_events(
    {"messages": [{"role": "user", "content": "What is the weather in SF?"}]},
    version="v3",
)
for message in stream.messages:
    for token in message.reasoning:
        print(f"[thinking] {token}", end="")
    for token in message.text:
        print(token, end="", flush=True)

Streaming tool calls

if you want to stream both:

Partial JSON as tool calls are generated
The completed, parsed tool calls that are executed

前者可以通过指定stream_mode= “message” 后者,根据是否在state中被tracked: True: 指定stream_mode = [“messages”,“updates”],并通过state updates来访问(如下所示) False: 使用custom updates 或者 aggregate the chunks during the streaming loop (next section).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


for chunk in agent.stream(
    {"messages": [input_message]},
    stream_mode=["messages", "updates"],
    version="v2",
):
    if chunk["type"] == "messages":
        token, metadata = chunk["data"]
        if isinstance(token, AIMessageChunk):
            _render_message_chunk(token)
    elif chunk["type"] == "updates":
        for source, update in chunk["data"].items():
            if source in ("model", "tools"):  # `source` captures node name
                _render_completed_message(update["messages"][-1])

Accessing completed messages

在message不通过state updates反映时采取的途径之一通过aggrrgate message chunks in the streaming loop

1.使用stream_writer inside the tool 2.如果你不能添加自定义的时间到stream中时,你可以流式传输中聚合消息块

Streaming with human-in-the-loop

解决human-in-the-loop interrupts:

为agent配置hunman-in-the-loop middleare 和 checkpointer
搜集在"update"流式传输模式下的interrupts
通过command来对这些interrupts进行响应

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72


from typing import Any

from langchain.agents import create_agent
from langchain.agents.middleware import HumanInTheLoopMiddleware
from langchain.messages import AIMessage, AIMessageChunk, AnyMessage, ToolMessage
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.types import Command, Interrupt


def get_weather(city: str) -> str:
    """Get weather for a given city."""

    return f"It's always sunny in {city}!"


checkpointer = InMemorySaver()

agent = create_agent(
    "openai:gpt-5.5",
    tools=[get_weather],
    middleware=[
        HumanInTheLoopMiddleware(interrupt_on={"get_weather": True}),
    ],
    checkpointer=checkpointer,
)


def _render_message_chunk(token: AIMessageChunk) -> None:
    if token.text:
        print(token.text, end="|")
    if token.tool_call_chunks:
        print(token.tool_call_chunks)


def _render_completed_message(message: AnyMessage) -> None:
    if isinstance(message, AIMessage) and message.tool_calls:
        print(f"Tool calls: {message.tool_calls}")
    if isinstance(message, ToolMessage):
        print(f"Tool response: {message.content_blocks}")


def _render_interrupt(interrupt: Interrupt) -> None:
    interrupts = interrupt.value  
    for request in interrupts["action_requests"]:
        print(request["description"])


input_message = {
    "role": "user",
    "content": (
        "Can you look up the weather in Boston and San Francisco?"
    ),
}
config = {"configurable": {"thread_id": "some_id"}}
interrupts = []
for chunk in agent.stream(
    {"messages": [input_message]},
    config=config,
    stream_mode=["messages", "updates"],
    version="v2",
):
    if chunk["type"] == "messages":
        token, metadata = chunk["data"]
        if isinstance(token, AIMessageChunk):
            _render_message_chunk(token)
    elif chunk["type"] == "updates":
        for source, update in chunk["data"].items():
            if source in ("model", "tools"):
                _render_completed_message(update["messages"][-1])
            if source == "__interrupt__":
                interrupts.extend(update)
                _render_interrupt(update[0])

接下来,为每一个interrupt收集对应的decision,并且decision的顺序与我们收集的action对应

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21


def _get_interrupt_decisions(interrupt: Interrupt) -> list[dict]:
    return [
        {
            "type": "edit",
            "edited_action": {
                "name": "get_weather",
                "args": {"city": "Boston, U.K."},
            },
        }
        if "boston" in request["description"].lower()
        else {"type": "approve"}
        for request in interrupt.value["action_requests"]
    ]

decisions = {}
for interrupt in interrupts:
    decisions[interrupt.id] = {
        "decisions": _get_interrupt_decisions(interrupt)
    }

decisions

我们可以通过向流式传输的循环传入一个command来resume

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


interrupts = []
for chunk in agent.stream(
    Command(resume=decisions),
    config=config,
    stream_mode=["messages", "updates"],
    version="v2",
):
    # Streaming loop is unchanged
    if chunk["type"] == "messages":
        token, metadata = chunk["data"]
        if isinstance(token, AIMessageChunk):
            _render_message_chunk(token)
    elif chunk["type"] == "updates":
        for source, update in chunk["data"].items():
            if source in ("model", "tools"):
                _render_completed_message(update["messages"][-1])
            if source == "__interrupt__":
                interrupts.extend(update)
                _render_interrupt(update[0])

Streaming from sub-agents

当agent中有多个LLMs时,需要对其来源进行消歧在创建agent时使用name参数,该参数在"message"的stream_mode中可以通过lc_agent_name键中的metadata访问 ps: 在创建stream时specify subgraph是= True

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35


from typing import Any

from langchain.agents import create_agent
from langchain.chat_models import init_chat_model
from langchain.messages import AIMessage, AnyMessage


def get_weather(city: str) -> str:
    """Get weather for a given city."""

    return f"It's always sunny in {city}!"


weather_model = init_chat_model("openai:gpt-5.5")
weather_agent = create_agent(
    model=weather_model,
    tools=[get_weather],
    name="weather_agent",
)


def call_weather_agent(query: str) -> str:
    """Query the weather agent."""
    result = weather_agent.invoke({
        "messages": [{"role": "user", "content": query}]
    })
    return result["messages"][-1].text


supervisor_model = init_chat_model("openai:gpt-5.5")
agent = create_agent(
    model=supervisor_model,
    tools=[call_weather_agent],
    name="supervisor",
)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34


def _render_message_chunk(token: AIMessageChunk) -> None:
    if token.text:
        print(token.text, end="|")
    if token.tool_call_chunks:
        print(token.tool_call_chunks)


def _render_completed_message(message: AnyMessage) -> None:
    if isinstance(message, AIMessage) and message.tool_calls:
        print(f"Tool calls: {message.tool_calls}")
    if isinstance(message, ToolMessage):
        print(f"Tool response: {message.content_blocks}")


input_message = {"role": "user", "content": "What is the weather in Boston?"}
current_agent = None
for chunk in agent.stream(
    {"messages": [input_message]},
    stream_mode=["messages", "updates"],
    subgraphs=True,
    version="v2",
):
    if chunk["type"] == "messages":
        token, metadata = chunk["data"]
        if agent_name := metadata.get("lc_agent_name"):
            if agent_name != current_agent:
                print(f"🤖 {agent_name}: ")
                current_agent = agent_name  
        if isinstance(token, AIMessageChunk):
            _render_message_chunk(token)
    elif chunk["type"] == "updates":
        for source, update in chunk["data"].items():
            if source in ("model", "tools"):
                _render_completed_message(update["messages"][-1])

Disable streaming

初始化时: set streaming = False

v2 streaming format

Pass version="v2" to stream() or astream() to get a unified output format. Every chunk is a StreamPart dict with type, ns, and data keys — the same shape regardless of stream mode or number of modes:

The v2 format also improves invoke() — it returns a GraphOutput object with .value and .interrupts attributes, cleanly separating state from interrupt metadata:

Structured Output

Mon, 01 Jan 0001 00:00:00 +0000

返回的形式: JSON objects, Pydantic models,or dataclasses that your application can use directly

LangChain’s create_agent 会自动处理结构化输出用户设置desired 结构化输出的schema 返回在 in the key strucured_responese key of the agent’s state

Field函数的作用: 结构化输出时,是创建元数据的重要途径,且其内部的description可以作为Prompt的一部分指导LLM的工作

1
2
3
4
5
6
7
8
9


def create_agent(
    ...
    response_format: Union[
        ToolStrategy[StructuredResponseT],
        ProviderStrategy[StructuredResponseT],
        type[StructuredResponseT],
        None,
    ]
)

Response format

使用response_format来控制agent返回结构化数据的方式

ToolStrategy[StructuredResponseT]: Uses tool calling for structured output
ProviderStrategy[StructuredResponseT]: Uses provider-native structured output
type[StructuredResponseT]: Schema type - automatically selects best strategy based on model capabilities
None: Structured output not explicitly requested

When a schema type is provided directly, LangChain automatically chooses:

ProviderStrategy if the model and provider chosen supports native structured output (e.g. OpenAI, Anthropic (Claude), or xAI (Grok)).
ToolStrategy for all other models.

Provider strategy

一些provider支持原生的结构化输出这是最可靠的方式要使用这种方式,请配置 Providerstrategy

1
2
3


class ProviderStrategy(Generic[SchemaT]):
    schema: type[SchemaT]
    strict: bool | None = None

参数 schema(required): 规定了结构化输出的format,取值有:

Pydantic models: BaseModel subclasses with field validation. Returns validated Pydantic instance.
Dataclasses: Python dataclasses with type annotations. Returns dict.
TypedDict: Typed dictionary classes. Returns dict.
JSON Schema: Dictionary with JSON schema specification. Must include top-level title and description keys. Returns dict.

strict: 可选;默认None;控制是否严格遵守schema

当你直接把schema type传给create_agent.response_format 时且模型支持原生的结构化输出时,LangChain会自动选择ProviderStrategy

Tool calling strategy

对于模型不支持原生的结构化输出,LangChain使用tool calling,这对于所有的支持tool calling的模型都适用使用:配置 ToolStrategy

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


class ToolStrategy(Generic[SchemaT]):
    schema: type[SchemaT]
    tool_message_content: str | None
    handle_errors: Union[
        bool,
        str,
        type[Exception],
        tuple[type[Exception], ...],
        Callable[[Exception], str],
    ]

schema: required 同上

tool_message_content: 生成结构化输出时的tool message的自定义内容如果没有提供,为显示结构化响应的响应数据

handle_errors: 默认为True 当结构化输出的验证失败的时候的错误处理策略,参数如下

True: Catch all errors with default error template
str: Catch all errors with this custom message
type[Exception]: Only catch this exception type with default message
tuple[type[Exception], ...]: Only catch these exception types with default message
Callable[[Exception], str]: Custom function that returns error message
False: No retry, let exceptions propagate

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29


from pydantic import BaseModel, Field
from typing import Literal, Union
from langchain.agents import create_agent
from langchain.agents.structured_output import ToolStrategy


class ProductReview(BaseModel):
    """Analysis of a product review."""
    rating: int | None = Field(description="The rating of the product", ge=1, le=5)
    sentiment: Literal["positive", "negative"] = Field(description="The sentiment of the review")
    key_points: list[str] = Field(description="The key points of the review. Lowercase, 1-3 words each.")

class CustomerComplaint(BaseModel):
    """A customer complaint about a product or service."""
    issue_type: Literal["product", "service", "shipping", "billing"] = Field(description="The type of issue")
    severity: Literal["low", "medium", "high"] = Field(description="The severity of the complaint")
    description: str = Field(description="Brief description of the complaint")

agent = create_agent(
    model="gpt-5.5",
    tools=[],
    response_format=ToolStrategy(Union[ProductReview, CustomerComplaint])
)

result = agent.invoke({
    "messages": [{"role": "user", "content": "Analyze this review: 'Great product: 5 out of 5 stars. Fast shipping, but expensive'"}]
})
result["structured_response"]
# ProductReview(rating=5, sentiment='positive', key_points=['fast shipping', 'expensive'])

Custom tool message content

the tool_message_content 参数允许你自定义结构化输出产生时呈现在会话历史中的message

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29


from pydantic import BaseModel, Field
from typing import Literal, Union
from langchain.agents import create_agent
from langchain.agents.structured_output import ToolStrategy


class ProductReview(BaseModel):
    """Analysis of a product review."""
    rating: int | None = Field(description="The rating of the product", ge=1, le=5)
    sentiment: Literal["positive", "negative"] = Field(description="The sentiment of the review")
    key_points: list[str] = Field(description="The key points of the review. Lowercase, 1-3 words each.")

class CustomerComplaint(BaseModel):
    """A customer complaint about a product or service."""
    issue_type: Literal["product", "service", "shipping", "billing"] = Field(description="The type of issue")
    severity: Literal["low", "medium", "high"] = Field(description="The severity of the complaint")
    description: str = Field(description="Brief description of the complaint")

agent = create_agent(
    model="gpt-5.5",
    tools=[],
    response_format=ToolStrategy(Union[ProductReview, CustomerComplaint])
)

result = agent.invoke({
    "messages": [{"role": "user", "content": "Analyze this review: 'Great product: 5 out of 5 stars. Fast shipping, but expensive'"}]
})
result["structured_response"]
# ProductReview(rating=5, sentiment='positive', key_points=['fast shipping', 'expensive'])

Error handing

LangChain提供了智能重试机制

Multiple structured outputs error

当模型错误地调用了多个结构化工具输出,会在ToolMessage中生成相应的反馈和提示词以便模型进行重试

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23


from pydantic import BaseModel, Field
from typing import Union
from langchain.agents import create_agent
from langchain.agents.structured_output import ToolStrategy


class ContactInfo(BaseModel):
    name: str = Field(description="Person's name")
    email: str = Field(description="Email address")

class EventDetails(BaseModel):
    event_name: str = Field(description="Name of the event")
    date: str = Field(description="Event date")

agent = create_agent(
    model="gpt-5.5",
    tools=[],
    response_format=ToolStrategy(Union[ContactInfo, EventDetails])  # Default: handle_errors=True
)

agent.invoke({
    "messages": [{"role": "user", "content": "Extract info: John Doe (john@email.com) is organizing Tech Conference on March 15th"}]
})

Schema validation error

当结构化输出和schema不匹配时,agent会返回特定的错误反馈

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


from pydantic import BaseModel, Field
from langchain.agents import create_agent
from langchain.agents.structured_output import ToolStrategy


class ProductRating(BaseModel):
    rating: int | None = Field(description="Rating from 1-5", ge=1, le=5)
    comment: str = Field(description="Review comment")

agent = create_agent(
    model="gpt-5.5",
    tools=[],
    response_format=ToolStrategy(ProductRating),  # Default: handle_errors=True
    system_prompt="You are a helpful assistant that parses product reviews. Do not make any field or value up."
)

agent.invoke({
    "messages": [{"role": "user", "content": "Parse this: Amazing product, 10/10!"}]
})

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27


================================ Human Message =================================

Parse this: Amazing product, 10/10!
================================== Ai Message ==================================
Tool Calls:
  ProductRating (call_1)
 Call ID: call_1
  Args:
    rating: 10
    comment: Amazing product
================================= Tool Message =================================
Name: ProductRating

Error: Failed to parse structured output for tool 'ProductRating': 1 validation error for ProductRating.rating
  Input should be less than or equal to 5 [type=less_than_equal, input_value=10, input_type=int].
 Please fix your mistakes.
================================== Ai Message ==================================
Tool Calls:
  ProductRating (call_2)
 Call ID: call_2
  Args:
    rating: 5
    comment: Amazing product
================================= Tool Message =================================
Name: ProductRating

Returning structured response: {'rating': 5, 'comment': 'Amazing product'}

Error handling strategies

你可以通过handle_errors参数自定义错误处理的方式

Custom error message:

1
2
3
4


ToolStrategy(
    schema=ProductRating,
    handle_errors="Please provide a valid rating between 1-5 and include a comment."
)

If handle_errors is a string, the agent will always prompt the model to re-try with a fixed tool message:

1
2
3
4


================================= Tool Message =================================
Name: ProductRating

Please provide a valid rating between 1-5 and include a comment.

Handle specific exceptions only:

1
2
3
4


ToolStrategy(
    schema=ProductRating,
    handle_errors=ValueError  # Only retry on ValueError, raise others
)

If handle_errors is an exception type, the agent will only retry (using the default error message) if the exception raised is the specified type. In all other cases, the exception will be raised.Handle multiple exception types:

1
2
3
4


ToolStrategy(
    schema=ProductRating,
    handle_errors=(ValueError, TypeError)  # Retry on ValueError and TypeError
)

If handle_errors is a tuple of exceptions, the agent will only retry (using the default error message) if the exception raised is one of the specified types. In all other cases, the exception will be raised.Custom error handler function:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32


from langchain.agents.structured_output import StructuredOutputValidationError
from langchain.agents.structured_output import MultipleStructuredOutputsError

def custom_error_handler(error: Exception) -> str:
    if isinstance(error, StructuredOutputValidationError):
        return "There was an issue with the format. Try again."
    elif isinstance(error, MultipleStructuredOutputsError):
        return "Multiple structured outputs were returned. Pick the most relevant one."
    else:
        return f"Error: {str(error)}"


agent = create_agent(
    model="gpt-5.5",
    tools=[],
    response_format=ToolStrategy(
                        schema=Union[ContactInfo, EventDetails],
                        handle_errors=custom_error_handler
                    )  # Default: handle_errors=True
)

result = agent.invoke({
    "messages": [{"role": "user", "content": "Extract info: John Doe (john@email.com) is organizing Tech Conference on March 15th"}]
})

for msg in result['messages']:
    # If message is actually a ToolMessage object (not a dict), check its class name
    if type(msg).__name__ == "ToolMessage":
        print(msg.content)
    # If message is a dictionary or you want a fallback
    elif isinstance(msg, dict) and msg.get('tool_call_id'):
        print(msg['content'])

On StructuredOutputValidationError:

1
2
3
4


================================= Tool Message =================================
Name: ToolStrategy

There was an issue with the format. Try again.

On MultipleStructuredOutputsError:

1
2
3
4


================================= Tool Message =================================
Name: ToolStrategy

Multiple structured outputs were returned. Pick the most relevant one.

On other errors:

1
2
3
4


================================= Tool Message =================================
Name: ToolStrategy

Error: 

No error handling:

1
2
3
4


response_format = ToolStrategy(
    schema=ProductRating,
    handle_errors=False  # All errors raised
)

Tools

Mon, 01 Jan 0001 00:00:00 +0000

Tools

Extend what agents can do!

Create tools

Basic tool definition

The easiest way to create a tool is to use the @tool decorator In this way the docstring will become the description of the tool

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


from langchain.tools import tool

@tool
def search_database(query: str, limit: int = 10) -> str:
    """Search the customer database for records matching the query.

    Args:
        query: Search terms to look for
        limit: Maximum number of results to return
    """
    return f"Found {limit} results for '{query}'"

docstring:’’' ada ’’' 的部分 docstring应该内容丰富且简明扼要参数的type hints是必须的,因为他们定义了工具的输入参数

Customize tool properties

Custom tool name

by default,the tool name is the function name. But you can override it.

1
2
3
4
5
6


@tool("web_search")  # Custom name
def search(query: str) -> str:
    """Search the web for information."""
    return f"Results for: {query}"

print(search.name)  # web_search

Custom tool description

Override the auto-generated tool description(refer to the docstring) for clearer model guidance

1
2
3
4


@tool("calculator", description="Performs arithmetic calculations. Use this for any math problems.")
def calc(expression: str) -> str:
    """Evaluate mathematical expressions."""
    return str(eval(expression))

Advanced schema definition

Define complex inputs with Pydantic models or JSON schemas

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23


from pydantic import BaseModel, Field
from typing import Literal

class WeatherInput(BaseModel):
    """Input for weather queries."""
    location: str = Field(description="City name or coordinates")
    units: Literal["celsius", "fahrenheit"] = Field(
        default="celsius",
        description="Temperature unit preference"
    )
    include_forecast: bool = Field(
        default=False,
        description="Include 5-day forecast"
    )

@tool(args_schema=WeatherInput)
def get_weather(location: str, units: str = "celsius", include_forecast: bool = False) -> str:
    """Get current weather and optional forecast."""
    temp = 22 if units == "celsius" else 72
    result = f"Current weather in {location}: {temp} degrees {units[0].upper()}"
    if include_forecast:
        result += "\nNext 5 days: Sunny"
    return result

Reserved argument names

Those can not be used as parameters’ name which will cause runtime errors config: reserved for RunnableConfig runtime: reserved for ToolRuntime

Access context

How to access and update this information(conversation history,user data,persistent memory)

How! By the ToolRuntime parameter

Component	Description	Use case
State	Short-term memory - mutable data that exists for the current conversation (messages, counters, custom fields)	Access conversation history, track tool call counts
Context	Immutable configuration passed at invocation time (user IDs, session info)	Personalize responses based on user identity
Store	Long-term memory - persistent data that survives across conversations	Save user preferences, maintain knowledge base
Stream Writer	Emit real-time updates during tool execution	Show progress for long-running operations
Execution Info	Identity and retry information for the current execution (thread ID, run ID, attempt number)	Access thread/run IDs, adjust behavior based on retry state
Server Info	Server-specific metadata when running on LangGraph Server (assistant ID, graph ID, authenticated user)	Access assistant ID, graph ID, or authenticated user info
Config	`RunnableConfig` for the execution	Access callbacks, tags, and metadata
Tool Call ID	Unique identifier for the current tool invocation	Correlate tool calls for logs and model invocations

Short-term memory(State)

Include the message history and any custom field you define in your graph state.

Access state

Use runtime: ToolRuntime to your signature to access state 在调用的时候,ToolNode会自动地注入value,但是这个参数不会出现在发送给model地schema上你需要通过runtime.state才能访问到当前对话的state

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24


from langchain.tools import tool, ToolRuntime
from langchain.messages import HumanMessage

@tool
def get_last_user_message(runtime: ToolRuntime) -> str:
    """Get the most recent message from the user."""
    messages = runtime.state["messages"]

    # Find the last human message
    for message in reversed(messages):
        if isinstance(message, HumanMessage):
            return message.content

    return "No user messages found"

# Access custom state fields
@tool
def get_user_preference(
    pref_name: str,
    runtime: ToolRuntime
) -> str:
    """Get a user preference value."""
    preferences = runtime.state.get("user_preferences", {})
    return preferences.get(pref_name, "Not set")

The runtime parameter is hidden from the model. For the example above, the model only sees pref_name in the tool schema.(上面说过了,结合例子更好理解)

Update state

Use Command to update the agent’s state. Useful to update custom state field. Include a ToolMessage in the update so the model can see the result of this tool call

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24


from langchain.agents import AgentState
from langchain.messages import ToolMessage
from langchain.tools import ToolRuntime, tool
from langgraph.types import Command


class CustomState(AgentState):
    user_name: str


@tool
def set_user_name(new_name: str, runtime: ToolRuntime[None, CustomState]) -> Command:
    """Set the user's name in the conversation state."""
    return Command(
        update={
            "user_name": new_name,
            "messages": [
                ToolMessage(
                    content=f"User name set to {new_name}.",
                    tool_call_id=runtime.tool_call_id,
                )
            ],
        }
    )

When tools update state variables, consider defining a reducer for those fields. Since LLMs can call multiple tools in parallel, a reducer determines how to resolve conflicts when the same state field is updated by concurrent tool calls. (创建reducer来应对并发修改同一个state值)

Context

Provides immutable configuration at invocation time Including user IDs,session details,application-specific settings

Access context through runtime.context. Pass it alongside a thread_id so the conversation is persisted across turns.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57


from dataclasses import dataclass

from langchain.agents import create_agent
from langchain.tools import tool, ToolRuntime
from langchain_core.utils.uuid import uuid7
from langchain_openai import ChatOpenAI


USER_DATABASE = {
    "user123": {
        "name": "Alice Johnson",
        "account_type": "Premium",
        "balance": 5000,
        "email": "alice@example.com",
    },
    "user456": {
        "name": "Bob Smith",
        "account_type": "Standard",
        "balance": 1200,
        "email": "bob@example.com",
    },
}


@dataclass
class UserContext:
    user_id: str


@tool
def get_account_info(runtime: ToolRuntime[UserContext]) -> str:
    """Get the current user's account information."""
    user_id = runtime.context.user_id

    if user_id in USER_DATABASE:
        user = USER_DATABASE[user_id]
        return (
            f"Account holder: {user['name']}\n"
            f"Type: {user['account_type']}\n"
            f"Balance: ${user['balance']}"
        )
    return "User not found"


model = ChatOpenAI(model="gpt-5.5")
agent = create_agent(
    model,
    tools=[get_account_info],
    context_schema=UserContext,
    system_prompt="You are a financial assistant.",
)

result = agent.invoke(
    {"messages": [{"role": "user", "content": "What's my current balance?"}]},
    config={"configurable": {"thread_id": str(uuid7())}},
    context=UserContext(user_id="user123"),
)

Long-term memory(Store)

The BaseStore provides persistent storage that survives across conversations. Access store through runtime.store. The store uses a namespace/key pattern to organize data

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43


from langgraph.store.memory import InMemoryStore
from langchain.agents import create_agent
from langchain.tools import tool, ToolRuntime
from langchain_openai import ChatOpenAI

# Access memory
@tool
def get_user_info(user_id: str, runtime: ToolRuntime) -> str:
    """Look up user info."""
    store = runtime.store
    user_info = store.get(("users",), user_id)
    return str(user_info.value) if user_info else "Unknown user"

# Update memory
@tool
def save_user_info(user_id: str, name: str, age: int, email: str, runtime: ToolRuntime) -> str:
    """Save user info."""
    store = runtime.store
    store.put(("users",), user_id, {"name": name, "age": age, "email": email})
    return "Successfully saved user info."

model = ChatOpenAI(model="gpt-5.5")

store = InMemoryStore()
agent = create_agent(
    model,
    tools=[get_user_info, save_user_info],
    store=store
)

# First session: save user info
agent.invoke({
    "messages": [{"role": "user", "content": "Save the following user: userid: abc123, name: Foo, age: 25, email: foo@langchain.dev"}]
})

# Second session: get user info
agent.invoke({
    "messages": [{"role": "user", "content": "Get user info for user with id 'abc123'"}]
})
# Here is the user info for user with ID "abc123":
# - Name: Foo
# - Age: 25
# - Email: foo@langchain.dev

Streaming writer

Stream real-time updates from tools during execution. Useful for providing progress feedback Use runtime.stream_Writer to emit custom updates

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


from langchain.tools import tool, ToolRuntime

@tool
def get_weather(city: str, runtime: ToolRuntime) -> str:
    """Get weather for a given city."""
    writer = runtime.stream_writer

    # Stream custom updates as the tool executes
    writer(f"Looking up data for city: {city}")
    writer(f"Acquired data for city: {city}")

    return f"It's always sunny in {city}!"

Execution info

Access thread ID,run ID,and retry state from within a tool via runtime.execution_info

1
2
3
4
5
6
7
8
9


from langchain.tools import tool, ToolRuntime

@tool
def log_execution_context(runtime: ToolRuntime) -> str:
    """Log execution identity information."""
    info = runtime.execution_info
    print(f"Thread: {info.thread_id}, Run: {info.run_id}")
    print(f"Attempt: {info.node_attempt}")
    return "done"

Server info

When your tool runs on LangGraph Server , access assistant ID,graph ID,and authenticated user via runtime.server_info

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


from langchain.tools import tool, ToolRuntime

@tool
def get_assistant_scoped_data(runtime: ToolRuntime) -> str:
    """Fetch data scoped to the current assistant."""
    server = runtime.server_info
    if server is not None:
        print(f"Assistant: {server.assistant_id}, Graph: {server.graph_id}")
        if server.user is not None:
            print(f"User: {server.user.identity}")
    return "done"

server_info is None when the tool is not running on LangGraph Server

Tool execution

Tool: used by agent Tool error handing is configured by middleware

Tool return values

a string for human-readable results;an object for structured results;a Command with optional message when you need to update the state

Return a string

when tool should provide plain text for the model to read and use in its next response

1
2
3
4
5
6
7


from langchain.tools import tool


@tool
def get_weather(city: str) -> str:
    """Get weather for a city."""
    return f"It is currently sunny in {city}."

The return value is converted to a ToolMessage

Return an object

when your tool produces structured data that the model should inspect

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


from langchain.tools import tool


@tool
def get_weather_data(city: str) -> dict:
    """Get structured weather data for a city."""
    return {
        "city": city,
        "temperature_c": 22,
        "conditions": "sunny",
    }

The object is serialized and sent back as tool output.
The model can read specific fields and reason over them.
Like string returns, this does not directly update graph state.

Return multimodal content

For multimodal content ,use standard content blocks so the model receives multimodal resources

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


from langchain.tools import tool


@tool
def capture_screenshot() -> list[dict]:
    """Capture a screenshot of the current page."""
    return [
        {"type": "text", "text": "Screenshot of the current page:"},
        {"type": "image", "url": "https://example.com/page.png"},
    ]

The return value is converted to a ToolMessage with multimodal content.
Use message.content_blocks to read the normalized block list after the tool runs.
The model must support the modalities you return. Check your model’s capabilities before returning images, audio, or video.

Return a Command

when the tool need to update graph state(or example, setting user preferences or app state)

When the Command targets the current graph, include a ToolMessage in the update whose tool call ID matches the current tool call. Every tool call in the message history must have a corresponding ToolMessage. 返回的一个Command的Command.update中必须包含一个对应的ToolMessage.对应是指tool_call_id对应

Use runtime.tool_call_id for the tool_call_id parameter. ToolNode enforces this requirement: if the update has no ToolMessage matching the tool call, it raises a ValueError. 否则会返回一个ValueError

Return directly from a tool

设置return_direct来使得agent-loop短路,这样工具输出不会传回给模型而是直接传给caller

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


from langchain.agents import create_agent
from langchain.tools import tool
from langchain_openai import ChatOpenAI


@tool(return_direct=True)
def fetch_order_status(order_id: str) -> str:
    """Fetch the current status of a customer order."""
    # In production, query your order management system here
    return f"Order {order_id} is shipped and will arrive in 2 days."


agent = create_agent(
    ChatOpenAI(model="gpt-4o-mini"),
    tools=[fetch_order_status],
)

result = agent.invoke({
    "messages": [{"role": "user", "content": "What is the status of order #12345?"}]
})
# The agent returns the tool output directly without another LLM call:
# "Order 12345 is shipped and will arrive in 2 days."

Behavior:

The tool executes normally and its output is wrapped in a ToolMessage.
The agent stops looping and returns the tool’s output as the final response, bypassing any additional model call.
Multiple parallel tool calls: When the model calls several tools in one step, all of them execute first. After all tools finish, the agent routes to END only if every tool in that batch has return_direct=True. The final response includes the ToolMessage outputs of every tool that was called in that step.

Use this when:

The tool’s output is the complete, user-ready answer (for example, a lookup that returns a ready-to-display result).
You want to avoid an extra model call when no additional reasoning is needed.
You need deterministic, unmodified output: the model cannot rephrase, summarize, or act on the tool result.

Return a Command with return_direct

A tool with return_direct=True can also return a Command to update graph state before the agent exits. Unlike a plain return value, a Command is not automatically converted into a ToolMessage. When the Command targets the current graph (graph is not set or is None), include a ToolMessage in Command.update matching the tool call’s tool_call_id. Omitting it causes ToolNode to raise a ValueError, because every AIMessage tool call must have a corresponding ToolMessage in the message history.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21


from langchain.messages import ToolMessage
from langchain.tools import ToolRuntime, tool
from langgraph.types import Command


@tool(return_direct=True)
def fetch_and_store_order(order_id: str, runtime: ToolRuntime) -> Command:
    """Fetch order status and store it in state."""
    status = f"Order {order_id} is shipped and will arrive in 2 days."
    return Command(
        update={
            "last_order_status": status,
            # Must include a ToolMessage so the message history stays valid
            "messages": [
                ToolMessage(
                    content=status,
                    tool_call_id=runtime.tool_call_id,
                )
            ],
        }
    )

Error handing

Using LangChain middleware to retry failed tool calls or return custom error message

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28


from collections.abc import Callable

from langchain.agents import create_agent
from langchain.agents.middleware import wrap_tool_call
from langchain.messages import ToolMessage
from langchain.tools.tool_node import ToolCallRequest


@wrap_tool_call
def handle_tool_errors(
    request: ToolCallRequest,
    handler: Callable[[ToolCallRequest], ToolMessage],
) -> ToolMessage:
    """Convert tool exceptions into ToolMessages the model can handle."""
    try:
        return handler(request)
    except Exception as e:
        return ToolMessage(
            content=f"Tool error: Please check your input and try again. ({e})",
            tool_call_id=request.tool_call["id"],
        )


agent = create_agent(
    model="openai:gpt-5.5",
    tools=[],
    middleware=[handle_tool_errors],
)

State injection

Tool access graph state through ToolRUntime

Dynamic tool section

太多tool会占用过多的context,太少则会限制能力动态工具允许在runtime时修改主要有两种方式实现,取决于工具是否事先已知

Filtering pre-registered tools

When all tools are known at agent creation time,you can pre-register them and dynamically filter which ones are exposed to the model based on state,permissions,or context

1

  代码有三个,自己去官网看

All possible tools are known at compile/startup time
You want to filter based on permissions, feature flags, or conversation state
Tools are static but their availability is dynamic

Runtime tool register

when tools are not discovered or created at runtime

wrap_model_call - Add the dynamic tools to the request
wrap_tool_call - Handle execution of the dynamically added tools

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36


from langchain.tools import tool
from langchain.agents import create_agent
from langchain.agents.middleware import AgentMiddleware, ModelRequest, ToolCallRequest

# A tool that will be added dynamically at runtime
@tool
def calculate_tip(bill_amount: float, tip_percentage: float = 20.0) -> str:
    """Calculate the tip amount for a bill."""
    tip = bill_amount * (tip_percentage / 100)
    return f"Tip: ${tip:.2f}, Total: ${bill_amount + tip:.2f}"

class DynamicToolMiddleware(AgentMiddleware):
    """Middleware that registers and handles dynamic tools."""

    def wrap_model_call(self, request: ModelRequest, handler):
        # Add dynamic tool to the request
        # This could be loaded from an MCP server, database, etc.
        updated = request.override(tools=[*request.tools, calculate_tip])
        return handler(updated)

    def wrap_tool_call(self, request: ToolCallRequest, handler):
        # Handle execution of the dynamic tool
        if request.tool_call["name"] == "calculate_tip":
            return handler(request.override(tool=calculate_tip))
        return handler(request)

agent = create_agent(
    model="gpt-5.5",
    tools=[get_weather],  # Only static tools registered here
    middleware=[DynamicToolMiddleware()],
)

# The agent can now use both get_weather AND calculate_tip
result = agent.invoke({
    "messages": [{"role": "user", "content": "Calculate a 20% tip on $85"}]
})

This approach is best when:

Tools are discovered at runtime (e.g., from an MCP server)
Tools are generated dynamically based on user data or configuration
You’re integrating with external tool registries

Headless tools

某些工具应该运行在用户应用运行的地方（通常是浏览器），而不是在进程内部。无头工具 (Headless tools)是工具定义（包括名称、描述和参数 schema），您在服务端向智能体进行注册。而实现仅在客户端注册，并在简短的中断/恢复（interrupt/resume）握手后执行

when to use

when the work depends on the environment, device, or UI that only exists on the client. For example:

Browser APIs
Privacy and locality
Latency(延迟)
Structured,safe effects

How it work

model can see it and call it but it actually happens outside the process

Define with (name, description,parameter) without in-process implementation
Register it with create agent so the model can call it normally
Handle the interrupt payload when the tool is invoked. Instead of running locally, the graph pauses with a payload shaped like {"type": "tool", "tool_call": {"id", "name", "args"}}.
Resume the graph

Use the optional onTool callback to observe lifecycle events (start, success, error) for UI feedback such as spinners or toasts.

Prebuilt tools

LangChain provides a large collection of prebuilt tools and toolkits for common tasks like web search, code interpretation, database access, and more. These ready-to-use tools can be directly integrated into your agents without writing custom code.See the tools and toolkits integration page for a complete list of available tools organized by category.

Tools from MCP

Model Context Protocol (MCP) is an open protocol that standardizes how applications expose tools to language models. Rather than hand-authoring a tool, you connect to an MCP server and adapt the tools it advertises into LangChain tools, ready to pass to an agent like any other tool.MCPAdapter discovers a server’s tools and converts them into LangChain tools. Open the adapter, call list_tools(), and hand the result to create_agent:

1
2
3
4
5
6


from langchain.agents import create_agent
from langchain.mcp import MCPAdapter

async with MCPAdapter("https://example.com/mcp") as adapter:
    tools = await adapter.list_tools()
    agent = create_agent("claude-sonnet-4-6", tools)

Server-side tool use

Some chat models feature built-in tools that are executed server-side by the model provider. These include capabilities like web search and code interpreters that don’t require you to define or host the tool logic.Refer to the individual chat model integration pages and the tool calling documentation for details on enabling and using these built-in tools.