Server-side context compaction for managing long conversations that approaching context window limits

1

Server-side compaction is the recommended strategy for managing context in long-running conversations and agentic workflows. It handles context management automatically, without client-side summarization code.

as a conversation grows, response quality degrades,Compaction extends the effective context length for long-running conversations and tasks compaction replaces older content with a concise summary.

This is ideal for:

• Chat-based, multi-turn conversations where you want users to use one chat for a long period of time
• Task-oriented prompts that require a lot of follow-up work (often tool use) that might exceed the context window

How compaction works

four steps:

1. Detects when input tokens reach your specified trigger threshold.
2. Generates a summary of the current conversation.
3. Creates a compaction block containing the summary.
4. Continues the response with the compacted context.

Basic usage

Enable compaction by adding the compact_20260112 strategy to context_management.edits in your Messages API request.

Parameters

ps: type must be “compact_20260112”: if u use Claude API, for sure; but if u def your own compaction function, it change use other value

Trigger configuration

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

client = anthropic.Anthropic()
messages = [{"role": "user", "content": "Hello, Claude"}]
response = client.beta.messages.create(
    betas=["compact-2026-01-12"],
    model="claude-opus-5",
    max_tokens=4096,
    messages=messages,
    context_management={
        "edits": [
            {
                "type": "compact_20260112",
                "trigger": {"type": "input_tokens", "value": 150000},
            }
        ]
    },
)

Custom summarization instructions

/compaction

default instructions:

1

You have written a partial transcript for the initial task above. Please write a summary of the transcript. The purpose of this summary is to provide continuity so you can continue to make progress towards solving the task in a future context, where the raw history above may not be accessible and will be replaced with this summary. Write down anything that would be helpful, including the state, next steps, learnings etc. You must wrap your summary in a <summary></summary> block.

Custom instructions don’t supplement the default prompt. They replace it completely

On Claude Fable 5.1 and Claude Mythos 5.1, a request with custom instructions summarizes from the visible conversation only: earlier thinking blocks are not part of the summarizer’s input.

Pausing after compaction

Use pause_after_compaction to pause the API after generating the compaction summary. This allows you to add additional content blocks

1
2
3

 context_management={
        "edits": [{"type": "compact_20260112", "pause_after_compaction": True}]
    }

Enforce a total token budget

When a model works on long tasks with many tool-use iterations, total token consumption can grow significantly. You can combine pause_after_compaction with a compaction counter to estimate cumulative usage and gracefully wrap up the task once a budget is reached.

This example appears in the SDK languages only: its value is the budget-tracking logic around the request. The raw request combines the trigger from Trigger configuration with pause_after_compaction from Pausing after compaction.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

client = anthropic.Anthropic()
messages = [{"role": "user", "content": "Hello, Claude"}]
TRIGGER_THRESHOLD = 100_000
TOTAL_TOKEN_BUDGET = 3_000_000
n_compactions = 0

response = client.beta.messages.create(
    betas=["compact-2026-01-12"],
    model="claude-opus-5",
    max_tokens=4096,
    messages=messages,
    context_management={
        "edits": [
            {
                "type": "compact_20260112",
                "trigger": {"type": "input_tokens", "value": TRIGGER_THRESHOLD},
                "pause_after_compaction": True,
            }
        ]
    },
)

if response.stop_reason == "compaction":
    n_compactions += 1
    messages.append({"role": "assistant", "content": response.content})

    # Estimate total tokens consumed; prompt wrap-up if over budget
    if n_compactions * TRIGGER_THRESHOLD >= TOTAL_TOKEN_BUDGET:
        messages.append(
            {
                "role": "user",
                "content": "Please wrap up your current work and summarize the final state.",
            }
        )

pause能够允许用户能够塞入自己的判断逻辑,防止超过总预算但是task未停止

Working with compaction blocks

When compaction is triggered, the API returns a compaction block at the start of the assistant response. The new compaction block will replace the old one

Passing compaction blocks back

You must pass the compaction block back to the API on subsequent requests to continue the conversation with the shortened prompt.

When the API receives a compaction block, all content blocks before it are ignored. You can either:

• Keep the original messages in your list and let the API handle removing the compacted content
• Manually drop the compacted messages and only include the compaction block onwards

Streaming

The compaction block streams differently from text blocks. You receive a content_block_start event, followed by a single content_block_delta with the complete summary content (no intermediate streaming), and then a content_block_stop event.

content_block_start -> content_block_delta -> content_block_stop

content_block_delta has the complete summary content

Prompt caching

Compaction works well with prompt caching. You can add a cache_control breakpoint on compaction blocks to cache the summarized content.

Maximining cache hits with system prompts

系统提示词的缓存和压缩新写入的内容会分开,这样写入的就只有压缩后的内容,而系统提示词的缓存不会失效,实现的方法是在你的系统提示词的后面打一个断点

When compaction occurs, the summary becomes new content that needs to be written to the cache. Without additional cache breakpoints, this would also invalidate any cached system prompt, requiring it to be re-cached along with the compaction summary.

To maximize cache hit rates, add a cache_control breakpoint at the end of your system prompt. This keeps the system prompt cached separately from the conversation, so when compaction occurs:

• The system prompt cache remains valid and is read from cache
• Only the compaction summary needs to be written as a new cache entry

his keeps long system prompts cached across multiple compaction events throughout a conversation.

Understanding usage

Compaction requires an additional sampling step, which contributes to rate limits and billing. The API returns detailed usage information in the response:

Output

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

{
  "usage": {
    "input_tokens": 23000,
    "output_tokens": 1000,
    "iterations": [
      {
        "type": "compaction",
        "input_tokens": 180000,
        "output_tokens": 3500
      },
      {
        "type": "message",
        "input_tokens": 23000,
        "output_tokens": 1000
      }
    ]
  }
}

The iterations array shows usage for each sampling iteration. When compaction occurs, you’ll see a compaction iteration followed by the main message iteration. The top-level input_tokens and output_tokens match the message iteration exactly in this example because there is only one non-compaction iteration. The final iteration’s token counts reflect the effective context size after compaction.

inerations数组中comapaction部分记录了压缩之前的上下文大小和压缩后的上下文大小,上面的usage和message相同是因为: 上述只有一次非compaction迭代

1
2
3

The top-level input_tokens and output_tokens do not include compaction iteration usage. They reflect the sum of all non-compaction iterations. To calculate total tokens consumed and billed for a request, sum across all entries in the usage.iterations array.

If you previously relied on usage.input_tokens and usage.output_tokens for cost tracking or auditing, you'll need to update your tracking logic to aggregate across usage.iterations when compaction is enabled. With the compaction beta enabled, every response includes usage.iterations, even if no compaction occurred. A compaction entry appears only when a new compaction is triggered during the request. Re-applying a previous compaction block incurs no additional compaction cost, and the top-level usage fields remain accurate in that case.

顶端的input_token和output_token不包含压缩迭代所产生的使用量,反应的是非压缩迭代的总和 要计算一次实际请求用量,要对usage.iteration中数组中的所有条目求和依赖usage.input_token和usage.output.token来进行成本统计的方式需要转变逻辑结构,转化为对于usage.interation中的每一项进行汇总 在启用compaction Beta之后,每个响应都会柏寒usage.interation,即使没有发生上下文压缩 但是只有出现了compaction,才会在interation中出现compaction条目 重新应用原来已经存在的compaction block不会产生额外的上下文压缩费用

Combing With other Features

Server tools

When using server tools (such as web search), the compaction trigger is checked at the start of each sampling iteration. Compaction might occur multiple times within a single request depending on your trigger threshold and the amount of output generated. 系统会在每个迭代之前检查是否需要压缩,这取决于你的设置和产出的数量

Token counting

The token counting endpoint (/v1/messages/count_tokens) applies existing compaction blocks in your prompt but does not trigger new compactions. Use it to check your effective token count after previous compactions 使用现存的compaction块而不会激活compaction,所哟不会产生新的上下文费用

Current limitations

• Same model for summarization: The model specified in your request is used for summarization. There is no option to use a different (for example, cheaper) model for the summary.

有点变态的是,Claudecode会给每一个模型都开新的缓存

• Compaction might fail when tools are defined: When your request includes tools, the model occasionally calls a tool during the internal summarization step instead of writing a summary. When this occurs, the response contains a compaction block with content: null. To prevent this, set instructions to a prompt that explicitly tells the model not to call tools, for example: 当你的请求中包含tools是,有时候压缩会因为调用工具而失败 解决方法:在Instruction中显式地声明不调用工具

Understand how the context window works, how extended thinking and tool use count toward it, and how to manage context as conversations grow.

the primary strategy is server-side-compaction

How the context window works

what is the context window:

The “context window” refers to all the text a language model can reference when generating a response, including the response itself. https://platform.claude.com/docs/images/context-window.svg

As token count grows, accuracy and recall degrade, a phenomenon known as context rot.

Chat interfaces manage the context window on a rolling “first in,first out “basis

• Progressive token accumulation: user message and assistant response accumulates within the context window and previous is preserved
• Context window capacity:
• Input-Output flow: Each turn consists of
  • input phase
  • output phase

Everything in the turns counts:

• the system prompt,every message in messages(including tool results,images, and documents) and your tool definition
• the output LLM generates, including the extended thinking

If you use prompt caching, the input count is split across input_tokens, cache_read_input_tokens, and cache_creation_input_tokens

Context window sizes by model

the max length of context window is up to the model

The context window with thinking

Thinking tokens are a subset of your max_tokens parameter, are billed as output tokens . Whether thinking blocks from previous assistant turns stay in the context window depends on the model. Most of LLMs keep previous thinking blocks by default, and they count toward the context window like any other input tokens.

The following diagram shows how tokens are managed when thinking is enabled on a model that strips previous thinking blocks: https://platform.claude.com/docs/images/context-window-thinking.svg

Billing: Thinking tokens are billed as output tokens once, when they are generated. On models that keep previous thinking blocks, the kept blocks are then part of later requests’ input and are billed as input tokens, like the rest of the conversation history.

The context with thinking and tool use

The following diagram illustrates how tokens are managed when you combine thinking with tool use on a model that strips previous thinking blocks: https://platform.claude.com/docs/images/context-window-thinking-tools.svg

A little bit different from my previous thought

1. First turn architecture

   • Input components: Tools configuration and user message
   • Output components: Thinking + text response + tool use request
   • Token calculation: All input and output components count toward the context window, and all output components are billed as output tokens.

2. Tool result handling (turn 2)

   • Input components: Every block in the first turn and the tool_result. You must return the thinking block with the corresponding tool results. This is the only case where you have to return thinking blocks.
   • Output components: After tool results have been passed back to Claude, Claude responds with only text (no additional thinking until the next user message, unless interleaved thinking is enabled).
   • Token calculation: All input and output components count toward the context window, and all output components are billed as output tokens.

3. New user turn (turn 3)

   • Input components: All inputs and the output from the previous turn are carried forward. The thinking block from the completed tool use cycle no longer has to stay in context: on models that strip previous thinking blocks, the API drops it automatically when you pass it back, and on models that keep previous thinking blocks, it stays unless you clear it with thinking block clearing. This is also where you add the next user turn.
   • Output components: Because there is a new user turn outside the tool use cycle, Claude generates a new thinking block and continues from there.
   • Token calculation: On models that strip previous thinking blocks, the previous thinking tokens no longer count toward the context window. All other previous blocks still count toward the context window, as does the thinking block in the current assistant turn.

Interleaved thinking:

Let LLMs think between tool calls.,including after it receives tool results

Context awareness

these models track their remaining context window (their “token budget”) throughout a conversation.

How it works

In the system prompt of every request, the API gives Claude its total context window:

1

<budget:token_budget>200000</budget:token_budget>

After each tool call, the API gives Claude an update on its remaining capacity:

1

<system_warning>Token usage: 35000/200000; 165000 remaining</system_warning>

For agents that span multiple sessions, design your state artifacts so that context recovery is fast when a new session starts. The memory tool’s multisession pattern walks through a concrete approach. See also Effective harnesses for long-running agents.

Manage context with compaction

If your conversations regularly approach context window limits, use server-side compaction. Compaction automatically summarizes earlier parts of the conversation on the server, so the conversation can continue past the context window limit. It is available in beta for Claude 4.6 and later models and Claude Mythos Preview.

• Tool result clearing: Clear old tool results in agentic workflows
• Thinking block clearing: Manage thinking blocks when you use extended thinking

Context window overflow behavior

If the input alone already exceeds the model’s context window, the API returns a 400 invalid_request_error (“prompt is too long”) on every model.

On Claude 4.5 models and newer, if input tokens plus max_tokens exceeds the context window size, the API accepts the request. If generation then reaches the context window limit, it stops with stop_reason: "model_context_window_exceeded"

On earlier models, the API returns a validation error instead.

Automatically manage conversation context as it grows with context editing 注意断句

Overview

The strategies on this page are useful for specific scenarios where you need more fine-grained control over what content is cleared.

更适用于在特定场景中,你需要清楚哪些内容进行更加细粒度的控制

Context editing allows you to selectively clear specific content from conversation history as it grows. Beyond optimizing costs and staying within limits, this is about actively curating what Claude sees: context is a finite resource with diminishing returns, and irrelevant content degrades model focus. Context editing gives you fine-grained runtime control over that curation. For the broader principles behind context management, see Effective context engineering.

上下文编辑允许你选择性地清楚会话内容

除了控制成本和避免超出限制以外,这也是在主动管理Claude对于上下文的可见性:上下文是一种有限的资源,且存在边界效应;无关内容会导致模型降级和降低注意力

主要提供了三种方法:

• Tool result clearing: 适用于工具调用频繁且旧的工具调用结果不再需要的场景
• Thinking block clear:可有有选择性地保留最近地think block
• Client-side SDK compaction:An SDK-based alternative for summary-based context management (server-side compaction is generally preferred)官方Compaction的替代

Server-side Strategies

Tool result clearing

when conversation context grows beyond your configured threshold Best for agentic workflows with heavy tool use and Older tool results are no longer needed once Claude has processed them.

通过清楚oldest tool result并且使用占位文本进行替换来告知模型结果已经被移除 默认情况下,只有results会被移除,但是你也可以在配置文件中将clear_tool_call 甚至为True,这样会同时清楚tool_calls以及调用时的tool use parameters和tool_results

Think block clearing

The clear_thinking_20251015 strategy manages thinking blocks in conversations when extended thinking is enabled. This strategy gives you control over thinking preservation: you can choose to keep more thinking blocks to maintain reasoning continuity, or clear them more aggressively to save context space. 用于启用了extend_thinking的会话中的think_blocks; 你可以选择保留更多的thinking_block也可以积极清理来节省上下文空间

Default behavior: The default varies by model class.

You can override the default,if your code runs across multiple models tiers,ser **keep **explicitly rather than relying on the per-model default

With interleaved thinking,there includes multiple thinking blocks

Context editing happens server-side

Context editing is applied server-side before the prompt reaches Claude.

Build a session-level mode that grants standing consent for multiagent fan-out, switched on and off with mid-conversation system messages.

会话级别的;持续性凭证;multiagent;mid-conversation

编排模式(orchestration mode)是一种会话级开关:当他开启时,model will pay most efforts to every request,先自行见检查和分析任务,然后默认将工作拆分给几个并发的子智能体 It is not an API parameter. It is built entirely from documented pieces

1. 1.An effort level: requests run at a documented effort value such as Max,there no other level higher than it
2. A mode reminder: a mid-conversation system message tells the model the mode is active, 每个几次对话发送一行通知 当推出时,会发出退出通知
3. Standing consent(持续授权) in the tool description:该模式开启时,对于每一个实质性的任务,都应该在不经过询问的情况下自己设计和运行一套工作流程

该模式会造成成本的增加,原因有二:Max effort level;Multi-agents,subagents

Ser up the loop

MAX_CONCURRENT限制并发的subagents的数量 MAX_TOTAL_SUBTASKS在一次工作流中的排队的任务的数量 当DOC_TEST_MODE开启时,会将循环次数限制为一次,这样地自动化文本测试框架就可以快速完成编译并快速结束,而不需要运行完整地编排流程 但是在运行个人的用例时,请不要开启或者leave it unset,不然你就会进入文档测试模式

Define the mode reminder

这些reminders都很简短,他们切换模式并且指向tool description 完整的文本只在刚开启时发送一次;the refresher每隔几轮对话发一次;退出通知只在退出的时候发送

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

MODE_ENTER = (
    "Orchestration mode is on: optimize for the most exhaustive, correct answer rather than "
    "the fastest one. Use the Workflow tool on every substantive task, sized to the problem's "
    "natural decomposition rather than the maximum the tool allows. See the Workflow tool's "
    "description for standing consent, granularity guidance, and quality patterns. Work solo "
    "only on conversational or trivial turns."
)
MODE_REFRESH = (
    "Orchestration mode is still on. Use the Workflow tool; see its standing consent section."
)
MODE_EXIT = (
    "Orchestration mode is off. The Workflow tool's standard opt-in rule applies again."
)

Grant standing consent in the tool description

Workflow工具承载了真正的行为约定(behavioral contract):包括准入规则,持续授权,模型可以采用的质量保障模式等等 subagent还会获得一个report_findings工具,将他们的结果以JSON结构化的形式返回;而bash工具则是anthropic定义的bash_20250124工具,在本地运行

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66

WORKFLOW_TOOL = {
    "name": "Workflow",
    "description": (
        "Orchestrate a multiagent workflow: split a large task into independent subtasks "
        "and run them as parallel agents, then collect their results.\n\n"
        "Opt-in: only use this tool when the user explicitly asks for a workflow, or when a "
        "system message confirms that orchestration mode is on.\n\n"
        "Quality patterns: adversarial verification (a second wave of agents checks the first "
        "wave's findings against the source), a completeness critic (one agent hunts for what "
        "the others missed), and multiphase sequencing (understand, design, implement, and "
        "review as separate workflow calls, reading results between phases). A useful default "
        "is hybrid: scout inline first to discover the work-list, then fan out over it.\n\n"
        "Granularity: scope each subtask to a distinct concern, component, or question rather "
        "than per line or per file section. Scale the count to what the user asked for: a "
        "focused review of a module of a few hundred lines rarely needs more than about ten "
        "subtasks; a broad audit of a large codebase can justify more.\n\n"
        "Standing consent: while a system message confirms orchestration mode is on, that "
        "opt-in is standing. Author and run a workflow for every substantive task by default, "
        "and lean toward verifying findings adversarially. Work solo only on conversational "
        "turns or trivial mechanical edits. When a system message says the mode is off, "
        "revert to the opt-in rule above."
    ),
    "input_schema": {
        "type": "object",
        "properties": {
            "subtasks": {
                "type": "array",
                "items": {"type": "string"},
                "description": "Independent subtask prompts to run as parallel agents",
            }
        },
        "required": ["subtasks"],
    },
}

BASH_TOOL = {"type": "bash_20250124", "name": "bash"}

REPORT_TOOL = {
    "name": "report_findings",
    "description": (
        "Report the final findings for your subtask. Call this exactly once, when you are "
        "done investigating; it ends your task."
    ),
    "input_schema": {
        "type": "object",
        "properties": {
            "summary": {"type": "string", "description": "Two or three sentences of synthesis"},
            "findings": {
                "type": "array",
                "items": {
                    "type": "object",
                    "properties": {
                        "claim": {"type": "string", "description": "The finding, one sentence"},
                        "evidence": {
                            "type": "string",
                            "description": "How it was verified (file, line, or command output)",
                        },
                        "severity": {"type": "string", "enum": ["high", "medium", "low", "info"]},
                    },
                    "required": ["claim", "evidence", "severity"],
                },
            },
        },
        "required": ["summary", "findings"],
    },
}

Run the bash tool locally

bash处理器运行命令并带运行超时时间,捕获合并后的stderr和stdout并且截断输出,以防止单次输出占满上下文窗口 命令会在启动的文件夹内运行 当DOC_TEST_MODE开启时,框架会给bash命令提供一个临时的,小型的测试目录,并且在退出时删除 没有沙箱, 示例每次调用都会在一个全新的子shell中运行,而不是维持持久化对话 而在生产环境中的agent则应该接入到一个长期运行的subshell

Run one subagent

每个workflow的子任务都会称为一个独立的小型的带有bash工具的agent循环,并且effort与主循环相同 每个请求都会设置一个单独的超时时间,这样即使链接中断也只是子agent受影响而不会让整个工作流瘫痪停滞

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52

def run_subagent(model: str, prompt: str) -> str:
    """One subagent: a small nested agent loop with the bash tool plus report_findings.
    Subagents inherit the main loop's effort level."""
    subagent_system = (
        "You are one agent in a larger parallel fan-out, assigned a single subtask. "
        "Investigate it directly, using bash to check facts rather than guessing, and finish "
        "by calling report_findings exactly once. Return findings, not narration."
    )
    messages = [{"role": "user", "content": prompt}]
    for _ in range(MAX_SUBAGENT_TURNS):
        with client.messages.stream(
            model=model,
            max_tokens=64000,
            system=subagent_system,
            output_config={"effort": EFFORT},
            tools=[BASH_TOOL, REPORT_TOOL],
            messages=messages,
            timeout=REQUEST_TIMEOUT_SECONDS,
        ) as stream:
            response = stream.get_final_message()
        messages.append({"role": "assistant", "content": response.content})
        if response.stop_reason == "pause_turn":
            continue
        if response.stop_reason != "tool_use":
            text = "".join(block.text for block in response.content if block.type == "text")
            if response.stop_reason == "max_tokens":
                text += "\n\n(warning: subagent response was truncated at max_tokens)"
            return text
        tool_results = []
        report = None
        for block in response.content:
            if block.type != "tool_use":
                continue
            if block.name == "report_findings":
                report = json.dumps(block.input, indent=2)
                output, is_error = "Findings recorded.", False
            elif block.name == "bash":
                output, is_error = handle_bash_block(block)
            else:
                output, is_error = f"unknown tool: {block.name}", True
            tool_results.append(
                {
                    "type": "tool_result",
                    "tool_use_id": block.id,
                    "content": output,
                    "is_error": is_error,
                }
            )
        if report is not None:
            return report
        messages.append({"role": "user", "content": tool_results})
    return "(subagent hit the turn limit before finishing)"

Journal results so reruns resume(记录结果这样重新运行时可以接着执行)

该模式会开启许多子agent,重启的成本很高.而一个基于内容寻址的日志会使他具有幂等性(idempotent) 在分发智能体之前,先计算器prompt的SHA-256,并且在本地的JSON文件中查找这个哈希值,如果存在,则直接返回之前的结果 如果运行过程被中断,也只有那些从未被完成的子任务才会被重启 该日志会在不同的运行之间去重,但是不会在同一次fan-out中去重 如果想要重头开始执行,只需要删除这个日志文件即可

Fan out,then verify

扇出最多接受 MAX_TOTAL_SUBTASKS 个 prompt，并通过日志（journal）运行这些任务，同时最多允许 MAX_CONCURRENT 个任务处于执行中（PHP 版本则是顺序执行）。 它会隔离各个子任务的失败：某个子智能体出错时，只会将其降级为一个错误字符串，而不会导致整个运行流程终止。 第一轮执行完成后，会进行第二轮扇出，并复用相同的子智能体执行路径，对第一轮的每个结果进行尝试反驳： 每个验证器（verifier）都会根据原始来源重新推导和核实这些结论；如果无法确定，则默认判定为**“反驳（refuted）”**。 最终，原始结果和对应的验证结论（verdict）都会返回给编排器（orchestrator），由编排器综合考虑两者。

Toggle the mode with mid-conversation system messages

通过 mid-conversation来切换模式 先追加user_message,然后才是system-message(可能full text,也可能是定时的refresher,也可能是退出通知) 并且这满足了它的位置限制: 某个user turn之后(role: user的tool call也可以视为user turn),对应的assistant之前.或者最后面: 原因之前有,主要是为了prompt caching

Change system instructions or tool availability partway through a conversation without invalidating the cached prefix that came before them. 在不使缓存失效的情况下,更改系统指令或者工具可用性 system instructions normally live in the top-level system field it is great for prompt caching: because it is stable prefix Mid-conversation system message will not change the cached prefix ,and you append a {“role”: “system”},and this instruction is still applied ad system instruction rather than as a ordinary user text

Mid-conversation tool changes

The tools array sets even earlier than the top-level system field. so editing it will make the whole cache invalided

Mid-conversation tool changes use tool_addition and tool_removal to offer a tool to model or withdraw it from a specific point in the conversation onward. The tool array never changes , so the prefix stays the same.

tool_addition and tool_removal are content blocks in the content array of role : “system” message and they can be mixed with text blocks The message follows the same placement rules as any mid-conversation system message

Each block’s tool field references a tool rather than defining one: {"type": "tool_reference", "name": "..."} names a tool declared in the request’s tools array, and MCP connector tools can be referenced individually with mcp_tool_reference (server_name and name) or as a whole toolset with mcp_toolset_reference (server_name). Referencing a name that is not declared in tools returns a 400 error.

可以使用defer_loading: true 来使得在tools offer的tool暂时隐藏,知道tool_addition激活它 并且tool_addition可以重新提供之前被 tool_removalde撤回的tool

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

client = anthropic.Anthropic()

response = client.beta.messages.create(
    model="claude-opus-5",
    max_tokens=1024,
    betas=["mid-conversation-tool-changes-2026-07-01"],
    # The full tool set is declared up front and never changes, so the
    # cached prefix stays intact.
    tools=[
        {
            "name": "get_weather",
            "description": "Get the current weather for a location.",
            "input_schema": {
                "type": "object",
                "properties": {
                    "location": {"type": "string", "description": "City name"},
                },
                "required": ["location"],
            },
        },
    ],
    messages=[
        {
            "role": "user",
            "content": "Say OK.",
        },
        # Withdraw get_weather from this point onward. The block references
        # the tool by name instead of editing `tools`, so earlier turns stay
        # byte-identical and the cache still hits.
        {
            "role": "system",
            "content": [
                {
                    "type": "tool_removal",
                    "tool": {"type": "tool_reference", "name": "get_weather"},
                },
            ],
        },
    ],
)

for block in response.content:
    if block.type == "text":
        print(block.text)

1. 是role: system的content块
2. parameter: tool_reference instead of tool_name

When to use a mid-conversation system message

prompt caching hashes the requests prefix in order: tools,system,message Mid-conversation system let you add the instruction at the end of the message history instead great for matching the unchanged parts

A few situations where this matters:(随便看看就行)

• Mid-session policy or persona changes. A long agentic session needs a new constraint (“from now on, write all SQL as parameterized queries”) after dozens of cached turns. Adding it to the top-level system field would re-process the entire history.
• Per-turn context that must be authoritative. You want to inject a freshness note, a session deadline, or a tool-availability change with system-level weight, and it changes too often to live in the cached prefix.
• Per-turn reminders that shouldn’t pile up. A harness nudges the model after each batch of tool results (“request independent reads together”, “the user hasn’t heard from you in a while”) and wants the model to see only the newest copy. A turn-scoped system message renders for one turn and then costs nothing, without deleting anything from the history.
• State changes your application observes. Your application notices something Claude should treat as an operator-level fact: files changed on disk, the user toggled an auto-approve setting, available tools changed, or the remaining token budget dropped below a threshold.
• User input that should not interrupt an agentic loop. A user types a follow-up while Claude is still executing tools for the previous request. Relaying it as a system message after the next tool result lets Claude fold the new input into the work it is already doing, instead of treating it as a fresh request to switch to. See Placement after tool results.
• Mode switches that grant standing permissions. A session-level mode can use a mid-conversation system message to grant standing consent to an expensive capability, such as automatically launching multiagent workflows, with a short refresher every several turns and an exit notice when the mode is turned off. For a worked example, see Build an orchestration mode.

A mid-conversation system message keeps that operator-level priority without paying the cache-miss cost of editing the top-level system field.

How it works

Add a message with “role” : “system” to the message array Use a plain string or content blocks for content , the same as a user or assistant turn.

when the later and the earlier conflict, the later will precedence the earlier mid-conversation system messages take precedence over the top-level system field for the turns that follow them

The position: A mid-conversation system message must immediately follow a user turn (or an assistant turn ending in a server tool result), and must either be the last entry in messages or be immediately followed by an assistant turn. Any other position, including between an assistant tool_use block and the tool_result that answers it, returns a 400 error.

Placement after tool results

Phrase the system content as context rather than as a command that overrides the user. State the fact (“new input arrived from the user: X”, “the remaining token budget is now Y”) and let Claude act on it. Claude is trained to resist instructions that appear to work against the user, and that protection still applies to the system role, so language such as “ignore what the user said” is less effective than stating what changed.

Turn-scoped system messages

要让有个role: system消息只作用与当前伦茨,可以设置它的clear_at,它的两个可选值: “never”:默认值,该小心会包含在它的每一次请求中 next_user_message:限定为当前伦茨的turn-scope,只有当它后面没有role: user的消息是,其文本才会被渲染 而一个只包含tool_result block的用户小心也会被视为role: user消息 一旦后面出现了用户消息,他就会被清楚:它仍然保留在message array但是不会被渲染也不会被计入input_token

主要用途是在 tool loop中提供每一轮次的提醒,这个可以从上面那个(而一个只包含tool_result block的用户小心也会被视为role: user消息)特性进行设计

模型只能看到最后一个user message后的副本,因此消息不会叠加

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68

{
  "model": "claude-fable-5-1",
  "max_tokens": 16000,
  "messages": [
    { "role": "user", "content": "Fix the failing test." },
    {
      "role": "assistant",
      "content": [
        { "type": "thinking", "thinking": "", "signature": "..." },
        {
          "type": "tool_use",
          "id": "toolu_01",
          "name": "read_file",
          "input": { "path": "test_auth.py" }
        }
      ]
    },
    {
      "role": "user",
      "content": [{ "type": "tool_result", "tool_use_id": "toolu_01", "content": "..." }]
    },
    {
      "role": "system",
      "clear_at": "next_user_message",
      "content": "Request independent reads in one turn."
    },
    {
      "role": "assistant",
      "content": [
        { "type": "thinking", "thinking": "", "signature": "..." },
        {
          "type": "tool_use",
          "id": "toolu_02",
          "name": "read_file",
          "input": { "path": "auth.py" }
        },
        {
          "type": "tool_use",
          "id": "toolu_03",
          "name": "read_file",
          "input": { "path": "tokens.py" }
        }
      ]
    },
    {
      "role": "user",
      "content": [
        { "type": "tool_result", "tool_use_id": "toolu_02", "content": "..." },
        {
          "type": "tool_result",
          "tool_use_id": "toolu_03",
          "content": "...",
          "cache_control": { "type": "ephemeral" }
        }
      ]
    },
    {
      "role": "system",
      "clear_at": "next_user_message",
      "content": "Request independent reads in one turn."
    },
    {
      "role": "system",
      "clear_at": "next_user_message",
      "content": "The shell exited with status 137."
    }
  ]
}

Rules for turn-scoped messages:

• Re-send cleared messages verbatim. A cleared message is still part of the conversation history. Rebuilding it from current state (a fresh token count, a timestamp), dropping it as redundant, or changing its clear_at value is an edit to an earlier message. The prompt cache misses from that point, and on Claude Fable 5.1 every thinking block produced after it fails the conversation check.
• Text only. content is one or more text blocks (or a string). tool_addition and tool_removal blocks return a 400 error on a turn-scoped message, and so does output_config. Use a separate role: "system" message without clear_at for those.
• No cache_control on its blocks. A cleared message is never part of a cache key, so a breakpoint on it could never match. Put the breakpoint on the last block of the preceding user turn instead, as the example does. The top-level automatic caching field skips turn-scoped messages when it picks a breakpoint. On the request that clears a message, the reusable cached prefix ends at the user turn before it, so only the one assistant turn between that message and the new user message is reprocessed.
• Placement rules still apply, cleared or not. A turn-scoped message must follow a user turn (or an assistant turn ending in a server tool result) and precede an assistant turn or end the array, like any mid-conversation system message. One that ends the array always renders. One followed directly by another user message is a 400 error, not a cleared message: put all of a tool round’s results in one user message and the reminders after it.
• Assistant turns don’t clear it. A prefilled or paused assistant turn after the message, or a server-side tool loop, adds no user message, so the message still renders on that continuation. To keep a reminder in view through a client-side tool loop, append it again after each tool_result message.
• Token counting follows what renders. A cleared message adds nothing to usage.input_tokens or to a token count.
• Imported history. In a transcript you construct in one step (few-shot examples, a migrated conversation), a turn-scoped message that already has an assistant turn and a user message after it is cleared from the first request and never renders. That is the right state for a per-turn reminder you are carrying over. Leave clear_at unset only on a message the model should see on every request.

Combining with prompt caching

对话中途的系统消息（Mid-conversation system messages）与提示缓存（prompt caching）是专门设计为可以结合使用的：

• 显式启用缓存。 只有当请求中包含 cache_control 时才会发生缓存，这可以是顶层的自动缓存（automatic caching）字段，也可以是内容块上的显式缓存断点（explicit breakpoint）。对话中途的系统消息本身不会自动创建缓存条目；如果没有启用缓存，也就没有可以保留的缓存收益。
• 像往常一样缓存稳定前缀。 将 cache_control 放在跨请求保持不变的最后一个内容块上。这个位置可以是顶层 system 字段的末尾、工具定义的末尾，或者消息历史中的某个稳定位置。
• 将系统消息追加在缓存断点之后。 因为它位于已经缓存的前缀之后，所以不会改变前缀的哈希值，缓存仍然可以命中。
• 对话中途的系统消息本身也可以被缓存。 一旦它进入对话，就会成为稳定历史的一部分。下一轮请求中，你可以将缓存断点向后移动到该系统消息之后（或者依赖自动缓存（automatic caching）自动将断点移动到那里），这样该系统消息就会像其他对话轮次一样从缓存中读取。

Limitations

• Not for the first message. A system message that carries content cannot be the first entry in messages. Use the top-level system field for instructions that apply from the very start.
• Placement is constrained. A system message that carries content (text, tool_addition, or tool_removal blocks) must immediately follow a user turn (including a user turn that carries tool_result blocks) or an assistant turn ending in a server tool result, and must precede an assistant turn or end the array. It cannot sit between a tool_use block and its tool_result. Placing it elsewhere returns a 400 error. One exception: tool_addition and tool_removal blocks are not accepted immediately after a paused assistant turn (one ending in a server tool result), though text blocks are; resume the paused turn first, then send the tool change in the next system message. A message with empty content that only sets output_config.effort renders nothing at its position and is accepted anywhere in messages, including first or between an assistant turn and a user turn. Consecutive system messages are judged together, so adding a text-carrying message next to an effort-only one makes the whole group follow the content rule.
• Turn-scoped messages are text-only and re-sent verbatim. A clear_at: "next_user_message" message carries no tool_addition, tool_removal, output_config, or cache_control, and once cleared it must stay in messages byte-for-byte on later requests. See Turn-scoped system messages.
• Not a place for untrusted content. Claude treats system content as operator instructions and follows it. Do not place text from outside the conversation, such as raw tool output, retrieved documents, or web content, directly in a system message; doing so gives that text operator-level authority. Keep that data in tool_result blocks and continue to follow Mitigate jailbreaks and prompt injections.

原文：Mid-conversation system messages and tool changes

在对话进行到一半时更改 system 指令或工具可用性，同时不使此前已缓存的 prefix 失效。

关于 zero data retention (ZDR) 如何适用于本功能，参见 API and data retention。

System 指令通常放在顶层 system 字段中，位于对话里所有 message 之前。这个位置非常适合 prompt caching：system prompt 属于稳定的 prefix，因此后续 turn 可以命中 cache。但如果你是在会话进行到一半才发现需要某条指令，这个位置就不合适了——因为修改顶层 system 字段会改变 prompt 的最开头，从而使其后全部内容的 cache 失效。

Mid-conversation system messages 补上了这个缺口。你不再去改顶层 system 字段，而是在新指令开始生效的那个对话位置，追加一条 {"role": "system"} message。cached prefix 保持不变，下一次请求仍会从 cache 读取它；同时，新指令仍按 system instruction 生效，而不是普通的 user 文本。

Mid-conversation system messages 可用于 Claude API、Claude in Amazon Bedrock 以及 Google Cloud。

该功能适用于 Claude Fable 5.1、Claude Mythos 5.1、Claude Fable 5、Claude Mythos 5、Claude Opus 4.8 和 Claude Opus 5。使用 mid-conversation system messages 不需要 beta header。Claude Sonnet 5 不支持此功能，请改用顶层 system 字段。

Mid-conversation tool changes 目前处于 beta，需要 mid-conversation-tool-changes-2026-07-01 这个 beta header。可用模型与平台同上：Claude API、Amazon Bedrock 和 Google Cloud。

Turn-scoped system messages（clear_at）目前处于 beta，需要 mid-conversation-system-clear-at-2026-08-21 这个 beta header；可用模型与平台与 mid-conversation system messages 相同。

Mid-conversation tool changes

tools 数组在 hashed request prefix 中的位置比顶层 system 字段还要靠前，因此改动它会使整段对话的 prompt cache 失效。Mid-conversation tool changes 是 mid-conversation system messages 在工具侧的对应能力：不必把工具列表在整段对话生命周期内写死，而是可以在 turn 与 turn 之间改变向模型提供哪些工具——先在 tools 里一次性声明完整工具集，再通过 tool_addition 和 tool_removal 这两个 block，从对话中的某个位置起向模型提供某工具，或将其撤回。tools 数组本身始终不变，因此 cached prefix 保持完好。

tool_addition 和 tool_removal 是 role: "system" message 的 content 数组中的 content block，可以与同一条 message 里的 text block 混用。该 message 的放置规则与任意 mid-conversation system message 相同（参见 Limitations），变更从该位置起向后生效。每个 block 的 tool 字段是引用工具，而不是定义工具：{"type": "tool_reference", "name": "..."} 指向请求 tools 数组中已声明的工具；MCP connector 工具可以用 mcp_tool_reference（server_name 和 name）逐个引用，也可以用 mcp_toolset_reference（server_name）引用整个 toolset。引用一个未在 tools 中声明的名字会返回 400 错误。

tools 中声明的每个工具从对话一开始就会提供给模型，除非声明时带上 defer_loading: true——后者会把它扣住，直到某条 tool_addition block 把它亮出来。tool_addition 也可以重新提供此前被 tool_removal 撤回的工具。

原文示例覆盖 cURL / CLI / Python / TypeScript / C# / Go / Java / PHP / Ruby。下文以 Python 为代表；其余语言的请求结构相同。使用本功能时需带上 beta header mid-conversation-tool-changes-2026-07-01。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

client = anthropic.Anthropic()

response = client.beta.messages.create(
    model="claude-opus-5",
    max_tokens=1024,
    betas=["mid-conversation-tool-changes-2026-07-01"],
    # 完整工具集在一开始就声明，且之后不再改动，因此 cached prefix 保持完好。
    tools=[
        {
            "name": "get_weather",
            "description": "Get the current weather for a location.",
            "input_schema": {
                "type": "object",
                "properties": {
                    "location": {"type": "string", "description": "City name"},
                },
                "required": ["location"],
            },
        },
    ],
    messages=[
        {
            "role": "user",
            "content": "Say OK.",
        },
        # 从这一位置起撤回 get_weather。该 block 按名字引用工具，而不是去改 `tools`，
        # 因此更早的 turn 仍逐字节相同，cache 仍然命中。
        {
            "role": "system",
            "content": [
                {
                    "type": "tool_removal",
                    "tool": {"type": "tool_reference", "name": "get_weather"},
                },
            ],
        },
    ],
)

for block in response.content:
    if block.type == "text":
        print(block.text)

Mid-conversation tool changes 目前处于 beta。使用时请在请求中带上 beta header mid-conversation-tool-changes-2026-07-01。

When to use a mid-conversation system message

Prompt caching 按如下顺序对 request prefix 做哈希：先 tools，再 system，再 messages。要命中 cache，prefix 必须与最近一次请求逐字节完全一致，直到 cache breakpoint 为止。

这个顺序意味着顶层 system 字段几乎位于 hashed prefix 的最开头。对它的任何改动——哪怕只是追加一句话——都会得到不同的 hash，于是这次请求会 miss cache：system prompt 本身，以及其后所有已缓存的 message，都无法复用。

Mid-conversation system messages 让你把指令加在 message history 的末尾。新指令之前的一切都没有改动，因此已有的 cache entry 仍然匹配，只有这条新 message 会作为 fresh input 被处理。

以下几种情况尤其值得这样做：

• 会话中途的 policy 或 persona 变更。 一段很长的 agentic session 已经缓存了几十个 turn，这时才需要加上新约束（例如 “from now on, write all SQL as parameterized queries”）。若把它写进顶层 system 字段，整段历史都要重新处理。
• 必须具备权威性的 per-turn 上下文。 你想注入一条 freshness note、session deadline，或工具可用性变更，并且希望它带有 system 级别的权重；同时它变化太频繁，不适合放进 cached prefix。
• 不应越积越多的 per-turn reminder。 harness 在每批 tool result 之后提醒模型（例如 “request independent reads together”、“the user hasn’t heard from you in a while”），并且希望模型只看到最新的那一份。Turn-scoped system message 只在一个 turn 内渲染，之后不再消耗 token，也不必从 history 里删除任何内容。
• 应用层观察到的状态变化。 你的应用注意到某些事实，希望 Claude 按 operator 级别对待：磁盘上的文件变了、用户切换了 auto-approve 设置、可用工具变了，或剩余 token budget 掉到阈值以下。
• 不应打断 agentic loop 的用户输入。 用户在 Claude 仍在为上一次请求执行工具时输入了 follow-up。把它作为 system message 接在下一条 tool result 之后转发，可以让 Claude 把新输入并入正在进行的工作，而不是把它当成一次需要立刻切换过去的全新请求。参见 Placement after tool results。
• 授予 standing permission 的 mode 切换。 会话级 mode 可以用 mid-conversation system message 为某项昂贵能力授予 standing consent（持续性授权），例如自动启动 multiagent workflow；每隔若干 turn 发一条简短 refresher，mode 关闭时再发一条退出通知。完整示例见 Build an orchestration mode。

上述所有情况，你也可以把指令放进普通 user message，而 Claude 确实会遵循出现在 user turn 中的指令。差别在于优先级：user message 被视为来自终端用户，system message 被视为来自你——应用的 operator。两者冲突时，system 指令优先。因此，对于即便终端用户提出不同要求也应继续成立的 operator 级事实与约束，请使用 system role。Mid-conversation system message 能保住这种 operator 级优先级，又不必付出改动顶层 system 字段所带来的 cache miss 成本。

How it works

向 messages 数组追加一条 "role": "system" 的 message。content 可以用纯字符串，也可以用 content block，与 user 或 assistant turn 相同。该指令从对话中的这个位置起向后生效。指令冲突时，后出现的 system message 优先于更早的；对跟随其后的那些 turn 而言，mid-conversation system messages 也优先于顶层 system 字段。

对整段对话都应生效的指令，仍然可以写在顶层 system 字段。Mid-conversation system messages 留给那些稍后才变得相关的指令，或那些你希望追加、同时又不想让 cached prefix 失效的指令。

role: "system" message 还可以携带 output_config.effort，从下一个 user turn 起改变 effort 级别。该能力在 Claude API 与 Google Cloud 上的 Claude Fable 5.1、Claude Mythos 5.1 和 Claude Opus 5 处于 beta，需要 mid-conversation-output-config-2026-07-01 这个 beta header。参见 Per-message effort。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

client = anthropic.Anthropic()

response = client.messages.create(
    model="claude-opus-5",
    max_tokens=1024,
    # 自动 prompt caching：每次请求都会缓存截至目前的对话，
    # 下一次请求则从 cache 读取未改动的 prefix。
    cache_control={"type": "ephemeral"},
    system="You are a code review assistant. Be concise.",
    messages=[
        {
            "role": "user",
            "content": "Review process() in utils.py for performance issues.",
        },
        {
            "role": "assistant",
            "content": "The list comprehension is fine for small inputs. For large inputs, consider a generator to avoid materializing the full list.",
        },
        {
            "role": "user",
            "content": "Now review the calling code that invokes process().",
        },
        # reviewer 在会话中途意识到：所有建议还必须满足团队严格的 typing 政策。
        # 把指令追加在这里，更早的 turn 仍逐字节相同，
        # 因此上一次请求缓存下来的 prefix 仍会从 cache 读取。
        {
            "role": "system",
            "content": "From now on, every suggestion must include explicit type annotations.",
        },
    ],
)

for block in response.content:
    if block.type == "text":
        print(block.text)

这个例子通过顶层 cache_control 字段启用了 automatic caching。Prompt caching 是 opt-in 的：如果请求既没有 cache_control 字段（无论是 automatic，还是 explicit breakpoint），就不会有任何内容被缓存，每次请求都按常规 input token 价格为整段对话付费。启用 caching 后，追加这条 system message 不会改动已经缓存的那些 turn，因此携带新指令的那次请求仍会从 cache 读取它们，而不是重新处理。Caching 还要求对话达到 minimum cacheable prompt length；像上面这样短的例子会低于该阈值，所以在对话变长之前，cache_creation_input_tokens 和 cache_read_input_tokens 都会保持为 0。

一条 mid-conversation system message 必须紧跟在某个 user turn 之后（或紧跟在以 server tool result 结尾的 assistant turn 之后），并且它要么是 messages 的最后一项，要么紧跟着一个 assistant turn。携带 tool_result block 的 user message 也算：在 agentic loop 中，你可以把 system message 紧接在 tool result 之后、Claude 的下一 turn 之前。任何其他位置——包括夹在 assistant 的 tool_use block 与回答它的 tool_result 之间——都会返回 400 错误。

Placement after tool results

在 agentic loop 中，system message 应放在投递 tool result 的那条 user message 之后。你的应用也可以在这里转发用户在 Claude 工作期间输入的内容，从而在不重启当前 turn 的情况下吸收新上下文：

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

[
  { "role": "user", "content": "Run the test suite and fix any failures." },
  {
    "role": "assistant",
    "content": [{ "type": "tool_use", "id": "toolu_01", "name": "run_tests", "input": {} }]
  },
  {
    "role": "user",
    "content": [
      { "type": "tool_result", "tool_use_id": "toolu_01", "content": "12 passed, 0 failed" }
    ]
  },
  {
    "role": "system",
    "content": "The user sent the following message while you were working: also update the changelog before you finish."
  }
]

把 system content 写成上下文，而不是一条覆盖用户意图的命令。陈述事实（“new input arrived from the user: X”、“the remaining token budget is now Y”），然后让 Claude 据此行动。Claude 被训练为抵制那些看起来与用户作对的指令，这种保护对 system role 同样生效，因此 “ignore what the user said” 这类措辞，效果不如直接说明发生了什么变化。

这种模式用于转发本段对话终端用户的输入。不要用它传递 tool output、检索到的文档或其他第三方内容；那些内容应放在 tool_result block 中（参见 Limitations）。

Turn-scoped system messages

若要把一条 role: "system" message 限定在当前 turn，设置它的 clear_at 字段。该字段只有两个取值：

• "never"（默认）：只要请求里包含这条 message，它就会在其所在位置渲染。省略该字段效果相同。
• "next_user_message"：这条 message 是 turn-scoped 的。只有当 messages 中它后面还没有 role: "user" message 时，其文本才会被渲染。这里，只携带 tool_result block 的 user message 也算 user message。一旦后面出现了更新的 user message，该条就会被 cleared：它仍留在数组中，但不再渲染任何内容，也不再消耗 input token——本次请求以及之后每一次都是如此。

Turn-scoped system messages 目前处于 beta。请带上 beta header mid-conversation-system-clear-at-2026-08-21。不带该 header 时，clear_at 会被当作未知字段而拒绝。

1
2
3
4
5

{
  "role": "system",
  "clear_at": "next_user_message",
  "content": "First privately list what you need next; then request every item that doesn't depend on another's result in this one response."
}

主要用途是 tool loop 中的 per-turn reminder。每次希望模型看到这条提醒时，就把它追加在 tool_result message 之后，更早的副本原样留着。模型只会看到位于最后一条 user message 之后的那些副本，因此 reminder 不会越积越多。messages 中更早的内容没有任何改动，所以 prompt cache 仍然匹配。在 Claude Fable 5.1 上，这样做还能让后续 thinking block 保持有效：删除更早的 reminder 会改变那些 block 之前的对话，导致 conversation check 失败；而被 cleared 的 message 仍留在数组中，对话保持不变。

下面这次请求是 agent loop 中较后的一步。messages[3] 在更早的那次请求中被渲染过，当时它是数组的最后一项。一旦存在 messages[5]（一条更后的 user message），messages[3] 就被 cleared：cleared message 仍留在数组中，因此 messages[4] 里 thinking block 之前的对话没有变化，但模型不再看到它的文本。messages[6] 和 messages[7] 都会按顺序渲染。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68

{
  "model": "claude-fable-5-1",
  "max_tokens": 16000,
  "messages": [
    { "role": "user", "content": "Fix the failing test." },
    {
      "role": "assistant",
      "content": [
        { "type": "thinking", "thinking": "", "signature": "..." },
        {
          "type": "tool_use",
          "id": "toolu_01",
          "name": "read_file",
          "input": { "path": "test_auth.py" }
        }
      ]
    },
    {
      "role": "user",
      "content": [{ "type": "tool_result", "tool_use_id": "toolu_01", "content": "..." }]
    },
    {
      "role": "system",
      "clear_at": "next_user_message",
      "content": "Request independent reads in one turn."
    },
    {
      "role": "assistant",
      "content": [
        { "type": "thinking", "thinking": "", "signature": "..." },
        {
          "type": "tool_use",
          "id": "toolu_02",
          "name": "read_file",
          "input": { "path": "auth.py" }
        },
        {
          "type": "tool_use",
          "id": "toolu_03",
          "name": "read_file",
          "input": { "path": "tokens.py" }
        }
      ]
    },
    {
      "role": "user",
      "content": [
        { "type": "tool_result", "tool_use_id": "toolu_02", "content": "..." },
        {
          "type": "tool_result",
          "tool_use_id": "toolu_03",
          "content": "...",
          "cache_control": { "type": "ephemeral" }
        }
      ]
    },
    {
      "role": "system",
      "clear_at": "next_user_message",
      "content": "Request independent reads in one turn."
    },
    {
      "role": "system",
      "clear_at": "next_user_message",
      "content": "The shell exited with status 137."
    }
  ]
}

Turn-scoped message 的规则：

• 按原文原样重新发送已 cleared 的 message。 一条 cleared message 仍是 conversation history 的一部分。按当前状态重建它（新的 token 计数、时间戳）、因为它冗余而丢掉它、或改它的 clear_at 值，都等于在编辑更早的 message。Prompt cache 会从该点起 miss；在 Claude Fable 5.1 上，它之后产生的每一个 thinking block 都会通不过 conversation check。
• 只允许文本。 content 是一个或多个 text block（或字符串）。在 turn-scoped message 上使用 tool_addition、tool_removal 会返回 400 错误，output_config 同样不行。这些内容请用另一条不带 clear_at 的 role: "system" message。
• 其 block 上不能带 cache_control。 Cleared message 从不参与 cache key，因此打在它上面的 breakpoint 永远无法匹配。请像示例那样，把 breakpoint 放在前一个 user turn 的最后一个 block 上。顶层 automatic caching 字段在选择 breakpoint 时会跳过 turn-scoped message。在把某条 message cleared 掉的那次请求上，可复用的 cached prefix 止于它之前的那个 user turn，因此只有「该 message 与新的 user message 之间」那一个 assistant turn 会被重新处理。
• 放置规则始终生效，无论是否已被 cleared。Turn-scoped message 必须跟在 user turn 之后（或以 server tool result 结尾的 assistant turn 之后），并且位于 assistant turn 之前或作为数组末项，与任意 mid-conversation system message 相同。位于数组末尾的那一条始终会渲染。若它后面直接跟另一条 user message，得到的是 400 错误，而不是一条 cleared message：请把同一轮 tool 的全部 result 放进一条 user message，再把 reminder 接在它后面。
• Assistant turn 不会把它 cleared。 该 message 之后若是一条 prefilled 或 paused 的 assistant turn，或一次 server-side tool loop，都不会新增 user message，因此在那次续写上它仍会渲染。若要在 client-side tool loop 中让 reminder 持续可见，请在每条 tool_result message 之后再追加一次。
• Token 计数以实际渲染为准。 Cleared message 不会计入 usage.input_tokens，也不会计入 token count。
• 导入的 history。 若你一次性构造整份 transcript（few-shot 示例、迁移过来的对话），一条 turn-scoped message 后面如果已经既有 assistant turn 又有 user message，那么从第一次请求起它就是 cleared 的，永远不会渲染。对于你正在沿用的 per-turn reminder，这正是正确状态。只有希望模型在每次请求都看到的 message，才不要设置 clear_at。

校验错误如下：

1
2
3
4
5
6

messages.3.clear_at: Extra inputs are not permitted
messages.3.clear_at: clear_at is only permitted on role 'system' messages
messages.3.clear_at: Input should be 'next_user_message' or 'never'
messages.3: a turn-scoped system message supports text blocks only (clear_at: 'next_user_message')
messages.3: output_config is not permitted on a turn-scoped system message (clear_at: 'next_user_message')
messages.3.content.0: cache_control is not permitted on a turn-scoped system message (clear_at: 'next_user_message')

第一条是未带 beta header 时返回的错误。在 Amazon Bedrock 和 Google Cloud 上，按 Beta headers 所述传入 beta 值。

通过 SDK 使用时，在 messages 里那条 role: "system" 条目上设置 clear_at，并发送 beta header。下面的例子在 user turn 之后追加一条 turn-scoped reminder；下一次请求中，一旦出现更后的 user message，这条 reminder 仍留在数组里，但不再渲染：

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

client = anthropic.Anthropic()

response = client.beta.messages.create(
    model="claude-fable-5-1",
    max_tokens=4096,
    messages=[
        {
            "role": "user",
            "content": "Draft a short status update on the database migration for the team channel.",
        },
        # Turn-scoped reminder：本 turn 会渲染；一旦后面出现更新的 user message，就会被 cleared。
        {
            "role": "system",
            "clear_at": "next_user_message",
            "content": "The reader is on call: keep this reply under 50 words.",
        },
    ],
    betas=["mid-conversation-system-clear-at-2026-08-21"],
)

for block in response.content:
    if block.type == "text":
        print(block.text)

Combining with prompt caching

Mid-conversation system messages 与 prompt caching 就是为一起使用而设计的：

• 显式启用 caching。 只有请求包含 cache_control 时才会发生 caching——可以是顶层 automatic caching 字段，也可以是某个 content block 上的 explicit breakpoint。Mid-conversation system message 本身不会创建 cache entry；若不启用 caching，也就没有可保全的节省。
• 照常缓存稳定的 prefix。 把 cache_control 放在跨请求保持不变的最后一个 block 上，无论那是顶层 system 字段的末尾、工具定义的末尾，还是 message history 中某个稳定位置。
• 把 system message 追加在 breakpoint 之后。 因为它位于 cached prefix 之后，不会改变 prefix hash，cache 仍然命中。
• Mid-conversation system message 本身也可被缓存。 一旦它进入对话，就成为稳定 history 的一部分。下一 turn 你可以把 cache breakpoint 移过它（或依赖 automatic caching 自动前移），这条 system message 就会像其他 turn 一样从 cache 读取。

避免编辑或删除已经发送过的 mid-conversation system message。与改动更早的任意 message 一样，这会从该点起使 cache 失效。在 Claude Fable 5.1 上，它还会使之后每一个 assistant turn 中的 thinking block 失效。只应对单个 turn 生效的指导，请使用 turn-scoped system message，并把它留在原位。若指令需要演进，请追加一条新的 system message，而不是改写旧的。连续的 system message 会被接受，并视为同一个 system section，整体仍遵循同一套放置规则。

Limitations

• 不能作为第一条 message。 携带 content 的 system message 不能是 messages 的第一项。从一开始就应生效的指令，请使用顶层 system 字段。
• 放置位置受限。 携带 content（text、tool_addition 或 tool_removal block）的 system message 必须紧跟在 user turn 之后（包括携带 tool_result block 的 user turn），或紧跟在以 server tool result 结尾的 assistant turn 之后，并且必须位于 assistant turn 之前或作为数组末项。它不能夹在 tool_use block 与对应的 tool_result 之间。放在其他位置会返回 400 错误。有一个例外：tool_addition 和 tool_removal block 不能紧接在 paused 的 assistant turn（即以 server tool result 结尾的那条）之后，尽管 text block 可以；请先恢复被 pause 的 turn，再在下一条 system message 中发送工具变更。一条 content 为空、只设置 output_config.effort 的 message 在其所在位置不渲染任何内容，可以出现在 messages 的任意位置，包括第一项，或夹在 assistant turn 与 user turn 之间。连续的 system message 会被一并判定，因此若在一条仅含 effort 的 message 旁边再加一条带文本的 message，整组都会按「带 content」的规则处理。
• Turn-scoped message 只能是文本，且必须按原文原样重发。 clear_at: "next_user_message" 的 message 不能携带 tool_addition、tool_removal、output_config 或 cache_control；一旦被 cleared，后续请求必须把它逐字节原样留在 messages 中。参见 Turn-scoped system messages。
• 不要放入不可信内容。 Claude 把 system content 当作 operator 指令并会遵循它。不要把对话之外的文本——例如原始 tool output、检索到的文档或网页内容——直接放进 system message；那样会赋予这些文本 operator 级权威。把那些数据放在 tool_result block 中，并继续遵循 Mitigate jailbreaks and prompt injections。

Related

使用cache_control来缓存提示词前缀,使用自动缓存或者使用显示缓存断点:具有5分钟或者1h的存活时间的断点

Prompt caching允许你从提示词中的特定前缀继续处理,来优化API的使用

• Automatic caching: Add a single cache_control field at the top level of your request. The system automatically applies the cache breakpoint to the last cacheable block and moves it forward as conversations grow. Best for multi-turn conversations where the growing message history should be cached automatically.

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

  client = anthropic.Anthropic() response = client.messages.create( model="claude-opus-5", max_tokens=1024, cache_control={"type": "ephemeral"}, system="You are a helpful assistant that remembers our conversation.", messages=[ {"role": "user", "content": "My name is Alex. I work on machine learning."}, { "role": "assistant", "content": "Nice to meet you, Alex! How can I help with your ML work today?", }, {"role": "user", "content": "What did I say I work on?"}, ], ) print(response.usage.model_dump_json())

• Explicit cache breakpoints: Place cache_control directly on individual content blocks for fine-grained control over exactly what gets cached.

其中都有breakpoint的存在,由此可知breakpoint的作用时为显式划分缓存地边界

我刚去学习了一下机制: 首先. 有一个消息队列,在它的最前面有形如: system prompt,tools,agent.md(项目级别文档,看自己的harness规定)等等(这些是看harness规定地), 然后每次新的对话,就会在消息队列中新增内容,并且在automatic的情况下,就会不断地移动断点来显式标注prompt prefix地边界,这是客户端的内容 然后发送给服务端后,在推理过程中,会从前缀开始一个token一个token地进行比对,命中的部分就会复用KV cache中已有的部分 而未命中的部分就会创建新的KV Prefix Cache 是连续的：从 Prompt 开头开始匹配；第一个 token mismatch 出现后，该缓存链在这里截断，后续内容需要重新 Prefill

下面有个例子,自己看

上面算是题外话,接下来继续:

How prompt caching works

How prompt caching works

When you send a request with prompt caching enabled:

1. The system checks if a prompt prefix, up to a specified cache breakpoint, is already cached from a recent query.
2. If found, it uses the cached version, reducing processing time and costs.
3. Otherwise, it processes the full prompt and caches the prefix once the response begins. 但是并不是全面重新cache哦

缓存的生命周期是从这条请求的开始就开始计算了,而不是从response开始计算,一般默认是5分钟 那有人就会问了(其实是我):我早上开了一个会话,下午继续问.缓存是不是就失效了?是的 那么还有人就要问:TTL五分钟,我一次会话都不止 实际上,只要你有新的information产生并会先hit 然后创建新的cache内容或者你不断地hit cache,它的生命周期会一直延续下去,比如你不断调用工具之类的,so don’t worry 不过如果你觉得5分钟太短了,可以有1h的,不过那是另外的价钱

Automatic caching

How automatic caching works in multi-turn conversations

With automatic caching, the cache point moves forward automatically as conversations grow. Each new request caches everything up to the last cacheable block, and previous content is read from cache.

TTL support

y default, automatic caching uses a 5-minute TTL. You can specify a 1-hour TTL at 2x the base input token price:

1

{ "cache_control": { "type": "ephemeral", "ttl": "1h" } }

Combining with block-level caching

自动缓存与显式缓存断点（explicit cache breakpoints）兼容。当两者一起使用时，自动缓存断点会占用 4 个可用缓存断点槽位中的一个

what stays the same

自动缓存使用相同的底层缓存基础设施。其定价、最低 Token 数量要求、上下文顺序要求，以及 20 个 block 的回溯窗口，都与显式缓存断点的规则相同。

Edge cases

• If the last block already has an explicit cache_control with the same TTL, automatic caching is a no-op.
• If the last block has an explicit cache_control with a different TTL, the API returns a 400 error.
• If 4 explicit block-level breakpoints already exist, the API returns a 400 error (no slots left for automatic caching).
• If the last block is not eligible as an automatic cache breakpoint target, the system silently walks backwards to find the nearest eligible block. If none is found, caching is skipped.

Explicit cache breakpoints

This is useful when you need to cache different sections that change at different frequencies, or need fine-grained control over exactly what gets cached. 适用于你想要控制缓存的block

How automatic prefix checking works

Three core principle

• Cache writes happen only at your breakpoint.
• Cache reads look backward for entries that prior requests wrote
• The lookback window is 20 blocks.

简言之:只在断点处写入;会向前回溯(指的是在不匹配的情况下,镶向前回溯);回溯窗口为20个Block(最多回溯20个窗口,如果超过了.就会停止检查)

Example: Lookback in a growing conversation

You append new blocks each turn and set cache_control on the final block of each request:

• Turn 1: 10 blocks, breakpoint on block 10. No prior cache entries exist. The system writes an entry at block 10.
• Turn 2: 15 blocks, breakpoint on block 15. Block 15 has no entry, so the system walks back to block 10 and finds the turn-1 entry. Cache hit at block 10; the system processes only blocks 11 through 15 fresh and writes a new entry at block 15.
• Turn 3: 35 blocks, breakpoint on block 35. The system checks 20 positions (blocks 35 through 16) and finds nothing. The turn-2 entry at block 15 is one position outside the window, so there is no cache hit. Adding a second breakpoint at block 15 starts a second lookback window there, which finds the turn-2 entry.

核心要点： 将 cache_control 放在跨请求保持不变的、最后一个 block 上，这样你希望共享同一个缓存的请求就可以复用该缓存。

在不断增长的对话中，只要每一轮新增的内容少于 20 个 block，将缓存断点放在最后一个 block 上就可以正常工作：之前的内容不会发生变化，因此下一次请求的回溯机制能够找到上一次请求写入的缓存。

对于带有变化后缀的提示词（例如时间戳、每次请求特有的上下文、用户当前发送的消息），应该将缓存断点放在静态前缀的末尾，而不是放在会发生变化的 block 上。

When to use multiple breakpoints

最多4个断点,应用情景如下:

• Cache different sections that change at different frequencies (for example, tools rarely change, but context updates daily)
• Have more control over exactly what gets cached
• Ensure a cache hit when a growing conversation pushes your breakpoint 20 or more blocks past the last cache write

对于一次write,超过20个block会导致`miss

Understanding cache breakpoints cost

断点本身不付费 付费的是:缓存读写和常规输入

Caching strategies and considerations

Limitations:

the minimum cacheable prompt length is:

To verify whether a prompt was cached, check the response usage fields: if both cache_creation_input_tokens and cache_read_input_tokens are 0, the prompt was not cached (likely because it did not meet the minimum length requirement).

what can be cached

• Tools: Tool definitions in the tools array
• System messages: Content blocks in the system array
• Text messages: Content blocks in the messages.content array, for both user and assistant turns
• Images & Documents: Content blocks in the messages.content array, in user turns
• Tool use and tool results: Content blocks in the messages.content array, in both user and assistant turns

what cannot be cached

• Thinking blocks cannot be cached directly with cache_control. However, thinking blocks CAN be cached alongside other content when they appear in previous assistant turns. When cached this way, they DO count as input tokens when read from cache.

• Sub-content blocks (like citations) themselves cannot be cached directly. Instead, cache the top-level block.

  In the case of citations, the top-level document content blocks that serve as the source material for citations can be cached. This allows you to use prompt caching with citations effectively by caching the documents that citations will reference.

• Empty text blocks cannot be cached.

What invalidates the cache

Modifications to cached content can invalidate some or all of the cache.

Tracking cache preformance

1

total_input_tokens = cache_read_input_tokens + cache_creation_input_tokens + input_tokens

计费则是在对应的部分乘上price

Understanding the token breakdown

The input_tokens field represents only the tokens that come after the last cache breakpoint in your request - not all the input tokens you sent.

To calculate total input tokens:

1

total_input_tokens = cache_read_input_tokens + cache_creation_input_tokens + input_tokens

Spatial explanation:

• cache_read_input_tokens = tokens before breakpoint already cached (reads)
• cache_creation_input_tokens = tokens before breakpoint being cached now (writes)
• input_tokens = tokens after your last breakpoint (not eligible for cache)

Example: If you have a request with 100,000 tokens of cached content (read from cache), 0 tokens of new content being cached, and 50 tokens in your user message (after the cache breakpoint):

• cache_read_input_tokens: 100,000
• cache_creation_input_tokens: 0
• input_tokens: 50
• Total input tokens processed: 100,050 tokens

This is important for understanding both costs and rate limits, as input_tokens will typically be much smaller than your total input when using caching effectively.

Caching with thinking blocks

thinking blocks在不同cache策略下的缓存方法: 在auto:alongside other content 不能直接缓存,和其他内容一起,比如tool call

**Input token counting:**当think block在缓存中被读,就会被视为input_token

Cache storage and sharin

Prompt caching uses workspace-level isolation. Caches are isolated between organizations. Caches are also isolated per workspace within an organization Exact matching: Cache hits require 100% identical prompt segments, including all text and images up to and including the block marked with cache control. Output token generation: Prompt caching has no effect on output token generation. The response you receive is identical to what you would get if prompt caching were not used.(显而易见的,只是费用有区别罢了)

Best practices for effective caching

To optimize prompt caching performance:

• Start with automatic caching for multi-turn conversations. It handles breakpoint management automatically.
• Use explicit block-level breakpoints when you need to cache different sections with different change frequencies.
• Cache stable, reusable content like system instructions, background information, large contexts, or frequent tool definitions.
• Place cached content at the prompt’s beginning for best performance.
• Use cache breakpoints strategically to separate different cacheable prefix sections.
• Place the breakpoint on the last block that stays identical across requests. For a prompt with a static prefix and a varying suffix (timestamps, per-request context, the incoming message), that is the end of the prefix, not the varying block.
• Regularly analyze cache hit rates and adjust your strategy as needed.

Optimizing for different use cases(泛话,懒得看)

对话式智能体（Conversational agents）： 降低长对话的成本和延迟，尤其适用于包含较长指令或上传文档的对话。

编程助手（Coding assistants）： 通过在提示词中保留代码库的相关部分，或代码库的摘要版本，提升代码自动补全和代码库问答的效果。

大型文档处理（Large document processing）： 将完整的长篇材料（包括图片）放入提示词中，而不会增加响应延迟。

详细的指令集（Detailed instruction sets）： 通过共享大量指令、流程和示例，让 Claude 的响应更加符合要求。开发者通常会在提示词中加入一两个示例，但借助提示词缓存，可以通过加入 20 个以上不同的高质量答案示例，获得更好的性能。

智能体工具调用（Agentic tool use）： 提升涉及多次工具调用和迭代式代码修改的场景的性能，因为每一步通常都需要发起一次新的 API 请求。

与书籍、论文、文档、播客文字稿及其他长篇内容对话： 通过将完整文档嵌入提示词，让任何知识库都变得生动，并允许用户向其提问。

Troubleshooting common issues(也是,懒得看)

If experiencing unexpected behavior:

Cache diagnostics (beta) has the API compare consecutive requests and report exactly where the prompt prefix diverged, which automatically handles many of the steps in this list.

• Ensure cached sections are identical across calls. For explicit breakpoints, verify that cache_control markers are in the same locations
• Check that calls are made within the cache lifetime (5 minutes by default)
• Verify that tool_choice, image usage, the thinking configuration, and output_config.effort remain consistent between calls
• Validate that you are caching at least the minimum number of tokens for your model and platform (see Cache limitations)
• Confirm your breakpoint is on a block that stays identical across requests. Cache writes happen only at the breakpoint, and if that block changes (timestamps, per-request context, the incoming message), the prefix hash never matches. The lookback does not find stable content behind the breakpoint; it only finds entries that earlier requests wrote at their own breakpoints
• Verify that the keys in your tool_use content blocks have stable ordering as some languages (for example, Swift, Go) randomize key order during JSON conversion, breaking caches
• Use cache diagnostics to have the API compare consecutive requests and report which part of the prompt diverged

1-hour cache duration

Mixing different TTLs

You can use both 1-hour and 5-minute cache controls in the same request, but with an important constraint: Cache entries with longer TTL must appear before shorter TTLs (that is, a 1-hour cache entry must appear before any 5-minute cache entries).

Pre-warming the cache

load your system prompt or tool definitions into the prompt cache before a user triggers a real request.

How it works

在请求中设置 max_tokens: 0。API 会将你的提示词读入模型，并在任何 cache_control 断点处写入缓存，然后立即返回，不生成任何输出。响应中的 content 数组为空，stop_reason 为 "max_tokens"，并且包含完整的 usage 信息。

将 cache_control 断点放在与后续请求共享的最后一个 block 上（通常是系统提示词或工具定义），而不是占位用户消息上。否则，缓存条目会以占位内容作为键，后续请求就无法命中该缓存。

同时，预热请求也应使用与后续请求相同的 thinking 配置和 output_config.effort。这些值会被渲染到提示词中（参见什么会使缓存失效（What invalidates the cache） ），因此，如果预热请求使用了不同的配置，就可能写入一个真实流量永远无法命中的缓存条目。

这意味着应使用**显式缓存断点（explicit cache breakpoint） ，而不是自动缓存（automatic caching） **，因为自动缓存会将断点放在最后一个 block 上，而在这里，最后一个 block 正是占位消息。

占位用户消息可以是任何包含非空白字符的字符串（示例中使用 "warmup"）；其内容会被读入模型，但不会得到回答。

Typical usage pattern

Fire a pre-warm request when your application starts (or on a scheduled interval), then send real user requests after the pre-warm completes:

Limitation

A max_tokens: 0 request is rejected with an invalid_request_error if any of the following are set, since each implies output that a zero-token budget cannot produce:

• stream: true
• Extended thinking (thinking.type: "enabled")
• Structured outputs (output_config.format)
• tool_choice of {"type": "tool", ...} or {"type": "any"}

max_tokens: 0 is also rejected inside a Message Batches request. Pre-warming targets time-to-first-token, which does not apply to batch processing, and a cache entry written during batch processing would likely expire before the follow-up request runs.

Data retention

Prompt caching (both automatic and explicit) is ZDR eligible. Anthropic does not store the raw text of your prompts or Claude’s responses.

KV (key-value) cache representations and cryptographic hashes of cached content are held in memory only and are not stored at rest. Cached entries have a minimum lifetime of 5 minutes (standard) or 1 hour (extended), after which they are promptly, though not immediately, deleted. Cache entries are isolated between organizations and, on the Claude API, Claude Platform on AWS, and Microsoft Foundry, between workspaces within an organization.

符合ZDR(零数据保留),KV cache和加密哈希值仅保存在内存中,不会持久化,最短生命周期为5min或者1h 缓存条目在不同组织上相互隔离