> ## Documentation Index
> Fetch the complete documentation index at: https://docs.naga.ac/llms.txt
> Use this file to discover all available pages before exploring further.

# Prompt caching

> Mark the long, repeated start of your prompt so the provider reads it from cache and you pay the cache-read price for it.

Mark where the repeated start of your prompt ends with `cache_control`, and a provider that caches prompts keeps that prefix. The next request that begins with the same prefix reads it from the cache instead of processing it again. It answers sooner, and the prefix costs the `cached_input_token` price rather than the input price. Caching pays off for a long system prompt, a document you ask several questions about, a set of tool definitions or a long conversation.

NagaAI passes `cache_control` to the providers that take it. These are most providers of Claude models, OpenAI from GPT-5.6 on, and some providers of Gemini and Qwen models. A provider without prompt caching ignores it, and the request runs as if you hadn't set it.

> [tab] Responses
>
> ```python Python theme={"theme":{"light":"github-light-default","dark":"github-dark-dimmed"}}
> import os
> from openai import OpenAI
>
> client = OpenAI(base_url="https://api.naga.ac/v1", api_key=os.environ["NAGA_API_KEY"])
>
> handbook = "Refunds are issued within 14 days of purchase. " * 300
>
> conversation = [{"role": "user", "content": "How long do I have to ask for a refund?"}]
>
> for follow_up in ["Is the period counted in days?", None]:
>     response = client.responses.create(
>         model="anthropic/claude-sonnet-5.5",
>         instructions=handbook,
>         input=conversation,
>         extra_body={"cache_control": {"type": "ephemeral"}},
>     )
>     print(response.output_text)
>     print(response.usage.input_tokens_details)
>     if follow_up:
>         conversation += [
>             {"role": "assistant", "content": response.output_text},
>             {"role": "user", "content": follow_up},
>         ]
> ```
>
> ```javascript Node.js theme={"theme":{"light":"github-light-default","dark":"github-dark-dimmed"}}
> import OpenAI from "openai";
>
> const client = new OpenAI({ baseURL: "https://api.naga.ac/v1", apiKey: process.env.NAGA_API_KEY });
>
> const handbook = "Refunds are issued within 14 days of purchase. ".repeat(300);
>
> const conversation = [{ role: "user", content: "How long do I have to ask for a refund?" }];
>
> for (const followUp of ["Is the period counted in days?", null]) {
>   const response = await client.responses.create({
>     model: "anthropic/claude-sonnet-5.5",
>     instructions: handbook,
>     input: conversation,
>     cache_control: { type: "ephemeral" },
>   });
>   console.log(response.output_text);
>   console.log(response.usage.input_tokens_details);
>   if (followUp) {
>     conversation.push(
>       { role: "assistant", content: response.output_text },
>       { role: "user", content: followUp },
>     );
>   }
> }
> ```

> [tab] Chat Completions
>
> ```python Python theme={"theme":{"light":"github-light-default","dark":"github-dark-dimmed"}}
> import os
> from openai import OpenAI
>
> client = OpenAI(base_url="https://api.naga.ac/v1", api_key=os.environ["NAGA_API_KEY"])
>
> handbook = "Refunds are issued within 14 days of purchase. " * 300
>
> for question in ["How long do I have to ask for a refund?", "Is the period counted in days?"]:
>     completion = client.chat.completions.create(
>         model="anthropic/claude-sonnet-5.5",
>         messages=[
>             {
>                 "role": "system",
>                 "content": [
>                     {"type": "text", "text": handbook, "cache_control": {"type": "ephemeral"}}
>                 ],
>             },
>             {"role": "user", "content": question},
>         ],
>     )
>     print(completion.choices[0].message.content)
>     print(completion.usage.prompt_tokens_details)
> ```
>
> ```javascript Node.js theme={"theme":{"light":"github-light-default","dark":"github-dark-dimmed"}}
> import OpenAI from "openai";
>
> const client = new OpenAI({ baseURL: "https://api.naga.ac/v1", apiKey: process.env.NAGA_API_KEY });
>
> const handbook = "Refunds are issued within 14 days of purchase. ".repeat(300);
>
> for (const question of ["How long do I have to ask for a refund?", "Is the period counted in days?"]) {
>   const completion = await client.chat.completions.create({
>     model: "anthropic/claude-sonnet-5.5",
>     messages: [
>       {
>         role: "system",
>         content: [{ type: "text", text: handbook, cache_control: { type: "ephemeral" } }],
>       },
>       { role: "user", content: question },
>     ],
>   });
>   console.log(completion.choices[0].message.content);
>   console.log(completion.usage.prompt_tokens_details);
> }
> ```

> [tab] Messages
>
> ```python Python theme={"theme":{"light":"github-light-default","dark":"github-dark-dimmed"}}
> import os
> from anthropic import Anthropic
>
> client = Anthropic(base_url="https://api.naga.ac", api_key=os.environ["NAGA_API_KEY"])
>
> handbook = "Refunds are issued within 14 days of purchase. " * 300
>
> for question in ["How long do I have to ask for a refund?", "Is the period counted in days?"]:
>     message = client.messages.create(
>         model="anthropic/claude-sonnet-5.5",
>         max_tokens=256,
>         system=[{"type": "text", "text": handbook, "cache_control": {"type": "ephemeral"}}],
>         messages=[{"role": "user", "content": question}],
>     )
>     print("".join(block.text for block in message.content if block.type == "text"))
>     print(message.usage)
> ```
>
> ```typescript TypeScript theme={"theme":{"light":"github-light-default","dark":"github-dark-dimmed"}}
> import Anthropic from "@anthropic-ai/sdk";
>
> const client = new Anthropic({ baseURL: "https://api.naga.ac", apiKey: process.env.NAGA_API_KEY });
>
> const handbook = "Refunds are issued within 14 days of purchase. ".repeat(300);
>
> for (const question of ["How long do I have to ask for a refund?", "Is the period counted in days?"]) {
>   const message = await client.messages.create({
>     model: "anthropic/claude-sonnet-5.5",
>     max_tokens: 256,
>     system: [{ type: "text", text: handbook, cache_control: { type: "ephemeral" } }],
>     messages: [{ role: "user", content: question }],
>   });
>   for (const block of message.content) {
>     if (block.type === "text") console.log(block.text);
>   }
>   console.log(message.usage);
> }
> ```

The first request writes the handbook to the cache, and its `usage` shows the written tokens: `cache_write_tokens` in Responses and Chat Completions, `cache_creation_input_tokens` in Messages. The second request starts with the same prefix, so `usage` shows those tokens as read: `cached_tokens`, or `cache_read_input_tokens` in Messages. In Responses the breakpoint sits at the top level and caches up to the end of the input. So that example's second request repeats the first conversation and adds to it, rather than asking a new question after the handbook. When both counts stay at zero, the provider that served the request doesn't cache, or the prefix is shorter than the model's minimum.

## Breakpoint positions by format

A breakpoint caches everything before it and the block it sits on, in the order tools, system prompt, messages. Each format takes it in the shape you already know from Anthropic or OpenRouter:

| | Responses | Chat Completions | Messages |
| - | - | - | - |
| Automatic: on the last block that can be cached | `cache_control` at the top level | `cache_control` at the top level | `cache_control` at the top level |
| On a block | Not available | On a `text`, `image_url` or `file` part of any message | On a tool, a `system` text block, and a `text`, `image`, `document`, `tool_use` or `tool_result` block |

The automatic form is the simplest choice for a conversation. The breakpoint moves forward as the conversation grows, so each turn reads the turns before it from the cache. Put a breakpoint on a block to keep a fixed part, such as the system prompt or the tool list, cached while the rest changes.

In Responses and Chat Completions an SDK has no argument for the top-level field. Pass it through `extra_body` in Python, and as a plain field in Node.js.

## Lifetime and price

`{"type": "ephemeral"}` keeps the prefix for five minutes after its last use, and every read starts the five minutes again. `{"type": "ephemeral", "ttl": "1h"}` keeps it for an hour. Any other `ttl` fails with `400`.

A write costs the model's `cache_write_input_token` price, and a one-hour write costs `cache_write_1h_input_token`. Both are higher than the input price, and a read costs the `cached_input_token` price, which is lower. A prefix you reuse at least once within its lifetime therefore costs less than sending it uncached. The prices are in the model's `pricing` in [`/v1/models`](https://docs.naga.ac/get-started/models), and [Billing](https://docs.naga.ac/build/billing.md#price-of-a-request) explains how NagaAI charges a request. NagaAI charges a write only when the request carried `cache_control`. A write the provider made on its own is reported in `usage` and costs nothing.

## Explicit OpenAI breakpoints

OpenAI takes breakpoints on GPT-5.6 and later models, and NagaAI turns `cache_control` into OpenAI's own breakpoint. A breakpoint on a text part, in Chat Completions or Messages, caches up to that part. Responses takes `cache_control` only at the top level. When a request has only block breakpoints, OpenAI caches only those prefixes. It writes nothing after the last one. The top-level `cache_control` keeps OpenAI's automatic breakpoint at the end of the latest message. OpenAI keeps an entry for thirty minutes, and a one-hour `ttl` gets the same thirty minutes. The write costs the model's `cache_write_input_token` price.

OpenAI's wire has no place for a breakpoint on an image, a file, an assistant turn or a tool result. NagaAI sends the request without it. A request writes at most four prefixes, and the top-level breakpoint is one of them. When you send more, NagaAI keeps the last ones. These models also write to the cache on their own when you set no `cache_control`, and that write costs nothing.

```python Python theme={"theme":{"light":"github-light-default","dark":"github-dark-dimmed"}}
import os
from openai import OpenAI

client = OpenAI(base_url="https://api.naga.ac/v1", api_key=os.environ["NAGA_API_KEY"])

contract = "The supplier delivers within 30 days and bears the shipping risk. " * 300

for question in ["Who bears the shipping risk?", "How long is the delivery period?"]:
    response = client.chat.completions.create(
        model="openai/gpt-6.1-sol",
        messages=[
            {
                "role": "user",
                "content": [
                    {
                        "type": "text",
                        "text": contract,
                        "cache_control": {"type": "ephemeral"},
                    },
                    {"type": "text", "text": question},
                ],
            }
        ],
    )
    print(response.choices[0].message.content)
    print(response.usage.prompt_tokens_details)
```

The first request writes the contract to the cache at the `cache_write_input_token` price. The second starts with the same contract, so its `usage` shows it as `cached_tokens`.

## OpenAI cache key and lifetime

GPT models cache on their own, with no `cache_control`. Two OpenAI fields steer that cache in Responses and Chat Completions, and NagaAI passes both to OpenAI. `prompt_cache_key` groups requests that share a prefix, so OpenAI sends them to the same cache and more of them read it. Use one stable key per prompt you reuse, such as one per customer or per agent, of at most 64 characters. A longer key fails with `400 invalid_request_error`. OpenRouter takes the key too, and other providers ignore it.

`prompt_cache_retention` sets how long OpenAI keeps the prefix on models before GPT-5.6: `in_memory`, or `24h` on the models OpenAI lists for extended retention, such as `gpt-5.4` and `gpt-4.1`. GPT-5.5 takes only `24h`, and GPT-5.6 and later have no such field. NagaAI drops a value the model doesn't take, and the request runs at the model's default lifetime. Neither field orders a write, so neither changes the price of a request.

```python Python theme={"theme":{"light":"github-light-default","dark":"github-dark-dimmed"}}
import os
from openai import OpenAI

client = OpenAI(base_url="https://api.naga.ac/v1", api_key=os.environ["NAGA_API_KEY"])

handbook = "Refunds are issued within 14 days of purchase. " * 300

for question in ["How long do I have to ask for a refund?", "Is the period counted in days?"]:
    response = client.responses.create(
        model="openai/gpt-6.1-sol",
        instructions=handbook,
        input=question,
        prompt_cache_key="refund-handbook",
    )
    print(response.output_text)
    print(response.usage.input_tokens_details)
```

The second request starts with the same handbook and the same key, so its `usage` shows the handbook as `cached_tokens` once OpenAI has cached it. On a model before GPT-5.6, add `prompt_cache_retention="24h"` to keep the prefix for up to a day.

## Limits

A request carries at most four breakpoints, and the automatic one counts as one of them. When you send more, NagaAI keeps the last ones and drops the earliest, because the last breakpoint covers the longest prefix.

A prefix shorter than the model's minimum isn't cached, and the request runs without an error. For current Claude models the minimum is 512 to 4,096 tokens, depending on the model, as Anthropic lists in its [prompt caching guide](https://platform.claude.com/docs/en/build-with-claude/prompt-caching#cache-limitations).

A one-hour breakpoint has to come before any five-minute one, in the order tools, system prompt, messages, with the automatic one last. When a one-hour breakpoint follows a five-minute one, NagaAI sends it as a five-minute one. The prefix still caches, at the cheaper write price.

The cache belongs to one provider. A request that NagaAI routes to a different provider than the last one writes the cache again. So a request with `cache_control` runs under the [`cache` routing strategy](https://docs.naga.ac/build/routing#strategies) unless you set another one. Among providers that sell the model at the same price, it prefers those that read your prefix back.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.