How to Reduce AI Token Costs？Try Prompt Caching and Sticky Routing strategy

Modern AI agents rarely send a small, independent prompt. A coding agent may include system instructions, repository context, tool definitions, security policies and JSON schemas in every request. A customer-service agent may repeatedly submit product documentation, conversation history and operating rules. Even when only the latest user message changes, the application can still resend tens of thousands of identical input tokens. Without prompt caching, those repeated tokens are processed and billed again on every turn.

Prompt caching changes the economics of multi-turn AI applications. Instead of fully processing the same prompt prefix repeatedly, the model provider stores reusable content and retrieves it during later requests. However, effective caching involves more than enabling a single API option. Developers must understand cache-write pricing, cache-read pricing, expiration times and the routing behaviour that determines whether the next request reaches the same warm cache.

This guide explains how cached token pricing works, why writes cost more than reads, how sticky routing and session_id protect cache continuity, and how to confirm that prompt caching is actually reducing your AI API costs.

What Is Prompt Caching in AI models?

Prompt caching allows an AI provider to reuse previously processed input when the beginning of a new request matches an earlier request.

Consider an agent that sends the following information on every turn:

• A 5,000-token system prompt
• 8,000 tokens of tool definitions
• A 10,000-token technical reference document
• The latest user message and tool result

The first three sections remain mostly unchanged. Only the final part develops as the agent completes its task.

With a normal AI API request, the entire input is billed at the standard input-token price each time. With prompt caching, the provider can store the stable prefix after the first request. Later turns then pay a lower cache-read rate for the reusable section while processing only the new content at the normal input rate.

The greatest savings therefore appear in long conversations, coding agents, retrieval-augmented generation systems and automated workflows that repeatedly send large blocks of stable context.

How Much Do Cached Tokens Cost in Claude and Chatgpt?

Cached tokens are not normally billed at the same rate as fresh input tokens. Depending on the provider and model, a cache read may cost approximately 10% to 50% of the standard input price. The first request may also create a cache write, which can cost the same as normal input or carry an additional premium.

The following multipliers illustrate several common provider-side pricing structures at the time of publication:

Pricing and supported models can change. Production applications should always review the current AI token pricing for the selected model and provider route before estimating savings.

A Practical Prompt-Caching Cost Example

Assume an agent repeats 20,000 stable input tokens across eight requests. The standard input price is $3 per million tokens.

Without caching, the repeated content costs:

20,000 × 8 × $3 / 1,000,000 = $0.48

With Anthropic’s five-minute cache, the first turn creates a write at 1.25 times the normal input rate. The following seven turns read the same content at 0.1 times the input rate:

Cache write: $0.075

Seven cache reads: $0.042

Total cached cost: $0.117

In this simplified example, prompt caching reduces the cost of the repeated input by approximately 75.6%. The calculation excludes changing messages and output tokens, but it illustrates why caching becomes increasingly valuable as an agent completes more turns. Anthropic states that its five-minute cache can become economical after one reuse, while the more expensive one-hour write generally requires two reads to recover its additional cost.

Why Do Cache Writes Cost More Than Cache Reads?

A cache write is a front-loaded operation. During the first request, the provider must process the original prompt and create a reusable cache entry. Depending on the provider, the price may include an additional charge for storing and managing that cached context.

A cache read is cheaper because the provider can reuse the stored prompt prefix instead of repeating the complete input-processing operation. The model still generates a new answer, so output tokens remain billable at the normal output rate. Only the reusable input section receives the cache discount.

This difference explains why prompt caching is not automatically beneficial for every request. A one-time prompt may pay a cache-write premium without ever producing a cache read. By contrast, an AI agent that reuses the same instructions over many turns can recover the write cost quickly and continue generating savings throughout the session.

Choosing Between a Short and Long Cache TTL

Anthropic provides a default five-minute cache lifetime and an optional one-hour lifetime. The five-minute option is more economical for active conversations in which requests arrive quickly. The one-hour option has a higher write multiplier but is better suited to workflows that pause between steps, such as extended research, human approval processes or long-running coding tasks.

A longer TTL should not be selected automatically. The additional write cost is worthwhile only when the session is likely to remain inactive long enough for a shorter cache to expire and later return to the same context.

Why a Warm Cache Can Still Miss

Creating a cache does not guarantee that the next request will use it.

A provider-side cache normally exists on the endpoint or route where it was created. In a multi-provider API, the first request might be sent through one provider endpoint, while the second request is routed through another route because of price, latency or availability. The second route does not necessarily have access to the first route’s cached state. The API request may still succeed, but the stable prompt is processed at full input-token pricing again.

This problem is especially important for multi-turn AI agents. Intelligent routing improves availability, yet uncontrolled provider switching can reduce cache-hit rates. Sticky routing solves the conflict by keeping related requests on the same compatible provider endpoint while the cache remains useful. If that route becomes unavailable, a well-designed router can still fail over rather than allowing the entire workflow to stop.

How session_id Keeps an AI Agent’s Cache Warm

A session_id gives the routing layer a stable identifier for a conversation or agent workflow. Without an explicit identifier, some routing systems infer whether requests belong to the same conversation by examining or hashing the opening messages. That approach works when the first messages never change. However, agent applications frequently rewrite context, add runtime metadata, summarize earlier turns or change the order of tool definitions.

Once the opening content changes, the routing system may interpret the request as a new conversation and send it to another provider.

An explicit session_id avoids that ambiguity. OpenRouter, for example, uses the value as a sticky-routing key and activates route stickiness after the first successful request—even before a cache hit has been detected. The same value must remain attached to every turn in the conversation.

{
  "model": "your-selected-model",
  "session_id": "support-ticket-83422",
  "messages": [
    {
      "role": "system",
      "content": "You are a technical support agent..."
    },
    {
      "role": "user",
      "content": "Help me diagnose this API error."
    }
  ]
}

The identifier should represent the entire unit of work, such as a chat thread, customer ticket, coding session or workflow run. Creating a new session_id for every request defeats the purpose because each turn is treated as a separate session.

Start Using the Session ID on Turn One

Do not wait until the second or third request to add the identifier.

The first request determines where the initial cache is written. Supplying the session_id from the beginning allows the routing layer to associate that provider route with the session immediately. Follow-up turns can then return to the endpoint that already holds the reusable prompt context.

This is particularly valuable for agents whose first two requests contain no cache hit yet. The first request creates the cache; the second request needs sticky routing to find it.

How to Verify That Prompt Caching Is Working

Lower latency alone is not reliable proof of a cache hit. Network conditions, provider load and output length can all change response time.

Instead, inspect the usage metadata returned by the API.

Where supported, usage.prompt_tokens_details.cached_tokens reports the number of prompt tokens retrieved from cache. A value greater than zero confirms that at least part of the request used cached input.

{
  "usage": {
    "prompt_tokens": 18560,
    "completion_tokens": 420,
    "total_tokens": 18980,
    "prompt_tokens_details": {
      "cached_tokens": 18000,
      "cache_write_tokens": 0
    }
  }
}

In this example, 18,000 prompt tokens were read from an existing cache. Because cache_write_tokens is zero, the request reused an earlier entry rather than creating a new one.

Some routing platforms also expose a cache-discount field or request-level billing record. A write request may initially show little savings—or even a temporary premium—while later cache-read requests should show lower input costs.

Compare Several Consecutive Requests

A reliable cache test should include at least two requests:

1. Send a long, stable prompt and record its usage data.
2. Send a second request with the same prefix, the same session identifier and a small new user message.
3. Check whether cached_tokens increases.
4. Compare the billed input cost between the write turn and the read turn.

Testing only the initial request cannot prove that caching works because the first turn normally creates the cache instead of reading it.

Common Reasons for Cache Misses

When cached-token usage remains at zero, the cause is usually straightforward.

The prompt may be below the model provider’s minimum cacheable length. Alternatively, the cache may have expired between requests. Another common problem is an unstable prefix: timestamps, request IDs and frequently changing metadata placed near the beginning of the prompt can make every request appear different.

Provider drift creates a fourth cause. Even a perfectly identical prompt cannot reuse a cache if the request moves to another endpoint that does not hold the original entry.

To improve cache-hit rates, place stable content first:

• System instructions
• Tool definitions
• JSON schemas
• Policies and guardrails
• Long-lived reference documents

Place changing information later, including user messages, tool results, timestamps and temporary workflow state.

Optimizing Cached AI Tokens With CostRouter

Prompt caching becomes more valuable when combined with clear AI token accounting. CostRouter provides one balance and one AI API key for accessing multiple supported model families, including GPT 5-6, Claude 5, Gemini. Each request records the selected model, token usage, billing calculation and final charge, allowing teams to compare the real cost of different models and routing configurations.

For developers, this creates a practical optimization workflow. A team can test the same AI agent across several models, compare input and output token costs, review request-level usage records and determine where provider-side prompt caching produces the greatest savings. Because cache fields and routing behaviour can differ between model routes, developers should still inspect the response metadata exposed by the selected endpoint. A lower published cache price is only valuable when the application consistently achieves cache hits.

Rather than choosing an AI model based only on its standard input price, production teams should evaluate four variables together: normal token pricing, cache-write pricing, cache-read pricing and the probability that consecutive requests reach the warm cache.

Frequently Asked Questions

Does Prompt Caching Reduce Output Token Costs?

No. Prompt caching generally reduces the cost of repeated input context. The model still generates a new response, so output tokens continue to use the applicable output-token price.

Is Prompt Caching Useful for a One-Off API Request?

Usually not. If content is never reused, a paid cache write may cost the same as or more than normal input processing. Caching is most effective for multi-turn conversations and agent workflows.

What Should a session_id Represent?

It should identify one continuous conversation or unit of work, such as a chat thread, support ticket, coding task or agent run. The value should remain unchanged throughout that session.

How Do I Know Whether Cached Tokens Saved Money?

Check cached_tokens, cache-write usage and request-level billing data. Compare the first write request with later read requests rather than looking at one API call in isolation.

Conclusion

Prompt caching is not merely a technical performance feature. For long-context AI agents, it is an important part of cost architecture. The first request creates the reusable cache. Later requests generate savings through lower-priced reads. Sticky routing ensures that those requests return to the provider endpoint holding the cached context, while a stable session_id protects the relationship from the first turn onward.

Most importantly, caching should always be verified rather than assumed. Inspect cached-token metadata, monitor request-level charges and test several consecutive turns under real production conditions. Combined with a multi-model API and transparent AI token records, prompt caching allows developers to build longer, more capable agent sessions without repeatedly paying full price for the same context.