# Prompt Caching Cuts Input Token Costs by 90%: How Growth Companies Build a Context Cache for Production AI Agents

> Prompt caching for AI agents cuts input token costs by up to 90% and speeds up responses. See how growth companies structure prefixes in production.

- Published: 2026-10-17
- Author: Master Mind
- Canonical: https://aimaster.fi/en/artikkelit/prompt-caching-leikkaa-ai-agentin-syotekuluista-90-prosenttia-tuotannossa

Cached input tokens cost just 5% to 10% of standard base input prices, cutting repetitive context costs by 90% to 95% (Anthropic Platform Docs, 2026; OpenAI API Docs, 2026). Yet many growth companies pay full price on every single agent step because a single dynamic timestamp at the top of the prompt breaks the cache. When you move a multi-step agent from a sandbox demo into daily production, locking your static prefix into a prompt cache is the difference between a profitable workflow and a runaway API bill.

Sixty-six percent of Finnish enterprises already use generative AI tools — the highest share in the EU (Statistics Finland / Eurostat, 2025). In a light chat interface, nobody notices input token waste. In an autonomous production agent that calls ERP and CRM tools ten times per task, resending 30,000 tokens of tool schemas and instructions from scratch on every loop multiplies both latency and cost.

## How does prompt caching for AI agents work in production?

Prompt caching for AI agents works by storing the language model's precomputed mathematical key-value (KV) states for the unchanged beginning of a prompt, known as the prefix. When the next API call starts with the exact same byte sequence, the model skips recalculating those tokens and reads the ready state directly from GPU memory at a 90% to 95% discount (OpenAI API Docs, 2026; Anthropic Platform Docs, 2026).

A language model does not cache raw text strings; it caches intermediate attention tensors (key-value states). For the model to reuse those tensors, every single token before your cache breakpoint — a marker that defines where reusable content ends — must match the previous request character for character.

On Anthropic's Claude platform, writing to a 5-minute ephemeral cache costs 1.25 times the base input token price, while a 1-hour cache write costs 2 times the base price (Anthropic Platform Docs, 2026). Subsequent cache reads cost 0.1 times the base rate on standard models and 0.05 times on Claude Opus 5.5 and Claude Sonnet 5.5 (Anthropic Platform Docs, 2026). OpenAI's GPT-5.6 and newer models follow the same economics: a 1.25x write multiplier, a 0.1x read multiplier (0.05x on GPT-6.1 Sol), and a minimum 30-minute retention window (OpenAI API Docs, 2026). Across ten agent steps sharing one static prefix, one write and nine reads at the 0.1x rate cost 2.15 times a single request's input cost — compared with 10 times without caching (OpenAI API Docs, 2026).

## Why does a single timestamp break your agent's entire cache?

A single timestamp breaks your agent's cache because language model APIs evaluate the prompt prefix as a cumulative cryptographic hash in a strict hierarchy: tools first, system instructions second, and conversation messages third (Anthropic Platform Docs, 2026). If any token changes before a cache breakpoint, the hash changes and invalidates every cached layer after that point, forcing a fresh computation.

In production audits, the most expensive line of code is almost always `Current time: 2026-10-17T14:03:12Z` injected into the first line of the system prompt. When that clock ticks one second forward on the next tool call, the prefix hash changes at token 5. Even if 25,000 tokens of ERP schemas and company policies after that line remain identical, the cache misses completely.

Worse still, if a developer places an explicit `cache_control` breakpoint (or enables automatic caching) on a block that changes every request, the company pays the 1.25x cache-write surcharge on every single call and never gets a single discounted read (Anthropic Platform Docs, 2026). Slide consultants rarely draw cumulative prefix hashes in their strategy decks, because a broken hash shows up on the monthly API invoice rather than in the steering committee presentation.

- Tool definitions (`tools`): Modifying a tool name, parameter schema, or tool order invalidates the entire cache — tools, system prompt, and conversation history (Anthropic Platform Docs, 2026; OpenAI API Docs, 2026).
- System and developer instructions (`system` / `developer`): Toggling web search, changing citations settings, or injecting user-specific variables into the top block invalidates both the system cache and message history (Anthropic Platform Docs, 2026).
- Lookback window limit (20 blocks): On the Claude API, automatic prefix lookup walks backward at most 20 content blocks from a breakpoint; if a long agent loop adds more than 20 blocks without an intermediate breakpoint, the lookup misses the earlier write (Anthropic Platform Docs, 2026).
- Minimum token threshold: Prefixes shorter than 1,024 tokens on GPT-5.6+ or 512–4,096 tokens on Claude models are silently processed without caching (OpenAI API Docs, 2026; Anthropic Platform Docs, 2026).

## How should a growth company structure an AI agent's prompt prefix?

A growth company should structure an AI agent's prompt from most stable to most volatile: lock frozen tool definitions and JSON schemas first, place static company rules second, append growing conversation and tool history third, and inject dynamic timestamps or per-request user data only after the final cache breakpoint (Anthropic Platform Docs, 2026; OpenAI API Docs, 2026).

Well-designed [context engineering for production AI agents](https://aimaster.fi/en/artikkelit/konteksti-ratkaisee-ei-prompti-context-engineering-tekee-ai-agentista-luotettava) separates static reference material from per-request state. Both Anthropic and OpenAI allow up to four explicit cache breakpoints per request (Anthropic Platform Docs, 2026; OpenAI API Docs, 2026). Using those four slots deliberately protects your agent when different layers of context update at different speeds.

| Prompt Layer (Order) | Content in Production Agent | Update Frequency | Cache Breakpoint Strategy |
| --- | --- | --- | --- |
| 1. Tools & JSON Schemas | ERP/CRM tool definitions and strict schemas | Weekly / per release | Breakpoint #1 at the end of the fixed tool array |
| 2. Static System Prompt | Role rules, decision guardrails, domain glossary | Daily / static | Breakpoint #2 after the last immutable instruction block |
| 3. Multi-Turn History & Tool Results | Appended user messages, tool calls, and outputs | Grows every agent step (append-only) | Breakpoint #3–#4 on recent tool results (within 20 blocks) |
| 4. Dynamic Request Suffix | Current timestamp, user ID, live ticket payload | Changes every API call | Left after the final breakpoint — never written to cache |

Notice how tool management works in this architecture. If an agent does not need a specific ERP write tool on step three of a workflow, do not delete the tool from the `tools` array — removing it rewrites the prefix and flushes the cache. Instead, keep the tool array identical and restrict callable tools using `tool_choice` or `allowed_tools` parameters, which preserves the cached tool prefix intact (OpenAI API Docs, 2026; Anthropic Platform Docs, 2026). Combining frozen tool definitions with [structured outputs and JSON schema in AI integrations](https://aimaster.fi/en/artikkelit/structured-outputs-ja-json-skeema-tekoalyintegraatiossa-estavat-rikkinaiset-erp-kutsut) keeps both your ERP API calls deterministic and your cache hit rate above 90% (OpenAI API Docs, 2026).

## How do you deploy prompt caching into production in 3-day sprints?

Deploying prompt caching into production succeeds when you audit token consumption by workflow first, refactor prompt hierarchies into append-only layers, and track realized cache-hit rates continuously in production. At AIMASTER, we build this architecture into production systems in 3-day sprints so unit economics stay predictable from day one.

**Master Plan** is an AI strategy sprint that maps where AI generates the most value for your company — measured in euros. When we evaluate an agent workflow in the [Master Plan](https://aimaster.fi/tuotteet/master-plan) sprint, we calculate the exact token volume per business transaction: how many tool definitions, product catalog rows, and loop steps a single order or service ticket requires.

**Master OS** is a company's AI Operating System: a platform that securely connects your existing systems (CRM, ERP, documents), data, and agents for AI use. In the [Master OS](https://aimaster.fi/tuotteet/master-layer) layer, we enforce strict prefix ordering, lock up to four explicit cache breakpoints, and isolate dynamic metadata to the uncached suffix. **Custom AI solutions** are delivered using an agile sprint model: one sprint is 3 development days. In a single 3-day sprint, refactoring a multi-step agent — such as the automated workflows we run in our own portfolio products like Lyyli.ai and BookyAI — turns repeated 40,000-token context loads into lightweight cache reads.

**Master Mind** is a suite of AI agents that runs on top of Master OS and handles business processes autonomously. In [Master Mind](https://aimaster.fi/tuotteet/master-mind) operations, we monitor `cache_read_input_tokens` and `cache_creation_input_tokens` metrics on every run (Anthropic Platform Docs, 2026). If a deployment accidentally drops the cache hit rate below target, the alert triggers immediately — not when the finance team opens the monthly cloud bill.

Want to see where AI agents will generate the highest euro-measured return in your operations — and how to build them on a cost-predictable architecture from the start? Start with a [Free Master Mind Analysis](https://aimaster.fi/analyysi).

## Frequently asked questions

### How much does prompt caching reduce AI agent token costs?

Prompt caching stores the model's precomputed key-value (KV) states for the unchanged beginning of a prompt. When a subsequent API call shares the exact same prefix, the model reads those states directly from memory at a 90–95% discount instead of recalculating every token from scratch (Anthropic Platform Docs, 2026; OpenAI API Docs, 2026).

### Why is my AI agent's prompt cache hit rate zero?

The most common mistake is placing a dynamic timestamp, session ID, or changing user variable at the beginning of the system prompt or directly on a cache breakpoint block. Because prefix hashes are cumulative, changing a single character at the start invalidates the entire cache and forces a 1.25x cache-write charge on every request (Anthropic Platform Docs, 2026).

### What is the minimum token length required for prompt caching?

A prompt prefix must exceed the model's minimum token threshold before caching activates. On OpenAI's GPT-5.6 and newer models, the minimum is 1,024 visible input tokens (OpenAI API Docs, 2026). On Anthropic's Claude models, the threshold ranges from 512 tokens on Claude Sonnet 5.5 and Opus 5.5 up to 4,096 tokens on Opus 4.6 and Haiku 4.5 (Anthropic Platform Docs, 2026).

### Does prompt caching lower the quality of an AI agent's responses?

No. Prompt caching reuses the exact mathematical key-value states of the identical prefix and generates new output tokens normally for the uncached suffix. It reduces time-to-first-token and input cost without lowering reasoning accuracy or altering tool call reliability (OpenAI API Docs, 2026).

### Do cached input tokens count toward API rate limits?

Yes. Both Anthropic and OpenAI count cached input tokens toward your organization's tokens-per-minute (TPM) rate limits alongside uncached tokens (OpenAI API Docs, 2026; Anthropic Platform Docs, 2026). Prompt caching cuts invoice cost and processing latency, not API rate-limit consumption.