Prompt Caching & Cost Optimization: 90% Savings on Repetitive Prompts

Claude Sonnet 4 has the highest absolute savings from caching: standard input rate $3/M tokens, cached read rate $0.30/M (90% off). A 10K-token system prompt saves $0.027 per request at cache hit. Compare: Claude Haiku 4 saves $0.00072 per request from same cache (base rate $0.80/M, cached $0.08/M) — caching is 37x more valuable on Sonnet than Haiku in absolute terms.

Alternatives: Claude Haiku 4 (caching saves proportionally less due to lower base cost), GPT-4o (auto-caching, no code changes), Gemini 2.0 Flash (context caching with explicit TTL API)

Anthropic's cache is prefix-based: the prompt must start with identical content for a cache hit. Design your prompt structure as: [system prompt (static)] → [tool schemas (static)] → [few-shot examples (static)] → [RAG docs if semi-static] → [user messages (dynamic)]. Any deviation in the static prefix causes a cache miss and a cache write charge.

Alternatives: Hash-based (for reordered content), Semantic cache (Redis + embeddings, for paraphrased queries)

Anthropic's default cache TTL is 5 minutes. For workloads with >12 requests/minute (720/hour), the cache stays warm indefinitely. For lower-traffic workloads, calculate your break-even: cache_write_cost / (standard_rate - cache_read_rate). A 10K-token prefix breaks even at 1.25 cache hits (write premium = 1.25x standard, read = 0.10x). Even 2 hits in the TTL window yields net savings.

Alternatives: Anthropic extended TTL (available on higher tiers), Application-level keep-alive (periodic no-op requests to refresh TTL), OpenAI auto-caching (1-hour TTL, simpler)

Helicone captures `cache_read_input_tokens`, `cache_creation_input_tokens`, and `input_tokens` from every Anthropic response and displays a live cost dashboard. Free tier covers up to 100K requests/month. Without monitoring, teams don't know if caching is actually working — misplaced cache_control blocks or dynamic content before the cache marker are silent failures.

Alternatives: Custom Prometheus metrics (parse usage from Anthropic API response), Langfuse cost tracking, DataDog LLM Observability

Prompt caching saves on the per-token LLM computation cost. A semantic cache (store query embedding → LLM response, check similarity at query time) saves the entire LLM call cost for repeated/similar queries. At 20-30% query repeat rate, a semantic cache on top of prompt caching reduces total LLM costs by another 20-30%. Combined savings: 60-80%.

Alternatives: Momento (managed semantic cache), Upstash Vector + Redis, LangChain SemanticCache

For workloads where real-time response is not required (nightly document processing, bulk evaluations, data enrichment), Anthropic Batch API provides 50% cost reduction on top of prompt caching. Combine both: use prompt caching for the static system prompt + batch API for the processing run. A 10K-token system prompt on 100K batch requests costs $0.30 (cached read) vs $300 (standard) — a 1,000x saving.

Alternatives: OpenAI Batch API (50% discount), Scheduled jobs with Inngest or Temporal

Mitigation: The most common mistake: placing a timestamp, request ID, or personalized content BEFORE the cache_control block. Anthropic's cache is prefix-based — any change before the marker causes a full cache miss and a write charge. Validate your prompt structure by checking the API response: `usage.cache_read_input_tokens` should be >0. If it's always 0, your cache_control placement is wrong.

Mitigation: Routes with <6 requests per 5-minute window will write more cache entries than they read — net cost increase. Identify these routes with your monitoring dashboard and disable caching for them (simply omit the cache_control block). Apply caching only to routes where: (requests_per_5min > 2) AND (cached_tokens > 2048).

Mitigation: You deploy a system prompt update, but the old cached version serves requests for up to 5 minutes. For most use cases, this is acceptable. For critical updates (safety rule changes, pricing changes), use a cache-busting strategy: include a version token in the prompt that changes on deployment (`# System v2.1.4`). The version change invalidates the old cache entry immediately.

Mitigation: OpenAI auto-caches prompts >1,024 tokens with a 1-hour TTL, but cache hit rates are not visible in the standard API response (only in the usage dashboard). Low-traffic hours cause cache expiry and the next request pays the full rate. Monitor total input token costs hourly — a 2-3x spike during low-traffic hours indicates cache expiry is a significant cost driver. Solution: schedule a warm-up request at the start of each hour for critical routes.