LLM Cost Optimization: Cutting API Bills 50–90% (2026)

When to Use

• ✓Monthly LLM API bills exceeding $1,000 — at this scale, optimization pays for itself in days
• ✓Before scaling a feature from prototype to production — optimization at design time is 10x easier than retrofitting
• ✓When a model upgrade would improve quality but the cost increase is unacceptable
• ✓Building multi-tenant SaaS where LLM cost per user directly affects unit economics
• ✓Optimizing developer tooling or internal tools that run thousands of queries per day

How It Works

1. 1Model routing: classify queries by complexity and route to the cheapest model that can handle them. Claude Haiku ($0.80/M input) is 3.75x cheaper than Sonnet ($3/M). GPT-4o-mini is 16x cheaper than GPT-4o. For most applications, 60–80% of queries are simple enough for a cheap model.
2. 2Prompt caching: cache your system prompt and static context. Anthropic prompt caching cuts cached token cost by 90%. For agents with 2K+ token system prompts making 1M requests/month, prompt caching alone saves $2,400/month on Claude Sonnet.
3. 3Output constraints: set max_tokens to 1.5x your expected output length, not the model maximum. An unconstrained max_tokens doesn't guarantee longer output — it just keeps the door open for unexpected verbosity that costs money.
4. 4Batch processing: use the Batch API (Anthropic, OpenAI) for non-real-time workloads. 50% cost discount, responses within 24 hours. Perfect for: nightly data processing, bulk classification, report generation.
5. 5Response caching: cache LLM responses for repeated queries. A semantic cache (embed the query, check for similar past queries) can achieve 20–40% cache hit rates on high-volume applications, with near-zero marginal cost on hits.

Examples

from anthropic import Anthropic

client = Anthropic()

# Complexity classifier (uses cheap model)
def classify_complexity(query: str) -> str:
    response = client.messages.create(
        model='claude-3-5-haiku-20241022',  # Cheapest model for classification
        max_tokens=10,
        messages=[{
            'role': 'user',
            'content': f'Classify query complexity: simple (factual lookup, short answer) or complex (analysis, reasoning, long output). Query: {query}. Reply: simple or complex'
        }]
    )
    return response.content[0].text.strip().lower()

def cost_optimized_call(query: str, system: str) -> str:
    complexity = classify_complexity(query)
    model = 'claude-3-5-haiku-20241022' if complexity == 'simple' else 'claude-3-5-sonnet-20241022'
    
    response = client.messages.create(
        model=model, max_tokens=1024,
        system=system,
        messages=[{'role': 'user', 'content': query}]
    )
    return response.content[0].text

# Bad: leaves max_tokens unlimited
response = client.messages.create(
    model='claude-3-5-sonnet-20241022',
    max_tokens=4096,  # Maximum — you'll pay if it uses all 4096
    messages=[{'role': 'user', 'content': 'Summarize this article in 2-3 sentences.'}]
)

# Good: constrain to task requirements
response = client.messages.create(
    model='claude-3-5-sonnet-20241022',
    max_tokens=150,  # 2-3 sentences ≈ 50-100 tokens, buffer for safety
    messages=[{
        'role': 'user',
        'content': 'Summarize this article in 2-3 sentences (maximum 100 words). Article: ...'
    }]
)
# Also constrain in the prompt itself for double enforcement

Common Mistakes

• ✗Optimizing prematurely before measuring — don't guess which queries are expensive. Instrument your application first: log (model, input_tokens, output_tokens, cost) per request. You'll discover the actual cost drivers are often not what you expected.
• ✗Routing all queries to cheap models — some tasks genuinely require frontier model quality. Measure quality degradation when routing to a cheaper model; don't just assume the cost savings are worth it.
• ✗Forgetting tool definition tokens — tool definitions sent with every request can be 1,000–4,000 tokens. For agents making 1M requests/month, 2,000 extra tokens = $6,000/month on Sonnet. Cache tool definitions and use prompt caching on them.
• ✗Not monitoring costs after optimization — cost optimization can be undone by new features adding tokens, increased query volume, or provider price changes. Set up daily cost tracking with alerts for 2x budget overruns.

FAQ