LLM-as-Judge: Using AI to Evaluate AI Outputs (2026)

When to Use

• ✓Evaluating open-ended outputs (summaries, explanations, creative text) where rule-based metrics fail
• ✓Scaling evaluation to thousands or millions of examples where human labeling is cost-prohibitive
• ✓Building continuous eval pipelines that monitor production LLM quality automatically
• ✓Comparing two model versions on the same task set to decide which to deploy
• ✓Evaluating RAG faithfulness — checking if answers are grounded in retrieved context

How It Works

1. 1Choose evaluation dimensions: correctness (factually accurate?), faithfulness (supported by context?), helpfulness (addressed the question?), harmlessness (no harmful content?). Each dimension needs a separate criterion in the judge prompt.
2. 2Design the judge prompt: provide the question, the answer to evaluate, and optionally a reference answer or context. Ask for a score (1-5 or 1-10) AND reasoning. Reasoning is essential — it catches judge errors and explains low scores.
3. 3Use a different, stronger model as judge than the one being evaluated. Evaluating GPT-4o outputs with GPT-4o introduces self-preference bias. Use Claude as judge for GPT-4o outputs and vice versa.
4. 4Mitigate position bias: for pairwise comparisons (A vs. B), run the comparison twice with A/B order swapped. A clear winner should score higher in both orderings.
5. 5Calibrate the judge: create a small set of human-labeled examples and measure judge-human agreement (Spearman correlation or Cohen's kappa). A well-calibrated judge achieves 0.75+ correlation with human experts.

Examples

import anthropic
import json

client = anthropic.Anthropic()

FAITHFULNESS_PROMPT = '''
You are evaluating whether a response is faithful to the provided context.

Context: {context}
Question: {question}
Response: {response}

Evaluate faithfulness: does the response contain ONLY information that can be verified from the context?

Score:
1 = Response contains significant hallucinations or unsupported claims
2 = Response contains some unsupported claims 
3 = Response is mostly faithful with minor unsupported details
4 = Response is faithful with trivial extensions
5 = Response is completely faithful, every claim verifiable from context

Return JSON: {"score": N, "unsupported_claims": ["list of any claims not in context"], "reasoning": "brief explanation"}'''

def judge_faithfulness(context, question, response):
    result = client.messages.create(
        model='claude-3-5-sonnet-20241022',
        max_tokens=300,
        messages=[{'role': 'user', 'content': FAITHFULNESS_PROMPT.format(
            context=context, question=question, response=response
        )}]
    )
    return json.loads(result.content[0].text)

def pairwise_compare(question, response_a, response_b, criteria):
    # Run A vs B
    score_ab = judge_pairwise(question, response_a, response_b, criteria)
    # Run B vs A (swapped)
    score_ba = judge_pairwise(question, response_b, response_a, criteria)
    
    # Consistent winner in both orderings
    if score_ab['winner'] == 'A' and score_ba['winner'] == 'B':
        return 'A'  # A wins regardless of position
    elif score_ab['winner'] == 'B' and score_ba['winner'] == 'A':
        return 'B'  # B wins regardless of position
    else:
        return 'tie'  # Inconsistent — likely a marginal difference

Common Mistakes

• ✗Using the same model as judge and generator — self-evaluation inflates scores due to shared biases and writing style preferences. Always use a different model family for judging than the one being evaluated.
• ✗Asking for a single holistic score — holistic scores conflate multiple quality dimensions and produce less actionable feedback. Break into specific dimensions: correctness, completeness, conciseness, tone. Separate scores are more useful for diagnosing failures.
• ✗Not validating judge calibration — an uncalibrated judge may systematically score 3-4 regardless of actual quality, or show strong recency/position bias. Always correlate judge scores with 50+ human labels before using the judge in production.
• ✗Using LLM judge for safety evaluation — LLM judges are unreliable for detecting subtle policy violations, jailbreaks, or harmful content. Use specialized safety classifiers (Llama Guard, OpenAI Moderation API) for safety evaluation, not general-purpose LLM judges.

FAQ