Invoice Structured Extraction

Sonnet 4 is the most reliable at following a strict JSON schema on messy multi-column invoices in 2026. Gemini 2.5 Pro wins on very long multi-page invoices because of its larger context. GPT-4o trails on schema adherence when line items are in a wrapped table.

Alternatives: Gemini 2.5 Pro, GPT-4o

Forcing the model to emit a tool call against a defined schema is more reliable than free-form JSON parsing, even with JSON mode. Pydantic gives runtime validation and auto-retry with errors as input — the pattern that works in production.

Alternatives: Instructor (Python), Zod + OpenAI structured outputs, BAML, Outlines for constrained decoding

Vision LLMs can read scans directly, but Textract returns bounding boxes which help when you need to highlight the source region during human review. For EU data residency, Google Document AI in europe-west regions is the common pick.

Alternatives: Azure Form Recognizer, Tesseract (on-prem only), Skip OCR and rely entirely on vision LLM

Money math must be deterministic. Decimal with ROUND_HALF_UP and a 1-cent tolerance catches all real invoices including rounding quirks. Never use floats. Never trust the LLM to do arithmetic — have it emit numbers and verify them externally.

Alternatives: Integer cents (safest), LLM self-check

Invoice vendor names vary (ACME CORP vs Acme Corporation LLC). Fuzzy match handles typographic variance; embedding rerank handles semantic variance (dba names, subsidiaries). Both steps together get you to >99 percent match accuracy on a clean vendor master.

Alternatives: Trigram match in Postgres, Exact match with aliases table

The relevant metric is per-field accuracy, not per-document accuracy. Track vendor name, invoice number, invoice date, subtotal, tax, total, and line-item count separately. A 95 percent document accuracy can hide a 60 percent line-item accuracy.

Alternatives: Braintrust, Manual spot check

Vision LLM alone vs OCR plus LLM

Modern vision LLMs can read scans directly with 90-95 percent accuracy. Adding a dedicated OCR pass (Textract) costs an extra $0.015 per page but boosts accuracy to 98 percent on poor-quality scans and gives you bounding boxes for audit trails. For any invoice workflow that must survive a finance audit, use both.

Zero-shot extraction vs per-vendor templates

Zero-shot extraction with a strong schema handles the long tail of one-off vendors and is what makes the pipeline scalable. Templates are faster and cheaper for the top 20 vendors (who are often 80 percent of volume). A hybrid where the top-N vendors get a template-first extraction with LLM fallback wins on both cost and accuracy, but adds maintenance burden.

Reject on math mismatch vs post-correct

Rejecting every invoice with a 1-cent discrepancy to human review is conservative and correct for finance workloads — you never want the AI silently fixing numbers on a supplier bill. Auto-correcting rounding errors saves human time but erodes trust the first time it corrects something it should not have. Reject and escalate.

Line items wrap across pages, extractor emits duplicates or truncates

Mitigation: Pass all pages of a multi-page invoice in a single LLM call rather than per-page. Include an explicit instruction in the schema that line items must be listed once with the total quantity even if the description wraps. Validate line-item count against the line-item count declared in the invoice footer when present.

Totals do not reconcile due to rounding or fee lines

Mitigation: Use Decimal arithmetic with 1-cent tolerance. If mismatch exceeds tolerance, do not silently accept — route to human review with both the extracted fields and a diff showing the reconciliation gap. Track mismatch rate per vendor to spot systematic extraction bugs.

Vendor name on invoice does not match master vendor table

Mitigation: Layered resolution: exact match on tax ID if present, then fuzzy match on name with rapidfuzz, then embedding rerank. If top match scores below a confidence threshold, create a 'suggested match' in the review queue rather than auto-binding. Track vendor auto-bind rate as a quality metric.

Schema violation on edge-case invoices (missing currency, foreign characters, negative line items)

Mitigation: Implement a retry loop: on Pydantic validation failure, send the original prompt plus the schema error back to the model up to 2 times. If still failing, route to human review with the raw model output. Log every schema-error-retry pair and periodically use them as fine-tuning or evaluation examples.

PII and tax ID exposure in observability tools

Mitigation: Redact tax IDs, bank account numbers, and anything in an 'account number' field before sending to your LLM observability vendor. Never log raw extraction JSON for invoices with PII flags. Keep raw invoice storage in your own encrypted S3 bucket with access logging.