From PDF to Production: Sub-2% Extraction on Messy Docs

A finance team drops 40,000 scanned invoices into a bucket on Monday and asks for line-item totals by Friday. The PDFs have rotated pages, mixed languages, two-column layouts, and stamps over the amounts. Getting extraction error below 2% at that volume is not a model problem — it is a pipeline problem.

The failure mode nobody names

Most PDF extraction demos work on clean, born-digital documents. Production traffic is different: enterprise PDFs are often image-based scans, and tables regularly span page columns. A single-shot LLM call over raw text tokens will hallucinate cell boundaries and quietly average two rows into one. The European Commission’s regulatory framework for trustworthy AI treats such silent errors as systemic risk, not a UX bug.

The fix is to stop asking the LLM to read the PDF. Ask it to read a structured intermediate representation you already trust.

The four-stage pipeline

Each stage carries its own error budget. Layout parsing (a fine-tuned DocLayout model backed by Tesseract 5 for OCR fallback) targets a 99.5% region-recall SLO. Region typing pushes tables into a dedicated table-transformer; key-value blocks go to a smaller extractor. Only after typing does an LLM see the content, and it sees it under a strict JSON schema, not free text. This mirrors what NIST’s AI Risk Management Framework calls structured intermediate grounding.

A production confidence log for one batch looks like this:

2026-09-03T04:12:11Z extract.batch id=b-8821 pages=1247
  layout_recall=0.996 table_f1=0.981
  llm_schema_valid=0.994 human_queue=17 (1.4%)
  final_error_rate=0.017 model=qwen3-8b-instruct

Why routing matters more than the model

ENISA’s 2024 review of AI cybersecurity challenges foregrounds data preparation as a primary risk surface. A frontier model over messy input often scores worse than a smaller open model over clean, region-typed input. Cost-aware routing follows the same logic — send high-confidence tables to a cheap model, escalate ambiguous scans only. We use the same discipline described in our cost-aware LLM routing notes and inside our RAG engineering practice.

Compliance teams routinely ask whether extracted data falls under GDPR Art. 6(1)(f) legitimate interest when the source document carries personal data. Guidance from ANSPDCP is clear: the extraction step is processing, and the retention policy on the intermediate JSON must match the retention policy of the source document. Log it or lose the audit.

Where CAI Technology sits

We build extraction pipelines the way we build security telemetry: every stage emits a signal with a versioned threshold, checked in alongside the model weights. Sub-2% error is not achieved by a bigger model. It is achieved by a smaller LLM asked a smaller, well-typed question. If your PDF pipeline plateaued at 8% error and you cannot find where the bleed is, come look at how we debug retrieval and extraction inside our RAG platform.

Read further

• Debugging graph-shaped retrieval when text-only RAG stalls
• Threat surfaces every RAG pipeline inherits from day one
• When retrieval augmentation actively hurts extraction quality