Document AI · Technical document processing

Document Intelligence & Structured Extraction

Dense technical documents — inspection forms, census sheets, engineering reports — are still transcribed by hand into spreadsheets. This platform runs GPU-accelerated OCR on a remote machine over SSH, then extracts structured data from the result with LLMs, validated against a domain schema so the output is a dataset rather than a wall of text.

The problem

OCR alone does not solve document data entry. It converts pixels into text and leaves the hard part untouched: deciding which number is the span length, which date is the inspection date, and what to do when a field is missing or ambiguous.

Getting a language model to fill a schema is easy to demo and hard to trust — the same document can produce different answers on different runs, and without measurement nobody knows how often.

What I built

Extraction engine

  • Remote GPU-accelerated OCR executed over SSH, so heavy vision models run on the machine that has the GPU while the application stays lightweight.
  • HTML and Markdown chunkers that preserve document structure — tables stay tables — because structure is most of the signal in technical forms.
  • An LLM extraction engine with schema validation, integrated with both Ollama and vLLM so the same pipeline runs against locally served open-weight models.
  • Output as standardized JSON and CSV datasets, ready to load rather than ready to re-type.

Application

  • A visual schema editor, so a domain expert can define and adjust the target structure without writing JSON schema by hand.
  • Configuration helpers for model, chunking and prompt settings.
  • A live extraction viewer over server-sent events, showing fields being filled as the run proceeds.
  • A multi-run evaluation and benchmarking suite: run the same documents repeatedly, score against ground truth, and measure both accuracy and run-to-run consistency.

Why the evaluation suite matters

The benchmarking suite is the part I would keep if I could keep only one. It converts “the extraction looks good” into a number, and it makes regression visible when a model, prompt or chunking strategy changes. Without it, a document extraction system is a demo; with it, it is something you can put in front of a client and defend.

Outcome

Shipped an automated extraction and evaluation engine with multi-run accuracy and consistency scoring against ground truth.

Estimate, not a logged metric: manual transcription of roughly 20–30 minutes per report reduced to under two minutes of automated processing.

Stack

Note: this case study is deliberately written without client names, product names or customer data. The time saving is an engineering estimate; accuracy and consistency are measured by the platform's own evaluation suite against ground truth.