Many business PDFs are not streams of paragraphs. They are spatial documents: tables, columns, forms, diagrams, screenshots, handwriting, stamps, footnotes, and visual relationships all carry meaning. A 2026 document-AI system should preserve that structure and choose text, layout, vision, and structured extraction per document—not flatten every page into plain text by default.
RAG still matters. The change is that retrieval and evidence can now be multimodal and layout-aware.
Why plain extraction fails
A conventional parser may produce readable words while destroying the relationships that make them useful.
Common failures include:
- a table's values detach from their row and column headers;
- two columns are interleaved in the wrong reading order;
- a chart caption survives but the chart's data does not;
- checkboxes, signatures, stamps, and handwriting disappear;
- headers and footers overwhelm the meaningful content;
- a scanned page returns weak OCR without a confidence signal;
- a diagram's labels are extracted without their connections;
- page and section locations are lost, making citations unverifiable.
The result can look searchable while returning evidence that no longer says what the original page said.
Classify documents before choosing a pipeline
Run a lightweight intake step that records file type, native text availability, page count, language, scan quality, layout complexity, table density, image content, and required citation precision.
Then route accordingly:
Native text and simple layout
Use structured text extraction, preserving headings, lists, page boundaries, links, and document metadata. This remains the cheapest and most reliable path for well-formed material.
Scans and photographed pages
Use OCR with page images and confidence data. Flag low-confidence regions for a visual model or human review when the field matters. Do not silently index corrupted identifiers, amounts, or dates.
Tables, forms, and invoices
Extract both the visual region and a structured representation. Preserve row and column headers, merged cells, units, field labels, bounding boxes, and page location. Validate important values with domain rules.
Charts, diagrams, and image-heavy pages
Keep the original image or region available to a vision-language model. Text extracted from labels may support search, but answering a question about a trend, connection, or visual anomaly requires the visual evidence too.
Mixed documents
Use a page- or region-level pipeline. A 100-page report may contain ordinary prose, five critical tables, and two scanned appendices; it should not pay the same processing cost on every page.
Preserve an evidence object, not just a chunk
Each retrievable unit should retain enough information to reconstruct and verify its meaning:
- stable document and version ID;
- page, section, and region coordinates;
- native or OCR text;
- heading and neighboring context;
- table or form structure where relevant;
- image reference or visual embedding;
- source permissions and tenant;
- parser/model version and confidence;
- a stable preview or deep link.
This evidence object supports several retrieval modes without duplicating the source of truth.
Retrieval can combine four signals
Lexical search
Best for exact names, clauses, product codes, invoice numbers, and uncommon terminology.
Semantic retrieval
Useful when the question and source express the same idea with different wording.
Structural filters and queries
Dates, document types, tenants, fields, table columns, and source authority should often be handled deterministically rather than embedded into prose.
Visual retrieval
Useful when similarity depends on page appearance, diagrams, screenshots, or regions whose meaning is not captured by extracted text. Late-interaction multimodal approaches can compare a question with page-level visual and textual representations without requiring one lossy summary upfront.
The 2026 ACL survey on multimodal retrieval-augmented generation treats retrieval over text, images, and document layout as a distinct system problem, reflecting this shift beyond text-only chunking.
Long context does not remove document engineering
A large context window can be excellent for a bounded set of selected pages. It does not decide which files a user may access, remove stale versions, correct broken reading order, or guarantee that a model attends to the decisive table among hundreds of pages.
Use long context after document selection, not as a replacement for source governance. A practical system may retrieve a small group of relevant pages, preserve their visual form, and then let the model reason across them together.
Add iterative research only when the task needs it
Simple questions should use simple retrieval. Complex tasks may need an agentic retrieval loop that searches multiple sources, inspects evidence gaps, follows references, and stops on a defined budget.
Examples include comparing clauses across document versions, reconciling an invoice with a purchase order and policy, or combining an internal report with current public evidence. The agent should keep an evidence ledger so each material claim can be traced back to the exact page or region.
Permissions and provenance survive every representation
If one page becomes OCR text, a table object, an image crop, and several embeddings, each derivative must carry the source's access policy and deletion identity. When the source is removed or access changes, every representation and cache must follow.
Apply permissions before retrieval and generation. Also protect page previews, signed URLs, thumbnails, exports, traces, and conversation history. A text answer that respects permissions is not enough if its visual citation exposes the restricted page.
Evaluate by document type and task
One average retrieval score can hide systematic failures on the documents that matter most. Build test groups for native PDFs, scans, tables, forms, charts, multi-column pages, handwriting, and conflicting versions.
Measure:
- parsing and reading-order correctness;
- OCR and field accuracy for important values;
- table structure preservation;
- retrieval coverage across text and visual evidence;
- citation page/region correctness;
- claim support and conflict handling;
- abstention on unreadable or missing evidence;
- permission isolation;
- per-page and per-answer cost and latency.
For extraction-heavy workflows, compare values against labeled fields. For open-ended answers, review whether the cited region actually supports the claim. Model graders can help scale semantic review, but critical amounts and identifiers deserve deterministic checks.
Control cost with selective depth
Multimodal processing can be more expensive than text extraction. Keep it practical:
- fingerprint files so unchanged pages are not reprocessed;
- use native structure when it is reliable;
- classify pages and apply vision only where it adds value;
- cache versioned extraction artifacts;
- retrieve before sending full-resolution pages to a model;
- use smaller models for classification and validation where measured quality allows;
- monitor cost by document type and successful task, not only by token.
The objective is not to run the largest vision model over every page. It is to avoid losing material evidence while spending depth where the document requires it.
Production checklist
- [ ] Authoritative source, owner, version, and deletion path
- [ ] Page-level document classification
- [ ] Reading-order and layout preservation
- [ ] OCR confidence and low-quality fallback
- [ ] Structured handling for tables and forms
- [ ] Visual evidence retained for charts and diagrams
- [ ] Hybrid retrieval with trusted metadata filters
- [ ] Page- or region-level citations users can open
- [ ] Permissions propagated to every derived artifact
- [ ] Evaluation split by source type and user task
- [ ] Selective multimodal processing and cost budgets
The right architecture is often mixed: native parsing for ordinary text, OCR for scans, structured extraction for forms and tables, visual reasoning for the pages that need it, and retrieval to select the evidence. “Multimodal” should describe preserved meaning, not an expensive model call added to a broken ingestion pipeline.