Document Retrieval-Aware Chunking (D-RAC): Universal Retrieval-Aware Ingestion of Enterprise Documents via PDF Normalization and Multimodal Markdown Conversion
Abstract
Retrieval-Augmented Generation (RAG) systems over enterprise knowledge bases must ingest heterogeneous document formats -- PDFs, Word documents, presentations, and scans -- whose content is locked inside complex visual layouts, multi-column pages, and dense tables. Rule-based extraction and OCR destroy reading order, flatten tables, and lose heading hierarchy, while fully agentic chunking over extracted text incurs high token costs and hallucination risk. We present Document Retrieval-Aware Chunking (D-RAC), an extension of our Web Retrieval-Aware Chunking (W-RAC) framework to arbitrary document formats. D-RAC first normalizes any input document into PDF, exploiting the fact that virtually every format has a faithful, deterministic PDF rendering. A single multimodal LLM pass then converts rendered pages into retrieval-optimized Markdown -- rewriting tables as self-contained prose statements and preserving heading hierarchy -- after which chunking proceeds exactly as in W-RAC: deterministic parsing into ID-addressable units followed by lightweight LLM-based chunk planning over identifiers rather than text. Source text is never regenerated during chunking, preserving W-RAC's cost, determinism, and observability benefits while unlocking every renderable format as a first-class input. On the 236-document, 795-page PDF subset of the RAG-Multi-Corpus benchmark spanning five enterprise domains, D-RAC converts and chunks the entire corpus in 72 minutes with zero errors, producing 1,748 retrieval-ready chunks. Compared to agentic chunking with frontier LLMs, D-RAC reduces chunking-stage output tokens by 95.7%, cutting chunking cost by 77.8% (GPT-4.1 pricing) to 85.6% (Gemini 2.5 Pro pricing) and chunking time by 75%. D-RAC scales linearly to documents of 500+ pages.
Community
D-RAC: Universal Retrieval-Aware Ingestion of Enterprise Documents
Enterprise RAG has to ingest PDFs, DOCX, PPTX, XLSX and scans — formats where text extraction breaks reading order, flattens tables and loses heading hierarchy. Agentic chunking repairs some of that, but it regenerates the whole document as output tokens, which is where the cost and the hallucination risk live.
D-RAC extends our earlier W-RAC framework to arbitrary formats with two additions. First, every input is deterministically normalized to PDF and rendered to page images, since virtually any document format has a faithful PDF rendering. Second, a single multimodal LLM pass converts those pages into retrieval-optimized Markdown: every table row becomes one self-contained sentence carrying its column headers, disjunctive merges ("16 or 20 years") are forbidden, decorative imagery is dropped, and heading hierarchy is made explicit. Chunking then proceeds exactly as in W-RAC — deterministic parsing into ID-addressable elements, then LLM planning over identifiers, never text. The multimodal model touches content once; source text is never regenerated.
Every agentic chunking pipeline I've run hits the same wall: the PDF parser decides what the retriever ever sees. So the real test here isn't whether retrieval-aware chunking beats fixed-size — it's whether the normalization step survives tables that span pages and headings buried in footers. If the markdown conversion mangles those, you've just moved the failure from chunking to parsing, and the retrieval-aware part is optimizing garbage. I'd want retrieval quality measured against a bad-parser baseline, not just against naive chunking. That's the number that tells me if this is worth wiring into prod.
This is an automated message from the ResearchStudio team.
We created an interactive ResearchStudio Reel for this paper. It includes a visual poster, a video, and a blog, all available for download in editable formats.
Open the ResearchStudio Reel →
Download all files from Hugging Face
Please give this comment a thumbs up if you find the Reel helpful!
Want to explore or create Reels for more papers? Visit the ResearchStudio demo.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Structure-Aware Semantic Chunking with Title-Chain Prefixes: A 1600-Query Evaluation and the Measurement Trap in Text-Transform Ablations (2026)
- VecTree-RAG: An Agentic Retrieval-Augmented Generation Framework Combining Vector and Tree Retrieval for Efficiency and Accuracy (2026)
- Beyond Top-K: Replacing Black-Box Retrieval with Interpretable Agentic Operations (2026)
- What the Reranker Sees: Multi-Aspect Page Annotation for Long-Document Multimodal Question Answering (2026)
- Semantic Compression Trees: Multi-Resolution Knowledge Retrieval via Hierarchical Semantic Residuals (2026)
- Evaluating Modern RAG: Textual, Multimodal, Dense, and Late Interaction Pipelines (2026)
- HAM-RAG: Hierarchy-Aware Multimodal RAG for Structure-Faithful Interleaved Generation (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Get this paper in your agent:
hf papers read 2609.24220 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
