How does structure-aware ingestion fix RAG accuracy at scale?

Answer

RAG accuracy comes from structure-aware ingestion and separate indexes, not query-time prompt tweaks.

Structure-aware chunking keeps headings, code fences, and tables intact with full breadcrumb paths.
LLM descriptions and dual Markdown plus CSV storage make tables searchable by meaning and readable at answer time.
Separate prose, table, and summary indexes run in parallel so strong matches never lose to noisy fragments.
Hard tenant partitions and idempotent batch upserts keep storage isolated and re-ingestion safe.

Applies toMulti-tenant corpusmillions of chunksprose, tables, code

Structure-Aware Multi-Index RAG Pipeline

Diagram structure

Structure Parser
Splits files along headings, code fences, and word boundaries while attaching Report > Performance > Benchmarks paths.
Table Enricher
Keeps each table whole with Markdown and CSV copies plus an LLM summary of what the table compares.
Doc Summarizer
Creates one file-level summary vector so broad document questions return complete overviews.
Batched Ingestor
Deletes old vectors before upserting new batches within API rate limits and tracks live progress.
Tenant Guard
Scopes every write and search to the tenant partition so data never leaks across tenants.
Parallel Merger
Queries prose, table, and summary indexes in parallel and ranks all hits on one relevance scale.

Failure Mode	Naive v1 (Fixed-Size Chunks)	Structure-Aware v2 (7 Ingestion Upgrades)
Mid-sentence cuts & broken code	FragmentedFixed token windows split sentences and treat `#` in code as headings	Intact boundariesParses real sections, respects code fences, and splits long sections at word boundaries
Missing citation context	OrphanedChunks lose parent headings and section order	AddressedEvery chunk stores its heading path and parent/section indices
Flattened, unreadable tables	CorruptedTables split across chunks with misaligned columns	Dual formatOne chunk per table storing full `Markdown` and a clean `CSV` copy
Semantic table queries miss	MissedCell values lack words like "faster" or "before/after"	Meaning-indexedLLM writes a semantic description indexed alongside the table
Broad "what is in this doc?" queries	No overviewReturns random low-context fragments	File summaryDedicated file-level summary vector answers document-wide questions
Prose crowds out tables	Unfair rankingOne shared index biases toward wordy paragraphs	Fair competitionSeparate prose and table indexes queried in parallel and merged by score
Multi-tenant & re-ingest risk	Leak & driftShared namespace and duplicate vectors on re-run	Isolated & idempotentHard tenant partitions, delete-before-write idempotency, and batched API calls

Parallel tenant-isolated retrieval

Diagram structure