Offline document intelligence for Uzbek-language archives

Problem

Uzbek is a low-resource language: documents mix Latin and Cyrillic script, apostrophes are inconsistent and many files are scans. Off-the-shelf RAG tools retrieve poorly, and cloud models were not an option for confidential documents.

Approach & architecture

• Researched before building: eight findings documents on model choice, Uzbek language support, ingestion and retrieval, plus a live bake-off of local models.
• Layout-aware ingestion with OCR for scans, followed by Uzbek normalisation — apostrophe repair, Unicode normalisation and Cyrillic-to-Latin transliteration.
• Hybrid dense and sparse retrieval with rank fusion and a reranker; a topic gate refuses questions the document does not cover; follow-ups are rewritten into standalone queries.
• Everything runs on one laptop within a planned memory budget: a quantised local LLM, the vector store in a container and a React UI with a side-by-side PDF viewer.

Result

• Hit@10 of 0.96 on a 30-question Uzbek golden set.
• Streaming answers with page citations, per-document summaries and insights, and a library mode that searches across all documents.

What made it robust

• Golden and follow-up evaluation sets with stored model-comparison reports, so model changes are measured rather than guessed.
• An implementation plan with phases, a memory budget and a risk register; phases 0–5 completed.
• Automated tests over normalisation, retrieval and the API.