Building an enterprise AI platform: permission-aware RAG that holds up in production
A reference architecture for enterprise AI: hybrid retrieval, user-scoped tools, fail-closed access control, event-driven deletes, and eval-gated CI/CD.
HLAB field notes
RAG
Stories and analysis filed under RAG.
07 stories
Test model output with deterministic rules
Add deterministic rule checks to a prompt-evaluation workflow, then measure exactly where simple substring rules produce false positives and false negatives.
Benchmark retrieval quality
Turn six labeled retrieval questions into a repeatable BM25 quality report using Hit@k, Recall@k, and mean reciprocal rank, then use the report to find what lexical retrieval misses.
Merge rankings with Reciprocal Rank Fusion
Combine BM25 and vector rankings with Reciprocal Rank Fusion, promoting chunks found by multiple retrievers without ever comparing their incompatible raw scores.
Build and search a vector index
Build a small exact vector index that stores L2-normalized vectors, ranks results by cosine similarity, and rejects vectors from an incompatible model or dimension.
Implement Okapi BM25 from scratch
Build the keyword-ranking function behind a local retrieval pipeline: preserve identifiers, calculate Okapi BM25 scores, and return deterministic ranked results without a search library.
Chunk Python and Markdown for retrieval
Build the ingestion pipeline that splits Python and Markdown files into stable, citable chunks, the foundation every retrieval guide in this series searches.