Learning path · 6 lessons
Claude Chat CLI with hybrid RAG retrieval
Build the hybrid retrieval pipeline behind a Claude-powered command-line chat assistant: chunk a real codebase, rank chunks with Okapi BM25 and cosine similarity, fuse both rankings with Reciprocal Rank Fusion, then benchmark and evaluate the results.
6 of 6 availableAll lessons are available
Expected outcome
What you will build
A from-scratch hybrid retriever, BM25 plus vector search fused with Reciprocal Rank Fusion, with a repeatable quality benchmark and deterministic response checks, all runnable offline without an API key.
Before you begin
Prerequisites
- Python 3.12 or later
- uv (the Python package and project manager)
- A local clone of the claude-with-the-anthropic-api repository with dependencies installed
6 available · 0 upcoming
Lessons in this series
- Lesson 1Chunk Python and Markdown for retrievalAn ingestion module that turns a repository's Python and Markdown files into small, retrievable chunks, each with a stable, citable chunk ID, assembled into the corpus later retrieval guides search.
- Lesson 2Implement Okapi BM25 from scratchA BM25 index that tokenizes prose and source-code identifiers, scores every chunk with term frequency, inverse document frequency, and length normalization, and returns deterministic ranked results without a search library.
- Lesson 3Build and search a vector indexA VectorIndex class that stores normalized chunk vectors, ranks results by cosine similarity with a deterministic tie-break, persists to JSON, and rejects vectors from an incompatible model or dimension.
- Lesson 4Merge rankings with Reciprocal Rank FusionA reciprocal_rank_fusion() function that combines multiple ordered result lists by position rather than source score, promotes chunks found by more than one retriever, and returns one deterministic result list.
- Lesson 5Benchmark retrieval qualityA repeatable retrieval benchmark that scores BM25 against labeled questions with Hit@k, Recall@k, and mean reciprocal rank (MRR), and names exactly which questions the retriever misses.
- Lesson 6Test model output with deterministic rulesA run_rule_checks() function that detects banned hype terms, repeated exclamation marks, and overly long responses, and reports every reason a response fails.