What Hybrid Search Looks Like When It Has to Run on Your Laptop

Suppose you are working in a codebase you did not write, and you type the following question: Where is ERR_CONN_RESET handled? The answer exists. One file contains a catch block that checks for exactly that constant and retries the request. A purely semantic search may not put that file first. It may return a connection-pool module, a general error-handling utility and a retry helper. All three are about the right topic. None of them necessarily contains the string you asked for. Now consider a second question: How do we stop a user from logging in too many times? The relevant function is called throttleAuthAttempts. Its body talks about a counter and a lockout window. It never uses the words "logging in" or "too many times". A keyword search may miss it. An embedding model has a much better chance of placing it near the question. This is the trade-off that hybrid retrieval exists to address, and it has been described well elsewhere: lexical search preserves exact identifiers, dense search preserves meaning, and neither preserves both. What is described less often is what happens after you accept that argument and actually build the system. Which BM25 parameters do you use? How does the tokenizer treat getUserById? What does a fused score of 0.03 mean to a user? Where do the chunks come from in the first place? ctx is a useful place to look for answers, because it is small enough to read and has to make all of those decisions in code. It is a TypeScript command-line tool (published on npm as yodaman-ctx, a maintained fork of an earlier project called Context Expert) that indexes local projects into SQLite, searches them with BM25 and dense vectors fused by RRF, and answers questions through a local model served by Ollama. There is no cloud provider in the code path. Everything below describes ctx 1.3.0; file paths are relative to the repository root. This article walks through that pipeline in the order data moves through it: chunking, embedding, the two retrievers, fusion, optional reranking, context assembly and evaluation. At each stage, the same two questions apply: what does this step preserve, and what does it give up? 1. Before Retrieval: Deciding What a Document Is A retriever can only return the units you gave it. In code, choosing those units is most of the work. Retrieval papers usually start with a corpus of passages. A code-search tool has to create those passages first, and the choice determines what can be found at all. ctx routes each file by type in src/indexer/chunker/chunker.ts. Config files become a single segment. Markdown goes to a section-based extractor. TypeScript, JavaScript and Python go to a tree-sitter extractor. Every other language is treated as one segment and split by size. The tree-sitter extractor (code-extractor.ts) walks the top-level statements of the file in order. Each definition — a function, class, interface, type alias, enum, or a const bound to an arrow function — becomes its own segment, together with its export keyword, decorators, and any comments that sit directly above it with no blank line in between. Everything that is not a definition (imports, constants, module setup, a Python if __name__ == "__main__": block) is grouped with its neighbours into a module segment. The comment at the top of the file states the invariant directly: every line of the file lands in exactly one segment. That invariant is worth noticing because, according to the project's changelog, it was not always true. Version 1.2.0 records that earlier indexing kept only functions and classes, so constants and module-level code were in no chunk and could not be found. The maintainers report that on 42 code questions, 23 answers were previously in no chunk at all, and that after the change the answer appeared in the top five results for 69% of questions instead of 21%. Those are the project's own measurements, not independent benchmarks, but they illustrate a point that is easy to forget: no ranking function can retrieve text that was never indexed. Markdown follows the same principle at a different granularity. markdown-extractor.ts uses the marked lexer to find headings — so a # inside a code fence is not mistaken for one — and makes each heading plus everything under it, up to the next heading, one segment. A table stays attached to its title. A heading with nothing under it is merged into the next section. Segments that fit within a size limit are kept whole. The limits are set per content type in chunker/config.ts: Content type Target size Overlap code 512 tokens 50 tokens docs 1,024 tokens 100 tokens config 256 tokens 25 tokens "Tokens" here is an estimate, not a tokenizer count: estimateTokens returns Math.ceil(text.length / 4). Oversized code segments are split by a recursive character splitter. Oversized documentation segments are split by a semantic chunker that embeds sentences and starts a new chunk where similarity between neighbours drops below 0.5. If that embedding call fails, the code catches the error, records a warning, and falls back to the size-based splitter instead of aborting the index. What this step preserves: the boundaries a reader would recognise — a whole function with its doc comment, a whole section with its heading. What it gives up: precision on very large units. A 2,000-line class is still one definition, so it will be split by size, and the four-characters-per-token estimate means the real token count of a chunk can differ from the configured limit. 2. The Dense Side: One Vector per Chunk, Scanned Exactly ctx uses a single local embedding model and compares every vector on every query. The default embedding model is BAAI/bge-large-en-v1.5, which the provider rewrites to Xenova/bge-large-en-v1.5 so that it can run through @huggingface/transformers in the Node process. It produces 1,024-dimensional vectors, L2-normalised. For BGE models the vector is taken from the first ([CLS]) token, which is how BAAI trained them; other models use the mean of all tokens. Until version 1.3.0, ctx mean-pooled BGE as well, so indexes built with earlier versions have to be rebuilt. The README states that the model is about 340 MB and is cached in ~/.ctx/models. An Ollama embedding model can be configured as a fallback, but the provider code refuses a fallback whose dimension differs from the primary model's, because the stored vectors would no longer be comparable. At query time, the vector store is not an approximate nearest-neighbour index. In src/search/store.ts, chunks are loaded from SQLite into an in-memory store with distanceMetric: 'cosine', and memory-store.ts scores the query against every stored Float32Array, sorts, and slices the top k. In other words, it is a brute-force scan. The comment above that code explains the choice. ctx builds this store afresh in every process, so the cost of building an HNSW graph would be paid on every search. The comment records 8.3 seconds to build HNSW over 3,714 vectors, compared with 43 ms for a scan of 58,728 vectors. The changelog adds that HNSW also held a second copy of every vector and could return few or no results under narrow metadata filters, and that it was removed in 1.2.0. This is a decision that only makes sense in context. For a long-running server with millions of vectors, an approximate index is the normal choice. For a CLI that starts, answers one question and exits, the build cost dominates, and an exact scan over tens of thousands of vectors is both simpler and fast enough. Two loading details matter at this scale. Chunks are read in batches paged by rowid > ? rather than by OFFSET, which the comment explains turned a 16-second load of a 23,000-chunk project into an index seek per page. And every loaded embedding is checked against the expected dimension; a mismatch throws an error that tells the user to re-index with --force, rather than silently comparing incompatible vectors. What this step preserves: paraphrase and intent. A question about "logging in too many times" can land near throttleAuthAttempts. What it gives up: any explicit guarantee that a rare exact string will dominate. The identifier has to compete with everything else in the chunk for influence over a single 1,024-dimensional vector. 3. The Lexical Side: BM25 in About 300 Lines The formula is the textbook one. The interesting decisions are in the tokenizer. ctx's BM25 retriever lives in src/sdk/rag/retrieval/bm25.ts, inside a vendored copy of the parts of the ContextAI SDK the tool uses. It has no external dependencies. The parameters are the common defaults, k1 = 1.2 and b = 0.75, and the IDF is the Robertson–Spärck Jones variant with a +1 inside the logarithm so it never goes negative: // IDF formula: log((N - df + 0.5) / (df + 0.5) + 1) const idf = Math.log((docCount - docFreq + 0.5) / (docFreq + 0.5) + 1); Enter fullscreen mode Exit fullscreen mode The per-term score is the standard saturated, length-normalised term frequency: const numerator = tf * (this.k1 + 1); const denominator = tf + this.k1 * (1 - this.b + this.b * (doc.length / this.avgDocLength)); score += idf * (numerator / denominator); Enter fullscreen mode Exit fullscreen mode Here, a "document" is a chunk. Candidate chunks are found from the union of the query terms' posting lists, filtered by file type or language when asked, scored, sorted, cut to k, and finally divided by the top score so the best BM25 hit is 1.0. The part that most affects code search is not the formula but the tokenizer: const CAMEL_BOUNDARY = /(? 1) tokens.push(word.toLowerCase()); for (const part of parts) tokens.push(part.toLowerCase()); } return tokens.filter((token) => token.length >= 2); Enter fullscreen mode Exit fullscreen mode Every run of characters that is not a Unicode letter or digit becomes a separator. A camelCase word is kept whole and also split at its case boundaries. Tokens shorter than two characters are dropped. Running it on a few identifiers shows what it does: Input Tokens ERR_CONN_RESET err, conn, reset ZX-4821 zx, 4821 getUserById getuserbyid, get, user, by, id HTTPServer httpserver, http, server get user by id get, user, by, id Keeping both forms serves two kinds of query. A query that contains getUserById verbatim still matches the whole-word token, which is rare and so carries a high IDF — exactly what BM25 is for. A query that says "get user by id" in plain words now matches the parts. Before version 1.3.0, the tokenizer kept camelCase as one token only, and that second query had no lexical match on the identifier at all. Other limits remain. ZX-4821 becomes two tokens, so a chunk containing ZX-4812 still matches on zx, and only the numeric token distinguishes the two. The parts of a camelCase word are also common words (get, user, id), so their IDF is low; they add a modest amount to a chunk's score rather than deciding it. It is still worth running a retriever's tokenizer on your own identifiers before assuming what "exact match" means. What this step preserves: exact rare strings — constants, error codes, function names typed verbatim — and, through the split parts, a lexical link between an identifier and the words it is made of. What it gives up: any connection between words that do not share a token, such as "authenticate" and checkLogin. 4. Fusion: RRF, Then a Rescale So the Number Means Something Reciprocal Rank Fusion solves the incompatible-scores problem. It creates a readability problem of its own. FusionService.search in src/search/fusion.ts runs the dense and BM25 searches in parallel with Promise.all, then fuses the two ranked lists. The underlying function is the standard one, with k = 60: $$ \text{RRF}(d) = \sum_{r \in R} \frac{1}{60 + \text{rank}_r(d)} $$ A chunk that is missing from one list receives zero from that list. The implementation also records each retriever's contribution, so a fused result carries a record of which list it came from and at what rank. Raw RRF scores are small. A chunk ranked first by both retrievers scores 2/61 ≈ 0.033. The changelog for 1.2.0 describes the practical consequence: results showed fusion values near 0.03, which looked like poor matches, and any minScore above that value removed every result. The fix is three lines: // Raw RRF tops out near 0.03 (2/61), which read as a poor match and made // any 0-1 minScore drop every result; divide by the best possible score. const bestPossible = (denseWeight + bm25Weight) / (k + 1); Enter fullscreen mode Exit fullscreen mode After this division, 1.0 means "ranked first by both", and about 0.5 means "ranked first by one and absent from the other". The order of results does not change; only the scale does. Applying it to a small example: Chunk BM25 rank Dense rank Scaled score A: the catch block for ERR_CONN_RESET 2 3 0.976 B: a changelog line mentioning ERR_CONN_RESET 1 8 0.949 C: a general retry helper — 1 0.500 Chunk A is first in neither list but first after fusion, because both retrievers placed it near the top. This is the behaviour hybrid search is designed for. There is one practical constraint on this example. Each retriever is called with the same topK as the final result count. ctx search defaults to 10, so a dense rank of 8 is inside the list. ctx ask uses final_k, which defaults to 5, so each retriever contributes at most five candidates and chunk B's dense rank of 8 would not be seen at all. In that case, B would score 1/61 ÷ 2/61 = 0.5. The candidate depth of each retriever directly bounds how much agreement fusion can detect. It is tempting to conclude that deeper lists are always better. A comment in fusion.ts records a test that points the other way. Fetching 50 candidates from each retriever lost an expected file on the project's eight-question fixture, with recall falling from 0.94 to 0.88. With k = 60, a chunk ranked seventh by both retrievers scores 2/67 ≈ 0.030, while a chunk ranked first by one and absent from the other scores 1/61 ≈ 0.016. When the lists are deep enough to contain most of a small corpus, middling agreement can outrank a strong single signal. The fixture is too small to settle the question for large projects, so the code keeps candidate depth equal to topK. minScore is also applied only after fusion. Before version 1.3.0, the same threshold was also passed to the dense search, where it was compared with raw cosine similarity, and to BM25, where it was compared with a score scaled to the best hit — three different scales for one number. The code also supports per-retriever weights; when either weight differs from 1.0, contributions are multiplied and the list is re-sorted. Nothing in the CLI's configuration exposes these weights, so at the default settings both retrievers count equally. What this step preserves: agreement. A chunk supported by both kinds of evidence rises above a chunk supported by only one. What it gives up: the magnitude of the original scores. A BM25 match that is far stronger than everything else and a BM25 match that barely beats second place both count as rank 1. 5. The Reranker That Was Turned Off A cross-encoder is often presented as a free improvement. In this project, it was measured and made optional. ctx includes a cross-encoder reranker (src/search/reranker.ts), using Xenova/bge-reranker-base. When enabled, it takes up to 50 fused candidates, scores each query–document pair, and returns the top k. In practice the limit is lower: each retriever contributes topK results, so the reranker sees at most 20 candidates at the default of 10. If the model cannot load, the code warns once and returns the results in their fused order rather than failing the search. It is not enabled by default. It runs only with ctx search --rerank; ask and chat do not rerank. The changelog gives the reason. On 42 questions over two projects, each scored by whether the chunk containing the answer came back, the maintainers report that hybrid search alone put the answer first 45% of the time against 26% with reranking, with mean reciprocal rank 0.56 against 0.45. Their note is that the cross-encoder helped with prose but ranked Markdown tables and code lower. The same release also records that the reranker had previously been passed the query without the document, so every result received the same score — a bug that would have been easy to miss without measuring. This is a small sample on two projects, and it should not be generalised to a claim about cross-encoders. What it does show is that a stage that improves results on one kind of content can make them worse on another, and that the only way to know which applies is to measure on your own content. 6. From Results to Prompt: Deduplicate, Order, Budget Retrieval ends with a ranked list. The model sees something else. For ctx ask, retrieved chunks pass through an assembler (src/sdk/rag/assembly/base-assembler.ts) before reaching the model. The steps run in a fixed order: Keep the top final_k results (default 5). Remove near-duplicates: a chunk whose word set has Jaccard similarity of 0.8 or more with an already-kept chunk is dropped. Include chunks in relevance order until the token budget (default 4,000) is used. Reorder the chunks that fit with the configured strategy (default: sandwich). Format as XML, with each carrying its file path and line range. The sandwich ordering is a response to the observation that models tend to use information at the start and end of a long context more reliably than information in the middle. With n results and the default start count of ⌈n/2⌉, the top half goes first and the remainder goes at the end in reverse order. For five results, the order becomes 1, 2, 3, 5, 4. The order of steps 3 and 4 matters, and until version 1.3.0 it was the other way round. When the budget ran after ordering, the chunks it considered last were those at the end of the sandwich — the positions the ordering reserves for the fourth- and fifth-ranked results — so a tight budget cut those first, while the least relevant chunks in the middle survived. Now the budget sees chunks in relevance order and drops the least relevant, and the sandwich is built from what remains. A test in src/agent/__tests__/assembler.test.ts covers this: with room for four of five chunks, the result is 1, 2, 4, 3, where the old order would have kept 1, 2, 3, 5. The budget loop still includes any chunk that still fits rather than stopping at the first that does not, so a short, lower-ranked chunk can be included after a long one is skipped. With five documentation chunks near the 1,024-token limit, a 4,000-token budget holds three of them. It is worth inspecting the final context rather than only the ranked list, because stages like these interact. ctx makes that straightforward: ctx ask --context-only --json returns the assembled context without calling the model. 7. Which Project? Routing Without a Model, Mostly Searching every indexed project is the safe default until there are a dozen of them. ctx can index many projects and search one or all of them. In ctx chat, a question first goes through QueryIntentClassifier in src/agent/query-router.ts, which uses string matching and no model call. Its rules, in order: Condition Intent Confidence Two or more project names mentioned multi-project 0.95 A multi-project keyword (e.g. "compare", "across") and at least one project name multi-project 0.85 Exactly one project name mentioned single project 0.9 No name, but a phrase like "this project" or "this repo" single project 0.85 A multi-project keyword with no project name general 0.6 Anything else general 0.5 A comment in the code explains the second-to-last rule: "compare error handling" with no project named is deliberately not classified as multi-project, to avoid false positives. What happens to a "general" question depends on state. If a project is focused (with /focus), the question stays in that project. Only when nothing is focused and at least two projects exist does the router ask the local model, with a 5-second timeout and a confidence threshold of 0.7; if the call fails or the confidence is too low, it searches every project. The changelog explains the ordering: when the model was consulted first, a local model often exceeded the 5-second limit, the fallback searched every project, and with a dozen projects that took about 50 seconds — past the agent's 30-second tool limit — so answers arrived with no sources. When multiple projects are searched, the multi-project service applies RRF at two levels: the dense and BM25 managers each merge results across projects, and the service then fuses the dense and BM25 lists. 8. Measuring Retrieval Separately from Answers If the right chunk never enters the context, no prompt can recover it. ctx ships an evaluation command, ctx eval, with metrics implemented directly in src/eval/metrics.ts: reciprocal rank, precision@k, recall@k, hit rate, NDCG@k with binary relevance, and average precision. Each golden entry pairs a question with the file paths expected in the results: { "id": "fixture-002", "query": "Where is the JWT token verified?", "expectedFilePaths": ["src/auth/middleware.ts"], "tags": ["auth"], "source": "manual" } Enter fullscreen mode Exit fullscreen mode The repository includes a small fixture project (eight source files) and a golden set of eight queries in fixtures/eval/. A GitHub Actions workflow runs this evaluation on pull requests that touch search, indexing, agent or eval code, and weekly on a schedule. scripts/check-eval-thresholds.js fails the build if MRR falls below 0.6, precision@k below 0.5, or hit rate below 0.8. The script's comment describes these as a minimum quality floor, intentionally lower than the configuration defaults of 0.7, 0.6 and 0.85. Eight queries over eight files is a regression check, not a benchmark, and the project does not present it as one. Relevance is also judged at the file level, so a result counts as relevant if it comes from the right file even if it is the wrong chunk of that file. Within those limits, it does the job it is designed for: a change that breaks retrieval on a known set of questions fails before it is merged. Conclusion: The Algorithm Is the Short Part The core of ctx's retrieval is short. RRF is a loop over two lists. BM25 is two formulas and an inverted index. Cosine similarity over normalised vectors is a dot product. The original argument for hybrid search — that lexical and semantic retrieval preserve different evidence — is fully visible in a few hundred lines. Most of the remaining code, and most of the changelog, is about everything around those lines: which units get indexed, how identifiers are tokenised, what a score of 0.03 means to a person reading it, how many candidates each retriever contributes, whether a reranker helps on this kind of content, and what survives a token budget. Several of the fixes recorded in the changelog were not ranking improvements at all. They were cases where the right text was not indexed, where a score was misleading, or where a stage was silently doing nothing. That is the practical lesson from reading this codebase. Choosing hybrid search is a decision that takes a paragraph to justify. Making it work takes attention to the steps before and after it. The Practical Takeaway If you are building retrieval over source code, a few checks follow directly from the code above: Check coverage before ranking. Confirm that every line of a file ends up in some chunk. Constants and module-level code are easy to lose with definition-only chunking. Run your tokenizer on your identifiers. Look at what happens to camelCase, snake_case and hyphenated codes. "Exact match" means whatever the tokenizer says it means. Look at candidate depth, and measure before changing it. RRF can only reward agreement that appears inside both lists. If each retriever returns five results, a document ranked sixth by one of them gets no credit for it. Deeper is not automatically better, though: deep lists let middling agreement outrank a strong single signal. Make scores readable. If users or downstream code filter by score, rescale RRF output so its range has a meaning. Measure optional stages on your own content. A reranker, a router or an approximate index is a trade-off, not a guaranteed improvement. Inspect the final context, not just the ranking. Ordering and token budgets change which chunks the model actually sees. ctx is MIT-licensed and available at github.com/Yoda-Man/ctx. The files referenced in this article are under src/search/, src/indexer/chunker/, src/sdk/rag/ and src/eval/. References Robertson, S., & Zaragoza, H. (2009). The Probabilistic Relevance Framework: BM25 and Beyond. Foundations and Trends in Information Retrieval, 3(4), 333–389. https://doi.org/10.1561/1500000019 Cormack, G. V., Clarke, C. L. A., & Büttcher, S. (2009). Reciprocal Rank Fusion outperforms Condorcet and individual rank learning methods. Proceedings of SIGIR 2009, 758–759. https://doi.org/10.1145/1571941.1572114 Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024). Lost in the Middle: How Language Models Use Long Contexts. Transactions of the Association for Computational Linguistics, 12, 157–173. https://doi.org/10.1162/tacl_a_00638 Xiao, S., Liu, Z., Zhang, P., Muennighoff, N., Lian, D., & Nie, J.-Y. (2024). C-Pack: Packed Resources For General Chinese Embeddings. Proceedings of SIGIR 2024. arXiv:2309.07597. (The paper introducing the BGE embedding models.) Yoda-Man. ctx source code and CHANGELOG.md, version 1.3.0. https://github.com/Yoda-Man/ctx