Inside Alaan's brain

Ranking across sources, measuring whether the ranking works, and getting the model to search at all.

A product manager asked our internal agent where a piece of onboarding behavior was implemented. The agent searched the codebase, pulled back eight results, read them, and answered that the code did not exist.

It existed. The identifier from the question had been sitting in the index the whole time, and a direct search of the right repository returned it on the first try.

Nothing had failed. That is the part worth sitting with for a moment. The search returned a full result set, at normal latency, at the top of the score range, holding real code out of a real repository. It was the wrong repository. It had been the wrong repository for every question anyone asked, for months.

That agent answers questions people are about to act on. What a spend limit actually evaluates. Whether a receipt satisfies a particular VAT rule. Why one card declined at one merchant on one evening. Which of two similar-looking numbers is the one a customer sees on a statement. Answers like those get pasted into customer conversations, often within the hour. A retrieval layer that fails quietly is a poor fit for that job.

An earlier post, How we built Alaan's brain, described the shape of the knowledge base underneath that agent: one store for every source, thin connectors, a context sentence written before every embedding, two retrieval modes. This one goes several levels further down, into three problems.

Ranking comes first. When a single question searches several sources, the results arrive carrying scores from different producers, and those scores are not the same unit.

Then measurement, because retrieval fails without complaining, and the assertion most people reach for passes cleanly during a total outage.

Then routing, which is the failure where the retrieval stack is in perfect health and the model never calls it.

What ties the three together is the property that made that first answer so expensive. None of them throws. The agent receives a healthy-looking result set, finds nothing relevant in it, and reports that the thing does not exist. From the outside, a broken search and a genuine finding are the same shape.

A score only means something inside the path that produced it

Brain picks a retrieval mode from the size of the thing it is searching.

A small collection skips retrieval entirely. Every chunk comes back in reading order, up to a cap that protects the model's context window. There is nothing worth ranking when the model can read the whole corpus, and the read costs one query rather than an embed, two index searches, a fusion, and a rerank.

Anything larger goes through hybrid retrieval, which is the standard arrangement. It is worth writing out precisely, because everything in this section turns on the details.

Three of those steps can be the last thing to touch a chunk, and each one hands back a different kind of number.

A reranked score runs from roughly 0 to 1. A model has read the question and the passage together and answered how well the second addresses the first. Of the three, it is the only one that means relevance.

A rank-fusion score sits near 1/60 and is a function of where a chunk placed in each list. Nothing else goes into it. Outside the particular set of lists that produced it the number means nothing at all, though inside that set it orders correctly.

The whole-corpus read ranks nothing. It has no opinion to express, so it stamps a constant on the way out. The constant was 1.

One relevance score, one positional artifact, one placeholder. All three are floating point numbers between 0 and 1, and the placeholder is parked at the top of the range the real one uses.

Why one collection is fine and several are not

None of this matters while you are searching one collection. Everything comes out of one path, every score is in one unit, and sorting them is valid.

It stops being valid the instant a single question reaches several collections, which for us is the ordinary case on two surfaces. Code search ranks across every indexed repository at once, because "how does this work" never arrives with the repository attached. A knowledge question asked in a Slack channel can reach that channel's own reference material and the asker's personal uploads in the same turn.

The original implementation did the obvious thing, which is usually where these live. One search per collection, concatenate the results, sort the union by score, descending.

Ask for eight results and you get eight chunks of one small repository, in file order, whatever it was you asked.

The reflex at this point is to argue about the constant. No value works.

Set it to 1 and the unranked collection takes every slot. Set it to 0 and it takes none, so a small reference document holding the actual answer never reaches the model. Set it somewhere in the middle and the outcome turns on how generous the reranker happened to be about the competing passages that day. The path has no opinion about relevance, so whatever number it emits is fiction. The bug was never the value. The bug was that a value was emitted.

Normalizing per collection is the other reflex, and it fails for a duller reason. Min-max scaling needs a spread, and the whole-corpus path has none, so the transform is undefined on exactly the input that caused the trouble. Where a spread does exist, rescaling makes two distributions overlap without making them mean the same thing. A 0.9 out of a reranker says a passage answers the question. A 0.9 produced by stretching rank positions says a chunk landed near the top of a list that may have been ordered by something else entirely. Making the second look like the first is how you manufacture a confident number with nothing behind it.

Two questions usually arrive about here, and the answers shaped the design.

Why are the sources separate collections at all? Merging everything into one index would delete the multi-source ranking problem outright. It would also delete the permission model. A collection is the unit of ownership and the unit of access: a Slack channel's reference material, one person's uploads, one repository. Read access is decided per collection before retrieval runs, and an answer built from one collection may be disclosed in places an answer built from another may not. A single index turns every access decision into a post-filter over retrieved rows, which is the design where one filtering bug becomes a disclosure. The same store also backs customer-facing conversations and internal ones, and several of the internal tools query across every customer with no tenant filter, because that is genuinely the question being asked. A collection is what stands between those two audiences. The separation is the security boundary, so the ranking problem is ours to solve rather than to avoid.

Why keep the whole-corpus read at all? For one small document it beats retrieval outright. The model sees every chunk in reading order, so there is no relevance judgment to get wrong and no passage that ranking can bury. It is also the cheapest good answer available. The mode earns its place. What went wrong was never that it existed. It was what it reported about itself once its output met somebody else's.

How we proved it

A wrong answer out of a language model has a hundred available explanations, and most of them point at the model. That is the honest reason this ran for months.

The test that settled it took about a minute. We ran two completely unrelated queries against the live index and compared the result sets byte for byte. Identical. A result set that does not move when the query moves is proof that the query never reached the ranking.

The index held 14 code collections then, three of them small enough for the whole-corpus path. Both queries came back with 8 of 8 hits from the same small repository, drawn from two files, every one at score 1. The two largest repositories, which between them hold the code that answers those questions, did not appear in either result. One of the queries named an identifier that is in the index and comes straight back from a direct search of the right collection.

Then the part that cost somebody an answer. The agent asked for code, received a full result set at the top of the score range, could not find the identifier in it, and concluded the code was not there. It said so to a product manager without a hedge, because from where it sat the search had worked.

What kept the bug alive was that every signal an operator actually looks at was healthy. Nothing threw. Latency was normal, the result count was right, and the passages were real code out of a real repository, so anyone skimming a trace saw output that looked reasonable. The only symptom was that answers were sometimes wrong, and a language model is always available to take the blame for that.

The fix, and why none of it is optional

Producing one ranked set instead of several took four changes. Drop any of them and the defect returns.

Rank once, over the union. Candidates are still gathered per collection, because the indexes are per collection, but no ranking decision happens at that level. The union is ranked in a single pass by a single scorer. Two scorers can disagree about scale while both are perfectly correct.

Retrieve the small collections through the same halves when they are part of a multi-collection query. A collection that would have taken the whole-corpus path goes through vector and lexical retrieval like everything else, so its chunks arrive carrying a real position instead of a placeholder. The whole-corpus read survives for the single-collection case, where it is still the better behavior. It was only ever safe when nothing was being ranked beside it.

Rerank the union in one call. One rerank puts every candidate on one scale by construction, and it re-scores the formerly unranked chunks from their own text instead of trusting the constant they came in with. A collection that produces no retrieval signal at all still contributes its chunks as candidates, so a small knowledge base cannot quietly vanish from a query it should have answered.

Fall back to rank position, never to score. When the reranker is unavailable, the per-collection lists are fused by position instead. The fused score for a chunk is the sum, across every list it appears in, of

1 / (k + rank)          with k = 60 and rank counted from 1

Nothing on the right-hand side is a score. A chunk that comes first in one list contributes 1/61, and it contributes the same 1/61 whether that list was ordered by a reranker, by cosine distance, or by term frequency. Position 1 means the same thing in every list, in a way that 0.83 does not. Fusing positions cannot be poisoned by a scale because it never reads one.

The constant damps the head of the list. Without it, first place is worth 1.0 and second place 0.5, so a single list can carry the whole result. At k = 60 the gap between rank 1 and rank 2 is about 1.6%, which tilts the fusion toward chunks that several lists agree on rather than a chunk that one list happened to put first.

Rank fusion is worse than a rerank. It never reads the question against the passage, so it cannot tell you that a top-ranked chunk is off-topic. What it is, is comparable, and comparable beats better when the alternative is a scale nobody can interpret.

Two things came out of running this rather than designing it.

The rerank is the expensive call in the chain, so the candidate set going into it is capped. Taking those candidates in list order turns out to be the wrong way to apply a cap: one large collection fills the budget and a small one never gets scored, which reproduces the original defect by a different route. Candidates are drawn round-robin across the per-collection lists instead, so every source reaches the reranker.

Gathering candidates serially cost one retrieval round trip per collection before the first rerank could start, and across 14 collections that dominated the latency of the whole call. Gathering is a set of independent reads, so it runs concurrently now, with input order preserved. Order matters because both the interleave and the fusion depend on it, and the same question has to return the same set twice.

One more thing in the fix's favor, which we did not plan. Ranking per collection meant one rerank call per collection. Ranking the union means one call, however many sources the question touches. The correct version is also the cheaper one.

The week we shipped the same fix twice

The knowledge search hit this bug first. A personal knowledge base is usually small, therefore inline, therefore stamped with 1s, and it swept every slot of a search that was also supposed to read a project collection. We fixed it on July 29 and moved on.

Code search had the identical defect and kept it until August 5. For that week, search_code was still returning one small repository for every query, and the fix was already written.

The reason is worth more than the embarrassment. We applied the first fix where the symptom appeared, which was the caller. Both callers merged results from the same layer underneath them, so the correct home for it was that layer, where "rank across N collections" is one operation with one answer. Once it moved down, code search inherited the fix and there was nothing left to remember.

If two callers both merge results from a layer below them, the merge belongs to that layer. We knew that. We wrote it in the wrong place anyway, because the bug arrived wearing one caller's name.

The change that mattered more than the ranking

What reaches the model now looks roughly like this, with the internals stripped out:

{
  "source": "<which collection this came from>",
  "path": "<file or document name>",
  "score": 0.71,
  "scoreKind": "rerank",
  "text": "…"
}

scoreKind takes three values, and the third one is the point. rerank means a model scored this passage against this question. rrf means the ordering is positional and no relevance judgment was made at all. inline means nothing was ranked, and the passage is here because it lived in a collection small enough to hand over whole.

The model is the last consumer in the chain and the only one that has to decide what a result is worth. Given a passage and a number, it cannot tell a confident hit from a positional artifact. Given a passage, a number, and the word inline, it can treat what it is holding as a document it was handed rather than a document the system chose.

The obvious alternative was to normalize the three units into one number so they would be comparable. That throws the distinction away instead of reporting it, which is the same mistake wearing better clothes. An unranked result is not a low-confidence result. It is a result about which no confidence statement exists, and to anything downstream that has to decide whether to trust it, those are very different objects.

The label also has to survive one more stage. On the internal profiles the answer is scanned for personal data before it is delivered, and so are the rows behind it, because the model is shown a short preview while the full set goes to a table, a CSV export, and a Slack message that never pass through the answer scan. That is separate work with its own failure modes and its own post. What matters here is that a result travels a long way after it is ranked, and everything downstream reads the number.

If I could keep one idea out of this post it would be this one, not the ranking fix. The ranking fix is something you can look up. Reporting the unit alongside the number is a habit, and it is the habit that would have made the original bug visible in a trace on day one.

How we know the ranking works

That bug ran in production for months, which is a testing failure at least as much as a ranking failure. So it is worth being exact about which assertion would have caught it.

A test that checks the search returned results would have passed the entire time. A corpus read in file order returns results for every query. So does a broken merge. Result count, latency, and the absence of an exception are all healthy during total retrieval failure, which makes all three useless as retrieval assertions.

A retrieval test has to name the document that must come back.

It helps to picture what a real one would have reported while the bug was live. Every query returned chunks from one small repository in file order, so for any case whose answer lives anywhere else, the correct file appears at no position at all. recall@10 collapses toward the share of cases that happen to be answered inside that one repository, and recall@1 goes down with it. Both numbers fall together, by a lot, on every case at once. There is no way to read that report as ordinary variation.

Building the goldset

Each case is a question paired with the one file that answers it.

{
  "query": "add a void state to the settlement status enum",
  "expect": "<the migration that adds it>"
}

The questions are phrased as intent rather than as identifiers, because that is how people ask. Writing the question as the identifier would test string matching and tell us nothing about whether someone can find a file without already knowing its name.

Every case has exactly one correct file, and that constraint is deliberate. With several acceptable answers per question, recall becomes ambiguous and the graded metrics start depending on a relevance rubric that nobody maintains. With one, recall@k and hit rate are the same number and the report can be read without interpretation. It costs some realism, since real questions sometimes have three good answers. It buys a number that means one thing, which at this stage is the better trade.

Cases come out of the repository rather than out of imagination. We took real files and real commits, wrote the question a person would have asked before that file existed, then checked two things for each case: that the target file is in the index, and that its chunks embedded successfully. A case whose target was never ingested measures the pipeline rather than the ranking, and it will drag a retrieval number down for a reason no retrieval change can fix. The runner reports how many cases were measurable separately from how many scored, so that failure stays visible instead of dissolving into an average.

We do not grade any of this with a model. A judge would let us score partial credit and questions with several good answers, and it would also introduce a second system whose failures look exactly like retrieval failures, inside a harness whose entire job is to be the thing you trust when the agent is wrong. A fixed target file compared by string equality has no failure mode of its own.

The set has to be big enough that one case moving does not look like a result. Ours holds 150 cases over one product repository, and that size was chosen rather than stumbled into. At 150, each case is worth 0.67 points, so a one-point move is one case and means nothing. Treating a recall figure as a proportion, the standard error near 0.6 at this sample size is about 4 points, so anything under roughly 8 points sits inside the noise and anything past 12 does not. The runs we compare are paired, since both pass over the same cases, which makes a like-for-like comparison tighter than that estimate suggests. We quote the independent-sample number anyway. It is conservative, and you can compute it in your head while reading a report, which is when you need it.

What each metric is actually telling you

We report recall@k, mean reciprocal rank, and normalized discounted cumulative gain, and they answer different questions.

recall@10 asks whether the system can find the file at all. It is a property of ingestion and retrieval together. When it is low, either the document is not indexed properly or neither retrieval half is reaching it. Reranking will not help, because a reranker can only reorder what it is given.

recall@1 asks whether the ranking is right, and in practice it is the number that matters most. The first result is the one the model reads most carefully and the one a citation points at.

Mean reciprocal rank averages 1 divided by the rank of the correct file. It notices the difference between position 2 and position 8, which recall@10 hides completely. When recall holds flat across a change and MRR moves, the change affected ordering rather than retrieval.

Normalized discounted cumulative gain discounts by position. With one relevant document per case it tracks MRR closely, so we read it as a cross-check rather than as independent evidence.

Reading recall@10 and recall@1 together is what makes the report actionable. High recall@10 with low recall@1 is a ranking problem. Low recall@10 makes recall@1 uninterpretable, because the ranker never saw the answer to begin with.

The run that decided something

The clearest thing this harness has produced is the case for the lexical half. Same 150 cases, same reranker, same corpus. The only variable was whether the full-text index took part.

The 20-point move in recall@10 is the one to look at. In case counts, 30 of the 150 target files were not in the vector candidate set at any depth, so the reranker never had the chance to promote them. The second half did not reorder those results. It made them exist.

The three recall numbers move by different amounts, and the gaps say where the help arrived. recall@1 gains 13 points, recall@3 gains 18, recall@10 gains 20, which in cases is 20, 27, and 30 of 150. So a good number of the newly found files land at position 2 or 3 rather than first: the token match puts them in front of the reranker, and the reranker still often prefers a semantically closer passage from somewhere else. That is useful when you are deciding what to work on next, because it says most of the remaining loss is a ranking problem rather than a retrieval one.

Why the lexical half finds them at all is in the questions. They name enum values, column names, and status codes, which are exact tokens. An embedding of an identifier lands near other identifiers, and near is not the same as the one you typed. A full-text index matches the token.

Neither half is a fallback for the other, so both stay on, and each degrades independently. With the lexical index unavailable the search runs vector-only rather than failing.

The second goldset, and what it does not prove

A second set re-words 20 of the cases and runs them against the same target files. Its job is narrow: it checks that retrieval is not keyed to the phrasing we happened to use the day we wrote the case.

It scores recall@1 of 0.80, recall@5 of 0.95, and MRR of 0.86. Those numbers are better than the main run and they are not evidence of anything improving. It is a 20-case subset, and a small subset of a corpus is easier than the corpus. We read it as a phrasing check that passed. Nothing more.

What the harness cannot see

It measures retrieval, not answers. A flawless result set can still be summarized wrong.

It measures one corpus. The numbers do not transfer to a different source type and we do not quote them as though they do.

Its cases are written by people who already know the corpus. So it undersamples the questions asked by people who do not, which are the questions the system exists to answer.

And it calls retrieval directly. Whether the model decides to search at all sits outside the loop entirely, and that turns out to be a failure class of its own.

Day to day we watch recall@1 and recall@10 on the main set, and a drop of more than a few points is something to explain before it is accepted. Both come out of one run, so a drop in one and not the other already names the layer to go and look at.

The model has to know what it can search

The third problem produces a wrong answer with a perfectly healthy retrieval stack underneath it. The search would have worked. It never ran.

Left to itself the model sees a tool description and nothing about what the tool can reach, so it has to guess whether searching is worth a turn. It guesses wrong in both directions. It searches for material we never indexed, which costs a turn. It skips a search that would have worked, which costs the answer.

So the system prompt carries a manifest of what this person, in this conversation, is allowed to search. Four rules keep it honest.

It is built from the same authorization path the tool uses. Not a copy of the rules. The same resolver, handing an already-authorized list to both readers. When the two drift, both directions fail silently: a manifest that advertises what the tool refuses makes the model retry a search that cannot succeed and promise a file it can never quote, and a manifest that omits what the tool allows makes it skip a search that would have worked.

It gives true counts. Names are capped and the remainder is stated. A truncated list that reads as complete is worse than no list, because it lets the model conclude a document is absent.

It says what is still processing. A document part-way through ingest is not searchable yet, and a model that does not know that will confidently deny it exists.

It says that a filename is not its contents. Without that line, the model answers from titles.

A longer tool description was the obvious alternative and it cannot carry this. A description is written once and shipped to everybody. The manifest is assembled per turn out of the documents this person can reach at this moment, which is the only form in which the information is true. A description listing the corpus would be wrong for every user it was not written for, and stale on the next upload.

A filename is not a subject either

The manifest still was not enough, and the case that showed it is worth laying out in full, because every layer underneath it was working.

A Slack channel had its own knowledge base holding a single reference document. The manifest named the file. Somebody asked how activation is defined, which is a question that document answers directly.

The model made no search call at all. It went to code search and the help center instead, and produced an invented activation checklist, complete with completion percentages that the document contradicts.

The control was one question away. Naming the document in the question made it search immediately and answer correctly out of the file. So the tool worked, the scope resolution worked, the retrieval worked. Only the decision to search was wrong.

That control did more work than the observation. A wrong answer is compatible with a dozen causes, most of them somewhere in the retrieval stack. Asking the same question with the source named, and getting a correct answer from that source in one turn, rules all of them out together. We run that pair now whenever an answer looks like a knowledge gap: the question as a user would ask it, then the question with the source named. If the second works and the first does not, the bug is in routing, and no amount of retrieval work is going to touch it.

The reason for the original failure is that a filename is a token, not a topic. A name assembled out of an internal acronym tells the model nothing about whether definitions live inside. It could have guessed, and a wrong guess costs a turn, so it did not.

Two changes fixed it, and both are needed.

The first is a one-line description on each knowledge base, naming the subjects it covers, rendered before the file list, so the manifest says what questions the thing can answer rather than what it is called:

Knowledge base available:
- This channel's reference "<name>": definitions, metrics and lifecycle stages
  used by the go-to-market team. 1 file: <filename>
Search this before general knowledge, the help center, or the codebase on those
subjects. A filename is not its contents.

The second is a routing directive. When a channel's own reference is in scope, it outranks general knowledge, the help center, and the codebase on the subjects it names, and it gets searched first.

Two reasonable instructions that cancelled each other

The directive did nothing at first, and that is the part of this worth generalizing.

The same prompt block ended with a line telling the model that if none of the listed sources plausibly covers the question, it should say so instead of searching. That line is a sensible guard. It stops the model burning turns rummaging through somebody's personal file folder for a question about the weather.

It also contradicts "search this first". Handed both instructions, the model followed the more cautious one, every single time. Dropping that line when a channel reference is present, and keeping it when the scope is only somebody's personal files, is what finally let the directive take effect.

Prompt instructions are not additive. Two rules that are each correct can cancel, and nothing in the output tells you so. The model does not report the conflict. It picks one and answers.

There is no linter for this. The two lines sat in the same prompt block about twenty lines apart, and each arrived in a change that was right on its own. The practice we settled on is narrow: when a prompt block gains an instruction about when to use a tool, read the whole block back and find the instruction that already governs the same decision.

Testing a decision, not a call

The goldset harness cannot see any of this, because it calls retrieval directly and the model's decision is nowhere in the path. Every unit test passed while this bug was live, and so did the end-to-end one, because both go straight at the search service.

So there is a second harness that runs whole turns. It builds the real prompt, applies the real authorization gate, calls the real model, and binds the conversation to a surface exactly the way production binds one. It reports which tools actually ran, read from the persisted record of the turn rather than from the response stream. The stream carries no tool-call event, so reading it will tell you "no tools" for a turn that called eight.

That harness is what found the problem. The failing assertion was not "retrieval returned the wrong thing". It was "the model never called retrieval".

The manifest is not free, either, and its cost shapes part of its design. It is assembled on every turn, for every conversation, before the model is called. Each collection in scope costs a count query and a bounded name lookup, and an ingested folder can hold thousands of rows, so the lookup is capped and the projection is narrow. The rendered line is then cached for a few minutes.

The cache key is where the interesting bug lived. It includes the document counts, so an upload or a finished ingest invalidates the line by changing the key, with no explicit invalidation for anyone to forget. It also has to include a hash of the description, because the description is part of what the line renders. Without that, an edited description never reaches the prompt, and an edit that appears to do nothing is the worst kind to debug, because the natural next move is to edit it again.

Failure there is non-fatal on purpose. Any error building the manifest logs a warning and yields no block. Prompt construction never breaks over it, and the model falls back to the behavior it had before the manifest existed, which is worse but not broken.

Where this differs from the usual hybrid-search write-up

None of the components here are novel. Chunk, contextualize, embed, index, retrieve with a vector half and a lexical half, fuse by rank, rerank, apply a floor. That recipe is well documented and we followed it.

All three failures in this post sit in the seams between those components rather than inside any of them. Every component did its own job correctly. Retrieval returned what the index held, the reranker ordered what it was given, and the permission gate admitted only what the asker could read.

Three seams account for it. Most write-ups assume a single index, so the question of what a score means when two indexes meet never comes up. Most treat a small corpus as a degenerate case of the same retrieval path rather than as a different path, so no unranked placeholder is ever produced to leak into a merge. And most stop at offline retrieval metrics, which measure the tool and say nothing about whether the agent in front of it chooses to call the tool.

If you are building one of these, the recipe is the easy part. The seams are where the time goes.

What to copy

Label every result with how it was scored. Not a confidence number. The provenance of the number. No consumer, human or model, can recover it any other way.

Never sort a union of scores from different producers. Convert to a unit they share, which in practice means rank position, or re-score everything in one pass with one scorer.

Put the merge in the layer that owns the data, not in the caller. We shipped the same fix twice a week apart because we did not.

Make retrieval tests name a document. A test that asserts a count, a latency, or the absence of an exception will pass during total retrieval failure.

Work out what one case is worth before you quote a percentage. And report how many cases were measurable separately from how many scored.

Test the model's decision, not only the tool call. Run whole turns through the real prompt and the real model, and read which tools ran from the persisted record rather than the response stream.

Keep the two-question diagnostic. Ask it the way a user would, then ask again with the source named. If the second works and the first does not, the bug is in routing.

What is still open

The goldset covers one corpus. Every number in this post describes retrieval over source code, and we have no idea what the same harness would report over meeting notes or product documentation. Building the second goldset is the next piece of work, and it is harder, because "the one file that answers this" is a much softer claim about a set of meeting notes than it is about a migration.

The turn-level harness runs by hand. It needs the real model, so it is slow and it costs money per run, which is why it is not in continuous integration. Until it is, a prompt change can silently cancel a routing rule and nothing will go red.

We have never tuned the fusion constant. 60 is the value everyone uses and the value we use, and "we have not found a reason to move it" is a weaker statement than it sounds, because we have not gone looking.

Nothing measures the join between retrieval and delivery. The goldset scores retrieval before the redaction stage runs, and the answer is checked after it. A passage that survives ranking and then loses the exact token the question was about falls between the two harnesses, and neither of them will say so.

Code chunking is still generic. We split on blank lines, which works without being good, and splitting on function and class boundaries would make excerpts a great deal more coherent. It is the one place where the generic pipeline is a poor fit for its source, and it has been on the list for a while.

We are hiring engineers in Dubai to work on the list above.