agent-search
Why I Built agent-search This Way
This is not meant to read like product documentation. It is a short explanation of the tradeoffs, motivations, and opinions behind the system: why it decomposes questions, why it keeps human review narrow, and why the runtime tries to stay configurable without turning into a framework-shaped blob.
The Core Problem
Most retrieval stacks work well right up to the point where a question stops being simple. As soon as a user asks for synthesis across multiple ideas, time periods, or entities, a single retrieval pass starts to feel fragile. You either miss important evidence, overfit to one phrasing of the query, or generate something that sounds coherent but is only weakly grounded.
I built agent-search around that failure mode. The system assumes that complex questions
deserve structure. Instead of trying to brute-force one retrieval and one answer, it breaks the problem
into smaller units, works those units independently, and only then pulls them back together into a final
response.
Why Decomposition Comes First
Decomposition is the center of gravity for the whole runtime. If the system can identify the real subquestions inside a user request, the rest of the pipeline becomes easier to reason about. Retrieval gets more targeted, citations become less noisy, and synthesis stops having to invent missing structure on the fly.
This is also why the one human-in-the-loop checkpoint lives here. If a person is going to intervene, the most valuable time to do it is before expensive retrieval and answer generation have already started. Editing subquestions is a high-leverage way to steer the run without turning the system into a fully manual workflow.
Why HITL Is Intentionally Narrow
A lot of systems become less usable when every stage is pausable. Technically that sounds flexible, but operationally it makes runs harder to understand and harder to resume. I wanted the runtime to preserve one clear checkpoint instead of exposing review prompts everywhere.
That is why hitl_subquestions is the only review stage in the SDK contract. It gives an
operator one meaningful place to inspect the shape of the work, while keeping the rest of the graph
deterministic enough to reason about.
Why Query Expansion And Reranking Are Toggles
Query expansion and reranking are both useful, but neither is universally correct. Some datasets benefit from broader retrieval vocabulary. Others get worse because expansion introduces noise. Some collections need reranking to surface the best evidence. Others are already clean enough that reranking adds cost without real upside.
That is why both are explicit runtime controls instead of hardcoded behavior. I wanted callers to be able to choose a more aggressive search strategy when recall matters, or a leaner one when latency and simplicity matter more.
Advanced RAG Techniques
The decomposition step is just an LLM transformation: give the model one large question and ask it to rewrite that question as a small set of answerable subquestions. There is no mystery math hiding there. It is sequence prediction constrained by instructions. In practice, the model learns that a broad prompt like “compare X and Y over time” is easier to answer if it emits pieces like “what happened to X,” “what happened to Y,” and “what changed between them.”
Query expansion uses the same basic idea on a smaller scale. Instead of trusting one literal wording, the model generates nearby phrasings and search variants for a subquestion. Logically, this improves the chance that at least one query lands near the right documents in embedding space or keyword space. If one query misses because the wording is too narrow, a neighboring query can still retrieve the evidence.
Reranking is simpler than it sounds: start with a candidate set of retrieved chunks, then score each chunk
against the subquestion and sort by that score. Conceptually it is just a function like
score(chunk, question), where higher means “more useful for answering this exact question.”
The top-ranked chunks are then passed forward. This is effective because retrieval is usually optimized for
broad recall, while reranking is optimized for local precision.
So the flow is: one big question becomes several smaller questions, each smaller question can generate a few search variants, retrieval brings back a noisy pool, reranking sorts that pool, and answer generation uses the best evidence. The benefit is not magic. It is just breaking one hard problem into smaller scoring and selection steps that are easier to get right.
Why The SDK Stays Narrow
The supported sync entrypoint is still advanced_rag(...) because I do not want consumers
rebuilding the orchestration surface area in slightly different ways. The value of the SDK is not just
that it wraps the runtime. It is that it gives one opinionated interface for model + vector store usage
and keeps the public contract from drifting too fast.
The config surface is still flexible. Callers can toggle rerank and query expansion, opt into HITL,
inject callbacks, resume paused runs, and override prompts through runtime_config and
custom_prompts. But the top-level shape remains intentionally small.
Why Prompt Overrides Are Limited
I wanted prompt customization without letting callers accidentally break the execution model. That is why prompt overrides replace only the instruction layer and not the runtime-supplied payload. The system still appends the live question, subquestion, and evidence sections itself.
In practice this means teams can change tone, emphasis, or answer style without severing the connection between the generated text and the evidence that produced it.
Why Checkpointing Is Optional
Non-HITL runs should not pay a complexity tax for resumability they do not use. That is why checkpoint
storage is optional overall and only becomes important for pause-and-resume flows. If you need it, you can
hand the SDK a checkpoint_db_url and let it manage the saver, or inject your own
checkpointer if your application already owns that lifecycle.
What I Wanted The Experience To Feel Like
The system is meant to feel inspectable instead of magical. You should be able to explain what it is doing, why it paused, which features are enabled, and how it arrived at the final answer. That is more important to me than hiding the workflow behind a single black-box prompt.
In other words, agent-search is trying to be opinionated where structure helps and flexible
where retrieval reality varies. That balance is the whole product.