The Model Is Not the Development Environment
I spent 12 million input tokens on a small monorepo change. The problem was not the model. It was the development environment around it.
A few weeks ago I added a boring feature to an unboring codebase: a monorepo containing a Next.js frontend, a Laravel backend, a couple of supporting microservices, shared TypeScript packages, some Python services, migrations, infrastructure definitions, and enough historical baggage that nobody could explain the full request lifecycle without opening the code.
The feature: users could update most of their profile, but changing their email required an administrator. I wanted users to update their own email after verifying the new address. The final implementation was unremarkable — a new endpoint, validation, a verification flow, two frontend changes, an event for downstream consumers, a handful of tests.
The coding agent produced around 100,000 output tokens for the session. It consumed roughly 12 million input tokens.
Before you object: yes, I know most of that 12M was prompt-cache hits. Cumulative input in an agent loop mostly re-reads the same context, and the billed cost is a fraction of the naive number. The dollar figure is not the problem. The problem is what that ratio says about the working set. A context stuffed with several candidate implementations, most of which turned out to be irrelevant, degrades attention on the one that matters. It forces compaction earlier, and compaction is where agents forget things they knew an hour ago. It adds latency to every turn. The 120:1 ratio is a symptom of an agent doing archaeology when it should be doing surgery.
I read the transcript to find the catastrophic mistake. There wasn’t one. The agent searched for the profile implementation, got several candidates, opened most of them. Searched for email validation, which pulled in auth code, admin workflows, invitation flows, forgotten-password handling. Noticed that user changes emit events, so it investigated consumers in two microservices. Discovered the shared frontend profile package and read that too. When a test failed, the runner printed two thousand lines and the agent consumed all of them. It re-searched for a symbol it had found twenty minutes earlier and reopened files it had already read.
Every individual action was locally reasonable. Collectively they were an absurd development process. The agent was using the context window as a filesystem, search index, debugger, notebook and long-term memory simultaneously — and once I saw it that way, a bigger context window stopped looking like the fix. The environment was the fix.
I didn’t set out to build another coding-agent setup. I wanted something small enough that I could control how context entered the model — intercept tool calls, change search behaviour, decide how repository context gets retrieved, and eventually experiment with compaction itself. Claude Code and Codex are excellent, and both are extensible at the edges, but the core retrieval loop is not mine to modify.
That led me to Pi, Mario Zechner’s deliberately minimal coding agent. Pi doesn’t hide the agent behind a framework; its extension model let me change the parts I cared about without rebuilding the runtime. The plan: use Pi as the base, add the missing infrastructure around it.
The first addition was ripgrep, plus fd for filename discovery. Agents are wasteful without a good lexical primitive. If I already know I’m looking for EmailVerificationRequested, an exact search is cheap, deterministic, and all that’s required — no semantic retrieval, no opening ten files.
The second was LSP. A large share of agent exploration consists of questions a language server already answers: where is this symbol defined, who calls this method, what implements this interface, which references may be affected if I change this function. These are not language-model questions. Instead of grepping for updateEmail and inferring relationships from twenty textual matches, the agent asks the language server for definitions, references and implementations directly.
The third was Zoekt, which builds a trigram index over source code and makes repeated searches across large corpora extremely fast. Ripgrep is superb but it still walks files, which you notice when repositories get large, when several must be searched together, or when the same corpus is searched all day. Zoekt gave me a retrieval layer between “grep the filesystem” and “ask a model to understand everything.”
Assembling these clarified something: “search the repository” is not one operation. Where is PAYMENT_FAILED used? wants ripgrep. Who calls UserService.updateEmail()? wants LSP. Where in this enormous repo is email synchronization handled? wants indexed search. What parts of the system assume an email never changes? may eventually want semantic retrieval. Collapsing these into one generic tool forces the model to solve a routing problem before it can solve the programming problem.
Everything above concerns getting useful information into context, but the other direction was just as wasteful. Run npm test in this monorepo and the process may print two thousand lines. Three tests failed; the agent needs the failing tests, the assertion messages, and the useful part of the stack traces. The same applies to git diff, Docker logs, compilers repeating one error twenty times through different dependency paths, linters, package managers, Kubernetes tooling. All of it is designed for humans at terminals, not for models paying attention to every line they consume.
So I added RTK — not mine; it’s an open-source Rust CLI proxy that sits in front of common development commands and reduces their output to what the agent actually needs before it enters context. The distinction turned out to be a useful way to think about the whole harness: search tools answer what should the agent retrieve; RTK answers once the agent executes something, what part of the result deserves to come back. Two directions of context flow, and I wanted control over both.
Once you start improving an agent environment, there is always another piece of infrastructure that looks obvious in hindsight. I wanted persistent repository instructions, because every new session was rediscovering basic invariants. One of my applications has rules like:
Users are never permanently deleted.Project heads may override team leads.Generated API clients must not be modified manually.Database changes require backwards-compatible migrations.Business logic does not belong in controllers.These are enormously valuable and completely task-independent. The agent should not spend twenty minutes inferring them from code, so I structured projects around AGENTS.md files, with more specific instructions deeper in the tree where needed.
Then I wanted reusable skills — workflow descriptions rather than domain dumps, because debugging production follows a different shape from implementing a feature, and exploring an unfamiliar repo should encourage broad search then narrow reading, while implementing an understood feature should discourage exploration entirely. Then subagents. Then better editing tools. Then browser access, Python, model routing, session handoff.
At some point I looked at the pile and realised I was building Oh My Pi — a fork of Pi that already ships LSP support, subagents, browser capabilities, Python integration and richer editing, while keeping the extensibility I liked.
There is a particular satisfaction in discovering someone has built the thing you’re halfway through building, and a particular annoyance. Satisfaction won. I switched the base of the harness to OMP, deleted a surprising amount of my work, and kept only what was specific to my workflow.
The deletion clarified the project. I did not want to build a coding agent. I wanted to build infrastructure around coding agents. OMP owns the runtime; I own context, reproducibility and repository intelligence.
OMP removed a lot of engineering, but it didn’t know my retrieval preferences, didn’t reduce every noisy command, didn’t know which architectural knowledge should persist — and, most importantly, it didn’t exist on every machine I work on. I move between a primary Linux workstation, remote dev machines and occasionally disposable systems, and I did not want a handcrafted snowflake environment that works perfectly until I format the machine and can no longer remember why.
So the setup became a repository:
coding-harness/├── install.sh├── config/│ ├── omp/│ ├── rtk/│ └── models/├── skills/│ ├── debugging/│ ├── code-review/│ ├── feature-development/│ └── repository-exploration/├── templates/│ └── AGENTS.md└── bin/ ├── dev ├── index-repo └── refresh-indexThe layout is unimportant. The decision is that the machine is no longer the source of truth — the repository is. install.sh installs OMP, RTK, ripgrep, fd, Zoekt and my language servers, then symlinks configuration and skills. A fresh system is git clone, ./install.sh, done.
The harness also becomes versioned: a changed index scheme is a commit, an updated debugging skill is a commit, a removed useless tool is a commit. The environment becomes software I can reason about rather than folklore.
A sophisticated coding setup that only works on one laptop is not infrastructure. It is a pet.
Machine setup solved, project setup wasn’t. I was still entering repositories and manually checking whether indexes existed, language servers were available, instructions were present. So: one deliberately boring command.
devThe launcher detects the Git repository, identifies languages, ensures the corresponding language servers, checks the Zoekt index, loads project instructions and shared skills, enables the RTK wrappers, and starts OMP. Repository state can be kept under something like ~/.cache/dev-agent/, with enough information to decide whether the current index is still useful or should be refreshed:
$ cd platform$ dev
Repository: platformLanguages: TypeScript, PHP, Python
✓ OMP✓ RTK✓ ripgrep✓ PHP language server✓ TypeScript language server✓ Python language server✓ Zoekt index ready✓ AGENTS.md loaded✓ shared skills available
>dev is not intelligent and must not become intelligent. Its job is to make intelligence available — predictable bootstrapping, then get out of the way. I do not want a giant orchestration framework that breaks every time one tool changes.
The interesting shift is that I no longer think of OMP as the development environment. OMP is one component inside the environment. That distinction is the most important thing the whole exercise taught me.
The improvement did not come from one clever tool. It came from reducing the number of situations where the model had to solve problems ordinary developer infrastructure already solves. If I know a string, grep should find it. If I know a symbol, LSP should navigate it. If the repo is large, an index should search it. If a command produces noise, a wrapper should reduce it. If a project has invariants, they should persist outside the conversation. If I use the same workflow on five machines, installation should be automated.
None of this sounds futuristic — which is exactly why it works. We spent decades building tooling so human developers didn’t reconstruct their environment from first principles every morning. Coding agents deserve the same treatment.
Three mistakes worth naming.
Reaching for semantic retrieval first. It is tempting to embed the repository, add a vector database and a reranker, and call it a context engine. A surprising amount of development navigates perfectly well on ripgrep, LSP and Zoekt — fast, deterministic, cheap. Semantic retrieval should answer questions lexical and structural search cannot, not replace them.
Making Pi the product. Pi was an excellent foundation for experimentation, but I was recreating features OMP already shipped. Continuing would have been engineering for its own sake.
Believing output compression alone would fix tokens. RTK helps a lot, but compressing a two-thousand-line test log after running an unnecessary suite is still worse than running the correct targeted test. The hierarchy matters: avoid unnecessary work, then retrieve narrowly, then filter mechanical output, then compact when necessary. Reversing that order treats symptoms.
An honest caveat: I have not yet instrumented a before/after on a comparable task, so I cannot tell you the 12M became 2M. Qualitatively, sessions are shorter, compaction fires later, and the agent appears to re-read the same material much less often — but “qualitatively” is exactly the kind of answer this post argues against. So before I build much of the next phase, I want to measure what the current harness is actually doing. Per task, I want searches performed, files opened, files reopened, source tokens returned, tool-output tokens returned, compactions triggered and wall-clock time. When I have those numbers for a task comparable to the email change, I will publish them, whichever way they point.
The agent has better tools. It still decides how to use them.
Ask what code runs after a user changes their email? and the model might use LSP references, query Zoekt, or run rg email and start opening files. The harness provides good primitives but assumes the model consistently picks the right one. Modern models are much better at tool selection than earlier ones; I am still not convinced repository discovery should remain entirely their responsibility.
The next step, if I build it, is not another search engine. It is a layer between the model and the existing search tools.
I would start with a small semantic interface such as:
code_findcode_searchcode_symbolcode_referencescode_contextThe model asks for the information it wants; the context layer decides which mechanism is appropriate.
This is not an escalation ladder where every query begins with fd, proceeds through rg, then LSP, then Zoekt, and finally semantic search. These tools answer different questions.
A path or filename query belongs to fd. A literal belongs to ripgrep. A known symbol belongs to LSP. A broad search over a large corpus belongs to Zoekt. A conceptual question with no useful lexical anchor may belong to semantic retrieval.
Conceptually:
query │ ┌──────────────┼──────────────┐ │ │ │ filename/path literal symbol │ │ │ ▼ ▼ ▼ fd rg LSP
broad corpus? │ ▼ Zoekt
conceptual intent? │ ▼ semantic retrievalEscalation happens only when the selected mechanism does not provide sufficient evidence.
A fully qualified symbol may go straight to LSP. A quoted string may go straight to ripgrep. A question such as what parts of the system assume the user’s email is immutable? may need semantic retrieval much earlier because the implementation might never contain those exact words.
The router itself does not necessarily need another large model. A substantial amount of this classification is deterministic from the query shape and repository metadata.
Once retrieval moves into its own layer, a dependency graph becomes useful as a prior.
For a monorepo, I can derive an approximate graph from imports, package manifests, service definitions and LSP references:
frontend/profile │ ▼packages/user-client │ ▼services/users │ ├──► events/user-updated │ ▼services/notificationsIf a change begins inside services/users, related packages should be searched before unrelated ones. But the graph should be a search prior, not a search boundary. Event buses, queues, HTTP calls, shared databases, configuration-driven integrations and runtime discovery can create relationships that imports do not reveal. “Search here first” is useful. “Never search outside this graph” would be dangerous.
The more interesting change is what the retrieval layer returns.
Most coding tools still expose something roughly equivalent to:
read this fileI would rather expose:
give me the evidence relevant to this questionFor example:
code_context UserService.updateEmailcould return:
Definition----------app/Services/UserService.php:141-198
References----------app/Http/Controllers/ProfileController.php:82app/Listeners/SyncUserEmail.php:34tests/Feature/ProfileUpdateTest.php:117
Calls-----EmailVerificationService::verify()UserUpdated::dispatch()
Relevant types--------------UpdateEmailRequestUserUpdatedEmailChangedPayloadThe agent then requests exactly the sections it wants.
The pattern changes from:
readunderstanddiscardto:
discoverselectreadThat is how good developers navigate unfamiliar systems. When someone tells me UserUpdated is involved, I do not open every file containing the word User. I trace the relevant path.
A context router gives me something I currently do not have: one place through which repository evidence flows.
That makes context observable.
Instead of discovering after the session that twelve million tokens went somewhere, I can see:
Repository exploration
Search operations: 8Symbols inspected: 11Files opened: 6Files reopened: 2Source lines returned: 940Source tokens: 18,420Tool-output tokens: 3,810Now better questions become possible.
Why did this feature need twenty-eight searches? Why was the same file reopened four times? Why did debugging one test consume more repository context than implementing the feature?
Once those numbers exist, I can experiment with soft exploration budgets. Not hard limits; a forcing function.
Perhaps initial discovery gets:
10 searches5 file reads20,000 source tokensIf the agent needs more, it can continue. But it should first state what remains unknown and why more exploration is justified.
Instead of silently performing another fifteen searches, the agent might say:
I still need to determine which service consumes UserUpdated.The current evidence is insufficient, so I am extendingrepository exploration to inspect event consumers.The point is not to ration tokens for their own sake. The point is to make exploration an explicit resource-consuming phase rather than invisible background behaviour.
Centralizing retrieval also creates another possibility: memory.
If the agent asks for the same symbol three times, the harness should know that it has already investigated it.
The cache key cannot simply be:
commit + querybecause during active development HEAD often remains unchanged while the worktree changes underneath it.
A more realistic identity is closer to:
repository identity+ HEAD+ relevant worktree state+ query+ retrieval-tool/config versionThat does not mean hashing the entire repository before every lookup. The harness can invalidate cached evidence for files it observes being modified, or maintain lightweight fingerprints for touched files.
The useful distinction is between information that has been seen and information that is currently resident in context.
Suppose the agent inspected InvoiceService::recalculate() an hour ago and compaction has since removed those lines from the active context. If the file has not changed, the system should not need to rediscover where the method lives and who calls it. It can rehydrate the relevant evidence directly.
At that point the context layer is becoming less like a search API and more like a memory manager.
There is another conclusion hiding inside all of this: reducing context after work has happened is weaker than avoiding unnecessary work in the first place.
RTK can compress a giant test log. It is still better not to run the giant test suite unnecessarily.
If the harness knows which files changed, which package owns them, which symbols are involved and which tests reference those paths, it can infer a narrow verification set before escalating to broader suites.
For example:
changed:services/users/app/Services/UserService.php
affected area:users-service / profile update flow
likely targeted verification:ProfileUpdateTestemail verification testsevent consumer testsThe mapping will not always be exact. Package ownership alone is not enough; a useful implementation could combine imports, references, test locations, previous CI history and eventually coverage data.
The goal is not to magically know the perfect test. The goal is to prefer a targeted test before running everything in the monorepo, and widen the verification scope only when necessary.
The cheapest token is still the token you never generated.
The same principle eventually applies to the conversation itself.
A long debugging session accumulates an initial hypothesis, failed implementation, test failure, second hypothesis, temporary workaround, more logs and finally the actual root cause. Traditional compaction tries to summarize the entire archaeological record.
I increasingly prefer a different operation: handoff.
At the end of a meaningful phase, produce a structured state:
Goal
Current understanding
Architecture involved
Decisions made
Files changed
Verification completed
Known risks
Remaining workThen start a fresh session.
This is how engineers hand work between shifts. The next engineer does not need every false start; they need the state required to continue.
Compaction asks:
How can I shrink the conversation?
Handoff asks:
What should the next engineer know?
The second question produces better context.
Don’t copy my setup; the tools matter less than the architecture.
Start with an agent you can extend — OMP, Pi, or whatever already gives you the hooks you need. Solve the cheap problems first: a good lexical search tool, LSP, indexed search if your repositories justify it, filtered output for noisy commands, an AGENTS.md containing actual invariants rather than a novel, and a few workflow skills that change behaviour.
Then make it reproducible.
Put the environment in Git. Create an installer. Make machines disposable. Wrap project preparation and agent startup behind one command. Mine is dev; yours can be anything.
Only then start building the clever layer.
You need to observe where your agents waste context before deciding what your router should optimise. Maybe it is repeated source reads. Maybe it is test logs. Maybe you span fifty repositories and cross-repo search dominates. Maybe you work in one compact Rust codebase where LSP already solves almost everything.
Perhaps you do not need Zoekt. Perhaps you need Sourcegraph because the code lives across hundreds of repositories. Perhaps semantic retrieval solves a real problem in your organisation because the architecture is distributed across source code, documentation and runbooks. Perhaps RTK removes ninety percent of your biggest pain and there is no reason to build a context router at all.
That is why I would treat my stack as one implementation of the idea, not the recipe.
Context engineering should be personalised because development workflows are.
The email-change task worked, and twelve million input tokens taught me more than an efficient session would have. The model was capable of implementing the feature very early in the process. What it lacked was an environment that helped it reach the right evidence efficiently.
Much of the discussion around coding agents focuses on the wrong abstraction — whether GPT beats Claude, whether a model holds a million tokens, which agent tops a benchmark. Those things matter. But human software engineering did not become productive because programmers got dramatically smarter every year. It became productive because we built compilers, IDEs, debuggers, language servers, search engines, version control, package managers, CI and reproducible environments around them.
The model is the intelligence. It should not also be the search engine, symbol index, log parser, dependency graph, project memory and environment bootstrapper. My first version extended Pi until I realised I was rebuilding OMP. The second used OMP and moved the remaining infrastructure around it. The third, if I build it, moves one more responsibility away from the model: deciding what information deserves its attention.
Not context windows large enough to hold the entire repository. Infrastructure good enough that the model rarely needs to.
Comments
Loading comments…