Hisham AlhussainSometime in late June, "context engineering" started showing up everywhere. Andrej Karpathy and Shopify's Tobi Lütke both pushed the term, and I'll admit my first reaction was an eye roll. Great, another rebrand.
But the more I thought about it, the more I came around. "Prompt engineering" always made it sound like the job was finding magic words. That was never really what the work was. The work is deciding what goes into the model's context window, and in what shape. Now it has a better name.
The context window is a budget
Every token you put in front of the model competes for its attention. System instructions, tool definitions, retrieved documents, conversation history, scratch notes from earlier steps. It all adds up fast, especially for agents that run for many turns.
I've started thinking of it like a budget. For every piece of context, I ask: does the model actually need this to do the next step well? If not, it goes.
Things that made a real difference for me
- Fewer, better tools. Trimming a tool list from 25 to 8 well-described tools improved tool selection more than any prompt tweak I tried.
- Summarize old turns. Long conversations drift. Compressing older history into a short running summary keeps the model focused on what matters now.
- Structured retrieval results. Instead of dumping raw chunks, I pass the source, the date, and the relevant snippet. The model cites better and hallucinates less.
- Write things down outside the window. For longer tasks, having the agent keep notes in a file or a scratchpad and re-read them beats keeping everything in context.
Bad context looks like a bad model
This is the part that gets people. When an agent goes off the rails, the first instinct is to blame the model or try a bigger one. Nine times out of ten, when I dig into the trace, the model was doing something reasonable with confusing or conflicting context. Two instructions that disagree. A stale document. A tool description that's ambiguous.
Kimi K2 and Grok 4 both came out this month, and I've been testing them too. They're impressive. But honestly, the gains I've gotten from cleaning up context have been bigger than the gains from swapping models.
Where I'd start
If you've got an agent or assistant in production, pull up a few full traces and just read exactly what the model saw. Not what you think it saw. What it actually saw. I promise you'll find something to cut.