Field note

When the Model Eats the Harness

This is a reading note on Thariq Shihipar's article, The new rules of context engineering for Claude 5 generation models.

Anthropic removed over 80% of Claude Code's system prompt for its newest models without a measurable loss on coding evaluations. Rules that once kept older models out of trouble had become constraints that newer models had to reason around. The new advice is looser: let the model use judgment, make tool interfaces expressive enough that they explain themselves, and load detailed guidance only when the work calls for it.

The obvious lesson is to simplify prompts. I think there is a stranger lesson for skills.

A skill library should sometimes shrink. Once a capability becomes common enough, the model may already know the routine. A long skill can collapse into a short phrase that wakes up behavior learned during training. At that point, the old instructions are scaffolding the model has to climb through before doing something it already knows how to do.

This is good for users. Nobody wants to remember a wall of prompt text just to trigger a familiar workflow. It also changes the competitive value of the harness.

A strong harness makes models easier to swap. If the shell handles the planning, tools, memory, and verification, the model underneath can be treated more like a component. A thinner harness moves more of the product's behavior into the model's weights. Competitors can inspect the shell, but they cannot see which habits the model learned or which plain words reliably bring those habits out.

That makes benchmarking harder too. An open-weight model can be tested inside a popular harness today, then look worse tomorrow because the harness was simplified around abilities that another model has absorbed. Part of the benchmark has moved behind the API, where competitors cannot inspect it.

From a model vendor's point of view, that is an attractive result. Users get shorter prompts and fewer conflicting rules. The vendor gets a product whose best behavior is harder to reproduce by swapping in another model. I do not know whether this is the strategy behind Anthropic's changes, but it is a useful side effect.

The awkward part is compatibility. Instructions that get in Claude 5's way may still be doing real work for an older Sonnet model. A short trigger that frees the new model could leave the older one guessing. Pruning skills therefore needs the same discipline as adding them: delete something, run the evaluations again, and keep a model-specific layer when the results still depend on it.

OpenAI has taken a somewhat different route. The open-source Codex CLI supports custom model providers, so the harness still has value when the model changes. It works more like an agent operating layer than a shell for one specific model. If native model judgment keeps improving, I still expect parts of that layer to get simpler.

My rule for skills is becoming clearer. Keep the ones that carry taste, private knowledge, or an important safety boundary. Cut the ones that merely explain a popular capability the model has already absorbed. The model may eat a popular harness the same way it eats a popular skill: by absorbing its familiar patterns until a much simpler trigger is enough.

To comment and continue discussing this article, please comment on the X Article.