Better Models: Worse Tools
- ID
- 2631
- Status
- summarized
- Published
- 05 Jul 2026, 6:53 AM
- Fetched
- 05 Jul 2026, 7:33 AM
- Provider
- Simon Willison
- Category
- developer-ai
- Original URL
- https://simonwillison.net/2026/Jul/4/better-models-worse-tools/
- Source URL
- https://simonwillison.net/atom/everything/
Summary
- Score
- 8.0
- Created
- 05 Jul 2026, 7:34 AM
- Tags
- Audience
- developersvibe_codersai_agent_usersai_ml_learners
What happened
Armin reports that newer Anthropic models like Opus 4.8 and Sonnet 5 increasingly invent extra fields when calling custom edit tools, causing schema rejections—while older models did not. The likely cause is that these models have been RL-trained specifically for Claude Code's built-in search-and-replace edit tool, degrading their ability to use third-party harness tools correctly. This raises the question of whether coding harnesses should implement multiple edit tools to match each model's preferred mechanism.
Why it matters
For anyone building or using AI coding agents, this is a concrete reminder that SOTA models can regress on tool-use fidelity as vendors optimize them for their own first-party harnesses. If you're wiring custom tools in Malaysia or elsewhere, expect more schema validation failures with newer models and consider matching your tool design to the model's native format rather than assuming improvements are universal.
Discussion angle
Should your agent harness implement vendor-specific edit tools (Claude's search-replace vs OpenAI's apply_patch) to reduce tool-call failures, or is it better to build a single robust schema and rely on retries?