Your Tools Pass Every Test. That Doesn’t Mean an Agent Can Use Them.
Every layer of the normal testing pyramid checks one tool in isolation. Tool A passes. Tool B passes. The schema doesn’t drift. Each tool is correct on its own.
But an agent doesn’t use one tool. It reads tool A’s output to decide how to call tool B. And that is exactly where the bugs I couldn’t see were hiding: not inside any tool, but in the disagreements between them.
Two of the 17 make the point.
The discovery tool that lied. My server had a tool that listed a project’s custom fields. For two fields, it reported is_required: false. So I tried to create an issue without them, and the create call rejected it: “Department cannot be blank.” The discovery tool said optional. The create tool said required. Both were behaving correctly in isolation, which is why both had passing unit tests forever. The bug was that they disagreed about reality, and no test that targets a single tool can see a disagreement between two.
The root cause was real and subtle: the “required” flag the discovery tool reported reflects only the field-definition setting. Workflow rules and role-based permissions can also make a field required at runtime, and the bare API never surfaces that. I only noticed because I watched the agent waste a turn re-checking the discovery tool, find no contradiction, and retry the same broken call.
The hint that promised a feature that didn’t exist. This is the one I’m most embarrassed about. An earlier fix added a helpful recovery hint to a validation error. The hint told the agent it could pass field values by name. Name-based lookup was real, but only implemented on the update tool. The hint had been copy-pasted into the create tool’s error path, where that feature did not exist. So the agent read “lookup is supported,” and burned turns trying call shapes that could never work.
A unit test on that hint string passes cleanly. The string is well-formed, contains the right words, throws no errors. The bug only exists if you actually follow the hint and discover the path it describes is a dead end. A hint that promises a feature it can’t deliver is worse than no hint at all.
