The tool was deployed. The model said it didn't exist.

I shipped two new tools to a chat agent this morning, watched the pipeline go green, then asked the agent to use one of them. It said:

I don't have a tool called printer_readiness. I won't make up output for tools I can't call.

Which was annoying, because I'd just deployed it, and irritatingly correct, because from where the model was sitting the tool genuinely did not exist.

Two registries is the obvious number

Most agent codebases end up with a pair of structures:

TOOL_DEFINITIONS = [
    {"name": "printer_readiness", "description": "...", "input_schema": {...}},
    ...
]

TOOL_HANDLERS = {
    "printer_readiness": printer_readiness,
    ...
}

One is what the model sees. One is what dispatch calls. Add a tool, add it to both, done. That's the mental model, and it's right up until the tool list gets big enough that you stop sending all of it.

The third one nobody remembers

Once you're past a couple hundred tools, handing the model every definition on every turn is expensive and makes it worse at choosing. So you classify the incoming message first and send a subset:

TOOL_CATEGORIES = {
    "tickets":  ["search_tickets", "get_ticket", ...],
    "devices":  ["search_devices", "reboot_device", ...],
    "identity": ["find_user", "reset_password", ...],
}

def tools_for(categories):
    names = set(ALWAYS_INCLUDE)
    for c in categories:
        names.update(TOOL_CATEGORIES.get(c, []))
    return [t for t in TOOL_DEFINITIONS if t["name"] in names]

I'd added the new tools to one category — the one I happened to be looking at. My question about them classified into a different category. The router built a tool list that didn't include them, the model got that list, and it answered honestly about the list it was given.

Nothing errored. Nothing logged. The deploy was green, the tests passed, the handler was importable, and the feature was unreachable. The only symptom was a polite refusal, which is exactly the symptom you get when you ask an agent for something it genuinely can't do — so it reads as normal behaviour instead of a bug.

The check that should have existed

The invariant is easy to state: every tool offered to the model must be dispatchable and selectable. Three sets, two subset relations.

@pytest.mark.parametrize("tool", sorted(DEFINED))
def test_every_defined_tool_is_reachable_from_some_category(tool):
    assert tool in REACHABLE, (
        f"{tool} is defined but listed in no category, so the router "
        f"can never hand it to the model")

Parameterised per tool, on purpose. A single test asserting DEFINED <= REACHABLE tells you the set is wrong; one test per tool tells you which tool you broke, in the failure name, without reading a diff.

When I ran it, six other tools were already in that state. Including the onboarding orchestrator every onboarding path in the system is supposed to funnel through. Nobody had noticed, because the failure mode of an unreachable tool is an agent that quietly does the job some other way.

One wrinkle: the module wouldn't import

The file this lives in can't be imported outside its runtime — cloud-only imports at module scope, and it uses f-strings with backslashes that need a newer Python than the box I test on. Easy trap here is to decide the invariant is untestable and move on.

It isn't. The registries are literals. You can read the source as text:

TEXT = Path("lambda_function.py").read_text()

def block(header, open_ch, close_ch):
    """Slice a top-level literal by balancing brackets from its header."""
    i = TEXT.index(open_ch, TEXT.index(header))
    depth = 0
    for j in range(i, len(TEXT)):
        depth += (TEXT[j] == open_ch) - (TEXT[j] == close_ch)
        if depth == 0:
            return TEXT[i:j + 1]
    raise AssertionError(f"unterminated block for {header}")

DEFINED   = set(re.findall(r'"name":\s*"([a-z0-9_]+)"', block("\nTOOL_DEFINITIONS = [", "[", "]")))
REACHABLE = set(re.findall(r'"([a-z0-9_]+)"', block("\nTOOL_CATEGORIES = {", "{", "}")))

Yes, it's regex over source. It is also the difference between an invariant that's enforced and an invariant that's a paragraph in a README. Balance the brackets rather than matching a closing line, and it survives reformatting fine.

The part worth generalising

Routing to a subset is a good idea — smaller tool lists really do improve tool choice. But it converts "did I register this?" from a question with one answer into a question with three, and the third one fails silently and politely.

If your agent narrows its own tool list, write the test that says every tool can still be chosen. And while you're there, tell the classifier the new capability exists in words too, because a router can't pick a category it was never told about.