toolcontract

Contract testing for LLM agent tool-calls — built, published to PyPI, and maintained solo.

When a provider ships a new model version, an agent's tool-calling can silently change: an argument invented or dropped, a different tool chosen for the same input, a value that no longer matches the schema downstream code depends on. Nothing in a normal CI pipeline asserts on which tools were called with what, so the drift reaches production before anyone notices. toolcontract pins a golden set of expected tool-call trajectories, replays them against a live model, and returns a pass/fail/inconclusive verdict with a diff — Pact for microservice contracts, or Percy for visual regressions, but for tool calls. Ships with a comparator engine, a verification ledger, OpenAI/Anthropic/LiteLLM adapters, a CLI, a pytest plugin, and an LLM-judge tier for semantic matching.