How to Version an API Your Model-Serving Clients Depend On

A model-serving contract should pin down:

• The request schema: required fields, optional fields, and validation rules, independent of which model handles the request.
• The response schema: field names, types, and what happens when the model returns nothing usable.
• Error codes and what each one means for the caller: retry, don't retry, or fall back.
• The streaming protocol, if you support one, including how a client detects the end of a response.

It should not pin down which model version serves the request; that's an implementation detail the contract should let you change freely.

Treat the API contract's version as separate from the model's version. A model swap that doesn't change the request or response shape doesn't need a new API version at all; a change to required fields or error semantics does, every time.

When you do need a breaking change, run the old and new contract versions in parallel behind different routes, with an announced deprecation window for the old one. Give clients a way to test against the new version before it becomes mandatory, not just a changelog entry and a deadline.

The teams that get burned here usually skipped the parallel period because the change felt small; a small schema change can still break a client's parser.

Streaming responses are where model-serving contracts most often go undocumented. Clients need to know exactly how to detect the end of a stream, what a partial or malformed chunk looks like, and whether tool calls or structured output arrive as one block or get built up incrementally across chunks.

Document the exact termination signal, and test what happens when a client disconnects mid-stream: does the model server keep generating for a response nobody will read, wasting GPU time on an abandoned request. A contract that's silent on this usually means every client team reinvents its own, slightly different, parsing logic.

Say you want to add a confidence indicator to every response. Adding it as a new optional field that old clients simply ignore is safe. Making it required, or changing an existing field's meaning to accommodate it, is not, even if the change looks small in a pull request.

The safe pattern is additive by default: new fields are optional, existing fields never change type or meaning, and anything that has to break gets its own contract version with a deprecation window. Reserve required, breaking changes for cases where the additive version genuinely isn't possible.

Decide up front whether you're supporting a maintained SDK, raw HTTP calls, or both, and be explicit about which one gets contract guarantees. An SDK can absorb a lot of the versioning pain for you, translating a breaking API change into a non-breaking SDK update, but only if you actually maintain it as carefully as the API itself.

Teams that publish an SDK and then let it lag the API end up with the worst of both worlds: a contract that changed and a client library that quietly stopped reflecting it.

• Documenting the happy path only, leaving every error code as an exercise for the client team to discover in production.
• Letting SDK behavior drift from the documented contract because the SDK got a quick fix the docs never caught up to.
• Assuming internal teams don't need the same contract discipline as external ones; internal teams break just as often, and debugging it takes just as long.

Each of these is cheap to avoid up front and expensive to unwind once several clients depend on the undocumented behavior.