Designing an API Contract for Your Retrieval Service

A retrieved result needs more than the chunk text: a stable document ID, a source reference the caller can show a user, a similarity or relevance score, and whatever metadata downstream code actually filters or displays on. Nail this shape down early, since callers will start depending on every field you return, including ones you didn't mean to commit to. If you're not sure a field is stable, don't return it yet, or mark it explicitly as unstable in your documentation.

Keep the score in the response even if no caller uses it today. It's cheap to include and expensive to add later once clients assume its absence means something.

Be explicit about units and ranges too. A cosine similarity score and a raw distance metric can both look like an unlabeled number between zero and one, and a caller who assumes the wrong one will silently sort results backward.

Internal changes, a new embedding model, a different reranker, a new chunking strategy, shouldn't force every caller to change their code. Put a version in the path or a header, and keep the response shape stable within a version even as the retrieval logic behind it evolves. When you do need a breaking change, ship the new version alongside the old one for a deprecation window rather than cutting over in place.

Document what "breaking" means for your service specifically: removing a field is breaking, adding one usually isn't, and changing what a score number means without changing its name absolutely is, even though nothing in the schema looks different.

A generic 500 tells a caller nothing useful. Distinguish a genuinely empty result set (200, with an empty array) from the vector store being unreachable (503) from an embedding provider timing out upstream (504, or a custom code your callers can check for and retry on). Callers building retry logic need to know which failures are safe to retry immediately and which mean back off.

Include a machine-readable error code alongside any human-readable message, since the message text will change over time and code that parses it for logic will quietly break the next time someone rewords an error string.

Any client that calls an ingestion endpoint will eventually retry a request that actually succeeded, after a timeout or a dropped connection on their end. If ingesting the same document twice creates two vectors instead of updating one, you'll accumulate duplicates that degrade retrieval quality over time in a way that's hard to trace back to its cause. Use a stable external ID or a content hash as the dedupe key, and make a repeated ingestion call with the same key an update, not an insert.

Without an explicit limit, a caller can request a much larger top-k than your system was sized for, either by mistake or through a bug in their own retry logic. Set a hard maximum, document it clearly, and return a real error rather than silently truncating the request, since a silent truncation looks like a bug in the caller's own code and takes far longer to diagnose.

It's tempting to skip contract discipline for a service that's only called by teams inside your own company, on the assumption that a Slack message can fix any confusion. In practice, an internal caller is harder to track down than an external one, since there's no formal registration process and no obvious owner list. Publish the schema, the error codes, and the deprecation policy the same way you would for a public API, even if the only consumers today sit two desks away. It saves the exact conversation you'd otherwise have to have later, after someone's dashboard silently breaks.

Decide these points before a second team depends on the service:

• Return a stable document ID, a source reference, a relevance score, and only the metadata that callers genuinely filter or display on.
• Put a version in the path or a header, and run old and new versions side by side during a deprecation window.
• Distinguish an empty result, an unreachable vector store, and an upstream embedding timeout, so callers know which failures are safe to retry.
• Make ingestion idempotent with a stable external ID, so a retried request updates a vector instead of creating a duplicate.
• Set a hard maximum top-k and return a real error rather than silently truncating the request.