How to Swap Embedding Models Without Taking Search Down

Cosine similarity between a vector from your old embedding model and one from the new model is meaningless, even if both are technically valid float arrays of the same dimension. If you re-embed new documents with the new model but leave existing documents on the old one, queries will silently favor whichever set happens to score higher for reasons that have nothing to do with relevance. Keep the two vector spaces in entirely separate indexes for the whole migration, never partially mixed.

Stand up a second index, re-embed your full corpus into it with the new model, and let it build fully before any query traffic touches it. This costs more compute and storage temporarily than an in-place swap would, but it means your production index keeps serving correct results the entire time the new one is being built, instead of degrading gradually as it fills with a mix of old and new vectors.

Before routing real traffic to the new index, run a fixed set of test queries, the kind your team actually cares about getting right, against both indexes and compare the results side by side. A new embedding model can be measurably better on a public benchmark and still return worse results for your specific corpus and query patterns, so a benchmark score alone isn't a green light. Fix issues you find here while the old index is still serving live traffic and nothing is at risk.

Keep this comparison query set somewhere durable, not in a scratch notebook someone deletes after the migration. The same queries are what you'll run against the next model swap, and a comparison set that grows over time, adding cases where the old model got something wrong, becomes a real regression check instead of a one-off spot check you invent fresh each time.

Route a small share of query traffic to the new index first, watch relevance and latency, then increase the share over hours or days rather than flipping every user at once. Keep the old index fully intact and queryable during this whole window; the moment something looks wrong, you want to route traffic back to it in minutes, not rebuild it from scratch. Only decommission the old index once the new one has run at full traffic for long enough that you're confident, not on the day the migration finishes.

The first embedding model swap on a given corpus is always the most expensive, mostly because nothing is scripted yet. Write down the re-embedding job, the comparison queries, and the traffic-shifting steps as you go, so the next time a vendor deprecates a model or a better one ships, you're running a known procedure instead of relearning it under time pressure. Vector databases and embedding providers both deprecate old models on their own timelines, not yours, so this isn't a hypothetical.

• Re-embedding new documents with the new model while old documents stay on the old model, mixing vector spaces silently
• Cutting over all traffic at once with no gradual rollout
• Deleting the old index as soon as the new one is built, before it's been proven under real traffic
• Comparing embedding models only on a public benchmark instead of your own queries
• Treating the migration as a one-time event instead of a repeatable runbook, so the next model swap starts from scratch