Cleaning Up Feature Flags After a Model Rollout
A flag controlling which model version a request routes to is a useful, standard tool during a rollout. The problem shows up later, once the rollout is finished and the flag is still there, still evaluated on every request, and slowly turning into a piece of logic nobody remembers the original reason for.
Flags accumulate quietly because removing one feels riskier than leaving it, even after the situation it was built for no longer exists, and each additional flag makes the next one feel a little more normal to leave behind too.
Why Model Routing Flags Are Riskier to Leave Around Than Ordinary Flags
An ordinary feature flag usually toggles a UI element or a minor behavior. A model routing flag can determine which version answers a request, which means a stale flag left in an unexpected state can silently route real traffic to a model version you believed was retired. That is a more consequential kind of leftover than most flag hygiene advice, written for ordinary application flags, accounts for, and it deserves its own stricter standard rather than borrowing a generic flag policy wholesale.
Setting an Expiration Before You Ship the Flag
Decide when a routing flag should be removed at the same time you create it, not after the rollout is finished and the urgency to clean it up has already faded. A flag with a target removal date attached from the start is far more likely to actually get removed than one left open-ended with an implicit understanding that someone will get to it eventually. Treat that date the same way you would treat any other commitment with an owner and a deadline.
Auditing What's Actually Still Live
Periodically list every model routing flag currently in your codebase and check, for each one, whether the rollout it was built for is actually finished and whether it is still being evaluated on live traffic. A flag that always evaluates to the same result for every request is functionally dead code with extra risk attached, since it still represents a branch that could theoretically be triggered incorrectly by a bug elsewhere. This audit is worth running on the same cadence as your other infrastructure reviews, rather than waiting for a flag-specific incident to prompt it.
For example, put every model routing flag in a simple table with its owner, the rollout it belonged to, the date it should have been removed, and whether it still branches on live traffic. Flags past their date that no longer branch are removal candidates. Flags past their date that still branch need a decision from the owner about finishing the rollout. Reviewing that table at each infrastructure review takes little time and prevents the situation where nobody can say which flag decides where a request goes.
Removing a Flag Safely Once the Rollout Is Done
Removing a routing flag means committing to the single code path it was choosing between, so confirm the winning path has actually been stable in production for a meaningful stretch before removing the flag and the alternative path it was protecting. Removing it too early, before the winning path has proven itself, defeats the safety the flag was providing in the first place.
A Worked Example: Three Flags Deep and Nobody Sure Which One Fires
Say a model server has accumulated three separate routing flags over successive rollouts, each added for a different migration and none removed once its migration finished. A new engineer trying to understand why a specific request routed to an older model version now has to trace the interaction between all three flags rather than reading a single, current routing rule. Untangling that interaction after the fact takes longer than removing each flag would have taken right after its own rollout finished, which is the core argument for treating removal as part of the rollout rather than a separate task nobody schedules.
Making Flag Cleanup Part of the Rollout, Not an Afterthought
Add flag removal as a specific step in your rollout checklist, right alongside the steps for creating and ramping the flag in the first place, so it has the same visibility and the same expectation of completion. A general task tracking tool can hold that checklist and its due date, but the tool itself does not do the cleanup. What matters is that removal is written down as a required step with an owner, not left as an informal intention that quietly falls off everyone's list once the rollout feels done.
A rollout checklist that includes cleanup can look like this:
- Create the routing flag with a named owner and a target removal date attached from the start.
- Ramp the rollout, then confirm the winning path has been stable in production for a meaningful stretch.
- Remove the flag and the alternative code path it was protecting, committing to the single winning path.
- Record the removal in the same tracker that holds the rest of the rollout steps.
- Close the rollout only after the removal step is marked done, not when the ramp finishes.
What Good Looks Like
Every model routing flag has a removal date set when it is created, and a periodic audit confirms which flags are still actively branching versus quietly settled on one outcome.
Building The Capability (5-Stage Skill Ladder)
How to Get Started
Frequently Asked Questions
How is a model routing flag different from an ordinary feature flag?
A stale model routing flag can silently route real traffic to a version you believed was retired, which is a more consequential failure mode than most ordinary flags carry. That extra consequence is why routing flags need a stricter cleanup routine.
When should we decide a routing flag's removal date?
At the same time you create the flag, not after the rollout finishes. A flag with a target removal date attached from the start is far more likely to actually get cleaned up than one left open-ended.
How do we find routing flags that are effectively dead but still in the codebase?
Check whether each flag always evaluates to the same result for every current request. A flag that never actually branches anymore is dead code with the added risk of a bug elsewhere triggering the unused path unexpectedly.
About the numbers
This guide doesn't quote a sourced benchmark. Figures in it are estimates or general guidance, so check them against your own numbers.
Related Guides
A Rollout Checklist for Swapping Models in Production
A rollout checklist for swapping AI models in production: evaluation gates, canary and shadow traffic, and fast rollback paths.
Why Inference Latency Creeps Up After You Ship
Where AI model-serving latency actually hides: tokenization, queueing, batching windows, and network hops, plus a worked example fix.
Zero Trust for Machines Calling Your Model Endpoints
Why internal services calling your inference endpoints still need identity checks, and where CrowdStrike-style posture checks and Tenable-style scanning fit.
What a Real Security Audit of Model Serving Should Cover
A practical checklist for auditing AI model serving and inference: endpoint access, weight security, prompt logging, and patch timelines.
A 30-Minute Audit for Feature Flags Nobody Remembers
A short, repeatable way to find stale feature flags before they turn into a security gap or a confusing bug nobody can trace back to its actual cause.
What SOC 2 Actually Expects From a Model-Serving Team
What SOC 2 expects from a team serving AI models: how change, access, patch, and vendor controls apply, and the evidence to have ready.