Building a CI/CD Pipeline That Tests Models, Not Just Code

• An evaluation stage that runs the changed prompt template or model configuration against a fixed test set, not just the unit tests for surrounding code.
• A comparison step that diffs the new outputs against the previous version's outputs for the same inputs, surfacing changes for review rather than assuming silence means nothing changed.
• A cost check that flags a change likely to meaningfully increase token usage or GPU time, before it reaches production traffic.
• A gate that blocks merge, not just warns, when the evaluation stage fails, the same way a failing unit test blocks any other merge.

Treat a prompt template or model configuration change exactly like a code change: it goes on a branch, opens a pull request, and needs review before merge. The temptation is to let prompt tweaks skip this because they feel like content edits rather than engineering changes; that's exactly the category of change most likely to cause a quiet regression.

Reviewers for these pull requests need to look at the evaluation diff specifically, not just read the prompt text and judge it by eye. A prompt that reads fine can still shift model behavior in ways that only show up in the comparison output.

DORA's research clusters engineering organizations by deploy frequency, from teams deploying on demand down to a slowest cluster deploying between once a month and twice a year¹. The same pattern applies to model changes specifically: a team shipping small prompt or config updates weekly catches problems in a smaller blast radius than a team that batches months of changes into one release.

If your model pipeline currently ships rarely because the manual evaluation process is slow, that's the thing to fix first; frequency itself becomes a safety property once the evaluation gate is automated enough to keep up.

A narrow exception is worth naming explicitly: a genuinely cosmetic change, fixing a typo in a system prompt that doesn't touch instructions or behavior, doesn't need the full evaluation stage every time. Define that exception in writing, narrowly, rather than letting individual engineers decide case by case what counts as cosmetic.

Without a written exception, teams either run the full evaluation on trivial changes and start resenting the process, or skip it inconsistently on changes that weren't actually as safe as they looked. A narrow, explicit carve-out beats an unwritten judgment call either way.

Say an engineer tightens a prompt's instructions to reduce verbosity, tests it manually against three examples, and it looks good. The automated evaluation stage runs it against the full test set and flags a drop in accuracy on a category of question the manual check never covered.

Without the automated stage, this ships and shows up as a support complaint two weeks later. With it, the pull request fails a check the same way a broken unit test would, and the engineer sees exactly which cases regressed before merging anything.

• Running the evaluation stage only on a schedule instead of on every pull request, which lets several risky changes stack up before anyone looks.
• Treating an evaluation warning as optional because it's not a hard failure, which trains reviewers to click past it.
• Skipping the comparison step for a change deemed too small to matter, which is exactly the category that slips through.