Testing Agent Workflows in CI When the Output Isn't Deterministic

Say you're building an automation that reads incoming invoices, extracts line items, and routes them to a client's accounting system. A traditional test asserts the output equals an exact string; that test will fail constantly once an LLM is generating part of the response, even when the extraction is correct.

Instead, your CI pipeline should run the agent against a fixed set of sample invoices on every pull request and check structural properties: did it extract the right number of line items, do the dollar amounts match, did it flag the one deliberately malformed invoice in your test set as unreadable rather than guessing. That's a test an agent can pass consistently even though its exact wording varies run to run.

Both platforms run this kind of evaluation job the same way a normal test suite runs, as a step in your pipeline. The difference shows up in matrix jobs: GitHub Actions' matrix strategy makes it simple to run your evaluation suite against two or three model versions in parallel and compare pass rates side by side in the pipeline summary, which matters when you're deciding whether to upgrade a client's automation to a newer model.

GitLab CI supports the same pattern through parallel jobs and its pipeline DAG, and its merge request widget surfaces failed evaluation jobs directly in the diff view, which is useful when a prompt change and a code change land in the same merge request and you need to know which one caused a regression.

A prompt change is a behavior change, and it deserves the same review and testing as a code change, not a quiet edit to a config file nobody reviews. Store prompts and agent configuration in the repository, require a pull request for changes to them, and run your evaluation suite against prompt changes exactly as you would against code changes.

This also gives you something client-facing: when an automation's behavior shifts, you can point to the exact commit and evaluation results that explain why, instead of guessing at what changed.

Unlike a unit test suite, an agent evaluation set usually produces a score, ninety four out of a hundred sample cases handled correctly, say, rather than a clean pass or fail. Set a minimum score threshold in your pipeline below which a deploy is blocked automatically, and treat anything between that threshold and a perfect score as something a human reviews before merging, since a small drop can be genuine noise or a real regression.

A deploy that goes out to a client's live automation should never happen purely on a passing test suite when the underlying model output is probabilistic. Build a manual approval step into the environment for production, even on a project that otherwise moves fast.

Run through this before flipping a client automation from staging to production:

• Does the evaluation set include the edge cases the client actually flagged during discovery, not just the happy path?
• Is there a rollback path if the model provider changes behavior on their end without you changing anything?
• Are API keys and model credentials scoped per client, so one client's usage can't exhaust another's budget?
• Does the pipeline log which model version handled each production run, so you can explain a client's specific outcome later?

These checks matter more for agent-based automations than for traditional software, because the thing you're shipping keeps behaving slightly differently after you ship it.

Unlike a dependency version you control, a hosted model can change behavior on a date the provider picks, not one you scheduled. Your evaluation suite is the thing that catches this: run it on a schedule against production, not just on pull requests, so a silent behavior shift shows up as a dropping score within a day rather than as a client complaint weeks later.

When a provider does change something, having your evaluation history gives you a specific before-and-after comparison to bring to the client, instead of an apology with no explanation attached. Agencies that skip this step tend to find out about a regression the same way the client does, which is a much worse position to explain from.