Catching a Broken Schema Before It Reaches a Client's Dashboard

Say a client's CRM export quietly renames a customer_id column to account_id during a routine vendor update. A code-only test suite that checks whether your transformation code runs without throwing an error will pass, because nothing in the SQL itself is broken, it just silently starts joining on a column that no longer means what your model assumes.

A data quality test, checking row counts, null rates, and whether key columns still exist with the expected name, catches this before it reaches a client's dashboard. Tools like dbt's built-in testing framework or a lightweight custom check running as a pipeline step both work; what matters is that the check runs on every pipeline execution, not just when someone remembers to look.

A software pipeline's tests ask whether the code behaves correctly against known inputs. A data pipeline needs a second layer: does the data itself still look like what the code assumes it looks like. Build both into your CI setup, code tests that run against fixture data on every pull request, and data quality checks that run against the actual current source data on a schedule.

The second kind of test is the one clients actually notice when it's missing, since a wrong number quietly appearing on a client's dashboard is a worse outcome than a build failure they can see and understand.

Both platforms run scheduled pipeline triggers well, which matters for data work since a lot of your pipeline runs are date-driven rather than triggered by a code change. GitHub Actions' cron-based scheduled workflows and GitLab CI's scheduled pipelines both cover this cleanly.

The real difference shows up in how each surfaces a failure. GitLab CI's pipeline dashboard groups scheduled and merge-triggered runs together in one view, which is convenient when you're triaging whether last night's data refresh failed for a code reason or a data reason. GitHub Actions requires a bit more setup, usually a dedicated Slack or email notification step, to get the same visibility without someone manually checking the Actions tab each morning.

A dbt project's compiled documentation, showing lineage from source to dashboard, is genuinely useful to a client during a handoff or an incident, and it's easy to generate as a pipeline step rather than something someone runs locally and forgets to share. Publish it as a build artifact on every successful pipeline run so it's never more than one deploy out of date.

On deployment frequency: teams with pipelines mature enough to deploy small model changes routinely tend to sit in the higher-performing DORA clusters rather than batching changes into large, infrequent releases¹. For a data pipeline specifically, that usually looks like shipping one model change at a time instead of a monthly bundle of unrelated fixes.

Before merging a change to a client-facing data model, confirm:

• Does the change include a test for the specific data quality issue it's meant to fix, not just a code-level assertion?
• Will the change break a downstream dashboard or report that isn't obviously connected to the model being changed?
• Is there a rollback plan if the change produces wrong numbers that reach a client before anyone notices?
• Does the pipeline log which source data version fed a specific run, so a wrong number can be traced back to its cause?

A data pipeline's failure mode is usually silent, not loud, which is exactly why these checks need to be automated rather than left to someone noticing a number looks off.

Eventually a wrong number reaches a client despite your checks, and how you explain it matters almost as much as how fast you fix it. Because your pipeline logs which source data version and which model version produced a given run, you can point to the specific upstream change that caused it rather than offering a vague apology with no explanation attached.

Clients tolerate an occasional data issue far better when the explanation is specific and the fix includes a new automated check preventing the same failure mode next time, rather than a promise to be more careful.