Hardening the Containers Running Your Pipeline Workers

A default base image runs processes as root unless the Dockerfile explicitly switches users, and pipeline worker images built quickly during an early build-out often never got that line added. Running as a non-root user limits what an attacker can do if they get code execution inside the container. Check your worker images specifically, since these often get less security attention than public-facing services, on the assumption that they're not internet-exposed, which is often wrong.

A pipeline worker image built from a base image that hasn't been rebuilt in a year is carrying every vulnerability patched in that base image since, even if your own application code is fine. Rebuild worker images on a recurring schedule against the current base image, not only when you have an application code change to ship, since a security-only rebuild is a legitimate reason to redeploy on its own.

A worker container that can reach every internal service by default, rather than only the specific topic, database, or API it needs, turns a single compromised container into a much bigger problem than it needs to be. Scope network policy and IAM roles per worker type to exactly what that worker touches, not a broad role shared across every service in the cluster for convenience.

Scanning images for known vulnerabilities is common; acting on the results within a defined window is less common. Federal guidance requires fixing a critical, internet-accessible vulnerability within 15 days and a high-severity one within 30¹. Apply at least that standard to worker images, and track it explicitly: a scan result sitting unaddressed in a dashboard nobody checks isn't remediation, it's just visibility.

Set the deadline from the scan date, not from whenever someone happens to notice the finding. A finding that sat unread in a dashboard for three weeks before anyone saw it has already consumed most of a 30-day window before remediation even starts, and a deadline measured from discovery rather than from occurrence hides exactly how exposed you were during that gap.

A worker image built with an API key, a database credential, or a signing key baked into a layer, even one that was later removed in a subsequent layer, still has that secret recoverable from the image's history. Scan images specifically for this, not just for known CVEs, and pull secrets from a secrets manager at runtime instead of build time. This is a common finding in older worker images that predate a team's current secrets-handling standard, and it's worth an explicit audit pass rather than assuming newer practices were retroactively applied.

Continuous scanning and endpoint-level threat detection across your whole container fleet is exactly what platforms like CrowdStrike or Tenable are built for, and rebuilding that capability yourself for one pipeline rarely makes sense. What they don't replace is the application-level decisions: which base image you build from, whether your Dockerfile runs as root, and what network access a specific worker actually needs. Those choices are still yours to make correctly before a scanning tool has anything good to report on.

Treat the two as complementary layers rather than picking one. A scanning platform tells you what's wrong across the fleet at scale; the hardening choices in the checks above are what determine how much a single compromised container can actually do once something does get through.

A summary of what to confirm on every worker image:

• The container runs as a non-root user, set explicitly in the Dockerfile rather than inherited from the base image.
• The image is rebuilt on a recurring schedule against the current base image, not only when application code changes.
• Network policy and IAM roles are scoped per worker type to the specific topic, database, or API that worker touches.
• Scan findings are remediated within a defined window based on severity, and the window is tracked explicitly.
• No API keys, credentials, or signing keys are baked into any layer, and secrets are pulled from a secrets manager at runtime.