Data Engineering & Real-Time Event StreamsPlaybook3 min readUpdated September 2026

A Cloud WAF Audit Checklist That Catches Rules Nobody's Touched in Years

A web application firewall configured once during initial setup and never revisited tends to drift into one of two bad states: rules loosened over time to fix false positives until they're barely blocking anything, or rules left in log only mode during an early rollout and simply forgotten, providing visibility but no actual protection. Neither state shows up on a dashboard that just reports the WAF is on.

An actual audit, walking through the specific things that tend to drift, catches both problems faster than a general review of whether a WAF exists at all.

Vendors Covered in this Article

Disclosure: We may earn a commission if you buy through some links on this page. It doesn't change what we recommend.

How do you find WAF rules stuck in log only mode?

It's common practice to deploy a new WAF rule in log only mode first, watching for false positives before switching it to actually block traffic, and it's just as common for that switch to never actually happen once the initial monitoring period ends and attention moves elsewhere. Pull a full list of every rule and its current mode, not just the ones you remember configuring, since rules added by a managed rule set update can quietly land in a non blocking mode too.

For every rule still in log only mode, check how old it is and whether there's a specific known reason it hasn't been switched over. A stale log only rule that's simply been forgotten provides a false sense of protection that's worse than knowing you have a documented gap.

Audit rule modes with these steps:

  1. Pull the full list of every rule and its current mode, not only the ones you remember configuring.
  2. Include rules that arrived through managed rule set updates, since those can quietly land in a non blocking mode too.
  3. For each log only rule, check how old it is and whether a specific known reason explains why it never switched to blocking.
  4. Switch forgotten rules to blocking, and document any rule that stays in log only mode on purpose as a known gap.

Reviewing IP and path allowlists for entries nobody remembers

Allowlist entries added for a specific, temporary reason, a vendor integration test, a one off migration script, a contractor's IP address, tend to outlive their original purpose by a wide margin. Go through every entry and require a documented reason and an owner for each one still in place; anything without both is a candidate for removal.

Pay particular attention to broad allowlist entries, an entire IP range or a wildcard path, since these carry the most exposure if they're stale and forgotten. A narrow, well documented allowlist entry is a much smaller risk than a broad one nobody can explain, and reviewing quarterly keeps that risk from quietly accumulating again after the first cleanup.

Confirming managed rule sets are actually current

Cloud WAF providers regularly update their managed rule sets to cover newly disclosed attack patterns, but many WAF configurations pin to a specific rule set version at setup time and never update it, which means new coverage silently isn't being applied even though the WAF appears fully configured. Check whether your managed rule sets are set to auto update or pinned to an old version, and if pinned, check how long ago that version was current.

Test a rule set update in a lower stakes environment or in log only mode before flipping it to blocking in production, since a rule set update can introduce new false positives against your specific traffic patterns. This is a real operational cost of staying current, but it's a much smaller cost than running years behind on coverage.

How do you test that a WAF actually blocks attacks?

A configuration review alone doesn't confirm the WAF behaves as configured in practice. Run a controlled test, sending known malicious patterns like a basic SQL injection payload or a path traversal attempt against a non production endpoint, and confirm they're actually blocked rather than just assuming the rule configuration works as written.

Repeat this test after any significant configuration change, not just once during initial setup. A misconfiguration during a later change, a rule accidentally disabled, an exception added too broadly, is exactly the kind of gap a periodic functional test catches that a static configuration review can miss entirely.

Putting the audit on a recurring calendar instead of a one time pass

A single thorough audit fixes the drift that's already accumulated, but a WAF configuration keeps drifting afterward: new exceptions get added under deadline pressure, new managed rule set versions get released, a temporary allowlist entry gets added again for the next vendor integration. Put this checklist on a recurring quarterly calendar rather than treating it as a one time cleanup project.

Assign clear ownership for the recurring review, not just for the initial audit, since a checklist with no owner tends to quietly stop happening after the first pass loses its novelty. A short, consistently run quarterly review catches drift while it's still small and easy to explain, rather than letting it accumulate back into the same tangled state the original audit was meant to fix.

Executive Capability Standard

What Good Looks Like

A well maintained WAF has no forgotten log only rules, no stale allowlist entries without a documented owner, current managed rule sets, and a periodic functional test confirming it actually blocks known malicious traffic patterns.

Building The Capability (5-Stage Skill Ladder)

1. Learn:Pull a full list of every WAF rule, its mode, and every allowlist entry to see what's actually configured versus what you assume is configured.
2. Do Manually:Manually review each log only rule and allowlist entry for a documented reason and owner, and switch or remove anything that lacks one.
3. Delegate:Assign a security or platform engineer to own a recurring WAF audit on a quarterly schedule.
4. Automate:Automate managed rule set updates in a staged rollout, log only first, then blocking, along with a scheduled functional test against known attack patterns.
5. Buy:Bring in a fractional CTO or security specialist if your WAF hasn't been reviewed since initial setup and you're preparing for a security review or compliance audit.

How to Get Started

Disclosure: We may earn a commission if you buy through some links on this page. It doesn't change what we recommend.

Frequently Asked Questions

Why would a WAF rule still be in log only mode years after it was created?

Usually because it was deployed intentionally in log only mode during an initial rollout to catch false positives, and the follow up step to switch it to blocking mode was never completed once attention moved elsewhere. This is one of the most common gaps found in a WAF audit, and it provides visibility without any actual protection.

How do we know if our WAF's managed rule sets are actually up to date?

Check whether your configuration is set to auto update managed rule sets or pinned to a specific version, and if pinned, check how long ago that version was released. A rule set pinned at initial setup and never revisited misses coverage for attack patterns disclosed since then, even though the WAF still appears fully configured.

Is a configuration review enough to confirm our WAF is working correctly?

No. Run a controlled functional test against a non production endpoint, sending known malicious patterns and confirming they're actually blocked. A configuration that looks correct on paper can still have a gap from a rule accidentally disabled or an exception added too broadly, which only a real functional test reliably catches.

About the numbers

This guide doesn't quote a sourced benchmark. Figures in it are estimates or general guidance, so check them against your own numbers.

Related Guides