Building a Role Matrix for a Distributed System

Start with who actually needs access, not what they might need someday. Pull the list of humans and services touching your system and group them by the job they do: engineer on call, support staff resolving tickets, the billing service reading customer records, the reporting job that runs nightly.

Resist the urge to create a role for every individual person. A handful of well-defined roles that map to actual jobs is easier to audit than fifty one-off permission sets built up over two years of individual requests.

For each role, write down exactly what it needs to read, write, and where. The support role probably needs read access to customer records and write access to a ticket system, not write access to billing. The nightly reporting job needs read access to specific tables, not admin rights on the whole database.

Where a role seems to need broad access, dig into why. Usually it's because a narrower permission wasn't available yet, not because the job genuinely requires that much reach.

It's easy to focus a role matrix on human users and wave through service-to-service access with a shared API key that can do anything. That's usually the bigger risk, since a compromised service account often has broader, longer-lived access than any single employee.

Give each service its own credential scoped to exactly what it calls, not a shared key reused across five services because provisioning a new one felt like extra work at the time.

Say a twelve-person engineering team is splitting into three feature teams, each owning its own services. Before the split, most engineers had broad access because the team was small enough that it didn't seem to matter.

The matrix for the new structure gives each team full access to its own services, read-only access to shared infrastructure they depend on, and no access at all to the other teams' production databases unless there's a specific, logged reason. On-call engineers get temporary elevated access during their rotation, not permanently.

• Access granted for a one-time project that never gets revoked once the project ends
• A role's permissions expanding gradually over time as small exceptions pile up
• New hires copied from an existing employee's access instead of assigned from the matrix
• Contractors and former employees left with valid credentials well past their last day

Any one of these turns a clean matrix into an inaccurate one within a few months if nobody's checking.

A quarterly access review, checking every role against who actually still needs it, catches drift before it becomes a real exposure. This is exactly the kind of recurring evidence that platforms like Vanta are built to automate, turning a manual spreadsheet review into a scheduled, logged process.

Whether you automate it or not, the review has to actually happen. A policy that says quarterly reviews occur, with no calendar reminder and no owner, is a policy that gets skipped the first busy quarter.

Every access model eventually meets a 2 a.m. incident where the on-call engineer needs access they don't normally have, right now, with no time for a ticket. Without a designed exception, the usual outcome is a standing admin credential kept around just in case, which defeats the rest of the matrix.

A break-glass process solves this properly: a short-lived credential, granted through a specific request that's logged and time-boxed, expiring on its own after a few hours whether or not anyone remembers to revoke it. The access still happens, but it happens through a path that leaves a record instead of through a permanent exception nobody reviews. Treat every use of it as an input to the next access review, since a role that needs break-glass access often shouldn't be in the matrix the way it's currently defined.