Moving From Request-Response to Event-Driven Without a Rewrite

The clearest candidates for an event-driven approach are workflows where the client triggering the action doesn't need to wait for it to finish: sending a welcome email after signup, generating a report, processing an uploaded file. If your current implementation makes the caller wait synchronously for one of these to complete, that's added latency and a fragile dependency for no real benefit, since nothing about the interaction actually requires an immediate response.

Introduce a message queue for the chosen workflow by having the existing endpoint publish an event and return immediately, with a separate consumer picking up the event and doing the actual work asynchronously. Keep the old synchronous code path available behind a flag during the transition, so you can compare behavior and roll back cleanly if the new path misbehaves under real traffic, rather than committing to the new path before you've seen it handle production load.

For example, suppose signup currently sends a welcome email inside the request, so a slow mail provider makes signup slow. The endpoint can instead publish a signup event and return right away, while a consumer sends the email. Keep the old synchronous path behind a flag, and compare results on real traffic for a while. The common mistake is removing the old path on the same day the new one ships. If the consumer misbehaves, flipping the flag back should restore the old behavior immediately, with no emergency deploy.

A message that fails processing needs a defined path: a retry with backoff, a dead letter queue for messages that fail repeatedly, and an alert so someone notices a stuck queue rather than discovering it days later when a customer asks why their report never arrived. Write this behavior down and test it deliberately, by forcing a failure in a test environment, before the first real failure in production forces you to improvise the policy under pressure. Decide how many retries is too many and what happens next, since a message that retries forever against a permanently broken dependency is its own kind of stuck queue.

A producer publishing to a queue and a consumer reading from it are two separate identities that need their own authentication and authorization, the same as any other service to service call, not an implicit trust relationship because they're both internal. Decide who can publish to which topics and who can consume from them, and log that access the same way you'd log an API call, since an unauthenticated or overly permissive queue is a quiet way to reintroduce the exact trust assumptions a zero-trust architecture is meant to remove.

Most message queues either don't guarantee strict ordering or can deliver the same message more than once under specific failure conditions, and code written assuming synchronous, exactly once behavior will misbehave in ways that are hard to reproduce later. Make consumers idempotent, safe to process the same message twice without a bad side effect, from the start, rather than treating it as an edge case to handle later once a duplicate delivery has already caused a real problem. A simple, durable record of which message IDs have already been processed is usually enough to make this safe without redesigning the whole consumer.

Once the first migrated workflow has run reliably through a normal traffic cycle, including a failure or two that the retry and dead letter handling caught cleanly, that's the signal to consider moving the next workflow, not a fixed timeline. Treat each migration as its own small project with its own verification, rather than declaring the whole system event-driven after the first success and rushing the rest. A system with a handful of well understood asynchronous workflows and the rest synchronous is a perfectly reasonable stopping point, not a compromise you need to apologize for.

The migration path, one workflow at a time:

1. Pick one workflow where the caller does not need an immediate answer, such as a welcome email or report generation.
2. Have the existing endpoint publish an event and return, with a separate consumer doing the work, and keep the old path behind a flag.
3. Define retries with backoff, a dead letter queue and an alert for stuck messages, and test a forced failure.
4. Give producers and consumers their own authentication and authorization, and log that access.
5. Make consumers idempotent, then move the next workflow only once this one has become boring.