Lesson 02: Infrastructure

Message Queues & Pub/Sub

To implement an asynchronous architecture, we need a piece of infrastructure called a Message Broker (like RabbitMQ, Amazon SQS, or Redis Pub/Sub).

A message broker sits between the Producer (the Registration API) and the Consumers (Email, Billing, CRM). It acts as a highly reliable post office.

Publish-Subscribe (Pub/Sub)

In a Pub/Sub model, the Producer does not know who is listening. It simply publishes an event to an Exchange (or Topic) named user.events.

The Message Broker receives this event and copies it into three separate, independent Queues. Each consumer worker connects to its own queue and processes the messages at its own speed.

Pub/Sub Simulator

Click "Fire Event" rapidly. Notice how the Email worker processes events quickly, while the CRM worker is slow and builds up a backlog. The Registration API (Producer) is never blocked.

Producer
Registration API
Message Broker
Exchange: user.events
Billing Worker
(Med: 800ms)
CRM Worker
(Slow: 2000ms)
System is idle. Producer response time: 0ms.

Resilience

The beauty of this architecture is decoupling. If the CRM API goes offline completely, its queue will simply fill up with messages. The Registration API keeps working perfectly. The Email API keeps working perfectly.

When the CRM API comes back online hours later, it will simply start processing its backlog of messages, and no data is lost.

Check yourself

1. How does a Message Queue (like RabbitMQ) improve system resilience?

Decoupling means the Producer (e.g. Registration API) does not care if the Consumer (e.g. CRM API) is offline. It fires the event into the queue and instantly returns success to the user.