Message Queues Are Amazing

Before creating Robust Design, I had very little knowledge of message queues. I've heard RabbitMQ mentioned and just nodded along, but I never truly understood what it was. I assumed it was another software tool engineers used. Deep down, I wanted to understand it. I wanted to truly understand system design and why it was so important for software engineers.

Then I built Robust Design. The initial build consisted of an API service, a user request component, and a load balancer, but it was missing something. I couldn't put my finger on it, but I felt it was missing something. Then came the time when I wanted to test out Robust Design by designing actual production systems like LinkedIn and Netflix. Then it hit me, I needed to add async processes that do not affect the user's response time. The only way to do that is by adding a message queue. Message queue allows you to send a "message" to a different service to perform a new operation, like calculating analytics, recommendations, updating a different user's information, with the actual requests experiencing little delays. It's like driving your car and waving "hi" to someone without stopping.

Learning this was a light-bulb moment for me, and it allowed me to implement my own message queue in Robust Design. I implemented a simple message queue with two operations: after_response and immediate. With an after_response, you can send a message to the message queue with data from the connected api service, but with an immediate response, both the parent api service and the message queue receive the same data. Message queues are one of the reasons you're able to scroll on social media and watch Netflix with minimal delays