This repo is aimed to learn RabbitMQ or any MQ for general.
I was reading an article on NodeJS describing where it shines and what are its downfalls (Article link). NodeJS operates on event-based calbacks in a single thread event-loop. Therefore, NodeJS really shines in building fast, scalable application capable of handling a huge number of simultaneous/concurrent connections.
The main idea of Node.js: use non-blocking, event-driven I/O to remain lightweight and efficient in the face of data-intensive real-time applications that run across distributed devices.
But where it falls is heavy server-side computation/processing. In general, any CPU intensive operation will nullify benefits of Node because incoming requests will be blocked while the thread is occupied with the number-crunching.
No, you definitely don’t want to build a Fibonacci computation server in Node.js.
One of the solution he suggests is to,
offload all heavy computation to background processes written in a more appropriate environment, and have them communicate via a message queue server like RabbitMQ.
From there I started reading and learning about RabbitMQ and MQ(message queue) in general.
A Message Queue provides an asynchronous communication protocol, a system that puts a message onto a message queue and does not require an immediate response to continuing processing. This way of handling messages decouple the producer from the consumer. The producer and the consumer of the message do not need to interact with the message queue at the same time.
In simple words a producer places a message in queue, and continues its work. A consumer takes the message from queue processes it and returns the result. Thus decoupling two compnents.
- Producing means nothing more than sending. A program that sends messages is a producer
- A queue is essentially a large message buffer. Many producers can send messages that go to one queue, and many consumers can try to receive data from one queue.
- Consuming has a similar meaning to receiving. A consumer is a program that mostly waits to receive messages:
Example
Imagine that your web service has to be highly available and ready to receive new request instead of being blocked by the processing of previous requests. In this case it is ideal to put a queue between the web service and the processing service. The web service can put a "start processing" message on a queue and the other process can handle messages in order.
RabbitMQ is a message broker: it accepts and forwards messages.
In the diagram below, P is our producer and C is our consumer. The box in the middle is a queue - a message buffer that RabbitMQ keeps on behalf of the consumer.
- module:
npm install amqplib - send.js: Send message to queue
- receive.js: Receive message form queue
- module:
pip install pika - receive.py: Receive message from queue
- Install RabbitMQ
- Start MQ server:
sudo rabbitmq-server - Send Message:
node send.js - Receive Message:
node receive.jsorpython receive.py
-
Starting RabbitMQ server
-
Sending and receiving message
