Getting Started with RabbitMQ: Python

In this blog, I am going to explain RabbitMQ.

RabbitMQ is a message broker. It accepts and forwards messages. A message broker acts as an intermediary platform when it comes to processing communication between two applications

Let’s first discuss the scenario where RabbitMQ could be useful:

Suppose you own a restaurant. Now imagine the situation where users are placing the order simultaneously and back-end servers are not processing the orders as fast as they should, or some backend error has occurred or hardware is malfunctioned. Since all the request were directly handled by the server, these request will not be processed.

Now let’s see how RabbitMQ can resolve this:

We can place a service in between the two services, i.e Frontend, and Backend. That service is Rabbit. It will consume all messages from Front-end and will only release when the backend is ready to process it.

Message brokers do many things such as:

1. 1. Decouple message publisher and consumer: A message queue provides an asynchronous communications protocol. You have the option to send a message from one service to another without having to know if another service is able to handle it immediately or not. Messages can wait until the responsible service is ready. A service publishing a message does not need to know anything about the inner workings of the services that will process that message. This way of handling messages decouple the producer from the consumer.

1. 1. Store the messages

1. 1. Routing of messages

1. Monitoring and management of messages

Let’s understand a few terms first:

Producer: A program that sends messages.

Consumer: A program that receives messages.

Channel: A channel is a virtual connection inside a connection. When you are publishing or consuming messages from a queue – it’s all done over a channel.

Connection: A connection is a TCP connection between your application and the RabbitMQ broker.

Routing Key: Routing keybinding with the queue are ruled that allow the exchange to put messages into the queue.

Exchange: The exchange receives messages from the producer and from the other side it pushes them to the queues. The exchange must know exactly what to do with the messages it receives. Should it be appended to a particular queue? Should it be appended to many queues? Or should it get discarded? The rules for determining that are defined by the exchange type.

There is mainly four type of exchanges available:

1. Direct: Sends a message to the queue whose binding key matches.
2. Topic: Sends the message on basis of the pattern.
3. Headers: A headers exchange is an exchange which routes messages to queues based on message header values instead of routing key.
4. Fanout: Sends a message to all the queues.

The RabbitMQ server is written in the Erlang programming language and is built on the Open Telecom Platform(OTP) framework for clustering and failover.

The producer creates a message and sends (publishes) into the message broker (RabbitMQ). A message must have two parts: a payload and a label(routing key). The payload is data and it can be anything from a simple JSON to MPEG-4 file.

Each queue is bound to a routing key or a pattern of routing keys. This routing keybinding with the queue is ruled that allow the exchange to put messages into the queue. The label describes the payload and how RabbitMQ will determine who should get a copy of the message.

The communication between publisher and RabbitMQ is one directional and fire and forget. The consumer, on the other hand, attaches to the broker and subscribes to a queue to get the message.

RabbitMQ speaks AMQP 0.9.1 Advanced Message Queuing Protocol, which is an open, general-purpose protocol for messaging.

In null-set, AMQP defines:

1. 1. Where to send messages (Routing)

1. 1. How to get there (Delivery)

1. What goes in must come out (Fidelity)

RabbitMQ provides persistence, delivery acknowledgments, publisher confirms and high availability.

Let’s understand using an example how RabbitMQ works:

Pika is a python client for RabbitMQ in python.

Pika is a pure-Python implementation of the AMQP 0-9-1 protocol that tries to stay fairly independent of the underlying network support library. To install pika run:

pip install pika

publish.py

import pika

#Create a new instance of the Connection object
connection = pika.BlockingConnection(pika.ConnectionParameters(host='localhost'))

#Create a new channel with the next available channel number or pass in a channel number to use
channel = connection.channel()

#Declare queue, create if needed. This method creates or checks a queue. When creating a new queue the client can specify various properties that control the durability of the queue and its contents, and the level of sharing for the queue.
channel.queue_declare(queue='hello')

channel.basic_publish(exchange='', routing_key='hello', body='Hello World!')    

print("[x] Sent 'Hello World!'")

connection.close()

subscribe.py

import pika

connection = pika.BlockingConnection(pika.ConnectionParameters(host='localhost'))

channel = connection.channel()

channel.queue_declare(queue='hello')

def callback(ch, method, properties, body):
print(" [x] Received %r" % body) 

channel.basic_consume(callback, queue='hello', no_ack=True)

print(' [*] Waiting for messages. To exit press CTRL+C')

channel.start_consuming()

Now start the Consumer, it will run continuously waiting for deliveries:

Now start the producer. The producer program will stop after every run:

Thanks for reading!!!