While building applications these days, developers are often met with the need to decouple systems and separate them into components. However, with applications built into different components, each component cannot communicate with or understand how other components function.
The only communication between them that can provide this communication and information is by using the publish/subscribe (pub/sub) pattern.
In this article, we will look into the publish/subscribe pattern, how to set it up in a Node.js application, how to implement the pattern, and understand the alternatives to using Redis. This covers:
Before we do this, it’s important to understand the pattern fully before going into the nitty-gritty of using it in Node.js. Also, ensure you have the following installed:
Pub/sub is a messaging pattern where different components publish and subscribe to each other to notify and send data to each other.
It also allows the decoupling of components and each component relies on a message broker. Basically, a publisher publishes a message and a subscriber subscribes to the publisher to receive the message from a message broker.
Using this pattern provides easy development so developers can work and worry about their component or service without needing to understand the interface of other components.
This pattern also provides scalability so components are loosely coupled and easily made robust and secure.
Redis is an in-memory system that can store all types of data in a specific format and its main purpose is caching data, and it’s often chosen over alternates, especially memcached, because of its ability to store any kind of data in a robust manner.
It’s also important to note that Redis uses a data structure to hold data, and there is no query language or no flexibility to perform some ad-hoc queries.
I do steer towards Redis as a choice when using pub/sub because of its simplicity; there’s no hard science required, but only if an application is simple with no complexity.
Also, note that Redis is an in-memory system and data can only be handled as long the RAM allocated to it is capable. This means that Redis is good for real-time messaging and systems that send data quickly without delay.
Other use cases with caching include the following.
Before we begin, we know that Redis implements the pub/sub messaging pattern, which means the publisher sends a message to the subscriber, and the subscriber receives it. Note that more than one component can subscribe to a publisher.
Let’s see how it works on the Redis-CLI:
Vectormikes-MacBook-Pro:Projects macbookpro$ redis-cli 127.0.0.1:6379> SUBSCRIBE article Reading messages... (press Ctrl-C to quit) 1) "subscribe" 2) "article" 3) (integer) 1
The terminal above shows the subscriber subscribing to a channel called article.
Let’s now open a new terminal window:
Vectormikes-MacBook-Pro:Projects macbookpro$ redis-cli 127.0.0.1:6379> PUBLISH article Pub/Sub (integer) 1 127.0.0.1:6379>
Then, we can publish the message Pub/Sub to the same channel, article:
Vectormikes-MacBook-Pro:Projects macbookpro$ redis-cli 127.0.0.1:6379> SUBSCRIBE article Reading messages... (press Ctrl-C to quit) 1) "subscribe" 2) "article" 3) (integer) 1 1) "message" 2) "article" 3) "Pub/Sub"
All subscribers of the same channel get the same message on their end.
Note that the Redis server must be installed on your computer before you begin.
Next, create a folder to contain the working directory and name it redisNode:
$ mkdir redisNode && cd redisNode
Ideally, we want to create two different files that will act as the publisher and subscriber. But, let’s first create two folders that will be individual components with their own servers:
$ mkdir publish subscribe
Next, initialize the project for the publish component:
$ cd publish && npm init -y
Then, initialize the project for the publish component:
$ cd subscribe && npm init -y
Now, we have two separate components with their servers, using one as the publisher and the other as the subscriber.
Our focus here is to implement the component that will handle publishing messages to a specific channel.
To use Redis in Node.js, we need to install the redis package. It serves as a client to the Redis instance on our machine itself:
$ npm install redis
Now we can create a server.js file in the publish directory that will contain our code:
Learn more →const redis = require('redis');
const publisher = redis.createClient();
(async () => {
const article = {
id: '123456',
name: 'Using Redis Pub/Sub with Node.js',
blog: 'Logrocket Blog',
};
await publisher.connect();
await publisher.publish('article', JSON.stringify(article));
})();
In the code above, we import Redis and call the createClient() method. Now, we want to publish an article object containing the id, name, and blog.
With this object, we can send it as a message to the article channel:
$ publisher.publish('article', JSON.stringify(article));
Since we have a component ready to publish a message, let’s implement a subscribe component that will receive these messages. Ideally, we can create or have multiple components.
First, ensure Redis is installed:
$ npm install express redis
Next, create a server.js file in your subscribe directory to contain the implementation:
const redis = require('redis');
(async () => {
const client = redis.createClient();
const subscriber = client.duplicate();
await subscriber.connect();
await subscriber.subscribe('article', (message) => {
console.log(message); // 'message'
});
})();
To subscribe, open your terminal and run the subscribe file, server.js, and expect a message after you publish a message:
Vectormikes-MacBook-Pro:subscribe macbookpro$ node server.js
{"id":"123456","name":"Using Redis Pub/Sub with Node.js","blog":"Logrocket Blog"}
To publish, open another terminal and run the publish file.
On the contrary, if you want to implement pub/sub on the same server rather than on a different server, it is important to note that you cannot do so on the same Redis client because one Redis client cannot handle pub/sub; two must be available.
So instead, you must add two Redis clients to hand the subscribe and publish functions:
const subscriber = redis.createClient({ ... })
const publisher = redis.createClient({ ... })
Also, do not forget to connect your Redis client in an async function:
await client.connect()
Aside from using Redis, we do have alternates to Redis that implement the pub/sub pattern, especially in Node.js applications. Below is a list of the common alternatives to Redis.
Apache Kafka is a lot faster than Redis and was made to handle more data than Redis. Kafka also tends to have a long period of data retention.
RabbitMQ is a dedicated message broker that is more advanced than Redis, since it was designed to be a message broker for pub/sub patterns, offering more configurations and scenarios. Redis is just a memory store, which is an ideal database.
Google Cloud Pub/Sub is more scalable if not highly scalable compared to Redis. But, it is less configurable than Redis.
When it comes to Firebase Cloud Messaging, the choice is yours, but Firebase offers a document model rather than the in-memory in Redis.
In this short tutorial, we looked at how we can quickly set up a Node.js application to initiate the pub/sub pattern using Redis. This approach is geared to the microservice or the loosely coupled architecture.
Monitor failed and slow network requests in productionDeploying a Node-based web app or website is the easy part. Making sure your Node instance continues to serve resources to your app is where things get tougher. If you’re interested in ensuring requests to the backend or third-party services are successful, try LogRocket.
LogRocket is like a DVR for web and mobile apps, recording literally everything that happens while a user interacts with your app. Instead of guessing why problems happen, you can aggregate and report on problematic network requests to quickly understand the root cause.
LogRocket instruments your app to record baseline performance timings such as page load time, time to first byte, slow network requests, and also logs Redux, NgRx, and Vuex actions/state. Start monitoring for free.
Create a dynamic demo blog site using Django and React to demonstrate Django’s server-side functionalities and React’s interactive UI.
Explore how the aoi.js library makes it easy to create Discord bots with useful functionalities for frontend applications.
Evaluate Web Components, a set of standards that allow you to create custom HTML tags for more reusable, manageable code.
Leverage services like AWS Lambda, CloudFront, and S3 to handle images more effectively, optimizing performance and providing a better UX.
One Reply to "Using Redis pub/sub with Node.js"
“Apache Kafka is a lot faster than Redis” – I find that hard to believe… Except possible at massive scale (When Redis starts choking on its single thread)
Do you have any stats to confirm that statement?
I found another page that indicates Redis is in ‘single-digits’ of milliseconds, and Kafka is in 10’s of milliseconds. The opposite of your statement here.