Sponsor: Do you build complex software systems? See how NServiceBus makes it easier to design, build, and manage software systems that use message queues to achieve loose coupling. Get started for free.
Do you need message ordering? Processing messages in order as they were sent/published to a queue or topic sounds simple but has some implications. What happens when processing a message fails? What happens to all subsequent messages? How is throughput handled when using the competing consumers’ pattern, which loses the guarantee processing in order. Lastly, is ordering even important?
YouTube
Check out my YouTube channel where I post all kinds of content that accompanies my posts including this video showing everything in this post.
Message Ordering
What are some reasons that most people think they need to process messages in a particular order? The most common reason is workflow. As an example events are expected to have occurred in a particular order such as OrderPlaced, OrderBilled, OrderShipped.
The second most common is because of CRUD & Property-Based Events. Generally, this is more referring to Event Carried State Transfer, where you are trying to propagate the state of entities to other services. As an example, ProductCreated and ProductNameUpdated. You must process these events in order. If you process ProductNameUpdated before ProductCreated, you have no data/record to update and change the Product Name.
So how do you achieve message ordering? Well if you have a message broker that supports FIFO (First In, First Out) and you have a single consumer that is processing messages in a single-threaded or one at a time, you will process the messages in the same order as they were delivered to the broker.
As an example, we have a producer (or many) creating messages and sending them to the broker.
As the prior example, let’s say the first Event was ProductCreated, and then the user immediately changed the name, so another event was generated ProductNameUpdated. Both events are now on Topic waiting to be consumed.
With a single consumer, it will process the first event (ProductCreated).
Once it’s done and updated its local cache of the product, the consumer will then process the next event ProductNameUpdated.
Competing Consumers
When you want to scale and process more messages you’ll start using the competing consumers’ pattern. This is basically having multiple instances of the consumer running so you can process more messages concurrently.
This poses a problem because if we have two events ProductCreated and ProductNameUpdated sitting in our Topic waiting to be consumed, we now have two consumers able to process those events.
Since they are independent and process messages concurrently, this means that one consumer could be processing ProductCreated at the same time that the other consumer is processing ProductNameUpdated.
Now there is a race condition and we need ProductCreated to finish processing first, otherwise, ProductNameUpdated will fail since we haven’t finished processing ProductCreated yet.
Competing consumers applies to Topics, often called Consumer Groups as well as Queues.
One solution to solve this issue with processing messages in order when using competing consumers’ pattern is to process related messages one at a time. To achieve this, different messaging platforms will call these Partitions, Message Groups, or an Ordering Key.
To illustrate this, let’s say we have a Partition/Message Group/Ordering Key on a ProductID. This means that only a single consumer will get process the messages for that ProductID.
We have two messages sent to a Topic. ProductCreated for productID=1 and ProductCreated for ProductID=2.
The first (top in the diagram) consumer is responsible for handling all events where ProductID=1
And the bottom consumer is responsible for ProductID=2.
Now if we get another event ProductNameChanged for ProductID=1, it will to the first partition.
And the same consumer will handle it since it is handling all the messages for that partition.
This strategy allows you to process messages in the order that relate to each other and you want to process them one at a time. However, it still allows you to process many messages concurrently that don’t relate. Consumers can be responsible for many different Partitions/Message Groups/Ordering Keys.
Failures
Failures are another thing to consider. If you’re processing messages in order, how do you want to handle not being able to process a message? What happens to all the messages behind it waiting to be processed that relate to the failed message?
If the first event to be processed is ProductCreated, and the following event is ProductNameChanged. They are waiting to be consumed by a single consumer to be processed one at a time.
If the consumer attempts to process ProductCreated but it fails to do so, because of a bug, serialization issue, or whatever the reason, how do you now process ProductNameChanged?
This is very situational. Maybe you can discard the failed message and continue on. Maybe you need to treat it as a poison pill and stop processing any future messages. Again, this needs to be considered if you want to process messages in order.
WorkFlow
Do you really need to process messages in order? Often times you can create a policy to understand when all the relevant events have occurred so that you can then perform a specific action.
When an order is placed in our system, we need to charge the customer in billing, and then create a shipping label in the warehouse.
Billing will consume the OrderPlaced event and charge the customer.
Once it charges the customer, it will publish an OrderBilled event.
The warehouse will have a Policy (illustrated by an NServiceBus Saga below), that will keep track that both OrderPlaced and OrderBilled have occurred.
Now depending on message ordering and when these are published and how we consume them, these can be processed out of order. We could of processed OrderBilled first, and then later OrderPlaced was consumed, even though that’s not the order they were published.
Once both events have been consumed by the Policy/Saga, it can send a CreateShippingLabel command to the Warehouse.
Here’s an example of what this looks like with NServiceBus.
Message Ordering
So do you need to process messages in order? Maybe, maybe not. I’d take a look at the workflow of what you’re trying to achieve to be sure that you actually require to process messages in order. If you do, you’ll have to leverage FIFO (first-in, first-out) queues or topics. As well as use a broker that supports single consumers to process partitions, message groups, or ordering keys.
Join!
Developer-level members of my YouTube channel or Patreon get access to a private Discord server to chat with other developers about Software Architecture and Design as well as access to source code for any working demo application that I post on my blog or YouTube. Check out the YouTube Membership or Patreon for more info.