Understand messaging system In Nodejs
When talking about messages and messaging systems, there are four fundamental elements to take into consideration; these are as follows:
- The direction of the communication, which can be one-way only or a request/reply exchange
- The purpose of the message, which also determines its content
- The timing of the message, which can be sent and received immediately or at a later time (asynchronously)
- The delivery of the message, which can happen directly or via a broker
In the sections that follow, we are going to formalize these aspects in order to provide a base for our later discussions.
One-way and request/reply patterns
The most fundamental aspect in a messaging system is the direction of the communication, which often also determines its semantics.
The most simple communication pattern is when the message is pushed one-way from a source to a destination; this is a trivial situation, and it doesn’t need much explanation:
A typical example of one-way communication is an e-mail or a web server that sends a message to a connected browser using WebSockets, or a system that distributes tasks to a set of workers.
The request/reply pattern is, however, far more popular than the one-way only communication; a typical example is the invocation of a web service. The following figure shows this simple and well-known scenario:
The request/reply pattern might seem a trivial pattern to implement; however, we will see that it becomes more complicated when the communication is asynchronous or involves multiple nodes. Take a look at the example in the following figure:
With the setup shown in the preceding diagram, we can appreciate the complexity of some request/reply patterns. If we consider the direction of the communication between any two nodes, we can surely say that it is one-way. However, from a global point of view, the initiator sends a request and in turn receives an associated response, even if from a different node. In these situations, what really differentiates a request/reply pattern from a bare one-way loop is the relationship between the request and the reply, which is kept in the initiator. The reply is usually handled in the same context of the request.
A message is essentially a means to connect different software components and there are different reasons for doing so: it might be because we want to obtain some information held by another system or a component, to execute operations remotely, or to notify some peers that something has just happened. The message content will also vary depending on the reason for the communication. In general, we can identify three types of messages, depending on their purpose:
- Command Message
- Event Message
- Document Message
The Command Message. The purpose of this type of message is to trigger the execution of an action or a task on the receiver. For this to be possible, our message has to contain the essential information to run the task, which is usually the name of the operation and a list of arguments to provide when it’s executed. The Command Message can be used to implement Remote Procedure Call (RPC) systems, distributed computations, or more simply used to request some data. RESTful HTTP calls are simple examples of commands; each HTTP verb has a specific meaning and is associated with a precise operation: GET, to retrieve the resource; POST, to create a new one; PUT, to update it; and DELETE, to destroy it.
An Event Message is used to notify another component that something has occurred. It usually contains the type of the event and sometimes also some details such as the context, the subject, or the actor involved. In web development, we are using an Event Message in the browser when using long-polling or WebSockets to receive notifications from the server that something has just happened, as, for example, changes in the data or, in general, the state of the system. The use of events is a very important integration mechanism in distributed applications, as it enables us to keep all the nodes of the system on the same page.
The Document Message is primarily meant to transfer data between components and machines. The main characteristic that differentiates a document from a command (which might also contain data) is that the message does not contain any information that tells the receiver what to do with the data. On the other hand, the main difference from an Event Message is mainly the absence of an association with a particular occurrence, with something that happened. Often, the replies to the Command Messages are Document Messages, as they usually contain only the data that was requested or the result of an operation.
Asynchronous messaging and queues
As Node.js developers, we should already know the advantages of executing asynchronous operations. For messaging and communications, it’s the same story.
We can compare synchronous communication to a phone call: the two peers must be connected to the same channel at the same time and they should exchange messages in real time. Normally, if we want to call someone else, we either need another phone or to close the ongoing communication in order to start a new one.
Asynchronous communication is similar to an SMS: it doesn’t require the recipient to be connected to the network the moment we send it, we might receive a response immediately or after an unknown delay, or we might not receive a response at all. We might send multiple SMSes to multiple recipients one after the other, and receive their responses (if any) in any order. In short, we have a better parallelism with the use of fewer resources.
Another important advantage of asynchronous communications is that the messages can be stored and then delivered as soon as possible or at a later time. This might be useful when the receiver is too busy to handle new messages or when we want to guarantee delivery. In messaging systems, this is made possible using a message queue, a component that mediates the communication between the sender and the receiver, storing any message before it gets delivered to its destination, as shown in the following figure:
If for any reason the receiver crashes, disconnects from the network, or experiences a slowdown, the messages are accumulated in the queue and dispatched as soon as the receiver comes online and is fully working. The queue can be located in the sender, or split between the sender and receiver, or live in a dedicated external system acting as middleware for the communication.
Peer-to-peer or broker-based messaging
Messages can be delivered directly to the receiver, in a peer-to-peer fashion or through a centralized intermediary system called a message broker. The main role of the broker is to decouple the receiver of the message from the sender. The following figure shows the architectural difference between the two approaches:
In a peer-to-peer architecture, every node is directly responsible for the delivery of the message to the receiver. This implies that the nodes have to know the address and port of the receiver and they have to agree on a protocol and message format. The broker eliminates these complexities from the equation: each node can be totally independent and can communicate with an undefined number of peers without directly knowing their details. A broker can also act as a bridge between the different communication protocols, for example, the popular RabbitMQ broker (http://www.rabbitmq.com) supports Advanced Message Queuing Protocol (AMQP), Message Queue Telemetry Transport (MQTT), and Simple/Streaming Text Orientated Messaging Protocol (STOMP), enabling multiple applications supporting different messaging protocols to interact.