What is an MQ and why do I want to use it?

Question

On my team at work, we use the IBM MQ technology a lot for cross-application communication. I've seen lately on Hacker News and other places about other MQ technologies like RabbitMQ. I have a basic understanding of what it is (a commonly checked area to put and get messages), but what I want to know what exactly is it good at? How will I know where I want to use it and when? Why not just stick with more rudimentary forms of interprocess messaging?

Answer 1

All the explanations so far are accurate and to the point - but might be missing something: one of the main benefits of message queueing: resilience.

Imagine this: you need to communicate with two or three other systems. A common approach these days will be web services which is fine if you need an answers right away.

However: web services can be down and not available - what do you do then? Putting your message into a message queue (which has a component on your machine/server, too) typically will work in this scenario - your message just doesn't get delivered and thus processed right now - but it will later on, when the other side of the service comes back online.

So in many cases, using message queues to connect disparate systems is a more reliable, more robust way of sending messages back and forth. It doesn't work well for everything (if you want to know the current stock price for MSFT, putting that request into a queue might not be the best of ideas) - but in lots of cases, like putting an order into your supplier's message queue, it works really well and can help ease some of the reliability issues with other technologies.

Answer 2

MQ stands for messaging queue.

It's an abstraction layer that allows multiple processes (likely on different machines) to communicate via various models (e.g., point-to-point, publish subscribe, etc.). Depending on the implementation, it can be configured for things like guaranteed reliability, error reporting, security, discovery, performance, etc.

You can do all this manually with sockets, but it's very difficult.

For example: Suppose you want to processes to communicate, but one of them can die in the middle and later get reconnected. How would you ensure that interim messages were not lost? MQ solutions can do that for you.

Answer 3

Message queueuing systems are supposed to give you several bonuses. Among most important ones are monitoring and transactional behavior.

Transactional design is important if you want to be immune to failures, such as power failure. Imagine that you want to notify a bank system of ATM money withdrawal, and it has to be done exactly once per request, no matter what servers failed temporarily in the middle. MQ systems would allow you to coordinate transactions across multiple database, MQ and other systems.

Needless to say, such systems are very slow compared to named pipes, TCP or other non-transactional tools. If high performance is required, you would not allow your messages to be written thru disk. Instead, it will complicate your design - to achieve exotic reliable AND fast communication, which pushes the designer into really non-trivial tricks.

MQ systems normally allow users to watch the queue contents, write plugins, clear queus, etc.

Answer 4

MQ simply stands for Message Queue.

You would use one when you need to reliably send a inter-process/cross-platform/cross-application message that isn't time dependent.

The Message Queue receives the message, places it in the proper queue, and waits for the application to retrieve the message when ready.

Answer 5

reference: web services can be down and not available - what do you do then? As an extension to that; what if your local network and your local pc is down as well?? While you wait for the system to recover the dependent deployed systems elsewhere waiting for that data needs to see an alternative data stream. Otherwise, that might not be good enough 'real time' response for today's and very soon in the future Internet of Things (IOT) requirements.

if you want true parallel, non volatile storage of various FIFO streams(at least at some point along the signal chain) use an FPGA and FRAM memory. FRAM runs at clock speed and FPGA devices can be reprogrammed on the fly adding and taking away however many independent parallel data streams are needed(within established constraints of course).