Message Queues and Structured IPC

• Define a message queue as a kernel-managed sequence of discrete, bounded messages, contrasted with a pipe's undifferentiated byte stream and shared memory's raw, unmediated region.
• Explain what "the kernel mediates every access" buys back compared to shared memory, and what it costs compared to shared memory's zero-copy speed.
• Describe message priority and selective receipt as real capabilities a message queue provides that neither a pipe nor raw shared memory does.
• Place message queues correctly on the spectrum this cluster has built so far: faster/less-safe (shared memory) to slower/more-structured (message queues), with pipes occupying a specific middle point.

This cluster has now covered two IPC mechanisms sitting at opposite ends of a real tradeoff: a pipe (kernel-mediated, but an undifferentiated byte stream with no message boundaries) and shared memory (blazingly fast, but with zero built-in structure or synchronization, leaving the cooperating processes to build both themselves). Message queues occupy a third, distinct point on this spectrum: like a pipe, the kernel mediates every single access, but unlike a pipe, the kernel treats each transfer as a discrete, self-contained message rather than a raw sequence of bytes — and unlike either pipe or shared memory, a real message queue can support message priorities and selective receipt, letting a consumer choose which waiting message to take next rather than being strictly limited to first-in-first-out order.

Understanding message queues completes this cluster's real picture of the actual tradeoffs system designers face: speed versus safety versus structure are not free — every mechanism this cluster covers buys one of these at some cost to the others, and no single mechanism dominates the other two on every axis.

A discrete message versus an undifferentiated byte stream

A pipe delivers bytes in the exact order they were written, with no concept of where one logical message ends and the next begins — if a writer performs three separate write() calls of 10 bytes each, a reader is not guaranteed to receive three separate 10-byte reads; it might see one 30-byte read, or ten 3-byte reads, entirely at the mercy of how the kernel happened to buffer and deliver the bytes. A message queue removes this ambiguity entirely: each send-equivalent operation enqueues one complete, self-contained message with a known length, and each receive-equivalent operation dequeues exactly one whole message, never a fragment and never more than one message merged together. This is a real, structural guarantee a pipe simply does not provide, and applications that need to reason about discrete units of work — one queued job, one logged event, one request — benefit directly from not having to re-implement message-framing themselves on top of a raw byte stream.

Kernel mediation: safety reinstated, speed given back

Because the kernel touches and manages every message in the queue — allocating space for it, tracking which messages are pending, delivering them to whichever process asks to receive next — message queues reinstate the two things shared memory gave up: built-in ordering and blocking (a receive call on an empty queue blocks until a message arrives, exactly like a pipe's read) with no need for the two cooperating processes to manage their own semaphores. The cost is symmetric to the benefit: every send and receive is a real system call, and the message's actual bytes are still copied — once into the kernel's queue storage, once back out to the receiving process — the same per-message overhead a pipe pays, now applied to whole structured messages instead of a raw stream.

Priority and selective receipt: capabilities neither pipes nor shared memory offer directly

A pipe's strict FIFO ordering means the very first byte written is always the very first byte a reader can see — there is no way for an urgent, later-arriving message to jump ahead of an earlier, lower-priority one already queued. Real message queue APIs (POSIX message queues, and System V message queues before them) support an explicit priority value attached to each message, letting a receiver retrieve the highest-priority pending message first regardless of arrival order — genuinely useful when, for example, an "abort now" control message needs to be processed ahead of a backlog of ordinary work items already queued. Some real APIs additionally let a receiver filter by a message-type tag, retrieving only messages matching a specific type and leaving others in the queue for a different consumer — a form of selective receipt neither a pipe (which has no concept of message identity at all) nor raw shared memory (which has no concept of a queue at all) provides.

Where message queues sit on this cluster's real tradeoff spectrum

Placing all three mechanisms on the same axis makes the tradeoff concrete: shared memory is fastest but provides no structure or synchronization of its own; a pipe is kernel-mediated (safe, ordered, blocking) but only delivers an undifferentiated byte stream; a message queue is also kernel-mediated, but additionally delivers discrete messages with optional priority and selective receipt, at the cost of being no faster than a pipe, and in practice often somewhat slower due to the added bookkeeping the kernel performs per message.

Example 1: A pipe's byte-stream ambiguity versus a message queue's discrete delivery

text
Writer performs three separate calls:
  write(fd, "AAAAAAAAAA", 10);
  write(fd, "BBBBBBBBBB", 10);
  write(fd, "CCCCCCCCCC", 10);

A pipe reader calling read(fd, buf, 30) may receive:
  "AAAAAAAAAABBBBBBBBBBCCCCCCCCCC"   -- all 30 bytes merged into
                                          one read, with no way to
                                          tell where one write ended
                                          and the next began.

A message-queue receiver, using send()/receive() instead, is
GUARANTEED three separate receive() calls, each returning exactly
one of "AAAAAAAAAA", "BBBBBBBBBB", "CCCCCCCCCC" -- never merged,
never split.

Example 2: Priority-based delivery, concretely

text
Queue receives, in this arrival order:
  msg1 (priority 1, "process batch job")
  msg2 (priority 1, "process batch job")
  msg3 (priority 10, "ABORT EVERYTHING")

A FIFO-only mechanism (like a pipe) would deliver msg1, then msg2,
then msg3, in that exact arrival order.

A priority-aware message queue instead delivers msg3 FIRST, despite
arriving last, because its priority (10) exceeds msg1 and msg2's
priority (1) -- the urgent control message reaches the consumer
before either queued batch job, without the consumer needing to
inspect and discard lower-priority messages itself.

Example 3: The three IPC mechanisms so far, compared on the same three axes

text
Mechanism        Kernel-mediated?   Message structure?   Relative speed
---------------  -----------------  --------------------  ---------------
Pipe             Yes                Byte stream only      Moderate
Shared memory    No (after setup)   None (raw region)      Fastest
Message queue    Yes                Discrete + priority     Moderate/slower

No single row dominates every column — each mechanism is a genuinely different point in the same design space, chosen based on which property (speed, structure, or built-in safety) a given application actually needs most.

• "A message queue is just a pipe that happens to be implemented differently." A pipe guarantees only byte-level ordering, with no concept of message boundaries; a message queue guarantees each send corresponds to exactly one, never-merged, never-split receive — a structural guarantee a pipe does not make.
• "Message queues are strictly better than pipes, since they do everything a pipe does plus more." They cost more per message (additional kernel bookkeeping for message boundaries and, if used, priority ordering) and are not meaningfully faster — for a simple, single-stream, FIFO-only use case, a pipe remains the simpler, adequate choice.
• "Message priority means higher-priority messages are processed faster once received." Priority only affects the ORDER in which pending messages are handed out by receive() calls — it says nothing about how quickly the receiving process itself processes a message once it has it.
• "Message queues avoid the copying overhead pipes have, since they're a different mechanism." They still copy each message's bytes into kernel storage and back out to the receiver — the same per-message copying cost a pipe pays, now applied to discrete messages instead of a raw stream, not eliminated by the added structure.

A message queue is a kernel-managed, kernel-mediated queue of discrete, bounded messages — unlike a pipe, which delivers an undifferentiated byte stream with no guaranteed message boundaries, and unlike shared memory, which provides no structure or synchronization at all. Because the kernel touches every message, message queues reinstate ordering and blocking automatically, the same benefit a pipe already provides, while additionally supporting priority-based delivery and selective receipt by message type — real capabilities neither a pipe nor raw shared memory offers. The cost is that a message queue is no faster than a pipe, and in practice sometimes slower, due to the added per-message bookkeeping the kernel performs to track boundaries and priority. Placing all three mechanisms side by side makes this cluster's real lesson explicit: speed, safety, and structure trade off against each other, and the right IPC mechanism depends on which of the three a given application genuinely needs most.