Message queues vs event logs

Two quite different systems are both called "message brokers".

#Message queues

RabbitMQ, Amazon SQS, Google Cloud Tasks, ActiveMQ.

• A message is delivered to one consumer, which processes it and acknowledges it; then it's deleted.
• If the consumer doesn't acknowledge in time (SQS's visibility timeout), the message is redelivered to another consumer.
• Add consumers to process faster ("competing consumers").
• Usually no ordering guarantee, or ordering per group in FIFO queues.

Best for jobs: send this email, resize this image, call this webhook.

#Event logs

Apache Kafka, Amazon Kinesis, Redpanda, Apache Pulsar.

• Messages are appended to a partitioned, ordered log and retained for a set time (or forever), regardless of whether they've been read.
• Each consumer group tracks its own offset per partition. Many groups read the same events independently: billing, analytics and search indexing each get every event.
• Order is guaranteed within a partition; choose a key (such as order ID) so related events share a partition.
• Within a consumer group, each partition is read by at most one consumer, so partition count caps parallelism. (Kafka 4.2's share groups let several consumers share a partition queue-style, at the cost of ordering.)
• Consumers can replay history by resetting offsets — to rebuild a derived store or fix a bug.

Best for events: facts that several systems react to, or that you want to keep as a history.

#Choosing

For modest volumes, a database table used as a queue (SELECT … FOR UPDATE SKIP LOCKED) is often enough (when not to add infrastructure). Whatever you choose, design for redelivery: see delivery guarantees.