Delivery guarantees

Messaging systems describe what happens when something fails between sending a message and finishing its processing. The guarantee is mostly decided by when the consumer acknowledges.

#At most once

Acknowledge first, then process. If the consumer crashes mid-processing, the message is gone. No duplicates, possible loss. Acceptable for data that's cheap to lose: metrics samples, typing indicators.

#At least once

Process first, then acknowledge. If the consumer crashes after processing but before acknowledging, the broker redelivers and the message is processed again. No loss, possible duplicates. This is the default in SQS, RabbitMQ, Kafka consumers and almost every webhook system.

Duplicates also come from producers: a publisher whose acknowledgement from the broker was lost will resend.

#Exactly once

A broker can't guarantee exactly-once effects on systems it doesn't control: it can't know whether your database write or your call to an email API happened just before a crash. What you can build is effectively once:

effectively once = at-least-once delivery + idempotent processing

Kafka's "exactly-once semantics" (idempotent producers plus transactions) gives exactly-once within Kafka — reading from one topic and writing to another atomically. As soon as the effect leaves Kafka (a database, an HTTP call), you're back to idempotency.

#How to get idempotent processing

• Make the operation naturally idempotent: "set status to shipped" rather than "advance status".
• Record processed message IDs in the same transaction as the effect, and skip IDs already seen (idempotent consumers).
• Pass an idempotency key to external services.
• Carry version numbers so an older event can't overwrite newer state.

#On the producer side

Publishing reliably alongside a database change needs the transactional outbox; otherwise a crash between the commit and the publish loses the event.