Idempotent consumers

With at-least-once delivery, every message may arrive more than once. An idempotent consumer makes the second processing a no-op.

#Natural idempotency

Some effects are safe to repeat as written:

• Absolute writes: SET status = 'shipped' rather than "move to the next status".
• Upserts keyed by a natural ID: INSERT … ON CONFLICT (order_id) DO UPDATE with values from the message.
• Versioned updates: apply only if the event's version is newer: UPDATE … SET … , version = $v WHERE id = $id AND version < $v.

Prefer these when you can; they also handle reordering.

#Deduplication table

For effects that aren't naturally idempotent ("add 10 credits"), record each message ID in the same transaction as the effect:

BEGIN;
INSERT INTO processed_messages (message_id) VALUES ($1) ON CONFLICT DO NOTHING;
-- if that inserted 0 rows: this message was already processed → COMMIT and acknowledge
UPDATE accounts SET credits = credits + $2 WHERE id = $3;
COMMIT;
-- acknowledge the message only after the commit

A crash before the commit undoes both; a crash after it leaves the ID recorded, so the redelivery skips. This is sometimes called the inbox pattern. Prune old IDs after the broker's maximum redelivery window (for example, keep 7 days).

#Effects outside your database

You can't put an email provider's send in your transaction. Instead, pass a stable idempotency key derived from the message ID (msg-8f2c:email) so the provider deduplicates, or record "sent" before calling and reconcile failures (idempotency keys).

#What a message needs

Give every message a unique, stable ID set by the producer (not by the broker on each delivery), and include enough data — versions, timestamps from the source — for the consumer to make these decisions. See delivery guarantees.