How to Avoid Processing the Same Message Twice
Using a message broker?
Assume a message might arrive more than once.
One common approach is to make the consumer idempotent.
For example:
if (await processedMessages.ExistsAsync(message.Id))
{
return;
}
await ProcessMessageAsync(message);
await processedMessages.AddAsync(message.Id);
An idempotent handler produces the same result even when the same message is delivered repeatedly.
Common techniques include:
- Store processed message IDs
- Use unique database constraints
- Use idempotency keys
- Make updates naturally idempotent
- Execute state changes and deduplication atomically where needed
At-least-once delivery means duplicates are part of the design problem.
If the same message arrives twice, processing it twice should not create two different business outcomes.