Webhook Idempotency: A Production-Ready Guide

If a webhook can retry, you must assume you will see the same event more than once. That is not an edge case. It is the normal delivery model for most providers, and the only safe response is to make your side effects idempotent.

The goal is not literal exactly-once delivery. The goal is exactly-once effect: one customer update, one invoice, one state transition, even if the provider sends the payload twice, three times, or out of order.

Stripe's live-mode checklist makes the same expectation explicit: production webhook handlers should tolerate duplicate notifications and out-of-order delivery. In other words, duplicate-safe behavior is not a nice-to-have. It is part of the integration contract.

Webhook duplicates are usually caused by normal delivery behavior:

• timeout on the sender side
• transient network failure
• consumer crash after partial processing
• retry after ambiguous acknowledgement

That is why webhook systems usually behave more like at-least-once delivery than exactly-once delivery. If the consumer cannot tolerate that reality, duplicate side effects become inevitable.

Idempotency is not just "ignore duplicates." It is a contract about durable side effects. The same incoming event should lead to the same business outcome regardless of how many times it is delivered.

That means the consumer must decide what the unit of work is:

• a provider event ID
• a business key such as subscription_id + billing_period
• a generated idempotency key tied to the actual action

The idempotent consumer pattern is a useful mental model here: message delivery can repeat, so the durable effect must be safe to repeat too.

Different workflows need different defenses. Use this comparison as the main design lens:

The boring but strong default is often two layers: short-circuit duplicates early with a dedupe record, then protect the final write with a unique constraint.

Start by identifying the first durable write. That is where the duplicate must be blocked.

The safe flow is:

1. verify authenticity
2. extract the event ID or business key
3. check the dedupe store or unique constraint
4. if already processed, return success
5. persist the event marker and perform the side effect in the same transactional boundary when possible
6. record enough metadata for replay and debugging

The most dangerous design is marking the event as processed after the side effect. If the process crashes in that gap, a retry can execute the side effect again.

If the provider does not send a stable event identifier, you may need to build your own idempotency key from the business action. The key should represent the effect you want to protect, not just the raw payload.

TTL policy matters too:

• too short, and legitimate retries fall outside the protection window
• too long, and you keep unnecessary state forever

Ordering is the other hidden trap. Many providers do not guarantee a strict global order across related events. If your consumer assumes created always arrives before updated, you will eventually hit an impossible state.

Signature verification is separate from idempotency, but it must happen first. A duplicate signed event should still be treated as a duplicate. An unsigned or tampered event should be rejected before it reaches business logic.

If the provider supports replay protection or timestamp validation, use it. Authenticity and replay safety are part of the same boundary.

Stripe's webhook guide on signature verification and replay protection is a good concrete example of this boundary: verify the sender, preserve the raw body, and reject stale or tampered deliveries before business logic runs.

Replay should not be an afterthought you invent during an incident. If you expect to reprocess payloads, store enough evidence to do it safely:

• event ID
• source system
• received timestamp
• processing status
• business key used for dedupe
• correlation ID for tracing

That makes replay an operational capability instead of a gamble.

Use this checklist before shipping:

• confirm the provider's retry semantics
• identify the stable dedupe key
• create the dedupe store before the first side effect
• make signature verification mandatory
• define dedupe TTL and replay policy
• log event ID, processing status, and correlation ID
• make side effects safe to replay
• define how out-of-order events converge
• decide when manual replay or DLQ handling is needed

Run these tests too:

• same event arrives twice in quick succession
• same event arrives after a crash between dedupe write and side effect
• two events map to the same business object
• out-of-order update arrives before create
• valid replay arrives after the original was already handled
• unsigned payload is rejected before any write

The most common mistake is treating deduplication like a cache lookup instead of a durability guarantee. If the dedupe record is not written atomically with business state, duplicates will leak through.

Another mistake is assuming the provider will only resend an identical payload forever. In practice, the business effect may matter more than the raw payload.

The third mistake is poor observability. If you cannot trace a webhook from ingestion to side effect, you will not know whether the duplicate is real or whether your own system retried internally. That is why this topic connects directly to Observability for product engineers, Event-driven systems without folklore, and Queues are not a silver bullet.