Webhooks arrive twice, late, or out of order
“An order is fulfilled twice, a subscription flips back to an old status, or an update never lands because the webhook handler trusted delivery order.”
THE PRINCIPLE
A webhook is a hint that something changed, not an ordered log of the truth.
FIRST MOVES
- Verify the signature against the raw request body before parsing; reject stale timestamps.
- Insert the event ID into an inbox table with a unique constraint; skip IDs already seen.
- Return 2xx immediately after persisting; process in a background worker.
- In the worker, fetch the current object from the provider API instead of applying payload fields in arrival order.
- Reconcile periodically: retries eventually stop, and some providers never redeliver automatically.
TOOLS: THEN → NOW
Business logic inside the HTTP handlerDurable inbox table or queue plus idempotent workers
Trusting event timestamps for orderingRefetching current state or comparing object versions
PATTERN SNAPSHOT
app.post("/webhooks/stripe", express.raw({ type: "application/json" }), async (req, res) => {
let event;
try {
event = stripe.webhooks.constructEvent(req.body, req.get("stripe-signature"), secret);
} catch {
return res.sendStatus(400);
}
// Durable inbox: dedupe by event id; a worker does the real work.
await db.query(
"INSERT INTO webhook_inbox (event_id, type, body) VALUES ($1, $2, $3) ON CONFLICT (event_id) DO NOTHING",
[event.id, event.type, req.body.toString("utf8")],
);
res.sendStatus(200);
});CLOSE THE AI. EXPLAIN THIS.
If invoice.paid arrives before invoice.created, what should the handler do?HOW IT WORKS UNDERNEATH
RELATED SYMPTOMS
SOURCES
Guide reviewed