Webhook Idempotency Patterns: Preventing Duplicate Processing
Webhooks are often delivered multiple times. Learn battle-tested patterns to ensure your handlers process each event exactly once, even under retry conditions.
HookWatch Team
January 18, 2026
Webhook providers retry failed deliveries. Network issues cause duplicates. Your handler might receive the same event multiple times. Without proper idempotency handling, you could charge customers twice, send duplicate emails, or corrupt your data.
Why Duplicates Happen
Webhooks can be delivered multiple times for several reasons:
- Automatic retries: Provider didn't receive your 200 response
- Network timeouts: Response was sent but never received
- Race conditions: Multiple requests arrive simultaneously
- Manual replays: Debugging or recovery operations
The Idempotency Key
Every webhook should include a unique identifier. Use this as your idempotency key:
// Common idempotency key locations
const idempotencyKey =
req.headers['x-idempotency-key'] || // Custom header
req.headers['x-request-id'] || // Request ID
req.body.id || // Event ID (Stripe, Shopify)
req.body.event_id; // Alternative field namePattern 1: Simple In-Memory Cache
For low-volume webhooks, an in-memory set works:
const processedEvents = new Set();
app.post('/webhook', async (req, res) => {
const eventId = req.body.id;
if (processedEvents.has(eventId)) {
console.log(`Duplicate event: ${eventId}`);
return res.status(200).json({ received: true });
}
processedEvents.add(eventId);
try {
await processEvent(req.body);
res.status(200).json({ received: true });
} catch (error) {
// Remove from cache so retry can succeed
processedEvents.delete(eventId);
res.status(500).json({ error: 'Processing failed' });
}
});Limitation: Memory is lost on restart, and doesn't work across multiple server instances.
Pattern 2: Database Deduplication Table
For production use, store processed events in your database:
// Create a processed_events table
// CREATE TABLE processed_events (
// event_id VARCHAR(255) PRIMARY KEY,
// processed_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
// );
app.post('/webhook', async (req, res) => {
const eventId = req.body.id;
try {
// Try to insert - fails if duplicate
await db.query(
'INSERT INTO processed_events (event_id) VALUES ($1)',
[eventId]
);
} catch (error) {
if (error.code === '23505') { // Unique violation
return res.status(200).json({ received: true });
}
throw error;
}
await processEvent(req.body);
res.status(200).json({ received: true });
});Pattern 3: Atomic Upsert with Status
Combine idempotency check with business logic in one transaction:
app.post('/webhook/orders', async (req, res) => {
const order = req.body;
await db.transaction(async (tx) => {
// Upsert with idempotency check
const result = await tx.query(`
INSERT INTO orders (external_id, status, data)
VALUES ($1, 'processing', $2)
ON CONFLICT (external_id)
DO UPDATE SET
updated_at = CURRENT_TIMESTAMP
WHERE orders.status = 'pending'
RETURNING id, status
`, [order.id, JSON.stringify(order)]);
// Only process if we actually inserted
if (result.rows[0]?.status === 'processing') {
await fulfillOrder(order);
await tx.query(
'UPDATE orders SET status = $1 WHERE external_id = $2',
['completed', order.id]
);
}
});
res.status(200).json({ received: true });
});Pattern 4: Redis-Based Locking
For distributed systems, use Redis for coordination:
const Redis = require('ioredis');
const redis = new Redis();
async function processWithLock(eventId, handler) {
const lockKey = `webhook:lock:${eventId}`;
const processedKey = `webhook:processed:${eventId}`;
// Check if already processed
if (await redis.exists(processedKey)) {
return { duplicate: true };
}
// Acquire lock with 30-second timeout
const acquired = await redis.set(
lockKey, '1', 'NX', 'EX', 30
);
if (!acquired) {
// Another instance is processing
return { duplicate: true };
}
try {
await handler();
// Mark as processed (expire after 24 hours)
await redis.setex(processedKey, 86400, '1');
return { success: true };
} finally {
await redis.del(lockKey);
}
}
app.post('/webhook', async (req, res) => {
const result = await processWithLock(
req.body.id,
() => processEvent(req.body)
);
res.status(200).json({ received: true });
});Pattern 5: Idempotent Operations
Design your operations to be naturally idempotent:
// Bad: Incrementing (not idempotent)
await db.query(
'UPDATE users SET credits = credits + $1 WHERE id = $2',
[amount, userId]
);
// Good: Set to specific value (idempotent)
await db.query(
'UPDATE users SET credits = $1 WHERE id = $2',
[newBalance, userId]
);
// Good: Use external ID for upserts
await db.query(`
INSERT INTO payments (external_id, amount, status)
VALUES ($1, $2, 'completed')
ON CONFLICT (external_id) DO NOTHING
`, [paymentId, amount]);Cleanup Strategy
Don't let your deduplication table grow forever:
-- Daily cleanup of old events
DELETE FROM processed_events
WHERE processed_at < NOW() - INTERVAL '7 days';Testing Idempotency
Always test duplicate handling:
describe('Webhook Idempotency', () => {
it('should handle duplicate events', async () => {
const event = { id: 'evt_123', type: 'order.created' };
// First request should process
const res1 = await request(app)
.post('/webhook')
.send(event);
expect(res1.status).toBe(200);
// Second request should be idempotent
const res2 = await request(app)
.post('/webhook')
.send(event);
expect(res2.status).toBe(200);
// Verify only one order created
const orders = await db.orders.findAll();
expect(orders.length).toBe(1);
});
});Proper idempotency handling is essential for production webhook systems. Choose the pattern that fits your scale and infrastructure, and always test with duplicate events.