API Design Patterns for Modern Web Applications: REST, Webhooks & Idempotency

# API Design Patterns for Modern Web Applications: REST, Webhooks & Idempotency As applications grow and interact with payment processors, mobile apps, CRM systems, and microservices, the quality of your API design directly determines system reliability. A poorly designed API causes silent duplicate billing, dropped customer webhooks, and painful breaking changes for client integrations. In this article, I cover four essential API design patterns that separate hobby projects from enterprise-grade production platforms. ## 1. The Idempotency Key Pattern Network failures are inevitable. A user clicks "Pay $100", their mobile connection drops, and the client retries the request. Without idempotency, the user gets billed twice. An **idempotent operation** is one where making the exact same request multiple times produces the identical outcome as making it once. While `GET`, `PUT`, and `DELETE` are naturally idempotent by HTTP specification, `POST` is not. Implement idempotency using an `Idempotency-Key` header: ``` POST /api/v1/charges Idempotency-Key: 7b8b2a1a-3c99-4d6b-95bb-599d1469e38f Content-Type: application/json { "amount": 10000, "currency": "usd" } ``` ### How the Idempotency Middleware Works: 1. Client generates a unique UUID and attaches it to the request header. 2. The API checks Redis or PostgreSQL for the key. 3. If the key exists with a completed response, the API returns the cached response immediately without re-executing logic. 4. If the key is currently being processed, return HTTP 409 Conflict. 5. If the key does not exist, lock the key, execute the charge, store the final response with a 24-hour TTL, and return the result. ```typescript export async function withIdempotency(key: string, handler: () => Promise) { const cached = await redis.get(`idempotency:${key}`); if (cached) { return JSON.parse(cached); } // Set processing lock const locked = await redis.set(`lock:${key}`, "1", "NX", "EX", 30); if (!locked) { throw new AppError(409, "A request with this idempotency key is already processing."); } try { const result = await handler(); await redis.set(`idempotency:${key}`, JSON.stringify(result), "EX", 86400); // 24h return result; } finally { await redis.del(`lock:${key}`); } } ``` ## 2. Reliable Webhook Delivery with Exponential Backoff When triggering webhooks to third-party endpoints (e.g. notifying a client system that an order was shipped), never deliver webhooks synchronously within the HTTP request lifecycle. If the destination server is slow or offline, your own application will hang. ### Resilient Webhook Architecture: 1. **Queue the event:** Store the event in a persistent queue (RabbitMQ, Redis/BullMQ, or PostgreSQL transactional outbox). 2. **Cryptographic Signing:** Sign the payload with HMAC-SHA256 using a shared secret so the receiver can verify message integrity. 3. **Retry Strategy with Exponential Backoff:** If the receiver returns a non-2xx status code or times out after 5 seconds, retry at increasing intervals: - Attempt 1: Immediate - Attempt 2: After 1 minute - Attempt 3: After 5 minutes - Attempt 4: After 30 minutes - Attempt 5: After 6 hours ```typescript import crypto from "crypto"; export function generateWebhookSignature(payload: string, secret: string): string { return crypto.createHmac("sha256", secret).update(payload).digest("hex"); } // Attach to outgoing headers: // X-Webhook-Signature: // X-Webhook-Timestamp: ``` ## 3. Cursor-Based Pagination for Large Data Sets Offset-based pagination (`OFFSET 10000 LIMIT 20`) suffers from two major problems: - **Performance:** PostgreSQL must scan and discard the first 10,000 rows, resulting in noticeable query latency as the database grows. - **Data drift:** If a new record is inserted while the user is paging, records shift and duplicates appear across pages. **Cursor-based pagination** solves this by filtering records relative to the last seen item ID or timestamp: ```sql -- High-performance cursor pagination SELECT id, title, created_at FROM articles WHERE created_at < '2026-03-10T14:00:00Z' ORDER BY created_at DESC LIMIT 20; ``` The API returns an opaque base64-encoded cursor token that the client sends in the next request: `?cursor=ZXlKMGVYQWlPaUkr...`. ## 4. API Versioning Without Breaking Changes When introducing changes to an API, adhere to strict backwards compatibility rules: - **Additive changes are safe:** Adding a new field to a JSON response rarely breaks compliant clients. - **Breaking changes require a new version:** Renaming fields, removing endpoints, or modifying authentication requirements must be released under a new URI prefix (`/api/v2/`). For external integration strategies, read my guide on [Third-Party API Integration Best Practices](/en/blog/third-party-api-integration-best-practices). For foundational data modeling, review [Database Schema Design Best Practices](/en/blog/database-schema-design-best-practices). Building a mission-critical API or multi-service architecture? Learn more about my [backend and API development services](/en/services/backend-api-development) or [reach out to discuss your project requirements](/en/contact).