Rate Limiting

Master API traffic control by implementing request limits that protect your system from overload while ensuring fair access.

Without these guardrails, a single misbehaving client could overwhelm your servers with thousands of requests per second. A bot scraping your endpoints could consume bandwidth meant for legitimate users. An infinite loop in someone's code could drain your database connections.

Rate limiting acts as a traffic control system for your API. It measures incoming requests against predefined rules and blocks or delays requests that exceed those limits.

The technique protects your infrastructure while maintaining predictable performance for well-behaved clients. When implemented correctly, most users never notice the limits exist.

How Rate Limiting Works

The system needs three pieces of information: who is making the request, what limits apply to them, and how many requests they have made recently. Client identification typically uses IP addresses, API keys, or user authentication tokens.

Request counting requires temporary storage that tracks usage over specific time windows. Redis and Memcached excel at this because they offer fast read/write operations and automatic key expiration.

When limits are exceeded, the API returns HTTP status code 429 along with headers indicating when the client can retry. Well-designed clients respect these signals and implement backoff strategies.

Common Rate Limiting Algorithms

Fixed window counting divides time into discrete periods. A limit of 100 requests per minute means clients can make 100 requests between 14:00:00 and 14:00:59, then another 100 between 14:01:00 and 14:01:59. The approach is simple but allows traffic spikes at window boundaries.

Sliding windows track requests over rolling time periods. Instead of fixed minute boundaries, the system counts requests made in the last 60 seconds from any given moment. This smooths traffic patterns but requires more complex counting logic.

Token bucket systems maintain a bucket of tokens that refill at a steady rate. Each request consumes one token. When tokens run out, requests are rejected. This allows short bursts while maintaining average rate limits over time.

Implementing Rate Limiting

They decide on tiered limits: 60 requests per minute for free accounts, 600 for paid plans, and 6000 for enterprise customers. The system uses API keys to identify clients and Redis to track request counts.

// APIForge rate limiting middleware
const redis = require('redis');
const client = redis.createClient();

async function rateLimitMiddleware(req, res, next) {
  const apiKey = req.headers['x-api-key'];
  const clientTier = await getClientTier(apiKey);
  
  const limits = {
    free: 60,
    paid: 600,
    enterprise: 6000
  };
  
  const windowSize = 60; // seconds
  const limit = limits[clientTier];
  const key = `rate_limit:${apiKey}:${Math.floor(Date.now() / 1000 / windowSize)}`;
  
  const current = await client.incr(key);
  await client.expire(key, windowSize);
  
  if (current > limit) {
    return res.status(429).json({
      error: 'Rate limit exceeded',
      limit: limit,
      reset: (Math.floor(Date.now() / 1000 / windowSize) + 1) * windowSize
    });
  }
  
  res.set({
    'X-RateLimit-Limit': limit,
    'X-RateLimit-Remaining': limit - current,
    'X-RateLimit-Reset': (Math.floor(Date.now() / 1000 / windowSize) + 1) * windowSize
  });
  
  next();
}

Rate Limiting Strategies

Client identification determines who shares limits. IP-based limiting is simple but problematic for users behind corporate NATs or mobile networks where hundreds of people share the same external IP. API key limiting provides better granularity but requires authentication.

User-based limiting offers the most precise control, allowing different limits for different user tiers. However, it only works for authenticated endpoints and requires user session management.

Some APIs combine approaches: strict IP limits for unauthenticated requests, generous authenticated limits, and premium limits for paid users. This encourages authentication while preventing anonymous abuse.

Expensive operations deserve stricter limits than cheap ones. Database writes, complex searches, and file uploads consume more resources than simple reads. Shopify limits product creation to 2 requests per second but allows 40 product reads per second.

Time-based limits can reflect business needs. A stock trading API might allow 1000 requests during market hours but only 100 after close. Social media APIs often have daily posting limits but no limits on reading content.

Advanced Rate Limiting

Distributed rate limiting becomes necessary when APIs run across multiple servers. Redis clustering or dedicated rate limiting services like Kong or Ambassador provide shared counters that work across server instances.

Fair queuing prevents a single client from monopolizing resources. Instead of rejecting excess requests immediately, the system queues them and processes at a controlled rate. This smooths traffic while maintaining throughput.

Adaptive limits adjust automatically based on system load. When CPU usage or response times increase, limits tighten to preserve stability. When resources are abundant, limits can relax to allow higher throughput.

# APIForge adaptive rate limiting
import time
import psutil
from redis import Redis

class AdaptiveRateLimiter:
    def __init__(self):
        self.redis = Redis()
        self.base_limit = 1000
        self.current_limit = self.base_limit
        
    def check_system_load(self):
        cpu_usage = psutil.cpu_percent(interval=1)
        memory_usage = psutil.virtual_memory().percent
        
        if cpu_usage > 80 or memory_usage > 85:
            # Reduce limits under high load
            self.current_limit = int(self.base_limit * 0.5)
        elif cpu_usage < 30 and memory_usage < 50:
            # Increase limits when resources available
            self.current_limit = int(self.base_limit * 1.5)
        else:
            self.current_limit = self.base_limit
            
        return self.current_limit
    
    def is_allowed(self, client_id):
        current_limit = self.check_system_load()
        window = int(time.time() // 60)  # 1-minute windows
        key = f"adaptive:{client_id}:{window}"
        
        current_count = self.redis.incr(key)
        self.redis.expire(key, 60)
        
        return current_count <= current_limit

Circuit breaker patterns complement rate limiting by cutting off traffic to failing services. When error rates exceed thresholds, the circuit opens and rejects all requests immediately rather than overwhelming struggling services.

Priority queues can exempt critical traffic from normal rate limits. Health checks, authentication requests, and administrative operations might bypass user-imposed limits to ensure core functionality remains available.

Client-Side Rate Limiting

Client-side limiting prevents applications from sending requests faster than servers can handle them. Instead of getting 429 responses, clients queue requests internally and send them at appropriate intervals.

Exponential backoff algorithms help clients recover from rate limit violations gracefully. When a request returns 429, the client waits before retrying. Each subsequent failure doubles the wait time until requests succeed again.

Request batching reduces the total number of API calls by combining multiple operations into single requests. Instead of creating 100 users with 100 API calls, batch creation might accomplish the same work with 5 calls.

// APIForge client SDK with built-in rate limiting
class APIForgeClient {
    constructor(apiKey, rateLimit = 10) {
        this.apiKey = apiKey;
        this.requestQueue = [];
        this.rateLimit = rateLimit; // requests per second
        this.lastRequest = 0;
        this.retryDelay = 1000; // start with 1 second
    }
    
    async makeRequest(url, options) {
        return new Promise((resolve, reject) => {
            this.requestQueue.push({ url, options, resolve, reject });
            this.processQueue();
        });
    }
    
    async processQueue() {
        if (this.requestQueue.length === 0) return;
        
        const now = Date.now();
        const timeSinceLastRequest = now - this.lastRequest;
        const minInterval = 1000 / this.rateLimit;
        
        if (timeSinceLastRequest < minInterval) {
            setTimeout(() => this.processQueue(), minInterval - timeSinceLastRequest);
            return;
        }
        
        const { url, options, resolve, reject } = this.requestQueue.shift();
        this.lastRequest = Date.now();
        
        try {
            const response = await fetch(url, {
                ...options,
                headers: {
                    'X-API-Key': this.apiKey,
                    'Content-Type': 'application/json',
                    ...options.headers
                }
            });
            
            if (response.status === 429) {
                // Server rate limit hit, implement backoff
                const retryAfter = response.headers.get('retry-after') * 1000 || this.retryDelay;
                setTimeout(() => {
                    this.requestQueue.unshift({ url, options, resolve, reject });
                    this.processQueue();
                }, retryAfter);
                this.retryDelay = Math.min(this.retryDelay * 2, 30000);
                return;
            }
            
            this.retryDelay = 1000; // reset on success
            resolve(response);
            
        } catch (error) {
            reject(error);
        }
        
        // Process next request
        if (this.requestQueue.length > 0) {
            setTimeout(() => this.processQueue(), minInterval);
        }
    }
}

Caching frequently accessed data reduces API calls significantly. Instead of fetching user profiles on every page load, cache them locally and refresh periodically. This pattern works especially well for reference data that changes infrequently.

Request deduplication prevents sending identical requests multiple times. If a user clicks a button rapidly, the client should recognize duplicate requests and only send one to the server.

Effective rate limiting requires balancing protection with usability, choosing appropriate algorithms for your traffic patterns, and providing clear feedback to clients about limits and violations.

Quiz