Rate Limiting

Rate limiting caps how many requests are allowed within a time window. It protects a service from being overwhelmed by bursts of traffic, keeps usage fair across clients, and helps you stay within the quotas that third-party APIs enforce. When a caller exceeds its allowance, the limiter rejects or delays the extra requests, usually answering with an HTTP 429 (Too Many Requests) response.

Common algorithms include the token bucket, the leaky bucket, and fixed or sliding window counters. They trade off differently between burst tolerance, memory cost, and smoothness, but they share the same goal: shape the flow of requests so a backend stays within a load it can sustain.

Rate limiting with HARP

Because HARP sits between your application and the remote APIs it calls, it is a natural place to apply this kind of control, without touching application code. HARP does not ship a dedicated rate limiter, but two building blocks let you keep request volume in check.

The rules engine runs Python at each stage of a request's lifecycle. In an on_request rule you can inspect the incoming traffic and short-circuit it, returning an early response instead of forwarding it to the remote service. That hook is what you would use to count requests and answer with a 429 once a threshold is reached.

Remote pools spread outgoing requests across several endpoints in a round-robin manner, so the load toward an upstream is divided rather than concentrated on a single target. Combined with the Retry pattern and the circuit breaker, this helps keep a backend within the volume it can handle.