rate-limitingedgefairnessabuse-prevention
Token Bucket Rate Limiting
Allow controlled burst traffic while enforcing long-term request rate limits.
Definition
Token bucket refills tokens at fixed rate and permits requests while tokens remain, enabling bursts up to bucket capacity.
When To Use
- Public APIs requiring fairness and abuse protection.
- Traffic patterns with short bursts but predictable sustained rates.
- Edge enforcement where low-latency decisioning is required.
When Not To Use
- Workloads needing strict per-interval request ceilings without burst allowance.
- Low-traffic internal APIs where simpler static quotas are enough.
- Cases lacking globally consistent keying in multi-region deployments.
Tradeoffs
- Supports burst tolerance, but requires careful bucket sizing.
- Simple implementation, but can be gamed via key proliferation if identity is weak.
- Low decision latency, with complexity in cross-region token state sync.
Common Failure Modes
- Clock skew distorts refill behavior and fairness.
- Hot-key contention in centralized counter store causes p99 spikes.
- Regional state drift permits temporary quota overshoot.
Interview Framing
Use this structure when the interviewer asks for this pattern explicitly.
Call out bucket size/refill math, key design, multi-region consistency model, and behavior on store outage.
Related Project Deep Dives
API Rate-Limiting as a Multi-Region Service
Design a globally consistent rate limiting service with low latency and multi-region enforcement.
Feature Flag Evaluation Engine at Scale
Design a low-latency feature flag evaluation system with targeting rules, percentage rollouts, A/B testing integration, kill switches, and multi-region consistency.
Related Concepts
Backpressure
Control producer rate based on downstream capacity to avoid queue explosions and cascading failures.
Consistent Hashing
Distribute keys across nodes while minimizing remapped keys during node add/remove events.
Circuit Breaker
Protect services from cascading failures by short-circuiting calls to unhealthy dependencies.