Skip to main content
Ratelimitly logo
Ratelimitly

Keep your API fast—even at peak

Rate limiting and admission checks that help reduce overload and protect expensive work.

Request early access Customer Log in

Reliable at peak

Set admission budgets for launch spikes and promos before excess work reaches your handlers.

Lower cloud costs

Fewer moving parts to run and monitor. Spend less on Redis and ops.

Dev‑first adoption

Add via SDK. Keep your stack. Clear limits your customers understand.

Request early access

UNDERSTAND OVERLOAD · KEEP USEFUL WORK MOVING

Keep useful work moving.

For developers, architects, and managers: 27 narrated lessons on why services overload and how to protect them. No cloud experience required. Start with one request, or jump to any lesson.

Course map Lesson 3 of 27 Browse all lessons

Four chapters, one learning path. Read or listen in any order. ~41 minutes total · Times shown at 1×.

Keep enough capacity to finish work and recover.

  1. 01 A useful result starts with one request
  2. 02 Unfinished work grows when arrivals outrun completions
  3. 03 More waiting space does not make work finish faster You are here
  4. 04 A timeout ends waiting, not necessarily the work
  5. 05 Retained work can exhaust a process’s memory
  6. 06 One failed copy can push its neighbors over capacity
  7. 07 Refuse new work while the application can still recover
  8. 08 Different compute platforms still have finite resources
  9. 09 A shared database failure can spread back into every app
  10. 10 Restore capacity before restoring full traffic
  11. 11 A circuit breaker stops repeating a failing call

Contain one customer's excess without punishing everyone.

  1. 12 One customer’s excess should not ruin another’s day
  2. 13 A rate limit checks one customer’s recent allowance
  3. 14 Copies and languages must share the same intended policy
  4. 15 Fairness helps contain damage without reserving every resource
  5. 16 A legitimate crowd can exceed shared capacity

Reduce work entering a shared dependency when it slows.

  1. 17 Measure the operation whose pressure you want to observe
  2. 18 A shared tracker observes; a guard decides; your app acts
  3. 19 Even the fastest recent observations can become slow
  4. 20 Separate local histories can reopen into the same burst
  5. 21 Replace optional expensive work before refusing essential work
  6. 22 A local breaker learns after calls have already piled up
  7. 23 Fast errors explain why the protections remain complementary
  8. 24 Latency feedback helps without measuring exact spare capacity

Choose, combine, and test the right protections.

  1. 25 Ask three different questions before expensive work
  2. 26 Recovery attempts need limits too
  3. 27 Measure useful service, not how much work you accepted

Read the explanation or use the course map above. Audio is optional.

3 / 27

More waiting space does not make work finish faster

Queue fill and drain

100 / sarrive0 / 100 waiting80 / sfinishStart with an empty queueWaiting space does not do the work1 tile = 20 waiting requests · equal-cost, constant-rate illustration100 / s arrive0 / 100 waiting1 tile = 20 requestsWorker: 80 / s finishSame processing capacityStart with an empty queueWaiting space does not dothe work
  1. Queue Waiting requests

    The queue holds work; it does not perform it.

  2. Worker 80 completions/s

    This finishing capacity stays the same.

Equal-cost, constant-rate illustration. One tile holds 20 waiting requests; empty tiles are free slots.

Read the numbered steps in order. The explanation below follows the narration.

A queue stores waiting work; a buffer supplies bounded space.

Bigger buffers buy time. Caching can remove work. Backpressure reduces incoming work.

0:00
Read transcript & diagram walkthrough

What you will learn

Bigger buffers buy time. Caching can remove work. Backpressure reduces incoming work.

A queue holds requests waiting for their turn. The worker, not the queue, performs the work. A buffer provides waiting space. In our example, one hundred requests arrive each second and eighty finish. Twenty extra requests must wait each second.

Start with one hundred empty waiting slots. After five seconds, all one hundred slots are occupied. With two hundred slots, the same excess fills the buffer in ten seconds. The worker still finishes eighty per second. We bought five more seconds, not a faster worker. Once full, refuse work, slow its supplier, or move waiting elsewhere; that still needs limits.

Now consider a different case: the burst ends after four seconds, with eighty requests waiting. New arrivals fall to forty per second. The worker can now clear forty waiting requests each second, so those eighty waiting requests disappear in two seconds. If arrivals merely fall to eighty, the queue stops growing but never drains. This is when a bounded queue helps: the burst ends, and later capacity can catch up before the result becomes useless.

A durable queue keeps jobs in storage across a restart. It preserves waiting work, not processing capacity. That suits jobs whose results can arrive later. A cache reuses a computed answer, avoiding repeated work. Backpressure asks the supplier to slow down; it helps only if that supplier responds. For sustained overload, change the work required, the work arriving, or the capacity that can finish it.

Diagram walkthrough
  1. Waiting space and processing capacity are different.

    Requests wait in the queue until the worker can take them. The tiles represent waiting space, not extra workers or faster execution.

  2. 100 arrive − 80 finish = 20 more waiting each second.

    The worker stays busy, but each second brings twenty requests it cannot yet finish. This simple model assumes equal-cost work, constant rates, and no refusal or cancellation.

  3. Start the fill-time comparison from empty.

    Reset the illustration to an empty 100-slot buffer. The available slots, not the finishing rate, determine how long this particular excess can be stored.

  4. The 100-slot queue is full after five seconds.

    100 free slots ÷ 20 extra requests per second = 5 seconds. Every slot is occupied, while the worker still finishes only eighty requests each second.

  5. Twice the waiting space lasts twice as long—not forever.

    This alternative starts with 200 empty slots under the same continuing overload. After ten seconds it is full too: the larger buffer has not removed the twenty-request-per-second mismatch.

  6. A full queue needs an explicit policy.

    Keeping a full queue does not create another slot. Refuse new work, get its supplier to slow down, or recognize that waiting is moving elsewhere and needs limits there too.

  7. Alternative: the burst ends before the queue fills.

    Reset to a different case: after four seconds of twenty extra requests per second, eighty requests are waiting. The burst now ends; this is not a continuation of the full-buffer example.

  8. New work now uses only part of the worker’s capacity.

    Forty new requests arrive per second while the worker can finish eighty. The remaining forty completions per second can reduce the already waiting backlog.

  9. The queue shrinks by forty requests per second.

    After one second of the quieter period, the backlog falls from eighty to forty. Using the full finishing rate as the drain rate would forget the new requests still arriving.

  10. Two quieter seconds clear the backlog.

    80 waiting ÷ (80 finishing − 40 arriving per second) = 2 seconds. New requests still arrive, but the old backlog has drained in this constant-rate example.

  11. Matching capacity stops growth; it does not clear old work.

    Alternative outcome from the same eighty-request backlog: eighty arrivals and eighty completions per second leave zero spare finishing capacity. The old waiting count stays at eighty.

  12. A useful buffer needs a burst that ends and capacity to drain it.

    Return to the successful forty-arrivals-per-second case. The backlog cleared because capacity remained after serving new arrivals, while the waiting results were still useful.

  13. Preserving jobs changes survival—not processing speed.

    Durability preserves waiting work across a restart, not processing capacity. It can suit delayed results, but storage limits, waiting age, and sufficient later capacity still matter.

  14. Reuse can remove work from the expensive path.

    A valid previously computed answer can satisfy a repeat request without recomputing it. That changes the work required; it is different from storing more work for later.

  15. Slower input helps only when the supplier responds.

    Backpressure communicates that the receiver cannot keep up. A cooperating supplier reduces the rate; otherwise merely sending a signal does not stop accumulation.

  16. Change the mismatch, not just the waiting room.

    Reduce work per result, reduce the work arriving, or increase capacity at the limiting component. A bounded queue remains useful only within the waiting and recovery conditions you have actually tested.

Content revision: 07c7dd00db17

Download review copy (27 lessons)
Check your understanding & apply it

Eighty requests are waiting. Arrivals drop to 40/s while the worker finishes 80/s. When does this queue clear?

Show the explained answer

After two seconds: the worker has 40/s left for the backlog, so 80 ÷ (80 − 40) = 2s. At 80 arrivals/s, the backlog would not shrink.

Integration guidance

Bound queue size and waiting age. Test both the expected burst and the quieter period needed to drain it; decide what happens when the queue is full.

Where the promise stops

Durable asynchronous work can be useful; a queue is not inherently a bad design.

Technical depth & sources

Concepts used here

Implementation detail

This constant-rate, equal-cost model starts with a busy worker and no refusals or cancellations. Fill time = free slots ÷ (arrivals − completions); drain time = backlog ÷ (completions − new arrivals), only when the denominator is positive. Moving waiting upstream does not remove its storage or age cost.

Original explanation inspired by Fred Hébert and operational references; no endorsement implied.

AI-generated narration: ElevenLabs Eleven v3, Daniel stock voice. Audio streams only when you start listening.

ElevenLabs

Why teams choose us

Happier customers
Consistent response times when it matters most.
Faster delivery
Ship features, not DIY limiters and brittle ops workarounds.
Clear controls
Per‑tenant and per‑route limits your stakeholders can reason about.

Engineering blog

View all posts →

Why you shouldn’t use Redis as a rate limiter: Part 1 of 2

A tour of the common Redis-based rate limiter implementations — and the correctness and performance traps each one hides.

Read more →

Auto-Scaling Won’t Save You

The myth of infinite serverless scale — why adding machines doesn’t fix overload, and what to do instead.

Read more →