Skip to main content
Ratelimitly logo
Ratelimitly

Keep your API fast—even at peak

Rate limiting and admission checks that help reduce overload and protect expensive work.

Request early access Customer Log in

Reliable at peak

Set admission budgets for launch spikes and promos before excess work reaches your handlers.

Lower cloud costs

Fewer moving parts to run and monitor. Spend less on Redis and ops.

Dev‑first adoption

Add via SDK. Keep your stack. Clear limits your customers understand.

Request early access

UNDERSTAND OVERLOAD · KEEP USEFUL WORK MOVING

Keep useful work moving.

For developers, architects, and managers: 27 narrated lessons on why services overload and how to protect them. No cloud experience required. Start with one request, or jump to any lesson.

Course map Lesson 6 of 27 Browse all lessons

Four chapters, one learning path. Read or listen in any order. ~41 minutes total · Times shown at 1×.

Keep enough capacity to finish work and recover.

  1. 01 A useful result starts with one request
  2. 02 Unfinished work grows when arrivals outrun completions
  3. 03 More waiting space does not make work finish faster
  4. 04 A timeout ends waiting, not necessarily the work
  5. 05 Retained work can exhaust a process’s memory
  6. 06 One failed copy can push its neighbors over capacity You are here
  7. 07 Refuse new work while the application can still recover
  8. 08 Different compute platforms still have finite resources
  9. 09 A shared database failure can spread back into every app
  10. 10 Restore capacity before restoring full traffic
  11. 11 A circuit breaker stops repeating a failing call

Contain one customer's excess without punishing everyone.

  1. 12 One customer’s excess should not ruin another’s day
  2. 13 A rate limit checks one customer’s recent allowance
  3. 14 Copies and languages must share the same intended policy
  4. 15 Fairness helps contain damage without reserving every resource
  5. 16 A legitimate crowd can exceed shared capacity

Reduce work entering a shared dependency when it slows.

  1. 17 Measure the operation whose pressure you want to observe
  2. 18 A shared tracker observes; a guard decides; your app acts
  3. 19 Even the fastest recent observations can become slow
  4. 20 Separate local histories can reopen into the same burst
  5. 21 Replace optional expensive work before refusing essential work
  6. 22 A local breaker learns after calls have already piled up
  7. 23 Fast errors explain why the protections remain complementary
  8. 24 Latency feedback helps without measuring exact spare capacity

Choose, combine, and test the right protections.

  1. 25 Ask three different questions before expensive work
  2. 26 Recovery attempts need limits too
  3. 27 Measure useful service, not how much work you accepted

Read the explanation or use the course map above. Audio is optional.

6 / 27

One failed copy can push its neighbors over capacity

Replica capacity and redistribution

Three separate application copiesServer ASame service codeApplication copyReady to serveServer BSame service codeApplication copyReady to serveServer CSame service codeApplication copyReady to serveA, B and C serve the same applicationThree separate applicationcopiesServer AApplication copySame service codeReady to serveServer BApplication copySame service codeReady to serveServer CApplication copySame service codeReady to serveA, B and C serve the sameapplication
  1. Server A / B / C Separate application copies

    Each can serve a request.

  2. Replica A copy of the application

    Work does not have to pass through all three.

Illustrative equal-cost rates and even routing. Server positions stay fixed. Active arrows carry work; STOPPED supplies no capacity. Desktop gauges all use the same 0–240/s scale.

Read the numbered steps in order. The explanation below follows the narration.

A load balancer distributes work across application replicas.

240 requests/s ÷ 3 = 80 each; after one loss, 240 ÷ 2 = 120 each.

0:00
Read transcript & diagram walkthrough

What you will learn

240 requests/s ÷ 3 = 80 each; after one loss, 240 ÷ 2 = 120 each.

A service can run several copies of its application. Each copy is a replica. Here they are servers A, B, and C. A load balancer distributes requests between copies that can serve them. In our example, each can finish one hundred equal-cost requests per second.

With two hundred forty requests per second, divided evenly, each receives eighty. Now server A stops. Once the balancer detects that failure, it redirects those requests. Each survivor now receives one hundred twenty per second, but can still finish only one hundred. Demand did not increase. Available capacity decreased.

If the survivors retain all that work, each accumulates twenty extra unfinished requests every second. Waiting and memory can grow. If server B also stops, server C receives all two hundred forty requests per second. It cannot finish one hundred forty of those each second. If C then fails, no serving copies remain. This spread through the effects of an earlier failure is a cascading failure.

Is that chain inevitable? No. Enough spare capacity, or refusing excess work, can interrupt it. For example, with two survivors, admitting ninety requests per second to each and refusing sixty overall leaves room below their combined capacity. Those numbers illustrate the principle, not a safe production setting. Refusal itself costs resources. Test that remaining capacity can finish accepted work and respond when a replica disappears.

Diagram walkthrough
  1. These are three separate copies of the same application.

    A, B, and C are individual serving copies, not three stages in one request. Keeping their names and positions fixed lets us follow what happens to each one as capacity is lost.

  2. The load balancer chooses a serving copy.

    The branching arrows are alternative destinations for incoming requests. They do not mean that one request is copied to all three servers, and the load balancer does not increase each server’s processing speed.

  3. Each server has the same finite finishing capacity.

    In this worked example, each copy can finish 100 equal-cost requests per second. We have not assigned demand yet. Three healthy copies supply 300/s of combined capacity under these assumptions.

  4. 240 requests/s divided three ways is 80 per server.

    The same 240 requests per second are distributed evenly: A, B, and C each receive 80. Each is below its 100/s finishing capacity. The gauge scale stays fixed across every later step.

  5. A is still on the diagram, but contributes no capacity.

    A has stopped. B and C are still in the same positions. This intermediate picture separates the lost path from the following redistribution: 80/s previously directed to A must now be handled or fail.

  6. The same demand is routed onto two survivors.

    After detection, this example routes all 240/s to B and C. Real detection and routing can take time; the diagram is a causal sequence, not a timing benchmark or a guarantee of even balancing.

  7. 120 arrive, but only 100 can finish on each survivor.

    Both surviving servers cross the 100/s capacity marker. Demand has not increased; total usable capacity fell from 300/s to 200/s. More traffic reaches each remaining copy because there are fewer copies.

  8. Unfinished work grows by 20/s on each survivor.

    If each server keeps everything it cannot finish, 120 arriving minus 100 finishing leaves 20 extra unfinished requests every second on each. Their waiting work can retain memory; growth is conditional on that work not being bounded or removed.

  9. C now receives all 240/s alone.

    Follow the severe branch: B also stops and the whole demand is routed to C. C’s capacity has not grown: 240 arriving minus 100 finishing leaves 140/s of excess if it accepts and retains everything.

  10. No serving copies remain. Customers get no useful result.

    In this explicitly conditional severe outcome, the last application copy also fails. Incoming customer demand still exists, but there is no serving application capacity; failure has spread through the earlier losses.

  11. Return to the first loss: the chain can be interrupted.

    This is an alternative to the severe continuation, not an automatic restart. A is unavailable, while B and C still have a chance to survive if enough capacity remains or excess work is refused.

  12. Refuse some work so the accepted work can finish.

    An illustrative alternative admits 90/s to each survivor: 180/s accepted and 60/s refused out of the same 240/s offered. Both stay below the assumed 100/s finishing capacity; this is not a recommended production threshold.

  13. Containment needs measured room to respond.

    The refused requests still consume some network, routing, and response resources. Test the real request mix and loss behavior: the numerical margin here explains the idea but cannot prove a safe threshold for your deployment.

Content revision: 07c7dd00db17

Download review copy (27 lessons)
Check your understanding & apply it

Why can one failed replica put healthy survivors at risk?

Show the explained answer

Demand is redistributed onto fewer replicas. Each survivor can exceed its own capacity.

Integration guidance

Test replica loss at expected demand, including detection delay, redistribution, retained work, and the cost of refusing excess on survivors.

Where the promise stops

The example assumes even redistribution and no additional capacity. A failed replica need not cause a fleet outage.

Technical depth & sources

Concepts used here

Implementation detail

Aggregate healthy capacity is not necessarily evenly usable. Include uneven routing, expensive request mixes, and resources consumed by refusals. The 90/s admitted example is not a recommended production threshold.

Original explanation inspired by Fred Hébert and operational references; no endorsement implied.

AI-generated narration: ElevenLabs Eleven v3, Daniel stock voice. Audio streams only when you start listening.

ElevenLabs

Why teams choose us

Happier customers
Consistent response times when it matters most.
Faster delivery
Ship features, not DIY limiters and brittle ops workarounds.
Clear controls
Per‑tenant and per‑route limits your stakeholders can reason about.

Engineering blog

View all posts →

Why you shouldn’t use Redis as a rate limiter: Part 1 of 2

A tour of the common Redis-based rate limiter implementations — and the correctness and performance traps each one hides.

Read more →

Auto-Scaling Won’t Save You

The myth of infinite serverless scale — why adding machines doesn’t fix overload, and what to do instead.

Read more →