Skip to main content
Ratelimitly logo
Ratelimitly

Keep your API fast—even at peak

Rate limiting and admission checks that help reduce overload and protect expensive work.

Request early access Customer Log in

Reliable at peak

Set admission budgets for launch spikes and promos before excess work reaches your handlers.

Lower cloud costs

Fewer moving parts to run and monitor. Spend less on Redis and ops.

Dev‑first adoption

Add via SDK. Keep your stack. Clear limits your customers understand.

Request early access

UNDERSTAND OVERLOAD · KEEP USEFUL WORK MOVING

Keep useful work moving.

For developers, architects, and managers: 27 narrated lessons on why services overload and how to protect them. No cloud experience required. Start with one request, or jump to any lesson.

Course map Lesson 22 of 27 Browse all lessons

Four chapters, one learning path. Read or listen in any order. ~41 minutes total · Times shown at 1×.

Keep enough capacity to finish work and recover.

  1. 01 A useful result starts with one request
  2. 02 Unfinished work grows when arrivals outrun completions
  3. 03 More waiting space does not make work finish faster
  4. 04 A timeout ends waiting, not necessarily the work
  5. 05 Retained work can exhaust a process’s memory
  6. 06 One failed copy can push its neighbors over capacity
  7. 07 Refuse new work while the application can still recover
  8. 08 Different compute platforms still have finite resources
  9. 09 A shared database failure can spread back into every app
  10. 10 Restore capacity before restoring full traffic
  11. 11 A circuit breaker stops repeating a failing call

Contain one customer's excess without punishing everyone.

  1. 12 One customer’s excess should not ruin another’s day
  2. 13 A rate limit checks one customer’s recent allowance
  3. 14 Copies and languages must share the same intended policy
  4. 15 Fairness helps contain damage without reserving every resource
  5. 16 A legitimate crowd can exceed shared capacity

Reduce work entering a shared dependency when it slows.

  1. 17 Measure the operation whose pressure you want to observe
  2. 18 A shared tracker observes; a guard decides; your app acts
  3. 19 Even the fastest recent observations can become slow
  4. 20 Separate local histories can reopen into the same burst
  5. 21 Replace optional expensive work before refusing essential work
  6. 22 A local breaker learns after calls have already piled up You are here
  7. 23 Fast errors explain why the protections remain complementary
  8. 24 Latency feedback helps without measuring exact spare capacity

Choose, combine, and test the right protections.

  1. 25 Ask three different questions before expensive work
  2. 26 Recovery attempts need limits too
  3. 27 Measure useful service, not how much work you accepted

Read the explanation or use the course map above. Audio is optional.

22 / 27

A local breaker learns after calls have already piled up

Failure-detection delay

Suggestions serviceApplication copiesFirst 30 s timeout is not known yetOpening does not erase in-flight callsCalls overlap while waitingCompleted timeout observations
  1. New calls STILL ADMITTED

    Basic timeout-only breaker

  2. In-flight calls WAITING · 30 s deadline

    Memory remains occupied

  3. Observed timeouts NONE YET

    Failure detection has a delay

Illustrative mechanism. Highlight follows the explanation; not a benchmark.

Read the numbered steps in order. The explanation below follows the narration.

At 0–30 s, overlapping callers can wait before the first timeout becomes observable.

Observe the detection delay: calls already in flight keep consuming resources before a basic timeout-only breaker opens.

0:00
Read transcript & diagram walkthrough

What you will learn

Observe the detection delay: calls already in flight keep consuming resources before a basic timeout-only breaker opens.

Now the suggestions service stops answering. Consider a basic breaker that counts only completed timeouts, each with a thirty-second deadline. For almost thirty seconds, calls can overlap before the first timeout is known. Their waiting state retains memory. Opening the breaker does not erase calls already in flight.

If two hundred application copies each require three observed failures, that is six hundred threshold observations, provided every copy reaches its threshold in the window. It is not a cap on failed calls. A new copy with no history can repeat discovery. A shared circuit breaker can reduce this duplication too.

Consider a different case: the dependency slows down but still answers successfully. A shared latency guard can refuse new work once its eligible signal reaches the configured threshold, before a failure-only breaker has a timeout to count. Modern slow-call breakers can also respond before complete failure.

RateLimitly's benefit is shared latency evidence for admission, not advance knowledge of every outage. If a sudden failure produces no fresh signal, do not assume immediate protection. Bound waiting and retained work, and show a cheap fallback when possible.

Diagram walkthrough
  1. At 0–30 s, overlapping callers can wait before the first timeout becomes observable.

    A failure-only detector cannot count an outcome that has not arrived. Many calls can overlap before the first 30-second timeout is observed.

  2. Independent discovery can repeat across copies; 200 × 3 is not a failed-call ceiling.

    Derived example: 200 copies × 3 observations = 600 threshold observations if every copy reaches its threshold. That is not a cap on total failed calls.

  3. Gradual slowdown can produce usable latency evidence while calls still return successfully.

    Gradual successful slowdown is different from a sudden silent outage. Latency-based admission can act on slow observations; modern slow-call breakers can too.

  4. Use shared evidence with bounded waiting and application-owned fallback.

    Without fresh evidence, do not assume immediate refusal. Bound waiting and retained work independently of the detector.

Content revision: 07c7dd00db17

Download review copy (27 lessons)
Check your understanding & apply it

Are six hundred threshold observations a maximum of six hundred failed requests?

Show the explained answer

No. The arithmetic assumes all two hundred copies reach three observed failures. Overlapping calls, new copies and probes can add more failures.

Integration guidance

Load-test abrupt stalls and gradual slowdown separately. Share the intended observation scope, bound concurrency and keep a fallback path.

Where the promise stops

Shared evidence has freshness and routing boundaries. Shared breakers exist; low-latency shared admission is not an instantaneous outage oracle.

Technical depth & sources

Concepts used here

Implementation detail

Detection delay, admitted concurrency and cancellation determine retained work before a trip. Test new replicas without history and delayed observations.

Original explanation inspired by Fred Hébert and operational references; no endorsement implied.

AI-generated narration: ElevenLabs Eleven v3, Daniel stock voice. Audio streams only when you start listening.

ElevenLabs

Why teams choose us

Happier customers
Consistent response times when it matters most.
Faster delivery
Ship features, not DIY limiters and brittle ops workarounds.
Clear controls
Per‑tenant and per‑route limits your stakeholders can reason about.

Engineering blog

View all posts →

Why you shouldn’t use Redis as a rate limiter: Part 1 of 2

A tour of the common Redis-based rate limiter implementations — and the correctness and performance traps each one hides.

Read more →

Auto-Scaling Won’t Save You

The myth of infinite serverless scale — why adding machines doesn’t fix overload, and what to do instead.

Read more →