Skip to main content
Ratelimitly logo
Ratelimitly

Keep your API fast—even at peak

Rate limiting and admission checks that help reduce overload and protect expensive work.

Request early access Customer Log in

Reliable at peak

Set admission budgets for launch spikes and promos before excess work reaches your handlers.

Lower cloud costs

Fewer moving parts to run and monitor. Spend less on Redis and ops.

Dev‑first adoption

Add via SDK. Keep your stack. Clear limits your customers understand.

Request early access

UNDERSTAND OVERLOAD · KEEP USEFUL WORK MOVING

Keep useful work moving.

For developers, architects, and managers: 27 narrated lessons on why services overload and how to protect them. No cloud experience required. Start with one request, or jump to any lesson.

Course map Lesson 23 of 27 Browse all lessons

Four chapters, one learning path. Read or listen in any order. ~41 minutes total · Times shown at 1×.

Keep enough capacity to finish work and recover.

  1. 01 A useful result starts with one request
  2. 02 Unfinished work grows when arrivals outrun completions
  3. 03 More waiting space does not make work finish faster
  4. 04 A timeout ends waiting, not necessarily the work
  5. 05 Retained work can exhaust a process’s memory
  6. 06 One failed copy can push its neighbors over capacity
  7. 07 Refuse new work while the application can still recover
  8. 08 Different compute platforms still have finite resources
  9. 09 A shared database failure can spread back into every app
  10. 10 Restore capacity before restoring full traffic
  11. 11 A circuit breaker stops repeating a failing call

Contain one customer's excess without punishing everyone.

  1. 12 One customer’s excess should not ruin another’s day
  2. 13 A rate limit checks one customer’s recent allowance
  3. 14 Copies and languages must share the same intended policy
  4. 15 Fairness helps contain damage without reserving every resource
  5. 16 A legitimate crowd can exceed shared capacity

Reduce work entering a shared dependency when it slows.

  1. 17 Measure the operation whose pressure you want to observe
  2. 18 A shared tracker observes; a guard decides; your app acts
  3. 19 Even the fastest recent observations can become slow
  4. 20 Separate local histories can reopen into the same burst
  5. 21 Replace optional expensive work before refusing essential work
  6. 22 A local breaker learns after calls have already piled up
  7. 23 Fast errors explain why the protections remain complementary You are here
  8. 24 Latency feedback helps without measuring exact spare capacity

Choose, combine, and test the right protections.

  1. 25 Ask three different questions before expensive work
  2. 26 Recovery attempts need limits too
  3. 27 Measure useful service, not how much work you accepted

Read the explanation or use the course map above. Audio is optional.

23 / 27

Fast errors explain why the protections remain complementary

One call, two observations

DurationFastOutcomeFAILED responseDuration and outcome are different facts.DurationFastOutcomeFAILED responseDuration and outcome aredifferent facts.
  1. Duration Fast

    Time taken.

  2. Outcome FAILED response

    Not a usable suggestion.

5 ms and 500 ms are illustrative, not recommended settings. Duration and failure classification are separate signals; refusal decisions occur before later work.

Read the numbered steps in order. The explanation below follows the narration.

A quick answer can still be an error: measure duration and useful outcome separately.

Low latency does not prove success. Rate limits, latency guards, error-aware breakers and local bounds observe different problems.

0:00
Read transcript & diagram walkthrough

What you will learn

Low latency does not prove success. Rate limits, latency guards, error-aware breakers and local bounds observe different problems.

A service can fail quickly. Suppose the suggestions service returns an error in five milliseconds. That is a failed result with low latency, not an overloaded queue we have measured.

With a five-hundred-millisecond guard, a five-millisecond observation is below the threshold. RateLimitly's latency report carries a duration, not an HTTP success or failure classification. A latency guard alone therefore does not replace an error-aware circuit breaker.

An error-aware breaker can count those failures and open. Half-open can later restrict recovery probes. By contrast, insufficient or expired tracker history produces a zero signal in the current implementation. That is not proof of health, and it is not the same as a bounded recovery probe.

Choose protection by the question it answers. Is one tenant taking too much? Use its rate limit. Is shared work getting too slow? Use shared latency admission. Are operations failing? Consider an error-aware breaker. Is this application retaining too much work? Bound its local resources. Your application chooses the cheap alternative, whichever protection declines the expensive operation.

Diagram walkthrough
  1. Fast is not the same as useful.

    A call has both a duration and an outcome. Looking at only the duration loses the fact that the customer did not receive a usable result.

  2. The same failed call returned in only 5 ms.

    Keep these two facts attached to the same call. The short duration does not turn its failed result into success or prove that a queue is overloaded.

  3. 5 ms passes this duration comparison, even though the call failed.

    Five is below five hundred. This latency check alone may admit later work. The report protocol carries duration, not an HTTP success/failure classification.

  4. An error-aware breaker can act on the failure signal.

    Opening depends on its configured criteria and sufficient observations, not one universal number of failures. Low duration does not erase classified errors.

  5. A half-open breaker deliberately limits recovery probes.

    After the configured open wait, a bounded number of test calls can supply recovery evidence. Waiting alone does not repair the dependency.

  6. No eligible tracker history is a different case, not a recovery test.

    The zero signal is optimistic behavior when insufficient history qualifies. It is not a measured five-millisecond result and not proof of health or a bounded half-open probe.

  7. Choose the signal for the problem, then implement the fallback.

    Tenant fairness, shared latency, classified errors, and local retained work answer different questions. Application code chooses a correct cheap alternative when protected work is declined.

Content revision: 07c7dd00db17

Download review copy (27 lessons)
Check your understanding & apply it

Will a 500 ms latency guard necessarily reject a service returning errors in 5 ms?

Show the explained answer

No. Five milliseconds is below that latency threshold. Classify errors separately; keep an error-aware breaker when the failure pattern needs one.

Integration guidance

Record both duration and outcome without disguising an error as fabricated latency. Configure bounded recovery and cheap alternatives explicitly.

Where the promise stops

RateLimitly does not claim every circuit-breaker feature. Shared state and graceful fallback are possible with other designs too.

Technical depth & sources

Concepts used here

Implementation detail

The current latency-report protocol contains duration but no outcome field; an insufficient-history minimum of zero is optimistic admission, not a half-open state machine.

Original explanation inspired by Fred Hébert and operational references; no endorsement implied.

AI-generated narration: ElevenLabs Eleven v3, Daniel stock voice. Audio streams only when you start listening.

ElevenLabs

Why teams choose us

Happier customers
Consistent response times when it matters most.
Faster delivery
Ship features, not DIY limiters and brittle ops workarounds.
Clear controls
Per‑tenant and per‑route limits your stakeholders can reason about.

Engineering blog

View all posts →

Why you shouldn’t use Redis as a rate limiter: Part 1 of 2

A tour of the common Redis-based rate limiter implementations — and the correctness and performance traps each one hides.

Read more →

Auto-Scaling Won’t Save You

The myth of infinite serverless scale — why adding machines doesn’t fix overload, and what to do instead.

Read more →