Skip to main content

Keep your API fast—even at peak

Rate limiting and admission checks that help reduce overload and protect expensive work.

Reliable at peak

Set admission budgets for launch spikes and promos before excess work reaches your handlers.

Lower cloud costs

Fewer moving parts to run and monitor. Spend less on Redis and ops.

Dev‑first adoption

Add via SDK. Keep your stack. Clear limits your customers understand.

UNDERSTAND OVERLOAD · KEEP USEFUL WORK MOVING

Keep useful work moving.

For developers, architects, and managers: 27 narrated lessons on why services overload and how to protect them. No cloud experience required. Start with one request, or jump to any lesson.

Course map Lesson 4 of 27 Browse all lessons

Four chapters, one learning path. Read or listen in any order. ~41 minutes total · Times shown at 1×.

Keep enough capacity to finish work and recover.

  1. 01 A useful result starts with one request
  2. 02 Unfinished work grows when arrivals outrun completions
  3. 03 More waiting space does not make work finish faster
  4. 04 A timeout ends waiting, not necessarily the work You are here
  5. 05 Retained work can exhaust a process’s memory
  6. 06 One failed copy can push its neighbors over capacity
  7. 07 Refuse new work while the application can still recover
  8. 08 Different compute platforms still have finite resources
  9. 09 A shared database failure can spread back into every app
  10. 10 Restore capacity before restoring full traffic
  11. 11 A circuit breaker stops repeating a failing call

Contain one customer's excess without punishing everyone.

  1. 12 One customer’s excess should not ruin another’s day
  2. 13 A rate limit checks one customer’s recent allowance
  3. 14 Copies and languages must share the same intended policy
  4. 15 Fairness helps contain damage without reserving every resource
  5. 16 A legitimate crowd can exceed shared capacity

Reduce work entering a shared dependency when it slows.

  1. 17 Measure the operation whose pressure you want to observe
  2. 18 A shared tracker observes; a guard decides; your app acts
  3. 19 Even the fastest recent observations can become slow
  4. 20 Separate local histories can reopen into the same burst
  5. 21 Replace optional expensive work before refusing essential work
  6. 22 A local breaker learns after calls have already piled up
  7. 23 Fast errors explain why the protections remain complementary
  8. 24 Latency feedback helps without measuring exact spare capacity

Choose, combine, and test the right protections.

  1. 25 Ask three different questions before expensive work
  2. 26 Recovery attempts need limits too
  3. 27 Measure useful service, not how much work you accepted

Read the explanation or use the course map above. Audio is optional.

4 / 27

A timeout ends waiting, not necessarily the work

Caller clock and remote outcomes

Caller’s waiting budgetMaya / callerWaiting for the reportA finite time to waitDeadline: when the caller stops waitingCaller’s waiting budgetMaya / callerWaiting for the reportA finite time to waitDeadline: when the callerstops waiting
  1. Maya Waiting for a report

    Not a completed request.

  2. Waiting budget Finite

    The caller needs a bound on this wait.

Narrated snapshots of a 30-second caller budget, not a database execution timer. Cancellation must propagate and be honored. Unknown outcomes are alternatives, not simultaneous facts.

Read the numbered steps in order. The explanation below follows the narration.

The caller has a chosen 30-second deadline.

Timeout, cancellation, connection failure, and operation outcome are different facts.

0:00
Read transcript & diagram walkthrough

What you will learn

Timeout, cancellation, connection failure, and operation outcome are different facts.

Maya cannot wait forever. Her caller has a deadline: a point after which it stops waiting for an answer. In this example that deadline is thirty seconds. A timeout is the caller reaching that deadline without the expected response.

At five seconds, Maya has twenty-five seconds left to wait. The interface must still look unfinished. At thirty seconds, it shows a timeout, not a successful report and not an error message supposedly sent by the database.

The database may still be doing the work. Cancellation is a separate instruction asking that work to stop, and each part of the system must carry it through and honor it. A connection is the communication link used by two components. If that link fails, an application may learn about the failure immediately rather than waiting thirty seconds.

Does Maya timing out prove that a database change did not happen? No. A caller stopping its wait does not establish whether the operation stopped, completed, or changed stored data.

Diagram walkthrough
  1. Maya needs a result within a finite wait.

    Follow the caller separately from the database. Waiting for a result consumes time, but the caller’s decision to stop waiting does not by itself establish what happened to remote work.

  2. Start with the full 30-second waiting budget.

    At this starting snapshot, zero seconds have elapsed and thirty seconds remain. The bar represents the caller’s elapsed waiting time, not percent completion of a database operation.

  3. A timeout means no expected response before the deadline.

    This defines the event we will show next. The request is still waiting in the starting snapshot; when its budget is exhausted without the expected answer, the caller times out.

  4. Five seconds have passed; Maya still has 25 seconds to wait.

    The caller remains visibly pending. Only five of the thirty seconds have elapsed, leaving twenty-five. Nothing in this snapshot says a report was delivered or that the database finished.

  5. The caller’s budget reaches zero: TIMEOUT, no report.

    The failure is shown on Maya’s caller, not as an error response supposedly sent by the database. The caller reached thirty seconds without its expected report; this is not a successful completion.

  6. The caller stops waiting; remote work may continue.

    The separate database track remains unresolved. A caller timing out does not automatically cancel the database operation, stop its resource use, or establish whether a write took effect.

  7. Cancellation must travel to the work and be honored.

    This is a separate signal asking the operation to stop, not an automatic consequence of the earlier timeout. Each component must propagate or handle cancellation and the downstream operation must actually honor it.

  8. The communication link can fail separately from the deadline.

    The application and database exchange calls and answers over a connection. A connection’s condition is a different fact from the caller’s waiting budget or the database operation’s final outcome.

  9. Some failures are learned about before the timeout.

    A broken connection may report an error immediately, without consuming the full thirty-second waiting budget. A fast communication error still does not automatically prove that remote work had no effect.

  10. A timeout leaves the remote operation’s outcome uncertain.

    No. The operation may still be running, may have stopped, or may have completed without its answer reaching the caller. A timeout does not prove that stored data stayed unchanged, or that a write was rolled back.

Content revision: 07c7dd00db17

Download review copy (27 lessons)
Check your understanding & apply it

Maya times out. Can the database still be working?

Show the explained answer

Yes. Cancellation must reach the operation and be honored; the timeout alone does not stop it.

Integration guidance

Propagate deadlines and cancellation where supported. Track caller and downstream outcomes separately.

Where the promise stops

Thirty seconds is this example’s chosen deadline, not a universal default or required setting.

Technical depth & sources

Concepts used here

Implementation detail

For writes, an unknown response may conceal a committed side effect. Do not label a timeout as proof of rollback.

Original explanation inspired by Fred Hébert and operational references; no endorsement implied.

AI-generated narration: ElevenLabs Eleven v3, Daniel stock voice. Audio streams only when you start listening.

ElevenLabs

Why teams choose us

Happier customers
Consistent response times when it matters most.
Faster delivery
Ship features, not DIY limiters and brittle ops workarounds.
Clear controls
Per‑tenant and per‑route limits your stakeholders can reason about.

Engineering blog

View all posts →

Why you shouldn’t use Redis as a rate limiter: Part 1 of 2

A tour of the common Redis-based rate limiter implementations — and the correctness and performance traps each one hides.

Auto-Scaling Won’t Save You

The myth of infinite serverless scale — why adding machines doesn’t fix overload, and what to do instead.