Skip to main content

Keep your API fast—even at peak

Rate limiting and admission checks that help reduce overload and protect expensive work.

Reliable at peak

Set admission budgets for launch spikes and promos before excess work reaches your handlers.

Lower cloud costs

Fewer moving parts to run and monitor. Spend less on Redis and ops.

Dev‑first adoption

Add via SDK. Keep your stack. Clear limits your customers understand.

UNDERSTAND OVERLOAD · KEEP USEFUL WORK MOVING

Keep useful work moving.

For developers, architects, and managers: 27 narrated lessons on why services overload and how to protect them. No cloud experience required. Start with one request, or jump to any lesson.

Course map Lesson 12 of 27 Browse all lessons

Four chapters, one learning path. Read or listen in any order. ~41 minutes total · Times shown at 1×.

Keep enough capacity to finish work and recover.

  1. 01 A useful result starts with one request
  2. 02 Unfinished work grows when arrivals outrun completions
  3. 03 More waiting space does not make work finish faster
  4. 04 A timeout ends waiting, not necessarily the work
  5. 05 Retained work can exhaust a process’s memory
  6. 06 One failed copy can push its neighbors over capacity
  7. 07 Refuse new work while the application can still recover
  8. 08 Different compute platforms still have finite resources
  9. 09 A shared database failure can spread back into every app
  10. 10 Restore capacity before restoring full traffic
  11. 11 A circuit breaker stops repeating a failing call

Contain one customer's excess without punishing everyone.

  1. 12 One customer’s excess should not ruin another’s day You are here
  2. 13 A rate limit checks one customer’s recent allowance
  3. 14 Copies and languages must share the same intended policy
  4. 15 Fairness helps contain damage without reserving every resource
  5. 16 A legitimate crowd can exceed shared capacity

Reduce work entering a shared dependency when it slows.

  1. 17 Measure the operation whose pressure you want to observe
  2. 18 A shared tracker observes; a guard decides; your app acts
  3. 19 Even the fastest recent observations can become slow
  4. 20 Separate local histories can reopen into the same burst
  5. 21 Replace optional expensive work before refusing essential work
  6. 22 A local breaker learns after calls have already piled up
  7. 23 Fast errors explain why the protections remain complementary
  8. 24 Latency feedback helps without measuring exact spare capacity

Choose, combine, and test the right protections.

  1. 25 Ask three different questions before expensive work
  2. 26 Recovery attempts need limits too
  3. 27 Measure useful service, not how much work you accepted

Read the explanation or use the course map above. Audio is optional.

12 / 27

One customer’s excess should not ruin another’s day

Tenant-tagged competing work

OrchidOSeparate customerNorthstarNMaya’s reportShared capacitySame finite resourceO square: Orchid · N circle: NorthstarOrchidOSeparate customerNorthstarNMaya’s reportShared capacitySame finite resourceO square: Orchid · Ncircle: Northstar
  1. Northstar Maya’s organization

    Circle N.

  2. Orchid Another organization

    Square O.

  3. Service Shared capacity

    Both use the same finite resource.

Schematic request tokens, not quotas or measured traffic. Letters and shapes preserve identity. Both tenants use shared capacity, not reserved physical pools.

Read the numbered steps in order. The explanation below follows the narration.

Two organizations share one service.

Contain one customer's excess before it harms other customers.

0:00
Read transcript & diagram walkthrough

What you will learn

Contain one customer's excess before it harms other customers.

Return to Maya's report. Her organization, Northstar, shares this service with another organization, Orchid. A tenant is a customer or organization whose usage we want to account for separately.

Orchid accidentally starts a bulk job that sends many expensive requests. Maya sends one ordinary request, but both organizations compete for the same finite capacity. A noisy neighbor is one tenant whose activity makes the service worse for others. It need not be deliberate. Here, abuse means exceeding an intended share, not judging the customer's intentions.

We want to refuse Orchid's excess before it performs expensive work, while allowing Maya's reasonable use. Rejecting every customer equally would miss that goal. The application therefore needs a trustworthy tenant identity attached to each request.

If one bulk job makes a thousand other customers wait, adding a boundary around that job protects real people. Maya should not lose her meeting because somebody else's integration ran out of control.

Diagram walkthrough
  1. Maya’s organization shares the service with another customer.

    Northstar and Orchid are distinct organizations using one service. Follow Northstar’s circular N token and Orchid’s square O token; their identities remain readable without relying on color.

  2. A tenant is the customer boundary used for separate accounting.

    The two customer labels define which organization each request belongs to. The drawn work area is shared; identifying tenants does not automatically create separate servers, memory, or reserved execution slots.

  3. Orchid’s bulk job creates many expensive requests.

    Orchid’s tokens occupy the shared work area in this illustrative congested state. The number of drawn tokens is schematic, not a traffic rate, limit configuration, or measured processing capacity.

  4. Maya waits even though she sent only an ordinary request.

    The same shared work area is occupied by Orchid’s activity while Maya’s Northstar request waits. The customer harmed by this congestion is not necessarily the customer producing the excess demand.

  5. Protect other customers without assuming malicious intent.

    The congestion can come from a mistake in an integration rather than an intentional attack. The protection goal is to enforce intended use and preserve service for other tenants, not to infer the customer’s motives.

  6. Refuse Orchid’s excess before it enters expensive work.

    This is the desired protected alternative. Orchid’s excess is refused, while permitted Orchid work and Maya’s reasonable request can enter the shared work area. The request tokens do not specify a production quota or guarantee reserved capacity.

  7. Blanket refusal would deny Maya as well.

    This contrasting policy rejects both tenants. It avoids adding this work but misses the noisy-neighbor goal: allowing reasonable use while containing the organization producing the excess.

  8. Use a trustworthy tenant identity before making the decision.

    Attach the tenant identity from trusted application context. A caller-provided name alone is not a trustworthy boundary; the admission decision must account against the correct organization before expensive work begins.

  9. Contain one runaway job so other people can keep working.

    Return to the selective policy: refuse the excess rather than treating every customer as the problem. This protects useful service for people like Maya; it does not promise isolation from every possible shared-resource failure.

Content revision: 07c7dd00db17

Download review copy (27 lessons)
Check your understanding & apply it

Orchid’s bulk job is accidental. Does the fairness boundary still apply?

Show the explained answer

Yes. Resource impact, not intent, determines whether excess usage needs to be refused.

Integration guidance

Derive tenant identity from authenticated application context. Choose the intended scope: user, organization, or another explicit customer boundary.

Where the promise stops

An IP address is not always one customer: people may share one address, and a customer may use many.

Technical depth & sources

Concepts used here

Implementation detail

Do not let an untrusted request choose another tenant’s identity. Combine authenticated policy scope with independent resource bounds.

Original explanation inspired by Fred Hébert and operational references; no endorsement implied.

AI-generated narration: ElevenLabs Eleven v3, Daniel stock voice. Audio streams only when you start listening.

ElevenLabs

Why teams choose us

Happier customers
Consistent response times when it matters most.
Faster delivery
Ship features, not DIY limiters and brittle ops workarounds.
Clear controls
Per‑tenant and per‑route limits your stakeholders can reason about.

Engineering blog

View all posts →

Why you shouldn’t use Redis as a rate limiter: Part 1 of 2

A tour of the common Redis-based rate limiter implementations — and the correctness and performance traps each one hides.

Auto-Scaling Won’t Save You

The myth of infinite serverless scale — why adding machines doesn’t fix overload, and what to do instead.