Keep your API fast—even at peak

The tracker holds a shared record of measurements supplied by clients. Observing those durations is separate from making an admission decision or stopping application work.

The guard uses the tracker’s observations to decide whether new work may start. The measurement record and the admission check have different responsibilities; they are not interchangeable names for the same thing.

The threshold supplies the guard’s configured comparison boundary. It is distinct from the observed durations: observations describe measured work; the threshold is the policy applied to those observations.

Observations inform the guard, but application code must start or skip the protected operation. Merely recording a latency metric does not prevent work from reaching a database.

Only the first side call exists at this step: application to RateLimitly. The database query has not begun. Put this check before acquiring or consuming the resource the guard is intended to protect.

The comparison includes equality. Below this illustrative positive threshold, the latency guard allows on this signal; at 500 ms or above it refuses. Other required checks can still refuse an operation.

The zero setting is an explicit configuration exception. It disables this latency guard rather than refusing every nonzero-latency operation. It does not disable other required admission checks.

Follow the direct application-to-database call. RateLimitly is a separate admission side call, not a proxy carrying the query. The result and subsequent report have not been shown yet.

The result returns directly from the database to the application. Measure from the agreed start to finish; this is the duration of the protected work, not the duration of the admission side call.

The last arrow reports the measured duration from the application to RateLimitly. That observation can inform future admissions; it does not retroactively prevent or cancel the work just measured.

The refusal comes back to application code, which does not start that database operation. There is no query arrow to the database on this branch. Return a correct cheap fallback or an explicit refusal.

The library does not automatically rewrite an expensive request into a cheap alternative. Your application handles the answer and must not perform the same protected operation while pretending to fall back.

All three responsibilities are needed: an observation, a guard decision, and application code that obeys it. Recording durations while unconditionally running the expensive operation leaves that work unprotected.

Return to the first call in the sequence. A guard evaluated after a database operation cannot prevent that operation. Test the refused path by checking that the protected call was never made.