Skip to main content
Ratelimitly logo
Ratelimitly

r-client specification

Edit on GitHub →

The normative specification every RateLimitly client implements. Client libraries and their API references live on the docs index.

On this page

r-client Specification

1. Overview

This document specifies the behavior of an r-client, a client designed to communicate with an r-server (ratelimitly-server) for distributed rate limiting and load shedding. The specification is language-agnostic and provides a blueprint for developing r-client implementations in various programming languages.

The r-client communicates with the r-server over UDP, sending requests and receiving responses according to the Ratelimitly Wire Protocol.

2. Protocol Basics

2.1. Transport

All communication between the r-client and r-server occurs over UDP. Each message (request or response) is sent as a single UDP datagram.

2.2. Message Structure

Messages are composed of a sequence of records:

  1. Tenant Header (TLV): Identifies the tenant and contains request metadata.
  2. Auth Header (TLV): Contains authentication information.
  3. Protocol Data Unit (PDU): Represents the specific operation (e.g., rate request).

All multi-byte integer values are encoded in little-endian format.

2.3. Data Alignment

The protocol is designed for high performance and zero-copy parsing. Data structures are aligned to 4 or 8-byte boundaries. Implementations should be mindful of this, especially when creating and parsing packets.

3. Connection Management

3.1. Server Discovery

The r-client discovers server instances through DNS resolution of a tenant-specific domain name. The discovery process is as follows:

  1. SRV Records: The client resolves _ratelimitly._udp.<tenant_dns_name>. The SRV records provide the authoritative list of server target hostnames and ports.
  2. Target Address Resolution: For each SRV target hostname returned, the client resolves that target via A/AAAA to obtain usable socket addresses.
  3. Explicit Configuration: The client can be explicitly configured with a static list of server addresses, bypassing DNS discovery.

DNS SRV Setup

To enable SRV discovery, publish SRV records under _ratelimitly._udp.<tenant_dns_name>. Each SRV record points at a target hostname and port; publish A/AAAA records for those targets. Direct A/AAAA records on <tenant_dns_name> do not replace the SRV layer.

The first DNS label of each SRV target hostname MUST encode the wire-protocol server_id in decimal as s-<server_id_decimal>. The server_id extracted from DNS MUST equal the server_id later returned by that server in the response Tenant Header.key_id.

Example target form:

s-1015809.rl1.example.com.

Example (BIND-style):

; tenant_dns_name = ratelimitly.example.com
_ratelimitly._udp.ratelimitly.example.com. 60 IN SRV 0 0 29292 s-1015809.rl1.example.com.
_ratelimitly._udp.ratelimitly.example.com. 60 IN SRV 0 0 29293 s-1048578.rl2.example.com.
s-1015809.rl1.example.com. 60 IN A 203.0.113.10
s-1048578.rl2.example.com. 60 IN A 203.0.113.11

Provider examples (fields and names may vary by UI):

Route 53 (SRV record):

  • Service: _ratelimitly
  • Protocol: _udp
  • Name: ratelimitly.example.com
  • Priority/Weight/Port/Target: 0 / 0 / 29292 / s-1015809.rl1.example.com

Cloudflare (SRV record):

  • Type: SRV
  • Name: _ratelimitly._udp
  • Priority/Weight/Port/Target: 0 / 0 / 29292 / s-1015809.rl1.example.com

GCP Cloud DNS (SRV record):

  • Name: _ratelimitly._udp.ratelimitly.example.com.
  • Priority/Weight/Port/Target: 0 / 0 / 29292 / s-1015809.rl1.example.com.

Namecheap (SRV record):

  • Service: _ratelimitly
  • Protocol: _udp
  • Host: ratelimitly.example.com
  • Priority/Weight/Port/Target: 0 / 0 / 29292 / s-1015809.rl1.example.com

3.2. Communication

The client sends UDP packets to one or more discovered server addresses. For operations that expect a response (like a rate request), the client should be prepared to receive a response from any of the servers it sent the request to.

4. Authentication

The r-client supports three authentication methods, specified in the Auth Header.

4.1. No Authentication (None)

  • tlv_type: 0x414E
  • No authentication is performed. This is suitable for trusted environments or testing.
  • tlv_type: 0x4143
  • The client sends a 32-byte cookie hash as raw binary data.
  • Cookie credentials are provisioned as a Bech32 key (rl-cookie...) that embeds:
    • auth_method = cookie
    • key_id (tenant ID)
    • 32-byte cookie hash payload

4.3. AES-256-GCM Authentication

  • tlv_type: 0x4541
  • This is the most secure method, providing authenticated encryption.
  • The PDU is encrypted using AES-256-GCM.
  • The Auth Header contains a 12-byte nonce and a 16-byte authentication tag.
  • Encryption scope is the PDU bytes only; Tenant TLV and Auth TLV remain plaintext.
  • AAD is tenant_tlv || auth_tlv_prefix, where auth_tlv_prefix = tlv_type || tlv_size || nonce.
  • Nonce MUST be unique per AES key.
  • AES credentials are provisioned as a Bech32 key (rl-aes...) that embeds:
    • auth_method = aes
    • key_id (tenant ID)
    • 32-byte AES key payload

5. Data Structures

The following are language-agnostic definitions of the core data structures used in the protocol.

5.1. Tenant Header

Field Size (bytes) Type Description
tlv_type 2 Integer 0x4C52 (“RL”)
tlv_size 2 Integer 40
key_id 8 Integer Tenant identifier.
unique_id 16 Binary Unique request ID (e.g., UUIDv4).
time_stamp 8 Integer Milliseconds since UNIX epoch.
steering_feedback 1 Boolean 0 = change port, 1 = keep port.
tenant_mgmt_flag 1 Boolean 0 = regular operation, 1 = admin. Client is 0.
padding 2 Binary MUST be zero.

5.2. Guard Block (Request)

Field Size (bytes) Type Description
latency_tracker_id 16 Binary Canonical content-defined latency-tracker ID.
ttl_ms 4 Integer Time-to-live for latency samples.
max_samples 4 Integer Max samples for latency tracking.
buffer_size 4 Integer Circular buffer size for latency tracking.
min_sample_threshold 4 Integer Minimum insertion rate for reliability.
latency_threshold 4 Integer Maximum acceptable latency in ms.
current_latency 4 Integer MUST be 0 in requests.

Before building a guard block, a conforming client MUST derive latency_tracker_id from the application-defined latency-tracker name, ttl_ms, max_samples, buffer_size, and min_sample_threshold by the canonical algorithm in wire_protocol.md.

A latency-tracker name identifies one logical latency signal within the tenant. It is not an r-server name or necessarily a DNS or network-service name. Guards and reports intended to share observations MUST use the same name and final effective tracker configuration.

latency_threshold is a query over the tracker and is not part of its identity. Different requests may therefore use different thresholds against the same tracker.

5.3. Resource Block (Request)

Field Size (bytes) Type Description
bucket_id 16 Binary Canonical content-defined resource identifier.
window_size_ms 4 u32 Rate-limiting window in milliseconds.
rate_limit 4 u32 Maximum tokens allowed in the window.
tokens_requested 2 Integer Number of tokens requested.
padding 2 Integer MUST be zero.

Before building a resource block, a conforming client MUST derive bucket_id from the logical bucket name, window_size_ms, and rate_limit by the canonical algorithm in wire_protocol.md. This makes each resource definition address independent tenant-local rate state without prior server-side registration.

tokens_requested is operation-specific consumption and is not part of the resource identity. Clients that intend to share rate state MUST use the same logical bucket name, window, and limit consistently. The response reuses rate_limit for actual consumption; it is not a new configured limit and MUST NOT be used to derive a future bucket_id.

An r-server uses the received bucket_id directly. While the corresponding state is live, it verifies that the accompanying window and limit match the values that established the state. A mismatch is a malformed request, not a rate denial: it consumes no tokens, produces no response, and is observed by the client as request failure.

5.4. Service Latency Block (Request)

The service latency block is 36 bytes on the wire.

Field Size (bytes) Type Description
latency_tracker_id 16 Binary Canonical content-defined latency-tracker ID.
ttl_ms 4 Integer Time-to-live for latency samples.
max_samples 4 Integer Max samples for latency tracking.
buffer_size 4 Integer Circular buffer size for latency tracking.
min_sample_threshold 4 Integer Minimum insertion rate for reliability.
observed_latency 4 Integer Client-observed latency in ms.

The report MUST derive latency_tracker_id from the same name and tracker configuration used by its corresponding guards. observed_latency is an individual sample and is not part of the identity.

An r-server uses the received latency_tracker_id directly. While the corresponding state is live, it verifies that all four tracker configuration values match the definition that established the state. A mismatch makes the containing PDU malformed and causes no tracker or resource-state mutation.

This section describes one recommended client-side usage pattern. It is not a wire-protocol requirement, and clients in Rust, C, Java, and JavaScript may choose other policies while preserving the semantics defined by wire_protocol.md.

The state machine below applies to both single-server and multi-server membership snapshots. Section 9 defines the parameterized fan-out, replay, and response-selection policy implemented by the MVP C client. Its design history and remaining study questions are documented in r-client-ha-design.md.

6.1. Transport states

The transport state is:

stateDiagram-v2
    [*] --> Uninitialized
    Uninitialized --> Discovering: start / DNS discovery required
    Uninitialized --> Ready: start / explicit endpoints usable
    Discovering --> Ready: usable endpoint set installed
    Discovering --> Unavailable: discovery failure
    Unavailable --> Discovering: retry timer
    Ready --> RebindPending: steering keep_port=false
    RebindPending --> Rebinding: no conflicting in-flight rate send
    Rebinding --> Ready: replacement socket registered
    Rebinding --> RebindPending: replacement failed
    Ready --> Closing: close
    Discovering --> Closing: close
    Unavailable --> Closing: close
    RebindPending --> Closing: close
    Rebinding --> Closing: close
    Closing --> Closed: socket, timers, and discovery released
    Closed --> [*]

Transport states have these meanings:

State Meaning
Uninitialized No endpoint set or client socket has been installed.
Discovering DNS or explicit-endpoint validation is in progress. No operation may be reported as successful solely because discovery has started.
Ready At least one usable server address and a usable UDP source socket exist.
Unavailable No usable endpoint currently exists; the DNS refresh configuration may schedule discovery.
RebindPending A trusted response requested keep_port = false; the current socket remains usable until replacement succeeds.
Rebinding A replacement socket is being opened and registered transactionally.
Closing New operations are rejected and existing operations are completed according to the API’s cancellation policy.
Closed All sockets, timers, callbacks, and owned operation state have been released.

The following transport properties are recommended for this usage pattern:

  • A failed discovery or rebind should not destroy a currently usable endpoint.
  • A replacement socket should be opened and registered before the old socket is closed.
  • A rebind should wait for the safety condition defined by the host integration (normally no current in-flight rate send and no active receive callback). It should not wait for a later latency report.
  • After keep_port = false, subsequent sends should use the replacement source port as soon as the safety condition permits. A reentrant send that is already executing MAY use the old port.
  • Closing should be idempotent. No callback or timer should start a new operation after Closing begins.

6.2. Logical operation states

Each resource request operation has its own immutable operation_id (the wire Tenant Header.unique_id), membership snapshot, logical request contents, current transmission round, absolute deadlines, response set, selected candidate, and terminal outcome. The operation state is independent of the UDP socket and survives a transactional source-port replacement.

stateDiagram-v2
    [*] --> Created
    Created --> Prepared: validate and encode
    Created --> Terminal: validation failure
    Prepared --> Terminal: no usable endpoint
    Prepared --> TransmissionRound: initial fan-out
    TransmissionRound --> Terminal: oldest response or preference selection
    TransmissionRound --> TransmissionRound: silent deadline / replay remains
    TransmissionRound --> FinalReceive: last round silent / final interval enabled
    TransmissionRound --> Terminal: last round silent / no final interval
    FinalReceive --> Terminal: selected response or final timeout
    TransmissionRound --> Cancelled: caller cancellation / owner teardown
    FinalReceive --> Cancelled: caller cancellation / owner teardown
    Terminal --> [*]
    Cancelled --> [*]

An attempted wire transmission is not an admission decision. A rate request is not successful until the policy selects a valid response. Latency reporting uses the separate fire-and-forget transitions in Section 6.2.2.

6.2.1. Rate-request transition table

Current state Event Required guard/action Next state
Created API call Allocate one operation ID, snapshot the current server membership, and capture immutable request inputs. Prepared
Created Invalid input Return a local validation error; send nothing. Terminal
Prepared Endpoint unavailable Apply the configured failure policy; do not invent an allow response. Terminal
Prepared Initial send succeeds Send the same logical request to every server in the membership snapshot and arm the first absolute deadline. TransmissionRound
TransmissionRound Valid correlated response Record the responding server and update the oldest candidate. Select immediately if it is the globally oldest server or the preference deadline has elapsed; otherwise re-arm to the preference deadline. Terminal(Allowed/Denied) or TransmissionRound
TransmissionRound Invalid or unrelated datagram Do not add it to the response set or candidate; remain able to receive a later valid response. TransmissionRound
TransmissionRound Preference deadline with candidate Select the oldest valid candidate. Terminal(Allowed/Denied)
TransmissionRound Preference deadline without candidate Continue waiting until the round replay deadline. TransmissionRound
TransmissionRound Replay deadline without candidate and replay remains Reuse the same operation ID and replay only to snapshot servers from which no valid response has arrived. TransmissionRound
TransmissionRound Last replay deadline without candidate Enter the configured final receive-only interval, or report timeout when it is disabled. FinalReceive or Terminal(Failed)
FinalReceive Valid correlated response Apply the final preference rule; with a zero final preference, select the first valid response immediately. Terminal(Allowed/Denied) or FinalReceive
FinalReceive Final deadline without candidate Report timeout. Terminal(Failed)
TransmissionRound or FinalReceive Cancellation/close Suppress later completion and release operation ownership exactly once. Cancelled

An accepted response is one that passes authentication, framing, correlation, and response-semantic checks. A response that fails a check is not a denial; it is invalid and must not complete the operation. Section 9.3 adds the trusted server-identity and response-selection checks.

The terminal rate outcomes are:

  • Allowed: every guard and resource condition grants the request;
  • Denied: the server returned a valid response whose guard or resource conditions reject the request;
  • Failed: the client could not obtain a valid response or local validation failed; the host integration applies its fail-open/fail-closed policy; and
  • Cancelled: the caller or owning host request ended before a terminal server decision was accepted.

Only Allowed and Denied are server decisions. A client should keep Failed and Cancelled distinguishable internally even if a host integration maps both to the same HTTP or application result.

6.2.2. Latency-report transition table

Current state Event Required guard/action Next state
Created Latency-report API call Validate and build a report with the observed latency and service blocks. No resource request is required. Prepared
Created Invalid input Return a local validation error; send nothing. Terminal
Prepared Endpoint unavailable Record a local send failure; do not affect the completed rate outcome. Terminal
Prepared Datagrams transmitted Send once to every server currently known to the client. Reported
Reported Any later event No response is expected and no rate operation is reopened. Reported

Latency reporting is best-effort, fire-and-forget, and independent of resource requests. A client may send only latency reports and never issue a resource request. A higher-level admission workflow may choose to report the protected work only after an allow decision, but that coupling belongs to the workflow, not to the latency-report protocol or core client operation.

6.3. Event ordering and exactly-once completion

An implementation should serialize state transitions for one logical operation, even when callbacks arrive from different threads or event-loop queues. The following ordering rules are recommended:

  1. Validate and correlate a datagram before changing operation state.
  2. Mark a rate operation terminal before invoking user/application completion.
  3. Re-arm the host timer from the request’s current absolute deadline after every nonterminal datagram or timeout transition. A valid non-oldest response can move that deadline earlier to the preference deadline.
  4. Disarm or invalidate the deadline before terminal completion; a later timer callback should be harmless.
  5. Invalidate operation ownership before cancellation or host-request teardown; a late UDP response should be ignored.
  6. Schedule source-port replacement from steering feedback, but do not perform replacement reentrantly inside response parsing or a receive callback.
  7. A latency report MAY be queued after rate completion, but its state and failure should remain independent of the rate operation.

A duplicate valid response for an already-terminal operation should be ignored. An implementation should not invoke the application completion callback twice, send a second latency report for one admitted operation, or turn a late valid response into a new operation.

6.4. Error taxonomy

The protocol state machine distinguishes these classes:

Error class Examples State-machine consequence
Validation malformed local input, impossible size, unsupported credential Created -> Terminal; no datagram
Discovery missing SRV, unusable target address, DNS timeout transport Unavailable; refresh according to persistent DNS configuration
Transport send failure, receive error, endpoint loss Current transmission fails locally or remains pending according to the host integration.
Timeout preference, replay, or final deadline elapsed Select a candidate, start the next replay round, enter final receive, or finish Terminal(Failed) according to Section 9.3.
Authentication failed credential validation or decryption ignore datagram; the current transmission or final-receive state remains pending
Correlation wrong tenant, operation ID, PDU kind, or server trust ignore datagram; the current transmission or final-receive state remains pending
Semantic valid response with denied guard/resource condition rate operation Terminal(Denied)
Cancellation caller abort, host teardown, client close rate operation Cancelled; suppress late completion

The API may expose a richer language-specific error type, while preserving the distinctions needed to apply the transitions and terminal-outcome rules above.

7. Client Operations

7.1. Rate Limiting (checkRateLimit)

This operation checks if a request is allowed based on defined guards and resource limits.

7.1.1. Request (PDU_RATE_REQUEST - 0x5452)

  • The client constructs a rate_request PDU containing:
    • A list of GuardBlocks (preconditions).
    • A list of ResourceBlocks (token requests).
    • An optional metrics_label TLV.
  • If tenant metrics label cardinality is exhausted server-side, the label may be internally rewritten to the fixed overflow label overflow.
  • The PDU is wrapped in Tenant and Auth headers.
  • The client sends the UDP datagram to the server(s) and waits for a response.

7.1.2. Response (PDU_RATE_RESPONSE - 0x5252)

  • The server sends back a rate_response PDU.
  • The client parses the response to determine the outcome.
  • Allowed: In state-machine terms, the request reaches Allowed if all of the following are true:
    • For every GuardBlock, current_latency < latency_threshold.
    • For every ResourceBlock, tokens_requested (reused as deficit in response) is 0.
  • Denied: If any guard fails or any resource has a non-zero deficit, the entire request reaches Denied and is considered rejected.
  • Quota Enforcement: Tenant quota breaches are handled atomically as request failures, except metrics label cardinality overflow, which uses label rewrite to overflow instead of hard-failing the request. max_samples is not a tenant-quota-enforced field.

7.2. Latency Reporting (reportLatency)

This is a “fire-and-forget” operation to send client-observed latency metrics to the server.

7.2.1. Request (PDU_LATENCY_REPORT - 0x524C)

  • The client constructs a latency_report PDU containing a list of ServiceLatencyBlocks.
  • The PDU is wrapped in Tenant and Auth headers.
  • The client sends the UDP datagram to the server(s).
  • No response is expected for this operation.

8. Error Handling

An r-client implementation should be robust against network issues and protocol errors.

  • Timeouts: The client should expose the next absolute policy deadline to the host. A deadline event selects an available candidate, starts the next configured replay round, enters the final receive-only interval, or ends in Terminal(Failed) as specified in Section 9.3.
  • Protocol Errors: The client should be able to gracefully handle malformed responses from the server.
  • Authentication Errors: If authentication fails, the server drops the request and sends no response (blackhole behavior). This manifests as a timeout on the client side.
  • Quota-related Rejections: When tenant quotas are exceeded for buckets/services/latency buffer size, responses should be treated as normal request rejections under the existing success/failure rules. max_samples remains application-managed and is not quota-rejected.

9. High Availability

The r-client is designed to work in a high-availability environment with multiple r-server instances.

  • Request Broadcasting: The client should send the same request (with the same unique_id) to all discovered server instances.
  • Independent Servers: The r-servers do not coordinate and do not share a commit authority. Deduplication is server-local; convergence comes from clients delivering the same logical operation to every server.
  • Membership Snapshot: A resource request should use one immutable snapshot of discovered server identities. Asynchronously discovered membership changes apply to later requests.
  • Trusted Identity: The client MUST compare the full 64-bit server_id for server identity, trust decisions, response tracking, and mismatch detection.
  • Age Ordering: The client decodes server_start_s from server_id, prefers lower startup times, and uses the lower full server_id as the deterministic tie-break.
  • Response Acceptance: The sole MVP policy retains the valid response from the oldest responding trusted server and applies the preference deadlines in Section 9.3. It does not use quorum, reliability scoring, or server-stability filtering.
  • Deduplication: The unique_id in the Tenant Header participates in the server-local deduplication key defined by the canonical wire-protocol rules.

9.1. server_id Decoding

The response Tenant Header.key_id field contains the 64-bit server_id defined by the wire protocol. Clients SHOULD decode it as follows:

  • start_s_since_2025 = server_id >> 23
  • listener_id = (server_id >> 15) & 0xFF
  • node_id = server_id & 0x7FFF
  • server_start_s = 1735689600 + start_s_since_2025

Where 1735689600 is January 1, 2025 00:00:00 +0000 UTC in Unix seconds.

When multiple trusted responses are available, the recommended ranking rule is:

  1. Prefer lower server_start_s values (older servers).
  2. If startup times are equal, prefer the lower full 64-bit server_id.

For clarity, server_start_s comes from the wire-level server_id; the SRV record port remains authoritative for routing.

Clients MUST still use the full 64-bit server_id as the server identity key even when they decode and compare server_start_s.

9.3. Parameterized HA Resource-Request Strategy

This section defines the recommended cross-language policy for resource requests and the sole request policy implemented by the MVP C client. It is not part of the wire protocol and does not constrain independently implemented clients that choose another policy. Latency reports remain independent, fire-and-forget operations and do not participate in this state machine.

The C API represents the strategy directly as one flat r_request_policy_t. It has no policy discriminator, nested strategy, compatibility alias, or alternative wait/quorum/retry execution path. Callers initialize it with r_client_default_request_policy() and then override individual fields.

The policy separates two concerns:

  1. Decision: choose the valid response that governs the caller’s action.
  2. Delivery: make a best-effort attempt to deliver the same logical operation to every server in the request’s membership snapshot.

The separation matters because choosing a response does not prove that every other server received the request.

9.3.1. Parameters

The timing and replay shape are described by these parameters:

Parameter C API field Meaning
U unit_ms Base time unit. It may be configured directly or derived from network observations, but its value is frozen for one request.
N replay_count Maximum number of replays after the initial transmission. There are therefore N + 1 transmission rounds.
B(k) replay_gap Duration of transmission round k, measured in units of U, for 0 <= k <= N. This is also the gap before the next replay when no valid response has arrived.
P(k) preference Oldest-server preference interval in round k, measured in units of U, where 0 <= P(k) <= B(k).
F final_receive_units Duration of the final receive-only interval, measured in units of U. F = 0 disables that interval.
P_final final_preference_units Oldest-server preference interval within the final receive-only interval, where 0 <= P_final <= F.
C completion_delivery Completion-delivery flag: false disables the final fire-and-forget convergence send and true enables it.

B(k) permits a fixed, linear, or exponential replay schedule. Examples include:

  • fixed: B(k) = b;
  • linear: B(k) = min(b + k * step, b_max); and
  • exponential: B(k) = min(b * factor^k, b_max).

The C API represents both B(k) and P(k) with r_ha_schedule_t: kind selects R_HA_SCHEDULE_FIXED, R_HA_SCHEDULE_LINEAR, or R_HA_SCHEDULE_EXPONENTIAL; initial_units and max_units bound the schedule; and the growth union supplies either linear_step_units or exponential_factor. Replay gaps must be positive. Preference values may be zero. The C client bounds replay_count by R_CLIENT_HA_MAX_REPLAY_COUNT, although the API-key deduplication limit normally imposes a much smaller practical maximum.

P(k) is intentionally independent of B(k). An exponential replay backoff can therefore reduce network traffic without forcing the client to retain a valid response for the entire, possibly long, replay interval. Once P(k) has elapsed, the next valid response may complete the request immediately.

The complete operation horizon is:

H = U * (sum(B(k), k = 0..N) + F)

The request deduplication TTL for this strategy is derived from H. Before the first send, the client MUST check that the derived TTL is representable on the wire and does not exceed the limit encoded by the API key. If it does, the client MUST reject the configuration or choose a shorter valid schedule; it MUST NOT silently send with a different replay contract. No replay or completion-delivery send may be initiated at or after the deduplication deadline.

9.3.2. Per-request state

For each resource request, the client records:

  • the immutable membership snapshot and deterministic oldest-to-youngest ranking over its server identities;
  • the encoded logical request and its single unique_id;
  • the current round and its preference and replay deadlines;
  • the set of servers from which a valid response has arrived; and
  • the current candidate, which is the response from the oldest server in that set.

Server age is ordered first by server_start_s. The MVP C client resolves ties by preferring the lower full 64-bit server_id, so one request has an unambiguous order.

Every transmission and replay for the operation MUST represent the same logical request and reuse the same unique_id. A client may regenerate transport framing when required, but it MUST preserve the wire-level deduplication identity.

9.3.3. Event algorithm

At the start of round 0, the client sends the request to every server in the membership snapshot. At the start of a later transmission round, it replays the request only to servers from which no valid response has yet arrived.

For each event in a transmission round:

  1. Validate and correlate received datagrams before changing request state. A response is usable only when its authentication, framing, tenant, unique_id, response kind, and trusted server identity are valid.
  2. Ignore invalid, unrelated, or untrusted datagrams. A duplicate valid response does not add another responding server or replace the candidate, but the current preference deadline still governs whether the existing candidate is now selectable.
  3. For a valid response, mark its server as having responded and replace the candidate if that server is older than the current candidate’s server.
  4. If the response is from the oldest server in the membership snapshot, select it immediately.
  5. If the round’s preference deadline has already elapsed, select the current candidate immediately.
  6. At the preference deadline, select the candidate if one exists. Otherwise, continue waiting until the replay deadline.
  7. At the replay deadline, if no candidate exists and the replay budget remains, start the next round and replay to the currently missing servers.
  8. After the last transmission round expires without a valid response, enter the final receive-only interval when F > 0; otherwise report transport failure.

The final receive-only interval applies the same response-ranking rule, but sends no replay. At its preference deadline, the client selects its current candidate if one exists. Any valid response arriving after that deadline is selected immediately. If its end deadline arrives first, the client reports transport failure.

When a receive event and a deadline are simultaneously ready, an event-loop implementation SHOULD process a bounded batch of already-readable datagrams before applying the deadline. This avoids making the result depend only on whether the operating system reported the timer or socket first, while still preventing receive starvation.

Allow and deny are equally valid responses and use exactly the same ordering and completion rules. Once a response is selected, later responses MUST NOT change the completed result.

9.3.4. Completion delivery

When C = 1, immediately before returning any selected response, the client performs one fire-and-forget send of the same logical request to every server in the membership snapshot from which no valid response has arrived. It does not wait for these sends, and their success or failure does not change the selected response.

This completion send is outcome-independent: it is performed for both allowed and denied responses. If one server denies while a missing, out-of-sync server would allow and consume the request, delivering the operation to that server is intentional. It makes the servers’ operation histories more alike and allows their TTL-governed state to converge; it is not a second client decision. Reusing the same unique_id also means that a server which processed an earlier copy but lost its response deduplicates the completion send.

The absence of a valid response is only evidence that delivery may have failed; it is not proof. Completion delivery is therefore a best-effort convergence mechanism, not an immediate consistency guarantee.

9.3.5. Default schedule

The recommended initial configuration preserves the bounded three-phase heuristic:

U       = 20 ms
N       = 1
B(0)    = 1
B(1)    = 1
P(0)    = 1
P(1)    = 1
F       = 1
P_final = 0
C       = 1
TTL     = 3 * U

With this schedule, the client:

  1. sends to all servers and waits for the oldest response for at most U, retaining the oldest valid fallback;
  2. if no server responded, replays to all still-missing servers and applies the same rule for another U; and
  3. if no server has responded after both transmissions, waits for one final U and returns the first valid response immediately.

At any phase, a response from the oldest server completes the request immediately. At either of the first two deadlines, the oldest valid response received so far completes it. With C = 1, every selected allow or deny is preceded by the fire-and-forget completion delivery to servers still missing a valid response.

9.4. DNS Refresh Is Separate

DNS refresh pacing belongs to persistent client configuration, not to the resource-request response-selection policy. In the C API it is configured through r_client_config_t.dns_refresh:

  • refresh_interval_ms controls periodic refresh and defaults to 300 seconds when zero;
  • forced_refresh_min_interval_ms bounds repeated forced refresh and defaults to one second when zero; and
  • forced_refresh_jitter_ms adds optional jitter and defaults to zero.

The C client copies r_request_policy_t during r_client_create(). A request then freezes its membership snapshot and derived schedule; DNS results installed asynchronously apply to later requests.

↑