perf(redis): collapse admission checks into one atomic Redis round trip

[CMGS]

Evidence

The online DAG serializes tenant QPS -> AK QPS -> product QPM -> model QPM -> AK TPM:

• nodes.rs
• default_layers

Realtime repeats the same checks sequentially for every generation. With Redis governance, each enabled check executes a separate command or script via governance.rs.

This is a static finding, not a production measurement. Five enabled checks cost roughly five serial Redis RTTs before the upstream call. Collapsing them to one would save approximately (N-1) * RTT: about 4 ms at 1 ms Redis RTT, or 12 ms at 3 ms.

Proposed scope

Add a structured Governance::admit(...) operation implemented by Redis as one Lua script. It should evaluate the enabled tenant, AK, product, model, quota, and TPM constraints and return a typed, deterministic denial reason.

Use the same operation from the REST DAG and realtime gate. Preserve the current fail-open policy for Redis outages and explicitly define whether denied requests consume earlier rate-window counters. Keep model-fallback resolution and cache lookup outside the aggregate where ordering requires it.

Document or guard Redis Cluster behavior: the script touches multiple logical keys and may require a deliberate hash-slot strategy.

Do not merely run the current checks concurrently; that would change denial precedence and counter-consumption semantics.

Acceptance criteria

• REST and realtime use the same aggregate admission implementation.
• Redis command tracing shows one admission round trip per admitted generation, excluding authentication/cache and settlement.
• Each limit returns the same user-visible denial precedence as today.
• A denial or Redis/script error leaves no leaked daily or TPM reservation.
• Memory and Redis governance pass a shared behavioral test suite.
• Benchmark all limits enabled with Redis at 0.2/1/3 ms injected RTT and report TTFT plus p50/p95/p99.

[CMGS]

Proposed approach (analysis — not yet implemented, not urgent)

Aggregate Governance::admit()

Collapse the current sequential admission chain — tenant QPS → AK QPS → product QPM → model QPM → per-user budget → AK daily reserve → AK TPM reserve — into a single Governance::admit(AdmitInput) -> AdmitOutcome, returning a typed, deterministic denial plus the daily/TPM amounts reserved (so settlement stays symmetric). On single-node Redis this is one atomic Lua round trip; the Memory backend runs the same checks in-process behind the same trait method. REST DAG nodes and the realtime gate both call it; cache lookup, model-fallback resolution, and reserve/settle stay outside the aggregate.

Semantics to preserve

• Denial precedence — unchanged (tenant → AK → product → model → user → daily → TPM), same user-visible message per limit.
• Counter consumption on denial — keep today's behavior: the rate/QPM counters (INCR-and-check) stay consumed even when a later check denies; the two reserves roll back among themselves (a TPM-reserve failure refunds the daily reserve), so no reservation leaks. Considered and rejected: all-or-nothing consumption — it would change rate-limit dynamics (a client hammering a denied model would no longer consume tenant/AK QPS permits).
• Fail-open — a Redis/script error admits (limits skipped) and logs loudly, matching the current posture.

The Redis Cluster limitation (the reason this is not a clean default)

A single multi-key Lua script cannot run on Redis Cluster: the seven keys (tenant, AK, product, model, user, daily, TPM) hash to different slots → CROSSSLOT. They also cannot be co-located under one hash tag, because the counters are shared across different scopes — the tenant counter is shared by all of a tenant's AKs; the product/model counters are cross-tenant — so forcing them onto one slot (e.g. {ak}) would fragment the shared counters and break their semantics.

Context: the current governance already uses a single-node client (redis::aio::ConnectionManager), so Cluster is not supported today by either path — the Lua collapse does not regress Cluster support, and a CROSSSLOT error on a misconfigured Cluster falls open exactly as the current sequential path does on MOVED. But single-node Redis is itself an availability risk, which is why a Cluster-capable design is preferable before this becomes load-bearing.

Cluster-friendly variant (for when this is picked up)

Group only the AK-scoped keys (AK rate + AK daily reserve + AK TPM reserve) under a {ak} hash tag into one script — that is 1 round trip on both single-node and Cluster — and keep tenant / product / model / user as separate calls. This drops ~7 round trips to ~5 on Cluster and to 1 on single-node, while preserving shared-counter semantics (the shared counters stay on their own slots). It is the honest ceiling on Cluster: cross-slot counters physically live on different nodes, so a true single-round-trip admit is only achievable on single-node.

Status

Deferred — not urgent. Prerequisite for the headline benchmark (all limits enabled, Redis at 0.2/1/3 ms injected RTT, report TTFT + p50/p95/p99) is a decision on whether to ship the single-node-only Lua collapse now or invest in the AK-scoped-group variant so Cluster is not left behind.