relay: global max_circuits pinned to per-peer value (512) throttles total relay throughput

[emlautarom1-agent]

Summary

Pluto's relay sets the global circuit ceiling (relay::Config.max_circuits) to max_res_per_peer (default 512). This is a cap across all peers, so the entire relay can host at most 512 concurrent circuits regardless of --p2p-max-connections (default 16384). At production fan-out this can cause spurious RESOURCE_LIMIT_EXCEEDED circuit denials and prevent peers from connecting via the relay.

There is already a todo(varex83) in the code questioning this exact value.

Where

crates/relay-server/src/config.rs:49-64:

relay::Config {
    max_reservations: config.max_conns,                 // 16384
    max_reservations_per_peer: config.max_res_per_peer, // 512
    ...
    max_circuits: config.max_res_per_peer,              // 512  <-- GLOBAL cap
    max_circuits_per_peer: config.max_res_per_peer,     // 512
    ...
}

rust-libp2p denies a circuit when self.circuits.len() >= self.config.max_circuits (global) or per-peer circuits exceed max_circuits_per_peer (libp2p-relay-0.21.1/src/behaviour.rs:540-548).

Semantics mismatch with go-libp2p / Charon

rust-libp2p splits circuit limits into global (max_circuits) and per-peer (max_circuits_per_peer). go-libp2p (circuitv2) has only one:

• Resources.MaxCircuits is documented as "the maximum number of open relay connections for each peer" — i.e. per-peer, default 16. There is no global circuit cap in go-libp2p.

(go-libp2p@v0.36.2/p2p/protocol/circuitv2/relay/resources.go)

Charon's relay config (charon/cmd/relay/p2p.go:60-72, built on relay.DefaultResources()):

relayResources := relay.DefaultResources()                 // MaxCircuits default = 16 (per-peer)
relayResources.MaxReservationsPerIP = config.MaxResPerPeer // 512
relayResources.MaxReservations      = config.MaxConns      // 16384
relayResources.MaxCircuits          = config.MaxResPerPeer // 512  (PER-PEER)

Charon CLI defaults (charon/cmd/relay.go:53-54) are the same ones Pluto adopted:

• p2p-max-reservations = 512
• p2p-max-connections = 16384

So Charon sets a per-peer circuit limit of 512 and relies on go-libp2p having no global circuit ceiling.

Mapping today

Limit	Pluto	Charon (go-libp2p)	Verdict
global reservations	max_reservations = max_conns (16384)	MaxReservations = MaxConns (16384)	✔ matches
per-peer reservations	max_reservations_per_peer = max_res_per_peer (512)	MaxReservationsPerIP = MaxResPerPeer (512)	✔ close analogue
per-peer circuits	max_circuits_per_peer = max_res_per_peer (512)	MaxCircuits = MaxResPerPeer (512, per-peer)	✔ matches
global circuits	max_circuits = max_res_per_peer (512)	(no equivalent)	✘ artificial throttle

Suggested fix

Decouple the global cap from the per-peer value. Keep max_circuits_per_peer = max_res_per_peer (mirrors Charon), but raise the global max_circuits so it doesn't throttle a healthy mesh:

max_circuits: config.max_conns,                 // align global circuit ceiling with the connection budget
max_circuits_per_peer: config.max_res_per_peer, // unchanged; mirrors Charon's per-peer MaxCircuits

Rationale: max_conns (16384) is the global reservation/connection budget; capping total circuits at that same value preserves a safety ceiling (which go-libp2p lacks entirely) while removing the artificial 512 throttle.

Alternative considered: usize::MAX to fully mirror Charon's "no global cap", but that removes the safety valve, so scaling with max_conns is preferred. Optionally, expose the global cap as its own CLI flag if operators need to tune it independently.

Notes

• Not observed in current dev-cluster logs (circuit counts are well below 512); this is a latent scaling limit, not an active incident. It came up while investigating relay WARN noise: today Pluto's relay logs do not include the deny status, so RESOURCE_LIMIT_EXCEEDED (this cap) and NO_RESERVATION (benign) are indistinguishable. Logging the status is being addressed separately.
• Resource-policy change — worth a quick review of the chosen default before merge.

References

• crates/relay-server/src/config.rs:49-64
• crates/cli/src/commands/relay.rs:153-169
• libp2p-relay-0.21.1/src/behaviour.rs:540-594
• charon/cmd/relay/p2p.go:60-72, charon/cmd/relay.go:53-54
• go-libp2p@v0.36.2/p2p/protocol/circuitv2/relay/resources.go