Feature set 134. - #61
Merged
Merged
Conversation
Rate-limit the caller, not the load balancer. Connection.RemoteIpAddress behind Railway is one of ~20 internal 100.64.x addresses, so every anonymous caller shared about twenty buckets and none of them was anybody. Recovered from production logs by hashing candidate ranges against the stored IpHash values: the ten busiest "clients" were 100.64.0.3 through .9. ClientIpResolver reads CF-Connecting-IP, but only on requests carrying a secret the Worker injects — Cloudflare appends to a caller-supplied X-Forwarded-For, and Railway answers on its own hostname where the header can just be typed in, so a spoofable key would be worse than a coarse one. Rate-limit rejections are now recorded. UseRateLimiter short-circuits ahead of the logging middleware, so the logs held zero 429s. Stats: date ranges, and metrics worth the trip. The page was fixed at 24 hours. It now takes any range, switches from hourly to daily buckets past three days, and reports the window it actually measured. Adds keyed-vs-anonymous split, top failing paths, per-endpoint p95, and busiest clients. Also drops the language describing this as a wiki-scraping project. The seeder is gone; attribution stays, because the wiki's text is CC BY-SA. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rate limiting was partitioning on the load balancer
Connection.RemoteIpAddressbehind Railway is not the caller. Recovered from three months ofproduction
IpHashvalues by hashing 4.7M candidate addresses through the sameSHA256(...)[..16]the logger uses — six of the ten busiest "clients" were:100.64.0.0/10is carrier-grade NAT space — Railway's internal load balancer pool. Not Cloudflareedge addresses either (all 15 published ranges were hashed; no hits). The aggregate agrees: distinct
addresses per day sat at 21–24 whether the day served 56 requests or 1,730.
So no caller could be isolated, legitimate callers shared buckets with abusers, and the anonymous
ceiling was ~60/min × proxy nodes × replicas — the source of the ~290/min measured in production.
ClientIpResolverresolvesCF-Connecting-IP, but only when the request carries a secret theWorker injects. That condition is the design, not belt-and-braces: Cloudflare appends to a
caller-supplied
X-Forwarded-Forso its first entry is attacker-controlled, and Railway answers onits own
*.up.railway.apphostname whereCF-Connecting-IPcan simply be typed in. A spoofablepartition key is worse than a coarse one — it lets one caller mint unlimited fresh windows. Without
the secret it falls back to the peer address. Seven tests cover the bypasses.
429s are now recorded.
UseRateLimitershort-circuits ahead of the logging middleware, which iswhy the logs contain zero of them — that zero was never evidence the limiter wasn't firing.
Reordering the pipeline was rejected as the fix: it would have exempted every static asset from rate
limiting.
OnRejectedwrites through the extractedRequestLogWriterinstead.Stats: ranges and metrics
Presets (24h / 7d / 30d / All) plus custom dates. Buckets switch hourly → daily past three days, and
the response echoes the window actually measured, because a chart whose axis silently changed unit
is a chart that lies about its shape. Very large ranges are trimmed to their most recent slice and
say so.
New: keyed vs anonymous, top failing paths, per-endpoint p95 (endpoints under five requests omitted),
busiest clients. The clients panel carries the caveat that pre-fix rows counted infrastructure.
Verified against production data by booting locally with a temporary auth bypass, since the endpoint
is owner-only — that bypass is reverted and not in this diff. It caught a real bug: model binding
returns
Kind=Localfor an ISO string ending inZ, and Npgsql refuses non-UTC fortimestamptz,so every custom range 500'd while the default 24 hours worked. Fixed and re-verified across eight
range shapes including reversed, empty, zone-less, and all-time.
Two findings worth your eye, from the new panels: 125 404s on
/api/v1/games(something iscalling a versioned path that doesn't exist), and
/api/arena/rosterat 3.3s p95, the slowestendpoint in the catalogue.
Language
Drops the framing of this as a wiki-scraping project — README, workflow headers, tool header. The
seeder is gone and the catalogue is curated by hand. Attribution to the wiki stays: its text is
CC BY-SA, which requires credit regardless of how much hand-editing has happened since. References
that describe live behaviour (the Referer-blocking CDN, the MediaWiki thumbnail URL the regex
actually parses) were generalised, not deleted.
Needs you
cloudflare/RATE-LIMITING.mdhas the steps I can't do from here:wrangler secret put EDGE_SECRET,the matching
Edge__Secreton Railway, and the edge rate-limiting rule. Until both sides of thesecret are set, this changes nothing — the resolver falls back to today's behaviour, silently.
🤖 Generated with Claude Code