Skip to content

fix(ingress): wildcard gets its own Ingress + Traefik router.priority (root cause now measured) - #516

Open
izzywdev wants to merge 1 commit into
masterfrom
claude/traefik-wildcard-priority
Open

fix(ingress): wildcard gets its own Ingress + Traefik router.priority (root cause now measured)#516
izzywdev wants to merge 1 commit into
masterfrom
claude/traefik-wildcard-priority

Conversation

@izzywdev

@izzywdev izzywdev commented Aug 3, 2026

Copy link
Copy Markdown
Owner

Makes the tenant-subdomain wildcard safe to re-enable. It stays disabled in prod — this ships the mechanism, not the rollout.

The root cause is no longer inferred

#431 shipped the wildcard as a second rule on the main Ingress and took FuzePlan down. #437 reverted it, and recorded the cause as inferred — the prod Traefik API was unreachable, so the router priorities were never read.

I reproduced it in isolation on rancher/mirrored-library-traefik:3.6.13, the exact prod image, using the file provider with two routers and no declared priority:

Host: plan.fuzefront.com     ->  wildcard-fuzefront@file  ->  :9002    WRONG

The exact-host router lost to the wildcard. Traefik orders routers by rule length, and the generated HostRegexp is longer than Host(`plan.fuzefront.com`).

With priority: 1 on the wildcard router only:

Host: plan.fuzefront.com     ->  exact-plan@file          ->  :9001    correct
Host: corpabc.fuzefront.com  ->  wildcard-fuzefront@file  ->  :9002    correct

Why a separate Ingress object

traefik.ingress.kubernetes.io/router.priority is a per-Ingress-object annotation. The wildcard cannot be de-prioritised while it shares an object with the canonical host — so it moves to fuzefront-tenant-wildcard. The template says so inline, with the measurement, so nobody merges it back.

On ingress-nginx (local) the annotation is inert: nginx follows the Kubernetes spec and prefers exact hosts on its own.

The YAML anchor had to go

An anchor cannot be referenced across YAML documents, so *fanout could not survive the split. Both call sites now include the same helper. The guarantee the anchor existed for — the two fan-outs cannot drift — is preserved, because there is still exactly one definition. The render asserts it rather than trusting it.

Renders

Scenario Result
prod (wildcard off) tenant-wildcard not rendered; main = 1 rule, 8 paths
local (nginx) tenant-wildcard rendered, no priority annotation (inert on nginx)
prod dry-run (wildcard on) router.priority=1, no tls: block, fan-out identical to canonical (8 paths), main still a single rule
helm lint 0 charts failed

Prod re-enablement is still a deploy-window decision

Deliberately not flipped here. The wildcard CNAME already exists at Cloudflare, so setting wildcardHost puts every *.fuzefront.com host live immediately — there is no DNS gate to stage behind. That assumption is what made #431 an outage rather than a no-op.

When you do enable it, verify plan.fuzefront.com straight after the sync, not the rendered YAML. helm template | kubeconform -strict and a parsed fan-out-identity check both passed on #431 and neither could see cross-namespace router precedence.

Refs #431, #437, FFRNT-91.

🤖 Generated with Claude Code

Makes the tenant-subdomain wildcard safe to re-enable. It stays DISABLED in
prod — this ships the mechanism, not the rollout.

#431 shipped the wildcard as a second rule on the main Ingress and took FuzePlan
down: plan.fuzefront.com served the FuzeFront shell. #437 reverted it. The root
cause was recorded then as INFERRED. It is now measured.

MEASURED on rancher/mirrored-library-traefik:3.6.13 (the exact prod image), file
provider, two routers, neither declaring a priority:

    Host: plan.fuzefront.com     -> wildcard-fuzefront@file -> :9002   WRONG

The exact-host router lost to the wildcard. Traefik orders routers by RULE
LENGTH, and the generated HostRegexp is longer than Host(`plan.fuzefront.com`).

With `priority: 1` on the wildcard router only:

    Host: plan.fuzefront.com     -> exact-plan@file         -> :9001   correct
    Host: corpabc.fuzefront.com  -> wildcard-fuzefront@file -> :9002   correct

So the fix is to de-prioritise the wildcard. `router.priority` is a
PER-INGRESS-OBJECT annotation, so the wildcard cannot be de-prioritised while it
shares an object with the canonical host — hence the split into
`fuzefront-tenant-wildcard`.

The YAML anchor is replaced by a direct include of the same helper: an anchor
cannot be referenced across documents, and the guarantee it existed for (the two
fan-outs cannot drift) is preserved because there is still exactly one
definition. Asserted in the render, not just claimed.

Renders:
  prod (wildcard off)  -> tenant-wildcard NOT rendered; main = 1 rule, 8 paths
  local (nginx)        -> tenant-wildcard rendered, no priority annot (inert on
                          nginx, which follows the k8s exact-beats-wildcard rule)
  prod DRY-RUN (on)    -> router.priority=1, no tls: block, fan-out identical to
                          canonical (8 paths), main still a single rule
  helm lint            -> 0 failed

Prod re-enablement remains a deploy-window decision: the wildcard CNAME already
exists at Cloudflare, so flipping wildcardHost puts every *.fuzefront.com host
live immediately. Verify plan.fuzefront.com straight after the sync.

Refs #431, #437, FFRNT-91.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@izzywdev izzywdev added the auto-merge Enable squash auto-merge once CI passes label Aug 3, 2026
@github-actions
github-actions Bot enabled auto-merge (squash) August 3, 2026 09:35
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Automated code review (gate-code-review)

Credit balance is too low

Report-only — this check never blocks merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-merge Enable squash auto-merge once CI passes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant