fix(terraform2.0/gateway): restart the service so it applies the written config; expose core_disconnect_grace_period - #173
Open
valentinvilar-conexa wants to merge 1 commit into
Conversation
…ten config The package postinst starts defguard-gateway with the gateway.toml shipped in the .deb, before setup.sh writes the module's configuration. The final `systemctl start` is a no-op on an already-active unit, so the service keeps running with the packaged defaults for the life of the instance -- most importantly `masquerade = false` even when `nat = true`, which on AWS makes the ENI's source/destination check silently drop all forwarded VPN traffic. Also expose core_disconnect_grace_period, which the module did not write, so the fail-closed purge window can be raised above a normal Core restart (see DefGuard/gateway#361). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem 1 — the Gateway never reads the config this module writes
terraform2.0/modules/gateway/setup.shdoes, in order:apt install -y defguard-gateway— the package'spostinststarts the service, using thegateway.tomlshipped with the.debgateway.tomlsystemctl start defguard-gatewaysystemctl starton an already-active unit is a no-op, so step 3 does not apply step 2. The Gateway keeps running with the package's default configuration for the entire life of the instance, unless someone restarts it by hand.The most damaging consequence is that
masqueradestaysfalseeven whennat = true. ThenfttableDEFGUARD-wg0is created but itsPOSTROUTINGchain is empty:On AWS this is fatal and silent. Forwarded VPN packets leave the ENI with a source address in the client range, the ENI's source/destination check drops them in the hypervisor — no security group, no log — and nothing inside the VPC answers. The tunnel still connects (the handshake does not involve forwarding), the client gets its address and routes, so it looks healthy. Because the client sends all DNS to the VPN resolver, name resolution fails completely, which sends you looking at DNS instead of NAT. It took us a full day.
An easy way to tell whether a Gateway is affected, from the startup log line
Starting Defguard Gateway ... with configuration: Config { ... }:stats_period: 60,masquerade: false→ running the.debdefault configstats_period: 30,masquerade: true→ running this module's configFix:
systemctl start→systemctl restart.Problem 2 —
core_disconnect_grace_periodis not configurableThe Gateway purges fail-closed when the Core has been unreachable for
core_disconnect_grace_period, whose default is30s. The purge tears downwg0and deletes the Gateway's gRPC certificates, dropping it back to setup mode. The Core still has it recorded as adopted, so it keeps dialing mTLS and loops onreceived corrupt message of type InvalidContentTypeindefinitely, with no automatic recovery. Filed separately as DefGuard/gateway#361.Until that is resolved, the practical mitigation is a grace period longer than any normal Core restart — but this module does not write the key, so there is no way to set it without editing the file on the instance, which is lost on every instance replacement (and
user_data_replace_on_change = truemakes replacements routine).This PR adds the variable and writes the key, defaulting to
300. Happy to change the default to30to preserve current behaviour if you prefer the module to stay neutral.The
coreandedgemodules have the same bug — they only appear to workBoth end their scripts with
systemctl startafter writing the config, exactly likegateway. We checked all three on a live deployment:gateway.debconfig for the life of the instance.masqueradestaysfalse.coreedgecoreis masked by a crash loop. The packagedcore.confhas no database configured, so the service thepostinststarted panics and core-dumps:By the time systemd's restart policy brings it back,
setup.shhas written the realcore.conf, so it comes up correctly. It works because the packaged config is unusable — not by design.edgenever restarts at all (0 restarts across the boot) and therefore runs the packagedproxy.tomlfor the life of the instance. It happens to behave correctly only because every value the module sets coincides with a default:So the module's
edge_http_port/edge_https_portvariables are effectively ignored: set either to a non-default value and the Edge will silently keep listening on the old ports. Same class of silent failure as the Gateway, just currently invisible.The one-word
start→restartfix should be applied to all three modules. This PR only changesgateway(the one that is actively broken) to keep the diff focused — happy to extend it tocoreandedgein this PR or a follow-up, whichever you prefer.Testing
Verified on Ubuntu 24.04 / EC2, gateway
2.0.4, Core2.0.3: after the change the startup log showsmasquerade: true,stats_period: 30andcore_disconnect_grace_period: 300, thePOSTROUTINGchain contains themasqueraderule, and restarting the Core no longer triggers a purge.