[Bubblewrap] Enforce proxy-only egress for private networking

[Soham Das (SohamDas2021)]

📖 Description

Stacked on #930 (user/sodas/bwrap-network-model2).

The base PR moves schema-0.8+ Bubblewrap proxy sandboxes into a private network namespace, but slirp4netns routes everything out of it — so the proxy is still only cooperative. A workload that ignores HTTP_PROXY/HTTPS_PROXY, or opens a raw socket, reaches the network directly. This PR makes the boundary real: inside the namespace, the proxy endpoint is the only reachable destination. Everything else is dropped, IPv4 and IPv6.

How it works

The supervisor (already introduced in the base PR) gains firewall responsibility:

1. slirp4netns moves from exec to a backgrounded child with its own internal readiness fd, so the supervisor regains control once routing is up.
2. The supervisor installs an MXC_EGRESS chain via nsenter --net=/proc/<pid>/ns/net, hooked from OUTPUT: accept lo, accept tcp -d <proxy-ip> --dport <proxy-port>, DROP everything else. The IPv6 chain accepts lo and drops the rest.
3. Only after every rule is installed does it write the external ready signal that releases the workload. The ordering is the security property — the sandbox can never observe an unenforced network. set -e means a failed rule kills the supervisor rather than proceeding.

bwrap joins the supervisor's user namespace and drops capabilities, so the workload sits inside the namespace that owns its own firewall but cannot modify it. That assumption is load-bearing, so it is now both documented and asserted by a test (below).

DNS is deliberately closed

No :53 accept rule is installed. An unscoped port-53 allowance is a standing DNS-tunnel exfiltration path, and this mirrors the policy already shipped for LXC model 2. Name resolution happens through the proxy.

Limitations

• Proxy endpoints must be a literal IPv4 address + port. A firewall rule needs an address, and resolving a hostname at rule-install time is a TOCTOU hole. Hostname and routable IPv6 endpoints are rejected at validation with an actionable error; localhost / 127.0.0.0/8 / 0.0.0.0 / :: are rewritten to slirp's gateway, while ::1 is rejected (an IPv6-loopback listener cannot accept the gateway's IPv4 connection). This is a narrowing of what the base PR accepts — hostname support requires porting LXC's ProxyHostPin (already present in
  wxc_common::models) and is the immediate follow-up.
• New host prerequisites for this mode: nsenter, iptables, ip6tables, added to the existing validate-time probe. None require root. iptables/ip6tables must also resolve to the nf_tables backend unless /run/xtables.lock is writable — the legacy backend opens that lock before touching any table, which an unprivileged same-uid supervisor cannot do on a stock host. validate refuses such a host rather than letting the supervisor die at the first rule.
• The 9 rule installations are serialized nsenter + iptables spawns, each passing -w, under a dedicated rule-installation budget rather than the 5s startup timeout. Batching them via iptables-restore was considered and dropped: -w plus the separate budget closes the correctness half, leaving a latency refactor that would trade per-rule failure attribution for one opaque batch failure.

Follow-ups filed: #935 (slirp liveness during the run), #936 (AF_VSOCK and other non-IP families), #937 (network.proxy enforced on LXC/bwrap-0.8 but cooperative on Seatbelt/WSLc).

🔍 Validation

Automated:

• cargo test -p bwrap_common — 112 pass, incl. 37 new tests covering endpoint validation (translation, ::1, hostname/IPv6, port-0, legacy-schema untouched, builtinTestServer exemption), iptables backend selection, and supervisor behaviour. The supervisor tests execute the script under a real sh with stubbed nsenter/slirp4netns — rule ordering, readiness-written-last, every rule position failing ⇒ no readiness, slirp dying early, and the exact ACCEPT set (which fails if a :53 rule is ever added).
• cargo clippy -p bwrap_common --all-targets -- -D warnings, cargo fmt --check — clean.
• node scripts/versioning/validate-configs.js — 233 configs.

Manual E2E on Ubuntu 24.04 (WSL2), unprivileged, no sudo:

• tests/scripts/run_bwrap_network_proxy_test.sh — 8/8 pass. The bubblewrap_network_proxy_egress_denied.json case asserts seven properties in one run: the proxy is reachable over a raw socket first (a positive control, so a blocked direct connection is attributable to the DROP rule rather than a dead network), direct egress blocked, loopback still exempt, CAP_NET_ADMIN dropped, an in-sandbox flush refused with a non-zero status, tampering ineffective, and the proxy still reachable at the end (so an offline host fails rather than passing vacuously).
• No leaked slirp4netns / bwrap processes after teardown.

Every new invariant was checked non-vacuous by injecting the matching regression (|| true on a rule, a swallowed rule failure, readiness moved ahead of the rules, the backend check stubbed to pass) and confirming the test fails. The first two are caught only by the executing supervisor tests — the text-based ones stayed green against an unenforced sandbox.

✅ Checklist

• Signed the Contributor License Agreement
• Linked to an issue
• Updated documentation (if applicable)
• Updated Copilot instructions (if build, architecture, or conventions changed)
• If this PR changes Cargo.lock, the dependency-feed-check check passes

📋 Issue Type

• Bug fix
• Feature
• Task

---

GitHub Actions runs the PR validation build automatically. The ADO pipeline
(MXC-PR-Build) is the Azure version of the PR pipeline, kept in parity with the GitHub
Actions build; it runs on merge to main, and Microsoft reviewers with write access can trigger it
on a PR with /azp run. See docs/pull-requests.md.

If the dependency-feed-check check fails on a new dependency, the crate must be added to
the feed before the PR can pass. See docs/pull-requests.md
for the steps.

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

Pull request overview

Enforces proxy-only Bubblewrap egress by installing namespace-local IPv4/IPv6 firewall rules before releasing workloads.

Changes:

• Adds fail-closed MXC_EGRESS firewall setup and dependency checks.
• Validates proxy endpoints and updates network-mode handling.
• Adds automated/E2E coverage and documentation.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/backends/bubblewrap/common/src/proxy_network.rs	Implements namespace firewall enforcement.
src/backends/bubblewrap/common/src/bwrap_runner.rs	Resolves and applies proxy egress endpoints.
src/backends/bubblewrap/common/src/bwrap_command.rs	Clarifies host versus namespace firewall modes.
tests/configs/bubblewrap_network_proxy_egress_denied.json	Adds egress and tampering checks.
tests/scripts/run_bwrap_network_proxy_test.sh	Runs the new enforcement test.
docs/bwrap-support/bubblewrap-backend.md	Documents requirements and enforcement behavior.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Suppressed comments (4)

src/backends/bubblewrap/common/src/bwrap_runner.rs:257

• This endpoint check is only reached during spawn_bwrap. ScriptRunner::run returns success immediately after validate_runner for a dry run, while this backend's validate only probes the dependencies, so --dry-run incorrectly accepts hostname and routable IPv6 proxy endpoints that execution rejects. Perform the endpoint validation in validate as well (with the existing loopback translation and builtin-proxy handling).

                .ok_or_else(|| {
                    "Bubblewrap: proxy-only networking requires a resolved proxy address"
                        .to_string()
                })
                .and_then(proxy_network::ProxyEgress::from_address)

src/backends/bubblewrap/common/src/bwrap_command.rs:102

• The proxy-env section later in this same builder (lines 364–369) still says clients using raw sockets are “NOT enforced.” That now contradicts this mode's contract for schema 0.8 and can mislead future maintenance. Update that comment to distinguish cooperative routing from the new enforced egress boundary, while retaining the legacy-version qualification.

    /// Cooperative proxy routing inside a slirp-backed private namespace, with
    /// egress closed to everything but the proxy.

docs/bwrap-support/bubblewrap-backend.md:395

• This change knowingly leaves the user-facing parser diagnostic false: config_parser.rs:1168-1182 still says Bubblewrap iptables requires privilege and calls this a cooperative-only model, which this PR disproves. Update that diagnostic to state the actual reason firewall/both remains incompatible, rather than documenting an incorrect error message.

  or `"both"` is rejected at config-parse time. (The rejection message cites a
  root requirement that proxy mode has since disproved; see the firewall
  section.)

src/backends/bubblewrap/common/src/proxy_network.rs:59

• The fixed descriptor 9 can collide with any of the dynamically allocated descriptors passed to the supervisor. For example, if ready.as_raw_fd() is 9, this redirection replaces the external readiness file with slirp.internal; the final readiness write then goes to the wrong file and startup always times out. Allocate the internal readiness file/FD in Rust and pass its distinct descriptor to the script instead of hard-coding an FD number.

exec 9> "$state_dir/slirp.internal"

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Suppressed comments (3)

src/backends/bubblewrap/common/src/proxy_network.rs:509

• This translates every 127.0.0.0/8 address, but slirp's 10.0.2.2 gateway maps specifically to host 127.0.0.1. A configured proxy bound only to 127.0.0.2 therefore passes validation, is advertised as supported in the updated docs, and is rewritten to an endpoint where nothing listens. Reject non-127.0.0.1 loopback literals (or add forwarding that preserves them) instead of treating the whole loopback range as reachable through the gateway.

        || parsed.is_some_and(|ip| ip.is_loopback() || ip.is_unspecified());

src/backends/bubblewrap/common/src/proxy_network.rs:39

• The deadline is exactly the sum of the nine maximum -w waits, but attach starts it before slirp readiness and each nsenter/iptables process also consumes execution and spawn time. Thus the fully contended case this budget claims to support necessarily exceeds the deadline and rejects a viable sandbox. Include slirp startup and command overhead (or enforce one explicit end-to-end deadline inside the supervisor) rather than sizing to the bare lock waits.

const RULE_INSTALL_TIMEOUT: Duration =
    Duration::from_secs(XTABLES_LOCK_WAIT.as_secs() * RULE_COMMAND_COUNT as u64);

tests/configs/bubblewrap_network_proxy_egress_denied.json:6

• This assertion can pass without any MXC egress rule: on hosts that already block direct Internet access (a common reason to require a proxy), the connection to 1.1.1.1:443 fails even if slirp remains unrestricted. The later proxied GitHub request only proves proxy connectivity, not that this direct destination was reachable before enforcement. Use a host-side control listener on a non-proxy port reachable through 10.0.2.2 (or establish an equivalent positive precondition) and assert that the sandbox cannot reach it.

    "commandLine": "bash -c 'set -u; pu=\"${HTTP_PROXY:-${http_proxy:-}}\"; if [ -z \"$pu\" ]; then echo NO_PROXY_ENV; exit 1; fi; pa=\"${pu#*://}\"; pa=\"${pa%%/*}\"; ph=\"${pa%%:*}\"; pp=\"${pa##*:}\"; if [ \"$pp\" = \"$ph\" ]; then pp=80; fi; if ! timeout 6 bash -c \"exec 3<>/dev/tcp/$ph/$pp\" >/dev/null 2>&1; then echo CONTROL_PROXY_UNREACHABLE; exit 1; fi; echo CONTROL_PROXY_REACHABLE_OK; timeout 6 bash -c \"exec 3<>/dev/tcp/1.1.1.1/443\" >/dev/null 2>&1; rc=$?; if [ \"$rc\" = 0 ]; then echo DIRECT_EGRESS_LEAKED; exit 1; fi; echo DIRECT_EGRESS_BLOCKED_OK; timeout 4 bash -c \"exec 3<>/dev/tcp/127.0.0.1/9\" >/dev/null 2>&1; lrc=$?; if [ \"$lrc\" = 124 ]; then echo LOOPBACK_DROPPED; exit 1; fi; echo LOOPBACK_EXEMPT_OK; if ! command -v iptables >/dev/null 2>&1; then echo NO_IPTABLES_BINARY; exit 1; fi; capeff=$(grep \"^CapEff\" /proc/self/status | cut -f2); if [ -z \"$capeff\" ]; then echo NO_CAPEFF; exit 1; fi; if [ $((0x$capeff & 0x1000)) -ne 0 ]; then echo CAP_NET_ADMIN_RETAINED; exit 1; fi; echo CAP_NET_ADMIN_DROPPED_OK; terr=$(iptables -F MXC_EGRESS 2>&1); trc=$?; if [ \"$trc\" = 0 ]; then echo \"TAMPER_FLUSH_SUCCEEDED: $terr\"; exit 1; fi; echo \"TAMPER_REFUSED_OK rc=$trc\"; timeout 6 bash -c \"exec 3<>/dev/tcp/1.1.1.1/443\" >/dev/null 2>&1; if [ $? = 0 ]; then echo TAMPER_DISABLED_EGRESS; exit 1; fi; echo TAMPER_INEFFECTIVE_OK; if ! curl -fsSL --max-time 15 https://api.github.com/zen >/dev/null; then echo PROXY_UNREACHABLE; exit 1; fi; echo PROXY_STILL_OK'"

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

Suppressed comments (3)

src/backends/bubblewrap/common/src/proxy_network.rs:43

• The parent starts this timeout before slirp startup, but the budget is exactly the sum of the nine maximum -w waits. If each command legitimately waits close to five seconds, slirp startup and process-spawn overhead push readiness past 45 seconds, so concurrent provisioning can be killed even though every rule command is still within its own timeout. Reserve the startup allowance (and therefore some overhead) in this deadline.

const RULE_INSTALL_TIMEOUT: Duration =
    Duration::from_secs(XTABLES_LOCK_WAIT.as_secs() * RULE_COMMAND_COUNT as u64);

src/backends/bubblewrap/common/src/proxy_network.rs:514

• :: does not guarantee a dual-stack listener: IPV6_V6ONLY may be enabled per socket or via net.ipv6.bindv6only. Rewriting an external [::] proxy to the IPv4 gateway therefore accepts a valid IPv6-only listener during validation but makes it unreachable at runtime. Reject :: with the other IPv6 endpoints (matching this PR's IPv4-only contract), or establish an IPv4 listener before translating it; update the tests and docs that currently promise this translation.

    // `::` is safe to rewrite to IPv4 because a dual-stack wildcard listener
    // accepts IPv4 connections, which `::1` does not.
    let is_host_local = host.eq_ignore_ascii_case("localhost")
        || parsed.is_some_and(|ip| ip.is_loopback() || ip.is_unspecified());

src/backends/bubblewrap/common/src/bwrap_command.rs:102

• This updated mode contract conflicts with the proxy-environment comment at lines 364–369, which still says raw sockets are “NOT enforced” and calls that a documented limitation. Scope that statement to LegacyProxy or explain that ProxyOnly enforces the destination with its in-namespace firewall, so future changes do not rely on the obsolete security model.

    /// Cooperative proxy routing inside a slirp-backed private namespace, with
    /// egress closed to everything but the proxy.