Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .azure-pipelines/templates/SDK.Integration.Test.Job.yml
Original file line number Diff line number Diff line change
Expand Up @@ -99,8 +99,8 @@ jobs:
- ${{ if eq(target.os, 'linux') }}:
- script: |
sudo apt-get update -qq
sudo apt-get install -y -qq lxc lxc-utils dnsmasq-base iptables bubblewrap
displayName: Install LXC and Bubblewrap
sudo apt-get install -y -qq lxc lxc-utils dnsmasq-base iptables bubblewrap slirp4netns
displayName: Install LXC, Bubblewrap, and slirp4netns

- script: sudo MXC_SKIP_LXC_NETWORK_TESTS=1 MXC_DEBUG=${{ parameters.debug }} npm test
workingDirectory: $(integrationDirectory)
Expand Down
2 changes: 1 addition & 1 deletion .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,7 @@ The Rust workspace (`src/`) implements multiple sandboxing backends behind the `
| WSLc | `wxc-exec.exe` | Windows | `backends/wslc/common/src/` β€” feature-gated behind `wslc`, experimental, uses the WSLc SDK (`wslcsdk.dll`, loaded at runtime) to run Linux containers in a WSL2 VM. Supports both one-shot (`WSLContainerRunner`, via `ScriptRunner` + streaming `SandboxBackend`) and state-aware (`state_aware.rs` `WslcStateAwareRunner`, via `StatefulSandboxBackend`) modes. Because the WSLc SDK has **no cross-process re-attach**, state-aware keeps the session (VM) + container warm across separate `wxc-exec` phase processes behind a persistent per-user daemon (`wxc-wslc-daemon.exe`, `backends/wslc/daemon/`) that owns the live `WslcSession`/`WslcContainer` handles; phase processes are thin named-pipe clients (`daemon_client.rs`). The daemon runs all SDK calls on one apartment-affine worker thread (so exec is currently serialized across sandboxes β€” see `docs/wsl/wslc-state-aware.md`). Honors `readwritePaths`/`readonlyPaths` at provision (β†’ container volumes) + `network.defaultPolicy` (`Block`β†’`None`, `Allow`β†’`Bridged`; networking is all-or-nothing β€” no per-host filtering, since the container lacks `CAP_NET_ADMIN`); rejects `deniedPaths` nested under a mount and rejects proxy/host-filtering at provision. exec honors `network.proxy` **url-form only** (injected as `HTTP_PROXY`/`HTTPS_PROXY`); start/stop/deprovision reject all policy. ID prefix `wslc` (`wslc:<32-hex>`). Idle-timeout is env-overridable via `MXC_WSLC_DAEMON_IDLE_TIMEOUT_SECS`/`MXC_WSLC_DAEMON_IDLE_POLL_SECS`. See `docs/wsl/wslc-state-aware.md`. |
| LXC | `lxc-exec` | Linux | `core/lxc/src/main.rs` + `backends/lxc/common/` |
| Seatbelt | `mxc-exec-mac` | macOS | `core/mxc_darwin/src/main.rs` + `backends/seatbelt/common/` β€” uses macOS App Sandbox (Seatbelt) profiles for process containment. Requires schema `0.7.0-alpha`+. Supports `network.proxy` via the same cooperative env-var model as Bubblewrap (injects `HTTP_PROXY`/`HTTPS_PROXY` into the sandbox, reusing `wxc_common::unix_proxy_coordinator`; `builtinTestServer` spawns the shared `unix-test-proxy`). See `docs/macos-support/seatbelt-backend.md`. |
| Bubblewrap | `lxc-exec` | Linux | `backends/bubblewrap/common/src/bwrap_runner.rs` β€” unprivileged sandboxing via Linux user namespaces and `bwrap`. Experimental β€” requires `--experimental`. Uses shared filesystem/network policy fields; per-host network filtering via `NetworkIptablesManager` from `backends/lxc/common`. See `docs/bwrap-support/bubblewrap-backend.md`. |
| Bubblewrap | `lxc-exec` | Linux | `backends/bubblewrap/common/src/bwrap_runner.rs` β€” unprivileged sandboxing via Linux user namespaces and `bwrap`. Experimental β€” requires `--experimental`. Uses shared filesystem/network policy fields; per-host network filtering via `NetworkIptablesManager` from `backends/lxc/common`. For schema 0.8+, proxy mode uses a private network namespace with rootless `slirp4netns` routing and fails if `slirp4netns` is unavailable; schema 0.6/0.7 and absent-version requests retain the legacy shared-network proxy behavior. See `docs/bwrap-support/bubblewrap-backend.md`. |

### Config flow

Expand Down
13 changes: 12 additions & 1 deletion .github/workflows/Build.Linux.Job.yml
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ jobs:
working-directory: ${{ github.workspace }}
run: |
sudo apt-get update
sudo apt-get install -y bubblewrap
sudo apt-get install -y bubblewrap slirp4netns iptables
# Ubuntu 24.04 runners restrict unprivileged user namespaces via
# AppArmor, which blocks `bwrap --unshare-user`. Relax it so the
# sandbox can start (no-op on kernels without this knob).
Expand Down Expand Up @@ -119,6 +119,17 @@ jobs:
run: cargo build --locked --release --target ${{ matrix.target }}
-p unix_test_proxy

# Exercises the real binary against the real dependencies: launches the
# slirp4netns supervisor, joins its user namespace, and asserts the
# sandbox lands in a private network namespace. Needs unix-test-proxy
# (builtinTestServer) alongside lxc-exec, which the two build steps
# above place in the same target directory.
- name: Test Bubblewrap proxy networking (end-to-end)
working-directory: ${{ github.workspace }}
env:
LXC_EXEC: ${{ github.workspace }}/src/target/${{ matrix.target }}/release/lxc-exec
run: bash tests/scripts/run_bwrap_network_proxy_test.sh

Comment on lines +122 to +132

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: FYI Elliot (@theelliotm) you might need to add slirp4netns to your prepare linux host script.

- name: Upload binaries
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
Expand Down
95 changes: 78 additions & 17 deletions docs/bwrap-support/bubblewrap-backend.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,22 @@ requiring root privileges or a container runtime.
newer** is required. Platform detection probes `bwrap --version` and reports
the backend as unavailable β€” with the detected version β€” when the host is
below that floor.
- **Schema 0.8 proxy mode only:** `slirp4netns` installed and on PATH. It is
not required when `network.proxy` is omitted or when a 0.6/0.7 policy uses
the legacy proxy behavior.
```bash
# Debian/Ubuntu
sudo apt install slirp4netns

# Fedora/RHEL
sudo dnf install slirp4netns

# Alpine
apk add slirp4netns
```
Proxy mode fails explicitly if `slirp4netns` is unavailable; it never falls
back to sharing the host network namespace. The host must also provide the
util-linux `unshare` command with `--map-current-user` and `--keep-caps`.
- User namespaces must be enabled:
```bash
# Check: should print "1"
Expand Down Expand Up @@ -219,7 +235,7 @@ namespace choice alone decides the outcome:

| `allowLocalNetwork` | Namespace | Result |
|---------------------|-----------|--------|
| `false` (default) | private (`--unshare-net`) | Honored at the sandbox boundary β€” nothing outside can reach in. `bind()`/`listen()` still succeed on the sandbox's own loopback, so its processes can talk to each other; that is already inside the caller's trust boundary |
| `false` (default) | private (`--unshare-net`, including 0.8 proxy mode) | Honored at the sandbox boundary β€” nothing outside can reach in. `bind()`/`listen()` still succeed on the sandbox's own loopback, so its processes can talk to each other; that is already inside the caller's trust boundary |
| `false` | shared with host | **Not honored** β€” the process can bind/listen on host-local addresses |
| `true` | private (`--unshare-net`) | **Partially honored** β€” the listener is reachable only from inside the sandbox |
| `true` | shared with host | Honored |
Expand All @@ -244,12 +260,19 @@ Standard `process` fields work as expected:
}
```

## Network proxy (cooperative, unprivileged)
## Network proxy (private namespace, unprivileged)

Bubblewrap supports an **unprivileged, cooperative network proxy** that
enforces `allowedHosts` / `blockedHosts` at the proxy layer instead of via
iptables. This is the **recommended** way to do per-host filtering on
Bubblewrap because it requires **no root and no `CAP_NET_ADMIN`**.
host-level iptables. The workload runs in a private network namespace and
reaches the proxy through rootless `slirp4netns` routing. This requires no root
privileges.

This private-network behavior applies to schema **0.8 and later**. Policies
using schema 0.6 or 0.7 retain the existing shared-host-network proxy behavior
for compatibility and do not require `slirp4netns`. An absent schema version is
also treated as legacy. The runner never silently falls back: a 0.8 proxy
request fails if its private namespace cannot be configured.

### How it works

Expand All @@ -258,17 +281,19 @@ Bubblewrap because it requires **no root and no `CAP_NET_ADMIN`**.
`unix-test-proxy` binary is used (`builtinTestServer: true`,
testing-only and gated behind `--allow-testing-features`); in production callers
supply their own proxy via `localhost: <port>` or `url: <url>`.
2. The sandbox is then started **without** `--unshare-net` so the sandbox
shares the host network namespace and can reach the loopback proxy.
3. The command builder sets `HTTP_PROXY`, `HTTPS_PROXY`, `ALL_PROXY`,
2. The runner creates a same-UID user-namespace supervisor, starts Bubblewrap
with `--unshare-net`, and keeps the workload behind a startup barrier.
3. The supervisor attaches `slirp4netns` to Bubblewrap's private network
namespace. Host-loopback proxy endpoints are presented to the sandbox
through slirp's `10.0.2.2` host gateway. The workload starts only after
slirp reports that `tap0` is configured.
4. The command builder sets `HTTP_PROXY`, `HTTPS_PROXY`, `ALL_PROXY`,
`FTP_PROXY`, and their lowercase variants inside the sandbox via
`bwrap --setenv` (caller-supplied values for these keys, including
`NO_PROXY` / `no_proxy`, are stripped before injection). The runner deliberately
does **not** set `NO_PROXY`: since the sandbox shares the host netns,
a `NO_PROXY=localhost,127.0.0.1` entry would let cooperating clients
bypass the proxy for host-loopback destinations, defeating
`allowedHosts` / `blockedHosts` enforcement for those targets.
4. Cooperative tools (curl, wget, Python `requests`, Node `https`, etc.)
`NO_PROXY` / `no_proxy`, are stripped before injection). The runner
deliberately does **not** set `NO_PROXY`, because exempt destinations would
bypass the configured proxy policy.
5. Cooperative tools (curl, wget, Python `requests`, Node `https`, etc.)
honor the env vars and traffic flows through the proxy, which applies
the `allowedHosts` / `blockedHosts` lists.

Expand Down Expand Up @@ -305,11 +330,47 @@ Bubblewrap because it requires **no root and no `CAP_NET_ADMIN`**.

### Caveats

- **Host loopback moves to the gateway address (breaking change in 0.8+
proxy mode)**: the sandbox gets its own network namespace, so inside it
`127.0.0.1` now means *the sandbox itself*, not the host. A config that
reaches a host-local service by loopback address β€” a database on
`127.0.0.1:5432`, a metadata endpoint, a second proxy β€” silently stops
connecting to the host and starts connecting to nothing. Under the schema
0.6/0.7 legacy proxy path the sandbox shared the **host's own network
namespace**, so `127.0.0.1` did reach the host; that is the behavior
changing here.

Host-local services remain reachable, but at slirp's gateway address
`10.0.2.2` instead β€” which is exactly how the runner rewrites a
`localhost` proxy endpoint so the sandbox can still find it.

**This reachability is not limited to the configured proxy.** slirp runs
without `--disable-host-loopback`, so the workload can open a connection
to *any* service bound to host loopback via `10.0.2.2:<port>` β€” a local
database, a metadata endpoint, an unrelated daemon. That is a deliberate
exception to the private-network boundary and a more sensitive one than
generic outbound internet egress, because host-loopback services often
assume that only host-local callers can reach them. `--disable-host-loopback`
would close the path, but it would also break the proxy rewrite above, so
the gateway stays reachable until a single-port forwarding mechanism
replaces it. Restricting egress to the configured proxy endpoint is the
job of the proxy-only enforcement work that builds on this change.
- **The supervisor's user namespace is visible to the sandbox**: in proxy
mode `bwrap` joins the supervisor's user namespace via `--userns` rather
than creating its own, and the namespace descriptor stays open in the
workload β€” `bwrap` keeps it across its own `fork`/`exec` and offers no flag
to close it. Re-entering the namespace with `setns` requires
`CAP_SYS_ADMIN`, which the sandbox cannot hold: `bwrap` empties the
capability bounding set before `exec`, so the workload runs with
`CapBnd`/`CapEff`/`CapPrm` all zero. The end-to-end test suite asserts
those are zero, because that assumption is what makes the exposed
descriptor inert.
- **Cooperative model**: the runner enforces by injecting
`HTTP_PROXY` / `HTTPS_PROXY` into the sandbox environment, so only
well-behaved clients that honor those vars are routed through the
proxy. Tools that bypass them (raw sockets, custom HTTP clients,
statically-linked binaries that ignore the env) are **not enforced**.
statically-linked binaries that ignore the env) can still use slirp's direct
egress and are **not yet enforced**.
This applies to **both** the builtin test proxy and external (BYO)
proxy modes β€” the limitation is in the env-var injection mechanism,
not in the proxy itself; a BYO proxy can do whatever it likes for
Expand Down Expand Up @@ -356,9 +417,9 @@ resolution.
|--------|-----|------------|
| Privileges | Root required | Unprivileged (user namespaces) |
| Rootfs | Downloads distro rootfs | Bind-mounts host filesystem |
| Startup | Create β†’ Start β†’ Attach | Single `bwrap` exec |
| Network isolation | iptables + veth | `--unshare-net` or iptables |
| Dependencies | `lxc-*` tools, templates | Single `bwrap` binary |
| Startup | Create β†’ Start β†’ Attach | Single `bwrap` exec (proxy mode adds a namespace supervisor) |
| Network isolation | iptables + veth | `--unshare-net`, private netns + slirp4netns, or iptables |
| Dependencies | `lxc-*` tools, templates | `bwrap`; proxy mode also needs `slirp4netns` and util-linux `unshare` |
| Lifecycle | Create/destroy containers | Process dies on exit |

**When to use Bubblewrap:**
Expand Down
1 change: 1 addition & 0 deletions src/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion src/backends/bubblewrap/common/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@ lxc_common = { workspace = true }
nix = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
tempfile = { workspace = true }
thiserror = { workspace = true }
url = { workspace = true }

[dev-dependencies]
tempfile = { workspace = true }
Loading
Loading