[LXC] State-aware sandbox lifecycle

[Darren Hoehna (dhoehna)]

📖 Description

Implements StatefulSandboxBackend for LXC, so a container can be driven through provision, start, exec, stop, and deprovision rather than only the one-shot spawn path.

• Adds the five lifecycle phases, with start serialized per container under an advisory flock.
• Adds network policy enforcement in both directions: the egress FORWARD chain in the host netns before the container starts, and the inbound INPUT chain inside the container's own netns once it is running.
• Adds ContainerPolicy::requires_firewall(), so enforcement follows the policy — a block default, a non-empty host list, or a proxy — and a start with no network section denies by default.
• Adds lxc to the Node SDK as a state-aware backend, with per-phase config types and LxcNetworkConfig narrowed to the three fields LXC enforces.
• Adds two E2E scripts run against a live LXC host: the provision-through-deprovision lifecycle, and a network matrix over 15 cases and 16 checked-in configs.

🔍 Validation

• cargo test --workspace --no-fail-fast — 2,785 passed, 2 failed. Both failures are pre-existing wslc_common tests that require an unlocked D: drive and reproduce at the base commit.
• npm test (Node SDK) — 283 tests, 277 passed, 0 failed, 6 skipped.
• cargo fmt --all -- --check and cargo clippy --workspace --all-targets — clean.
• run_lxc_state_aware_network_test.sh on a live LXC host — 70 passed, 0 failed, 0 quarantined.
• run_lxc_state_aware_test.sh on a live LXC host — 8 passed, 0 failed.

✅ Checklist

• Signed the Contributor License Agreement
• Linked to an issue
• Updated documentation (if applicable)
• Updated Copilot instructions (if build, architecture, or conventions changed)
• If this PR changes Cargo.lock, the dependency-feed-check check passes

📋 Issue Type

• Bug fix
• Feature
• Task

Microsoft Reviewers: Open in CodeFlow

[Copilot]

Pull request overview

Copilot reviewed 42 out of 43 changed files in this pull request and generated no new comments.

Suppressed comments (2)

src/core/wxc_common/src/models.rs:641

• This is a parse-derived presence bit, but unlike its peers it is included when ContainerPolicy is serialized (for example by diagnostic::redacted_request_json). Mark it skipped so internal parser state does not silently change the serialized diagnostic shape; filesystem_specified, network_specified, and network_mode_specified establish this convention at lines 627, 658, and 667.

    pub default_network_policy_present: bool,

tests/scripts/run_lxc_state_aware_network_test.sh:272

• A failed deprovision is counted as a failure, but the sandbox ID is then cleared unconditionally, so the EXIT trap cannot retry cleanup and the live container can leak into later cases. Keep the ID until deprovision actually succeeds, as the lifecycle test already does in run_lxc_state_aware_test.sh:170-175.

[Copilot]

Pull request overview

Copilot reviewed 42 out of 43 changed files in this pull request and generated no new comments.

Suppressed comments (5)

sdk/node/src/state-aware.ts:198

• For LXC dry-run execs, the executor keeps stdout as the protocol channel (exec_streams_stdout is false in src/core/lxc/src/main.rs:148), but this branch always inspects stderr. A dry-run validation/deserialization failure is therefore returned as a normal ExecResult instead of a typed MxcError. Select stderr only for non-dry-run LXC execs; dry-run should parse stdout like the other envelope-only paths.

    const errorEnvelope = backendKey === 'lxc'
      ? tryParseErrorEnvelopeFromLines(stderr)
      : tryParseErrorEnvelope(stdout);

tests/scripts/run_lxc_state_aware_network_test.sh:274

• A failed deprovision still clears SANDBOX_ID, so the EXIT trap cannot retry cleanup and the test leaks the container into subsequent runs. Preserve the ID unless deprovision succeeds, as the lifecycle test already does.
  src/backends/lxc/common/src/state_aware.rs:270
• This policy gate ignores policy.ui_specified, so raw state-aware requests can supply a UI lockdown and every LXC phase reports success without enforcing it. ContainerPolicy::ui defaults to deny and the parser deliberately preserves presence, so silently accepting it makes a security policy claim that the backend does not deliver. Reject any supplied UI block with policy_validation on start and the other phases, as unsupported-policy backends do.

fn reject_start_policy_on_other_phase(
    phase: &str,
    policy: &ContainerPolicy,
) -> Result<(), MxcError> {
    if has_filesystem_policy(policy) || has_network_policy(policy) {
        return Err(MxcError::policy_validation(format!(
            "LXC state-aware {phase} does not accept filesystem or network policy; pass it to start"
        )));

src/backends/lxc/common/src/state_aware.rs:1005

• The new live matrix verifies only the host-side FORWARD chain. Removing this state-aware apply_ingress_policy call would leave every added lifecycle test green, even though the PR's key behavior is installing INPUT policy inside the container namespace. Add a state-aware live assertion that the MXCI chain and INPUT hook exist in the provisioned container's namespace (similar to run_lxc_inbound_deny_test.sh).

            // Inbound enforcement lands only once the container's network
            // namespace exists, so unlike the egress chain it comes after start.
            if let Err(e) = apply_ingress_policy(&container, container_name, request, &mut logger) {

src/backends/lxc/common/src/signal_cleanup.rs:124

• NetworkOnly always runs StopContainer before cleanup. A signal can arrive after the FORWARD chain is installed but before container.start(); in that state lxc-stop -k returns nonzero for the already-stopped container, execute_rollback aborts, and the newly installed chain is stranded. Distinguish an already-stopped container from a failed stop so pre-start cancellation can still remove the owned firewall while preserving the fail-closed behavior for an unknown/live container.

        SignalRollback::NetworkOnly => {
            plan.push(RollbackStep::StopContainer);
            if owns_firewall {
                plan.push(RollbackStep::RemoveFirewall);
            }

[Copilot]

Pull request overview

Copilot reviewed 42 out of 43 changed files in this pull request and generated 1 comment.

Suppressed comments (4)

sdk/node/src/state-aware.ts:198

• LXC dry-run exec failures are emitted on stdout (exec_streams_stdout is false when dry_run), but this branch always inspects stderr. Consequently execInSandboxAsync(..., { dryRun: true }) returns an ordinary nonzero ExecResult instead of throwing the typed validation error. Select stderr only for non-dry-run LXC execs.

    const errorEnvelope = backendKey === 'lxc'
      ? tryParseErrorEnvelopeFromLines(stderr)
      : tryParseErrorEnvelope(stdout);

src/backends/lxc/common/src/signal_cleanup.rs:101

• This safety explanation contradicts the implemented rollback plan: the one-shot path no longer explicitly removes ingress rules before destroy; destroy removes their network namespace. The later DestroyContainer comment states the opposite correctly. Update this paragraph so future ordering changes are not based on the obsolete behavior.

/// The inbound rules are different in kind. They live inside the container's
/// own network namespace, reachable only by entering it through the init PID,
/// so they cease to exist when the container does. That is why only the
/// one-shot path removes them, and why it does so before `destroy` -- after
/// that there is no namespace left to enter. The stop path deliberately omits

tests/scripts/run_lxc_state_aware_network_test.sh:272

• The sandbox ID is cleared even when deprovision fails, so the EXIT trap cannot retry cleanup and the failed case can leak a container into subsequent runs. Retain the ID unless deprovision succeeds, matching the lifecycle test's cleanup behavior.
  src/core/wxc_common/src/models.rs:641
• This parse-derived presence bit is the only such ContainerPolicy field not marked #[serde(skip)] (unlike filesystem_specified, network_specified, and network_mode_specified). Serializing the public ExecutionRequest therefore exposes it, and deserializing can accept a caller-supplied value, contradicting the documented “never on the wire” contract.

    pub default_network_policy_present: bool,

[Copilot]

Pull request overview

Copilot reviewed 42 out of 43 changed files in this pull request and generated no new comments.

Suppressed comments (3)

src/backends/lxc/common/src/state_aware.rs:1005

• The live state-aware matrix exercises the new egress path only: its default-deny assertions inspect host FORWARD hooks and the terminal DROP, but nothing verifies that this state-aware call installs the container-netns INPUT chain or rolls back a failed ingress apply. The existing inbound test covers the one-shot runner, so this wiring could be removed while the new lifecycle suite remains green. Add a live state-aware inbound assertion (and a failed-apply rollback case) before relying on this security boundary.

            if let Err(e) = apply_ingress_policy(&container, container_name, request, &mut logger) {

tests/scripts/run_lxc_state_aware_network_test.sh:274

• If deprovision fails, this still clears SANDBOX_ID, so the EXIT trap cannot retry and the test leaks the container it just reported failing to remove. Keep the ID until deprovision succeeds, as the lifecycle test already does.
  src/core/wxc_common/src/models.rs:641
• This is documented as parse-derived and “never on the wire,” but unlike every adjacent presence bit it lacks serde(skip). Serializing the public ContainerPolicy therefore exposes default_network_policy_present, and deserialization accepts callers setting this internal parser fact. Skip it so only convert_wire_config can establish presence.

    pub default_network_policy_present: bool,

[Copilot]

Pull request overview

Copilot reviewed 42 out of 43 changed files in this pull request and generated no new comments.

Suppressed comments (3)

sdk/node/src/state-aware.ts:198

• For LXC dry-run execs, lxc-exec does not reserve stdout for guest output: exec_streams_stdout is false when dry_run is set, so dispatch errors are emitted on stdout. This branch always checks only stderr for LXC, causing execInSandboxAsync(..., { dryRun: true }) to return an ordinary nonzero result instead of throwing the typed dispatch error. Select stderr only for non-dry-run LXC execs.

    const errorEnvelope = backendKey === 'lxc'
      ? tryParseErrorEnvelopeFromLines(stderr)
      : tryParseErrorEnvelope(stdout);

tests/scripts/run_lxc_state_aware_network_test.sh:390

• The live state-aware default-deny check verifies only the host FORWARD hooks and terminal DROP. It never enters this started container's network namespace to assert that the new state-aware path installed and hooked its INPUT chain; the existing inbound script exercises the one-shot path instead. A regression that omits apply_ingress_policy from state-aware start would therefore leave all these cases green despite inbound being open. Add a namespace-level INPUT assertion for this case (and IPv6 when active).
  tests/scripts/run_lxc_state_aware_network_test.sh:274
• These state resets run even when stop or deprovision fails. In that case the EXIT trap no longer retries the failed phase, so this root-run test can leak a running container and its firewall state into later cases or test runs. Clear each flag only after the corresponding phase succeeds, as the lifecycle test already does for deprovision.

[Copilot]

Pull request overview

Copilot reviewed 42 out of 43 changed files in this pull request and generated no new comments.

Suppressed comments (2)

tests/scripts/run_lxc_state_aware_network_test.sh:272

• A failed deprovision is counted but then SANDBOX_ID is cleared. The next case can overwrite the only handle to that container, so the EXIT trap cannot retry cleanup and the test leaks a container. Abort while retaining the ID when deprovision fails; only clear it after success.
  src/core/wxc_common/src/models.rs:641
• This parse-derived presence bit is serialized by ContainerPolicy, even though its documentation says it is never on the wire. The neighboring filesystem_specified, network_specified, and ui_specified fields are all skipped; leaving this one exposed leaks internal parser state into serialized policies and makes round-trips inconsistent. Mark it with #[serde(skip)] as well.

    pub default_network_policy_present: bool,

[Copilot]

Pull request overview

Copilot reviewed 42 out of 43 changed files in this pull request and generated 1 comment.

Suppressed comments (2)

src/backends/lxc/common/src/signal_cleanup.rs:405

• A signal can arrive after the FORWARD chain is created but before container.start(). In that state lxc-stop -k reports that the container is not running, this step returns false, and execute_rollback deliberately skips RemoveFirewall, stranding the chain and blocking the next start. Treat a positively stopped container as a successful stop; preserve the current fail-closed behavior when the state probe is unknown or killing a live container fails.

                RollbackStep::StopContainer => LxcContainer::new(&name, None).kill().is_ok(),

tests/scripts/run_lxc_state_aware_network_test.sh:272

• This clears SANDBOX_ID even when deprovision fails. The next case then overwrites the only identifier the EXIT trap could retry, leaking the live container and potentially its firewall state. Stop the matrix on teardown failure while leaving the ID armed so the trap gets one final cleanup attempt.

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[Copilot]

Pull request overview

Copilot reviewed 48 out of 49 changed files in this pull request and generated no new comments.

Suppressed comments (3)

tests/scripts/run_lxc_state_aware_network_test.sh:314

• This clears SANDBOX_ID even when deprovision fails, so the EXIT trap cannot retry and the failed case leaks its container into subsequent cases/runs. The lifecycle test already preserves the ID on failure for this reason; do the same here.
  tests/scripts/run_lxc_state_aware_network_test.sh:434
• The assert_default_deny path verifies only the host-side FORWARD chain. If the new apply_ingress_policy call were removed, cases 3/10/11/12 would still pass, so the state-aware INPUT-chain behavior claimed by this suite is not covered. Add a container-netns iptables assertion or an inbound traffic probe that fails when the INPUT hook/terminal DROP is absent.
  sdk/node/src/state-aware-types.ts:158
• FilesystemConfig also exposes the SDK-only clearPolicyOnExit field, but state-aware envelopes forward filesystem unchanged and Rust's wire::Filesystem rejects that field via deny_unknown_fields. As written, a type-correct LXC start can therefore fail during parsing. Narrow this property to the three actual wire fields, just as LxcNetworkConfig excludes its SDK-only field.

  filesystem?: FilesystemConfig;

[Copilot]

Pull request overview

Copilot reviewed 53 out of 54 changed files in this pull request and generated no new comments.

Suppressed comments (3)

sdk/node/src/state-aware-types.ts:126

• This contract contradicts the PR description, which says restrictive LXC start policies require enforcementMode: 'firewall' (or 'both'). The implementation and tests instead accept omitted/capabilities modes and ignore the field. Please align the PR requirements with the shipped API, or restore the stated validation; currently the intended public behavior is ambiguous.

 * - `enforcementMode` is accepted, parsed, and ignored.  LXC enforces from
 *   `defaultPolicy`, `allowedHosts`, and `blockedHosts` alone, so no
 *   combination of this field with those is worth rejecting here.

sdk/node/src/state-aware-types.ts:143

• FilesystemConfig also exposes the SDK-only clearPolicyOnExit field, but buildStateAwareEnvelope copies it into the top-level filesystem object and Rust's wire::Filesystem rejects unknown fields. A type-valid LXC start using that option therefore fails at runtime. Narrow this property to the three wire path lists, as was already done for LxcNetworkConfig to exclude SDK-only fields.

  filesystem?: FilesystemConfig;

tests/scripts/run_lxc_state_aware_network_test.sh:425

• The live state-aware matrix only inspects the host FORWARD chain. It never verifies the new container-namespace INPUT hook or terminal DROP, so removing the apply_ingress_policy call would still leave this suite green—the exact regression this PR is meant to prevent. Add a state-aware assertion that enters the provisioned container's network namespace and checks the INPUT chain (ideally with traffic-level IPv4/IPv6 probes).

[Copilot]

Pull request overview

Copilot reviewed 53 out of 54 changed files in this pull request and generated 1 comment.

Suppressed comments (2)

tests/scripts/run_lxc_state_aware_network_test.sh:305

• Do not discard the sandbox ID when deprovision fails. Clearing it here disarms the EXIT trap's retry and leaks the container/firewall state; the lifecycle test correctly retains its ID on failure. Capture the return code and clear SANDBOX_ID only after success.
  src/backends/lxc/common/src/state_aware.rs:948
• The live state-aware matrix never verifies this newly wired ingress half: it inspects only host FORWARD rules, while run_lxc_inbound_deny_test.sh exercises the one-shot path. Removing this call would leave the new matrix green. Add a state-aware check that enters the started container's netns and confirms the MXCI chain is hooked from INPUT (for both usable families).

            // Inbound enforcement lands only once the container's network
            // namespace exists, so unlike the egress chain it comes after start.
            if let Err(e) = apply_ingress_policy(&container, container_name, request, &mut logger) {

[Copilot]

Pull request overview

Copilot reviewed 54 out of 55 changed files in this pull request and generated no new comments.

Suppressed comments (4)

sdk/node/src/state-aware.ts:111

• This channel description contradicts the new executor and the implementation below: lxc-exec writes dispatch error envelopes to stdout (src/core/lxc/src/main.rs:166), and execInSandboxAsync parses stdout at line 174. The streaming API also exposes the PTY's combined data rather than selecting a backend-specific error channel. Please document stdout here so callers do not look for LXC protocol errors on stderr.

 * On dispatch failure the executor emits a single error envelope; the SDK does
 * not parse it here — callers consuming `IPty.onData` see the raw bytes. The
 * channel is backend-specific: LXC puts it on stderr, because stdout is
 * carrying the container's raw output, while Windows Sandbox, IsolationSession,
 * and WSLc keep stdout as the executor's own channel and emit it there. Use
 * `execInSandboxAsync` when typed-error throwing is needed — it already selects
 * the right channel per backend.

sdk/node/src/state-aware.ts:153

• This still claims LXC errors are read from stderr, but the changed LXC entry point emits the envelope on stdout and this function only calls tryParseErrorEnvelope(stdout). Correct the public contract to match the actual parser.

 * field set) when the executor reports a dispatch failure, recognised by
 * exit != 0 together with a complete `{error}` envelope on the channel that
 * backend's executor owns -- stderr for LXC, stdout for the others.

tests/scripts/run_lxc_state_aware_network_test.sh:305

• This clears the only cleanup handle even when deprovision fails. The next case overwrites SANDBOX_ID, so the EXIT trap cannot retry and the leaked container can contaminate later cases. Keep the ID and stop the matrix on teardown failure so the trap gets one final retry.
  .github/copilot-instructions.md:186
• “In a firewall mode” is now incorrect: this PR deliberately ignores enforcementMode and installs egress filtering whenever requires_firewall() is true, including the default-deny request with no network section. Since this file defines repository guidance, describe the policy-driven predicate instead.

| LXC | `lxc-exec` | Linux | `core/lxc/src/main.rs` + `backends/lxc/common/` — supports both **one-shot** (single-invocation lifecycle) and **state-aware** (multi-invocation provision/start/exec/stop/deprovision, via `StatefulSandboxBackend`) modes. State-aware lives in `backends/lxc/common/src/state_aware.rs` (`LxcStateAwareRunner`) and needs **no daemon**: the container outlives each phase process in the LXC runtime, so `sandboxId` is just `lxc:<containerName>` and every phase re-acquires the handle with `LxcContainer::new`. In a firewall mode the host-side veth name is derived from the container name and pinned by a container-global `lxc.hook.start-host` installed **before** start, so the iptables FORWARD chain is scoped before the interface exists; the hook resolves the container's peer interface from `$LXC_PID` and renames it to that name, and exits nonzero — aborting the start — if it cannot find one. The interface set is read from liblxc (`lxc-info -c`) rather than by parsing the container's config, so an `lxc.include` that declares interfaces elsewhere is resolved rather than refused, and no interface *index* is read at all, so a container numbering its interface `lxc.net.3` is enforced exactly as one using `lxc.net.0`. Start is refused when the container declares anything other than exactly one interface, or does not declare that interface's type as `veth` (an undeclared type is refused too, since absence is not evidence of a veth). Teardown derives that same name rather than asking `lxc-info`, which keeps reporting the name liblxc recorded before the hook renamed it, and is ownership-scoped, so a concurrent start's chain is left alone; mount cleanup clears only MXC-marked `lxc.mount.entry` lines. Inbound default-deny (`IngressManager`) is installed on both paths; on the state-aware path it goes on after `container.start()` because the chain lives in the container's network namespace and needs its init PID, and a failure there rolls the start back. Start, stop, and deprovision each take a per-sandbox `flock` in the LXC root, so a teardown cannot strip a start's firewall in the window before the container runs. See `docs/state-aware-lifecycle/mxc-state-aware-sandbox-api.md`. |

[Copilot]

Pull request overview

Copilot reviewed 54 out of 55 changed files in this pull request and generated 1 comment.

Suppressed comments (3)

tests/scripts/run_lxc_state_aware_network_test.sh:743

• This case never verifies that allowedHosts produces an ACCEPT rule: it only checks that the chain is hooked and ends in DROP, which is identical to default-deny with the allow list ignored. Add a positive connectivity check to a controlled allowed endpoint (plus a denied control), or inspect the destination-specific ACCEPT rule; case 12 has the same gap.
  tests/scripts/run_lxc_state_aware_network_test.sh:305
• A failed deprovision still clears SANDBOX_ID, so the EXIT trap cannot retry cleanup and the container leaks into later cases/runs. Preserve the ID unless deprovision succeeds, as the lifecycle test already does in tests/scripts/run_lxc_state_aware_test.sh:168-175.
  tests/scripts/run_lxc_state_aware_network_test.sh:793
• This blocklist assertion is vacuous because omitting defaultPolicy inherits default block, so evil.example.com is denied even if blockedHosts is completely ignored. Exercise the list under defaultPolicy: allow and verify both the blocked destination and an allowed control, or inspect the explicit destination DROP rule.

[Copilot]

Pull request overview

Copilot reviewed 56 out of 57 changed files in this pull request and generated no new comments.

Suppressed comments (2)

sdk/node/src/state-aware-types.ts:140

• FilesystemConfig also exposes the SDK-only clearPolicyOnExit field. buildStateAwareEnvelope forwards this object unchanged, while Rust's wire::Filesystem uses deny_unknown_fields and has no such member, so a configuration accepted by this new public type fails at runtime as malformed. Narrow the filesystem type just as the network type is narrowed.

  filesystem?: FilesystemConfig;

tests/scripts/run_lxc_state_aware_network_test.sh:307

• The sandbox ID is cleared even when deprovision fails, which disarms the EXIT trap and can leave a container behind. The same issue occurs for SANDBOX_STARTED: a failed stop is recorded as stopped, so cleanup will not retry it. Retain both states until their corresponding phase succeeds.