Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitmodules
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[submodule "src/sandboxd"]
path = src/sandboxd
url = https://github.com/inclusionAI/sandboxd.git
url = https://github.com/akernel-dev/sandboxd.git
[submodule "src/distill-fs"]
path = src/distill-fs
url = https://github.com/inclusionAI/distill-fs.git
Expand Down
26 changes: 23 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,11 @@ AKernel provides cluster-backed remote sandbox environments for agents and
developer workflows. The current public user-facing surface is the Python
`akernel-sdk`, including the `akernel_sdk.Sandbox` API and the `ak` CLI.
The default sandbox runtime is gVisor runsc; callers may select Kata
Containers when the cluster has a KVM-capable node. Experimental whole-device
NVIDIA GPU and configurable writable-storage requests currently require
runsc.
Containers when the cluster has a KVM-capable node. Creation-time network
policies support unrestricted networking, blocking all traffic except the
YuanRong control proxy, or denying exact and leading-wildcard DNS names.
Experimental whole-device NVIDIA GPU and configurable writable-storage
requests currently require runsc.

Use AKernel when a task needs an isolated remote environment with command
execution, file operations, interactive PTYs, port forwarding, or reverse
Expand Down Expand Up @@ -170,6 +172,15 @@ runsc; a Kata request fails scheduling with a no-resource error when no
eligible node exists. Do not treat a configured runtime as an advertised
runtime.

The bundled sandboxd configuration enables per-sandbox network ACLs.
ACL-capable nodes require eBPF `SCHED_CLS`, TC `clsact`, writable bpffs,
permission to load BPF programs and manage TC filters, and free TCP/UDP port
53 on the sandbox bridge. Drain existing sandboxes before enabling ACLs or
upgrading a node from a pre-ACL configuration; sandboxd refuses to initialize
ACLs when old sandbox records remain. A sandbox without a network policy stays
unrestricted. See `deploy/README.md` for deployment requirements and
`sdk/python/README.md` for API limits.

Dragonfly distribution is optional and disabled by default. Enable it during
profile generation with `make config INSTALL_DRAGONFLY=true`. This installs the
pinned public chart and, by default, creates three seed nodes and one server
Expand Down Expand Up @@ -257,6 +268,15 @@ with Sandbox(xpu="gpu:l20:1", storage_mb=20 * 1024) as sb:
print(sb.commands.run("nvidia-smi -L").stdout)
```

Configure a creation-time network policy:

```python
from akernel_sdk import NetworkPolicy, Sandbox

with Sandbox(network=NetworkPolicy.block()) as sb:
print(sb.commands.run("echo control-plane-access").stdout)
```

Required environment:

```bash
Expand Down
10 changes: 10 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,16 @@ with Sandbox(cpu=2000, memory=4096) as sb:
print(result.stdout)
```

Creation-time network policies can either preserve unrestricted networking,
block all traffic except the YuanRong control proxy, or deny selected DNS
names:

```python
from akernel_sdk import NetworkPolicy

sandbox = Sandbox(network=NetworkPolicy.deny_dns("github.com", "*.github.com"))
```

### One-Click Deployment: From Laptop to Multi-Cloud

One all-in-one image, multiple deployment targets — deploy in under 10 minutes:
Expand Down
21 changes: 21 additions & 0 deletions deploy/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,27 @@ The node must support TC eBPF and bpffs. bpfnat does not manage host firewall
policy, so custom host-network deployments must allow forwarding to and from
the sandbox bridge when their `FORWARD` policy is `DROP`.

### Network ACLs

The bundled standalone, Helm, and Terraform sandboxd configurations enable
per-sandbox network ACLs. A sandbox created without a policy remains on the
unrestricted fast path. ACL nodes require Linux eBPF `SCHED_CLS`, TC
`clsact`, supported hash and array maps, a writable bpffs at
`/sys/fs/bpf` (or permission to mount one), and permission to load BPF
programs and manage TC filters. TCP and UDP port 53 on the sandbox bridge
must be free, and sandboxd must have at least one usable upstream nameserver.
AKernel's node container is privileged so it can meet these requirements.

Drain all sandboxes from a node before enabling ACLs or upgrading an existing
deployment to a release that enables them. Sandboxd deliberately refuses to
start ACL support when its store contains pre-ACL sandboxes, preventing a
silent fail-open migration. Start new sandboxes only after the upgraded
sandboxd is healthy.

ACL enforcement is independent of the selected `iptables` or `bpfnat` NAT
backend. DNS policies manage each sandbox's `/etc/resolv.conf`; a caller
mount that owns that path is rejected while ACL support is enabled.

`make config` is interactive by default. It writes:

- `.akernel/default/config.env`
Expand Down
1 change: 1 addition & 0 deletions deploy/akernel/charts/core/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -464,6 +464,7 @@ node:
[plugin.network]
ip_range="172.17.0.1/16"
nat_backend="iptables"
enable_network_acl=true

[plugin.resource]
cgroup_cache_size=800
Expand Down
8 changes: 8 additions & 0 deletions deploy/standalone/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,14 @@ later creation of `sandbox0` cannot change the advertised node address. Set
`AKERNEL_NODE_IP` only when a multi-homed deployment requires an explicit
override.

The standalone configuration enables per-sandbox network ACLs. Its privileged
node container can mount bpffs and manage the required eBPF TC filters. TCP
and UDP port 53 on the sandbox bridge must remain free for sandboxd's managed
DNS proxy. Before upgrading an existing standalone data directory to an
ACL-enabled image, terminate its sandboxes and stop the old node cleanly;
sandboxd refuses to initialize ACLs while pre-ACL sandboxes remain in its
store.

## Directory Structure

```
Expand Down
1 change: 1 addition & 0 deletions deploy/standalone/config/sandboxd_config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ stream_server_port=""
ip_range="10.88.0.1/16"
# start.sh overrides this value when AKERNEL_NAT_BACKEND is set.
nat_backend="iptables"
enable_network_acl=true
# The all-in-one frontend shares this network namespace with sandboxd and
# reaches forwarded sandbox ports through the node-local address.
enable_local_dnat=true
Expand Down
1 change: 1 addition & 0 deletions deploy/terraform/aliyun/values-akernel.yaml.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,7 @@ node:
[plugin.network]
ip_range="172.17.0.1/16"
nat_backend="${sandboxd_nat_backend}"
enable_network_acl=true

[plugin.resource]
cgroup_cache_size=800
Expand Down
1 change: 1 addition & 0 deletions deploy/terraform/huaweicloud/values-akernel.yaml.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,7 @@ node:
[plugin.network]
ip_range="172.17.0.1/16"
nat_backend="${sandboxd_nat_backend}"
enable_network_acl=true

[plugin.resource]
cgroup_cache_size=800
Expand Down
61 changes: 61 additions & 0 deletions sdk/python/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ It supports two backends:
- [Install and configure](#install-and-configure)
- [Create a sandbox](#create-a-sandbox)
- [Experimental GPU and writable storage](#experimental-gpu-and-writable-storage)
- [Network ACLs](#network-acls)
- [Sandbox runtimes](#sandbox-runtimes)
- [Commands](#commands)
- [Filesystem](#filesystem)
Expand Down Expand Up @@ -99,6 +100,7 @@ Sandbox(
*,
xpu: str | None = None,
storage_mb: int | None = None,
network: NetworkPolicy | None = None,
)
```

Expand Down Expand Up @@ -129,6 +131,63 @@ default 10 GiB memory-backed writable overlay. See
[`examples/gpu_sandbox.py`](./examples/gpu_sandbox.py) and
[`examples/storage_sandbox.py`](./examples/storage_sandbox.py).

### Network ACLs

Omit `network` to leave all sandbox networking unrestricted. An empty
`NetworkPolicy()` is equivalent and is omitted from the creation request:

```python
from akernel_sdk import NetworkPolicy, Sandbox

with Sandbox() as unrestricted:
print(unrestricted.commands.run("python3 -c 'import socket; "
"socket.getaddrinfo(\"github.com\", 443)'"))
```

Block all sandbox traffic except the YuanRong control proxy:

```python
with Sandbox(network=NetworkPolicy.block()) as sandbox:
result = sandbox.commands.run("printf 'control plane still works'")
assert result.exit_code == 0
```

Commands and lifecycle operations continue to work in block mode. The SDK
also falls back from its direct filesystem data path to RuntimeRPC transfers,
so file reads, writes, and copies keep working but bulk transfers can be
slower.

Deny conventional DNS lookups for exact names or leading `*.` suffix
patterns:

```python
policy = NetworkPolicy.deny_dns("github.com", "*.github.com")
with Sandbox(network=policy) as sandbox:
blocked = sandbox.commands.run(
"python3 -c 'import socket; socket.getaddrinfo(\"github.com\", 443)'"
)
assert blocked.exit_code != 0
```

An exact pattern matches only that name. For example, `github.com` does not
match `api.github.com`, while `*.github.com` matches descendants but not
the apex. Supply both when both should be denied. Patterns are normalized to
lower case without a trailing dot; international names must use ASCII
punycode.

Network policies are fixed when a sandbox is created. `block_network` and
`dns_blacklist` cannot be combined in the current SDK. DNS blacklists cover
ordinary UDP and TCP DNS and return a refused response for blocked queries;
DNS-over-HTTPS and connections to a known IP are outside their scope. The
packet ACL is currently IPv4 and stateless.

See [`examples/network_policy.py`](./examples/network_policy.py) for all
three modes. Deployment nodes must have network ACL support enabled; the
bundled standalone, Helm, and Terraform configurations enable it. Drain
existing sandboxes before upgrading a node to an ACL-enabled sandboxd
configuration, as described in the
[deployment guide](../../deploy/README.md#network-acls).

## Sandbox runtimes

AKernel uses the gVisor `runsc` runtime when `runtime` is omitted. Callers may also select `runsc` explicitly or request Kata Containers:
Expand Down Expand Up @@ -389,6 +448,7 @@ Maintained examples are under [`examples/`](./examples):
- `custom_image.py`
- `gpu_sandbox.py`
- `named_sandbox.py`
- `network_policy.py`
- `pty.py`
- `port_forwarding.py`
- `reverse_tunnel.py`
Expand Down Expand Up @@ -425,3 +485,4 @@ not part of the default test suite.
| `S3Config` | `endpoint`, `bucket`, `object`, optional credentials |
| `Mount` | `target`, one source, and `type` |
| `HttpReverseTunnel` | `target`, `reverse_port`, `listen_port`, `connect_timeout` |
| `NetworkPolicy` | `block_network`, `dns_blacklist` |
2 changes: 2 additions & 0 deletions sdk/python/akernel_sdk/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
EntryInfo,
HttpReverseTunnel,
Mount,
NetworkPolicy,
NodeInfo,
S3Config,
SandboxInfo,
Expand All @@ -38,6 +39,7 @@
"Sandbox",
"S3Config",
"Mount",
"NetworkPolicy",
"HttpReverseTunnel",
"CommandResult",
"CommandInfo",
Expand Down
2 changes: 2 additions & 0 deletions sdk/python/akernel_sdk/_backends/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
EntryInfo,
HttpReverseTunnel,
Mount,
NetworkPolicy,
S3Config,
SandboxInfo,
)
Expand Down Expand Up @@ -75,6 +76,7 @@ class SandboxSpec:
node_id: str | None
xpu: str | None
storage_mb: int | None
network: NetworkPolicy | None


class CommandsDriver(Protocol):
Expand Down
9 changes: 8 additions & 1 deletion sdk/python/akernel_sdk/_backends/openyuanrong_sandbox.py
Original file line number Diff line number Diff line change
Expand Up @@ -327,7 +327,7 @@ def create(self, spec: SandboxSpec) -> BackendSession:
self._validate(spec)
sandbox_type = yr_sandbox.Sandbox
if spec.runtime == "kata" and spec.image is None and spec.rootfs is None:
# openyuanrong-sandbox 0.9.3 forwards the isolation runtime only
# openyuanrong-sandbox forwards the isolation runtime only
# through an explicit rootfs. Keep this aligned with the actor
# backend until frontend can override the service rootfs runtime.
sandbox_type = _LocalRootfsSandbox
Expand All @@ -340,6 +340,12 @@ def create(self, spec: SandboxSpec) -> BackendSession:
access_key=spec.rootfs.access_key,
secret_key=spec.rootfs.secret_key,
)
network = None
if spec.network is not None:
network = yr_sandbox.NetworkPolicy(
block_network=spec.network.block_network,
dns_blacklist=spec.network.dns_blacklist,
)
mounts = [
yr_sandbox.Mount(
target=mount.target,
Expand Down Expand Up @@ -390,6 +396,7 @@ def create(self, spec: SandboxSpec) -> BackendSession:
node_id=spec.node_id,
xpu=spec.xpu,
storage_mb=spec.storage_mb,
network=network,
create_timeout=create_timeout,
)
except Exception as error:
Expand Down
1 change: 1 addition & 0 deletions sdk/python/akernel_sdk/_backends/openyuanrong_sdk.py
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,7 @@ def create(self, spec: SandboxSpec) -> BackendSession:
node_id=spec.node_id,
xpu=spec.xpu,
storage_mb=spec.storage_mb,
network=spec.network,
)
try:
instance = _impl.create_instance(
Expand Down
11 changes: 10 additions & 1 deletion sdk/python/akernel_sdk/_backends/openyuanrong_sdk_impl.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,13 @@
validate_storage_mb,
xpu_custom_resource,
)
from ..types import HttpReverseTunnel, Mount, NodeInfo, S3Config
from ..types import (
HttpReverseTunnel,
Mount,
NetworkPolicy,
NodeInfo,
S3Config,
)

logger = logging.getLogger(__name__)

Expand Down Expand Up @@ -159,6 +165,7 @@ def build_options(
node_id: str | None,
xpu: str | None,
storage_mb: int | None,
network: NetworkPolicy | None,
) -> Any:
"""Translate the stable SDK configuration to openYuanrong options."""

Expand Down Expand Up @@ -209,6 +216,8 @@ def build_options(
options.custom_resources[resource_name] = count
if storage_mb is not None:
options.custom_resources["storage"] = storage_bytes(storage_mb)
if network is not None:
options.custom_extensions["network_policy"] = json.dumps(network.to_dict())

forwarded = list(port_forwardings)
if reverse_tunnel is not None:
Expand Down
8 changes: 7 additions & 1 deletion sdk/python/akernel_sdk/sandbox.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
from .commands import Commands
from .filesystem import Filesystem
from .pty import Pty
from .types import HttpReverseTunnel, Mount, S3Config, SandboxInfo
from .types import HttpReverseTunnel, Mount, NetworkPolicy, S3Config, SandboxInfo

_SUPPORTED_RUNTIMES = ("runsc", "kata")
_traefik_internal_ip_cache: str | None = None
Expand Down Expand Up @@ -134,6 +134,7 @@ def __init__(
*,
xpu: str | None = None,
storage_mb: int | None = None,
network: NetworkPolicy | None = None,
) -> None:
"""Create and wait for a sandbox to become ready.

Expand Down Expand Up @@ -161,6 +162,8 @@ def __init__(
storage_mb: Experimental writable root filesystem quota in MiB.
When omitted, the configured default is used. Explicit quotas
currently require the ``runsc`` runtime.
network: Optional creation-time network policy. Omitting it leaves
sandbox networking unrestricted.

Raises:
TypeError: An argument has an invalid type.
Expand All @@ -181,6 +184,8 @@ def __init__(
)
normalized_xpu = normalize_xpu(xpu)
validate_storage_mb(storage_mb)
if network is not None and not isinstance(network, NetworkPolicy):
raise TypeError("network must be a NetworkPolicy or None")
if normalized_xpu is not None and runtime != "runsc":
raise ValueError("xpu is currently supported only by runsc")
if storage_mb is not None and runtime != "runsc":
Expand Down Expand Up @@ -265,6 +270,7 @@ def __init__(
node_id=node_id,
xpu=normalized_xpu,
storage_mb=storage_mb,
network=None if network is None or network.is_empty else network,
)
self._session = load_backend().create(spec)
try:
Expand Down
Loading
Loading