Skip to content

Support gateways with multiple replicas#3960

Open
jvstme wants to merge 1 commit into
masterfrom
gateway_replicas
Open

Support gateways with multiple replicas#3960
jvstme wants to merge 1 commit into
masterfrom
gateway_replicas

Conversation

@jvstme

@jvstme jvstme commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator

A gateway can now have multiple replicas for
improved availability.

type: gateway
name: example-gateway

backend: aws
region: eu-west-1

domain: example.com

certificate: null
replicas: 2

To balance requests between gateway replicas, add
DNS records for each replica or set up a load
balancer outside of dstack. Replica hostnames
are displayed in dstack CLI and UI.

$ dstack gateway list
 NAME             BACKEND          HOSTNAME        DOMAIN       DEFAULT  STATUS
 example-gateway                                   example.com  ✓        running
    replica=0     aws (eu-west-1)  34.244.128.46
    replica=1     aws (eu-west-1)  18.201.201.174

Limitations:

  • Changing the number of replicas or redeploying
    replicas is not supported.
  • HTTPS is not supported. Use an external load
    balancer for TLS termination.
  • An unavailable gateway replica prevents any new
    services or service replicas from being added.
  • All replicas are bound to the same backend and
    region.

Implementation notes:

  • GatewayComputeModel now represents a gateway
    replica.
  • In this version, the terms "compute" and
    "replica" are used interchangeably. The plan is
    to switch to using exclusively "replica" later.
  • In this version, replica provisioning and
    termination are still done in the gateway
    pipeline, for all replicas at once. The plan is
    to introduce gateway replica pipelines later to
    allow for independent replica processing.

#3959

A gateway can now have multiple replicas for
improved availability.

```yaml
type: gateway
name: example-gateway

backend: aws
region: eu-west-1

domain: example.com

certificate: null
replicas: 2
```

To balance requests between gateway replicas, add
DNS records for each replica or set up a load
balancer outside of `dstack`. Replica hostnames
are displayed in `dstack` CLI and UI.

```shell
$ dstack gateway list
 NAME             BACKEND          HOSTNAME        DOMAIN       DEFAULT  STATUS
 example-gateway                                   example.com  ✓        running
    replica=0     aws (eu-west-1)  34.244.128.46
    replica=1     aws (eu-west-1)  18.201.201.174
```

Limitations:
- Changing the number of replicas or redeploying
  replicas is not supported.
- HTTPS is not supported. Use an external load
  balancer for TLS termination.
- An unavailable gateway replica prevents any new
  services or service replicas from being added.
- All replicas are bound to the same backend and
  region.

Implementation notes:
- `GatewayComputeModel` now represents a gateway
  replica.
- In this version, the terms "compute" and
  "replica" are used interchangeably. The plan is
  to switch to using exclusively "replica" later.
- In this version, replica provisioning and
  termination are still done in the gateway
  pipeline, for all replicas at once. The plan is
  to introduce gateway replica pipelines later to
  allow for independent replica processing.
@jvstme jvstme requested a review from r4victor June 12, 2026 01:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant