Support gateways with multiple replicas#3960
Open
jvstme wants to merge 1 commit into
Open
Conversation
A gateway can now have multiple replicas for
improved availability.
```yaml
type: gateway
name: example-gateway
backend: aws
region: eu-west-1
domain: example.com
certificate: null
replicas: 2
```
To balance requests between gateway replicas, add
DNS records for each replica or set up a load
balancer outside of `dstack`. Replica hostnames
are displayed in `dstack` CLI and UI.
```shell
$ dstack gateway list
NAME BACKEND HOSTNAME DOMAIN DEFAULT STATUS
example-gateway example.com ✓ running
replica=0 aws (eu-west-1) 34.244.128.46
replica=1 aws (eu-west-1) 18.201.201.174
```
Limitations:
- Changing the number of replicas or redeploying
replicas is not supported.
- HTTPS is not supported. Use an external load
balancer for TLS termination.
- An unavailable gateway replica prevents any new
services or service replicas from being added.
- All replicas are bound to the same backend and
region.
Implementation notes:
- `GatewayComputeModel` now represents a gateway
replica.
- In this version, the terms "compute" and
"replica" are used interchangeably. The plan is
to switch to using exclusively "replica" later.
- In this version, replica provisioning and
termination are still done in the gateway
pipeline, for all replicas at once. The plan is
to introduce gateway replica pipelines later to
allow for independent replica processing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A gateway can now have multiple replicas for
improved availability.
To balance requests between gateway replicas, add
DNS records for each replica or set up a load
balancer outside of
dstack. Replica hostnamesare displayed in
dstackCLI and UI.$ dstack gateway list NAME BACKEND HOSTNAME DOMAIN DEFAULT STATUS example-gateway example.com ✓ running replica=0 aws (eu-west-1) 34.244.128.46 replica=1 aws (eu-west-1) 18.201.201.174Limitations:
replicas is not supported.
balancer for TLS termination.
services or service replicas from being added.
region.
Implementation notes:
GatewayComputeModelnow represents a gatewayreplica.
"replica" are used interchangeably. The plan is
to switch to using exclusively "replica" later.
termination are still done in the gateway
pipeline, for all replicas at once. The plan is
to introduce gateway replica pipelines later to
allow for independent replica processing.
#3959