Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions .gitlab-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,34 @@ publish:
tags:
- kubernetes

publish-cloudflared:
stage: docker
image:
name: moby/buildkit:rootless
entrypoint: [""]
script:
- |
CLOUDFLARED_IMAGE="$HARBOR_REGISTRY/$HARBOR_PROJECT/cloudflared-tunnel:latest"
IMAGE_TAG="${CI_COMMIT_SHORT_SHA:-$CI_PIPELINE_ID}"

OUTPUT_FLAGS="--output type=image,name=$CLOUDFLARED_IMAGE,push=true"
OUTPUT_FLAGS="$OUTPUT_FLAGS --output type=image,name=$HARBOR_REGISTRY/$HARBOR_PROJECT/cloudflared-tunnel:$IMAGE_TAG,push=true"
if [ -n "$CI_COMMIT_TAG" ]; then
OUTPUT_FLAGS="$OUTPUT_FLAGS --output type=image,name=$HARBOR_REGISTRY/$HARBOR_PROJECT/cloudflared-tunnel:$CI_COMMIT_TAG,push=true"
fi

buildctl-daemonless.sh build \
--frontend dockerfile.v0 \
--local context=./cloudflare \
--local dockerfile=./cloudflare/Dockerfile \
$OUTPUT_FLAGS

echo "Pushed cloudflared-tunnel to $CLOUDFLARED_IMAGE"
rules:
- if: '$CI_COMMIT_TAG'
tags:
- kubernetes

release:
stage: release
image: registry.gitlab.com/gitlab-org/cli:latest
Expand Down
6 changes: 6 additions & 0 deletions cloudflare/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Cloudflared with shell for k8s token injection
# Official image is distroless (no /bin/sh) - we need shell to pass token from secret
FROM harbor.dataknife.net/dockerhub/cloudflare/cloudflared:latest AS cf
FROM harbor.dataknife.net/dockerhub/library/alpine:3.19
COPY --from=cf /usr/local/bin/cloudflared /usr/local/bin/cloudflared
ENTRYPOINT ["/bin/sh", "-c", "exec cloudflared --no-autoupdate tunnel run --token \"$TUNNEL_TOKEN\""]
7 changes: 7 additions & 0 deletions cloudflare/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Cloudflare Tunnel Image

Builds the cloudflared image for Kubernetes deployment. The official `cloudflare/cloudflared` image is distroless (no shell), so we use Alpine + shell to inject the tunnel token from a k8s secret at runtime.

**Deployment:** See `../k8s/CLOUDFLARE_TUNNEL.md`

**GitLab CI:** The `publish-cloudflared` job builds and pushes this image on tags.
83 changes: 83 additions & 0 deletions k8s/CLOUDFLARE_TUNNEL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# Cloudflare Tunnel in Kubernetes

Expose High Command via Cloudflare Tunnel without port forwarding. The tunnel runs as a pod in the cluster and proxies traffic to internal services.

## Prerequisites

- Tunnel created in [Cloudflare Zero Trust](https://one.dash.cloudflare.com/) → Networks → Tunnels
- Connector token from the tunnel setup (Docker install step)

## Deploy

### 1. Create the secret with your tunnel token

```bash
kubectl create secret generic cloudflared-tunnel-credentials \
--from-literal=token='eyJhIjoi...' \
-n high-command
```

Use the token from the Docker command in the Cloudflare dashboard:
`docker run cloudflare/cloudflared:latest tunnel run --token <TOKEN>`

### 2. Deploy the tunnel and gateway alias

```bash
kubectl apply -f gateway-tunnel-service.yaml
kubectl apply -f cloudflared-tunnel-deployment.yaml
```

### 3. Configure ingress in Cloudflare dashboard

Route all traffic to the Envoy Gateway — it already handles path routing (`/api`, `/mcp`, `/`) and MCP path rewrite via the HTTPRoute.

In Zero Trust → Networks → Tunnels → your tunnel → Public Hostname:

| Public hostname | Path | Service | URL |
|-----------------|------|---------|-----|
| `hc.dataknife.ai` | `/` (or leave empty) | HTTPS | `https://high-command-gateway.high-command.svc.cluster.local:443` |

**Additional application settings** (expand the section when adding the route):

- **Origin Server Name**: `hc.dataknife.ai` — The Gateway's listener matches this hostname. Without it, cloudflared sends the internal k8s hostname as SNI and the Gateway resets the connection.
- **No TLS Verify**: Enable this. The Gateway's cert is for `hc.dataknife.ai`, but cloudflared connects to the internal k8s hostname — TLS verification would fail without it.

## Verify

```bash
kubectl get pods -n high-command -l app=cloudflared-tunnel
kubectl logs -n high-command -l app=cloudflared-tunnel -f
```

## Troubleshooting

**Connection refused or TLS errors:**

- Enable **No TLS Verify** in Additional application settings.
- Confirm the Gateway service name is correct (it may change if the Gateway is recreated).

**API or MCP returns 404:**

- The Gateway's HTTPRoute handles path routing. Verify the Gateway and HTTPRoute are healthy: `kubectl get gateway,httproute -n high-command`.

**If the Envoy Gateway service was recreated** (e.g. after a Gateway update), the `high-command-gateway` ExternalName may point to a stale service. Update it:

```bash
NEW_SVC=$(kubectl get svc -n envoy-gateway-system -o name | grep high-command | cut -d/ -f2)
kubectl patch svc high-command-gateway -n high-command -p "{\"spec\":{\"externalName\":\"${NEW_SVC}.envoy-gateway-system.svc.cluster.local\"}}"
```

## Image

The deployment uses `cloudflare/Dockerfile` (Alpine + cloudflared binary, shell for token injection). GitLab CI builds and pushes on tags. To build manually:

```bash
docker build -t harbor.dataknife.net/library/cloudflared-tunnel:latest cloudflare/
docker push harbor.dataknife.net/library/cloudflared-tunnel:latest
```

## Files

- `cloudflared-tunnel-deployment.yaml` - Deployment (token from secret, `--no-autoupdate`)
- `cloudflared-tunnel-secrets-example.yaml` - Secret template (do not commit real token)
- `gateway-tunnel-service.yaml` - Stable alias to Envoy Gateway for tunnel routing
48 changes: 29 additions & 19 deletions k8s/README.md
Original file line number Diff line number Diff line change
@@ -1,37 +1,47 @@
# Kubernetes Deployment Files

This directory contains Kubernetes manifests for deploying the High Command UI.
This directory contains Kubernetes manifests for deploying the High Command stack (UI, API, MCP, Gateway, Cloudflare Tunnel).

## Files
## Architecture

- `ui-deployment-blue.yaml` - Blue deployment (active version)
- `ui-deployment-green.yaml` - Green deployment (standby version)
- `ui-service.yaml` - Service to route traffic between blue/green
- `ui-pdb.yaml` - Pod Disruption Budget for availability
- `ui-ingress.yaml` - Ingress configuration for external access
Traffic flow: **Cloudflare Tunnel** → **Envoy Gateway** → **HTTPRoute** → UI/API/MCP. No nginx Ingress.

## UI Files

- `ui-deployment-blue.yaml`, `ui-deployment-green.yaml` - Blue/green deployments
- `ui-service.yaml` - Service routing
- `ui-pdb.yaml` - Pod Disruption Budget

## Full Stack Files

- `api-deployment-blue.yaml`, `api-deployment-green.yaml` - API deployments
- `api-service.yaml`, `api-pdb.yaml` - API service
- `httproute.yaml` - Gateway API HTTPRoute (/api, /claude, /mcp, /)
- `gateway.yaml`, `gatewayclass.yaml`, `gateway-certificate.yaml` - Envoy Gateway
- `gateway-tunnel-service.yaml` - Alias for Cloudflare Tunnel → Gateway
- `cloudflared-tunnel-deployment.yaml` - Cloudflare Tunnel pod
- `mcp-service.yaml`, `mcp-referencegrant.yaml`, `referencegrant.yaml` - MCP routing

See `CLOUDFLARE_TUNNEL.md` for tunnel setup.

## Secrets

**No secrets are stored in these files.**

If you need to use Claude integration, add the API key as a Kubernetes Secret:
**API secrets** (required): `database-url` and optionally `claude-api-key`:

```bash
kubectl create secret generic high-command-ui-secrets \
--from-literal=vite-claude-api-key='your-api-key-here' \
kubectl create secret generic high-command-api-secrets \
--from-literal=database-url='postgresql://user:password@high-command-postgres-rw.high-command.svc.cluster.local:5432/highcommand' \
--from-literal=claude-api-key='sk-ant-api03-...' \
-n high-command
```

Then update the deployments to reference the secret:
See `api-secrets-example.yaml` for details.

```yaml
env:
- name: VITE_CLAUDE_API_KEY
valueFrom:
secretKeyRef:
name: high-command-ui-secrets
key: vite-claude-api-key
```
## Cloudflare

The `../cloudflare/` folder contains the tunnel Dockerfile. GitLab CI builds the cloudflared-tunnel image from `cloudflare/Dockerfile`.

## Environment Variables

Expand Down
104 changes: 104 additions & 0 deletions k8s/api-deployment-blue.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: high-command-api-blue
namespace: high-command
labels:
app: high-command-api
version: blue
spec:
replicas: 3
selector:
matchLabels:
app: high-command-api
version: blue
template:
metadata:
labels:
app: high-command-api
version: blue
spec:
containers:
- name: api
image: harbor.dataknife.net/library/high-command-api:v0.11
imagePullPolicy: Always
env:
- name: MODE
value: "api"
- name: PYTHONUNBUFFERED
value: "1"
- name: LOG_LEVEL
value: "INFO"
- name: DATABASE_URL
valueFrom:
secretKeyRef:
name: high-command-api-secrets
key: database-url
- name: PORT
value: "5000"
- name: HOST
value: "0.0.0.0"
- name: HELLDIVERS_API_BASE
value: "https://api.helldivers2.dev/api/v1"
- name: HELLDIVERS_API_CLIENT_NAME
value: "High Command"
- name: HELLDIVERS_API_CONTACT
value: "lee@fullmetal.dev"
- name: CLAUDE_API_KEY
valueFrom:
secretKeyRef:
name: high-command-api-secrets
key: claude-api-key
optional: true
ports:
- name: http
containerPort: 5000
protocol: TCP
livenessProbe:
httpGet:
path: /api/health
port: http
initialDelaySeconds: 30
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 3
readinessProbe:
httpGet:
path: /api/health
port: http
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 3
failureThreshold: 2
resources:
requests:
cpu: "200m"
memory: "256Mi"
limits:
cpu: "1000m"
memory: "1Gi"
securityContext:
runAsNonRoot: true
runAsUser: 1000
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
volumeMounts:
- name: tmp
mountPath: /tmp
volumes:
- name: tmp
emptyDir: {}
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- high-command-api
topologyKey: kubernetes.io/hostname
Loading
Loading