Deploy on GCP

GCP is the practical target for self-hosting because Compute Engine exposes nested virtualization on standard machine types, so Firecracker gets /dev/kvm without bare metal. The Terraform modules set enable_nested_virtualization = true on every host that runs sandboxes.

There is no deploy-gcp.sh. Two entry points exist in the repo:

Prereqs

• gcloud CLI logged in (gcloud auth login) against a project with billing enabled
• terraform
• A Cloudflare API token with Zone:DNS:Edit, plus the zone ID — DNS records are managed by Terraform, not by a cloud DNS service
• An SSH public key and the CIDR you will SSH from

The multi-node script additionally requires jq and curl, and checks for all four up front.

Single dev host

Copy the example variables file and fill it in:

cd infra/terraform/envs/dev-gcp
cp terraform.tfvars.example terraform.tfvars

gcp_project          = "pandastack-production"
ssh_pubkey           = "ssh-ed25519 AAAA... you@host"
ssh_allowed_cidr     = "1.2.3.4/32"
cloudflare_api_token = "<token>"
cloudflare_zone_id   = "<zone-id>"

Then apply from the repo root:

make tf-gcp-init
make tf-gcp-plan
make tf-gcp-apply
make tf-gcp-output

The targets run terraform -chdir=infra/terraform/envs/dev-gcp with -var-file=terraform.tfvars; apply and destroy pass -auto-approve.

This builds three modules plus DNS:

• gcp-network — VPC, subnet, firewall scoped to ssh_allowed_cidr, static external IP
• gcp-storage — a GCS bucket for kernels, templates, and snapshots
• gcp-compute — one Compute Engine instance with nested virtualization enabled
• dns — Cloudflare A records for the dashboard, API, and www subdomains (set subdomain_www = "" to skip the last one)

Outputs are instance_name, external_ip, app_fqdn, api_fqdn, and gcs_bucket_name.

Defaults worth changing

Tear it down with make tf-gcp-destroy.

Multi-node

deploy/deploy-gcp-multi.sh drives the dev-gcp-multi environment. It reads .env.local from the repo root and fails immediately if that file is missing, so populate DATABASE_URL, CLOUDFLARE_API_TOKEN, and CLOUDFLARE_ZONE_ID before running anything.

./deploy/deploy-gcp-multi.sh up

up cross-builds pandastack-agent, pandastack-init, pandastack-daemon, and pandastack-api for linux/amd64, publishes them to a GCS build bucket, writes terraform.tfvars, runs terraform init -upgrade and terraform apply -auto-approve, waits up to 15 minutes for the load balancer health check, bakes templates if none are in the bucket, deploys the dashboard, and points api.<zone> at the LB IP through Cloudflare.

Subcommands

./deploy/deploy-gcp-multi.sh up                 # build + apply + bake + DNS
./deploy/deploy-gcp-multi.sh status             # cert status, LB IP, MIG instances, agent heartbeats
./deploy/deploy-gcp-multi.sh smoke              # /healthz + /version, plus sandbox lifecycle if SMOKE_API_TOKEN is set
./deploy/deploy-gcp-multi.sh bake-templates     # (re)bake templates on an agent VM and push to GCS
./deploy/deploy-gcp-multi.sh deploy-frontend    # build the dashboard and deploy it to Cloudflare Pages
./deploy/deploy-gcp-multi.sh cutover            # upsert the api A record to the LB IP
./deploy/deploy-gcp-multi.sh scale 3            # resize the agent MIG
./deploy/deploy-gcp-multi.sh down               # terraform destroy

Running the script with no subcommand prints this list.

bake-templates accepts template names (bake-templates code-interpreter browser) and honours FORCE=1 to rebuild artifacts that already exist. It runs the bake on an agent VM over IAP and polls for up to 90 minutes, because a full bake takes 10–60 minutes.

smoke targets api.pandastack.ai literally. Edit the script or run the checks by hand if you deployed to your own zone.

Topology

up builds, in one project:

• A private VPC with separate edge and agent subnets, Cloud NAT, and Secret Manager
• An edge MIG (e2-small by default) running the API behind a global HTTPS load balancer with a Google-managed certificate
• A stateful agent MIG (n2-standard-8 by default, Intel Cascade Lake minimum CPU platform, nested virtualization on, 400 GiB pd-ssd boot disk, no public IP). This pool carries the durable volumes disk, which is why GCP forbids an autoscaler on it — scale it by hand with scale N.
• An ephemeral agent pool, created at size 0 with its autoscaler mode OFF. It exists but is inert until you set agent_eph_count and agent_eph_autoscaler_mode in terraform.tfvars.
• A Cloud SQL instance, a ClickHouse VM, and a DB proxy VM with a static IP for managed databases

The agent boot disk is 400 GiB because each agent keeps a baked snapshot and rootfs on local disk for every public template.

Overrides

write_tfvars reads these from .env.local:

GCP_PROJECT=pandastack-production
GCP_REGION=us-central1
GCP_ZONE=us-central1-a
USE_PREEMPTIBLE=false
AGENT_MACHINE_TYPE=n2-standard-8
AGENT_COUNT=1
AGENT_MAX_COUNT=8
EDGE_COUNT=2
CLOUDFLARE_DOMAIN=pandastack.ai
SSH_PUBKEY_PATH=~/.ssh/id_ed25519.pub
SSH_ALLOWED_CIDR=1.2.3.4/32

Sizing the fleet

Capacity is bounded by memory, not by the NATID slot pool. Firecracker cannot resize vCPU or RAM at snapshot restore, so each template's guest RAM is fixed when the template is baked:

Concurrency per agent VM is roughly that VM's usable RAM divided by the baked RAM of the template you run, minus host overhead. base at 4 GiB is the expensive one, and it is also what git-driven Apps run on. When the fleet runs out of memory, sandbox creates return 502 — add agents with scale N before that happens.

Point clients at your deployment

export PANDASTACK_API=https://api.pandastack.ai
export PANDASTACK_API_KEY=pds_abc123def456...

Replace the API URL with your own origin.

Related

• Self-host overview
• Deploy on AWS
• Templates
• Scheduler internals