Deploying models
Swiss (swissd) on the cluster performs every deployment. Each install, upgrade and rollback first composes a plan and previews the change; nothing runs until you confirm.
First use: site setup
If swissd has no site profile yet, Model Serving opens on Site setup. Describe the cluster and save:
- model catalog: where deployable models come from
- chart and image registries
- model path template: where weights live on the nodes
You can change it any time in Model Serving → Site Profile, as a form or as YAML. The server validates it before saving.
1. Pick a model
Open Model Serving → Model Library:
- Narrow it down with search and filters (family, engine, hardware, tag); "Tuned only" keeps models with tuned variants.
- Click a card for details. The Variants tab lists the ways the model can be deployed (GPU model, GPU count, engine), each with "N nodes available" computed from the cluster's free GPUs.
- Click Deploy on the variant you want.
2. Fill in the form
| Section | Main fields | Notes |
|---|---|---|
| Basics | Service name, namespace | The service name also names the helm release, route, scaler and SLO; lowercase letters, digits and -, at most 53 characters. Cannot be changed later |
| Resources & scheduling | Autoscaling / replicas, GPU models, tolerations | Autoscaling is on by default: LLMScaler keeps replicas between the bounds |
| Routing | Publish route, route path, SLO, TTFT / TPS thresholds, Monitor, ServiceMonitor | Controls how requests reach the inference instances, and the route’s service targets and monitoring |
| Advanced | YAML overrides | Rarely needed |
Most fields can stay empty and take the site profile's and chart's defaults.
3. Review and confirm
Compose plan opens the review:
- Plan: the configuration layers this deployment is made of.
- Diff: what will change compared with the cluster.
- Change note: why you are making the change; it goes into the activity log.
Click Deploy to confirm.
4. Wait until ready
Watch the Model Serving → Inference Services list:
| State | Meaning |
|---|---|
| Applying | helm is running |
| Loading | pods are scheduled and reading weights |
| Serving | ready for traffic |
| Failed | the deployment failed; see its output in Activity |
| Not ready | not ready after an hour, longer than a normal load; investigate |
A cold start takes 20–40 minutes. Loading large weights is slow; "Loading" is usually not an error.
Click a service for its details. Overview shows progress step by step: applied → instances ready → routed → autoscaling. The other tabs:
| Tab | Contents |
|---|---|
| Instances | pods, their nodes and restart counts |
| Logs | recent logs per pod and container |
| Events | Kubernetes events, optionally warnings only (kept for one hour by default) |
| Versions | revision history; view the values of any revision and roll back |
| Activity | this service's installs, upgrades and rollbacks |
| Plan | how the current plan is composed |
Upgrade, roll back, uninstall
From a row's actions in the list, or the top of the detail page:
- Upgrade: change the model version, variant or settings; again you review the diff before confirming. Service name and namespace are fixed.
- Roll back: pick a revision under Versions to restore its plan as it was. To change settings, upgrade instead.
- Edit SLO: the latency and throughput targets and the replica bounds the scaler works with. Services deployed without SLO need it switched on in an upgrade first.
- Uninstall: stops the service and deletes its pods; the activity log is kept. Reinstalling loads the weights again.
"Behind catalog" in the list means the catalog has a newer version to upgrade to; "Untracked" means the release was not deployed by Swiss and cannot be upgraded here.
Nodes and activity
- Model Serving → Nodes: the cluster's GPU inventory — free GPUs, the largest free slot on one node, models in stock, and unavailable nodes. Check here before deploying.
- Model Serving → Activity: every install, upgrade, rollback and uninstall swissd has run; expand a row for its full output.
Read-only mode
If action buttons are greyed out with "this swissd is read-only", swissd on the cluster is not allowed to deploy and you can only look. An operator has to turn on allowDeploy in Swiss's configuration.