ModelSphere Helm Charts

Helm charts for running large language models on Kubernetes.

helm repo add modelsphere https://modelsphere.github.io/helm-charts
helm repo update
helm search repo modelsphere
ChartVersionApp versionDescription
modelsphere/autoconfig0.4.30.4.3k8s 后端发现 → 自动同步 openresty peers / cache-aware-router workers(ModelRoute controller)
modelsphere/bodylog0.1.160.1.15Body-log listener for the OpenResty router: receives full request/response frames over TCP, writes them as hourly JSONL, and serves rollups over HTTP
modelsphere/bodylog-exporter0.3.70.2.14Turns the body-log listener's detail records into Prometheus metrics, and polls the router's control-plane endpoints for live routing state
modelsphere/bodylog-victorialogs0.1.1v1.53.0Searchable body logs: Vector ships bodylog's JSONL into VictoriaLogs, registered as a Grafana datasource
modelsphere/cart0.2.40.6.6Cache-aware router: routes each request to the replica that already holds the longest matching prompt prefix, while keeping load balanced
modelsphere/catalog-mirror0.2.00.2.0In-cluster copy of the model catalog and this helm chart repository, for clusters whose route to GitHub is slow or unreliable; swiss is pointed at it instead of GitHub Pages
modelsphere/console0.2.20.2.2ModelSphere community portal — identity (users, roles, login) and a federation gateway to Swiss and other backends.
modelsphere/continuation-gateway0.1.40.2.7Continuation gateway: when a streaming completion stalls or drops mid-generation, resumes it once from what the client already received, so the client sees one continuous, complete stream
modelsphere/llm-autotune0.3.10.3.0Automated tuning of LLM inference deployments — campaigns, a launch stack, and policy-as-code search
modelsphere/llm-bench0.3.00.3.0LLM Benchmark Platform - FastAPI backend, Vue frontend, Dramatiq workers
modelsphere/llm-slo-decision-gen0.3.90.8.1LLM and async-job SLO stack: decision-gen (recommendation service), optional slo-api (SLO storage HTTP/UI), and the LLMSLORequirement and JobSLORequirement CRDs under inference.modelsphere.dev. CRD ownership: this chart owns llmslorequirements.inference.modelsphere.dev and jobslorequirements.inference.modelsphere.dev (see crds/). Both CRDs carry helm.sh/resource-policy: keep so uninstall leaves them in the cluster — existing LLMSLORequirement / JobSLORequirement objects are not deleted with the release. Do not install a second chart that also ships these CRDs into the same cluster. LLMScaler CRD and the operator that reconciles it are provided by a separate chart (llmscaleoperator / llmscaleoperator-system). Deploy that one first if you need autoscaling, not just SLO storage.
modelsphere/llmscaleoperator0.5.00.5.0Kubernetes operator that autoscales LLM inference workloads on LLM-specific signals -- KV-cache utilization, queue depth, TPM/capacity load -- rather than CPU and memory. An HPA specialized for token serving. Ships the LLMScaler CRD (autoscaling.modelsphere.dev/v1alpha1), the LLMService CRD (serving.modelsphere.dev/v1alpha1) and the controller that reconciles them. The SLO storage side lives in the llm-slo-decision-gen chart; the two are independent, and this one is what you need for autoscaling alone. Generated from the llm-operator repo (dist/chart, kubebuilder helm plugin) and maintained here, because this is where charts are published from.
modelsphere/minimax-h30.2.20.2.2Self-hosted MiniMax H3-compatible video generation for the open-weight MiniMax H3 models -- H3 Router, H3 Server and SGLang in one release, with separate FL2VA and Ref2VA pools
modelsphere/mooncake-master0.1.00.3.13Highly available Mooncake Store masters: two or more mooncake_master replicas electing a leader through etcd (bundled or external), with an optional sidecar that re-registers NVMe-oF volumes on whichever master wins
modelsphere/openresty0.1.280.1.28Session-affinity router for LLM inference backends: pins a conversation to the backend that already holds its prefix cache, with active health checking, adaptive concurrency and full body logging
modelsphere/rdma-injector0.2.30.2.0Mutating admission webhook that spares workload manifests the RDMA boilerplate: pods labelled rdma-ib:"true" get the /etc/gpu-node mount and an NCCL_IB_HCA sourced from the node's healthy InfiniBand ports injected automatically. The webhook code is baked into the image; its serving certificate is self-signed by the chart at install time, so it installs into any namespace.
modelsphere/sglang0.9.371.0.0SGLang inference deployment -- single-node, multi-node (LeaderWorkerSet) or prefill/decode-disaggregated (DisaggregatedSet or merged colocate pod), with optional cache-aware routing, autoscaling and hang detection
modelsphere/swiss0.7.20.7.2swissd -- the Swiss deploy control plane for the sglang and vllm charts. One instance per cluster, running inside the cluster it manages: it reads the model catalog over HTTP, reads this cluster's releases and nodes through its own ServiceAccount, and serves the web UI embedded in its binary.
modelsphere/vllm0.9.81.0.0vLLM inference deployment -- single-node or multi-node (LeaderWorkerSet), with optional cache-aware routing, autoscaling and hang detection