← Back to Blog

A production Buildbarn cluster is a set of services with shared Jsonnet, persistent CAS and Action Cache disks, and worker pools that scale separately from the scheduler. Upstream Buildbarn publishes deployment examples. It does not publish a maintained Helm chart. Hermetiq maintains one.

This page is the install path for that chart: what version 0.9.6 deploys, which values to change before the first upgrade, and which limits are intentional. The short companion page is Buildbarn Helm chart. Commands below are the chart README’s install section, checked against charts/buildbarn/README.md and Chart.yaml at version 0.9.6.

TL;DR

Copy the starter values, point storage and routing at your cluster, then install chart 0.9.6 from OCI. Run this from the directory that contains buildbarn-values.yaml:

helm upgrade --install --namespace hermetiq buildbarn oci://ghcr.io/hermetiq/buildbarn --version 0.9.6 --values buildbarn-values.yaml

That is oci://ghcr.io/hermetiq/buildbarn at --version 0.9.6. A shorter command without --namespace or --values installs chart defaults, including the StorageClass name premium-rwo and Contour routing. Use your values file for a cluster that does not match those defaults. Workers are not in this command. They come from the bb-worker-operator chart after this release exists.

Prerequisites

The repository installation guide expects:

  • Kubernetes 1.32 or newer, and namespace-administrator access.
  • Helm 3.8 or newer, which is the version that installs OCI charts, and kubectl aimed at the target cluster.
  • A Gateway API controller, Contour, or an Ingress controller that is already installed. This chart attaches routes. It does not install the controller or manage public DNS.
  • DNS and a TLS certificate for the external hosts. Contour and Ingress can use an existing Secret or a cert-manager Certificate. Gateway listeners terminate TLS themselves.
  • A StorageClass for the CAS and Action Cache volumes.
  • An OIDC issuer if you keep the starter file’s frontend JWKS and Browser SSO settings.

PostgreSQL, NATS, and Dragonfly belong to the Hermetiq application chart, not to this Buildbarn chart. The supported full-stack order is still the one in the repository installation guide: shared services, Hermetiq, the worker operator, Buildbarn, then RbeWorker pools. Hermetiq deployment choices, including what runs in your cluster versus Hermetiq Cloud, are documented in Deployments.

What the chart deploys

Chart 0.9.6 packages Buildbarn appVersion 20260908T142448Z. One release renders the services that share a Jsonnet config model. Worker pods are not among them.

WorkloadWhat the chart rendersRole
FrontendDeployment and frontend-grpc Service. Optional KEDA ScaledObject.Remote cache and remote execution API for Bazel.
SchedulerDeployment scheduler-ubuntu22-04 and scheduler Service.Queues actions and assigns them to workers that match the platform.
StorageStatefulSet and headless storage Service. Default replica count is 3.Sharded CAS and Action Cache on persistent disks.
BrowserDeployment and Service.Buildbarn Browser. The starter file puts oauth2-proxy in front of it.
WorkersConfigMap buildbarn-worker-config only.Shared common.libsonnet for pools the operator creates later.

Optional objects include the Remote Asset API, bb-portal, PodDisruptionBudgets, NetworkPolicies for storage and the scheduler, cert-manager Certificates, and VictoriaMetrics scrape and rule objects. Routing renders Contour HTTPProxy, Gateway API HTTPRoute and GRPCRoute, or Ingress resources, depending on routing.provider. Jsonnet is rendered into buildbarn-config. configOverrides can replace a single file when the values surface is not enough. The chart includes values.schema.json, so a value that does not match the schema fails at render time.

Clients use the frontend gRPC host. With the default hosts.domainBase of helm.hermetiq.dev, that host is bb.helm.hermetiq.dev. Browser, scheduler web, asset, portal, and BES names are derived the same way unless you set hosts.browser, hosts.frontendGrpc, or the other host fields. The repository examples show Bazel client wiring. This page does not add a .bazelrc that your project has not reviewed.

Install

The repository setup guide clones with SSH (git@github.com:Hermetiq/hermetiq-k8s.git). The same copy steps work over HTTPS. my-custom-values/ is gitignored.

git clone https://github.com/Hermetiq/hermetiq-k8s.git
cd hermetiq-k8s
cp -R custom-values my-custom-values
cd my-custom-values

Read the packaged docs before editing the file. helm show values is the full default set. The starter file is only an override.

helm show readme oci://ghcr.io/hermetiq/buildbarn --version 0.9.6
helm show values oci://ghcr.io/hermetiq/buildbarn --version 0.9.6

In buildbarn-values.yaml, set the hosts, identity provider, storage size, and routing, and confirm the StorageClass names with kubectl get storageclass. Then run the install command from the TL;DR. The chart README writes it across several lines; the flags are the same.

Check the workloads the chart owns before you add pools:

kubectl get deploy browser frontend scheduler-ubuntu22-04 -n hermetiq
kubectl get sts storage -n hermetiq
kubectl get endpoints browser frontend-grpc scheduler storage -n hermetiq

You can also render first with helm template buildbarn oci://ghcr.io/hermetiq/buildbarn --version 0.9.6 --namespace hermetiq --values buildbarn-values.yaml. The Buildbarn setup guide covers architecture choices that remain after the chart is installed: storage, workers, and what you operate yourself. The chart reference remains the source for flags.

Values to set first

ValueWhat the chart and starter file do
hosts.domainBaseDefaults to helm.hermetiq.dev. Derived names include browser. and bb. for frontend gRPC. Replace the suffix with a zone you control.
storage.persistence.cas.storageClassName and storage.persistence.ac.storageClassNameBoth default to premium-rwo. On EKS, set gp3 unless the cluster already has a premium-rwo alias backed by EBS gp3. Review the other persistence classes if you enable them.
routing.providerChart default is contour. The starter file sets gateway and names an existing Gateway. Use gateway-httproute-only for GKE Gateway, which has HTTPRoute and not GRPCRoute. Use none when another system owns the routes.
frontend.jwksThe starter file enables JWT checks and a JWKS sync Job. Issuer and audience must match the token. The file ships with Auth0 placeholders.
frontend.actionCache.putAuthorizer, CAS putAuthorizer, and executeAuthorizerStarter mode is requireCanWriteToCache. Reads stay allow. See the limitations below.
scheduler.predeclaredPlatformQueuesRequired for pools that scale to zero. Without a declaration, an unregistered platform is rejected with No workers exist, so KEDA never sees the queue signal it needs to start a worker.
browser.oauth2Proxy.enabledThe starter file turns this on and reuses the dashboard OAuth ConfigMap and Secret from the Hermetiq chart. Turn it off, or supply your own issuer, when that chart is not installed.

The starter file also sets storage and frontend resource requests and a shared maximum unary message size of 800 MiB (config.maximumMessageSizeBytes: 838860800). The file says to tune storage requests for shard count and cache load, and to size pod memory for large requests. Those figures are configuration for that example, not a capacity recommendation. Chart-managed pods use k8sNodeScheduling, which defaults to amd64 Linux nodes, unless a workload sets its own nodeSelector.

Workers need the operator

The Buildbarn chart does not create worker Deployments. The supported bundle pairs chart 0.9.6 with bb-worker-operator 0.3.8 (appVersion v0.3.9). For a namespace-scoped install, apply the CRD from the matching release tag first. The operator README owns that URL. Then, from the same copied values directory:

helm upgrade --install --namespace hermetiq bb-worker-operator \
  oci://ghcr.io/hermetiq/bb-worker-operator \
  --version 0.3.8 \
  --skip-crds \
  --values bb-worker-operator-values.yaml

If the Helm installer can create cluster-scoped resources, the operator README says the separate CRD step is optional on the first install: omit --skip-crds. Later upgrades still apply the CRD before the chart. Keep the CRD and the controller image on the same release.

After Buildbarn and buildbarn-worker-config exist, apply pools from the copied rbeworkers directory:

kubectl apply --namespace hermetiq --kustomize rbeworkers
kubectl -n hermetiq get rbeworkers

FUSE and Docker-in-Docker pools need privileged containers. The namespace has to allow those pods. Chart-managed frontend, scheduler, storage, and browser pods still set a restricted security context. Sample pools select amd64 Linux nodes and do not add custom tolerations. A pool with minReplicas: 0 shows zero ready replicas until KEDA sees queued work. That is expected. Check the RbeWorker and ScaledObject, not only kubectl get pods.

Limitations

  • Worker pools are a second chart. Installing oci://ghcr.io/hermetiq/buildbarn does not start workers.
  • Starter values target the GKE-oriented examples in the repository. Other clouds need equivalent storage, identity, and routing. The chart default StorageClass name is the GKE-style premium-rwo.
  • Frontend reads are open by design. With frontend.jwks.enabled, the authentication policy is any of a JWT check and allow. CAS get, FindMissing, and Action Cache reads stay allow. Writes and Execute are gated separately. Restrict network exposure if an open read cache is not what you want. Removing the allow fallback is a frontend.jsonnet override, and the chart README says the browser, bb-portal, and tokenless CI lose access at the same time.
  • No comparative performance numbers are published for this chart. Do not treat default replica counts, disk sizes, or the starter resource requests as a benchmark.
  • The chart does not install Gateway, Contour, Ingress, cert-manager, KEDA, or DNS. KEDA is required only if you enable frontend autoscaling or worker ScaledObjects.
  • Platform matching has no silent fallback. An action whose platform matches no pool fails or waits. A catch-all router is optional, and turning it on stops those mismatches from failing loudly.
Read the evidence and limitations

Install flags, the OCI URL, and version 0.9.6 are copied from the Buildbarn chart README and Chart.yaml on hermetiq-k8s main. The supported bundle table in that repository’s README pairs Buildbarn 0.9.6 (appVersion 20260908T142448Z) with bb-worker-operator 0.3.8. StorageClass defaults are premium-rwo in charts/buildbarn/values.yaml. The EKS gp3 note is in the chart README install section. routing.provider defaults to contour in values.yaml; the starter file sets gateway.

The repository README states that the examples are tailored for GKE and that the charts work on conformant Kubernetes 1.32 or newer when storage, identity, routing, DNS, and TLS are supplied. “Reads are open by design” is the chart README’s heading for the JWT-plus-allow policy. This page does not add measurements. There is no published benchmark to cite.

Upstream deployment examples are buildbarn/bb-deployments. The community chart is slamdev/buildbarn, chart version 0.0.1. Those are different artifacts. This page does not rank them. Artifact Hub’s Hermetiq package is hermetiq-buildbarn/buildbarn. Its publisher is moving from cesar to the Hermetiq organization. The install source on this page is the OCI URL, not the Artifact Hub publisher name.

The clone command in the repository guide uses SSH. The snippet above uses the HTTPS remote and the same cp -R custom-values step. The operator README’s namespace-scoped CRD URL is not reproduced here, because that page tells you to apply the CRD from the matching release tag and the Helm command shown is version 0.3.8.

FAQ

Is there an official Buildbarn Helm chart?

Upstream Buildbarn does not publish a maintained Helm chart. It publishes deployment examples in bb-deployments. Hermetiq maintains an Apache-2.0 chart at oci://ghcr.io/hermetiq/buildbarn, version 0.9.6, with source in Hermetiq/hermetiq-k8s. A separate community chart, slamdev/buildbarn (chart version 0.0.1), also exists. This page does not compare their features or performance.

What is the install command?

From the directory that contains your copied buildbarn-values.yaml: helm upgrade --install --namespace hermetiq buildbarn oci://ghcr.io/hermetiq/buildbarn --version 0.9.6 --values buildbarn-values.yaml. Inspect the packaged README and values before you rely on a values file.

Does the chart deploy workers?

No. The chart renders frontend, scheduler, storage, and browser workloads, plus shared worker Jsonnet. It does not render worker Deployments or worker KEDA ScaledObjects. Install the bb-worker-operator chart, then apply RbeWorker resources.

Which values should I set first?

Set hosts, the CAS and AC StorageClasses, and routing.provider. The chart default StorageClass name is premium-rwo; on EKS use gp3 unless the cluster provides that name. The starter values enable frontend JWKS and Browser SSO, so set the issuer, audience, and JWKS URL before installing that file unchanged.

Are frontend reads authenticated?

No. With frontend JWKS enabled, unauthenticated callers are still admitted. CAS reads, FindMissing, and Action Cache reads are allow. The starter values require a valid JWT for CAS writes, Action Cache writes, and Execute. Closing reads is a frontend.jsonnet override, and it also blocks readers that do not send a token.

Will the starter values work outside GKE?

The repository examples are tailored for GKE. The charts target conformant Kubernetes 1.32 or newer, but EKS, AKS, and other clusters need their own storage, workload identity, Gateway or Ingress, DNS, and TLS. The chart does not install a routing controller or public DNS.

Are there published benchmarks for this chart?

No. The chart repository and this page do not publish comparative performance numbers. Storage sizes and replica counts in the defaults are configuration, not a measured result for your workload.

Can I install the chart without the Hermetiq application?

Yes. The chart is Apache-2.0 and deploys Buildbarn components on its own. The supported full-stack order in the repository installs the Hermetiq chart and the worker operator as well. The starter Browser SSO reuses the Hermetiq dashboard OAuth Secret, and the optional grpc-cache-proxy sidecar is what sends cache events to Hermetiq. Change those values if you are not installing Hermetiq.

Where is the chart listed?

Artifact Hub currently lists it at artifacthub.io/packages/helm/hermetiq-buildbarn/buildbarn. That listing is moving from publisher cesar to the Hermetiq organization. Install from oci://ghcr.io/hermetiq/buildbarn; that OCI reference is the source this page uses.

Open the Buildbarn Helm chart page for the OCI install command, architecture, and limits. Add build intelligence on a Buildbarn cluster you already run, or book a demo. Managed cache and execution is a separate operating decision from installing this chart.

Related reading: