What Is Hermeticity and Why It Matters

TL;DR: Hermeticity means a build's output is determined entirely by its declared inputs — nothing leaks in from the host machine, the network, or the environment. It's the property that makes builds reproducible, cacheable, and parallelizable at scale. It's also the hardest thing to get right in Bazel, and the root cause of most build failures teams can't explain.

You've probably heard the term "hermetic" thrown around in conversations about Bazel. It shows up in documentation, conference talks, and migration pitches. It's often described as one of Bazel's core properties — right alongside reproducibility and incrementality.

But if you ask five engineers what hermeticity actually means, you'll get five different answers. Some will say it means builds are reproducible. Others will say it means builds don't touch the network. A few will wave their hands and say something about sandboxing.

None of those answers are wrong, exactly. But none of them capture why hermeticity is the single most important property in modern build systems — and why getting it wrong is the source of the most painful, hardest-to-debug build failures you'll ever encounter.

The Formal Definition

A build is hermetic when its output is determined entirely by its explicitly declared inputs. Nothing else influences the result. Not the state of the host machine. Not environment variables. Not the system clock. Not the contents of your home directory. Not whatever version of Python happens to be installed at /usr/bin/python3.

If you give a hermetic build the same inputs, it will produce the same outputs — on any machine, at any time, in any order. Every time.

This is a deceptively simple idea with profound consequences. It means build outputs are a pure function of their inputs. And like pure functions in programming, that purity unlocks everything else.

Why It's the Foundation

Hermeticity isn't just a nice property to have. It's the foundation that every other advantage of modern build systems rests on. Remove it, and the entire value proposition collapses.

Caching only works if builds are hermetic

Bazel's remote caching system works by computing a content hash of each action's inputs and using that hash as a cache key. If the inputs match, the cached output is reused. This is what allows teams to skip rebuilding code that hasn't changed — sometimes saving hours per day across an organization.

But this only works if the same inputs actually produce the same outputs. If a build action secretly depends on the system clock, or reads a file that isn't declared in its inputs, or behaves differently based on the number of CPU cores available — the cache key will match, but the output will be wrong. You'll get a "successful" build with incorrect artifacts. Or worse, you'll get a build that works on your machine but fails in CI, and nobody can figure out why.

Every cache miss you can't explain is a hermeticity violation waiting to be found.

Parallelism only works if builds are hermetic

Bazel builds your dependency graph in parallel — executing independent actions simultaneously across local cores or a remote execution cluster. This is where Bazel's speed advantage comes from.

But parallel execution only produces correct results if actions are truly independent — meaning they don't share hidden state. If action A writes to a temp file and action B reads from it, and neither declares that dependency, the build is correct only by accident of scheduling. Run those actions in a different order — or on different machines — and you get a different result.

Hermeticity guarantees that the dependency graph is the truth. If an action's inputs are declared, and those inputs haven't changed, the action can safely run anywhere, at any time, in any order. That's what makes distributed builds possible.

Reproducibility is hermeticity across time

When someone says a build is "reproducible," what they mean is: you can check out the same commit next week, next month, or next year, and get the same build output. Reproducibility is what allows you to bisect regressions, audit artifacts, and trust that what you tested is what you shipped.

Reproducibility is just hermeticity extended across time. If a build is truly hermetic — if nothing outside the declared inputs influences the output — then it's automatically reproducible. The inputs are versioned in your repository. The outputs are deterministic. Time is irrelevant.

When builds aren't reproducible, the cause is almost always a hermeticity violation. Something is leaking in that isn't tracked: a system library version, a network dependency, a timestamp embedded in an artifact.

What Breaks Hermeticity (And How It Hurts)

In theory, every Bazel build is hermetic. In practice, hermeticity is a spectrum — and most organizations are further from the ideal than they think.

Here are the most common ways hermeticity breaks:

Undeclared system dependencies

A build rule shells out to gcc or python3 without pinning the version. On your laptop, that's GCC 13.2. On the CI machine, it's GCC 12.1. The build "works" on both — until someone uses a C++20 feature that only compiles on 13.x, and suddenly CI is red for no apparent reason.

The fix is toolchain management: declaring your compilers, interpreters, and system tools as explicit, versioned inputs to the build. Bazel's toolchain framework exists specifically for this purpose. But configuring it correctly is non-trivial, which is why many teams skip it early and pay for it later.

Network access during builds

A build rule downloads a dependency at build time — a JAR from Maven Central, a wheel from PyPI, a pre-trained model from S3. If the remote artifact changes (even subtly, like a re-published version with the same tag), the build is no longer hermetic. If the network is unavailable, the build fails entirely.

Bazel addresses this with repository rules that download external dependencies during the fetch phase — separate from the build phase — and verify them against expected checksums. But if a rule bypasses this mechanism and hits the network directly during an action, hermeticity is broken.

Timestamps and non-deterministic outputs

Many tools embed timestamps, process IDs, or random values in their outputs. A compiler might include a build date in the binary. A documentation generator might insert the current time in its output. An archiver might record file modification times.

These don't change the functional correctness of the output, but they destroy cache effectiveness. If the same inputs produce different output bytes — even if the difference is just a timestamp — the cache hit rate drops. Across a large organization, this kind of invisible non-determinism can quietly cost hours of wasted computation per day.

Host configuration leakage

Environment variables, locale settings, filesystem ordering, available memory, number of CPU cores — any of these can influence build behavior if a tool reads them. A test that passes when the filesystem returns entries in alphabetical order might fail when it doesn't. A build that fits in 8GB of RAM might OOM on a machine with 4GB.

Bazel's sandboxing is designed to isolate actions from the host environment, but sandboxing has limits. Not all platforms support it equally. Not all rules are compatible with it. And teams sometimes disable it for performance or compatibility reasons — trading hermeticity for speed without fully understanding the trade-off.

Hermeticity Is Hard. That's the Point.

If hermeticity were easy, everyone would have it. The reason it's valuable is precisely because it's difficult to achieve and maintain at scale.

The challenge isn't the initial setup. A small project can be perfectly hermetic with modest effort. The challenge is keeping it hermetic as the codebase grows, as new engineers join the team, as new languages and tools are added, as third-party dependencies evolve, and as the build graph becomes too large for any single person to understand.

Hermeticity violations are silent. They don't cause immediate, obvious failures. They cause intermittent failures — the worst kind. A build that fails 5% of the time. A test that's flaky on Tuesdays. A cache that works for team A but not team B. An artifact that passes all tests but behaves differently in production.

These are the bugs that consume weeks of engineering time. Not because they're intellectually difficult, but because they're invisible. You can't debug what you can't see. You can't fix a hermeticity violation if you don't know it exists.

Why We Named the Company After It

We named Hermetiq after hermeticity because we believe it's the most important — and most underserved — property in build infrastructure.

When we talk to platform engineering teams running Bazel at scale, the same story comes up over and over. The build system is powerful. The caching is fast when it works. The remote execution cluster is provisioned and running. But something is off. Cache hit rates are lower than expected. Builds that should be fast are slow. Tests that passed yesterday fail today for no reason.

In our experience, the root cause is almost always a hermeticity problem — and the reason it goes undiagnosed is that there's no tooling to detect it.

Hermetiq's observability platform is designed, in part, to surface exactly these issues. When we show you that a specific target's cache hit rate dropped from 95% to 40% after a toolchain update, that's a hermeticity signal. When we identify a test that produces different results depending on which remote execution worker it runs on, that's a hermeticity signal. When we trace a build failure to an action that reads an undeclared input, that's the hermeticity violation itself.

Build observability and hermeticity are deeply connected. You can't maintain hermeticity at scale without observability — because hermeticity violations are, by their nature, the things you can't see without instrumentation.

The Spectrum of Hermeticity

It's worth being honest: perfect hermeticity is an asymptote, not a destination. Even Google — which invented the concept — doesn't achieve 100% hermeticity across all builds. The practical goal is to be hermetic enough that your caches are reliable, your builds are reproducible in the common case, and violations are caught and fixed quickly rather than silently accumulating.

Where your organization sits on that spectrum depends on your investment in three things:

1. Toolchain management: Are your compilers, interpreters, and build tools declared as versioned inputs, or do they come from the host machine?
2. Dependency pinning: Are your external dependencies fetched with verified checksums during a controlled fetch phase, or downloaded ad-hoc during the build?
3. Observability: Can you detect when hermeticity degrades — when cache hit rates drop unexpectedly, when builds produce different outputs for the same inputs, when test results vary across environments?

The first two are engineering problems with known solutions. The third is the one most teams neglect — and it's the one that determines whether your hermeticity investment actually holds over time.

Getting Started

If you're running Bazel and want to improve your build hermeticity:

1. Audit your toolchains. Check whether your builds depend on host-installed tools. If which gcc matters, your build isn't hermetic. Move to Bazel-managed toolchains with explicit version pinning.
2. Lock your external dependencies. Every external dependency should have a pinned version and a verified checksum. If a build can produce different results because a remote package changed, you have a hermeticity gap.
3. Enable sandboxing. Bazel's sandboxing catches many classes of undeclared dependencies by restricting what files an action can access. If you've disabled it, understand what you're trading away.
4. Monitor your cache hit rates. Unexplained drops in cache hit rates are the most reliable signal that hermeticity has degraded somewhere. If you're not tracking this metric, hermeticity violations are accumulating silently.
5. Or instrument everything at once. Hermetiq gives you full visibility into your Bazel builds — including cache behavior, execution patterns, and the signals that indicate hermeticity problems — so you can catch violations before they compound.

Hermeticity is the property that makes modern build systems work. It's what turns a build from a hopeful script into a reliable, reproducible, parallelizable machine. Getting it right is hard. Keeping it right is harder. But the teams that invest in it — and in the observability to maintain it — ship faster, debug less, and trust their builds. The teams that don't spend their time chasing ghosts.

Related Articles

• What Is Build Observability? — Apply production-grade monitoring to your build system to detect hermeticity violations.
• We Built Hermetiq to Solve the Problem Created by the AI Coding Boom — The build bottleneck created by AI coding and why observability is critical.
• Why Everyone Is Talking About MCP — Connect AI assistants to your build telemetry for instant root cause analysis.