Hermetiq Cloud pilot: execution, cache invalidation, and a controlled test repair Recorded November 19, 2026. Bazel 9.2.0, macOS arm64 client, shared Hermetiq Cloud Buildbarn workers. No customer source, credentials, or customer workload. What ran The manifest rule declares one text input and one environment variable. A shell action writes the mode and copies the input. The sample test deliberately fails for TEST_RUN_NUMBER=2 until the one-line repair removes that condition. It is not a naturally flaky test or a demonstration of automatic root-cause repair. All six scenarios used new output bases. The first disabled remote cache reads. The test scenarios used --runs_per_test=3 and --cache_test_results=no. Observed cold: 1 remote manifest execution, exit 0. warm: 1 remote cache hit from a fresh output base, exit 0. environment-change: PILOT_MODE changes from release to debug; 1 remote execution, a different action key and output digest, exit 0. restored: release restores the original key and remote cache hit, exit 0. test-failure: remote runs 1 and 3 pass; run 2 fails; Bazel exits 3. test-fixed: after the supplied patch, all three remote runs pass; Bazel exits 0. Hermetiq verification The invocations appeared in the authenticated Hermetiq dashboard. On the changed manifest invocation, Cache Metrics reported one remote Action Cache lookup, one miss and Env Changed. A project-scoped Barn Owl investigation called find_cache_events and compare_build_configurations. The retained tool-result excerpts identify ENV_CHANGED and the added --define=PILOT_MODE=debug. The independent Bazel action records show the exact release-to-debug value change, unchanged shell arguments and input, and the restored hit. The Command digest includes environment variables; its change does not imply changed argv. Reproduce in your own project 1. Extract the ZIP into a new directory. Install Bazelisk. The .bazelversion file selects 9.2.0. Keep the original BUILD.bazel for the first five scenarios. 2. Connect this workspace to your own Hermetiq Cloud project using Quickstart. Install its credential helper and store your own credentials outside source. Enable the shared-worker remote-execution settings in the hermetiq config. No helper, credentials, or usable project configuration is included here. See https://www.hermetiq.com/docs/remote-execution#managed-rbe-start 3. Use the commands below. Choose a new directory for PILOT_OUTPUT. Do not reuse another experiment's output bases. Omit disk caching from this config. PILOT_OUTPUT=$(mktemp -d) bazelisk --batch --output_base="$PILOT_OUTPUT/cold" build --config=hermetiq //:manifest --noremote_accept_cached --remote_download_outputs=all bazelisk --batch --output_base="$PILOT_OUTPUT/warm" build --config=hermetiq //:manifest --remote_download_outputs=all bazelisk --batch --output_base="$PILOT_OUTPUT/changed" build --config=hermetiq //:manifest --define=PILOT_MODE=debug --remote_download_outputs=all bazelisk --batch --output_base="$PILOT_OUTPUT/restored" build --config=hermetiq //:manifest --remote_download_outputs=all bazelisk --batch --output_base="$PILOT_OUTPUT/test-before" test --config=hermetiq //:test_sample --runs_per_test=3 --cache_test_results=no --test_output=errors patch -p1 < fix.diff bazelisk --batch --output_base="$PILOT_OUTPUT/test-after" test --config=hermetiq //:test_sample --runs_per_test=3 --cache_test_results=no --test_output=errors The penultimate test command is expected to fail. Check each execution log and testResult event; a green cached result does not establish a fresh test run. To capture your own records, add --execution_log_json_file= and --build_event_json_file= to each command. Do not overwrite this bundle's recorded files. Your invocation IDs, platform paths and action keys may differ. Verify this recording offline python3 verify.py The verifier checks action keys and outputs across the four build cases, unchanged inputs/argv/platform, all six test outcomes and remote test-setup actions, the exact repair, and every SHA256SUMS entry. Bazel also labels its test.xml generation actions TestRunner; some of those auxiliary actions hit cache in the fixed run. The actual test-setup actions all executed remotely. No service or credentials are needed to verify the published recording. Evidence format and limits recorded/*.actions.json contains selected original execution-log fields, not complete logs. recorded/*.bep-excerpt.json retains selected started, finished, testResult and testSummary events. Project IDs in artifact URIs are replaced by DEMO_PROJECT. Text logs replace local recording/workspace paths with placeholders. The two tool-result excerpts select fields from the actual Barn Owl tool details; they omit project identifiers, user identifiers, local paths and unrelated fields. recorded/runs.json records invocation IDs, outcomes and output hashes. Shared service versions were not pinned; this is a dated observation, not a cross-version guarantee. It does not measure production speedups, detection accuracy, migration effort or universal workload compatibility. Three passes after removing a deliberately failing condition do not establish that an arbitrary real test is free from intermittent failures. References https://bazel.build/remote/cache-remote https://bazel.build/reference/test-encyclopedia https://github.com/bazelbuild/remote-apis/blob/main/build/bazel/remote/execution/v2/remote_execution.proto