Validate Bazel changes with the right tests and evidence

A patch can pass the test beside the edited file and still break a consumer in another package. Before accepting a Bazel change, check which tests were selected, which configuration they used, and whether their results came from a new execution or a cached result.

This matters when a coding agent prepares the patch, but the same problem exists in a manual workflow. “Tests passed” is useful only with the scope attached. A successful command that tested one target cannot establish that every affected consumer still works.

This guide builds a small example, uses reverse dependencies to find a missed test, and records enough evidence to distinguish a narrow pass from a broader failure. It also explains where this technique stops being a safe shortcut.

Reproduce a change that passes the nearest test

Our example has a shared shell function and two tests in different Bazel packages. The library's test checks that an identifier is nonempty. The API's test checks the identifier format promised to its consumer. That deliberately weak library assertion makes the coverage gap easy to see.

Download the example source and runner. The example was run locally on Linux x86-64 with Bash, Bazel 8.4.2 and rules_shell 0.6.1. Those are reproduction pins, not a recommendation to change your project's versions. The first run needs network access to fetch Bazel dependencies. No Hermetiq service, remote cache or remote executor is required or used.

The source is small enough to inspect before running it. MODULE.bazel declares the shell rules:

module(name = "change_validation_demo")
bazel_dep(name = "rules_shell", version = "0.6.1")

The shared function in lib/id.sh returns an identifier:

format_id() { printf 'user-%s\n' "$1"; }

lib/nonempty_test.sh checks only that a value exists:

#!/usr/bin/env bash
set -euo pipefail
source "$1"
[[ -n "$(format_id 42)" ]]

api/id_contract_test.sh checks the consumer's stronger requirement:

#!/usr/bin/env bash
set -euo pipefail
source "$1"
actual="$(format_id 42)"
[[ "$actual" == "user-42" ]] || {
  printf 'expected user-42; got %s\n' "$actual" >&2
  exit 1
}

Both tests declare the shared script as an input. lib/BUILD.bazel exports it and defines the nearby test:

load("@rules_shell//shell:sh_test.bzl", "sh_test")

exports_files(["id.sh"], visibility = ["//visibility:public"])

sh_test(
    name = "nonempty_test",
    srcs = ["nonempty_test.sh"],
    data = ["id.sh"],
    args = ["$(rootpath :id.sh)"],
)

api/BUILD.bazel declares the cross-package dependency:

load("@rules_shell//shell:sh_test.bzl", "sh_test")

sh_test(
    name = "id_contract_test",
    srcs = ["id_contract_test.sh"],
    data = ["//lib:id.sh"],
    args = ["$(rootpath //lib:id.sh)"],
)

From the extracted example directory, make the test scripts executable and establish the baseline:

chmod +x lib/nonempty_test.sh api/id_contract_test.sh
bz() { bazel --ignore_all_rc_files "$@"; }
evidence="$(mktemp -d)"

bz test //... \
  --build_event_json_file="$evidence/baseline.bep.json" \
  --test_output=errors

The bz function isolates this disposable example from your normal .bazelrc settings. Do not use it to strip required configuration from a real project. There, retain the repository's CI wrapper, platform settings and approved execution configuration.

Now change the separator in the helper and run only its nearby test:

printf '%s\n' 'format_id() { printf "user_%s\n" "$1"; }' > lib/id.sh
bz test //lib:nonempty_test

The test passes: user_42 is still nonempty. The assertion does exactly what it says. It simply does not protect the API's format contract.

Select consumers through the declared dependency graph

Use Bazel's reverse-dependency query to find tests that depend on the changed source file:

if ! bz query 'tests(rdeps(//..., //lib:id.sh))' \
  --output=label --order_output=full \
  > "$evidence/affected-tests.txt"; then
  echo "Target query failed; no tests will be run." >&2
  exit 1
fi

cat "$evidence/affected-tests.txt"

The result in this example is:

//lib:nonempty_test
//api:id_contract_test

rdeps(universe, changed_targets) traverses reverse dependencies inside the chosen universe's dependency closure. tests(...) selects test rules and expands test suites. The Bazel query guide and query reference document these operations.

Save the selection before executing it. In this controlled example, both returned labels are appropriate to run:

if [[ ! -s "$evidence/affected-tests.txt" ]]; then
  echo "No tests selected; review the selection before continuing." >&2
  exit 1
fi

status=0
bz test --target_pattern_file="$evidence/affected-tests.txt" \
  --build_event_json_file="$evidence/changed.bep.json" \
  --test_output=errors \
  > "$evidence/changed.log" 2>&1 || status=$?

cat "$evidence/changed.log"
printf 'Bazel exit code: %s\n' "$status"

The consumer test fails with:

expected user-42; got user_42

Bazel returned exit code 3 in our run: the build completed, but a test failed. Preserve that status in a CI wrapper rather than letting a later successful cat, upload or summary command replace it. Bazel documents its exit-code meanings, including the distinction between test failures and environmental failures.

The query guard stops the workflow on a loading error; a partially written target file is not a valid selection. The test snippet displays the expected failure so you can continue this tutorial. A CI wrapper must retain the artifacts and return that original nonzero status. The downloadable demonstration runner instead checks that the observed failure is exactly the expected one.

This is the useful result of the example: the source change was visible to Bazel, and normal dependency-aware execution exposed the broken consumer once the consumer was actually selected. Disabling caches would not repair an omitted test.

Know when reverse dependencies are insufficient

The query gives you graph relationships, not a proof of complete behavioral coverage. Apply a broader validation policy when the change can invalidate the assumptions behind that graph.

The distinction between query and cquery also matters. Traditional query operates before configuration choices are resolved, so it can include alternatives from select(). cquery analyzes a configured graph using build options. It can help investigate the platform-specific relationship, but it does not support the tests() operator; swapping the command name in the example is not sufficient. See the configured-query documentation.

A smaller universe is another deliberate limitation. Replacing //... with //lib/... would omit the API consumer in this example. In a large repository, scope the query for cost and relevance, but record the boundary. Do not describe a subtree result as repository-wide coverage.

Finally, review execution policy before feeding query output directly into a production job. Bazel's manual tag excludes tests from ordinary bazel test wildcard expansion, but not from query expansion. Explicitly passing those labels can therefore select work your normal wildcard command would omit. Platform-incompatible explicit targets can also fail rather than being skipped. Preserve your CI's approved test selection, prerequisites and platform policy. These behaviors are documented in Bazel's common rule attributes.

Use cached results deliberately

A cached passing test is useful evidence when the test's declared inputs and execution assumptions are valid. It should not be counted as another independent execution.

Bazel's default --cache_test_results=auto reuses eligible results and reruns tests in documented cases, including detected dependency changes and prior failures. Set --cache_test_results=no when you specifically need the selected tests to execute again. That does not require rebuilding every prerequisite from scratch. See the test-caching options.

We also checked normal reuse and deliberate re-execution against the passing version. To reproduce those checks, first restore the original helper and execute both tests:

cat > lib/id.sh <<'RESTORED'
format_id() { printf 'user-%s\n' "$1"; }
RESTORED
bz test //... --cache_test_results=no

# Normal unchanged rerun: eligible test results can be reused.
bz test //...

# Deliberate fresh test execution; build prerequisites may still be reused.
bz test //... --cache_test_results=no

The recorded results were:

These are observations from a synthetic local example, not a customer benchmark or a performance comparison. They demonstrate selection and result provenance, not the quality of an AI-generated patch or a remote execution service.

Do not make cache disabling a reflex across every presubmit. First identify the question. Checking a normal source edit, probing a suspected flaky test, and validating a changed execution environment need different evidence. A non-hermetic test may depend on something its cache key does not represent; repeatedly forcing it to run does not fix the missing dependency. Bazel's test specification describes the intended dependency and execution model.

Keep failures and repetitions distinguishable

If a failure disappears on retry, retain the failed attempt. Bazel can report a test as FLAKY after a failed attempt followed by success, while treating it as passed for the command's exit status. A green exit code alone can therefore hide information a reviewer needs.

For a bounded repeatability check, explicitly set both the number of runs and the retry limit:

bz test //api:id_contract_test \
  --cache_test_results=no \
  --runs_per_test=5 \
  --flaky_test_attempts=1 \
  --build_event_json_file="$evidence/repeated.bep.json"

Run this after restoring the correct separator. Our five runs passed. That establishes only that these five executions passed under the recorded conditions; it does not establish that a real test is flake-free. Choose a repetition budget appropriate to the failure and cost. The test-run and retry options explain their different roles.

Do not respond to the example's contract failure by weakening the assertion or adding retries. Decide whether the format change is intended. If it is, update the contract and consumers through the normal review process. If it is not, fix or revert the implementation, then rerun the appropriate tests.

Record enough evidence to review the result

A useful change-validation record contains:

• The base revision, proposed revision and exact patch, including uncommitted edits when applicable.
• The selection query, its universe, the saved target list and intentional exclusions.
• The Bazel version, CI command, relevant configuration and platform.
• The command exit code, test outcomes, cached-result information and all retry or repeated-run outcomes.
• The invocation reference and retained failure logs, with a note about anything that did not run.

The JSON BEP files above make test outcomes inspectable without scraping terminal formatting. TestResult identifies individual attempts, shards and runs; TestSummary aggregates a test's outcome. The BEP glossary explains these event roles.

Bazel's versioned BEP schema also distinguishes locally cached test results from remote-cache information in execution metadata. A cached result can carry a start time from before the current invocation. Do not infer a fresh execution merely because the test appears in today's event file.

The included runner retains separate logs and BEP files for each phase. Keep referenced test outputs too: an event file that points to an ephemeral local path is not a durable archive of that log. If a run is interrupted or its evidence is incomplete, report that limitation instead of filling in a pass.

Make the agent's handoff reviewable

When a coding agent validates a patch, ask it to return the selected targets, selection rationale, actual command, result provenance and remaining gaps alongside the patch. Keep repository execution permissions and merge requirements in your existing CI and review process.

For teams investigating Bazel outcomes across builds, Hermetiq Build Analytics & AI connects build evidence with Barn Owl investigations while letting you keep your existing infrastructure. Available analysis depends on the data sources and access connected to your deployment. Target-selection policy, test adequacy and change approval still require your project's engineering judgment.

Start with one concrete change. Confirm that its consumers were considered, preserve the exact result, and make the next verification step explicit. That gives an engineer something stronger to review than “the tests passed.”

Related reading:

• Adopt an AI build engineer without handing it production control
• Bazel BEP integration guide
• Recorded AI-assisted Bazel build fix

• Bazel performance guides