Claude Code repairs an authentication refresh path and runs the focused test:
npm test -- refresh-token
It passes. While the agent prepares its summary, a dependency bot changes package-lock.json. A developer then rebases the branch onto a commit that changes shared test setup. Claude Code runs the full suite against this new state:
npm test
That passes too.
The final handoff says all tests passed. Every word is technically true, but the claim is still misleading. The focused test belongs to the old branch and lockfile. The broad suite belongs to a different working tree. Neither result proves that the final patch passed both checks in one known state.
A command and exit code are not enough. Each result needs a run fingerprint that identifies the code and tool state that produced it.
record the state beside the result
A useful evidence record starts with the command, but it does not stop there:
evidence:
command: npm test -- refresh-token
exit_code: 0
started_at: 2026-08-05T09:42:11Z
finished_at: 2026-08-05T09:42:39Z
fingerprint:
id: run-state-42
commit: 8f31c2a
working_tree_digest: sha256:b71d...
lockfile_digest: sha256:2ac1...
test_config_digest: sha256:e17a...
test_runner: vitest@3.2.4
tool_policy: claude-code-policy-v7
mcp_servers:
- name: repository-context
version: 2.4.1
Now a reviewer can compare the result with the patch under review. If the final candidate has commit af42e91 and a different lockfile digest, the focused result is stale. It still helps diagnose the original failure, but it cannot approve the new state.
Do not fingerprint the entire machine. Record the inputs that can change the behaviour you care about. For a unit test, that may be the commit, uncommitted files, dependency lock, test configuration, runtime, and test runner. For an integration test, add the container image, named environment profile, feature flag version, and relevant MCP server identity.
The record should contain identifiers and digests, not secrets. Do not copy tokens, passwords, customer data, or raw environment values into a review packet. Hashing a short or guessable secret does not make it safe. Record the approved configuration profile and the presence of a required variable instead.
fingerprint before and after the command
Capturing state only after execution leaves a race. A formatter, another agent, or a developer can change a relevant file while a long suite is running. The command returns zero, but it tested a moving target.
Capture the fingerprint when the command starts. Check it again when the command finishes:
validation_result:
command: npm test
started_on_fingerprint: run-state-44
finished_on_fingerprint: run-state-45
exit_code: 0
status: state_changed_during_execution
changed_inputs:
- tests/fixtures/auth-session.json
usable_for_approval: false
rerun_required: true
Keep the output. It may explain a failure or reveal an unstable fixture. Just do not present it as current evidence for approval.
This matters even more when several agents share a checkout. One subagent can run tests while another edits a shared fixture. The safest answer is isolation: one worktree or ephemeral checkout per serious run. A fingerprint remains useful because external dependencies, policy, and MCP tools may still change around that isolated repository.
define invalidation before Claude Code starts
If Claude Code decides whether evidence is stale while writing its final summary, the rule is too easy to bend. Put invalidation in the task contract:
evidence_policy:
invalidate_when:
- base_commit_changes
- relevant_working_file_changes
- dependency_lock_changes
- test_config_changes
- tool_policy_changes
- mcp_server_version_changes
on_invalidation:
preserve_old_result: true
mark_status: stale
request_smallest_valid_rerun: true
completion_requires:
- all_required_checks_current
- all_results_match_candidate_fingerprint
The smallest valid rerun is important. A lockfile change does not always justify running every expensive check again. Map each input to the evidence it can invalidate. A change to the authentication fixture may require the focused authentication tests and their dependent integration suite. It should not automatically trigger an unrelated mobile build.
The runtime should make that relationship explicit rather than leaving the model to improvise a cheap answer under time or cost pressure.
show stale evidence in the review packet
A Claude Code review packet should not flatten mixed results into passed and failed. Show which result belongs to the current candidate and which result needs replacement:
review_packet:
candidate_fingerprint: run-state-44
validation:
- command: npm test -- refresh-token
fingerprint: run-state-42
exit_code: 0
status: stale
reason: base commit and lockfile changed
- command: npm test
fingerprint: run-state-44
exit_code: 0
status: current
missing_evidence:
- npm test -- refresh-token on run-state-44
decision: not_ready_for_review
That packet tells the reviewer exactly what is missing. It also prevents a polished agent summary from combining two separate truths into one stronger claim.
Bind the final packet to the candidate commit after the rerun. If Claude Code edits the patch again, invalidate the affected evidence again. A green result belongs to a particular state, not to a branch name or pull request forever. This is the same reason an agent must prove diff ownership before it commits.
make the run replayable
A transcript can explain what Claude Code said. It cannot reconstruct a missing dependency version or identify which policy an MCP server enforced. A replay record needs the candidate commit, relevant file digests, dependency set, command order, runtime versions, configuration profile, tool policy, and MCP endpoints used.
If the environment cannot be rebuilt, state that limitation. Replay does not mean asking the model to tell the same story twice. It means giving another engineer enough recorded state to reproduce the checks and understand why a result was accepted.
The same distinction applies outside the repository. An MCP effect receipt proves what an external write changed. A run fingerprint proves which code and tool state produced a command result. Production workflows need both when tests and external actions share one agent run.
turn state drift into an eval
Build fixtures that move the state between steps:
- update the lockfile after focused tests finish
- rebase the branch before the broad suite starts
- change the test configuration while a command runs
- upgrade an MCP server during the task
- let a subagent modify a shared fixture after another agent tests it
The agent passes the eval when it preserves the old result, marks it stale, names the changed input, and requests the smallest necessary rerun. It fails when it carries the green result into the final approval claim.
My operating rule is blunt: Claude Code cannot say tests passed unless the required results match the exact state under review.
If your team needs Claude Code runs that can be reviewed and replayed, Claude Code: Building Production Agents That Actually Scale covers the operating layer around test evidence, MCP tools, permissions, evals, rollback, and human approval. Leanpub readers receive future updates as these production patterns develop.