At 16:42, Claude Code changed a deployment rule after an MCP catalogue tool reported that ledger-api had no regulated-data tag. The change passed its checks and reached production.

The next morning, the team replayed the same prompt against the same repository commit. Claude Code left the rule alone. The review concluded that the incident could not be reproduced.

One detail had changed. The replay called the live service catalogue. An operator had restored the missing tag overnight.

incident run, 16:42     regulated_data=false   response digest 31a7...
replay, 09:18           regulated_data=true    response digest 8c04...

The team had rerun the task. It had not replayed the incident.

All systems, identifiers, commands, and records below are fictional examples.

Claude Code forensic replay compared with a live rerun

A transcript cannot recreate an external decision

A prompt, chat transcript, and Git commit preserve only part of a Claude Code run. The decision may also depend on MCP responses, policy verdicts, retrieved documents, environment variables, feature flags, clock time, and the authority attached to each tool call.

If one of those inputs drifts, the repeated run answers a different question. A clean result can prove that today’s system behaves correctly. It says little about what the agent saw during the incident.

Start the review with a blunt inventory:

input class                 pinned?    independently captured?
repository tree            yes        yes
system and task prompts     yes        yes
model and parameters        partial    no
MCP responses               no         no
policy decisions            yes        yes
credential scope            no         no
runtime image               yes        yes
clock and feature flags     no         no

That table is more useful than a confident reproduced: false verdict. It tells the reviewer what the test could never have established.

This is also why an audit trail needs proof beyond the transcript. The transcript records what the model said. Incident reconstruction needs the evidence that shaped what it did.

Capture a replay capsule during the original run

Capture the decision inputs before newer state replaces them. Give each item a stable identity and a digest.

replay_capsule:
  operation_id: op-example-913
  repository:
    commit: 7f2c-example
    dirty_tree_digest: sha256:example-tree
  prompts:
    system_digest: sha256:example-system
    task_digest: sha256:example-task
  model:
    provider: example-provider
    model: example-model-4
    parameters_digest: sha256:example-params
  runtime:
    image_digest: sha256:example-image
    started_at: 2026-09-13T16:42:03Z
  authority:
    principal: claude-worker-example-12
    scopes: [repo_write, catalogue_read]
    policy_version: policy-example-44
  tool_evidence:
    - call_id: call-example-07
      server: service-catalogue
      method: service.get
      request_digest: sha256:example-request
      response_digest: sha256:31a7-example
      captured_response_ref: evidence://op-example-913/call-example-07
      observed_at: 2026-09-13T16:42:11Z
  completeness: unproven

Store sensitive values behind a controlled evidence reference instead of copying secrets into a general log. A digest proves which object the run used, but only if an authorised reviewer can retrieve that object later.

The operation ID must also remain stable across the plan, tool calls, effects, and evidence. Two parallel writes sharing an ID can corrupt that chain. I covered that failure in Claude Code gave two parallel writes the same operation ID.

You can download the blank replay capsule YAML and adapt its required fields to your own runtime.

Separate forensic replay from live verification

The two exercises have different jobs.

A forensic replay feeds captured tool responses and policy decisions through a simulator. It asks: given the evidence available at 16:42, can we reproduce the agent’s choice?

A live verification calls current systems in read-only mode. It asks: would the same task cause the same decision now?

Run both, but do not combine their verdicts:

replay_results:
  forensic:
    evidence_time: 2026-09-13T16:42:03Z
    external_calls: blocked
    captured_responses_used: 6
    decision_reproduced: true
  live_verification:
    evidence_time: 2026-09-14T09:18:22Z
    external_calls: allowed_read_only
    changed_inputs: [service_catalogue_tag]
    decision_reproduced: false

When the results differ, the difference is the finding. Here, the forensic replay explains why the deployment rule changed. The live check shows that the repaired catalogue now produces a safer decision.

Make missing evidence stop the replay

A replay harness will eventually meet a missing response, an unsupported MCP method, or a damaged evidence object. The tempting shortcut is to call the live tool and carry on.

Do not do that quietly.

missing_evidence_policy:
  default: stop
  permitted_fallbacks: []
  verdict_on_missing_input: replay_incomplete

If a reviewer permits a live read, record the substitution. Mark every downstream decision as contaminated by current state. The harness must not present that path as faithful reproduction.

This stop rule belongs beside the controls that bind tool effects to the approval. The run should fail closed when it cannot prove either the authority used or the evidence observed.

Compare decisions and effects, not prose

Model wording can change while the operational choice stays the same. Comparing two transcripts line by line creates noise. Compare structured checkpoints instead:

  • chosen tool and intended effect
  • request digest and target scope
  • policy result and effective identity
  • proposed patch digest
  • tests selected and omitted
  • committed external effects
  • stop, approval, and rollback decisions

A divergence record should identify the first point where the paths split:

divergence:
  first_checkpoint: tool_response.service_catalogue
  incident_digest: sha256:31a7-example
  replay_digest: sha256:8c04-example
  downstream_changes:
    - deployment_rule_patch_not_proposed
    - regulated_data_eval_not_triggered
  classification: external_evidence_drift

That record keeps the review on the changed deployment decision instead of harmless differences in phrasing. It also preserves causal order. A reviewer needs to know which input changed first, then which decisions and effects followed.

Test the replay harness before you need it

Build fixtures that prove the harness stops or reports drift correctly:

  1. Remove one captured MCP response. Expect replay_incomplete.
  2. Change an evidence object without changing its stored digest. Expect an integrity failure.
  3. Let the MCP server return a new output schema. Expect no silent coercion.
  4. Omit the effective credential scope or policy version. Expect the authority checkpoint to fail.
  5. Attempt a network call in forensic mode. Expect the sandbox to block it and the eval to fail.
  6. Match the repository commit but change the dirty tree. Expect a state mismatch.
  7. Cross a time-dependent policy boundary. Expect the clock input to appear in the divergence report.
  8. Produce the same patch through a different tool path. Expect the effect comparison to detect it.

The eval should fail whenever the replay reaches a verdict with an unpinned decision input. Reproducing the final patch is insufficient if the path relied on substituted evidence.

Put the verdict in the review packet

The final record should be short enough for an incident reviewer to challenge:

replay_verdict:
  capsule_complete: true
  forensic_decision_reproduced: true
  live_decision_reproduced: false
  first_divergence: service_catalogue.response
  external_effects_replayed: false
  unexpected_live_calls: 0
  reviewer_decision: incident_explained_by_catalogue_drift

External writes should normally be simulated or sent to an isolated target. A forensic replay exists to explain an effect, not cause it again.

Then connect the verdict to rollback. Test the recovery against the captured incident state as well as today’s repaired state. If rollback works only after the catalogue was fixed, the team still does not know whether it could have recovered safely at 16:42.

My rule is simple: do not call a rerun a replay when it reads new evidence. Pin every input that can change the agent’s decision, stop when one is missing, and report the first structured divergence between the incident path and current behaviour.

Claude Code: Building Production Agents That Actually Scale covers tool evidence, bounded permissions, failure-path evals, rollback, and review packets for coding agents that can change real systems.