Trace replay

Trace replay means: take the activity-frame log of a past session, feed those frames back into the system, and compare new behavior to old. It's how a meta-harness validates that a prompt or tool change actually changed the agent's outputs rather than just the model's.

What you have to work with today

The platform persists everything you need to reconstruct a session:

Resource	Location
Messages (full)	`session_messages` SQLite rows
Tool args/results	embedded in those rows (`tool_args`, `tool_result`)
Activity frames	`IAgentSessionStore.loadActivityFrames(sessionId, requestId?)`
Snapshot at session start	`session_state(session_id, key='snapshot')`
REPL state	`session_state(session_id, key='repl_state')`
Prompt templates	`<host-home>/prompts/` if user-overridden, else bundled

Reconstructing means: load the snapshot, load the prior messages, load the frames, and replay either the LLM output (deterministic re-prompt) or the tool dispatches (deterministic invoke loop).

The `$traceQuery` accept

AgentsRoot.accepts.$traceQuery returns activity frames for (sessionId, requestId). This is the canonical read path.

bash

matrix invoke system.agents '$traceQuery' '{ "sessionId": "ses_…", "requestId": "req_…" }'

Returns { ok: true, frames: ActivityFrame[] }. Frames carry enough information (target/op/args for tool.call, result/durationMs for tool.result) to reconstruct what happened.

Replay strategies

Two strategies are useful in practice:

Tool replay. Walk the frames in order; for each tool.call, re-invoke the same (target, op, args) and compare the result against the recorded tool.result.data. Differences indicate underlying-actor non-determinism (e.g. a search hit a different memory because new memories were added).
Prompt replay. Reconstruct the system prompt + prior messages, then re-issue the prompt to the model with the same provider/model. The new response is compared against the recorded final response. Use temperature: 0 for any chance at determinism.

There is no built-in command for either today; both are programmable through the actor surface.

What's NOT wired

A first-class harness.replay op or CLI. You write it on top of $traceQuery and AgencyInvokeTool for now.
A diff renderer. Director's session view shows live frames but not "this run vs that run."
Replay against a different model. The recorded frames don't carry full prompt context — to replay against another model you must reconstruct the prompt yourself from the snapshot + prior messages + active prompt template.

Operator considerations

Replay is read-heavy. If you're replaying many sessions, RecordBackedSessionStore (against the observability record store) scales better than FileAgentSessionStore.
Replays produce new activity frames, even if the new run is identical to the old one. They don't share frames.
Replay against the live runtime competes for the same actor as production traffic. For comparisons, run replays in a sandbox Host (a separate --home).

Trace replay ​

What you have to work with today ​

The $traceQuery accept ​

Replay strategies ​

What's NOT wired ​

Operator considerations ​