Evaluation schema

The shapes a harness produces and consumes. All are defined in harness/HarnessRun.ts and harness/HarnessProtocol.ts.

`HarnessStatus`

type HarnessStatus = 'running' | 'converged' | 'failed' | 'stopped';

`HarnessRun`

harness/HarnessRun.ts:25-34:

interface HarnessRun {
  readonly runId: string;
  readonly targetRef: TargetRef;
  readonly harnessKind: string;
  readonly status: HarnessStatus;
  readonly iterations: readonly HarnessIteration[];
  readonly activeSessions: Readonly<Record<string, HarnessActiveSession>>;
  readonly createdAt: number;
  readonly updatedAt: number;
}

activeSessions is keyed by profileKey — at most one active session per profile.

`HarnessIteration`

harness/HarnessRun.ts:12-23:

interface HarnessIteration {
  readonly iteration: number;
  readonly promptHash: string;
  readonly agentSessionId: string;
  readonly profileKey: string;
  readonly gitSha?: string;
  readonly score?: number;
  readonly constraintResults?: Readonly<Record<string, boolean>>;
  readonly trace?: string;
  readonly startedAt: number;
  readonly completedAt?: number;
}

Notes:

promptHash — the hash of the rendered prompt for replay/comparability.
score — free-form number; semantics defined by the harness, not by the schema.
constraintResults — per-named-check pass/fail.
gitSha — optional pin to the repo state at the time of the iteration (used by propose runs).
trace — opaque pointer to the activity-frame log (a sessionId+requestId pair, encoded).

`HarnessActiveSession`

harness/HarnessRun.ts:5-10:

interface HarnessActiveSession {
  readonly profileKey: string;
  readonly workerRef?: WorkerRef;
  readonly runtimeKind?: WorkerKind;
  readonly sessionId: string;
}

Pure constructors

harness/HarnessRun.ts:36-78:

function createHarnessRun(input): HarnessRun;
function attachHarnessSession(run, session, now?): HarnessRun;
function recordHarnessIteration(run, iteration, now?): HarnessRun;
function completeHarnessRun(run, status, now?): HarnessRun;

All four are pure: they return a new HarnessRun with updatedAt advanced. Callers persist the returned value.

`HARNESS_ACCEPTS`

The protocol accepts (harness/HarnessProtocol.ts:3-9):

Op	Description
`harness.start`	Begin a harness workflow run
`harness.stop`	Stop an active harness run
`harness.steer`	Inject human guidance into an active harness run
`harness.status`	Return current harness phase, iteration, score, and active sessions
`harness.history`	Return harness iteration history with agent session pointers

These are the names a harness implementation exposes; @open-matrix/agents ships the contract but not a reference implementation. Smithers implements the equivalent flow with its own op names (smithers.start, etc.).

`HARNESS_EVENTS`

const HARNESS_EVENTS = {
  'iteration.start':      { description: 'Harness iteration started' },
  'agent.dispatched':     { description: 'Harness dispatched an agency profile session' },
  'evaluation.complete':  { description: 'Harness evaluation completed' },
  'agent.streaming':      { description: 'Harness observed agent activity' },
  converged:              { description: 'Harness reached convergence' },
};

`HarnessStatusSnapshot` (returned by `harness.status`)

interface HarnessStatusSnapshot {
  readonly runId: string;
  readonly status: HarnessStatus;
  readonly phase?: string;
  readonly iteration: number;
  readonly score?: number;
  readonly activeSessions: readonly {
    readonly profileKey: string;
    readonly sessionId: string;
    readonly runtimeKind?: string;
  }[];
}

`AgencyInvokeInput` / `AgencyInvokeResult`

Defined in harness/AgencyInvokeTool.ts and exported from the package. These are the input/output shapes for agencyInvoke() calls — the canonical entry point for spawning agency-profile sessions from a harness.

Evaluation schema ​

HarnessStatus ​

HarnessRun ​

HarnessIteration ​

HarnessActiveSession ​

Pure constructors ​

HARNESS_ACCEPTS ​

HARNESS_EVENTS ​

HarnessStatusSnapshot (returned by harness.status) ​

AgencyInvokeInput / AgencyInvokeResult ​

See also ​