Skip to content

Tool testing

Three layers of testing exist for tools today, plus one that is filed as TODO.

Unit tests on the dispatch layer

The runtime/__tests__/ directory holds tests for the runtime adapters and worker projection. These are the mechanical tests: given a tool projection, the adapter emits the correct sink events and surfaces the correct error codes. They are the fastest signal that a refactor of executeActorOracle did not break the dispatch path.

Run with the standard package test:

bash
cd projects/matrix-3
pnpm --filter @open-matrix/agents test

Integration tests through MCP bridge

McpBridgeActor is exercised by integration tests that mount it on a test runtime, fire a tool.invoke, and check the result envelope. These cover:

  • allowedTargets enforcement.
  • approvalRequiredFor round-trip.
  • Error propagation when the underlying actor op returns ok: false.

Deterministic profile for end-to-end work

The evaluate profile (BUILT_IN_AGENCY_PROFILES key: 'evaluate') exists specifically to be a deterministic loop: runtime.preferred = 'deterministic', tools.toolContextRefs = ['test-runner'], context.includeSnapshot = false. It is the closest thing today to a "tool harness" — drive a known input through a frozen tool set and assert on the output.

What's NOT tested today

prove-cognitive-core.ts is filed in WORKSTREAMS/loose-ends/items/P1.11-cognitive-product-proof.md as the missing E2E that exercises tools through Director/Chat with a mock provider. It does not exist. Status: TODO. When it lands, it becomes the canonical full-stack tool test.

The acceptance ladder there is:

1. Director "what is this actor for?" → prompt → response
2. ActivityFrames captured ($thinking, $token, $response)
3. Session persisted; second invocation finds it
4. Chat targeting same TargetRef sees session group
5. Exit code 0 on success

Until the script exists, the manual test is:

bash
hivecast install --home /tmp/test-home --no-start
hivecast start --home /tmp/test-home
# Open http://127.0.0.1:3100/apps/director/, select an actor, ask "what is this for?"
# Watch system.agents traces live.

See also

Source: projects/matrix-3/packages/agents/src/runtime/__tests__/, WORKSTREAMS/loose-ends/items/P1.11-cognitive-product-proof.md (P1.11 — TODO).