Chat and agents

Chat is a UI for a remote cognitive runtime. The cognitive runtime is system.agents (in @open-matrix/agents). Chat sends $prompt ops there and renders the streaming $activity frames it receives. Sessions, memory, and embeddings all live in system.agents (or its compat aliases). Chat owns no cognition.

Adapter pattern

Chat declares a IChatConversationService contract (chat/src/contracts/IChatConversationService.ts):

interface IChatConversationService {
  prompt(request: IChatConversationPromptRequest): Promise<IChatConversationPromptResponse>;
  cancel(request: { requestId: string; targetMount?: string }): Promise<{ ok: boolean }>;
  listSessions(request: { principalId?: string }): Promise<{ sessions: IChatSessionSummary[] }>;
  deleteSession(request: { sessionId: string }): Promise<{ deleted: boolean }>;
  readState(request: { sessionId: string; key: string }): Promise<{ ok: boolean; data: unknown }>;
  writeState(request: { sessionId: string; key: string; data: unknown }): Promise<{ ok: boolean }>;
}

Two implementations:

LocalConversationService (chat/src/app/services/impl/) — fully in-memory, no actor. Used by standalone-local mode.
RemoteActorConversationAdapter (chat/src/app/services/adapters/) — calls actors over the bus. Used by standalone-connected and daemon-hosted modes.

RemoteActorConversationAdapter.prompt():

RequestReply.sendToInbox(this._context, promptTarget, '$prompt', {
  prompt: request.text,
  requestId: request.requestId,
  traceId: request.traceId,
  activityTo: request.activityTo,
  callerSnapshot: request.callerSnapshot,
  ...(callerRealm ? { callerRealm } : {}),
  ...(callerMount ? { callerMount } : {}),
  sessionId: request.sessionId,
  principalId: request.principalId,
});

It uses sendToInbox (fire-and-forget) rather than execute, because the response arrives as $activity frames, not as a request-reply pair. The requestId correlates frames to the originating prompt.

For session list / read / write, the adapter uses RequestReply.execute against _backendTarget (typically system.agents):

'$sessionList'         { principalId? }     timeoutMs: 10_000
'$sessionDelete'       { sessionId }        timeoutMs: 10_000
'session_state.read'   { sessionId, key }   timeoutMs: 10_000
'session_state.upsert' { sessionId, key, data } timeoutMs: 10_000

The chat-component surface uses execute, not sendToInbox

chat-component/src/surface/ConversationSurfaceActor.ts uses RequestReply.execute for its $prompt to receive an immediate ACK (line 389-395):

const ack = await RequestReply.execute(
  this._context!,
  'system.agents',
  '$prompt',
  payload,
  this._requestOptionsForRoot(serviceRoot, 30_000),
);

The reason is recorded in the chat-side auto-memory (Inference Deadlock note 2026-04-17): RPC $prompt deadlocks observed inside the same NATS process motivated the chat-app side to switch to sendToInbox. The surface variant uses execute because it operates from a different lifecycle: it expects an ACK before declaring the request "in flight" so it can disable the input. The $activity stream still drives the rendering.

The pragmatic split: app-side uses sendToInbox to avoid deadlock; surface-side uses execute for ACK semantics. Both rely on $activity frames for streaming.

$activity frame consumption

MatrixChatApp.on$Activity (chat/src/ChatApp.ts:~2200-2400) and ConversationSurfaceActor.on$Activity (chat-component/src/surface/ConversationSurfaceActor.ts:191-287) handle the same frame kinds:

phase: 'thinking' | 'token' | 'tool-start' | 'tool-result' | 'done' | 'error' | 'cancelled'

Each phase maps to a renderer op:

Phase	Op into renderer
`thinking`	`renderer.stream-thinking { text }`
`token`	`renderer.stream-append { text }`
`tool-start`	`renderer.add-tool { op, target }`
`tool-result`	`renderer.add-tool-result { op, summary, duration }`
`done`	`renderer.stream-end { text }` (and updates `sessionId` from `data.sessionId`)
`error`	`renderer.add-message { role: 'error', content }`
`cancelled`	`renderer.add-message { role: 'system', content: 'Request cancelled' }`

Snapshot tree (target context)

When the user prompts an actor that is not itself system.agents (e.g., asking "what does this do" about system.config), Chat passes a snapshotTree describing the target:

{
  mount, root,
  tag,                                     // componentClass
  accepts: [{ op, params: [...] }, ...],   // from $introspect
  emits: [...],
  skills: [...],
  cognitiveState: { purpose, systemPrompt, memoryScope, regime },
  children: [...],
}

system.agents uses this snapshot to give the answering model context about what target the user is asking about.

What's not yet proven E2E

WORKSTREAMS/loose-ends/items/P1.11-cognitive-product-proof.md (P1, TODO as of 2026-05-05) describes the missing E2E test:

User asks Director: "what is system.registry for?" → Director sends $prompt to system.agents → AgentActor calls system.inference (mock) → ActivityFrames stream back → session persists → Chat sees the session.

The code wiring is verified. The proof script does not yet exist. Treat the call chain in this page as wired but not yet proven end-to-end.

Chat and agents ​

Adapter pattern ​

The chat-component surface uses execute, not sendToInbox ​

$activity frame consumption ​

Snapshot tree (target context) ​

What's not yet proven E2E ​