Chat and inference

Chat does not run inference. The dependency chain is:

Chat UI
   └─ $prompt → system.agents
                   ├─ session lookup, snapshotTree merge, system prompt
                   └─ infer → system.inference
                                  ├─ provider routing (status.get → effective)
                                  ├─ credentials via system.factotum
                                  └─ provider driver (driver-openai, driver-anthropic, driver-ollama)
                   ├─ writes session_state.transcript
                   └─ emits $activity frames → Chat UI

system.inference is owned by @open-matrix/inference. Provider drivers are separate packages (@open-matrix/driver-*). system.factotum (in @open-matrix/factotum) is the only actor that touches credentials.

Where Chat directly touches `system.inference`

Only one place: a dependency probe. MatrixChatApp.buildPromptDependencyProbes() (src/ChatApp.ts:3336-3384) builds two probes:

const probes: PromptDependencyProbe[] = [];

// 1. Agent service (only when conversation.mode === 'remote-actor')
probes.push({
  label:   'agent service',
  target:  resolveChatServiceTarget(this._context, conversationRef),
  op:      '$ping',
  payload: this._principalId ? { principalId: this._principalId } : {},
  timeoutMs: ...,
});

// 2. Inference service (only when not standalone-local)
probes.push({
  label:   'inference service',
  target:  this.resolveRootScopedTarget('system.inference'),
  op:      'status.get',
  payload: {},
  timeoutMs: PROMPT_DEPENDENCY_TIMEOUT_MS,
  validate: (response) => {
    const effective = response.effective;
    if (effective && typeof effective === 'object'
        && typeof effective.provider === 'string'
        && effective.provider.trim().length > 0) {
      return null; // OK
    }
    return 'reachable but no effective provider/model route is configured';
  },
});

These probes run only after a prompt has been sent and no $activity frames arrive within PROMPT_DEPENDENCY_PROBE_DELAY_MS. If both probes succeed, Chat keeps waiting. If a probe returns a validation error or timeout, Chat surfaces a "Dependency Unavailable" message in the transcript explaining what's missing.

That's the entire system.inference surface from Chat's perspective. No inference.complete, no inference.stream, no provider routing. Only a probe to fail honestly.

The historical `DaemonLlmProxy` exception

chat/src/services/DaemonLlmProxy.ts (a headless MatrixActor) accepts an infer op and forwards via FederationPeer.invoke() to a configured target. This is not part of the modern Chat → system.agents flow; it is a federation-routing artifact that lets a chat actor target a remote daemon's LLM-proxy mount via URL parameter ?llm_to=ROOT/mount.

In daemon-hosted and standalone-connected modes, this proxy is rarely used. The canonical path is the system.agents route. DaemonLlmProxy exists for legacy federation scenarios and must be configured explicitly via session-info or URL params.

What you'll see when inference is misconfigured

Symptoms:

User submits a prompt, Chat shows "thinking..." indicator.
After ~2 seconds (the dependency-probe delay), Chat probes system.inference status.get.
If effective.provider is empty, Chat appends a system message:

[Dependency Unavailable: No activity completed on the prompt stream.
 Verify system.agents and system.inference are reachable and healthy.]

The exact wording is in ChatApp.ts:3296. The fix is to configure an effective provider:

bash

matrix invoke system.inference status.get '{}'
# Expected: { effective: { provider: 'anthropic', model: 'claude-...' } }

Configure via the Inference Settings webapp (/apps/inference-settings/) or by editing the inference config file the runtime owns.

OAuth vs API key

Per the auto-memory note "OAuth tokens are NOT API keys" (2026-04-02): Codex/Claude providers use OAuth JWT from a browser flow (~/.codex/auth.json), not static keys. The user must complete an OAuth flow locally; system.factotum reads the resulting credentials. Chat is unaware of this — it sees only system.inference status.get succeeding.

Why probing is the only direct call

Chat is a UI. The architectural rule from CLAUDE.md:

"Inference API calls — Local runtime through Factotum-owned credentials. Tokens are local."

If Chat called inference directly, the browser would need credentials. The whole point of routing through system.agents → system.inference → system.factotum is to keep credentials and provider routing on the trusted runtime side.

Chat and inference ​

Where Chat directly touches system.inference ​

The historical DaemonLlmProxy exception ​

What you'll see when inference is misconfigured ​

OAuth vs API key ​

Why probing is the only direct call ​