Runtime smoke tests

A runtime smoke test brings up a real Host process — bundled NATS, gateway, runtime supervisor — into a temp directory, installs the package under test, starts the host, asserts that healthcheck passes, then tears everything down. It catches integration failures unit tests cannot see: install scripts, port conflicts, runtime supervision, gateway routing.

The fixture lives in @matrix/test-utils: createDaemonHarness in projects/matrix-3/packages/test-utils/src/daemon-harness.ts.

Note: The function is named createDaemonHarness for historical reasons — when the SDK's host process was called matrixd. The retired daemon path is intentionally disabled for product work; the harness now exercises Host Service via the same install-and-start scripts. The name will change with a future deprecation cycle. See Deprecated APIs.

The harness shape

interface DaemonHarness {
  readonly rootDir: string;            // temp directory
  readonly matrixDir: string;          // <root>/.matrix
  readonly brokerPort: number;         // NATS client port
  readonly wsPort: number;             // NATS WebSocket port
  readonly httpPort: number;           // gateway HTTP port
  readonly httpUrl: string;            // 'http://127.0.0.1:<httpPort>'
  readonly realm: string;              // unique authority root
  readonly pidFile: string;
  readonly projectsPath: string;
  readonly packagesPath: string;

  createProject(name?: string): string;
  installWorkspacePackage(packageName: string): void;
  start(opts?: { matrixdPath?, fromPath?, webapps?, timeoutMs? }): Promise<void>;
  healthcheck(timeoutMs?: number): Promise<boolean>;
  teardown(): Promise<void>;
  cleanup(): void;
}

Source: projects/matrix-3/packages/test-utils/src/daemon-harness.ts:15-35.

Anatomy of a smoke test

import { test } from 'node:test';
import { createDaemonHarness } from '@matrix/test-utils';
import * as http from 'node:http';

test('Host starts and responds to /healthz', async () => {
  const harness = await createDaemonHarness('my-package-smoke', 'my-package');
  try {
    // Install the package under test.
    harness.installWorkspacePackage('@open-matrix/my-package');

    // Start the host.
    await harness.start({ webapps: ['my-package'] });

    // Health.
    const ok = await harness.healthcheck(5000);
    if (!ok) throw new Error('host never became ready');

    // Drive whatever assertion you need: HTTP, NATS, file-system inspection.
    const code = await fetchStatus(`${harness.httpUrl}/api/whoami`);
    assert.equal(code, 200);
  } finally {
    await harness.teardown();
  }
});

function fetchStatus(url: string): Promise<number> {
  return new Promise((resolve) => {
    http.get(url, (res) => { res.resume(); resolve(res.statusCode ?? 0); }).on('error', () => resolve(0));
  });
}

What createDaemonHarness sets up

projects/matrix-3/packages/test-utils/src/daemon-harness.ts:54-272:

1. Creates a temp rootDir under os.tmpdir().
2. Allocates three free ports (broker, ws, http) via getAvailablePorts(3).
3. Creates a <rootDir>/.matrix/ skeleton — packages/node_modules/, projects.json.
4. Returns a harness object with start, teardown, etc.

installWorkspacePackage(name) (:78-124):

• Locates the workspace package by name.
• If dist/ is missing, runs pnpm --filter <name> build to build it.
• Copies dist/ and matrix.json/package.json/schema.sql into the harness's packages/node_modules/<scope>/<name>/.

start(opts) (:147-211):

1. Runs matrixd init --tier local --no-seed against the harness home.
2. Fixes nats-server permissions.
3. Edits daemon.json to pin the harness ports.
4. Optionally copies webapp dist/ directories from sibling packages.
5. Spawns node matrixd.js start --foreground as a child process.
6. Polls /healthz until the host is up or the timeout fires.

teardown() (:213-245):

• Kills the daemon process (SIGTERM, then SIGKILL after 5s).
• Tries matrixd stop as a backup.
• fs.rmSync(rootDir, { recursive: true, force: true }).

When to write a smoke test

• Adding install/uninstall logic to a package — verify it actually installs.
• Adding a new system actor that needs to come up at host start — verify boot.
• Changing port allocation, gateway routing, or webapp discovery — verify HTTP responds.
• Adding a new webapp — verify it's served at the expected route.

When NOT to write a smoke test:

• Testing a single op handler — use a unit test (10ms) instead.
• Testing actor logic in isolation — MockRuntime/InMemoryTransport.
• Testing wire-format translation — stub INatsLikeConnection.

Smoke tests cost real time (5-30 seconds each). One per package is the right cardinality.

Skip conditions

Smoke tests need nats-server on the system. Without it, the test should skip rather than fail:

import { createDaemonHarness, isNatsServerAvailable } from '@matrix/test-utils';

test('Host starts', async (t) => {
  if (!isNatsServerAvailable()) {
    t.skip('nats-server binary missing');
    return;
  }
  // ... harness ...
});

CI environments include nats-server; some dev machines don't.

Common failure modes

Symptom	Likely cause
Daemon never became ready on http://127.0.0.1:N	Look at the harness's <matrixDir>/logs/matrixd.log (the start method dumps its tail in the error). Usually a build artifact missing or a port conflict.
Workspace package dist missing	The package wasn't built before the test ran. The harness will try to build, but if the build itself fails the harness reports the build's stderr/stdout.
matrixd init exited 1	Init failed. Check <rootDir>/.matrix/ for partial state and inspect the init log.
Test passes locally, fails in CI	Port allocation race — multiple harnesses running concurrently. The harness uses getAvailablePorts per instance; each gets fresh ports. If still flaky, run smoke tests serially in CI.

Cleanup discipline

The try { ... } finally { await harness.teardown(); } pattern is mandatory. A test that throws without teardown leaks:

• A node child process holding ports.
• A NATS sibling process.
• A temp directory.

Multiply by a CI matrix and the build host runs out of ports or disk. The harness's cleanup() is a non-async best-effort fallback; teardown() is the proper async path.

See also

• Unit tests — when you don't need a real Host.
• Package proof tests — end-to-end proofs that build on smoke tests.
• Transport testing — createNatsHarness for NATS-only tests.

Source: projects/matrix-3/packages/test-utils/src/daemon-harness.ts:54-272 (createDaemonHarness), :296-307 (createNatsHarness).