Matrix SDK

Appearance

Browser tests ​

A browser test drives a real browser against a running --serve Host. It loads the package's webapp, performs user actions, and asserts on observable UI + bus state. Playwright is the tool of choice across the tree.

This page covers what to test, how it fits with smoke and proof tests, and the conventions for end-to-end UI verification. Memory.md calls this out as a hard rule:

NEVER declare done without Playwright MCP browser test. Login as user, check ALL apps, read console errors, verify inference. HTTP 200 is NOT proof — check for NATS permission errors in console.

When you need a browser test ​

  • The change touches a UI component's behaviour (interaction, state, rendering).
  • The change touches the bootstrap path (transport, auth, root resolution).
  • The change touches gateway routing for served webapps.
  • The change touches anything that crosses the boundary "I'm a browser tab and I'm talking to a Matrix actor."

For pure backend changes, smoke tests + unit tests are usually enough. The browser test is for the cases where the headless test would pass but the real product is broken.

The recipe ​

The end-to-end flow against a developer Host:

bash
# 1. Build the package whose UI you're testing.
pnpm --filter @open-matrix/my-package build

# 2. Start a fresh Host with the package served.
node projects/matrix-3/packages/hivecast/bin/hivecast.mjs install --home /tmp/matrix-home
node projects/matrix-3/packages/hivecast/bin/hivecast.mjs start --home /tmp/matrix-home
matrix run @open-matrix/my-package --serve --port 5001 \
  --runtime-id MY-PKG --env hivecast --startup auto --restart always \
  --home /tmp/matrix-home

# 3. Open the served URL in Playwright.
# http://127.0.0.1:5001/apps/my-package/

In a CI environment, you'd wrap steps 1-2 inside a createDaemonHarness test that programmatically spins up the Host before Playwright connects.

What to verify ​

For every browser test, at minimum:

  1. Page loads. No 4xx/5xx, no JS module-load errors.
  2. Console is clean. No uncaught exceptions, no NATS permission errors.
  3. Bootstrap completes. The <matrix-dsl-host> element gets context; root component reaches data-matrix-ready.
  4. UI responds to interaction. Click a button, fill a form, assert the resulting state.
  5. Bus traffic happens. Observe $events or end-state from the actor side via a parallel test connection.

The HTTP-200 trap (memory.md): don't conclude "the page loaded ⇒ the package works." A page can load without errors and still fail to bootstrap because of a NATS auth issue. Read the console.

Console-error discipline ​

ts
// Playwright pattern
const errors: string[] = [];
page.on('console', (msg) => {
  if (msg.type() === 'error') errors.push(msg.text());
});
page.on('pageerror', (err) => errors.push(err.message));

await page.goto('http://127.0.0.1:5001/apps/my-package/');
await page.waitForSelector('[data-matrix-ready]');

// Assert absolutely no errors.
const noise = errors.filter(e => !isExpectedNoise(e));
expect(noise).toEqual([]);

isExpectedNoise is per-test — some packages emit benign warnings during dev that aren't worth a test failure. Whitelist them explicitly with comments.

Where browser tests live ​

There are no per-package tests/browser/ directories yet — browser tests for product paths live alongside proof scripts under projects/matrix-3/scripts/ or in the projects/matrix-3/tests/integration/ tree, depending on whether they need a full Host or just a static --serve.

Status: target state, not yet implemented. A canonical tests/browser/ directory and Playwright config baked into every webapp package. Today, the structure is more ad hoc; follow the closest existing browser test as a template.

Pairing with the daemon harness ​

For programmatic CI runs, combine createDaemonHarness with Playwright:

ts
import { createDaemonHarness } from '@matrix/test-utils';
import { chromium } from 'playwright';

test('chat loads and shows the conversation panel', async () => {
  const harness = await createDaemonHarness('chat-browser', 'chat-test');
  try {
    harness.installWorkspacePackage('@open-matrix/chat');
    harness.installWorkspacePackage('@open-matrix/matrix-web');
    await harness.start({ webapps: ['chat', 'matrix-web'] });

    const browser = await chromium.launch();
    const page = await browser.newPage();
    const errors: string[] = [];
    page.on('console', (m) => m.type() === 'error' && errors.push(m.text()));
    page.on('pageerror', (e) => errors.push(e.message));

    await page.goto(`${harness.httpUrl}/apps/chat/`);
    await page.waitForSelector('[data-matrix-ready]', { timeout: 10000 });
    expect(errors.filter(/* whitelist */)).toEqual([]);

    await browser.close();
  } finally {
    await harness.teardown();
  }
});

Two-runtime topology and browser tests ​

For testing both the platform shell and the Edge shell side by side, the project root CLAUDE.md § "Two-Runtime Dev Topology" describes pinning ports:

bash
matrix up @open-matrix/matrix-web --serve --port 5001 --runtime-id WEB ...
matrix up @open-matrix/matrix-edge --serve --port 5002 --runtime-id EDGE ...

Then point Playwright at both:

http://127.0.0.1:5001/apps/web/   ← platform shell
http://127.0.0.1:5002/apps/edge/  ← Device shell

Both heartbeat to the same local system.devices. Cross-runtime ops route through local NATS. This is the canonical local proof for any feature that touches both shells.

Anti-patterns ​

  • HTTP 200 = green. Read the console. Always.
  • Snapshot tests on rendered DOM. Brittle. Test behaviour ("after click, state.count === 1"), not the exact serialized HTML.
  • Skipping cleanup. Browser tests leak Chromium processes if browser.close() and harness.teardown() aren't in a finally.
  • Tests that depend on production HiveCast. Browser tests run against a local Host. If you need pairing, run the device-link proof, not a browser test.

See also ​

Source: projects/matrix-3/packages/test-utils/src/daemon-harness.ts:54-272 (harness), root CLAUDE.md § "Two-Runtime Dev Topology" (recipe), memory.md "END TO END TESTING".