NATS connection failed

NATS is the bus. Any failure to talk to NATS surfaces as broken actor invocations, browser tabs that never finish loading, or runtimes that crash during bootstrap. This page covers the most common scenarios.

Symptom-driven triage

Symptom	Likely cause
Webapp loads, spinner never resolves, browser console shows WebSocket connection to 'ws://.../nats-ws' failed	Gateway is up but nats-ws proxy is misconfigured or NATS WebSocket port isn't bound
Runtime fails on start with connection refused, ECONNREFUSED nats://127.0.0.1:4222	NATS sibling isn't running OR is listening on a different port than host.json says
hivecast invoke <mount> <op> errors with Matrix Host Service is not running	Either the Host or NATS is down — run Host not running first
nats-server: Listen on 127.0.0.1:4222: address already in use	Another process owns the port
Sudden drop in connectivity after hivecast stop ... && hivecast start	Race against half-shutdown NATS still binding the port — fixed in current code, but symptom worth watching

Verify NATS is up

# Pidfile exists and points to a live process?
cat <host-home>/nats/host-default/nats-server.pid
ps -p $(cat <host-home>/nats/host-default/nats-server.pid)

# Port accepting connections?
nc -zv 127.0.0.1 4222

# WebSocket port accepting connections?
nc -zv 127.0.0.1 4223

# Look at the conf file the wrapper generated
cat <host-home>/nats/host-default/nats-server.conf

Expected nats-server.conf shape (from ensureNatsConfigFile in hivecast.mjs:606-624):

host: "127.0.0.1"
port: 4222
jetstream {
  store_dir: "<MATRIX_HOME>/nats/host-default"
}
websocket {
  host: "127.0.0.1"
  port: 4223
  no_tls: true
}

If any of those is wrong, the conf file was hand-edited or the home was set up under a different version. The wrapper regenerates the conf on every start (per the comment block in hivecast.mjs:606-624), so simply hivecast stop && hivecast start corrects it.

Port already in use

nats-server: Listen on 127.0.0.1:4222: address already in use. Two scenarios:

Another NATS is running

lsof -i :4222
# nats-serv 12345 user 5u IPv4 ... TCP 127.0.0.1:4222 (LISTEN)

If pid 12345 is one you started somewhere else, stop it. If it's the wrapper-spawned one, hivecast stop should already have removed it — but if its pidfile points at an unrelated dead pid, it might have spawned a new one. Force-kill, then:

rm <host-home>/nats/host-default/nats-server.pid
hivecast start --home <host-home>

A non-NATS process is on the port

Pick a different port:

hivecast install --home <host-home> --port 4242 --ws-port 4243
# (or for an existing install, edit host.json transport.nats.port and wsPort, then restart)

-p and --port set the NATS port; --ws-port sets the WebSocket port. The gateway will pick up the new ports from host.json's transport.nats.url and wsUrl automatically.

Note (latent bug, fixed): if you only updated host.json's nats.url without updating nats.port, older code wrote the wrong port back to nats-server.conf on the next start, which then bound the wrong port. Current code keeps port/wsPort and the URLs in sync (writeHostProductConfig in hivecast.mjs:281-346 sets both explicitly). Surfaced by P1.30 stop/start cycles.

Browser → NATS WebSocket

A common subtle failure: the gateway runs and serves HTTP, but the browser's WebSocket can't connect. Check:

1. The browser's network panel for the /nats-ws request — should be a 101 (Switching Protocols).
2. The gateway runtime's stderr log:

   tail -100 <host-home>/logs/runtimes/RUNTIME-HOST-LOCAL-*-GATEWAY.stderr.log

3. The NATS WebSocket port (default 4223) is bound and reachable from the gateway runtime.

Per CLAUDE.md Rule 7:

Browser connects via same-origin WebSocket. Always /nats-ws on the SAME origin. Never absolute WebSocket URLs from bootstrap.

If the bootstrap returned an absolute wss://... URL, browser CORS / mixed-content rules will reject it. Bootstrap should ALWAYS return a relative /nats-ws so the browser uses the same origin and the same auth context as the page itself.

Mismatched authority root

If a runtime starts but immediately disconnects with auth errors, check the wire root. Each runtime carries runtimeWireRoot in its runtime.json:

cat <host-home>/runtimes/<runtime-id>/runtime.json | jq '.runtimeWireRoot'

That should match host.json transport.root (or transport.authorityRoot) for purely-local runtimes. For paired Devices, the hivecast-link.json authorityRoot may diverge, and runtime records created during pairing get the linked root.

hivecast doctor runs a "root collision guard" check (hivecast.mjs:948-1029) that flags this. Run:

hivecast doctor --home <host-home>

Look for the root collision guard line. If it says mismatches: ..., you have stale runtime records pointing at a different root than the current Host link. The remedy:

relink the Device, restart the Host service, then stop or recreate runtime records whose runtimeWireRoot does not match the linked authority root

Sibling vs child NATS

Per the comment block in hivecast.mjs:561-582:

NATS runs as an INDEPENDENT sibling process, not a child of host-service. host-service uses mode:'external' to connect to it instead of supervising it.

If hivecast stop only kills host-service but leaves NATS running, that's the SIBLING design — NATS lifetime is independent. The next hivecast start will see NATS already up and just connect.

If you want NATS to fully restart, hivecast stop (which now correctly tears down NATS) followed by hivecast start is the right path. Don't try to kill NATS manually unless the pidfile is broken.

Verifying the bus is actually working

# From the same machine
hivecast invoke system.runtimes runtimes.list '{}' --home <host-home>
hivecast invoke system.devices devices.list '{"includeOffline":true}' --home <host-home>

If both return real data, the bus path is healthy. If invoke errors with ECONNREFUSED or EPIPE, NATS isn't listening on the port the wrapper expects.

See also

• Host not running — start here if hivecast status doesn't return.
• Local Device → Logs — nats.stderr.log and per-runtime stderr.
• Reference → Ports — the default port set and how to pin them.
• Reference → Config files — host.json transport.nats schema.

Source: projects/matrix-3/packages/hivecast/bin/hivecast.mjs lines 561-625 (the NATS sibling design comment + config generation), 654-685 (startNatsSibling), 948-1029 (rootCollisionDiagnostics).