Appearance
Package failed to start
You ran hivecast up <pkg> and got a non-zero exit, or hivecast runtimes shows the runtime in status: "failed". This page is the triage path.
Check 1 — Read the runtime record
bash
ls <host-home>/runtimes/
# Find the right runtime-id
cat <host-home>/runtimes/<runtime-id>/runtime.json | jq .Look at:
status— should berunning. Iffailed, see step 2. Ifstarting, the spawn hasn't completed yet (give it 10-30s, then check again — if stillstarting, something is wedged).pid— should be set when running. Runps -p <pid>to confirm aliveness.metadata.staleReason— set if the supervisor detected the recorded pid is dead or wrong (seeruntimeRecordStaleReasoninhivecast.mjs:1200-1230).localMounts— should list the mounts the runtime claimed.metadata.commandandmetadata.args— the actual exec.
Check 2 — Read the runtime stderr
bash
tail -100 <host-home>/logs/runtimes/<runtime-id>.stderr.logThe stderr typically reveals:
- module-resolution failures (
Cannot find module ...) — the package or its deps aren't in the package store - port-bind failures (
EADDRINUSE) —--portcollision with another runtime - mount-claim failures (
mount already claimed by ...) — duplicate mount - crash on bootstrap (uncaught exception during the runtime's own init)
Check 3 — Is the package present and shaped correctly?
bash
ls <host-home>/packages/global/node_modules/<package-name>/
cat <host-home>/packages/global/node_modules/<package-name>/matrix.jsonIf matrix.json is missing, the package is malformed (this can happen if you npm installed something that's not a Matrix package). The wrapper's bundled set always has matrix.json.
If package.json is missing, the package store is corrupt. Re-seed:
bash
hivecast seed --home <host-home>The output shows healthy + corrupt records; re-seed only re-mirrors the bundled set, so it won't help for packages you brought in via matrix install.
Check 4 — Is there a duplicate runtime?
hivecast doctor checks for duplicate live runtime claims (hivecast.mjs:1102-1109):
bash
hivecast doctor --home <host-home>
# Look for "duplicate live runtime claims" in the outputIf two runtimes claim the same logical mount or web route, the second one fails to register. The fix is hivecast down <duplicate>.
Check 5 — Stale pid (record says running, process is dead)
If the record's status is running but ps -p <pid> says nothing, the supervisor lost track. hivecast doctor will catch this and hivecast.mjs:859-881 (repairDeadRuntimeRecords) will rewrite the record to failed with metadata.staleReason. Once the record reads failed, restart it:
bash
hivecast down <runtime-id> --home <host-home>
hivecast up <package> --runtime-id <runtime-id> --startup auto --restart always --home <host-home>Check 6 — Did the runtime crash during its bootstrap?
If you see in stderr:
- a stack trace from your package's init code
- timeouts waiting for external services (DB, API)
- assertion failures
…then the package's own init logic is the problem. Fix the code or the config it depends on. Browser-package init issues won't show here — those are in the browser console.
Check 7 — The "host-service short-circuited because already running"
If you ran hivecast up <pkg> and it printed:
json
{ "ok": true, "skipped": true, "runtimeId": "...", "reason": "already-running" }…the runtime was already running with the given id. That's success, not a failure. To restart it, hivecast down <runtime-id> && hivecast up <package> --runtime-id <runtime-id>.
Common error signatures
| Stderr line | Meaning | Fix |
|---|---|---|
Cannot find module '@open-matrix/foo' | Package's deps aren't installed | Re-install the package: matrix install <package> |
EADDRINUSE: address already in use 0.0.0.0:5050 | Webapp's --port is already bound | Pick a different port or --serve (auto-allocate) |
RegistryError: mount 'chat' already claimed by RUNTIME-... | Duplicate mount claim | hivecast down <conflicting> |
Timed out waiting for runtime to register | Runtime didn't call system.runtimes.register within the timeout | The runtime's init is hanging or crashed silently — check stderr earlier |
Failed to parse runtime.json | Corrupt record on disk | Run hivecast seed, then hivecast start will self-heal |
When hivecast up succeeds but you don't see the webapp
Confirm the gateway picked it up:
bash
hivecast invoke system.gateway.http routes.list '{}' --home <host-home>Should list every webapp's routePrefix. If your runtime is in hivecast runtimes but missing here, the gateway's route registration didn't fire. Restart the runtime (down + up) — usually a transient registration race.
When the runtime restarts repeatedly
restart: always policy + a crash on every start = restart loop. systemd protects against this for the host-service unit (StartLimitBurst=5); host-control protects similarly for runtimes (per package-launch readiness work). After a few failures, the runtime will be left in failed and not auto-restarted.
To break the loop manually:
bash
hivecast down <runtime-id> --home <host-home>
# fix the underlying issue
hivecast up <package> --runtime-id <runtime-id> --restart on-failure --home <host-home>--restart on-failure is gentler than always for development.
See also
- Local Device → Runtime records —
runtime.jsonschema reference. - Local Device → Logs — where stderr files live.
- CLI → hivecast up / down — the up/down reference.
- NATS connection failed — if the runtime crashes specifically with NATS errors.
Source:
projects/matrix-3/packages/hivecast/bin/hivecast.mjslines 1264-1273 (isRuntimeRecordRunning);projects/matrix-3/packages/host-service/src/types.tslines 79-115 (record schema).