HiveCast Platform

Appearance

Operational runbooks ​

Common repair procedures for a HiveCast platform Host or a paired user Device.

Diagnostic first steps ​

bash
# 1. Is the Host running?
hivecast status --home /var/lib/hivecast/host-home
# 2. Are runtimes healthy?
matrix invoke host.control status '{}' --home /var/lib/hivecast/host-home
# 3. Is the gateway responding?
curl -fsS http://127.0.0.1:3100/healthz | jq .
# 4. Are paired Devices online?
matrix invoke system.devices devices.list '{"includeOffline":true}'

If any of these fail, follow the matching runbook below.

Host won't start ​

Symptom: hivecast start exits with an error.

bash
# Inspect supervisor log
tail -50 <host-home>/logs/host-service.log

Common causes:

  • NATS already running on port. A previous NATS sibling process did not stop. pkill nats-server, then hivecast start.
  • Stale PID file. Delete <host-home>/host.pid and <host-home>/nats.pid if the processes are not actually running.
  • Port collision (gateway port). Pass --http-port 0 to let the OS allocate, or kill the holder.

After fix, hivecast start again.

Runtime won't start ​

Symptom: host.control runtimes.list shows the runtime in failed or it never appears.

bash
tail -50 <host-home>/logs/runtimes/<runtime-id>.log

Common causes:

  • Empty/corrupt runtime record JSON. hivecast seed self-heals.
  • Missing package. Verify <host-home>/.matrix/packages/<package-name>/ exists; if not, reinstall the wrapper.
  • Bad env config. Check host.json transport.root matches MATRIX_AUTHORITY_ROOT if set.
  • Pinned port collision. Edit the runtime record's port to 0 or kill the holder.

system.devices shows zero rows ​

Symptom: devices.list returns count: 0 even though Host Links exist in auth-state.json.

This is a registration-propagation lag. The Host's host.control heartbeats register the Device with system.devices. If the Host just restarted, allow ~30 seconds.

If still empty:

bash
# Verify the Host Link exists
matrix invoke system.auth auth.hostLink.list '{"principalId":"p_xxx"}'

# Verify the Host is heartbeating
tail -20 <host-home>/logs/runtimes/host-control.log

# Force a register
matrix invoke host.control register '{}'

If auth.hostLink.list returns the link but system.devices.devices.list still shows zero, the registration step on system.devices has a bug — file an issue.

Device shows offline but is running ​

Symptom: Device row has status: 'offline' and recent lastSeenAt.

Verify on the Device itself:

bash
# On the Device
matrix invoke host.control status '{}' --home <host-home>
# Look for last heartbeat outcome

If the Device is heartbeating but the platform records old lastSeenAt, check:

  • Heartbeat token validity (re-pair if stale).
  • Network reachability from Device to platform.
  • Platform's _auth/host-link/heartbeat endpoint logs.

Cookies fail to authenticate ​

Symptom: Browser is logged in but every API request returns 401.

  • Check mx_session cookie is present (browser dev tools, Application > Cookies).
  • Check auth.sessionSecret was not rotated (rotation invalidates all sessions; users must re-login).
  • Check the platform's iss matches the cookie. auth.session.validate checks claims.iss === issuer where issuer is matrix-host:<authorityRoot>.

Lost the session HMAC secret ​

Recovery: Generate a new one, deploy, accept that all users must re-login.

bash
# On the platform Host
node -e 'console.log(require("crypto").randomBytes(32).toString("hex"))'
# Copy to host.json auth.sessionSecret
hivecast stop && hivecast start

There is no secret-derivation path; if the secret is lost, every existing session is unrecoverable. Plan for periodic rotation.

Lost a paired Device's credentials ​

Symptom: A Device's <host-home>/credentials/hivecast-link.json was deleted or corrupted.

The Device cannot heartbeat. The platform still has the Host Link record.

Recovery: revoke the lost Host Link (so its NATS keys can't be reused if recovered) and re-pair:

bash
# On platform
matrix invoke system.auth auth.hostLink.revoke '{"hostLinkId":"<id>"}'

# On Device
hivecast login --device --cloud https://hivecast.ai

NATS hub failed to reload after revoke ​

Symptom: auth.hostLink.revoke returned ok: true, but the revoked Device's NATS user JWT still authenticates.

bash
# Force NATS to reload the account JWT
nats account reload  # via NATS CLI on the platform NATS

# Verify revocation took effect
matrix invoke system.auth auth.hostLink.get '{"hostLinkId":"<id>"}'
# Status should be revoked.

Audit a principal's full state ​

bash
# All Host Links
matrix invoke system.auth auth.hostLink.list '{"principalId":"p_xxx"}'

# All namespace claims
matrix invoke system.auth auth.namespace.list '{"principalId":"p_xxx"}'

# Default Space
matrix invoke system.auth auth.space.default '{"principalId":"p_xxx"}'

# Authority root
matrix invoke system.auth auth.principal.authorityRoot '{"principalId":"p_xxx"}'

# Their Devices
matrix invoke system.devices devices.list '{"principalId":"p_xxx","includeOffline":true,"includeRevoked":true}'

Cross-reference with the dashboard or with auth-state.json if needed.

Suspend a principal ​

Until auth.principal.suspend ships, hand-edit:

bash
# On the platform Host. CARE: this is direct state mutation.
hivecast stop
sudo jq '.principals |= map(if .id == "p_xxx" then .status = "suspended" else . end)' \
  <host-home>/state/auth-state.json > /tmp/auth-state.json
sudo mv /tmp/auth-state.json <host-home>/state/auth-state.json
hivecast start

# Revoke their Host Links so paired Devices stop authenticating
matrix invoke system.auth auth.hostLink.list '{"principalId":"p_xxx"}'
# For each id, call revoke

See also ​

Source: Operational experience captured in WORKSTREAMS/loose-ends/items/P1.22* progress logs and WORKSTREAMS/product-launch/HIVECAST-DEVICE-ENROLLMENT-SPEC.md.