Runtime Host

Appearance

Logs ​

The Host writes logs in four places. None are unified under a single "hivecast logs" command yet — see docs-hivecast/content/cli/exit-codes-and-diagnostics.md for the rationale.

1. Per-runtime stdout/stderr files ​

<home>/logs/runtimes/<runtimeId>/stdout.log
<home>/logs/runtimes/<runtimeId>/stderr.log

Created on every startRuntimeProcess call (matrix-host-service.ts:375-378). Append-mode; never truncated by the Host.

bash
# Tail one runtime
tail -f <home>/logs/runtimes/RUNTIME-HOST-LOCAL-ABC123-CHAT/stderr.log

# Tail every runtime's stderr at once
find <home>/logs/runtimes -name 'stderr.log' -print0 \
  | xargs -0 tail -F

When a runtime fails to become ready, the Host appends a tail of these files to the error message via appendRuntimeLogTail (matrix-host-service.ts:1322-1354) — up to 4 KiB of stderr and 2 KiB of stdout. So the supervisor's failure message already contains the runtime's last words.

2. NATS sibling logs (wrapper installs) ​

<home>/logs/nats.stdout.log
<home>/logs/nats.stderr.log

hivecast.mjs:663-665 opens these on startNatsSibling and pipes the detached nats-server process into them. The .deb install routes NATS through journald instead — see below.

3. Host bootstrap log (.deb installs) ​

/var/lib/hivecast/logs/host-bootstrap.log

The systemd ExecStart shell wrapper (build-deb-installer.js:216-270) runs hivecast bootstrap-default-runtimes in the background up to 6 times, 5 seconds apart, and appends each attempt's stdout+stderr to this file. It is the right place to look when default runtimes are not appearing after a fresh install.

4. journald (.deb installs) ​

The Host's own stdout/stderr go to journalctl -u hivecast-host.service, NATS goes to journalctl -u hivecast-nats.service. The postinst installs a journald drop-in at /etc/systemd/journald.conf.d/hivecast.conf (build-deb-installer.js:368-388) with these caps:

[Journal]
Storage=persistent
SystemMaxUse=2G
SystemKeepFree=4G
SystemMaxFileSize=128M
MaxFileSec=1week
RuntimeMaxUse=200M
ForwardToSyslog=no
RateLimitIntervalSec=30s
RateLimitBurst=10000

This was added by the launch-readiness atomic-writes work (WORKSTREAMS/launch-readiness-atomic-writes-and-bootstrap/) after a runaway log emitter filled the disk during a crash loop. The cap composes with the per-unit LogRateLimitIntervalSec=10s / LogRateLimitBurst=200 to bound catastrophe.

What lives where ​

FileSourceLifetimeRotation
<home>/logs/runtimes/<id>/{stdout,stderr}.logruntime child processfrom startRuntimeProcess to runtime stop or Host wipenone built-in
<home>/logs/nats.stdout.log, nats.stderr.logwrapper-managed NATS siblingfor the life of the NATS siblingnone built-in
<home>/logs/host-bootstrap.log.deb ExecStart wrappergrows on each Host startnone built-in
journalctl -u hivecast-host.servicesystemd capture of Host stdout/stderrjournald-managedjournald cap
journalctl -u hivecast-nats.servicesystemd capture of NATS stdout/stderrjournald-managedjournald cap

Reading practical signals ​

bash
# Did the Host start cleanly?
sudo journalctl -u hivecast-host.service -n 100

# Did NATS bind both ports?
sudo journalctl -u hivecast-nats.service -n 100

# Did the bootstrap finish or hit the 6-attempt cap?
sudo tail -n 200 /var/lib/hivecast/logs/host-bootstrap.log

# Why is one runtime in 'failed' state?
hivecast status | jq '.runtimes[] | select(.status=="failed")'
tail -n 200 <home>/logs/runtimes/<runtimeId>/stderr.log

# Is the gateway WebSocket healthy?
# (browser console matters more than HTTP 200; see Worker Bootstrap Gotchas
# in repo AGENTS.md.)

What's NOT logged ​

  • Bus messages. Subscribing to *.> for debugging is a separate exercise (use nats sub against the NATS URL).
  • Runtime-record state changes are not separately logged; the runtime.json record itself is the audit trail. updatedAt ticks on every change.
  • The Host does not write to syslog. The journald drop-in explicitly turns off ForwardToSyslog=no.

See also ​