Skip to content

Heartbeats

A Device's presence in system.devices is driven by heartbeats. The Device's own host.control actor posts them at startup and on a fixed interval, both to the local system.devices actor (over the bus) and, if the Device is cloud-linked, to a cloud HTTP endpoint.

Two heartbeat targets

text
host.control

   ├──► local bus: system.devices.devices.register
   │                + system.devices.devices.runtimes.register

   └──► cloud HTTPS: POST <cloudUrl>/_auth/host-link/heartbeat
                     Authorization: Bearer <hostLinkToken>

A linked Device posts to BOTH targets every interval. A local-owner Host (not linked) posts only to the local bus.

Frequency and timeouts

Constants live in HostControlActor.ts:

ConstantValueUse
DEVICE_HEARTBEAT_INTERVAL_MS(interval, fixed in code)timer firing rate
DEVICE_HEARTBEAT_TIMEOUT_MSbus call timeoutper-op RequestReply.execute() timeout
DEVICE_HEARTBEAT_TTL_MSpassed in heartbeatTtlMshow long the receiver should treat this Device as online

On the receiver side:

ConstantValueUse
DEFAULT_HEARTBEAT_TTL_MS45_000fallback if heartbeat omits its own TTL
PRUNE_INTERVAL_MS15_000how often _pruneStaleDevices() runs

Local heartbeat path

HostControlActor._publishDeviceHeartbeat() runs at startup and on interval. It:

  1. Reads <host-home>/credentials/hivecast-link.json via readHiveCastHostLink(). If the link is missing or incomplete (no hostId, principalId, or authorityRoot), it does nothing — local-only Hosts skip heartbeating.
  2. Reads runtime records via _listRuntimeRecordsForHeartbeat(link). Only runtimes whose runtimeWireRoot matches the Host Link's authorityRoot are advertised.
  3. (If cloud-linked) calls postCloudDeviceHeartbeat().
  4. Calls _registerLocalDeviceHeartbeat() which fires devices.register and one devices.runtimes.register per runtime.

The Device payload (from deviceHeartbeatPayload() at lines 393-413) is:

ts
{
  deviceId: link.hostId,
  hostLinkId?: link.hostLinkId,
  principalId: link.principalId,
  hostName?: link.hostName,
  deviceSlug?: link.deviceSlug,
  spaceId?: link.spaceId,
  linkStatus: 'active',
  authorityRoot: link.authorityRoot,
  publicNamespace?: link.publicNamespace,
  routeKey?: link.routeKey,
  heartbeatTtlMs: DEVICE_HEARTBEAT_TTL_MS,
  metadata: {
    source: 'host-control',
    reason,
    cloudUrl?: link.cloudUrl,
    deviceRegistryRoot?: link.deviceRegistryRoot,
  },
}

reason is 'startup' on the first heartbeat and 'interval' thereafter.

Cloud heartbeat path

postCloudDeviceHeartbeat() is the cloud-side post. It:

ts
// projects/matrix-3/packages/host-control/src/HostControlActor.ts:474-492
const endpoint = new URL('/_auth/host-link/heartbeat', link.cloudUrl);
const response = await fetch(endpoint, {
  method: 'POST',
  headers: {
    authorization: `Bearer ${link.hostLinkToken}`,
    'content-type': 'application/json',
  },
  body: JSON.stringify({
    hostLinkId: link.hostLinkId,
    hostId: link.hostId,
    principalId: link.principalId,
    device: deviceHeartbeatPayload(link, reason),
    runtimes: runtimes
      .filter((runtime) => readString(runtime.runtimeId).length > 0)
      .map((runtime) => runtimeHeartbeatPayload(link, runtime)),
  }),
});

The cloud gateway calls auth.hostLink.verifyHeartbeat({ token, hostLinkId, hostId, principalId }) to validate the bearer. If the token does not match the SHA-256 hash on the active Host Link, or the link is revoked, the request is rejected.

After successful verification, the gateway updates system.devices with the device + runtime payloads, and updates lastRefreshedAt / updatedAt on the Host Link via HostLinkStore.refresh() / verifyHeartbeatToken().

What identity comes from where

Per CLAUDE.md Rule 5 ("Identity from transport metadata, NEVER from payload"):

Identity fieldSource on cloud-side heartbeat
principalIdderived from the bearer token's verified Host Link record, NOT trusted from request body
authorityRootderived from the bearer token's verified Host Link record
spaceIdderived from the bearer token's verified Host Link record
hostId / hostLinkIdmatched against the body for shape, but the verified Host Link is canonical

Body fields that disagree with the verified link are rejected. This is the property that scenarios A-E in WORKSTREAMS/loose-ends/items/P1.17-device-heartbeat-auth-proof.md tests.

Failure modes

SymptomCauseFix
device.heartbeat.failed { reason: 'register-failed' } eventsystem.devices returned { ok: false } from devices.register. Most common cause: the previous record's identity scope disagrees with the new payload (see Ownership).Inspect the existing Device record. If it's a stale leftover, remove it via the appropriate mutation op or re-pair the Device.
device.heartbeat.failed { reason: '<error message>' } eventEither readHiveCastHostLink() failed (corrupt or missing hivecast-link.json) or the cloud heartbeat HTTP request threwCheck <host-home>/credentials/hivecast-link.json is well-formed. Check the cloud is reachable.
Cloud device heartbeat failed: HTTP 401The bearer token is wrong. Either rotate happened cloud-side without the Device picking up the new value, or the link was revoked.hivecast link-status to confirm; re-pair if revoked.
Cloud device heartbeat failed: HTTP 403The link exists but is revoked, or the body identity does not match the bearer-verified identity.Per CLAUDE.md Rule 5 the server uses the bearer-derived identity. If the body says a different principalId, the request is rejected. Investigate the heartbeat caller.

What heartbeat does NOT do

  • It does NOT push individual op invocations. Heartbeats only update presence and runtime metadata.
  • It does NOT mint NATS credentials. Credential rotation is a separate op (auth.hostLink.credentials.refresh).
  • It does NOT re-create the Host Link. If the cloud-side row is missing (e.g. revoked + cleaned up), heartbeats fail and require re-pairing.

See also

Source: projects/matrix-3/packages/host-control/src/HostControlActor.ts (_publishDeviceHeartbeat(), deviceHeartbeatPayload(), postCloudDeviceHeartbeat()); projects/matrix-3/packages/system/src/SystemDevicesActor.ts (onDevicesRegister() accepts the heartbeat).