Skip to content

Reference: Runbooks (admin)

A quick-lookup index for admin repair procedures. For non-admin runbooks, see Reference: Operational runbooks. For incident-class procedures, see Operations: Resolve incident.

Index

Account / principal

Devices

Spaces

Runtimes

Secrets


Account / principal

Suspend a principal

bash
# Until auth.admin.principal.suspend ships:
hivecast stop --home <host-home>
sudo jq '.principals |= map(if .id == "p_xxx" then .status = "suspended" else . end)' \
  <host-home>/state/auth-state.json > /tmp/auth-state.json
sudo mv /tmp/auth-state.json <host-home>/state/auth-state.json
hivecast start --home <host-home>

# Then revoke their Devices
matrix invoke system.auth auth.hostLink.list '{"principalId":"p_xxx"}'
# For each id, call auth.hostLink.revoke

Find every Device for a principal

bash
matrix invoke system.devices devices.list \
  '{"principalId":"p_xxx","includeOffline":true,"includeRevoked":true}'

Find every Space for a principal

bash
matrix invoke system.auth auth.namespace.list '{"principalId":"p_xxx"}'
# Or read state file:
sudo jq '.spaces[] | select(.ownerPrincipalId == "p_xxx")' \
  <host-home>/state/auth-state.json

Force re-login of a specific principal

bash
# Add every active session for the principal to revokedSessions[].
# Today there is no per-principal session-list op; the simplest path:
hivecast stop
sudo jq '.principals |= map(if .id == "p_xxx" then .updatedAt = now|todate else . end)' \
  <host-home>/state/auth-state.json > /tmp/auth-state.json
sudo mv /tmp/auth-state.json <host-home>/state/auth-state.json
# Then rotate the session HMAC secret to fully invalidate sessions.
# (Affects every user, not just one — this is a big-hammer fix.)

A scoped op (auth.admin.principal.invalidateSessions) is target.

Devices

bash
matrix invoke system.auth auth.hostLink.revoke '{"hostLinkId":"<id>"}'
matrix invoke system.devices devices.list '{"includeRevoked":true}'  # verify

Revoke every Device for a principal

bash
matrix invoke system.auth auth.hostLink.list '{"principalId":"p_xxx"}' \
  | jq -r '.hostLinks[].id' \
  | while read id; do
      matrix invoke system.auth auth.hostLink.revoke "{\"hostLinkId\":\"$id\"}"
    done

Identify a Device by leaked NATS user public key

bash
# State-file scan
sudo jq --arg key "<leaked-key>" \
  '.hostLinks[] | select(.natsUserPublicKeys[]? == $key)' \
  <host-home>/state/auth-state.json

Spaces

Suspend a Space

bash
hivecast stop
sudo jq '.spaces |= map(if .id == "spc_xxx" then .status = "suspended" else . end)' \
  <host-home>/state/auth-state.json > /tmp/auth-state.json
sudo mv /tmp/auth-state.json <host-home>/state/auth-state.json
hivecast start

List every Space (cross-principal)

bash
sudo jq '.spaces' <host-home>/state/auth-state.json

No auth.space.list op today.

Find a Space by spacePath

bash
matrix invoke system.auth auth.namespace.resolve '{"routeKey":"<spacePath>"}'
# Returns spaceId, authorityRoot, claim, space.

Runtimes

Repair a corrupt runtime record

bash
hivecast seed --home <host-home>
# Look at output's runtimeRecords.corrupt[].
# Subsequent hivecast start self-heals empty records.
hivecast start --home <host-home>

Restart a wedged runtime without affecting others

bash
matrix invoke system.runtimes runtimes.restart '{"mount":"<mount>"}'
# Or
matrix invoke host.control runtimes.restart '{"runtimeId":"<id>"}'

Secrets

Rotate session HMAC secret

bash
NEW=$(node -e 'console.log(require("crypto").randomBytes(32).toString("hex"))')
hivecast stop
sudo jq ".auth.sessionSecret = \"$NEW\"" <host-home>/host.json > /tmp/h.json
sudo mv /tmp/h.json <host-home>/host.json
hivecast start
# All users must re-login.

Rotate Google OIDC client secret

bash
# 1. Generate new secret in Google Cloud Console.
# 2. Update host.json:
hivecast stop
sudo jq ".auth.providers.google.clientSecret = \"$NEW_SECRET\"" \
  <host-home>/host.json > /tmp/h.json
sudo mv /tmp/h.json <host-home>/host.json
hivecast start
# 3. Old secret remains in Google's console for grace period; remove after.

See also

Source: Operational experience captured across WORKSTREAMS/loose-ends/items/P1.22* and WORKSTREAMS/product-launch/.