beefcake/impermanence: seed machine-id in initrd (fixes the dbus crash-loop)

lytedev commented

2026-07-08 16:33:11 -05:00

Owner

The fix for flip-attempt-#1's dbus crash-loop. ROOT CAUSE (proven from the failed boot's own persisted journal): on the @blank-wiped root, /etc/machine-id is absent when PID1 starts → systemd generates a random transient id + ConditionFirstBoot=true → dbus-broker's launcher fails its privilege-drop (launcher_run_child: Permission denied) → crash-loops every 90s → NetworkManager never starts → no network → boot hangs. The persistence layer's activation-time machine-id symlink lands ~50s too late.

FIX: an initrd service (persist-machine-id) copies the persisted machine-id onto the fresh root BEFORE switch-root (after rollback + the stage-1 /persist mount), so PID1 sees it and boots normally. Self-seeding so a missing machine-id can never re-trigger the loop.

PROVEN in the VM (rollback-config now runs dbus+NetworkManager+avahi on the wiped ZFS-rollback root with /persist mounted stage-1 exactly as beefcake does): machine-id stable across the wipe (==/persist), is-system-running=running (converged, not hung), dbus+NM active, no crash-loop — all PASS.

Flag stays OFF (held by #725). Retry #2 after this + #725 land = re-flip + deploy --boot + reboot. Datasets (@blank, /persist) still staged on beefcake.

🤖 Generated with Claude Code

https://claude.ai/code/session_01G7HHrn1sxz4ozvREyHmi4i

The fix for flip-attempt-#1's dbus crash-loop. ROOT CAUSE (proven from the failed boot's own persisted journal): on the @blank-wiped root, /etc/machine-id is absent when PID1 starts → systemd generates a random transient id + ConditionFirstBoot=true → dbus-broker's launcher fails its privilege-drop (launcher_run_child: Permission denied) → crash-loops every 90s → NetworkManager never starts → no network → boot hangs. The persistence layer's activation-time machine-id symlink lands ~50s too late. FIX: an initrd service (persist-machine-id) copies the persisted machine-id onto the fresh root BEFORE switch-root (after rollback + the stage-1 /persist mount), so PID1 sees it and boots normally. Self-seeding so a missing machine-id can never re-trigger the loop. PROVEN in the VM (rollback-config now runs dbus+NetworkManager+avahi on the wiped ZFS-rollback root with /persist mounted stage-1 exactly as beefcake does): machine-id stable across the wipe (==/persist), is-system-running=running (converged, not hung), dbus+NM active, no crash-loop — all PASS. Flag stays OFF (held by #725). Retry #2 after this + #725 land = re-flip + deploy --boot + reboot. Datasets (@blank, /persist) still staged on beefcake. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01G7HHrn1sxz4ozvREyHmi4i

lytedev added 1 commit

2026-07-08 16:33:11 -05:00

fix(beefcake/impermanence): seed machine-id in initrd — fixes the dbus crash-loop

/ check-format (push) Successful in 8s

Details

/ build (push) Has been skipped

Details

72be6be653

Root cause of flip-attempt-#1 (2026-07-08), proven from the failed boot's
own persisted journal: on the @blank-wiped root, /etc/machine-id is absent
when PID1 starts, so systemd generates a random transient id and declares
ConditionFirstBoot=true; in that first-boot state dbus-broker's launcher
fails its privilege-drop (launcher_run_child: Permission denied) and
crash-loops every 90s, blocking NetworkManager → no network → boot never
completes. The persistence layer restored machine-id via an activation-time
symlink (~50s too late).

Fix: an initrd systemd service (persist-machine-id) that copies the
persisted machine-id onto the fresh root BEFORE switch-root — after the
rollback + /persist (stage-1, neededForBoot) mount — so PID1 sees it and
boots normally. Self-seeding (generates + saves one if /persist lacks it)
so a missing machine-id can never re-trigger the loop; also makes a
from-scratch rebuild self-heal. machine-id dropped from environment.
persistence.files (the initrd owns it now; the activation symlink would
fight it).

PROVEN in the VM: prototypes/beefcake-impermanence/rollback-config.nix now
runs beefcake's failing surface (dbus + NetworkManager + avahi) on the
wiped ZFS-rollback root with /persist mounted stage-1 the same way beefcake
does (legacy + neededForBoot). rollback-demo asserts: machine-id stable
across the wipe (== /persist), is-system-running=running (converged, not
hung), dbus+NM active, no crash-loop. All PASS. Faithful-repro fixes along
the way: /persist must be a stage-1 (legacy+neededForBoot) mount for
sysroot-persist.mount to exist; the journal 'machine ID from random' grep
was ambiguous (initrd's own transient id) — replaced by the unambiguous
root==persist comparison.

Flag stays OFF (held by #725). Retry #2 = re-flip + deploy --boot + reboot
once this + #725 land.

lytedev merged commit 72be6be653 into main

2026-07-08 16:35:33 -05:00

lytedev deleted branch fix-impermanence-machineid

2026-07-08 16:35:33 -05:00

lytedev referenced this pull request from a commit

2026-07-08 17:30:38 -05:00

fix(beefcake/impermanence): boot the ephemeral root fully — sops initrd identity + /var parent perms

lytedev referenced this pull request

2026-07-08 17:31:03 -05:00

fix(beefcake/impermanence): boot the ephemeral root fully — sops initrd identity + /var parent perms #727

lytedev referenced this pull request from a commit

2026-07-08 19:37:36 -05:00

fix(beefcake/impermanence): boot the ephemeral root fully — sops initrd identity + /var parent perms

Rows
Columns

beefcake/impermanence: seed machine-id in initrd (fixes the dbus crash-loop) #726