On 2026-07-30 a kernel upgrade left the machine in systemd emergency mode
on three consecutive boots. The upgrade replaced /usr/lib/modules/<old>
with <new>, but the new kernel never reached the ESP, so the firmware kept
booting the OLD vmlinuz whose module tree had just been deleted. Nothing
modular could load after pivot-root — including vfat, which meant /boot
itself could not be mounted, local-fs.target failed, and systemd dropped to
an emergency shell. pacman reported the transaction as successful.
Root cause: pacman-git with ParallelDownloads > 1 segfaults every forked
alpm hook child, silently skipping 90-mkinitcpio-install and depmod.
boot-repair is self-contained (no repo libs — it must run in the emergency
shell) and fully offline (missing modules are recovered from
/var/cache/pacman/pkg). It repairs by default; --dry-run inspects.
It detects and fixes: read-only root, noauto/nofail fstab drift, orphaned
kernel files shadowing an unmounted ESP, unloadable vfat, unmounted ESP,
missing modules.dep, ESP kernel != installed tree, stale initramfs, and
ParallelDownloads > 1.
Two pacman hooks make the failure impossible rather than merely
recoverable: 05-boot-mounted-guard aborts a kernel transaction when the
ESP is unmounted (AbortOnFail), and 99-boot-autorepair fixes ESP drift
after the transaction, before the machine is ever rebooted.
Safety: a module tree counts as usable only if it has BOTH vmlinuz and
kernel/, so the metadata-only wreckage this bug leaves — which sorts
newer — is never chosen as the target. Files under /boot are deleted only
while it is not a mountpoint AND contains no EFI/ or loader/; deleting
after mounting would destroy the real kernel.
Verified:
- 31/31 fixture tests (--root against throwaway trees, no privileges)
- 11/11 reversible live tests on the real machine: unmounted the real ESP,
planted shadow files, confirmed they were deleted and the ESP then
mounted with EFI/, loader/ and kernel 7.1.5-arch1-2 intact
- Gate proven against a real transaction with the ESP unmounted:
"failed to commit transaction", no packages upgraded, nothing written
- shellcheck + shfmt clean, pre-commit passing
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
