pacman intermittently freezes at its FIRST PreTransaction hook - 5 abandoned
transactions since 2026-07-29, stalls of 4 minutes to overnight, always fine on
retry. This investigates it and repairs the real defects found on the way.
The stall is NOT fixed, and deliberately so: no verified defect was found, so
nothing pacman-side was changed.
- Not guard-lib. /etc/guard-lib/targets was empty, so that hook script
provably exits in milliseconds. The failures share POSITION (first hook of
the transaction), not identity: one stall hit the hosts hook back when that
one ran first.
- Not reproducible on demand. 65 real transactions (40 idle, 25 under 2.6 GB
of memory pressure) produced zero stalls. Against the historical 4-in-56
rate that is ~0.9% likely, so the trigger is conditional on something a
synthetic load does not recreate. Memory pressure is untested at the load
that actually occurred - not refuted.
So diagnose_pacman_hook_stall.sh drives real transactions with a watchdog, and
its --watch mode passively captures /proc/<pid>/{stack,wchan,syscall}, the
process tree, meminfo and PSI from the next REAL stall. No pacman-side fix
should ship before that dump exists.
Three real defects were found and fixed:
1. The deployed pacman wrapper was 19916 B against a 27510 B source, never
sourcing the shared pacman_lock_lib.sh that makepkg_wrapper already used.
It hid because check_pacman_wrapper() only tested that files EXIST. Both
installers now write a drift manifest covering source AND installed hashes;
the checker replays it with sha256sum -c and fails closed when it cannot
verify. Manifest paths are absolute - relative ones resolve to nothing under
systemd's cwd=/ and would have looped reinstalling forever.
2. The guard-lib migration (66c4698) was never finished here: guard-lib was
installed with zero registered targets while the legacy hosts-guard layer did
all the work, so every transaction ran two PreTransaction hooks and the first
one did nothing. migrate_hosts_guard_to_guard_lib.sh registers hosts,
nsswitch and resolved, retires the legacy hooks and units, and has a
--rollback that was actually exercised. Plugins install to a stable path
because guardctl records absolute plugin paths. Side effect: one hook pair
instead of two, transactions ~5-6s -> ~2-3s.
3. "One heavy job at a time" was a rule a human had to remember. heavy_job_lock.sh
makes the overlap impossible: one system-wide flock shared between root and
the user (a per-user lock would give root's pacman and the user's builds
separate locks and no exclusion at all). Wired into pacman TRANSACTIONS only
- read-only queries stay instant - plus makepkg_capped and the coverage hook.
Always fails open.
Two bugs surfaced only by running --rollback, both of which briefly left
/etc/hosts with no immutable flag: fg_uninstall clears +i on its way out while
legacy watchers only fire on PathModified, and `((i > 20)) && break` left the
loop body's status at 1, which under set -e aborted the rollback silently - but
only when the loop actually ran, so --dry-run looked healthy.
Not migrated: the shutdown-schedule guard (/etc/shutdown-schedule.conf does not
exist here). Not wired: Flutter/Gradle builds, which have no single entry point.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Krzysztof Rudnicki
79829ec8a2