scripts/vmbox/harness.sh provisions a disposable Arch guest, runs the REAL
install.sh in it, and adjudicates 8 cases on a host-side exit code. Plus
DOCS-vmbox.md, which states what a pass does and does not mean.
Covered: system-python imports, timer enabled, wake-alarm.service still
`static` (an [Install] there fires the alarm at *login*), the shutdown wrapper
shadowing /usr/bin/shutdown, the alarm-cache shape jq expects, a trigger tick,
the inhibitor, and the re-fire path. With --window it also opens the real
fullscreen Tk window and screenshots it from the host -- confirmed showing
"WAKE UP!", round 1/4 and a live flash challenge.
Not covered, and said plainly in the doc rather than implied away: audio (the
guest has no pactl/paplay), the phone, GPU and real outputs, and anything
bootloader-related (guest is BIOS, this machine is UEFI).
**The VM earned its place immediately by finding two live bugs that the host
could not reach and the unit suite passed straight through.**
1. The inhibitor claimed a block it did not have. Popen succeeding only means
the binary launched; logind can still refuse -- the guest's polkit denies a
non-interactive session outright ("Failed to inhibit: Access denied", exit
1) while the alarm cheerfully logged the power key as blocked.
take_power_key_inhibitor() now waits INHIBIT_CONFIRM_SECONDS and confirms
the child survived, returning None and logging the refusal otherwise.
2. An unsigned session record was written and then rejected on read. With no
HMAC key on the machine (/etc/gatelock absent, as in the guest),
verify_entry_hmac returns False for a *missing* key exactly as it does for
a forgery -- so every session record was unreadable and item 2's power-off
defence silently did not exist. An entry with no "hmac" field is now
honoured with a loud warning; a present-but-wrong signature is still
rejected, since that one really is tampering.
Two harness traps worth recording, both hit while building it:
- `vm run` reports the *VM's* verdict, not the guest command's exit code, so a
guest script exiting 1 still gives rc=0. A gate built on that fails OPEN.
guest_cases.sh prints ALL-CASES-PASSED only when every case passed and the
harness greps for it; verified by breaking a case on purpose and watching
the gate go red.
- `vm share`/`vm repo` key off the path basename, so REPO_DIR must be absolute
and symlink-resolved. A relative "scripts/vmbox/../.." makes rsync write
into /home and fail with permissions errors that look like a guest fault.
Guest checks live in real .py files beside the shell, not heredocs, so they
get linted like everything else -- ruff's autofix gutted an early heredoc
version (stripped the assignments AND the print) exactly as TODO-hardening
warned, and only running it showed the file had stopped producing output.
Also drops three stale fan references the guest install run surfaced:
install.sh still announced "Fans will ramp to 100%" and README.md still
described the fan ramp, three commits after item 14 deleted the feature.
Splits carried: test_inhibit.py -> + test_inhibit_cleanup.py,
test_alarm_session.py -> + test_alarm_session_refire.py.
455 tests pass, 100% branch coverage. Harness: 8/8 cases, exit 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EckWmyfetYq3Mi8MAnUudh
5.9 KiB
Testing wake_alarm in a disposable VM
scripts/vmbox/harness.sh is this repo's default gate for the shell and
system pieces — install.sh, the systemd units, shutdown-wrapper.sh, the
power-key inhibitor and the re-fire path. It provisions a throwaway Arch guest
(~/utils/vmbox), installs this repo into it exactly the way a real machine
does, runs the cases, and returns a host-side verdict.
bash scripts/vmbox/harness.sh # the gate
bash scripts/vmbox/harness.sh --window # also screenshot the real alarm window
bash scripts/vmbox/harness.sh --keep # leave the sandbox up to poke at
Exit codes: 0 everything passed, 1 a case failed, 2 the harness itself
could not run.
Why a VM at all
Every one of these touches something a test on the host cannot safely touch:
install.sh writes systemd user units, a sudoers entry and
/usr/local/bin/shutdown; the shutdown wrapper's whole job is to hibernate the
machine; the inhibitor takes a logind lock on the power button. Getting any of
those wrong on the host is not a failed test, it is a broken machine.
What it actually covers
| Case | What a pass means |
|---|---|
imports-from-system-python |
wake_alarm and gatelock import under /usr/bin/python3, the interpreter the units really use — not a venv |
trigger-timer-enabled |
wake-alarm-trigger.timer is enabled after install.sh |
alarm-service-never-auto-starts |
wake-alarm.service is still static. An [Install] section there fires the alarm at login, because _should_run_alarm() has no time gate |
shutdown-wrapper-shadows-real-shutdown |
shutdown resolves to /usr/local/bin/shutdown, ahead of /usr/bin/shutdown |
wrapper-parses-the-alarm-cache |
jq reads the {"hour","minute","fetched_at"} shape the wrapper expects — the only coupling left between it and _sync.py's _write_cache() |
trigger-dry-run |
one tick reaches a decision without crashing |
inhibitor-never-claims-a-block-it-lacks |
see the caveat below — this is not "the power button was blocked" |
cut-short-alarm-refires |
an unfinished session record makes the trigger fire even with the catch-up window long closed |
alarm-window-opens (--window) |
the fullscreen Tk window really paints, on a real X server, verified by a host-side screenshot |
install.sh itself running to completion is a precondition of all of them: if
it fails, the harness stops with exit 2 and dumps the log.
What it cannot cover — do not read a pass as covering these
- Audio. The guest has no
pactland nopaplay. Every sink, card-profile and reactivation path — the whole of item 6 — is untestable here and was verified directly on the host instead. - That the power button is actually blocked. The guest has no physical
power button, and its polkit refuses an inhibit from a non-interactive ssh
session outright (
Failed to inhibit: Access denied). So the case asserts something narrower but still useful: that the code never claims a block it did not get.REFUSEDis a passing outcome. - The phone. Nothing about the Flutter app, the sync round-trip, or the alarm the phone rings.
- GPU and the real display outputs. The guest's X server is not DP-2 and HDMI-0, so multi-monitor surface placement proves nothing here.
- Anything bootloader-related. The guest is BIOS/GRUB; this machine is
UEFI. A guest pass on boot-adjacent work is actively misleading — a sibling
repo's
boot-repairpasses in the guest while deleting the only bootable kernel on a UEFI host. - rtcwake actually waking the machine.
rtcwakeexists in the guest, but a VM's RTC alarm is not the host's firmware.
A sandbox pass never proves the host is fine. It is an added first step: sandbox → report → host run. Phrase results as "passed in vmbox, not verified on the host".
Guest provisioning, and why
The stock guest image is missing exactly one thing this package needs:
tk— without itimport tkinterfails withlibtk8.6.so: cannot open shared object fileand the alarm cannot open its window at all.
gatelock is not installed separately: it is a pyproject.toml dependency, so
install.sh's pip install -e pulls it from its pinned tag. That is
deliberate — the harness runs the real install path rather than a hand-rolled
approximation, so a break in install.sh fails the gate.
Two traps worth knowing before editing the harness
vm rundoes not propagate the guest command's exit code. It reports the VM's verdict (clean / dirty / rebooted / crashed / hung), so a guest script exiting 1 still returns 0 on the host. A gate built on that exit code fails open, which is worse than no gate.guest_cases.shtherefore printsALL-CASES-PASSEDonly when every case passed, and the harness greps for it.vm shareandvm repokey off the path's basename, so the repo path must be absolute and symlink-resolved. A relativescripts/vmbox/../..makes rsync write into/homeand fail with permission errors that look like a guest problem.
Cases this VM found that the host could not
Both were live bugs, both passed the unit suite:
- The inhibitor claimed a block it did not have.
Popensucceeding only means the binary launched; logind can still refuse. The alarm logged "power key inhibited" whilesystemd-inhibithad already exited 1.take_power_key_inhibitor()now waits briefly and confirms the child survived. - An unsigned session record was written and then rejected on read. With
no HMAC key on the machine (
/etc/gatelockabsent, as in the guest),verify_entry_hmacreturns False, so every session record was unreadable and the power-off defence silently did not exist. An entry with nohmacfield is now honoured with a loud warning, while a present but wrong signature is still rejected — that one really is tampering.