QEMU user-mode networking (libslirp 4.9.4 / qemu 11.1.1) freezes 2-4 of 5
concurrent 60 MB streams at ~10 MB until they time out, while one stream
does 45 MB/s. pacman's ParallelDownloads = 5 hit this on every guest
upgrade: a 90-package -Syu crawled at "500 KB/s" for minutes and then
finished in 12 s once set to 1. Window scaling and rmem do not help; a
tap/bridge NIC would but needs root, which vmbox avoids by design.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SS8M5wk88nJTXxDWz61KKk
The golden image logs in with agetty+startx and has no RGB controllers,
no NVIDIA card and no sound card, so the night-lockdown scripts — which
mask/stop lightdm.service and getty@.service and shell out to openrgb,
nvidia-smi and amixer — had nothing real to act on in a sandbox and
were being "verified" on the host instead.
- `vm lightdm <name>`: installs lightdm + lightdm-gtk-greeter into the
live overlay (pacman -Syu, not -S: the overlay's package database is
the image's and mirrors have rotated those versions out), configures
autologin -> i3 exactly like the host, turns tty1 back into a plain
getty, reboots the guest and proves lightdm.service is active with an
Xorg it owns. Never rebuilds the base: `vm build --force` re-seals a
new checksum while every existing sandbox still backs onto the old
file's name, which is silent qcow2 corruption.
- `vm shim <name> <tool> [exit[:N]|hang|sleep:S]`: a fake tool ahead of
the real one on PATH, every call logged with an epoch to
/var/log/vmbox-shim/<tool>.log so the host can assert what ran and in
what order. `hang` is the openrgb 1.0-2 that never returned on
2026-09-12 and defeated the curfew.
- The DISPLAY/XAUTHORITY resolver `vm run` sources becomes its own file,
guest/vmbox-x11.sh, installed by the image build and re-installed by
the lightdm switch; it now tests readability, not existence, and tries
~/.Xauthority before the startx serverauth glob, because lightdm's
Xorg -auth file is root-only.
- README: the worked example uses MIDNIGHT_SHUTDOWN_CONFIRM=y (the
installer refuses to run unattended now; `echo y |` no longer works),
and the "i2c RGB stays on the host" line is gone.
- bats: four host-side tests for shim.sh (exit, exit:N, hang, refusals).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015mLnUeoJ3mUwGRTowTUH6d
Markdown across every repo had accumulated a dozen ad-hoc names
(refactor_claude_todo.md x23, DESIGN_AUDIT_TODO.md, NEXT_SESSION,
HANDOFF, PLAN, SPEC) with nothing distinguishing a live task from a
finished record. An audit found docs asserting work was pending that
shipped months earlier, and one asserting completion while the work
was open.
Four namespaces now: README.md, CLAUDE*.md, DOCS*.md, TODO*.md. Task
files carry the literal "REMOVE ME AFTER FINISH" and are deleted when
the work lands.
Enforced bidirectionally -- a TODO without the marker fails, AND any
file carrying the marker that is not named TODO*.md fails. Without the
reverse rule a task can be renamed into the permanent namespace and
quietly live forever, which is the failure being fixed.
md_naming/ mirrors the file_length topology and imports its tables
(EXCLUDED_DIRS, THIRD_PARTY_REPOS, VENDORED_ANYWHERE, GENERATED_PATTERN)
rather than restating them, so the two gates cannot disagree about the
same file. 71 tests, 100% branch coverage.
stale.py warns on TODOs past 90 days but never fails: no machine can
prove a task is finished, so the honest mechanism is a nudge.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A47aX5DJwmA3QFn3pkEWi7
The previous commit skipped the poweroff settle when `ssh_rc != 255`, on the
reasoning that a successful poweroff kills the ssh connection and ssh reports
255. That holds for the ssh transport and for nothing else. `_run_transport`
falls back to the serial console whenever ssh does not answer -- and the
README calls serial "the transport of record for shutdown tests" -- where the
exit code comes from serial_exec.py and is 0 even while the machine is already
on its way down.
Result, reproduced: `VMBOX_TRANSPORT=serial vm run <vm> 'sudo systemctl
poweroff'` reported "still running" for a guest that had cleanly powered off,
with SHUTDOWN{guest:true} in events.jsonl and "reboot: Power down" in the
serial log. The one thing this tool exists to get right, silently wrong. My
five verdict re-tests all went through ssh, so none of them caught it.
Grepping the serial log for shutdown markers instead does not work either:
they land a second or two after the command returns, so the check loses the
race. The fix waits a short grace period (default 3s) for any sign of a stop
before committing to the 30s wait. Warm runs go from 0s to 3s; that is the
honest price of a correct verdict.
Re-tested: clean poweroff, reboot and panic on BOTH transports, plus
tests/destructive_demo.sh 7/7.
Also, per "a fix that needs a human to remember a flag is not a fix": the X
session wait is now automatic on the branch that actually boots the guest --
the only one that can race tty1's autologin -- instead of requiring
VMBOX_X_WAIT. Warm runs never reach that branch. The env var remains an
override. README numbers corrected to the measured 14s/3s and ~100s.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdyMBGCrGVGDWtJ9xe48f3
`vm run` took 41s cold / 30s warm to run `true`. Both were vmbox waiting on
itself. Now 12s / 0s, with all five shutdown verdicts re-tested unchanged.
- _run_settle waited up to 30s for the kernel's "reboot: Power down" marker
UNCONDITIONALLY, including on the majority of runs that leave the guest up,
where that marker is never going to appear. It now settles only when the
guest has actually stopped, or when ssh returned 255 (what a successful
poweroff looks like from the host).
- The ssh readiness polls used a 5s ConnectTimeout. QEMU's user-mode
networking ACCEPTS the forwarded connection while the guest still has
nothing on :22, so each probe blocked for the full timeout instead of
failing fast: a guest reachable at ~10s was reported up at ~45s. Fixed in
both the sandbox and build paths, via an env override -- ssh honours the
FIRST value for an option, so a second -o could not have worked.
Making runs fast exposed a race: sshd now answers BEFORE tty1's autologin has
started Xorg, so the X environment could be missing on the first run after a
boot. `VMBOX_X_WAIT=<seconds>` waits for the session; it defaults to 0 so an
ordinary run never pays for a graphical session it is not using.
Build: the host's /var/cache/pacman/pkg is offered to the build guest
read-only (`readonly=on` AND an `ro` mount, because it is the real host cache)
as an extra CacheDir after the guest's own writable one, so a package the host
lacks still downloads rather than failing the build. This removes the 396 MiB
the build used to fetch. Host cache verified byte-for-byte identical across
four builds: same file count, same size, same mtime.
Build time itself moved only 103.8s -> 100.3s. Per-phase timings (also added
here) show why: 47s of it is the build guest's cloud-init first boot, and that
is NOT datasource probing -- `cloud-init analyze blame` totals 2.27s. Left
unexplained rather than guessed at; SESSION_RESULTS.md records the evidence.
Also: `vm scp`, which lib/ssh.sh could always do but the CLI never exposed,
and provision.sh split at the 250-line cap.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdyMBGCrGVGDWtJ9xe48f3
Captured a real 1280x800 PNG of a sandbox's i3 session straight from qemu's
framebuffer -- no VNC or SPICE viewer installed on the host. This is the path
locker and fullscreen-gate tests use, and it is safer than testing those on
the host, where a fullscreen locker can hijack input and steal focus.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdyMBGCrGVGDWtJ9xe48f3
Ran kuhy's real setup_midnight_shutdown.sh inside a sandbox -- the installer
whose own output says "There is NO disable option - this is protected by a
monitor service". It created /etc/shutdown-schedule.conf under guard-lib's
immutable protection and enabled shutdown-timer-monitor.service,
day-specific-shutdown.timer and shutdown-timer-monitor-watchdog.timer.
The host was provably untouched afterwards: base image checksum OK, /etc/hosts
still carrying its +i flag, and the host's own shutdown-schedule.conf still
stamped 11:21 that morning rather than the 21:19 when the guest wrote its copy.
Two properties of that installer are now documented, both found by running it:
it is interactive (`read -r -p 'Do you want to proceed?'`, so pipe `echo y |`
since vm run closes stdin deliberately), and it is subcommand-driven, so a
bare invocation prints usage and changes nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdyMBGCrGVGDWtJ9xe48f3
The destructive demo asked the guest to confirm its own destruction, one step
after deleting /usr -- which removes bash, sshd and agetty. Nothing would have
been left to answer, so the check would have FAILED on a step that actually
succeeded. It now proves breakage from the host (the overlay grew, and the
guest can no longer answer at all), which is the same evidence the shutdown
verdict relies on and for the same reason: a corpse cannot report its own
death.
Step 3 also no longer recreates the sandbox. `vm reset` in step 4 is the claim
under test; recreating first would have quietly tested something easier.
README's verdict table now carries the MEASURED qmp events per guest command
rather than the inferred ones, and states the finding that motivates the whole
design: ssh returned an identical "Connection closed by remote host" for the
poweroff, the reboot AND the panic.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdyMBGCrGVGDWtJ9xe48f3
Much of testsAndMisc/linux_configuration cannot be tested on the host: 15
files use `chattr +i` (so a bad test is not simply revertable), the shutdown
installers write units that power the machine off, and boot_recovery installs
pacman hooks. The repo's own CI says as much -- its Ubuntu job runs only the
side-effect-free tests, noting the rest "drive pacman, /etc, systemd or adb".
vmbox runs them in a throwaway Arch guest and verifies the result from the
HOST, which is the part that is not obvious: a successful shutdown kills the
connection carrying the report, and ssh returns 255 for a clean poweroff, a
crash and a dropped link alike. So the verdict comes from two artefacts that
outlive the VM -- a QMP event log written by a recorder attached at launch
(attach later and qemu has already exited), and the serial console log.
Five outcomes are distinguished: clean poweroff (SHUTDOWN guest:true plus a
serial tail reaching "reboot: Power down"), dirty stop, guest-reset (rebooted
instead of powering off), GUEST_PANICKED, and hung. A POWERDOWN event proves
nothing -- it means the request was delivered, not that anything acted on it.
Reset is a qcow2 backing-file trick: the golden image is built once and sealed
(chmod 444 plus a sha256 sidecar verified on every launch, since booting the
base would silently invalidate every overlay), and each sandbox is a thin
overlay, so reset is rm + recreate. No root anywhere -- user-mode networking
with per-VM forwarded ssh ports, and a socket multicast segment for VM-to-VM.
The guest clock is pinned and re-applied on every launch, because the shutdown
installers gate on a 21:00-05:00 window; NTP is masked in the image or it
silently undoes the pin. Repos are cloned from a read-only share scoped to an
explicit directory, never $HOME, and credentials are sentinel values.
`vm build` ends by booting a throwaway sandbox off the sealed image and
refusing to ship if ssh does not answer. Several distinct faults produced an
identical "boots fine, connection reset" symptom; checks inside the build VM
only prove state at seal time, so the gate has to be an end-to-end boot.
Tests: 8 bats cases over the verdict table and name/meta/serial handling;
shellcheck and ruff clean; every file within the 250-line cap.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdyMBGCrGVGDWtJ9xe48f3
