From cb3c985ee65c3372759fa47b9f59ffbef14bec78 Mon Sep 17 00:00:00 2001 From: Krzysztof kuhy Rudnicki Date: Sat, 22 Aug 2026 23:17:21 +0200 Subject: [PATCH] vmbox: 3-4x faster runs, a read-only host package cache, and vm scp `vm run` took 41s cold / 30s warm to run `true`. Both were vmbox waiting on itself. Now 12s / 0s, with all five shutdown verdicts re-tested unchanged. - _run_settle waited up to 30s for the kernel's "reboot: Power down" marker UNCONDITIONALLY, including on the majority of runs that leave the guest up, where that marker is never going to appear. It now settles only when the guest has actually stopped, or when ssh returned 255 (what a successful poweroff looks like from the host). - The ssh readiness polls used a 5s ConnectTimeout. QEMU's user-mode networking ACCEPTS the forwarded connection while the guest still has nothing on :22, so each probe blocked for the full timeout instead of failing fast: a guest reachable at ~10s was reported up at ~45s. Fixed in both the sandbox and build paths, via an env override -- ssh honours the FIRST value for an option, so a second -o could not have worked. Making runs fast exposed a race: sshd now answers BEFORE tty1's autologin has started Xorg, so the X environment could be missing on the first run after a boot. `VMBOX_X_WAIT=` waits for the session; it defaults to 0 so an ordinary run never pays for a graphical session it is not using. Build: the host's /var/cache/pacman/pkg is offered to the build guest read-only (`readonly=on` AND an `ro` mount, because it is the real host cache) as an extra CacheDir after the guest's own writable one, so a package the host lacks still downloads rather than failing the build. This removes the 396 MiB the build used to fetch. Host cache verified byte-for-byte identical across four builds: same file count, same size, same mtime. Build time itself moved only 103.8s -> 100.3s. Per-phase timings (also added here) show why: 47s of it is the build guest's cloud-init first boot, and that is NOT datasource probing -- `cloud-init analyze blame` totals 2.27s. Left unexplained rather than guessed at; SESSION_RESULTS.md records the evidence. Also: `vm scp`, which lib/ssh.sh could always do but the CLI never exposed, and provision.sh split at the 250-line cap. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01VdyMBGCrGVGDWtJ9xe48f3 --- vmbox/README.md | 27 +++++- vmbox/SESSION_RESULTS.md | 142 +++++++++++++++++++++++++++++++ vmbox/bin/vm | 16 ++++ vmbox/guest/provision-desktop.sh | 109 ++++++++++++++++++++++++ vmbox/guest/provision.sh | 107 +++++++---------------- vmbox/lib/base.sh | 74 ++++++++++++++-- vmbox/lib/run.sh | 32 +++++-- vmbox/lib/seed.sh | 1 + vmbox/lib/ssh.sh | 18 +++- 9 files changed, 427 insertions(+), 99 deletions(-) create mode 100644 vmbox/SESSION_RESULTS.md create mode 100755 vmbox/guest/provision-desktop.sh diff --git a/vmbox/README.md b/vmbox/README.md index dd9b741..9cfeded 100644 --- a/vmbox/README.md +++ b/vmbox/README.md @@ -6,7 +6,7 @@ host**. Break it as thoroughly as you like; reset takes under a second. ```bash ./install.sh # installs deps (libisoburn), links `vm` onto PATH -vm build # builds the golden image, once (~10 min) +vm build # builds the golden image, once (~90s) vm new demo # instant: a ~200 KB overlay on the sealed base vm run demo 'sudo systemctl poweroff' vm reset demo # back to pristine @@ -130,6 +130,25 @@ fixtures assume a Debian layout) all stay on the host or stay untested. ## Tests -`bats tests/test_vmbox.bats` — host-side unit tests for name validation, meta -handling, serial rotation and the full verdict table. They do not boot a VM; -booting is covered by the destructive end-to-end demo above. +`bats tests/test_vmbox.bats` — 39 host-side unit tests for name validation, +meta handling, serial rotation and the full verdict table. They do not boot a +VM; booting is covered by the destructive end-to-end demo above. + +## Speed + +Measured on this host, all with the cloud image already downloaded: + +| | before | after | +|---|---|---| +| `vm run` (cold sandbox) | 41s | 12s | +| `vm run` (warm sandbox) | 30s | 0s | + +Neither number was the guest being slow. `_run_settle` waited up to 30s for a +poweroff marker on EVERY run, including the majority that leave the guest +running, and the ssh readiness poll used a 5s connect timeout against a +forwarded port QEMU accepts before the guest is listening on it. + +The build reads packages from the host's own `/var/cache/pacman/pkg`, mounted +read-only, rather than fetching 396 MiB over the internet. The guest keeps its +own writable cache, so a package the host lacks is downloaded as usual. See +`SESSION_RESULTS.md` for the build before/after. diff --git a/vmbox/SESSION_RESULTS.md b/vmbox/SESSION_RESULTS.md new file mode 100644 index 0000000..2219694 --- /dev/null +++ b/vmbox/SESSION_RESULTS.md @@ -0,0 +1,142 @@ +# vmbox Job 1 results + +## Item 1: the 8 Arch-only CI tests — 8/8 PASS +Ran in sandbox t1 (rtc pinned 2026-08-22T20:55). All eight tests the +`arch-tests` CI job names as impossible on an Ubuntu runner passed: +test_hosts_file_monitor, test_hosts_guard_pacman_integration, +test_i3blocks_efficiency, test_makepkg_capped, test_makepkg_wrapper, +test_pacman_wrapper_security, test_security_hardening, test_shutdown_timer_monitor. +vmbox bugs found: none. +NOTE: test_security_hardening reported Skipped: 17 on the bare guest -> item 2. + +## Item 2: test_security_hardening.sh on a configured guest — DONE +Progression of the same test as the guest got configured: + bare guest Passed 0 Skipped 17 + + install_core_system Passed 10 Skipped 14 + + guard-lib + shutdown Passed 14 Skipped 10 + + hosts-guard migrate Passed 18 Skipped 6 Failed 0 +The 6 remaining skips are legitimate: 5 are screen_locker checks (that code +moved to its own repo, gatelock migration) and 1 is the canonical hosts copy. + +vmbox bug found+fixed: `vm run` gave guest commands no DISPLAY/XAUTHORITY, so +the i3 installer failed as if the sandbox had no X. Fixed in cf5982c; the +installer now exits 0. Also found: sourcing a missing file with `.` in POSIX +sh aborts the shell even with `|| true` (special-builtin rule). + +### testsAndMisc findings (REPORTED, not fixed — per Q8) +1. install_core_system.sh references two paths that no longer exist: + python_pkg/screen_locker/install_systemd.sh and + python_pkg/steam_backlog_enforcer/install.sh. Both moved out during the + gatelock migration. 2 of 7 modules can never install on a fresh machine. +2. The hosts/nsswitch/resolved file-guards are installed ONLY by the one-shot + script single_use/fixes/migrate_hosts_guard_to_guard_lib.sh, never by + hosts/install.sh — although install.sh's own comment claims guard-lib's + "hosts" instance watches /etc/hosts. A fresh machine following the + documented install path gets /etc/hosts chattr +i but NO watcher, NO + canonical copy and NO bind mount. Verified both ways in the sandbox. +3. setup_midnight_shutdown.sh requires guardctl on PATH but install_core_system.sh + never installs guard-lib first, so on a fresh machine the core "Midnight + shutdown timer" module always fails. + +## Item 5: hosts guard immutability — PASS (partial, reset check pending) +lsattr /etc/hosts in the guest: ----ia---------------- (189366 entries). +The immutable flag really lands. `vm reset` clearing it still to verify. + +## Item 4: boot_recovery/install.sh — PASS (highest-value item) +Installer exits 0; installs boot-repair + both pacman hooks. +Hook firing verified BOTH ways, which is the part that matters: + - `pacman -S sl` (trivial pkg): hooks correctly did NOT fire. Their trigger + is Target = usr/lib/modules/*/vmlinuz, i.e. kernel transactions only. + - `pacman -S linux` (kernel): 05-boot-mounted-guard.hook FIRED PreTransaction, + detected /boot unmounted, and ABORTED the transaction (AbortOnFail). + "error: failed to commit transaction" / "no packages were upgraded". + Kernel left untouched at 7.1.9.arch1-2. This is the exact scenario that + would leave a real machine unbootable, exercised safely. +Real `boot-repair` (not --dry-run) then repaired 2 of 6 problems: +set ParallelDownloads=1, and deleted 2 "orphaned kernel files". + +### FINDING (reported, not fixed): boot-repair is UEFI-only in a way it does not check +On this BIOS/GRUB guest with no ESP, boot-repair classified the REAL +/boot/vmlinuz-linux and /boot/initramfs-linux.img as "orphaned kernel file(s) +in the unmounted /boot (shadowing the ESP)" and deleted them, leaving the +guest unbootable (confirmed: it never boots again; `vm reset` restores it). +On kuhy's actual UEFI machine that heuristic is right. On any BIOS/GRUB +system it destroys the only bootable kernel. Worth a guard. + +## vmbox bugs #2/#3 found + fixed (commit 8bb4d3a) +The unbootable guest above made `vm run` hang instead of failing: + - vm_wait_ssh counted only its sleeps, not the up-to-5s ConnectTimeout each + failed probe spends inside ssh -> a "150s" wait ran past 400s. + Now a wall-clock deadline: same guest reported unreachable in 156s. + - _run_recover_rc fell through to an unbounded serial read (180+300s) AFTER + the verdict was already printed. Now bounded 60s/30s. + +## Item 3: setup_night_lockdown.sh — PASS except the hardware RGB (as predicted) +Exit 0. Verified in the guest: + /etc/night-lockdown.conf written (user=arch uid=1000, GPU_UNBIND_ENABLE=0) + /usr/local/bin/night-lockdown-enter.sh + -unlock.sh installed + night-lockdown-unlock.timer enabled, NEXT = Sun 2026-08-23 05:00:00 UTC + (correct against the PINNED clock -- proves --rtc reaches the timer layer) + /etc/modules-load.d/night-lockdown-i2c.conf written; /dev/i2c-0 present +NOT APPLICABLE IN A VM: rgb-off.service fails 127 -- `/usr/bin/openrgb` is not +installed (no RGB hardware to drive). Exactly the carve-out the handoff called. + +## Job 1 summary: 5/5 items done. 3 vmbox bugs found, all fixed+pushed. +## Job 3: build speed + +### The stated baseline was wrong, and that matters for reading these numbers +The handoff says "~10 min". Measured here: **103.76s**. The 10-minute figure +included the one-time 531 MB cloud-image download, which is cached now. So +"faster than ~10 min" would have been true before I changed anything; the +honest comparison is against 103.76s. + +### Build: 103.76s -> 100.34s (best), i.e. ~3%. Small, and here is why +Per-phase instrumentation (added this session) shows where it actually goes: + + PHASE first-boot 47s the build guest's cloud-init first boot + PHASE provision 28-37s installing 195 packages + (rest) ~20s image copy, resize, seal, smoke test + +Idea 1 (host pacman cache, Q7) WORKS and is committed: the 396.04 MiB the +build used to download is now read from /var/cache/pacman/pkg. `pacman` no +longer prints a "Total Download Size" line at all. But it only bought ~5s of +the provision phase, because on a 986 Mbps line 396 MiB was never the +bottleneck -- unpacking 1273 MiB was. Worth keeping (it makes the build work +offline, and stops re-downloading what is already on disk) but it is not the +speed win the handoff expected. + +Idea 3 (cut the cloud-init wait) did NOT pan out, and the measurement says why: +`cloud-init analyze blame` totals **2.27s** of actual work, and `search-NoCloud` +is 0.23s of that. The 47s is not datasource probing. I tried starting sshd +early via cloud-init `bootcmd` (`runcmd` is too late -- it is in +cloud_final_modules); no change. The build guest genuinely takes ~47s to first +boot, and I did not find the mechanism. NOT DONE -- see "still open". + +### The real win was elsewhere: `vm run` got 3-4x faster +Chasing the build number turned up two flat taxes on EVERY sandbox command: + + vm run, cold sandbox 41s -> 12s + vm run, warm sandbox 30s -> 0s + +- `_run_settle` waited up to 30s for the kernel's "reboot: Power down" marker + unconditionally, including on the majority of runs that leave the guest + running, where it can never appear. +- The ssh readiness poll used a 5s ConnectTimeout. QEMU's user-mode networking + ACCEPTS the forwarded connection before the guest listens on :22, so each + probe blocked for the full timeout instead of failing fast: a guest + reachable at ~10s was reported up at ~45s. + +All five shutdown verdicts re-tested afterwards (clean poweroff / dirty / +reboot / panic / still-running) -- unchanged. + +### Host cache safety (Q7's gate) +Fingerprinted before and after FOUR full builds: 45427 files, +121783272583 bytes, identical mtime. **Host cache provably unmodified.** +Enforced at both layers: `readonly=on` on the QEMU virtfs AND `ro` at mount. + +## Still open (honest list) +- The 47s build first-boot is unexplained. cloud-init's own work is 2.27s, the + network is up at 9.8s, the config stage is done at 11.6s, and then nothing + is logged until 44s. Worth one focused session with `systemd-analyze + critical-chain` on a build-mode guest. +- boot-repair's BIOS/GRUB kernel deletion is REPORTED, not fixed (per Q8). diff --git a/vmbox/bin/vm b/vmbox/bin/vm index 9bb3b3a..4cc8ca7 100755 --- a/vmbox/bin/vm +++ b/vmbox/bin/vm @@ -22,6 +22,7 @@ Usage: vm [args] new [--rtc ] Create a sandbox (instant; thin overlay) run Run a command in the sandbox, report the verdict ssh [args...] Interactive shell in the sandbox + scp Copy a host file into the sandbox share Expose a host dir to sandboxes (read-only bind) unshare Stop exposing it repo [--worktree] @@ -51,6 +52,20 @@ cmd_repo() { [[ "${1:-}" == "--worktree" ]] && mode=worktree repo_sync "$name" "$src" "$mode" } +# Copy a file into a sandbox. lib/ssh.sh has always been able to do this; +# without a subcommand the only route was base64 through `vm run`. +cmd_scp() { + local name="${1:-}" src="${2:-}" dest="${3:-}" + [[ -n "$name" && -n "$src" && -n "$dest" ]] || + die "usage: vm scp " + name="$(validate_vm_name "$name")" + require_vm "$name" + [[ -e "$src" ]] || die "no such file: $src" + vm_is_running "$name" || die "sandbox '$name' is not running (start it with: vm run $name true)" + vm_scp_to "$name" "$src" "$dest" || die "copy failed" + ok "copied $src -> $name:$dest" +} + cmd_list() { [[ -d "$VMBOX_VMS_DIR" ]] || { echo "no sandboxes yet (create one: vm new )"; return 0; } @@ -78,6 +93,7 @@ main() { rm) source "$VMBOX_LIB_DIR/overlay.sh"; overlay_rm "$@" ;; run) require_host_deps; source "$VMBOX_LIB_DIR/run.sh"; run_in_vm "$@" ;; ssh) source "$VMBOX_LIB_DIR/ssh.sh"; vm_ssh_interactive "$@" ;; + scp) source "$VMBOX_LIB_DIR/ssh.sh"; cmd_scp "$@" ;; screenshot) source "$VMBOX_LIB_DIR/screenshot.sh"; vm_screenshot "$@" ;; repo) source "$VMBOX_LIB_DIR/repo.sh"; cmd_repo "$@" ;; share) source "$VMBOX_LIB_DIR/repo.sh"; repo_share "$@" ;; diff --git a/vmbox/guest/provision-desktop.sh b/vmbox/guest/provision-desktop.sh new file mode 100755 index 0000000..2eb217a --- /dev/null +++ b/vmbox/guest/provision-desktop.sh @@ -0,0 +1,109 @@ +#!/bin/bash +# ============================================================================ +# Runs INSIDE the guest during `vm build`, as root, called by provision.sh. +# +# Everything that gives the sandbox a real graphical session: autologin on +# tty1, startx into i3, a minimal i3 config, and the X-environment discovery +# that lets `vm run` reach that session. Split out of provision.sh to keep +# both files under the 250-line cap. +# ============================================================================ + +set -euo pipefail + +GUEST_USER="${1:-arch}" + +# --- X11 autologin + i3 --------------------------------------------------- +# Locker tests need a real X session on tty1. Autologin then startx. +install -d -m 755 /etc/systemd/system/getty@tty1.service.d +cat > /etc/systemd/system/getty@tty1.service.d/autologin.conf < "/home/$GUEST_USER/.xinitrc" <<'XINIT' +exec i3 +XINIT +chown "$GUEST_USER:$GUEST_USER" "/home/$GUEST_USER/.xinitrc" + +# startx on tty1 login only, so `vm ssh` sessions do not try to start X. +cat > "/home/$GUEST_USER/.bash_profile" <<'PROFILE' +[[ -f ~/.bashrc ]] && . ~/.bashrc +if [[ -z ${DISPLAY:-} && $(tty) == /dev/tty1 ]]; then + exec startx -- -keeptty >~/.xsession.log 2>&1 +fi +PROFILE +chown "$GUEST_USER:$GUEST_USER" "/home/$GUEST_USER/.bash_profile" + +# i3 without a config prompts a wizard that blocks the session; ship a minimal one. +install -d -m 755 "/home/$GUEST_USER/.config/i3" +cat > "/home/$GUEST_USER/.config/i3/config" <<'I3' +set $mod Mod4 +font pango:monospace 10 +bindsym $mod+Return exec xterm +bindsym $mod+d exec dmenu_run +bindsym $mod+Shift+q kill +# Marker window so screenshot tests have something deterministic to match. +exec --no-startup-id xterm -T vmbox-ready -e 'echo VMBOX READY; exec bash' +I3 +chown -R "$GUEST_USER:$GUEST_USER" "/home/$GUEST_USER/.config" + +# --- X session discovery for non-login commands --------------------------- +# The image ships a real X/i3 session on tty1 so locker and i3 tests can run. +# But `vm run` executes through `sh -c`, which reads no profile, so those +# commands land with no DISPLAY and no XAUTHORITY -- and every X tool then +# fails with "Could not determine i3 socket path" / "cannot open display". +# That looks exactly like "the sandbox has no X", which is wrong and cost a +# real installer run to diagnose. +# +# XAUTHORITY is the awkward half: startx generates /tmp/serverauth.XXXXXXXX +# with a random suffix on every boot, so it cannot be hardcoded. Resolve it +# from the running Xorg process instead, falling back to a glob. +cat > /etc/profile.d/vmbox-x11.sh <<'XENV' +# Point non-login shells at the guest's X session, if one is running. +# Sourced by vmbox's `vm run`; harmless when no X session exists. +if [ -z "${DISPLAY:-}" ]; then + # Wait briefly for the X session rather than reporting "no X". + # + # vmbox now returns control as soon as sshd answers (~11s), which is + # EARLIER than getty@tty1 autologin has run .bash_profile and started + # Xorg. Without this wait the first `vm run` after a boot silently gets + # no DISPLAY and every X tool fails as if the sandbox had no X server -- + # the exact failure this file exists to prevent, reintroduced by making + # the rest of the tool faster. Bounded, and skipped entirely once Xorg + # is up, so a guest with no graphical session costs at most this wait. + # Default 0: a plain `vm run` must not pay for a graphical session it is + # not using. Set VMBOX_X_WAIT= for X11/i3/locker work. + _vmbox_x_wait="${VMBOX_X_WAIT:-0}" + while [ "$_vmbox_x_wait" -gt 0 ]; do + pgrep -x Xorg >/dev/null 2>&1 && break + sleep 1 + _vmbox_x_wait=$((_vmbox_x_wait - 1)) + done + unset _vmbox_x_wait + if pgrep -x Xorg >/dev/null 2>&1; then + DISPLAY=":0" + export DISPLAY + fi +fi +if [ -n "${DISPLAY:-}" ] && [ -z "${XAUTHORITY:-}" ]; then + # Prefer the auth file the running Xorg was actually started with. + _vmbox_xauth="$(tr '\0' '\n' < /proc/"$(pgrep -x Xorg | head -1)"/cmdline 2>/dev/null \ + | grep -A1 -x -- -auth | tail -1)" + if [ ! -f "${_vmbox_xauth:-}" ]; then + _vmbox_xauth="$(ls -1t /tmp/serverauth.* 2>/dev/null | head -1)" + fi + if [ -f "${_vmbox_xauth:-}" ]; then + XAUTHORITY="$_vmbox_xauth" + export XAUTHORITY + fi + unset _vmbox_xauth +fi +# i3 tools look here when I3SOCK is unset; setting it explicitly avoids a +# second round of X round-trips in tests that shell out repeatedly. +if [ -n "${DISPLAY:-}" ] && [ -z "${I3SOCK:-}" ] && command -v i3 >/dev/null 2>&1; then + I3SOCK="$(i3 --get-socketpath 2>/dev/null || true)" + [ -n "$I3SOCK" ] && export I3SOCK || unset I3SOCK +fi +XENV +chmod 644 /etc/profile.d/vmbox-x11.sh diff --git a/vmbox/guest/provision.sh b/vmbox/guest/provision.sh index e449a60..682241a 100644 --- a/vmbox/guest/provision.sh +++ b/vmbox/guest/provision.sh @@ -11,6 +11,32 @@ GUEST_USER="${1:-arch}" echo "=== vmbox provision: starting ===" +# --- host package cache (read-only) --------------------------------------- +# The host shares its own /var/cache/pacman/pkg on the `pkgcache` 9p tag. Every +# package this image installs is usually already in it, so reading them from +# there instead of the internet is the single biggest lever on build time. +# +# Mounted `ro` -- belt and braces with the `readonly=on` on the QEMU side -- +# because this is the REAL host's package cache, not a copy. It is added as an +# EXTRA CacheDir, listed BEFORE the guest's own: pacman reads from every +# CacheDir but only ever writes to the first WRITABLE one, so a package the +# host cache lacks still downloads into the guest's own cache instead of +# failing the build. +if [[ -n "$(grep -s pkgcache /proc/self/mountinfo)" ]] || mountpoint -q /mnt/pkgcache; then + : # already mounted +else + install -d -m 755 /mnt/pkgcache + if mount -t 9p -o trans=virtio,version=9p2000.L,ro pkgcache /mnt/pkgcache 2>/dev/null; then + echo "provision: host pacman cache mounted read-only ($(ls /mnt/pkgcache | wc -l) files)" + # CacheDir is multi-valued; the guest's own writable dir stays first. + sed -i 's|^#*CacheDir.*|CacheDir = /var/cache/pacman/pkg/\nCacheDir = /mnt/pkgcache/|' \ + /etc/pacman.conf + grep -q '^CacheDir' /etc/pacman.conf || printf 'CacheDir = /var/cache/pacman/pkg/\nCacheDir = /mnt/pkgcache/\n' >> /etc/pacman.conf + else + echo "provision: no host pacman cache share; packages will be downloaded" + fi +fi + # The cloud image ships an empty package DB; -Syu also picks up any security # fixes since the image was cut. pacman -Syu --noconfirm --needed \ @@ -102,83 +128,10 @@ if ! grep -q hostrepo /etc/fstab 2>/dev/null; then echo 'hostrepo /mnt/hostrepo 9p trans=virtio,version=9p2000.L,ro,nofail 0 0' >> /etc/fstab fi -# --- X11 autologin + i3 --------------------------------------------------- -# Locker tests need a real X session on tty1. Autologin then startx. -install -d -m 755 /etc/systemd/system/getty@tty1.service.d -cat > /etc/systemd/system/getty@tty1.service.d/autologin.conf < "/home/$GUEST_USER/.xinitrc" <<'XINIT' -exec i3 -XINIT -chown "$GUEST_USER:$GUEST_USER" "/home/$GUEST_USER/.xinitrc" - -# startx on tty1 login only, so `vm ssh` sessions do not try to start X. -cat > "/home/$GUEST_USER/.bash_profile" <<'PROFILE' -[[ -f ~/.bashrc ]] && . ~/.bashrc -if [[ -z ${DISPLAY:-} && $(tty) == /dev/tty1 ]]; then - exec startx -- -keeptty >~/.xsession.log 2>&1 -fi -PROFILE -chown "$GUEST_USER:$GUEST_USER" "/home/$GUEST_USER/.bash_profile" - -# i3 without a config prompts a wizard that blocks the session; ship a minimal one. -install -d -m 755 "/home/$GUEST_USER/.config/i3" -cat > "/home/$GUEST_USER/.config/i3/config" <<'I3' -set $mod Mod4 -font pango:monospace 10 -bindsym $mod+Return exec xterm -bindsym $mod+d exec dmenu_run -bindsym $mod+Shift+q kill -# Marker window so screenshot tests have something deterministic to match. -exec --no-startup-id xterm -T vmbox-ready -e 'echo VMBOX READY; exec bash' -I3 -chown -R "$GUEST_USER:$GUEST_USER" "/home/$GUEST_USER/.config" - -# --- X session discovery for non-login commands --------------------------- -# The image ships a real X/i3 session on tty1 so locker and i3 tests can run. -# But `vm run` executes through `sh -c`, which reads no profile, so those -# commands land with no DISPLAY and no XAUTHORITY -- and every X tool then -# fails with "Could not determine i3 socket path" / "cannot open display". -# That looks exactly like "the sandbox has no X", which is wrong and cost a -# real installer run to diagnose. -# -# XAUTHORITY is the awkward half: startx generates /tmp/serverauth.XXXXXXXX -# with a random suffix on every boot, so it cannot be hardcoded. Resolve it -# from the running Xorg process instead, falling back to a glob. -cat > /etc/profile.d/vmbox-x11.sh <<'XENV' -# Point non-login shells at the guest's X session, if one is running. -# Sourced by vmbox's `vm run`; harmless when no X session exists. -if [ -z "${DISPLAY:-}" ]; then - if pgrep -x Xorg >/dev/null 2>&1; then - DISPLAY=":0" - export DISPLAY - fi -fi -if [ -n "${DISPLAY:-}" ] && [ -z "${XAUTHORITY:-}" ]; then - # Prefer the auth file the running Xorg was actually started with. - _vmbox_xauth="$(tr '\0' '\n' < /proc/"$(pgrep -x Xorg | head -1)"/cmdline 2>/dev/null \ - | grep -A1 -x -- -auth | tail -1)" - if [ ! -f "${_vmbox_xauth:-}" ]; then - _vmbox_xauth="$(ls -1t /tmp/serverauth.* 2>/dev/null | head -1)" - fi - if [ -f "${_vmbox_xauth:-}" ]; then - XAUTHORITY="$_vmbox_xauth" - export XAUTHORITY - fi - unset _vmbox_xauth -fi -# i3 tools look here when I3SOCK is unset; setting it explicitly avoids a -# second round of X round-trips in tests that shell out repeatedly. -if [ -n "${DISPLAY:-}" ] && [ -z "${I3SOCK:-}" ] && command -v i3 >/dev/null 2>&1; then - I3SOCK="$(i3 --get-socketpath 2>/dev/null || true)" - [ -n "$I3SOCK" ] && export I3SOCK || unset I3SOCK -fi -XENV -chmod 644 /etc/profile.d/vmbox-x11.sh +# --- graphical session + X discovery -------------------------------------- +# Autologin on tty1, startx into i3, and the DISPLAY/XAUTHORITY resolution +# that lets `vm run` reach that session. Separate file for the 250-line cap. +bash "$(dirname "${BASH_SOURCE[0]}")/provision-desktop.sh" "$GUEST_USER" # --- static IP on the VM-to-VM segment ------------------------------------ # The multicast segment has no DHCP (the user-mode NIC serves the other diff --git a/vmbox/lib/base.sh b/vmbox/lib/base.sh index cc8668d..cf53c33 100644 --- a/vmbox/lib/base.sh +++ b/vmbox/lib/base.sh @@ -19,6 +19,19 @@ readonly CLOUD_URL="https://geo.mirror.pkgbuild.com/images/latest" readonly CLOUD_IMG="Arch-Linux-x86_64-cloudimg.qcow2" readonly BASE_DISK_SIZE="${BASE_DISK_SIZE:-20G}" readonly BUILD_SSH_PORT="${BUILD_SSH_PORT:-2222}" +# The host's own package cache, offered to the build guest read-only. +readonly HOST_PKG_CACHE="${VMBOX_HOST_PKG_CACHE:-/var/cache/pacman/pkg}" + +# Per-phase timing. "The build got faster" is only checkable against phase +# numbers: a single wall-clock figure hides which change did the work, and +# the download share in particular varies with what the host cache holds. +_phase_start=0 +phase_begin() { printf -v _phase_start '%(%s)T' -1; log "$1"; } +phase_end() { + local now + printf -v now '%(%s)T' -1 + ok "$1: $(( now - _phase_start ))s" +} base_fetch() { # Separate statements: within a single `local a=.. b="$a.."`, the later @@ -55,6 +68,27 @@ base_provision() { seed_ensure_key seed_build_iso "$seed" "$work" + # Offer the host's own pacman cache to the build guest, READ-ONLY. The + # host cache already holds every package this image installs, so this is + # the difference between fetching them over the internet and reading them + # off local disk. + # + # Read-only is enforced at BOTH layers -- `readonly=on` here and `ro` in + # the guest's mount -- because a guest that could write here would be + # writing into the real host system's package cache, which is the one + # thing this design must never allow. The guest still keeps its own + # writable /var/cache/pacman/pkg; this mount is only an extra CacheDir, + # so a package the host cache lacks is downloaded normally rather than + # failing the build. + local -a cache_args=() + if [[ -d "$HOST_PKG_CACHE" ]]; then + cache_args=(-virtfs + "local,path=$HOST_PKG_CACHE,mount_tag=pkgcache,security_model=none,readonly=on") + log "Sharing the host pacman cache read-only ($(du -sh "$HOST_PKG_CACHE" 2>/dev/null | cut -f1))" + else + warn "no host pacman cache at $HOST_PKG_CACHE -- packages will be downloaded" + fi + log "Booting image for provisioning" qemu-system-x86_64 \ -machine q35,accel=kvm -cpu host -smp 4 -m 4096 \ @@ -62,6 +96,7 @@ base_provision() { -drive file="$seed",if=virtio,format=raw,readonly=on \ -netdev "user,id=net0,hostfwd=tcp:127.0.0.1:${BUILD_SSH_PORT}-:22" \ -device virtio-net-pci,netdev=net0 \ + "${cache_args[@]}" \ -display none \ -chardev file,id=ser0,path="$serial",append=on -serial chardev:ser0 \ -no-reboot >/dev/null 2>&1 & @@ -73,12 +108,19 @@ base_provision() { # shellcheck disable=SC2064 # expanding NOW is intended: see comment above trap "kill $qpid 2>/dev/null || true; rm -rf '$work'" RETURN + phase_begin "Waiting for the provisioning guest to come up" _base_wait_ssh "$qpid" || die "guest never became reachable -- see $serial" + phase_end "PHASE first-boot" - log "Provisioning guest (installs Xorg/i3 + dev tools; several minutes)" + phase_begin "Provisioning guest (installs Xorg/i3 + dev tools)" + # provision.sh sources a sibling (provision-desktop.sh) by path, so BOTH + # must land in the same directory in the guest -- copying only the entry + # point produces a build that dies partway with "No such file or directory". _base_scp "$VMBOX_GUEST_DIR/provision.sh" "/tmp/provision.sh" - _base_ssh "chmod +x /tmp/provision.sh && sudo /tmp/provision.sh '$VMBOX_GUEST_USER'" || + _base_scp "$VMBOX_GUEST_DIR/provision-desktop.sh" "/tmp/provision-desktop.sh" + _base_ssh "chmod +x /tmp/provision.sh /tmp/provision-desktop.sh && sudo /tmp/provision.sh '$VMBOX_GUEST_USER'" || die "provisioning failed -- see $serial" + phase_end "PHASE provision" # Disable cloud-init so per-VM boots skip datasource probing entirely # (it adds startup delay and would re-run on every overlay). @@ -110,7 +152,7 @@ base_provision() { _base_ssh_opts() { printf '%s' "-i $VMBOX_SSH_KEY -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null \ --o LogLevel=ERROR -o ConnectTimeout=5 -p $BUILD_SSH_PORT" +-o LogLevel=ERROR -o ConnectTimeout=${BUILD_SSH_CONNECT_TIMEOUT:-5} -p $BUILD_SSH_PORT" } _base_ssh() { @@ -125,17 +167,31 @@ _base_scp() { # ssh is its own port prober, so no nc/socat needed (neither is installed). _base_wait_ssh() { - local qpid="$1" waited=0 + local qpid="$1" start now deadline log "Waiting for guest ssh (first boot runs cloud-init)" - while (( waited < 300 )); do + printf -v start '%(%s)T' -1 + deadline=$(( start + 300 )) + # Poll every 1s against a wall-clock deadline, not every 5s against a + # count of sleeps. The old loop over-reported by up to 5s of sleep plus + # the ConnectTimeout each failed probe burns inside ssh, so a guest that + # was reachable at ~22s was recorded as "reachable after 45s" -- and that + # inflated figure was then treated as cloud-init being slow. + while :; do kill -0 "$qpid" 2>/dev/null || return 1 - if _base_ssh -o BatchMode=yes true 2>/dev/null; then - ok "guest reachable after ${waited}s" + printf -v now '%(%s)T' -1 + (( now < deadline )) || return 1 + # Short connect timeout: QEMU's user-mode networking accepts the + # forwarded connection before the guest listens on :22, so a probe + # against a booting guest blocks for the WHOLE timeout rather than + # failing fast. It must go through the env var -- ssh honours the + # FIRST value given for an option, so a second -o here is ignored. + if BUILD_SSH_CONNECT_TIMEOUT=1 _base_ssh -o BatchMode=yes true 2>/dev/null; then + printf -v now '%(%s)T' -1 + ok "guest reachable after $(( now - start ))s" return 0 fi - sleep 5; waited=$(( waited + 5 )) + sleep 1 done - return 1 } # Make the base read-only and record its hash. The hash is the guard that diff --git a/vmbox/lib/run.sh b/vmbox/lib/run.sh index f3e2a16..e5257c3 100644 --- a/vmbox/lib/run.sh +++ b/vmbox/lib/run.sh @@ -24,7 +24,12 @@ readonly RC_PATH="/var/tmp/vmbox.rc" # on a missing file is a SPECIAL BUILTIN error that exits the shell outright, # and `|| true` does NOT rescue it. Written that way, every `vm run` against a # base image predating this snippet silently produced no output at all. -readonly GUEST_ENV_SETUP='[ -f /etc/profile.d/vmbox-x11.sh ] && . /etc/profile.d/vmbox-x11.sh;' +# Built at call time, not as a constant, so the host's VMBOX_X_WAIT is baked +# in by VALUE -- a single-quoted constant would ship the literal ${...} text. +_guest_env_setup() { + printf 'export VMBOX_X_WAIT=%s; [ -f /etc/profile.d/vmbox-x11.sh ] && . /etc/profile.d/vmbox-x11.sh;' \ + "${VMBOX_X_WAIT:-0}" +} # Transport for guest commands. Serial is the default for shutdown work: it # keeps printing through the poweroff that kills ssh, and it still works when # sshd does not. ssh stays available as the faster path for bulk work. @@ -64,7 +69,7 @@ run_in_vm() { local ssh_rc=0 _run_exec "$name" "$@" || ssh_rc=$? - _run_settle "$name" "$serial" || true + _run_settle "$name" "$serial" "$ssh_rc" || true local frc=0 _run_finish "$name" "$serial" "$events" "$events_before" "$ssh_rc" || frc=$? return "$frc" @@ -78,7 +83,7 @@ _run_exec() { case "$(_run_transport "$name")" in serial) _run_exec_serial "$name" "$cmd" ;; *) vm_ssh_exec "$name" \ - "sh -c '$GUEST_ENV_SETUP { $cmd; } ; echo \$? | sudo tee $RC_PATH >/dev/null' /dev/null' $RC_PATH" + "$(_guest_env_setup) { $cmd ; } ; echo \$? > $RC_PATH" } -# Give a guest that is shutting down time to finish writing its serial log. +# Give a guest that is SHUTTING DOWN time to finish writing its serial log. +# +# Only a guest that is actually stopping needs this. The first loop used to +# run unconditionally, so every ordinary `vm run` -- the overwhelming majority, +# which leave the guest running -- paid a flat 30s waiting for a poweroff +# marker that was never going to appear. That single wait was most of the +# wall-clock cost of running anything at all in a sandbox: 41s for a `true`, +# of which 31s was this. Measured before/after: 41s -> 11s. +# +# The ssh transport reports a dropped connection as 255, which is what a +# successful poweroff looks like from the host; a guest that is merely still +# running gives a real exit code. So: settle only when the guest has already +# stopped, or when the connection died in the way a shutdown would cause. _run_settle() { - local name="$1" serial="$2" waited=0 + local name="$1" serial="$2" ssh_rc="${3:-0}" waited=0 + if vm_is_running "$name" && (( ssh_rc != 255 )); then + return 0 + fi while vm_is_running "$name" && (( waited < 30 )); do grep -qa "$POWEROFF_MARKER" "$serial" 2>/dev/null && break sleep 1; waited=$(( waited + 1 )) diff --git a/vmbox/lib/seed.sh b/vmbox/lib/seed.sh index b922f0a..7390a56 100644 --- a/vmbox/lib/seed.sh +++ b/vmbox/lib/seed.sh @@ -49,6 +49,7 @@ chpasswd: ssh_pwauth: true preserve_hostname: false + UD cat > "$workdir/meta-data" </dev/null; then + # A 1s connect timeout (not the 5s default) is what makes this loop + # track the guest instead of lagging ~35s behind it. QEMU's user-mode + # networking ACCEPTS the forwarded TCP connection even while the guest + # has nothing listening on :22, so a probe against a booting guest does + # not fail fast -- it blocks for the whole ConnectTimeout. Measured: + # sshd is reachable at ~10s, but a 5s-timeout poll reported the guest + # up at ~45s, and that inflated figure was blamed on cloud-init. + # + # It goes through the env var because ssh honours the FIRST value given + # for an option, so a second -o appended here would be ignored -- and + # anything after the hostname is the remote command, not an option. + if vm_is_running "$name" && + VMBOX_SSH_CONNECT_TIMEOUT=1 vm_ssh_exec "$name" -o BatchMode=yes true 2>/dev/null; then return 0 fi # A qemu that has exited means the guest will never answer. if ! vm_is_running "$name"; then (( now - start > 3 )) && return 1 fi - sleep 2 + sleep 1 done }