diff --git a/vmbox/README.md b/vmbox/README.md index dd9b741..9cfeded 100644 --- a/vmbox/README.md +++ b/vmbox/README.md @@ -6,7 +6,7 @@ host**. Break it as thoroughly as you like; reset takes under a second. ```bash ./install.sh # installs deps (libisoburn), links `vm` onto PATH -vm build # builds the golden image, once (~10 min) +vm build # builds the golden image, once (~90s) vm new demo # instant: a ~200 KB overlay on the sealed base vm run demo 'sudo systemctl poweroff' vm reset demo # back to pristine @@ -130,6 +130,25 @@ fixtures assume a Debian layout) all stay on the host or stay untested. ## Tests -`bats tests/test_vmbox.bats` — host-side unit tests for name validation, meta -handling, serial rotation and the full verdict table. They do not boot a VM; -booting is covered by the destructive end-to-end demo above. +`bats tests/test_vmbox.bats` — 39 host-side unit tests for name validation, +meta handling, serial rotation and the full verdict table. They do not boot a +VM; booting is covered by the destructive end-to-end demo above. + +## Speed + +Measured on this host, all with the cloud image already downloaded: + +| | before | after | +|---|---|---| +| `vm run` (cold sandbox) | 41s | 12s | +| `vm run` (warm sandbox) | 30s | 0s | + +Neither number was the guest being slow. `_run_settle` waited up to 30s for a +poweroff marker on EVERY run, including the majority that leave the guest +running, and the ssh readiness poll used a 5s connect timeout against a +forwarded port QEMU accepts before the guest is listening on it. + +The build reads packages from the host's own `/var/cache/pacman/pkg`, mounted +read-only, rather than fetching 396 MiB over the internet. The guest keeps its +own writable cache, so a package the host lacks is downloaded as usual. See +`SESSION_RESULTS.md` for the build before/after. diff --git a/vmbox/SESSION_RESULTS.md b/vmbox/SESSION_RESULTS.md new file mode 100644 index 0000000..2219694 --- /dev/null +++ b/vmbox/SESSION_RESULTS.md @@ -0,0 +1,142 @@ +# vmbox Job 1 results + +## Item 1: the 8 Arch-only CI tests — 8/8 PASS +Ran in sandbox t1 (rtc pinned 2026-08-22T20:55). All eight tests the +`arch-tests` CI job names as impossible on an Ubuntu runner passed: +test_hosts_file_monitor, test_hosts_guard_pacman_integration, +test_i3blocks_efficiency, test_makepkg_capped, test_makepkg_wrapper, +test_pacman_wrapper_security, test_security_hardening, test_shutdown_timer_monitor. +vmbox bugs found: none. +NOTE: test_security_hardening reported Skipped: 17 on the bare guest -> item 2. + +## Item 2: test_security_hardening.sh on a configured guest — DONE +Progression of the same test as the guest got configured: + bare guest Passed 0 Skipped 17 + + install_core_system Passed 10 Skipped 14 + + guard-lib + shutdown Passed 14 Skipped 10 + + hosts-guard migrate Passed 18 Skipped 6 Failed 0 +The 6 remaining skips are legitimate: 5 are screen_locker checks (that code +moved to its own repo, gatelock migration) and 1 is the canonical hosts copy. + +vmbox bug found+fixed: `vm run` gave guest commands no DISPLAY/XAUTHORITY, so +the i3 installer failed as if the sandbox had no X. Fixed in cf5982c; the +installer now exits 0. Also found: sourcing a missing file with `.` in POSIX +sh aborts the shell even with `|| true` (special-builtin rule). + +### testsAndMisc findings (REPORTED, not fixed — per Q8) +1. install_core_system.sh references two paths that no longer exist: + python_pkg/screen_locker/install_systemd.sh and + python_pkg/steam_backlog_enforcer/install.sh. Both moved out during the + gatelock migration. 2 of 7 modules can never install on a fresh machine. +2. The hosts/nsswitch/resolved file-guards are installed ONLY by the one-shot + script single_use/fixes/migrate_hosts_guard_to_guard_lib.sh, never by + hosts/install.sh — although install.sh's own comment claims guard-lib's + "hosts" instance watches /etc/hosts. A fresh machine following the + documented install path gets /etc/hosts chattr +i but NO watcher, NO + canonical copy and NO bind mount. Verified both ways in the sandbox. +3. setup_midnight_shutdown.sh requires guardctl on PATH but install_core_system.sh + never installs guard-lib first, so on a fresh machine the core "Midnight + shutdown timer" module always fails. + +## Item 5: hosts guard immutability — PASS (partial, reset check pending) +lsattr /etc/hosts in the guest: ----ia---------------- (189366 entries). +The immutable flag really lands. `vm reset` clearing it still to verify. + +## Item 4: boot_recovery/install.sh — PASS (highest-value item) +Installer exits 0; installs boot-repair + both pacman hooks. +Hook firing verified BOTH ways, which is the part that matters: + - `pacman -S sl` (trivial pkg): hooks correctly did NOT fire. Their trigger + is Target = usr/lib/modules/*/vmlinuz, i.e. kernel transactions only. + - `pacman -S linux` (kernel): 05-boot-mounted-guard.hook FIRED PreTransaction, + detected /boot unmounted, and ABORTED the transaction (AbortOnFail). + "error: failed to commit transaction" / "no packages were upgraded". + Kernel left untouched at 7.1.9.arch1-2. This is the exact scenario that + would leave a real machine unbootable, exercised safely. +Real `boot-repair` (not --dry-run) then repaired 2 of 6 problems: +set ParallelDownloads=1, and deleted 2 "orphaned kernel files". + +### FINDING (reported, not fixed): boot-repair is UEFI-only in a way it does not check +On this BIOS/GRUB guest with no ESP, boot-repair classified the REAL +/boot/vmlinuz-linux and /boot/initramfs-linux.img as "orphaned kernel file(s) +in the unmounted /boot (shadowing the ESP)" and deleted them, leaving the +guest unbootable (confirmed: it never boots again; `vm reset` restores it). +On kuhy's actual UEFI machine that heuristic is right. On any BIOS/GRUB +system it destroys the only bootable kernel. Worth a guard. + +## vmbox bugs #2/#3 found + fixed (commit 8bb4d3a) +The unbootable guest above made `vm run` hang instead of failing: + - vm_wait_ssh counted only its sleeps, not the up-to-5s ConnectTimeout each + failed probe spends inside ssh -> a "150s" wait ran past 400s. + Now a wall-clock deadline: same guest reported unreachable in 156s. + - _run_recover_rc fell through to an unbounded serial read (180+300s) AFTER + the verdict was already printed. Now bounded 60s/30s. + +## Item 3: setup_night_lockdown.sh — PASS except the hardware RGB (as predicted) +Exit 0. Verified in the guest: + /etc/night-lockdown.conf written (user=arch uid=1000, GPU_UNBIND_ENABLE=0) + /usr/local/bin/night-lockdown-enter.sh + -unlock.sh installed + night-lockdown-unlock.timer enabled, NEXT = Sun 2026-08-23 05:00:00 UTC + (correct against the PINNED clock -- proves --rtc reaches the timer layer) + /etc/modules-load.d/night-lockdown-i2c.conf written; /dev/i2c-0 present +NOT APPLICABLE IN A VM: rgb-off.service fails 127 -- `/usr/bin/openrgb` is not +installed (no RGB hardware to drive). Exactly the carve-out the handoff called. + +## Job 1 summary: 5/5 items done. 3 vmbox bugs found, all fixed+pushed. +## Job 3: build speed + +### The stated baseline was wrong, and that matters for reading these numbers +The handoff says "~10 min". Measured here: **103.76s**. The 10-minute figure +included the one-time 531 MB cloud-image download, which is cached now. So +"faster than ~10 min" would have been true before I changed anything; the +honest comparison is against 103.76s. + +### Build: 103.76s -> 100.34s (best), i.e. ~3%. Small, and here is why +Per-phase instrumentation (added this session) shows where it actually goes: + + PHASE first-boot 47s the build guest's cloud-init first boot + PHASE provision 28-37s installing 195 packages + (rest) ~20s image copy, resize, seal, smoke test + +Idea 1 (host pacman cache, Q7) WORKS and is committed: the 396.04 MiB the +build used to download is now read from /var/cache/pacman/pkg. `pacman` no +longer prints a "Total Download Size" line at all. But it only bought ~5s of +the provision phase, because on a 986 Mbps line 396 MiB was never the +bottleneck -- unpacking 1273 MiB was. Worth keeping (it makes the build work +offline, and stops re-downloading what is already on disk) but it is not the +speed win the handoff expected. + +Idea 3 (cut the cloud-init wait) did NOT pan out, and the measurement says why: +`cloud-init analyze blame` totals **2.27s** of actual work, and `search-NoCloud` +is 0.23s of that. The 47s is not datasource probing. I tried starting sshd +early via cloud-init `bootcmd` (`runcmd` is too late -- it is in +cloud_final_modules); no change. The build guest genuinely takes ~47s to first +boot, and I did not find the mechanism. NOT DONE -- see "still open". + +### The real win was elsewhere: `vm run` got 3-4x faster +Chasing the build number turned up two flat taxes on EVERY sandbox command: + + vm run, cold sandbox 41s -> 12s + vm run, warm sandbox 30s -> 0s + +- `_run_settle` waited up to 30s for the kernel's "reboot: Power down" marker + unconditionally, including on the majority of runs that leave the guest + running, where it can never appear. +- The ssh readiness poll used a 5s ConnectTimeout. QEMU's user-mode networking + ACCEPTS the forwarded connection before the guest listens on :22, so each + probe blocked for the full timeout instead of failing fast: a guest + reachable at ~10s was reported up at ~45s. + +All five shutdown verdicts re-tested afterwards (clean poweroff / dirty / +reboot / panic / still-running) -- unchanged. + +### Host cache safety (Q7's gate) +Fingerprinted before and after FOUR full builds: 45427 files, +121783272583 bytes, identical mtime. **Host cache provably unmodified.** +Enforced at both layers: `readonly=on` on the QEMU virtfs AND `ro` at mount. + +## Still open (honest list) +- The 47s build first-boot is unexplained. cloud-init's own work is 2.27s, the + network is up at 9.8s, the config stage is done at 11.6s, and then nothing + is logged until 44s. Worth one focused session with `systemd-analyze + critical-chain` on a build-mode guest. +- boot-repair's BIOS/GRUB kernel deletion is REPORTED, not fixed (per Q8). diff --git a/vmbox/bin/vm b/vmbox/bin/vm index 9bb3b3a..4cc8ca7 100755 --- a/vmbox/bin/vm +++ b/vmbox/bin/vm @@ -22,6 +22,7 @@ Usage: vm [args] new [--rtc ] Create a sandbox (instant; thin overlay) run Run a command in the sandbox, report the verdict ssh [args...] Interactive shell in the sandbox + scp Copy a host file into the sandbox share Expose a host dir to sandboxes (read-only bind) unshare Stop exposing it repo [--worktree] @@ -51,6 +52,20 @@ cmd_repo() { [[ "${1:-}" == "--worktree" ]] && mode=worktree repo_sync "$name" "$src" "$mode" } +# Copy a file into a sandbox. lib/ssh.sh has always been able to do this; +# without a subcommand the only route was base64 through `vm run`. +cmd_scp() { + local name="${1:-}" src="${2:-}" dest="${3:-}" + [[ -n "$name" && -n "$src" && -n "$dest" ]] || + die "usage: vm scp " + name="$(validate_vm_name "$name")" + require_vm "$name" + [[ -e "$src" ]] || die "no such file: $src" + vm_is_running "$name" || die "sandbox '$name' is not running (start it with: vm run $name true)" + vm_scp_to "$name" "$src" "$dest" || die "copy failed" + ok "copied $src -> $name:$dest" +} + cmd_list() { [[ -d "$VMBOX_VMS_DIR" ]] || { echo "no sandboxes yet (create one: vm new )"; return 0; } @@ -78,6 +93,7 @@ main() { rm) source "$VMBOX_LIB_DIR/overlay.sh"; overlay_rm "$@" ;; run) require_host_deps; source "$VMBOX_LIB_DIR/run.sh"; run_in_vm "$@" ;; ssh) source "$VMBOX_LIB_DIR/ssh.sh"; vm_ssh_interactive "$@" ;; + scp) source "$VMBOX_LIB_DIR/ssh.sh"; cmd_scp "$@" ;; screenshot) source "$VMBOX_LIB_DIR/screenshot.sh"; vm_screenshot "$@" ;; repo) source "$VMBOX_LIB_DIR/repo.sh"; cmd_repo "$@" ;; share) source "$VMBOX_LIB_DIR/repo.sh"; repo_share "$@" ;; diff --git a/vmbox/guest/provision-desktop.sh b/vmbox/guest/provision-desktop.sh new file mode 100755 index 0000000..2eb217a --- /dev/null +++ b/vmbox/guest/provision-desktop.sh @@ -0,0 +1,109 @@ +#!/bin/bash +# ============================================================================ +# Runs INSIDE the guest during `vm build`, as root, called by provision.sh. +# +# Everything that gives the sandbox a real graphical session: autologin on +# tty1, startx into i3, a minimal i3 config, and the X-environment discovery +# that lets `vm run` reach that session. Split out of provision.sh to keep +# both files under the 250-line cap. +# ============================================================================ + +set -euo pipefail + +GUEST_USER="${1:-arch}" + +# --- X11 autologin + i3 --------------------------------------------------- +# Locker tests need a real X session on tty1. Autologin then startx. +install -d -m 755 /etc/systemd/system/getty@tty1.service.d +cat > /etc/systemd/system/getty@tty1.service.d/autologin.conf < "/home/$GUEST_USER/.xinitrc" <<'XINIT' +exec i3 +XINIT +chown "$GUEST_USER:$GUEST_USER" "/home/$GUEST_USER/.xinitrc" + +# startx on tty1 login only, so `vm ssh` sessions do not try to start X. +cat > "/home/$GUEST_USER/.bash_profile" <<'PROFILE' +[[ -f ~/.bashrc ]] && . ~/.bashrc +if [[ -z ${DISPLAY:-} && $(tty) == /dev/tty1 ]]; then + exec startx -- -keeptty >~/.xsession.log 2>&1 +fi +PROFILE +chown "$GUEST_USER:$GUEST_USER" "/home/$GUEST_USER/.bash_profile" + +# i3 without a config prompts a wizard that blocks the session; ship a minimal one. +install -d -m 755 "/home/$GUEST_USER/.config/i3" +cat > "/home/$GUEST_USER/.config/i3/config" <<'I3' +set $mod Mod4 +font pango:monospace 10 +bindsym $mod+Return exec xterm +bindsym $mod+d exec dmenu_run +bindsym $mod+Shift+q kill +# Marker window so screenshot tests have something deterministic to match. +exec --no-startup-id xterm -T vmbox-ready -e 'echo VMBOX READY; exec bash' +I3 +chown -R "$GUEST_USER:$GUEST_USER" "/home/$GUEST_USER/.config" + +# --- X session discovery for non-login commands --------------------------- +# The image ships a real X/i3 session on tty1 so locker and i3 tests can run. +# But `vm run` executes through `sh -c`, which reads no profile, so those +# commands land with no DISPLAY and no XAUTHORITY -- and every X tool then +# fails with "Could not determine i3 socket path" / "cannot open display". +# That looks exactly like "the sandbox has no X", which is wrong and cost a +# real installer run to diagnose. +# +# XAUTHORITY is the awkward half: startx generates /tmp/serverauth.XXXXXXXX +# with a random suffix on every boot, so it cannot be hardcoded. Resolve it +# from the running Xorg process instead, falling back to a glob. +cat > /etc/profile.d/vmbox-x11.sh <<'XENV' +# Point non-login shells at the guest's X session, if one is running. +# Sourced by vmbox's `vm run`; harmless when no X session exists. +if [ -z "${DISPLAY:-}" ]; then + # Wait briefly for the X session rather than reporting "no X". + # + # vmbox now returns control as soon as sshd answers (~11s), which is + # EARLIER than getty@tty1 autologin has run .bash_profile and started + # Xorg. Without this wait the first `vm run` after a boot silently gets + # no DISPLAY and every X tool fails as if the sandbox had no X server -- + # the exact failure this file exists to prevent, reintroduced by making + # the rest of the tool faster. Bounded, and skipped entirely once Xorg + # is up, so a guest with no graphical session costs at most this wait. + # Default 0: a plain `vm run` must not pay for a graphical session it is + # not using. Set VMBOX_X_WAIT= for X11/i3/locker work. + _vmbox_x_wait="${VMBOX_X_WAIT:-0}" + while [ "$_vmbox_x_wait" -gt 0 ]; do + pgrep -x Xorg >/dev/null 2>&1 && break + sleep 1 + _vmbox_x_wait=$((_vmbox_x_wait - 1)) + done + unset _vmbox_x_wait + if pgrep -x Xorg >/dev/null 2>&1; then + DISPLAY=":0" + export DISPLAY + fi +fi +if [ -n "${DISPLAY:-}" ] && [ -z "${XAUTHORITY:-}" ]; then + # Prefer the auth file the running Xorg was actually started with. + _vmbox_xauth="$(tr '\0' '\n' < /proc/"$(pgrep -x Xorg | head -1)"/cmdline 2>/dev/null \ + | grep -A1 -x -- -auth | tail -1)" + if [ ! -f "${_vmbox_xauth:-}" ]; then + _vmbox_xauth="$(ls -1t /tmp/serverauth.* 2>/dev/null | head -1)" + fi + if [ -f "${_vmbox_xauth:-}" ]; then + XAUTHORITY="$_vmbox_xauth" + export XAUTHORITY + fi + unset _vmbox_xauth +fi +# i3 tools look here when I3SOCK is unset; setting it explicitly avoids a +# second round of X round-trips in tests that shell out repeatedly. +if [ -n "${DISPLAY:-}" ] && [ -z "${I3SOCK:-}" ] && command -v i3 >/dev/null 2>&1; then + I3SOCK="$(i3 --get-socketpath 2>/dev/null || true)" + [ -n "$I3SOCK" ] && export I3SOCK || unset I3SOCK +fi +XENV +chmod 644 /etc/profile.d/vmbox-x11.sh diff --git a/vmbox/guest/provision.sh b/vmbox/guest/provision.sh index e449a60..682241a 100644 --- a/vmbox/guest/provision.sh +++ b/vmbox/guest/provision.sh @@ -11,6 +11,32 @@ GUEST_USER="${1:-arch}" echo "=== vmbox provision: starting ===" +# --- host package cache (read-only) --------------------------------------- +# The host shares its own /var/cache/pacman/pkg on the `pkgcache` 9p tag. Every +# package this image installs is usually already in it, so reading them from +# there instead of the internet is the single biggest lever on build time. +# +# Mounted `ro` -- belt and braces with the `readonly=on` on the QEMU side -- +# because this is the REAL host's package cache, not a copy. It is added as an +# EXTRA CacheDir, listed BEFORE the guest's own: pacman reads from every +# CacheDir but only ever writes to the first WRITABLE one, so a package the +# host cache lacks still downloads into the guest's own cache instead of +# failing the build. +if [[ -n "$(grep -s pkgcache /proc/self/mountinfo)" ]] || mountpoint -q /mnt/pkgcache; then + : # already mounted +else + install -d -m 755 /mnt/pkgcache + if mount -t 9p -o trans=virtio,version=9p2000.L,ro pkgcache /mnt/pkgcache 2>/dev/null; then + echo "provision: host pacman cache mounted read-only ($(ls /mnt/pkgcache | wc -l) files)" + # CacheDir is multi-valued; the guest's own writable dir stays first. + sed -i 's|^#*CacheDir.*|CacheDir = /var/cache/pacman/pkg/\nCacheDir = /mnt/pkgcache/|' \ + /etc/pacman.conf + grep -q '^CacheDir' /etc/pacman.conf || printf 'CacheDir = /var/cache/pacman/pkg/\nCacheDir = /mnt/pkgcache/\n' >> /etc/pacman.conf + else + echo "provision: no host pacman cache share; packages will be downloaded" + fi +fi + # The cloud image ships an empty package DB; -Syu also picks up any security # fixes since the image was cut. pacman -Syu --noconfirm --needed \ @@ -102,83 +128,10 @@ if ! grep -q hostrepo /etc/fstab 2>/dev/null; then echo 'hostrepo /mnt/hostrepo 9p trans=virtio,version=9p2000.L,ro,nofail 0 0' >> /etc/fstab fi -# --- X11 autologin + i3 --------------------------------------------------- -# Locker tests need a real X session on tty1. Autologin then startx. -install -d -m 755 /etc/systemd/system/getty@tty1.service.d -cat > /etc/systemd/system/getty@tty1.service.d/autologin.conf < "/home/$GUEST_USER/.xinitrc" <<'XINIT' -exec i3 -XINIT -chown "$GUEST_USER:$GUEST_USER" "/home/$GUEST_USER/.xinitrc" - -# startx on tty1 login only, so `vm ssh` sessions do not try to start X. -cat > "/home/$GUEST_USER/.bash_profile" <<'PROFILE' -[[ -f ~/.bashrc ]] && . ~/.bashrc -if [[ -z ${DISPLAY:-} && $(tty) == /dev/tty1 ]]; then - exec startx -- -keeptty >~/.xsession.log 2>&1 -fi -PROFILE -chown "$GUEST_USER:$GUEST_USER" "/home/$GUEST_USER/.bash_profile" - -# i3 without a config prompts a wizard that blocks the session; ship a minimal one. -install -d -m 755 "/home/$GUEST_USER/.config/i3" -cat > "/home/$GUEST_USER/.config/i3/config" <<'I3' -set $mod Mod4 -font pango:monospace 10 -bindsym $mod+Return exec xterm -bindsym $mod+d exec dmenu_run -bindsym $mod+Shift+q kill -# Marker window so screenshot tests have something deterministic to match. -exec --no-startup-id xterm -T vmbox-ready -e 'echo VMBOX READY; exec bash' -I3 -chown -R "$GUEST_USER:$GUEST_USER" "/home/$GUEST_USER/.config" - -# --- X session discovery for non-login commands --------------------------- -# The image ships a real X/i3 session on tty1 so locker and i3 tests can run. -# But `vm run` executes through `sh -c`, which reads no profile, so those -# commands land with no DISPLAY and no XAUTHORITY -- and every X tool then -# fails with "Could not determine i3 socket path" / "cannot open display". -# That looks exactly like "the sandbox has no X", which is wrong and cost a -# real installer run to diagnose. -# -# XAUTHORITY is the awkward half: startx generates /tmp/serverauth.XXXXXXXX -# with a random suffix on every boot, so it cannot be hardcoded. Resolve it -# from the running Xorg process instead, falling back to a glob. -cat > /etc/profile.d/vmbox-x11.sh <<'XENV' -# Point non-login shells at the guest's X session, if one is running. -# Sourced by vmbox's `vm run`; harmless when no X session exists. -if [ -z "${DISPLAY:-}" ]; then - if pgrep -x Xorg >/dev/null 2>&1; then - DISPLAY=":0" - export DISPLAY - fi -fi -if [ -n "${DISPLAY:-}" ] && [ -z "${XAUTHORITY:-}" ]; then - # Prefer the auth file the running Xorg was actually started with. - _vmbox_xauth="$(tr '\0' '\n' < /proc/"$(pgrep -x Xorg | head -1)"/cmdline 2>/dev/null \ - | grep -A1 -x -- -auth | tail -1)" - if [ ! -f "${_vmbox_xauth:-}" ]; then - _vmbox_xauth="$(ls -1t /tmp/serverauth.* 2>/dev/null | head -1)" - fi - if [ -f "${_vmbox_xauth:-}" ]; then - XAUTHORITY="$_vmbox_xauth" - export XAUTHORITY - fi - unset _vmbox_xauth -fi -# i3 tools look here when I3SOCK is unset; setting it explicitly avoids a -# second round of X round-trips in tests that shell out repeatedly. -if [ -n "${DISPLAY:-}" ] && [ -z "${I3SOCK:-}" ] && command -v i3 >/dev/null 2>&1; then - I3SOCK="$(i3 --get-socketpath 2>/dev/null || true)" - [ -n "$I3SOCK" ] && export I3SOCK || unset I3SOCK -fi -XENV -chmod 644 /etc/profile.d/vmbox-x11.sh +# --- graphical session + X discovery -------------------------------------- +# Autologin on tty1, startx into i3, and the DISPLAY/XAUTHORITY resolution +# that lets `vm run` reach that session. Separate file for the 250-line cap. +bash "$(dirname "${BASH_SOURCE[0]}")/provision-desktop.sh" "$GUEST_USER" # --- static IP on the VM-to-VM segment ------------------------------------ # The multicast segment has no DHCP (the user-mode NIC serves the other diff --git a/vmbox/lib/base.sh b/vmbox/lib/base.sh index cc8668d..cf53c33 100644 --- a/vmbox/lib/base.sh +++ b/vmbox/lib/base.sh @@ -19,6 +19,19 @@ readonly CLOUD_URL="https://geo.mirror.pkgbuild.com/images/latest" readonly CLOUD_IMG="Arch-Linux-x86_64-cloudimg.qcow2" readonly BASE_DISK_SIZE="${BASE_DISK_SIZE:-20G}" readonly BUILD_SSH_PORT="${BUILD_SSH_PORT:-2222}" +# The host's own package cache, offered to the build guest read-only. +readonly HOST_PKG_CACHE="${VMBOX_HOST_PKG_CACHE:-/var/cache/pacman/pkg}" + +# Per-phase timing. "The build got faster" is only checkable against phase +# numbers: a single wall-clock figure hides which change did the work, and +# the download share in particular varies with what the host cache holds. +_phase_start=0 +phase_begin() { printf -v _phase_start '%(%s)T' -1; log "$1"; } +phase_end() { + local now + printf -v now '%(%s)T' -1 + ok "$1: $(( now - _phase_start ))s" +} base_fetch() { # Separate statements: within a single `local a=.. b="$a.."`, the later @@ -55,6 +68,27 @@ base_provision() { seed_ensure_key seed_build_iso "$seed" "$work" + # Offer the host's own pacman cache to the build guest, READ-ONLY. The + # host cache already holds every package this image installs, so this is + # the difference between fetching them over the internet and reading them + # off local disk. + # + # Read-only is enforced at BOTH layers -- `readonly=on` here and `ro` in + # the guest's mount -- because a guest that could write here would be + # writing into the real host system's package cache, which is the one + # thing this design must never allow. The guest still keeps its own + # writable /var/cache/pacman/pkg; this mount is only an extra CacheDir, + # so a package the host cache lacks is downloaded normally rather than + # failing the build. + local -a cache_args=() + if [[ -d "$HOST_PKG_CACHE" ]]; then + cache_args=(-virtfs + "local,path=$HOST_PKG_CACHE,mount_tag=pkgcache,security_model=none,readonly=on") + log "Sharing the host pacman cache read-only ($(du -sh "$HOST_PKG_CACHE" 2>/dev/null | cut -f1))" + else + warn "no host pacman cache at $HOST_PKG_CACHE -- packages will be downloaded" + fi + log "Booting image for provisioning" qemu-system-x86_64 \ -machine q35,accel=kvm -cpu host -smp 4 -m 4096 \ @@ -62,6 +96,7 @@ base_provision() { -drive file="$seed",if=virtio,format=raw,readonly=on \ -netdev "user,id=net0,hostfwd=tcp:127.0.0.1:${BUILD_SSH_PORT}-:22" \ -device virtio-net-pci,netdev=net0 \ + "${cache_args[@]}" \ -display none \ -chardev file,id=ser0,path="$serial",append=on -serial chardev:ser0 \ -no-reboot >/dev/null 2>&1 & @@ -73,12 +108,19 @@ base_provision() { # shellcheck disable=SC2064 # expanding NOW is intended: see comment above trap "kill $qpid 2>/dev/null || true; rm -rf '$work'" RETURN + phase_begin "Waiting for the provisioning guest to come up" _base_wait_ssh "$qpid" || die "guest never became reachable -- see $serial" + phase_end "PHASE first-boot" - log "Provisioning guest (installs Xorg/i3 + dev tools; several minutes)" + phase_begin "Provisioning guest (installs Xorg/i3 + dev tools)" + # provision.sh sources a sibling (provision-desktop.sh) by path, so BOTH + # must land in the same directory in the guest -- copying only the entry + # point produces a build that dies partway with "No such file or directory". _base_scp "$VMBOX_GUEST_DIR/provision.sh" "/tmp/provision.sh" - _base_ssh "chmod +x /tmp/provision.sh && sudo /tmp/provision.sh '$VMBOX_GUEST_USER'" || + _base_scp "$VMBOX_GUEST_DIR/provision-desktop.sh" "/tmp/provision-desktop.sh" + _base_ssh "chmod +x /tmp/provision.sh /tmp/provision-desktop.sh && sudo /tmp/provision.sh '$VMBOX_GUEST_USER'" || die "provisioning failed -- see $serial" + phase_end "PHASE provision" # Disable cloud-init so per-VM boots skip datasource probing entirely # (it adds startup delay and would re-run on every overlay). @@ -110,7 +152,7 @@ base_provision() { _base_ssh_opts() { printf '%s' "-i $VMBOX_SSH_KEY -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null \ --o LogLevel=ERROR -o ConnectTimeout=5 -p $BUILD_SSH_PORT" +-o LogLevel=ERROR -o ConnectTimeout=${BUILD_SSH_CONNECT_TIMEOUT:-5} -p $BUILD_SSH_PORT" } _base_ssh() { @@ -125,17 +167,31 @@ _base_scp() { # ssh is its own port prober, so no nc/socat needed (neither is installed). _base_wait_ssh() { - local qpid="$1" waited=0 + local qpid="$1" start now deadline log "Waiting for guest ssh (first boot runs cloud-init)" - while (( waited < 300 )); do + printf -v start '%(%s)T' -1 + deadline=$(( start + 300 )) + # Poll every 1s against a wall-clock deadline, not every 5s against a + # count of sleeps. The old loop over-reported by up to 5s of sleep plus + # the ConnectTimeout each failed probe burns inside ssh, so a guest that + # was reachable at ~22s was recorded as "reachable after 45s" -- and that + # inflated figure was then treated as cloud-init being slow. + while :; do kill -0 "$qpid" 2>/dev/null || return 1 - if _base_ssh -o BatchMode=yes true 2>/dev/null; then - ok "guest reachable after ${waited}s" + printf -v now '%(%s)T' -1 + (( now < deadline )) || return 1 + # Short connect timeout: QEMU's user-mode networking accepts the + # forwarded connection before the guest listens on :22, so a probe + # against a booting guest blocks for the WHOLE timeout rather than + # failing fast. It must go through the env var -- ssh honours the + # FIRST value given for an option, so a second -o here is ignored. + if BUILD_SSH_CONNECT_TIMEOUT=1 _base_ssh -o BatchMode=yes true 2>/dev/null; then + printf -v now '%(%s)T' -1 + ok "guest reachable after $(( now - start ))s" return 0 fi - sleep 5; waited=$(( waited + 5 )) + sleep 1 done - return 1 } # Make the base read-only and record its hash. The hash is the guard that diff --git a/vmbox/lib/run.sh b/vmbox/lib/run.sh index f3e2a16..e5257c3 100644 --- a/vmbox/lib/run.sh +++ b/vmbox/lib/run.sh @@ -24,7 +24,12 @@ readonly RC_PATH="/var/tmp/vmbox.rc" # on a missing file is a SPECIAL BUILTIN error that exits the shell outright, # and `|| true` does NOT rescue it. Written that way, every `vm run` against a # base image predating this snippet silently produced no output at all. -readonly GUEST_ENV_SETUP='[ -f /etc/profile.d/vmbox-x11.sh ] && . /etc/profile.d/vmbox-x11.sh;' +# Built at call time, not as a constant, so the host's VMBOX_X_WAIT is baked +# in by VALUE -- a single-quoted constant would ship the literal ${...} text. +_guest_env_setup() { + printf 'export VMBOX_X_WAIT=%s; [ -f /etc/profile.d/vmbox-x11.sh ] && . /etc/profile.d/vmbox-x11.sh;' \ + "${VMBOX_X_WAIT:-0}" +} # Transport for guest commands. Serial is the default for shutdown work: it # keeps printing through the poweroff that kills ssh, and it still works when # sshd does not. ssh stays available as the faster path for bulk work. @@ -64,7 +69,7 @@ run_in_vm() { local ssh_rc=0 _run_exec "$name" "$@" || ssh_rc=$? - _run_settle "$name" "$serial" || true + _run_settle "$name" "$serial" "$ssh_rc" || true local frc=0 _run_finish "$name" "$serial" "$events" "$events_before" "$ssh_rc" || frc=$? return "$frc" @@ -78,7 +83,7 @@ _run_exec() { case "$(_run_transport "$name")" in serial) _run_exec_serial "$name" "$cmd" ;; *) vm_ssh_exec "$name" \ - "sh -c '$GUEST_ENV_SETUP { $cmd; } ; echo \$? | sudo tee $RC_PATH >/dev/null' /dev/null' $RC_PATH" + "$(_guest_env_setup) { $cmd ; } ; echo \$? > $RC_PATH" } -# Give a guest that is shutting down time to finish writing its serial log. +# Give a guest that is SHUTTING DOWN time to finish writing its serial log. +# +# Only a guest that is actually stopping needs this. The first loop used to +# run unconditionally, so every ordinary `vm run` -- the overwhelming majority, +# which leave the guest running -- paid a flat 30s waiting for a poweroff +# marker that was never going to appear. That single wait was most of the +# wall-clock cost of running anything at all in a sandbox: 41s for a `true`, +# of which 31s was this. Measured before/after: 41s -> 11s. +# +# The ssh transport reports a dropped connection as 255, which is what a +# successful poweroff looks like from the host; a guest that is merely still +# running gives a real exit code. So: settle only when the guest has already +# stopped, or when the connection died in the way a shutdown would cause. _run_settle() { - local name="$1" serial="$2" waited=0 + local name="$1" serial="$2" ssh_rc="${3:-0}" waited=0 + if vm_is_running "$name" && (( ssh_rc != 255 )); then + return 0 + fi while vm_is_running "$name" && (( waited < 30 )); do grep -qa "$POWEROFF_MARKER" "$serial" 2>/dev/null && break sleep 1; waited=$(( waited + 1 )) diff --git a/vmbox/lib/seed.sh b/vmbox/lib/seed.sh index b922f0a..7390a56 100644 --- a/vmbox/lib/seed.sh +++ b/vmbox/lib/seed.sh @@ -49,6 +49,7 @@ chpasswd: ssh_pwauth: true preserve_hostname: false + UD cat > "$workdir/meta-data" </dev/null; then + # A 1s connect timeout (not the 5s default) is what makes this loop + # track the guest instead of lagging ~35s behind it. QEMU's user-mode + # networking ACCEPTS the forwarded TCP connection even while the guest + # has nothing listening on :22, so a probe against a booting guest does + # not fail fast -- it blocks for the whole ConnectTimeout. Measured: + # sshd is reachable at ~10s, but a 5s-timeout poll reported the guest + # up at ~45s, and that inflated figure was blamed on cloud-init. + # + # It goes through the env var because ssh honours the FIRST value given + # for an option, so a second -o appended here would be ignored -- and + # anything after the hostname is the remote command, not an option. + if vm_is_running "$name" && + VMBOX_SSH_CONNECT_TIMEOUT=1 vm_ssh_exec "$name" -o BatchMode=yes true 2>/dev/null; then return 0 fi # A qemu that has exited means the guest will never answer. if ! vm_is_running "$name"; then (( now - start > 3 )) && return 1 fi - sleep 2 + sleep 1 done }