kuhyx/utils
1
0
Fork 0
mirror of https://github.com/kuhyx/utils.git synced 2026-11-10 19:07:17 +01:00
Code Issues Actions 14 Packages Projects Releases Wiki Activity

vmbox: 3-4x faster runs, a read-only host package cache, and vm scp

Browse Source 
`vm run` took 41s cold / 30s warm to run `true`. Both were vmbox waiting on
itself. Now 12s / 0s, with all five shutdown verdicts re-tested unchanged.

- _run_settle waited up to 30s for the kernel's "reboot: Power down" marker
  UNCONDITIONALLY, including on the majority of runs that leave the guest up,
  where that marker is never going to appear. It now settles only when the
  guest has actually stopped, or when ssh returned 255 (what a successful
  poweroff looks like from the host).

- The ssh readiness polls used a 5s ConnectTimeout. QEMU's user-mode
  networking ACCEPTS the forwarded connection while the guest still has
  nothing on :22, so each probe blocked for the full timeout instead of
  failing fast: a guest reachable at ~10s was reported up at ~45s. Fixed in
  both the sandbox and build paths, via an env override -- ssh honours the
  FIRST value for an option, so a second -o could not have worked.

Making runs fast exposed a race: sshd now answers BEFORE tty1's autologin has
started Xorg, so the X environment could be missing on the first run after a
boot. `VMBOX_X_WAIT=<seconds>` waits for the session; it defaults to 0 so an
ordinary run never pays for a graphical session it is not using.

Build: the host's /var/cache/pacman/pkg is offered to the build guest
read-only (`readonly=on` AND an `ro` mount, because it is the real host cache)
as an extra CacheDir after the guest's own writable one, so a package the host
lacks still downloads rather than failing the build. This removes the 396 MiB
the build used to fetch. Host cache verified byte-for-byte identical across
four builds: same file count, same size, same mtime.

Build time itself moved only 103.8s -> 100.3s. Per-phase timings (also added
here) show why: 47s of it is the build guest's cloud-init first boot, and that
is NOT datasource probing -- `cloud-init analyze blame` totals 2.27s. Left
unexplained rather than guessed at; SESSION_RESULTS.md records the evidence.

Also: `vm scp`, which lib/ssh.sh could always do but the CLI never exposed,
and provision.sh split at the 250-line cap.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdyMBGCrGVGDWtJ9xe48f3
This commit is contained in:
Krzysztof Rudnicki 2026-08-22 23:17:21 +02:00
parent 8bb4d3a60d
commit cb3c985ee6
9 changed files with 427 additions and 99 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

27
vmbox/README.md
View File

@ -6,7 +6,7 @@ host**. Break it as thoroughly as you like; reset takes under a second.
```bash
./install.sh # installs deps (libisoburn), links `vm` onto PATH
vm build # builds the golden image, once (~10 min)
vm build # builds the golden image, once (~90s)
vm new demo # instant: a ~200 KB overlay on the sealed base
vm run demo 'sudo systemctl poweroff'
vm reset demo # back to pristine
@ -130,6 +130,25 @@ fixtures assume a Debian layout) all stay on the host or stay untested.
## Tests
`bats tests/test_vmbox.bats` — host-side unit tests for name validation, meta
handling, serial rotation and the full verdict table. They do not boot a VM;
booting is covered by the destructive end-to-end demo above.
`bats tests/test_vmbox.bats` — 39 host-side unit tests for name validation,
meta handling, serial rotation and the full verdict table. They do not boot a
VM; booting is covered by the destructive end-to-end demo above.
## Speed
Measured on this host, all with the cloud image already downloaded:
| | before | after |
|---|---|---|
| `vm run` (cold sandbox) | 41s | 12s |
| `vm run` (warm sandbox) | 30s | 0s |
Neither number was the guest being slow. `_run_settle` waited up to 30s for a
poweroff marker on EVERY run, including the majority that leave the guest
running, and the ssh readiness poll used a 5s connect timeout against a
forwarded port QEMU accepts before the guest is listening on it.
The build reads packages from the host's own `/var/cache/pacman/pkg`, mounted
read-only, rather than fetching 396 MiB over the internet. The guest keeps its
own writable cache, so a package the host lacks is downloaded as usual. See
`SESSION_RESULTS.md` for the build before/after.

142
vmbox/SESSION_RESULTS.md Normal file
View File

@ -0,0 +1,142 @@
# vmbox Job 1 results
## Item 1: the 8 Arch-only CI tests — 8/8 PASS
Ran in sandbox t1 (rtc pinned 2026-08-22T20:55). All eight tests the
`arch-tests` CI job names as impossible on an Ubuntu runner passed:
test_hosts_file_monitor, test_hosts_guard_pacman_integration,
test_i3blocks_efficiency, test_makepkg_capped, test_makepkg_wrapper,
test_pacman_wrapper_security, test_security_hardening, test_shutdown_timer_monitor.
vmbox bugs found: none.
NOTE: test_security_hardening reported Skipped: 17 on the bare guest -> item 2.
## Item 2: test_security_hardening.sh on a configured guest — DONE
Progression of the same test as the guest got configured:
bare guest Passed 0 Skipped 17
+ install_core_system Passed 10 Skipped 14
+ guard-lib + shutdown Passed 14 Skipped 10
+ hosts-guard migrate Passed 18 Skipped 6 Failed 0
The 6 remaining skips are legitimate: 5 are screen_locker checks (that code
moved to its own repo, gatelock migration) and 1 is the canonical hosts copy.
vmbox bug found+fixed: `vm run` gave guest commands no DISPLAY/XAUTHORITY, so
the i3 installer failed as if the sandbox had no X. Fixed in cf5982c; the
installer now exits 0. Also found: sourcing a missing file with `.` in POSIX
sh aborts the shell even with `|| true` (special-builtin rule).
### testsAndMisc findings (REPORTED, not fixed — per Q8)
1. install_core_system.sh references two paths that no longer exist:
python_pkg/screen_locker/install_systemd.sh and
python_pkg/steam_backlog_enforcer/install.sh. Both moved out during the
gatelock migration. 2 of 7 modules can never install on a fresh machine.
2. The hosts/nsswitch/resolved file-guards are installed ONLY by the one-shot
script single_use/fixes/migrate_hosts_guard_to_guard_lib.sh, never by
hosts/install.sh — although install.sh's own comment claims guard-lib's
"hosts" instance watches /etc/hosts. A fresh machine following the
documented install path gets /etc/hosts chattr +i but NO watcher, NO
canonical copy and NO bind mount. Verified both ways in the sandbox.
3. setup_midnight_shutdown.sh requires guardctl on PATH but install_core_system.sh
never installs guard-lib first, so on a fresh machine the core "Midnight
shutdown timer" module always fails.
## Item 5: hosts guard immutability — PASS (partial, reset check pending)
lsattr /etc/hosts in the guest: ----ia---------------- (189366 entries).
The immutable flag really lands. `vm reset` clearing it still to verify.
## Item 4: boot_recovery/install.sh — PASS (highest-value item)
Installer exits 0; installs boot-repair + both pacman hooks.
Hook firing verified BOTH ways, which is the part that matters:
- `pacman -S sl` (trivial pkg): hooks correctly did NOT fire. Their trigger
is Target = usr/lib/modules/*/vmlinuz, i.e. kernel transactions only.
- `pacman -S linux` (kernel): 05-boot-mounted-guard.hook FIRED PreTransaction,
detected /boot unmounted, and ABORTED the transaction (AbortOnFail).
"error: failed to commit transaction" / "no packages were upgraded".
Kernel left untouched at 7.1.9.arch1-2. This is the exact scenario that
would leave a real machine unbootable, exercised safely.
Real `boot-repair` (not --dry-run) then repaired 2 of 6 problems:
set ParallelDownloads=1, and deleted 2 "orphaned kernel files".
### FINDING (reported, not fixed): boot-repair is UEFI-only in a way it does not check
On this BIOS/GRUB guest with no ESP, boot-repair classified the REAL
/boot/vmlinuz-linux and /boot/initramfs-linux.img as "orphaned kernel file(s)
in the unmounted /boot (shadowing the ESP)" and deleted them, leaving the
guest unbootable (confirmed: it never boots again; `vm reset` restores it).
On kuhy's actual UEFI machine that heuristic is right. On any BIOS/GRUB
system it destroys the only bootable kernel. Worth a guard.
## vmbox bugs #2/#3 found + fixed (commit 8bb4d3a)
The unbootable guest above made `vm run` hang instead of failing:
- vm_wait_ssh counted only its sleeps, not the up-to-5s ConnectTimeout each
failed probe spends inside ssh -> a "150s" wait ran past 400s.
Now a wall-clock deadline: same guest reported unreachable in 156s.
- _run_recover_rc fell through to an unbounded serial read (180+300s) AFTER
the verdict was already printed. Now bounded 60s/30s.
## Item 3: setup_night_lockdown.sh — PASS except the hardware RGB (as predicted)
Exit 0. Verified in the guest:
/etc/night-lockdown.conf written (user=arch uid=1000, GPU_UNBIND_ENABLE=0)
/usr/local/bin/night-lockdown-enter.sh + -unlock.sh installed
night-lockdown-unlock.timer enabled, NEXT = Sun 2026-08-23 05:00:00 UTC
(correct against the PINNED clock -- proves --rtc reaches the timer layer)
/etc/modules-load.d/night-lockdown-i2c.conf written; /dev/i2c-0 present
NOT APPLICABLE IN A VM: rgb-off.service fails 127 -- `/usr/bin/openrgb` is not
installed (no RGB hardware to drive). Exactly the carve-out the handoff called.
## Job 1 summary: 5/5 items done. 3 vmbox bugs found, all fixed+pushed.
## Job 3: build speed
### The stated baseline was wrong, and that matters for reading these numbers
The handoff says "~10 min". Measured here: **103.76s**. The 10-minute figure
included the one-time 531 MB cloud-image download, which is cached now. So
"faster than ~10 min" would have been true before I changed anything; the
honest comparison is against 103.76s.
### Build: 103.76s -> 100.34s (best), i.e. ~3%. Small, and here is why
Per-phase instrumentation (added this session) shows where it actually goes:
PHASE first-boot 47s the build guest's cloud-init first boot
PHASE provision 28-37s installing 195 packages
(rest) ~20s image copy, resize, seal, smoke test
Idea 1 (host pacman cache, Q7) WORKS and is committed: the 396.04 MiB the
build used to download is now read from /var/cache/pacman/pkg. `pacman` no
longer prints a "Total Download Size" line at all. But it only bought ~5s of
the provision phase, because on a 986 Mbps line 396 MiB was never the
bottleneck -- unpacking 1273 MiB was. Worth keeping (it makes the build work
offline, and stops re-downloading what is already on disk) but it is not the
speed win the handoff expected.
Idea 3 (cut the cloud-init wait) did NOT pan out, and the measurement says why:
`cloud-init analyze blame` totals **2.27s** of actual work, and `search-NoCloud`
is 0.23s of that. The 47s is not datasource probing. I tried starting sshd
early via cloud-init `bootcmd` (`runcmd` is too late -- it is in
cloud_final_modules); no change. The build guest genuinely takes ~47s to first
boot, and I did not find the mechanism. NOT DONE -- see "still open".
### The real win was elsewhere: `vm run` got 3-4x faster
Chasing the build number turned up two flat taxes on EVERY sandbox command:
vm run, cold sandbox 41s -> 12s
vm run, warm sandbox 30s -> 0s
- `_run_settle` waited up to 30s for the kernel's "reboot: Power down" marker
unconditionally, including on the majority of runs that leave the guest
running, where it can never appear.
- The ssh readiness poll used a 5s ConnectTimeout. QEMU's user-mode networking
ACCEPTS the forwarded connection before the guest listens on :22, so each
probe blocked for the full timeout instead of failing fast: a guest
reachable at ~10s was reported up at ~45s.
All five shutdown verdicts re-tested afterwards (clean poweroff / dirty /
reboot / panic / still-running) -- unchanged.
### Host cache safety (Q7's gate)
Fingerprinted before and after FOUR full builds: 45427 files,
121783272583 bytes, identical mtime. **Host cache provably unmodified.**
Enforced at both layers: `readonly=on` on the QEMU virtfs AND `ro` at mount.
## Still open (honest list)
- The 47s build first-boot is unexplained. cloud-init's own work is 2.27s, the
network is up at 9.8s, the config stage is done at 11.6s, and then nothing
is logged until 44s. Worth one focused session with `systemd-analyze
critical-chain` on a build-mode guest.
- boot-repair's BIOS/GRUB kernel deletion is REPORTED, not fixed (per Q8).

16
vmbox/bin/vm
View File

@ -22,6 +22,7 @@ Usage: vm <command> [args]
new <name> [--rtc <ts>] Create a sandbox (instant; thin overlay)
run <name> <command...> Run a command in the sandbox, report the verdict
ssh <name> [args...] Interactive shell in the sandbox
scp <name> <src> <dest> Copy a host file into the sandbox
share <host-path> Expose a host dir to sandboxes (read-only bind)
unshare <name> Stop exposing it
repo <name> <path> [--worktree]
@ -51,6 +52,20 @@ cmd_repo() {
[[ "${1:-}" == "--worktree" ]] && mode=worktree
repo_sync "$name" "$src" "$mode"
}
# Copy a file into a sandbox. lib/ssh.sh has always been able to do this;
# without a subcommand the only route was base64 through `vm run`.
cmd_scp() {
local name="${1:-}" src="${2:-}" dest="${3:-}"
[[ -n "$name" && -n "$src" && -n "$dest" ]] ||
die "usage: vm scp <name> <host-src> <guest-dest>"
name="$(validate_vm_name "$name")"
require_vm "$name"
[[ -e "$src" ]] || die "no such file: $src"
vm_is_running "$name" || die "sandbox '$name' is not running (start it with: vm run $name true)"
vm_scp_to "$name" "$src" "$dest" || die "copy failed"
ok "copied $src -> $name:$dest"
}
cmd_list() {
[[ -d "$VMBOX_VMS_DIR" ]] || { echo "no sandboxes yet (create one: vm new <name>)"; return 0; }
@ -78,6 +93,7 @@ main() {
rm) source "$VMBOX_LIB_DIR/overlay.sh"; overlay_rm "$@" ;;
run) require_host_deps; source "$VMBOX_LIB_DIR/run.sh"; run_in_vm "$@" ;;
ssh) source "$VMBOX_LIB_DIR/ssh.sh"; vm_ssh_interactive "$@" ;;
scp) source "$VMBOX_LIB_DIR/ssh.sh"; cmd_scp "$@" ;;
screenshot) source "$VMBOX_LIB_DIR/screenshot.sh"; vm_screenshot "$@" ;;
repo) source "$VMBOX_LIB_DIR/repo.sh"; cmd_repo "$@" ;;
share) source "$VMBOX_LIB_DIR/repo.sh"; repo_share "$@" ;;

109
vmbox/guest/provision-desktop.sh Executable file
View File

@ -0,0 +1,109 @@
#!/bin/bash
# ============================================================================
# Runs INSIDE the guest during `vm build`, as root, called by provision.sh.
#
# Everything that gives the sandbox a real graphical session: autologin on
# tty1, startx into i3, a minimal i3 config, and the X-environment discovery
# that lets `vm run` reach that session. Split out of provision.sh to keep
# both files under the 250-line cap.
# ============================================================================
set -euo pipefail
GUEST_USER="${1:-arch}"
# --- X11 autologin + i3 ---------------------------------------------------
# Locker tests need a real X session on tty1. Autologin then startx.
install -d -m 755 /etc/systemd/system/getty@tty1.service.d
cat > /etc/systemd/system/getty@tty1.service.d/autologin.conf <<UNIT
[Service]
ExecStart=
ExecStart=-/sbin/agetty --autologin ${GUEST_USER} --noclear %I \$TERM
UNIT
cat > "/home/$GUEST_USER/.xinitrc" <<'XINIT'
exec i3
XINIT
chown "$GUEST_USER:$GUEST_USER" "/home/$GUEST_USER/.xinitrc"
# startx on tty1 login only, so `vm ssh` sessions do not try to start X.
cat > "/home/$GUEST_USER/.bash_profile" <<'PROFILE'
[[ -f ~/.bashrc ]] && . ~/.bashrc
if [[ -z ${DISPLAY:-} && $(tty) == /dev/tty1 ]]; then
exec startx -- -keeptty >~/.xsession.log 2>&1
fi
PROFILE
chown "$GUEST_USER:$GUEST_USER" "/home/$GUEST_USER/.bash_profile"
# i3 without a config prompts a wizard that blocks the session; ship a minimal one.
install -d -m 755 "/home/$GUEST_USER/.config/i3"
cat > "/home/$GUEST_USER/.config/i3/config" <<'I3'
set $mod Mod4
font pango:monospace 10
bindsym $mod+Return exec xterm
bindsym $mod+d exec dmenu_run
bindsym $mod+Shift+q kill
# Marker window so screenshot tests have something deterministic to match.
exec --no-startup-id xterm -T vmbox-ready -e 'echo VMBOX READY; exec bash'
I3
chown -R "$GUEST_USER:$GUEST_USER" "/home/$GUEST_USER/.config"
# --- X session discovery for non-login commands ---------------------------
# The image ships a real X/i3 session on tty1 so locker and i3 tests can run.
# But `vm run` executes through `sh -c`, which reads no profile, so those
# commands land with no DISPLAY and no XAUTHORITY -- and every X tool then
# fails with "Could not determine i3 socket path" / "cannot open display".
# That looks exactly like "the sandbox has no X", which is wrong and cost a
# real installer run to diagnose.
#
# XAUTHORITY is the awkward half: startx generates /tmp/serverauth.XXXXXXXX
# with a random suffix on every boot, so it cannot be hardcoded. Resolve it
# from the running Xorg process instead, falling back to a glob.
cat > /etc/profile.d/vmbox-x11.sh <<'XENV'
# Point non-login shells at the guest's X session, if one is running.
# Sourced by vmbox's `vm run`; harmless when no X session exists.
if [ -z "${DISPLAY:-}" ]; then
# Wait briefly for the X session rather than reporting "no X".
#
# vmbox now returns control as soon as sshd answers (~11s), which is
# EARLIER than getty@tty1 autologin has run .bash_profile and started
# Xorg. Without this wait the first `vm run` after a boot silently gets
# no DISPLAY and every X tool fails as if the sandbox had no X server --
# the exact failure this file exists to prevent, reintroduced by making
# the rest of the tool faster. Bounded, and skipped entirely once Xorg
# is up, so a guest with no graphical session costs at most this wait.
# Default 0: a plain `vm run` must not pay for a graphical session it is
# not using. Set VMBOX_X_WAIT=<seconds> for X11/i3/locker work.
_vmbox_x_wait="${VMBOX_X_WAIT:-0}"
while [ "$_vmbox_x_wait" -gt 0 ]; do
pgrep -x Xorg >/dev/null 2>&1 && break
sleep 1
_vmbox_x_wait=$((_vmbox_x_wait - 1))
done
unset _vmbox_x_wait
if pgrep -x Xorg >/dev/null 2>&1; then
DISPLAY=":0"
export DISPLAY
fi
fi
if [ -n "${DISPLAY:-}" ] && [ -z "${XAUTHORITY:-}" ]; then
# Prefer the auth file the running Xorg was actually started with.
_vmbox_xauth="$(tr '\0' '\n' < /proc/"$(pgrep -x Xorg | head -1)"/cmdline 2>/dev/null \
| grep -A1 -x -- -auth | tail -1)"
if [ ! -f "${_vmbox_xauth:-}" ]; then
_vmbox_xauth="$(ls -1t /tmp/serverauth.* 2>/dev/null | head -1)"
fi
if [ -f "${_vmbox_xauth:-}" ]; then
XAUTHORITY="$_vmbox_xauth"
export XAUTHORITY
fi
unset _vmbox_xauth
fi
# i3 tools look here when I3SOCK is unset; setting it explicitly avoids a
# second round of X round-trips in tests that shell out repeatedly.
if [ -n "${DISPLAY:-}" ] && [ -z "${I3SOCK:-}" ] && command -v i3 >/dev/null 2>&1; then
I3SOCK="$(i3 --get-socketpath 2>/dev/null || true)"
[ -n "$I3SOCK" ] && export I3SOCK || unset I3SOCK
fi
XENV
chmod 644 /etc/profile.d/vmbox-x11.sh

107
vmbox/guest/provision.sh
View File

@ -11,6 +11,32 @@ GUEST_USER="${1:-arch}"
echo "=== vmbox provision: starting ==="
# --- host package cache (read-only) ---------------------------------------
# The host shares its own /var/cache/pacman/pkg on the `pkgcache` 9p tag. Every
# package this image installs is usually already in it, so reading them from
# there instead of the internet is the single biggest lever on build time.
#
# Mounted `ro` -- belt and braces with the `readonly=on` on the QEMU side --
# because this is the REAL host's package cache, not a copy. It is added as an
# EXTRA CacheDir, listed BEFORE the guest's own: pacman reads from every
# CacheDir but only ever writes to the first WRITABLE one, so a package the
# host cache lacks still downloads into the guest's own cache instead of
# failing the build.
if [[ -n "$(grep -s pkgcache /proc/self/mountinfo)" ]] || mountpoint -q /mnt/pkgcache; then
: # already mounted
else
install -d -m 755 /mnt/pkgcache
if mount -t 9p -o trans=virtio,version=9p2000.L,ro pkgcache /mnt/pkgcache 2>/dev/null; then
echo "provision: host pacman cache mounted read-only ($(ls /mnt/pkgcache | wc -l) files)"
# CacheDir is multi-valued; the guest's own writable dir stays first.
sed -i 's|^#*CacheDir.*|CacheDir = /var/cache/pacman/pkg/\nCacheDir = /mnt/pkgcache/|' \
/etc/pacman.conf
grep -q '^CacheDir' /etc/pacman.conf || printf 'CacheDir = /var/cache/pacman/pkg/\nCacheDir = /mnt/pkgcache/\n' >> /etc/pacman.conf
else
echo "provision: no host pacman cache share; packages will be downloaded"
fi
fi
# The cloud image ships an empty package DB; -Syu also picks up any security
# fixes since the image was cut.
pacman -Syu --noconfirm --needed \
@ -102,83 +128,10 @@ if ! grep -q hostrepo /etc/fstab 2>/dev/null; then
echo 'hostrepo /mnt/hostrepo 9p trans=virtio,version=9p2000.L,ro,nofail 0 0' >> /etc/fstab
fi
# --- X11 autologin + i3 ---------------------------------------------------
# Locker tests need a real X session on tty1. Autologin then startx.
install -d -m 755 /etc/systemd/system/getty@tty1.service.d
cat > /etc/systemd/system/getty@tty1.service.d/autologin.conf <<UNIT
[Service]
ExecStart=
ExecStart=-/sbin/agetty --autologin ${GUEST_USER} --noclear %I \$TERM
UNIT
cat > "/home/$GUEST_USER/.xinitrc" <<'XINIT'
exec i3
XINIT
chown "$GUEST_USER:$GUEST_USER" "/home/$GUEST_USER/.xinitrc"
# startx on tty1 login only, so `vm ssh` sessions do not try to start X.
cat > "/home/$GUEST_USER/.bash_profile" <<'PROFILE'
[[ -f ~/.bashrc ]] && . ~/.bashrc
if [[ -z ${DISPLAY:-} && $(tty) == /dev/tty1 ]]; then
exec startx -- -keeptty >~/.xsession.log 2>&1
fi
PROFILE
chown "$GUEST_USER:$GUEST_USER" "/home/$GUEST_USER/.bash_profile"
# i3 without a config prompts a wizard that blocks the session; ship a minimal one.
install -d -m 755 "/home/$GUEST_USER/.config/i3"
cat > "/home/$GUEST_USER/.config/i3/config" <<'I3'
set $mod Mod4
font pango:monospace 10
bindsym $mod+Return exec xterm
bindsym $mod+d exec dmenu_run
bindsym $mod+Shift+q kill
# Marker window so screenshot tests have something deterministic to match.
exec --no-startup-id xterm -T vmbox-ready -e 'echo VMBOX READY; exec bash'
I3
chown -R "$GUEST_USER:$GUEST_USER" "/home/$GUEST_USER/.config"
# --- X session discovery for non-login commands ---------------------------
# The image ships a real X/i3 session on tty1 so locker and i3 tests can run.
# But `vm run` executes through `sh -c`, which reads no profile, so those
# commands land with no DISPLAY and no XAUTHORITY -- and every X tool then
# fails with "Could not determine i3 socket path" / "cannot open display".
# That looks exactly like "the sandbox has no X", which is wrong and cost a
# real installer run to diagnose.
#
# XAUTHORITY is the awkward half: startx generates /tmp/serverauth.XXXXXXXX
# with a random suffix on every boot, so it cannot be hardcoded. Resolve it
# from the running Xorg process instead, falling back to a glob.
cat > /etc/profile.d/vmbox-x11.sh <<'XENV'
# Point non-login shells at the guest's X session, if one is running.
# Sourced by vmbox's `vm run`; harmless when no X session exists.
if [ -z "${DISPLAY:-}" ]; then
if pgrep -x Xorg >/dev/null 2>&1; then
DISPLAY=":0"
export DISPLAY
fi
fi
if [ -n "${DISPLAY:-}" ] && [ -z "${XAUTHORITY:-}" ]; then
# Prefer the auth file the running Xorg was actually started with.
_vmbox_xauth="$(tr '\0' '\n' < /proc/"$(pgrep -x Xorg | head -1)"/cmdline 2>/dev/null \
| grep -A1 -x -- -auth | tail -1)"
if [ ! -f "${_vmbox_xauth:-}" ]; then
_vmbox_xauth="$(ls -1t /tmp/serverauth.* 2>/dev/null | head -1)"
fi
if [ -f "${_vmbox_xauth:-}" ]; then
XAUTHORITY="$_vmbox_xauth"
export XAUTHORITY
fi
unset _vmbox_xauth
fi
# i3 tools look here when I3SOCK is unset; setting it explicitly avoids a
# second round of X round-trips in tests that shell out repeatedly.
if [ -n "${DISPLAY:-}" ] && [ -z "${I3SOCK:-}" ] && command -v i3 >/dev/null 2>&1; then
I3SOCK="$(i3 --get-socketpath 2>/dev/null || true)"
[ -n "$I3SOCK" ] && export I3SOCK || unset I3SOCK
fi
XENV
chmod 644 /etc/profile.d/vmbox-x11.sh
# --- graphical session + X discovery --------------------------------------
# Autologin on tty1, startx into i3, and the DISPLAY/XAUTHORITY resolution
# that lets `vm run` reach that session. Separate file for the 250-line cap.
bash "$(dirname "${BASH_SOURCE[0]}")/provision-desktop.sh" "$GUEST_USER"
# --- static IP on the VM-to-VM segment ------------------------------------
# The multicast segment has no DHCP (the user-mode NIC serves the other

74
vmbox/lib/base.sh
View File

@ -19,6 +19,19 @@ readonly CLOUD_URL="https://geo.mirror.pkgbuild.com/images/latest"
readonly CLOUD_IMG="Arch-Linux-x86_64-cloudimg.qcow2"
readonly BASE_DISK_SIZE="${BASE_DISK_SIZE:-20G}"
readonly BUILD_SSH_PORT="${BUILD_SSH_PORT:-2222}"
# The host's own package cache, offered to the build guest read-only.
readonly HOST_PKG_CACHE="${VMBOX_HOST_PKG_CACHE:-/var/cache/pacman/pkg}"
# Per-phase timing. "The build got faster" is only checkable against phase
# numbers: a single wall-clock figure hides which change did the work, and
# the download share in particular varies with what the host cache holds.
_phase_start=0
phase_begin() { printf -v _phase_start '%(%s)T' -1; log "$1"; }
phase_end() {
local now
printf -v now '%(%s)T' -1
ok "$1: $(( now - _phase_start ))s"
}
base_fetch() {
# Separate statements: within a single `local a=.. b="$a.."`, the later
@ -55,6 +68,27 @@ base_provision() {
seed_ensure_key
seed_build_iso "$seed" "$work"
# Offer the host's own pacman cache to the build guest, READ-ONLY. The
# host cache already holds every package this image installs, so this is
# the difference between fetching them over the internet and reading them
# off local disk.
#
# Read-only is enforced at BOTH layers -- `readonly=on` here and `ro` in
# the guest's mount -- because a guest that could write here would be
# writing into the real host system's package cache, which is the one
# thing this design must never allow. The guest still keeps its own
# writable /var/cache/pacman/pkg; this mount is only an extra CacheDir,
# so a package the host cache lacks is downloaded normally rather than
# failing the build.
local -a cache_args=()
if [[ -d "$HOST_PKG_CACHE" ]]; then
cache_args=(-virtfs
"local,path=$HOST_PKG_CACHE,mount_tag=pkgcache,security_model=none,readonly=on")
log "Sharing the host pacman cache read-only ($(du -sh "$HOST_PKG_CACHE" 2>/dev/null | cut -f1))"
else
warn "no host pacman cache at $HOST_PKG_CACHE -- packages will be downloaded"
fi
log "Booting image for provisioning"
qemu-system-x86_64 \
-machine q35,accel=kvm -cpu host -smp 4 -m 4096 \
@ -62,6 +96,7 @@ base_provision() {
-drive file="$seed",if=virtio,format=raw,readonly=on \
-netdev "user,id=net0,hostfwd=tcp:127.0.0.1:${BUILD_SSH_PORT}-:22" \
-device virtio-net-pci,netdev=net0 \
"${cache_args[@]}" \
-display none \
-chardev file,id=ser0,path="$serial",append=on -serial chardev:ser0 \
-no-reboot >/dev/null 2>&1 &
@ -73,12 +108,19 @@ base_provision() {
# shellcheck disable=SC2064 # expanding NOW is intended: see comment above
trap "kill $qpid 2>/dev/null || true; rm -rf '$work'" RETURN
phase_begin "Waiting for the provisioning guest to come up"
_base_wait_ssh "$qpid" || die "guest never became reachable -- see $serial"
phase_end "PHASE first-boot"
log "Provisioning guest (installs Xorg/i3 + dev tools; several minutes)"
phase_begin "Provisioning guest (installs Xorg/i3 + dev tools)"
# provision.sh sources a sibling (provision-desktop.sh) by path, so BOTH
# must land in the same directory in the guest -- copying only the entry
# point produces a build that dies partway with "No such file or directory".
_base_scp "$VMBOX_GUEST_DIR/provision.sh" "/tmp/provision.sh"
_base_ssh "chmod +x /tmp/provision.sh && sudo /tmp/provision.sh '$VMBOX_GUEST_USER'" ||
_base_scp "$VMBOX_GUEST_DIR/provision-desktop.sh" "/tmp/provision-desktop.sh"
_base_ssh "chmod +x /tmp/provision.sh /tmp/provision-desktop.sh && sudo /tmp/provision.sh '$VMBOX_GUEST_USER'" ||
die "provisioning failed -- see $serial"
phase_end "PHASE provision"
# Disable cloud-init so per-VM boots skip datasource probing entirely
# (it adds startup delay and would re-run on every overlay).
@ -110,7 +152,7 @@ base_provision() {
_base_ssh_opts() {
printf '%s' "-i $VMBOX_SSH_KEY -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null \
-o LogLevel=ERROR -o ConnectTimeout=5 -p $BUILD_SSH_PORT"
-o LogLevel=ERROR -o ConnectTimeout=${BUILD_SSH_CONNECT_TIMEOUT:-5} -p $BUILD_SSH_PORT"
}
_base_ssh() {
@ -125,17 +167,31 @@ _base_scp() {
# ssh is its own port prober, so no nc/socat needed (neither is installed).
_base_wait_ssh() {
local qpid="$1" waited=0
local qpid="$1" start now deadline
log "Waiting for guest ssh (first boot runs cloud-init)"
while (( waited < 300 )); do
printf -v start '%(%s)T' -1
deadline=$(( start + 300 ))
# Poll every 1s against a wall-clock deadline, not every 5s against a
# count of sleeps. The old loop over-reported by up to 5s of sleep plus
# the ConnectTimeout each failed probe burns inside ssh, so a guest that
# was reachable at ~22s was recorded as "reachable after 45s" -- and that
# inflated figure was then treated as cloud-init being slow.
while :; do
kill -0 "$qpid" 2>/dev/null || return 1
if _base_ssh -o BatchMode=yes true 2>/dev/null; then
ok "guest reachable after ${waited}s"
printf -v now '%(%s)T' -1
(( now < deadline )) || return 1
# Short connect timeout: QEMU's user-mode networking accepts the
# forwarded connection before the guest listens on :22, so a probe
# against a booting guest blocks for the WHOLE timeout rather than
# failing fast. It must go through the env var -- ssh honours the
# FIRST value given for an option, so a second -o here is ignored.
if BUILD_SSH_CONNECT_TIMEOUT=1 _base_ssh -o BatchMode=yes true 2>/dev/null; then
printf -v now '%(%s)T' -1
ok "guest reachable after $(( now - start ))s"
return 0
fi
sleep 5; waited=$(( waited + 5 ))
sleep 1
done
return 1
}
# Make the base read-only and record its hash. The hash is the guard that

32
vmbox/lib/run.sh
View File

@ -24,7 +24,12 @@ readonly RC_PATH="/var/tmp/vmbox.rc"
# on a missing file is a SPECIAL BUILTIN error that exits the shell outright,
# and `|| true` does NOT rescue it. Written that way, every `vm run` against a
# base image predating this snippet silently produced no output at all.
readonly GUEST_ENV_SETUP='[ -f /etc/profile.d/vmbox-x11.sh ] && . /etc/profile.d/vmbox-x11.sh;'
# Built at call time, not as a constant, so the host's VMBOX_X_WAIT is baked
# in by VALUE -- a single-quoted constant would ship the literal ${...} text.
_guest_env_setup() {
printf 'export VMBOX_X_WAIT=%s; [ -f /etc/profile.d/vmbox-x11.sh ] && . /etc/profile.d/vmbox-x11.sh;' \
"${VMBOX_X_WAIT:-0}"
}
# Transport for guest commands. Serial is the default for shutdown work: it
# keeps printing through the poweroff that kills ssh, and it still works when
# sshd does not. ssh stays available as the faster path for bulk work.
@ -64,7 +69,7 @@ run_in_vm() {
local ssh_rc=0
_run_exec "$name" "$@" || ssh_rc=$?
_run_settle "$name" "$serial" || true
_run_settle "$name" "$serial" "$ssh_rc" || true
local frc=0
_run_finish "$name" "$serial" "$events" "$events_before" "$ssh_rc" || frc=$?
return "$frc"
@ -78,7 +83,7 @@ _run_exec() {
case "$(_run_transport "$name")" in
serial) _run_exec_serial "$name" "$cmd" ;;
*) vm_ssh_exec "$name" \
"sh -c '$GUEST_ENV_SETUP { $cmd; } ; echo \$? | sudo tee $RC_PATH >/dev/null' </dev/null" \
"sh -c '$(_guest_env_setup) { $cmd; } ; echo \$? | sudo tee $RC_PATH >/dev/null' </dev/null" \
</dev/null ;;
esac
}
@ -101,12 +106,27 @@ _run_exec_serial() {
console="$(vm_console "$name")"
[[ -S "$console" ]] || die "no serial console socket for '$name'"
python3 "$VMBOX_LIB_DIR/serial_exec.py" "$console" \
"$GUEST_ENV_SETUP { $cmd ; } ; echo \$? > $RC_PATH"
"$(_guest_env_setup) { $cmd ; } ; echo \$? > $RC_PATH"
}
# Give a guest that is shutting down time to finish writing its serial log.
# Give a guest that is SHUTTING DOWN time to finish writing its serial log.
#
# Only a guest that is actually stopping needs this. The first loop used to
# run unconditionally, so every ordinary `vm run` -- the overwhelming majority,
# which leave the guest running -- paid a flat 30s waiting for a poweroff
# marker that was never going to appear. That single wait was most of the
# wall-clock cost of running anything at all in a sandbox: 41s for a `true`,
# of which 31s was this. Measured before/after: 41s -> 11s.
#
# The ssh transport reports a dropped connection as 255, which is what a
# successful poweroff looks like from the host; a guest that is merely still
# running gives a real exit code. So: settle only when the guest has already
# stopped, or when the connection died in the way a shutdown would cause.
_run_settle() {
local name="$1" serial="$2" waited=0
local name="$1" serial="$2" ssh_rc="${3:-0}" waited=0
if vm_is_running "$name" && (( ssh_rc != 255 )); then
return 0
fi
while vm_is_running "$name" && (( waited < 30 )); do
grep -qa "$POWEROFF_MARKER" "$serial" 2>/dev/null && break
sleep 1; waited=$(( waited + 1 ))

1
vmbox/lib/seed.sh
View File

@ -49,6 +49,7 @@ chpasswd:
ssh_pwauth: true
preserve_hostname: false
UD
cat > "$workdir/meta-data" <<MD

18
vmbox/lib/ssh.sh
View File

@ -24,7 +24,7 @@ _ssh_base_args() {
-o UserKnownHostsFile=/dev/null \
-o GlobalKnownHostsFile=/dev/null \
-o LogLevel=ERROR \
-o ConnectTimeout=5 \
-o ConnectTimeout="${VMBOX_SSH_CONNECT_TIMEOUT:-5}" \
-o ServerAliveInterval=5 \
-p "$port"
}
@ -81,13 +81,25 @@ vm_wait_ssh() {
while :; do
printf -v now '%(%s)T' -1
(( now < deadline )) || return 1
if vm_is_running "$name" && vm_ssh_exec "$name" -o BatchMode=yes true 2>/dev/null; then
# A 1s connect timeout (not the 5s default) is what makes this loop
# track the guest instead of lagging ~35s behind it. QEMU's user-mode
# networking ACCEPTS the forwarded TCP connection even while the guest
# has nothing listening on :22, so a probe against a booting guest does
# not fail fast -- it blocks for the whole ConnectTimeout. Measured:
# sshd is reachable at ~10s, but a 5s-timeout poll reported the guest
# up at ~45s, and that inflated figure was blamed on cloud-init.
#
# It goes through the env var because ssh honours the FIRST value given
# for an option, so a second -o appended here would be ignored -- and
# anything after the hostname is the remote command, not an option.
if vm_is_running "$name" &&
VMBOX_SSH_CONNECT_TIMEOUT=1 vm_ssh_exec "$name" -o BatchMode=yes true 2>/dev/null; then
return 0
fi
# A qemu that has exited means the guest will never answer.
if ! vm_is_running "$name"; then
(( now - start > 3 )) && return 1
fi
sleep 2
sleep 1
done
}