mirror of
https://github.com/kuhyx/utils.git
synced 2026-11-10 20:27:15 +01:00
Krzysztof kuhy Rudnicki
a2d672f7fd
Much of testsAndMisc/linux_configuration cannot be tested on the host: 15 files use `chattr +i` (so a bad test is not simply revertable), the shutdown installers write units that power the machine off, and boot_recovery installs pacman hooks. The repo's own CI says as much -- its Ubuntu job runs only the side-effect-free tests, noting the rest "drive pacman, /etc, systemd or adb". vmbox runs them in a throwaway Arch guest and verifies the result from the HOST, which is the part that is not obvious: a successful shutdown kills the connection carrying the report, and ssh returns 255 for a clean poweroff, a crash and a dropped link alike. So the verdict comes from two artefacts that outlive the VM -- a QMP event log written by a recorder attached at launch (attach later and qemu has already exited), and the serial console log. Five outcomes are distinguished: clean poweroff (SHUTDOWN guest:true plus a serial tail reaching "reboot: Power down"), dirty stop, guest-reset (rebooted instead of powering off), GUEST_PANICKED, and hung. A POWERDOWN event proves nothing -- it means the request was delivered, not that anything acted on it. Reset is a qcow2 backing-file trick: the golden image is built once and sealed (chmod 444 plus a sha256 sidecar verified on every launch, since booting the base would silently invalidate every overlay), and each sandbox is a thin overlay, so reset is rm + recreate. No root anywhere -- user-mode networking with per-VM forwarded ssh ports, and a socket multicast segment for VM-to-VM. The guest clock is pinned and re-applied on every launch, because the shutdown installers gate on a 21:00-05:00 window; NTP is masked in the image or it silently undoes the pin. Repos are cloned from a read-only share scoped to an explicit directory, never $HOME, and credentials are sentinel values. `vm build` ends by booting a throwaway sandbox off the sealed image and refusing to ship if ssh does not answer. Several distinct faults produced an identical "boots fine, connection reset" symptom; checks inside the build VM only prove state at seal time, so the gate has to be an end-to-end boot. Tests: 8 bats cases over the verdict table and name/meta/serial handling; shellcheck and ruff clean; every file within the 250-line cap. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VdyMBGCrGVGDWtJ9xe48f3
172 lines
7.5 KiB
Bash
172 lines
7.5 KiB
Bash
#!/bin/bash
|
|
# ============================================================================
|
|
# Runs INSIDE the guest during `vm build`, as root, exactly once.
|
|
# Installs the test surface (Xorg/i3, dev tools), wires serial+autologin,
|
|
# then leaves the image ready to be sealed.
|
|
# ============================================================================
|
|
|
|
set -euo pipefail
|
|
|
|
GUEST_USER="${1:-arch}"
|
|
|
|
echo "=== vmbox provision: starting ==="
|
|
|
|
# The cloud image ships an empty package DB; -Syu also picks up any security
|
|
# fixes since the image was cut.
|
|
pacman -Syu --noconfirm --needed \
|
|
base-devel git openssh sudo python python-pip \
|
|
xorg-server xorg-xinit xorg-xrandr xorg-xset i3-wm xterm dmenu \
|
|
strace jq bats shellcheck kcov rsync inetutils \
|
|
iproute2 nftables which less vim
|
|
|
|
# --- serial console -------------------------------------------------------
|
|
# The verdict logic reads the serial log to tell a clean poweroff from one
|
|
# that died mid-sequence, so the guest must actually WRITE its console there.
|
|
# We keep the VGA device for screenshots, so we cannot rely on -nographic's
|
|
# implicit redirect: request ttyS0 explicitly on the kernel cmdline.
|
|
if [[ -f /etc/default/grub ]]; then
|
|
sed -i 's/^GRUB_CMDLINE_LINUX_DEFAULT="\(.*\)"/GRUB_CMDLINE_LINUX_DEFAULT="\1 console=tty0 console=ttyS0,115200"/' \
|
|
/etc/default/grub
|
|
grub-mkconfig -o /boot/grub/grub.cfg 2>/dev/null || true
|
|
fi
|
|
systemctl enable serial-getty@ttyS0.service
|
|
|
|
# --- ssh ------------------------------------------------------------------
|
|
# `systemctl enable sshd` alone is NOT enough on this image: systemd's
|
|
# ssh-generator provides a socket-activated AF_UNIX unit, so the enable can
|
|
# no-op and leave nothing listening on TCP:22. The guest then boots perfectly
|
|
# and every ssh gets "Connection reset by peer". Force the real service on and
|
|
# verify the symlink exists, failing the build loudly if it does not.
|
|
systemctl enable sshd.service
|
|
systemctl disable sshd.socket 2>/dev/null || true
|
|
if [[ ! -e /etc/systemd/system/multi-user.target.wants/sshd.service ]]; then
|
|
echo "provision: FAILED to enable sshd.service -- guest would be unreachable" >&2
|
|
exit 1
|
|
fi
|
|
# Host keys: sshd is "enabled" but FAILS TO START without them, which looks
|
|
# identical to a network fault from the host side. Generate now and assert.
|
|
ssh-keygen -A
|
|
ls /etc/ssh/ssh_host_*_key >/dev/null 2>&1 || {
|
|
echo "provision: FAILED to generate sshd host keys" >&2
|
|
exit 1
|
|
}
|
|
echo "provision: sshd.service enabled with $(ls /etc/ssh/ssh_host_*_key | wc -l) host keys"
|
|
|
|
# --- systemd --user session ----------------------------------------------
|
|
# Several target scripts use `systemctl --user`, which needs a real session
|
|
# bus. Lingering gives the user one without an interactive login.
|
|
loginctl enable-linger "$GUEST_USER" 2>/dev/null || true
|
|
|
|
# --- serial console autologin -------------------------------------------
|
|
# Debuggability: when ssh is broken, the serial console is the ONLY way in.
|
|
# Without autologin here an unreachable guest can only be diagnosed by
|
|
# inferring from absent log lines, which is exactly how a single sshd fault
|
|
# turned into several blind image rebuilds.
|
|
install -d -m 755 /etc/systemd/system/serial-getty@ttyS0.service.d
|
|
cat > /etc/systemd/system/serial-getty@ttyS0.service.d/autologin.conf <<SUNIT
|
|
[Service]
|
|
ExecStart=
|
|
ExecStart=-/sbin/agetty --autologin root --keep-baud 115200,57600,38400,9600 %I \$TERM
|
|
SUNIT
|
|
|
|
# --- host repo share ------------------------------------------------------
|
|
# Mounted read-only; the guest clones out of it rather than working in it.
|
|
install -d -m 755 /mnt/hostrepo
|
|
if ! grep -q hostrepo /etc/fstab 2>/dev/null; then
|
|
echo 'hostrepo /mnt/hostrepo 9p trans=virtio,version=9p2000.L,ro,nofail 0 0' >> /etc/fstab
|
|
fi
|
|
|
|
# --- X11 autologin + i3 ---------------------------------------------------
|
|
# Locker tests need a real X session on tty1. Autologin then startx.
|
|
install -d -m 755 /etc/systemd/system/getty@tty1.service.d
|
|
cat > /etc/systemd/system/getty@tty1.service.d/autologin.conf <<UNIT
|
|
[Service]
|
|
ExecStart=
|
|
ExecStart=-/sbin/agetty --autologin ${GUEST_USER} --noclear %I \$TERM
|
|
UNIT
|
|
|
|
cat > "/home/$GUEST_USER/.xinitrc" <<'XINIT'
|
|
exec i3
|
|
XINIT
|
|
chown "$GUEST_USER:$GUEST_USER" "/home/$GUEST_USER/.xinitrc"
|
|
|
|
# startx on tty1 login only, so `vm ssh` sessions do not try to start X.
|
|
cat > "/home/$GUEST_USER/.bash_profile" <<'PROFILE'
|
|
[[ -f ~/.bashrc ]] && . ~/.bashrc
|
|
if [[ -z ${DISPLAY:-} && $(tty) == /dev/tty1 ]]; then
|
|
exec startx -- -keeptty >~/.xsession.log 2>&1
|
|
fi
|
|
PROFILE
|
|
chown "$GUEST_USER:$GUEST_USER" "/home/$GUEST_USER/.bash_profile"
|
|
|
|
# i3 without a config prompts a wizard that blocks the session; ship a minimal one.
|
|
install -d -m 755 "/home/$GUEST_USER/.config/i3"
|
|
cat > "/home/$GUEST_USER/.config/i3/config" <<'I3'
|
|
set $mod Mod4
|
|
font pango:monospace 10
|
|
bindsym $mod+Return exec xterm
|
|
bindsym $mod+d exec dmenu_run
|
|
bindsym $mod+Shift+q kill
|
|
# Marker window so screenshot tests have something deterministic to match.
|
|
exec --no-startup-id xterm -T vmbox-ready -e 'echo VMBOX READY; exec bash'
|
|
I3
|
|
chown -R "$GUEST_USER:$GUEST_USER" "/home/$GUEST_USER/.config"
|
|
|
|
# --- static IP on the VM-to-VM segment ------------------------------------
|
|
# The multicast segment has no DHCP (the user-mode NIC serves the other
|
|
# interface), so without a static address sandboxes cannot reach each other.
|
|
# The last octet is injected per-VM at launch via the vmbox.peer kernel arg.
|
|
cat > /usr/local/bin/vmbox-peer-ip <<'PEER'
|
|
#!/bin/bash
|
|
# Assign the peer-network address from the vmbox.peer=<n> kernel argument.
|
|
set -euo pipefail
|
|
n=$(sed -n 's/.*vmbox\.peer=\([0-9]\+\).*/\1/p' /proc/cmdline)
|
|
[[ -n "$n" ]] || exit 0
|
|
subnet=$(sed -n 's/.*vmbox\.subnet=\([0-9.]\+\).*/\1/p' /proc/cmdline)
|
|
[[ -n "$subnet" ]] || subnet=10.77.0
|
|
# The peer NIC is the one WITHOUT a default route (user-mode NIC has it).
|
|
for i in /sys/class/net/e*; do
|
|
dev=$(basename "$i")
|
|
# Skip the primary NIC: it carries ssh and already has DHCP + a default
|
|
# route. Touching it is how the sandbox becomes unreachable.
|
|
ip route show default dev "$dev" | grep -q . && continue
|
|
ip -4 addr show dev "$dev" | grep -q 'inet ' && continue
|
|
ip addr add "${subnet}.${n}/24" dev "$dev" 2>/dev/null || true
|
|
ip link set "$dev" up
|
|
break
|
|
done
|
|
PEER
|
|
chmod 755 /usr/local/bin/vmbox-peer-ip
|
|
|
|
cat > /etc/systemd/system/vmbox-peer-ip.service <<'PSVC'
|
|
[Unit]
|
|
Description=vmbox peer network address
|
|
After=network.target
|
|
[Service]
|
|
Type=oneshot
|
|
ExecStart=/usr/local/bin/vmbox-peer-ip
|
|
RemainAfterExit=yes
|
|
[Install]
|
|
WantedBy=multi-user.target
|
|
PSVC
|
|
systemctl enable vmbox-peer-ip.service
|
|
|
|
systemctl enable systemd-networkd systemd-resolved 2>/dev/null || true
|
|
|
|
# --- keep the pinned clock pinned -----------------------------------------
|
|
# The host passes -rtc base=<ts> so time-gated scripts (the shutdown installers
|
|
# key off a 21:00-05:00 window) are deterministic. systemd-timesyncd reaches
|
|
# the network on boot and silently overwrites that with real time, which
|
|
# re-introduces exactly the nondeterminism -rtc exists to remove.
|
|
systemctl disable systemd-timesyncd 2>/dev/null || true
|
|
systemctl mask systemd-timesyncd 2>/dev/null || true
|
|
# Do NOT write a /etc/systemd/network/*.network file here: an en* match
|
|
# also catches the primary NIC and outranks the cloud image's own DHCP
|
|
# config, which breaks sshd's network and makes the sandbox unreachable.
|
|
# (Learned the hard way -- it cost a full rebuild.) Masking the unit above is
|
|
# sufficient to keep the pinned clock; UseNTP only matters if timesyncd runs.
|
|
# cloud-init also writes an NTP config on some images; neutralise it.
|
|
rm -f /etc/systemd/timesyncd.conf.d/*.conf 2>/dev/null || true
|
|
|
|
echo "=== vmbox provision: done ==="
|