utils/vmbox/lib/serial_exec.py
Krzysztof kuhy Rudnicki 0b1f86ae2c vmbox: add a serial transport so shutdown tests do not depend on sshd
The Arch cloud image leaves sshd enabled but never started -- no start line,
no failure line, dependencies satisfied. Four hypotheses (NTP, cloud-init
clean, a socket-disable, a blocking start) were each disproven by measurement,
so rather than keep paying a ~10 min rebuild per guess this adds the transport
the tool should arguably have used all along.

For the case vmbox exists for, serial beats ssh on the merits: it keeps
printing straight through the poweroff that kills an ssh connection, and it
works when sshd does not. `vm run` now picks ssh when it answers and the
serial console otherwise, and recovers the guest exit code either way. Guests
get a second serial port (ttyS1) on a unix socket with a root autologin, kept
separate from ttyS0 so driving a shell never interleaves with the append-only
log the verdict is read from.

serial_exec.py waits for an explicit per-command end marker rather than a
fixed sleep. That is not a detail: an earlier probe used a 4s window, the next
command was typed while the previous one still ran, and the tty echoed that
input -- which reads exactly like output in a transcript and produced a
confident but wrong conclusion. It also strips ANSI CSI and systemd's OSC 3008
shell-integration blocks, verified against a real captured boot log.

Also: a vmbox-sshd-kick unit that forces sshd up after multi-user.target as
belt and braces, and the smoke-test sandbox renamed to a name that actually
passes validate_vm_name -- "__smoke" was rejected, so the gate silently never
ran.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdyMBGCrGVGDWtJ9xe48f3
2026-08-22 20:58:22 +02:00

105 lines
3.7 KiB
Python
Executable File

#!/usr/bin/env python3
"""Run a command in a vmbox guest over its serial console root shell.
This exists because ssh is not the right channel for the thing vmbox is for.
A shutdown script kills the connection carrying its own result, and the serial
console keeps printing right through the poweroff -- so for the shutdown case
serial is not a fallback, it is the better transport. It also works when sshd
is broken, which makes an unreachable guest debuggable instead of opaque.
Every command is bracketed by explicit markers and waited on until its END
marker appears. Never use a fixed sleep: if the window closes early the next
command is typed while the previous one still runs, the tty echoes that input,
and the echoed text is indistinguishable from output in the transcript.
Usage:
serial_exec.py <console.sock> <command> -> runs, prints output
serial_exec.py <console.sock> --wait-prompt -> just wait for a shell
"""
from __future__ import annotations
import re
import socket
import sys
import time
import uuid
PROMPT = "[root@vmbox"
# Strip ANSI CSI sequences and the OSC 3008 shell-integration blocks that
# systemd's login emits, both of which otherwise corrupt marker matching.
_ANSI = re.compile(r"\x1b\[[0-9;?]*[a-zA-Z]")
_OSC = re.compile(r"\x1b?\]3008;[^\x07\\]*(?:\x07|\\)?")
def clean(raw: bytes) -> str:
text = raw.decode("utf-8", "replace")
text = _ANSI.sub("", text)
text = _OSC.sub("", text)
return text.replace("\r", "")
class Console:
"""A line-oriented client for the guest's serial console socket."""
def __init__(self, path: str) -> None:
self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
self.sock.connect(path)
self.sock.settimeout(0.4)
self.buf = b""
def read_until(self, needle: str, timeout: float) -> bool:
end = time.time() + timeout
while time.time() < end:
try:
chunk = self.sock.recv(4096)
except TimeoutError:
continue
except OSError:
return False
if not chunk:
return False
self.buf += chunk
if needle in clean(self.buf):
return True
return False
def wait_prompt(self, timeout: float = 180) -> bool:
# Nudge with a newline: if the shell is already up we get a fresh
# prompt immediately instead of waiting for unrelated output.
self.sock.sendall(b"\n")
return self.read_until(PROMPT, timeout)
def run(self, cmd: str, timeout: float = 300) -> str:
token = uuid.uuid4().hex[:8]
start, done = f"<<<S{token}>>>", f"<<<E{token}>>>"
self.buf = b""
# `; echo RC=$?` keeps the guest's exit status, which the ssh path
# cannot report once the machine stops.
self.sock.sendall(f"echo {start}; {cmd}; echo RC=$?; echo {done}\n".encode())
if not self.read_until(done, timeout):
return f"### TIMEOUT: no end marker after {timeout}s ###"
text = clean(self.buf)
body = text.split(start, 1)[-1].split(done, 1)[0]
# Drop the echoed command line itself.
lines = [ln for ln in body.splitlines() if start not in ln and done not in ln]
return "\n".join(lines).strip()
def main() -> int:
if len(sys.argv) < 3:
print(__doc__, file=sys.stderr)
return 2
console = Console(sys.argv[1])
if sys.argv[2] == "--wait-prompt":
return 0 if console.wait_prompt() else 1
if not console.wait_prompt():
print("serial_exec: no root prompt on the console", file=sys.stderr)
return 1
print(console.run(" ".join(sys.argv[2:])))
return 0
if __name__ == "__main__":
sys.exit(main())