Add an execution-trace harness for shell splits

verify_shell_split.sh sources the libs and never calls them, which is why
analyze_repo.sh shipped broken after passing every static check. This runs
the script for real with ~30 mutating binaries (sudo, pacman, systemctl,
adb, nft, mount, reboot...) shadowed by PATH stubs that record their calls
and exit 0, then prints a diffable trace: exit status, stubbed calls in
order, stdout, stderr.

Verified against both bugs the handoff documents: it catches the `set -e`
function-tail abort (exit 1, later calls absent from the trace) and the
self-referencing nameref (exit 0, correct stdout, `circular name
reference` in stderr). `systemctl is-active foo` stayed inactive across
all three runs, so nothing reached the live system.

This is what lets Decision 6 ("never execute an enforcement script") and
the user's "run-and-diff always" coexist. Handoff updated with both, plus
the win32_window.cpp resolution: it is tracked and listed in
CMakeLists.txt, so gitignore cannot work -- the exemption belongs in
~/utils/file_length/config.py.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011w2Pj4Ek3ZNMQVYcFHgXDY
This commit is contained in:
Krzysztof Rudnicki 2026-08-16 22:49:48 +02:00
parent 3b37d7b422
commit f4d45ce60f
3 changed files with 259 additions and 5 deletions

View File

@ -0,0 +1,47 @@
{
"intent": "Add an execution-trace harness so shell splits can be verified by running the script with mutating commands stubbed, closing the gap that let a green-but-broken split ship.",
"scope": [
"meta/scripts/trace_shell_split.sh (new)",
"Non-goal: does not replace verify_shell_split.sh; that still covers syntax, lint and source-line resolution"
],
"changes": [
"New harness that shadows 30 mutating binaries (sudo, pacman, systemctl, adb, nft, mount, mkinitcpio, reboot...) with PATH stubs that record their invocation and exit 0, runs the target script, and prints a diffable trace: exit status, stubbed calls in order with arguments, stdout, stderr.",
"Trace is the comparison artifact: capture at the pre-split commit and after the split, then diff."
],
"verification": [
{
"command": "trace_shell_split.sh on a fixture reproducing the `set -e` function-tail bug",
"result": "pass (bug detected)",
"evidence": "exit status 1, empty stdout, and the later `sudo systemctl restart foo` absent from the trace -- the script died silently at the false conditional tail. This is the analyze_repo.sh failure that bash -n, shellcheck, a function-set diff and the stubbed probe all passed."
},
{
"command": "trace_shell_split.sh on the same fixture with `|| true` appended",
"result": "pass",
"evidence": "exit status 0, trace records 'sudo systemctl restart foo', stdout shows 'detected: py=true' and 'done'. The two traces differ, so the harness discriminates rather than always agreeing."
},
{
"command": "trace_shell_split.sh on a fixture reproducing the self-referencing nameref bug",
"result": "pass (bug detected)",
"evidence": "exit status 0 and correct stdout, but stderr carries four 'circular name reference' warnings that a trace diff surfaces. This bug is invisible to exit-status checking alone."
},
{
"command": "systemctl is-active foo",
"result": "pass",
"evidence": "inactive -- the stub absorbed the call; the live system was not touched by any of the three runs."
},
{
"command": "shellcheck + shfmt -d on meta/scripts/trace_shell_split.sh",
"result": "pass",
"evidence": "Both clean. SC2155 fixed by splitting declare from assign rather than suppressing. Banner count 2 before and 2 after shfmt -w (that formatter is known to delete banners). Re-ran both fixtures after formatting and behaviour was unchanged."
}
],
"risks": [
"Stubs always exit 0, so a script branching on a real failure exit status takes a path it would not take live; such cases need a hand-written stub and a note in that split's evidence.",
"A mutating command not in STUBBED_COMMANDS runs for real. The list must be extended deliberately before tracing a script that uses something new -- check with a grep for mutating verbs first.",
"Not a substitute for reading the diff: an identical trace proves the same calls happened, not that the split is well-factored."
],
"rollback": [
"Delete meta/scripts/trace_shell_split.sh; no other file imports it.",
"Shell splits fall back to verify_shell_split.sh plus the by-hand checks in docs/shell-split-verification.md."
]
}

163
meta/scripts/trace_shell_split.sh Executable file
View File

@ -0,0 +1,163 @@
#!/usr/bin/env bash
# ============================================================================
# Capture an execution trace of a shell script with every mutating command
# stubbed, so a split can be verified by RUNNING it rather than by sourcing it.
#
# Why this exists: verify_shell_split.sh sources the libs and never calls them,
# which proves `source` lines resolve and nothing more. The analyze_repo.sh
# split passed that, plus bash -n, shellcheck and a function-set diff, and
# still shipped a script that aborted after language detection -- a `set -e`
# function-tail return and a self-referencing nameref, neither reachable
# without execution. See docs/shell-split-verification.md.
#
# The trace is the artifact: run this at the pre-split commit and again after
# the split, then diff. An identical trace means the same commands ran in the
# same order with the same arguments.
#
# Mutating binaries are shadowed by stubs earlier on PATH, so the live system
# is never touched. That is what makes this safe for the enforcement and
# installer scripts Decision 6 forbids executing for real.
#
# Usage:
# trace_shell_split.sh <script> [-- <script args>]
# trace_shell_split.sh <script> --out <file> [-- <script args>]
# ============================================================================
set -euo pipefail
SCRIPT_NAME="$(basename "$0")"
readonly SCRIPT_NAME
# Commands that change the system, the phone, or the package set. Each becomes
# a stub that records its invocation and exits 0. Extend deliberately: a
# missing name here means the real binary runs.
readonly STUBBED_COMMANDS=(
sudo pacman yay paru systemctl systemd-run
adb fastboot
nft iptables ip6tables firewall-cmd
mount umount swapoff modprobe
useradd usermod visudo chpasswd
mkinitcpio grub-mkconfig bootctl
npm pip pip3 flutter gradle
reboot shutdown poweroff halt
)
TARGET=""
OUT=""
TEMP_DIR=""
declare -a SCRIPT_ARGS=()
cleanup() {
if [[ -n "${TEMP_DIR:-}" && -d "$TEMP_DIR" ]]; then
rm -rf "$TEMP_DIR"
fi
}
trap cleanup EXIT
usage() {
echo "Usage: $SCRIPT_NAME <script> [--out <file>] [-- <script args>]"
echo " <script> script to trace (not modified)"
echo " --out <file> write the trace here (default: stdout)"
echo " -- everything after this is passed to the script"
exit 0
}
validate_requirements() {
if [[ -z "$TARGET" ]]; then
echo "Error: no script given" >&2
exit 1
fi
if [[ ! -f "$TARGET" ]]; then
echo "Error: no such script: $TARGET" >&2
exit 1
fi
}
# One stub per mutating command: append the call to the trace, succeed.
# Stubs report success because the point is to reach later code paths, not to
# simulate failure. A script whose logic branches on a real exit status needs a
# hand-written stub instead -- note that in the split's evidence file.
write_stubs() {
local bin_dir="$1" name
mkdir -p "$bin_dir"
for name in "${STUBBED_COMMANDS[@]}"; do
cat >"$bin_dir/$name" <<STUB
#!/usr/bin/env bash
printf '%s %s\n' "$name" "\$*" >>"\$TRACE_FILE"
exit 0
STUB
chmod +x "$bin_dir/$name"
done
}
main() {
validate_requirements
TEMP_DIR="$(mktemp -d)"
if [[ ! -d "$TEMP_DIR" ]]; then
echo "Error: failed to create temporary directory" >&2
exit 1
fi
local bin_dir="$TEMP_DIR/bin"
local trace="$TEMP_DIR/trace.txt"
: >"$trace"
write_stubs "$bin_dir"
# xtrace to a dedicated fd so the script's own stdout stays separate.
local xtrace="$TEMP_DIR/xtrace.txt"
set +e
(
export TRACE_FILE="$trace"
export PATH="$bin_dir:$PATH"
# A stubbed run must never believe it is root.
export EUID_OVERRIDE=1000
exec 9>"$xtrace"
export BASH_XTRACEFD=9
set -x
bash "$TARGET" "${SCRIPT_ARGS[@]+"${SCRIPT_ARGS[@]}"}"
) >"$TEMP_DIR/stdout.txt" 2>"$TEMP_DIR/stderr.txt"
local status=$?
set -e
{
echo "=== exit status: $status"
echo "=== mutating calls (stubbed)"
cat "$trace"
echo "=== stdout"
cat "$TEMP_DIR/stdout.txt"
echo "=== stderr"
cat "$TEMP_DIR/stderr.txt"
} >"${OUT:-/dev/stdout}"
}
while [[ $# -gt 0 ]]; do
case $1 in
--out)
OUT="$2"
shift 2
;;
-h | --help)
usage
;;
--)
shift
SCRIPT_ARGS=("$@")
break
;;
*)
if [[ -z "$TARGET" ]]; then
TARGET="$1"
shift
else
echo "Unknown option: $1" >&2
exit 1
fi
;;
esac
done
main "$@"

View File

@ -6,11 +6,15 @@
## Where things stand
**61 files** over 250 lines (was 183 — 67% cleared). Everything is committed
**and pushed** to `main` at `c4bdaad`. `focus_owner/analysis_options.yaml` has an
uncommitted change from an earlier session — **leave it unstaged**, it is not
ours. (It was briefly lost to a pre-commit stash this session and recovered from
`~/.cache/pre-commit/patch*`; see "If the tree looks suspiciously clean".)
**60 files** over 250 lines (was 183 — 67% cleared). Everything is committed
to `main`; see `git log` for the head.
`focus_owner/analysis_options.yaml` is **resolved** (2026-08-16). It was not a
stray edit: `flutter pub get` writes that `analyzer.exclude` block itself
("Upgrading analysis_options.yaml to exclude build and platform directories"),
reproduced byte-identically from a clean detached worktree. It is committed, so
it will stop reappearing. Earlier handoffs called it "not ours" — that was
wrong, and the reason it kept coming back.
**`python_pkg/` is DONE — 50 violations → 0.** Do not reopen it.
**All 12 prose files are done.** `kcd2_dice_solver` is 12 → 4, all four source
@ -60,6 +64,19 @@ Same in TypeScript: `kcd2`'s `search.ts` and `badgeValue.ts`, and
`billsplit`'s last file is Flutter's generated `win32_window.cpp`; editing it
would be undone by `flutter create`, so it needs an exemption or a shrug.
**Resolved 2026-08-16 — and gitignore is NOT the mechanism.** The file is
tracked and `billsplit/windows/runner/CMakeLists.txt:13` lists it as a build
source, so a `.gitignore` entry does nothing (git keeps tracking it) and
`git rm --cached` would break the Windows build for every fresh clone.
`check.py` does skip git-ignored files, which is what makes gitignore look
like it would work — it only works for untracked ones.
**Prerequisite of the gate-wiring step:** add a Flutter-runner exemption to
`~/utils/file_length/config.py` (a _different repo_, with its own gate and CI)
covering `windows/runner/`, `linux/runner/` and `macos/Runner/`. That is
platform scaffolding `flutter create` owns; it belongs beside the existing
`GENERATED_PATTERN` rules. Land it there before wiring the hook here.
## Read these BEFORE the first split
- `docs/shell-split-recipes.md` — how to **make** a shell split.
@ -91,6 +108,23 @@ reachable by any static check:
thing**: stub only what mutates the system, and diff the generated artifacts
against a detached worktree at the pre-split commit.
**`meta/scripts/trace_shell_split.sh` now does this** (added 2026-08-16). It
shadows ~30 mutating binaries (`sudo`, `pacman`, `systemctl`, `adb`, `nft`,
`mount`, `reboot`, …) with `PATH` stubs that record their calls and exit 0,
runs the script, and prints a diffable trace of exit status + stubbed calls +
stdout + stderr. Capture before and after a split, then diff.
Verified against both documented bug classes: it catches the `set -e`
function-tail abort (exit 1, later calls missing from the trace) and the
self-referencing nameref (exit 0 and correct stdout, but `circular name
reference` in stderr). Two caveats: stubs always exit 0, so a script branching
on a real failure status needs a hand-written stub; and a mutating command
**not** in `STUBBED_COMMANDS` runs for real — grep the script for mutating
verbs and extend the list before tracing something new.
This is what lets Decision 6 and "verify by running" coexist: the enforcement
and installer scripts are executed against stubs, never against the system.
### `--skip-install` is not permission to run a script
`shell_check.sh` gates only `install_if_missing`'s pacman branch on
@ -121,6 +155,16 @@ grep the flag's variable and confirm it gates **every** mutation path.
10. **When a seam splits code covered by a per-file-ignore, fix the lint — do
not copy the ignore onto the new file.** Ask only if the fix looks like it
changes behaviour.
11. **The named blockers get real refactors, not exemptions** (user, 2026-08-16:
"yes, do a real refactoring, we can take risks"). Risk is accepted on the
blocker list — but "accepted risk" means verified against stubs, not
unverified. Pair it with 12.
12. **Every split is verified by running, always** (user, 2026-08-16: "apply the
full run-and-diff always"). For scripts Decision 6 forbids executing live,
"running" means `meta/scripts/trace_shell_split.sh` — a real execution with
mutating binaries stubbed — and a trace diff against the pre-split commit.
Decisions 6 and 12 do not conflict; the harness is what reconciles them.
A split verified by static checks alone is not finished.
## The constraint that will bite you