From f6356d4725018a350eb11223a6042687a5573411 Mon Sep 17 00:00:00 2001 From: Krzysztof kuhy Rudnicki Date: Sun, 16 Aug 2026 23:31:49 +0200 Subject: [PATCH] Trace scripts that place files, via a prefix redirect Shell redirections are builtins, so no PATH stub can intercept `cat > /etc/modprobe.d/x.conf`. That left installer scripts unverifiable: stub cp/chmod/ln and the trace is nearly empty, don't and the run rewrites the live system. --prefix redirects HOME and every XDG_* base into a temp tree. XDG_DATA_HOME is exported explicitly, not left to default off HOME: install_leechblock.sh reads it directly and then runs `rsync -a --delete` at it. --bind-abs binds hardcoded absolutes via bwrap --unshare-user --uid 0, leaf directories only. uid 0 is load-bearing beyond permissions: common.sh's require_root does `exec sudo "$0"`, and under the sudo stub that truncated the whole run to one recorded call -- a trace that diffs clean against any split. Binding /etc wholesale was measured to shadow /etc/passwd, collapse $SUDO_USER to /home//, and mask a nested bind, so the scanner never emits it. The trace gains a manifest of every file written, with content hashes. That section is the point: with it, a dropped write in the fixture pair is a one-line diff; with it stripped, the two traces are identical, i.e. the old harness passed that broken split. Hashes normalize the prefix to @PREFIX@ because install_usage_monitoring.sh writes an unquoted heredoc that bakes $HOME into the artifact. Verified: ~/.config/systemd/user, ~/.local/bin, ~/.local/share and /etc/modprobe.d byte-identical before and after; usage-report units still 2. The three prior regressions are now committed fixtures rather than ad-hoc -- one was silently neutered mid-change when a trailing echo added for SC2034 became the function's last statement and turned exit 1 into exit 0. Stub generation moved to lib/ because --prefix pushed the harness to 252 lines, over the cap it exists to serve. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_011w2Pj4Ek3ZNMQVYcFHgXDY --- docs/shell-split-verification.md | 87 ++++++++- .../trace-prefix-writes-2026-08-16.json | 22 +++ .../trace-prefix-writes-2026-08-16.json | 73 ++++++++ meta/scripts/fixtures/circular_nameref.sh | 32 ++++ meta/scripts/fixtures/dropped_write_after.sh | 35 ++++ meta/scripts/fixtures/dropped_write_before.sh | 39 ++++ meta/scripts/fixtures/set_e_function_tail.sh | 32 ++++ meta/scripts/fixtures/valued_stub_branch.sh | 27 +++ meta/scripts/lib/trace_prefix.sh | 166 ++++++++++++++++++ meta/scripts/lib/trace_stubs.sh | 72 ++++++++ meta/scripts/trace_shell_split.sh | 122 ++++++------- 11 files changed, 641 insertions(+), 66 deletions(-) create mode 100644 docs/superpowers/contracts/trace-prefix-writes-2026-08-16.json create mode 100644 docs/superpowers/evidence/trace-prefix-writes-2026-08-16.json create mode 100755 meta/scripts/fixtures/circular_nameref.sh create mode 100755 meta/scripts/fixtures/dropped_write_after.sh create mode 100755 meta/scripts/fixtures/dropped_write_before.sh create mode 100755 meta/scripts/fixtures/set_e_function_tail.sh create mode 100755 meta/scripts/fixtures/valued_stub_branch.sh create mode 100755 meta/scripts/lib/trace_prefix.sh create mode 100755 meta/scripts/lib/trace_stubs.sh diff --git a/docs/shell-split-verification.md b/docs/shell-split-verification.md index c5d31541..8310db16 100644 --- a/docs/shell-split-verification.md +++ b/docs/shell-split-verification.md @@ -93,11 +93,88 @@ Three things to watch: target for network and build verbs, then pass them: `--stub git,curl,makepkg`. -**Scripts whose job is placing files** (`install_pacman_wrapper.sh`, -`block_compulsive_opening.sh`) are not yet tractable here: stub `cp`/`chmod`/ -`ln` and the trace is nearly empty, don't stub them and the run rewrites the -live system. Those need a fakeroot-style prefix redirect — a harness change, -not a split. Do not trace one until that exists. +### `--prefix` — scripts whose job is placing files + +Shell redirections (`cat > /etc/modprobe.d/x.conf`) are builtins, so no `PATH` +stub can intercept them. Stub `cp`/`chmod`/`ln` and the trace is nearly empty; +don't, and the run rewrites the live system. `--prefix` (added 2026-08-16) +resolves that in two layers — see `meta/scripts/lib/trace_prefix.sh`: + +```bash +# $HOME-rooted destinations: no bwrap, no root needed +meta/scripts/trace_shell_split.sh install_usage_monitoring.sh --prefix /tmp/pfx + +# hardcoded absolutes: bind each LEAF directory +meta/scripts/trace_shell_split.sh nvidia_troubleshoot.sh --prefix /tmp/pfx \ + --bind-abs /etc/modprobe.d --bind-abs /etc/X11 --stub git,lsmod,nvidia-smi +``` + +`HOME` **and every `XDG_*` base** are redirected. Exporting the XDG vars +explicitly is not belt-and-braces: `install_leechblock.sh` reads +`XDG_DATA_HOME` directly and then runs `rsync -a --delete` at it, so inheriting +the caller's real value would delete live data. + +The trace gains a `=== files written (prefix)` section listing every file with +its size and a content hash. **That section is the point**: without it a +dropped `cat >` shows up as nothing at all — same exit status, same stubbed +calls, same stdout — and the traces match while one lost a file. Measured on +the `dropped_write_{before,after}.sh` fixture pair: with the manifest the lost +`.timer` is a one-line diff; with the section stripped the two traces are +byte-identical, i.e. the pre-`--prefix` harness passed that broken split. + +Four traps, each of which cost a debugging round: + +- **Bind leaf directories, never `/etc`.** Binding `/etc` wholesale shadows + `/etc/passwd` and `/etc/os-release`. Measured: `$SUDO_USER` lookup collapsed + to `/home//pyroveil`, `/etc/profile` became "No such file or directory", and + a nested `/etc/X11` bind was masked so its write vanished from the manifest. + All three look like a broken split. The scanner therefore never emits a bare + `/etc`. +- **`require_root` truncates a stubbed run to nothing.** `lib/common.sh` does + `exec sudo "$0" "$@"`; under the `sudo` stub that records one line and exits + 0 — a three-line trace that diffs clean against _any_ split. `--bind-abs` + runs under `bwrap --unshare-user --uid 0`, so `$EUID` is 0 and the script + proceeds for real. Without a bind there is no uid 0 and the truncation is + silent, so check the trace actually entered the code you moved. +- **Unquoted heredocs bake the prefix into the artifact.** + `install_usage_monitoring.sh:222` uses `<< SCRIPT`, so `$HOME` interpolates + at write time. Hashes are taken over content with the prefix normalized to + `@PREFIX@`; without that, a `mktemp -d` prefix makes every run differ. +- **A timestamp the target embeds still varies.** `nvidia_troubleshoot.sh` + writes `$(date)` into its config, so its hash changes every run; compare the + file list and sizes. Don't freeze the clock — that would hide real changes. + +Anything absolute that was _not_ bound fails read-only/EPERM rather than +succeeding, so Decision 6 holds by construction. `--prefix` **refuses to run** +when the target writes to absolute paths and no `--bind-abs` was given, listing +the paths it found; the scan follows variables +(`MODPROBE_DIR="/etc/modprobe.d"` → `cat >"$CONFIG_FILE"`), because a +literal-only scan found nothing and produced a silent empty manifest. + +Verified untouched after the runs above: `~/.config/systemd/user`, +`~/.local/bin`, `~/.local/share`, `/etc/modprobe.d` (identical listings), and +`systemctl --user list-unit-files | grep -c usage-report` unchanged at 2. + +`block_compulsive_opening.sh` remains out of scope: `install_all` copies the +running script into `/usr/local/bin`, and the handoff records that a bad split +there breaks three daily-use apps plus the pacman rewrap hook. + +### Regression fixtures + +`meta/scripts/fixtures/` holds runnable reproductions of every trap above, so a +harness change can be checked instead of trusted: + +| Fixture | Reproduces | +| --------------------------------- | ----------------------------------------------- | +| `set_e_function_tail.sh` | exit 1, later calls missing from the trace | +| `circular_nameref.sh` | exit 0, correct stdout, warning only in stderr | +| `valued_stub_branch.sh` | the silent-stub trap (`--stub du` vs `du=4096`) | +| `dropped_write_{before,after}.sh` | a split that silently loses one write | + +A lint fix once broke one of these invisibly: adding a trailing `echo` to +satisfy SC2034 made _that_ echo the function's last statement, so the fixture +exited 0 and reproduced nothing while still "passing". Re-run the fixtures +after touching them, and check the exit status is still the one documented. **Run the baseline from the script's own directory** when it reads relative paths. `fresh-install/main.sh` reads `aur_packages.txt` and diff --git a/docs/superpowers/contracts/trace-prefix-writes-2026-08-16.json b/docs/superpowers/contracts/trace-prefix-writes-2026-08-16.json new file mode 100644 index 00000000..bdce6117 --- /dev/null +++ b/docs/superpowers/contracts/trace-prefix-writes-2026-08-16.json @@ -0,0 +1,22 @@ +{ + "title": "Make write-to-system installers traceable with a prefix redirect", + "objective": "Extend meta/scripts/trace_shell_split.sh so scripts whose job is placing files can be executed safely and diffed. Shell redirections are builtins and cannot be intercepted by PATH stubs, so this class of script was previously unverifiable: stubbing cp/chmod/ln yields a near-empty trace, and not stubbing them rewrites the live system. This is the harness change that unblocks those splits; it is explicitly not a split.", + "acceptance_criteria": [ + "trace_shell_split.sh can execute install_usage_monitoring.sh and nvidia_troubleshoot.sh and produce a diffable trace with the same format as before: exit status, ordered stubbed calls with arguments, stdout, stderr.", + "The trace SHOWS each redirected write with its destination and a content hash, so a dropped write is visible; a redirect that hid its writes would recreate the silent-stub trap it is meant to avoid.", + "The real ~/.config/systemd/user/, ~/.local/bin/ and /etc/modprobe.d/ are provably untouched after those runs, compared against snapshots taken before the first run.", + "A fixture pair proves the trace diff DETECTS a dropped write, and that the same diff without the manifest section is identical - i.e. that the harness catches a real break rather than merely running.", + "The three existing regression fixtures (set -e function-tail abort, self-referencing nameref, valued-stub branch) still behave as documented; they are committed as runnable files rather than left ad-hoc.", + "Traces captured WITHOUT the new flag remain byte-identical to previously captured ones, so the change does not invalidate existing baselines.", + "shellcheck and shfmt -d clean on every changed shell file, with findings fixed rather than suppressed.", + "Every changed file stays under the 250-line cap, including trace_shell_split.sh itself - the harness is not exempt from the rule it serves." + ], + "out_of_scope": [ + "Splitting any script. No target is split this session; the harness lands first and is proven first.", + "block_compulsive_opening.sh, whose install_all copies the running script into /usr/local/bin and which the handoff records as breaking three daily-use apps plus the pacman rewrap hook if split badly.", + "Making the sudo stub execute its argument. That would change every existing trace and turn sudo from a safety barrier into a passthrough; it is recorded as a named limitation instead.", + "Wiring the file-length pre-commit hook, which must land last, only once check_file_length.sh --all exits 0.", + "Weakening the gate in any way: no suppressions, no exclude entries." + ], + "verifier": "meta/scripts/trace_shell_split.sh install_usage_monitoring.sh --prefix (manifest lists all 5 artifacts), then diff of before/after snapshots of ~/.config/systemd/user, ~/.local/bin, ~/.local/share, /etc/modprobe.d plus systemctl --user list-unit-files | grep -c usage-report; diff of traces of fixtures/dropped_write_before.sh and dropped_write_after.sh (must differ by exactly the missing .timer line, and must be identical once the manifest section is stripped); the three regression fixtures re-run and checked against their documented exit statuses; shellcheck and shfmt -d on all changed shell files; wc -l under 250 for each." +} diff --git a/docs/superpowers/evidence/trace-prefix-writes-2026-08-16.json b/docs/superpowers/evidence/trace-prefix-writes-2026-08-16.json new file mode 100644 index 00000000..1fd7daf2 --- /dev/null +++ b/docs/superpowers/evidence/trace-prefix-writes-2026-08-16.json @@ -0,0 +1,73 @@ +{ + "intent": "Make write-to-system installer scripts traceable, so splits of them can be verified by running them and diffing, without mutating the real system. Before this, scripts whose job is placing files were the one class of remaining file-length violations that could not be verified at all: shell redirections (cat > /etc/...) are builtins and cannot be intercepted by the existing PATH stubs, so the choice was a near-empty trace or a rewritten live system.", + "scope": [ + "meta/scripts/trace_shell_split.sh - adds --prefix and --bind-abs; drops dead EUID_OVERRIDE; resolves SCRIPT_DIR through readlink -f", + "meta/scripts/lib/trace_prefix.sh - new: env redirect, absolute-path scan, bwrap argv, write manifest", + "meta/scripts/lib/trace_stubs.sh - new: stub generation extracted so the harness stays under the 250-line cap", + "meta/scripts/fixtures/ - new: five runnable fixtures (three existing regressions + a dropped-write pair)", + "docs/shell-split-verification.md - replaces the 'not yet tractable' paragraph with the --prefix section and its traps", + "NON-GOAL: no script was split this session; this is the harness change that unblocks those splits", + "NON-GOAL: block_compulsive_opening.sh stays out of scope (copies the running script into /usr/local/bin)" + ], + "changes": [ + "--prefix redirects HOME and every XDG_* base into a temp tree. XDG_DATA_HOME is exported explicitly rather than left to default off HOME, because install_leechblock.sh reads it directly and then runs `rsync -a --delete` at it; inheriting the caller's real value would delete live data.", + "--bind-abs binds hardcoded absolute destinations into the prefix via bwrap --dev-bind / / --unshare-user --uid 0. Only leaf directories are bound. uid 0 also makes require_root() pass, which matters because lib/common.sh does `exec sudo \"$0\"` and under the sudo stub that truncates the entire run to one recorded call.", + "The trace gains a '=== files written (prefix)' section: every file with size and a content hash. Emitted only under --prefix, so traces taken without the flag stay byte-identical to previously captured ones.", + "Hashes are computed over content with the prefix path normalized to a literal @PREFIX@, because install_usage_monitoring.sh:222 is an UNQUOTED heredoc that interpolates $HOME into the artifact at write time.", + "The harness refuses to run when the target writes to absolute paths and no --bind-abs was given, listing the paths found. The scan follows variable assignments (MODPROBE_DIR=\"/etc/modprobe.d\" then `cat >\"$CONFIG_FILE\"`), and never emits a bare /etc.", + "Stub generation moved to lib/trace_stubs.sh: --prefix pushed trace_shell_split.sh to 252 lines, over the cap it exists to serve. Now 197." + ], + "verification": [ + { + "command": "trace_shell_split.sh install_usage_monitoring.sh --prefix ", + "result": "pass", + "evidence": "exit 0; manifest lists all 5 artifacts (.config/systemd/user/{nvidia-pmon.service,usage-report-catchup.service,usage-report-catchup.timer}, .local/bin/{nvidia-pmon-logger.sh,usage-report-catchup.sh}) with sizes and hashes." + }, + { + "command": "ls -la ~/.config/systemd/user ~/.local/bin ~/.local/share /etc/modprobe.d; systemctl --user list-unit-files | grep -c usage-report", + "result": "pass", + "evidence": "Diffed before/after snapshots: all four listings byte-identical; usage-report unit count unchanged at 2. The live /etc/modprobe.d/nvidia-gsp-disable.conf and /etc/X11/xorg.conf.d/20-nvidia.conf are dated 2025-06-26 and their content differs from what the traced run generated, confirming no write escaped the prefix." + }, + { + "command": "trace_shell_split.sh nvidia_troubleshoot.sh --prefix --bind-abs /etc/modprobe.d --bind-abs /etc/X11 --stub git,lsmod,nvidia-smi", + "result": "pass", + "evidence": "Ran to section 3 under uid 0 instead of truncating at require_root; both /etc writes landed in the prefix. Unbound /etc/profile failed 'Permission denied' rather than writing live - fail-closed." + }, + { + "command": "diff trace(dropped_write_before.sh) trace(dropped_write_after.sh)", + "result": "pass", + "evidence": "Dropped .timer write detected as a one-line diff. With the manifest section stripped, the two traces are IDENTICAL - i.e. the pre-change harness would have passed that broken split. This is the proof the harness catches a real break, not just that it runs." + }, + { + "command": "trace_shell_split.sh fixtures/{set_e_function_tail,circular_nameref,valued_stub_branch}.sh", + "result": "pass", + "evidence": "R1 exit 1 with systemctl absent from the trace; R2 exit 0 + 4 'circular name reference' lines in stderr; R3 --stub du -> 'branch skipped', 0 sudo calls captured; --stub du=4096 -> 'BRANCH TAKEN', sudo rm captured." + }, + { + "command": "trace_shell_split.sh install_usage_monitoring.sh --prefix vs ", + "result": "pass", + "evidence": "Traces byte-identical across two different random prefix paths, so before/after diffing works. nvidia_troubleshoot.sh's hash varies only because the target embeds $(date) in its output; documented rather than papered over." + }, + { + "command": "shellcheck + shfmt -d on all changed shell files", + "result": "pass", + "evidence": "Both clean with zero suppressions. SC2034 and SC2178 in the fixtures were fixed by restructuring (reading both flags before the failing tail; declaring the array indirectly), not by # shellcheck disable." + }, + { + "command": "wc -l on changed files", + "result": "pass", + "evidence": "trace_shell_split.sh 197, lib/trace_prefix.sh 166, lib/trace_stubs.sh 71, docs/shell-split-verification.md 230 - all under the 250-line cap." + } + ], + "risks": [ + "The absolute-path scan is static and deliberately over-reports (a variable holding a system path may only ever be read). Over-reporting costs a redundant --bind-abs; under-reporting would cost a live-system write, so the bias is chosen.", + "A target that computes a destination at runtime (from command substitution or an argument) would be missed by the scan. The bwrap layer still fails such a write closed rather than letting it through, but the refusal message would not name the path.", + "--bind-abs without bwrap, or with unprivileged user namespaces disabled, errors out. Verified present on this machine; the error names the package.", + "A fixture can be neutered by an edit that still 'passes' - this happened during this change, when adding a trailing echo to satisfy SC2034 made that echo the function's last statement and turned exit 1 into exit 0. Documented in the doc's fixture table." + ], + "rollback": [ + "git revert the commit; trace_shell_split.sh returns to the stub-only harness and the lib/ and fixtures/ files disappear.", + "Traces captured WITHOUT --prefix remain valid across the revert: the manifest section and the prefix normalization are both gated on the flag, so default output is unchanged.", + "After rollback, re-run the three regression fixtures from a checkout that still has them (or the commands in docs/shell-split-verification.md) to confirm the stub path still behaves." + ] +} diff --git a/meta/scripts/fixtures/circular_nameref.sh b/meta/scripts/fixtures/circular_nameref.sh new file mode 100755 index 00000000..b6b80e5c --- /dev/null +++ b/meta/scripts/fixtures/circular_nameref.sh @@ -0,0 +1,32 @@ +#!/usr/bin/env bash + +# Regression fixture: the self-referencing nameref. +# +# `local -n LANG_CODE_FILES="$1"` pointing at a global of the same name still +# moves data on bash 5.3, but warns `circular name reference` on EVERY access, +# straight into the script's output. +# +# Traced, this shows up as the warning in stderr despite exit 0 and correct +# stdout -- which is why stderr belongs in the trace and not just the status. + +set -euo pipefail + +# The array is declared and populated indirectly. Declaring it as a literal +# `declare -A LANG_CODE_FILES=(...)` lets shellcheck tie the global to the +# same-named `local -n` below and raise SC2178, and suppressing that is barred. +# The runtime shape under test is untouched: at execution time there is still a +# global associative array whose name the local nameref reuses, which is what +# produces the circular-reference warning. +CODE_FILES_MAP="LANG_CODE_FILES" +declare -A "$CODE_FILES_MAP" +printf -v "${CODE_FILES_MAP}[python]" '%s' 7 + +count_files() { + # The bug: a local nameref bound to a global of the SAME name. bash still + # moves the data, but warns `circular name reference` on every access. + local -n LANG_CODE_FILES="$1" + echo "python=${LANG_CODE_FILES[python]}" +} + +count_files "$CODE_FILES_MAP" +echo "exit 0 with correct stdout, but stderr carries the warning" diff --git a/meta/scripts/fixtures/dropped_write_after.sh b/meta/scripts/fixtures/dropped_write_after.sh new file mode 100755 index 00000000..83dd8197 --- /dev/null +++ b/meta/scripts/fixtures/dropped_write_after.sh @@ -0,0 +1,35 @@ +#!/usr/bin/env bash + +# Fixture: the BROKEN split of dropped_write_before.sh. +# +# The .timer write was lost -- the kind of thing that happens when a heredoc +# falls across a seam during a split. Everything else is identical: same exit +# status, same stubbed systemctl call, same stdout "install complete". +# +# Diffing a --prefix trace of this against its partner must show the missing +# demo-agent.timer line. That diff is the whole point of the manifest. + +set -euo pipefail + +bin_dir="$HOME/.local/bin" +unit_dir="$HOME/.config/systemd/user" +mkdir -p "$bin_dir" "$unit_dir" + +cat >"$bin_dir/demo-agent.sh" <<'SCRIPT' +#!/usr/bin/env bash +echo "demo agent running" +SCRIPT +chmod +x "$bin_dir/demo-agent.sh" + +cat >"$unit_dir/demo-agent.service" <<'UNIT' +[Unit] +Description=Demo agent + +[Service] +ExecStart=%h/.local/bin/demo-agent.sh +UNIT + +# The .timer heredoc that belongs here was dropped by the split. + +systemctl --user daemon-reload +echo "install complete" diff --git a/meta/scripts/fixtures/dropped_write_before.sh b/meta/scripts/fixtures/dropped_write_before.sh new file mode 100755 index 00000000..8ba0fefe --- /dev/null +++ b/meta/scripts/fixtures/dropped_write_before.sh @@ -0,0 +1,39 @@ +#!/usr/bin/env bash + +# Fixture: the pre-split installer. Places three files under $HOME. +# +# Its partner, dropped_write_after.sh, is the "split" version that silently +# loses one of them -- the exact failure --prefix exists to catch. Without the +# manifest section, a dropped `cat >` shows up as NOTHING at all: same exit +# status, same stubbed calls, same stdout. The traces match and the bug ships. + +set -euo pipefail + +bin_dir="$HOME/.local/bin" +unit_dir="$HOME/.config/systemd/user" +mkdir -p "$bin_dir" "$unit_dir" + +cat >"$bin_dir/demo-agent.sh" <<'SCRIPT' +#!/usr/bin/env bash +echo "demo agent running" +SCRIPT +chmod +x "$bin_dir/demo-agent.sh" + +cat >"$unit_dir/demo-agent.service" <<'UNIT' +[Unit] +Description=Demo agent + +[Service] +ExecStart=%h/.local/bin/demo-agent.sh +UNIT + +cat >"$unit_dir/demo-agent.timer" <<'UNIT' +[Unit] +Description=Demo agent timer + +[Timer] +OnCalendar=hourly +UNIT + +systemctl --user daemon-reload +echo "install complete" diff --git a/meta/scripts/fixtures/set_e_function_tail.sh b/meta/scripts/fixtures/set_e_function_tail.sh new file mode 100755 index 00000000..701d6072 --- /dev/null +++ b/meta/scripts/fixtures/set_e_function_tail.sh @@ -0,0 +1,32 @@ +#!/usr/bin/env bash + +# Regression fixture: the `set -e` function-tail abort. +# +# Reproduces the analyze_repo.sh bug. Eight `((MAP[x] > 0)) && HAS_Y=true` lines +# were top-level statements, where a false `&&` merely leaves $? non-zero. +# Wrapped into a function they become its RETURN VALUE, and the last one -- +# false for most inputs -- kills the script under set -e. +# +# Traced, this shows up as exit 1 with every later call missing from the trace. + +set -euo pipefail + +declare -A LANG_FILES=([python]=3 [java]=0) + +detect_languages() { + local HAS_PYTHON=false HAS_JAVA=false + ((LANG_FILES[python] > 0)) && HAS_PYTHON=true + # Read both flags BEFORE the failing tail, not after. An earlier revision + # put `echo "java=$HAS_JAVA"` at the end to satisfy SC2034 -- which made + # that echo the function's last statement, gave it exit 0, and silently + # deleted the very bug this fixture exists to reproduce. The trace went + # from exit 1 to exit 0 and the check passed while testing nothing. Keep + # the bare conditional last. + echo "python=$HAS_PYTHON java=$HAS_JAVA" + # The tail. False, so the function returns 1 and set -e aborts the script. + ((LANG_FILES[java] > 0)) && HAS_JAVA=true +} + +detect_languages +echo "reached the line after detect_languages" +systemctl status something diff --git a/meta/scripts/fixtures/valued_stub_branch.sh b/meta/scripts/fixtures/valued_stub_branch.sh new file mode 100755 index 00000000..f3e38f2f --- /dev/null +++ b/meta/scripts/fixtures/valued_stub_branch.sh @@ -0,0 +1,27 @@ +#!/usr/bin/env bash + +# Regression fixture: the silent-stub trap. +# +# `du` stubbed to print NOTHING makes size="" , so `((size > 0))` is false, +# the guarded branch is skipped, and two traces match perfectly while +# exercising none of the code under test. +# +# trace_shell_split.sh valued_stub_branch.sh --stub du +# -> size=[] / branch skipped, never reaches the guarded sudo rm -rf +# trace_shell_split.sh valued_stub_branch.sh --stub 'du=4096' +# -> size=[4096] / BRANCH TAKEN, and the sudo call is captured +# +# The point of keeping this: an empty stub is not a passing test, it is a test +# that never ran. + +set -euo pipefail + +size="$(du -sk /some/dir 2>/dev/null | cut -f1)" +echo "size=[$size]" + +if ((${size:-0} > 0)); then + echo "BRANCH TAKEN" + sudo rm -rf /some/dir/cache +else + echo "branch skipped" +fi diff --git a/meta/scripts/lib/trace_prefix.sh b/meta/scripts/lib/trace_prefix.sh new file mode 100755 index 00000000..224c4de6 --- /dev/null +++ b/meta/scripts/lib/trace_prefix.sh @@ -0,0 +1,166 @@ +#!/usr/bin/env bash + +# ============================================================================ +# Prefix redirection for trace_shell_split.sh: run a script whose job is +# PLACING FILES without letting it touch the live system, and make every write +# it performs visible in the trace. +# +# Why a separate mechanism from the PATH stubs: `cat > /etc/modprobe.d/x.conf` +# is a bash builtin plus a redirection. No PATH entry can intercept it. The +# stubs handle mutating *binaries*; this file handles mutating *redirections*. +# +# Two layers, applied together by build_prefix_argv: +# +# 1. Environment redirect. HOME and every XDG_* base are pointed into the +# prefix. This covers the majority of real destinations, which are +# $HOME-rooted variables (`unit_dir="$HOME/.config/systemd/user"`), with +# NO change to the traced script. XDG_DATA_HOME is exported explicitly +# rather than left to default off HOME: install_leechblock.sh reads it +# directly and then runs `rsync -a --delete` at it, so inheriting a real +# XDG_DATA_HOME from the caller's environment would delete live data. +# +# 2. bwrap bind mounts for the hardcoded-absolute residue (/etc/modprobe.d, +# /usr/local/bin, /etc/systemd/system). Only the LEAF directories named on +# the command line are bound -- binding /etc wholesale would shadow +# /etc/passwd and /etc/os-release and produce fake failures that look like +# a broken split. `--unshare-user --uid 0` additionally makes the run +# believe it is root, which matters more than it sounds: the shared +# require_root() does `exec sudo "$0"`, and under the sudo stub that +# truncates the whole run to a single recorded call -- a near-empty trace +# that diffs clean against any split at all. +# +# Anything absolute that was NOT bound fails EPERM / read-only rather than +# silently succeeding, so Decision 6 (never mutate the real system to verify a +# split) holds by construction rather than by discipline. +# ============================================================================ + +# XDG bases redirected into the prefix. Each is exported explicitly so a value +# already present in the caller's environment cannot leak a live path through. +readonly TRACE_XDG_VARS=( + XDG_DATA_HOME:.local/share + XDG_CONFIG_HOME:.config + XDG_CACHE_HOME:.cache + XDG_STATE_HOME:.local/state +) + +# Populate the prefix with the directory skeleton and echo the env assignments +# needed to redirect a run into it. Caller exports them. +trace_prefix_env() { + local prefix="$1" entry var rel + mkdir -p "$prefix" + printf 'HOME=%s\n' "$prefix" + for entry in "${TRACE_XDG_VARS[@]}"; do + var="${entry%%:*}" + rel="${entry#*:}" + mkdir -p "$prefix/$rel" + printf '%s=%s\n' "$var" "$prefix/$rel" + done +} + +# Absolute destinations a script writes to, discovered statically. Used to tell +# the user which --bind-abs arguments a target needs, and to refuse clearly +# when bwrap cannot provide them. +# +# Both spellings must be caught, and the second is the one that bites: +# nvidia_troubleshoot.sh writes `cat >"$CONFIG_FILE"`, where CONFIG_FILE is +# built from MODPROBE_DIR="/etc/modprobe.d" many lines earlier. A scan that +# only looked for a literal path next to the `>` found nothing, the refusal +# never fired, and the run silently produced an empty manifest -- a false pass +# of exactly the kind this harness exists to prevent. So we also collect any +# variable ASSIGNED an absolute path under a system root, which over-reports +# (a path may only ever be read) but never under-reports. Over-reporting costs +# a redundant --bind-abs; under-reporting costs a live-system write. +trace_prefix_scan_absolute() { + local target="$1" + # `|| true` on each grep: a target with only one spelling of destination + # makes the other grep exit 1, and under the caller's `set -e` that aborted + # validate_requirements silently -- the refusal never fired and the run + # proceeded with an empty manifest. Exactly the false pass being guarded + # against, so the status is neutralised rather than relied upon. + { + # Literal destination directly after a redirection. + { grep -oE '>[[:space:]]*"?(/etc|/usr/local|/usr/share|/var/lib|/opt)[^"[:space:]]*' "$target" || true; } | + sed -E 's/^>[[:space:]]*"?//' | + xargs -r -n1 dirname + # Absolute system paths assigned to a variable, which redirections and + # `tee`/`mkdir` then reach indirectly. + { grep -oE '^[[:space:]]*(local[[:space:]]+)?[A-Za-z_][A-Za-z0-9_]*="?(/etc|/usr/local|/usr/share|/var/lib|/opt)[^"[:space:]]*' "$target" || true; } | + sed -E 's/^[[:space:]]*(local[[:space:]]+)?[A-Za-z_][A-Za-z0-9_]*="?//' | + while IFS= read -r path; do + # Emit the path itself, never its parent. Taking dirname of a + # config *directory* is what turned MODPROBE_DIR="/etc/modprobe.d" + # into a bare `/etc`, and binding /etc wholesale shadows + # /etc/passwd and /etc/os-release -- the traced script then fails + # on an unrelated read and it looks like a broken split. A + # variable holding a file path yields a directory that does not + # exist inside the prefix, which bwrap creates on demand; that + # is harmless, whereas an over-broad bind is not. + printf '%s\n' "$path" + done + } | sort -u +} + +# Build the argv that runs "$@" with the absolute destinations bound into the +# prefix. Echoes the command; empty output means "no bwrap needed". +# +# Each bound path is created inside the prefix first: bwrap requires the source +# of a --bind to exist. +trace_prefix_bwrap_argv() { + local prefix="$1" + shift + local -a abs_dirs=() + while [[ $# -gt 0 && $1 != "--" ]]; do + abs_dirs+=("$1") + shift + done + shift # drop the -- + + if [[ ${#abs_dirs[@]} -eq 0 ]]; then + return 0 + fi + + if ! command -v bwrap >/dev/null 2>&1; then + echo "Error: --bind-abs needs bubblewrap (pacman -S bubblewrap)" >&2 + return 1 + fi + + local -a argv=(bwrap --dev-bind / / --unshare-user --uid 0 --gid 0) + local dir + for dir in "${abs_dirs[@]}"; do + mkdir -p "$prefix/abs$dir" + argv+=(--bind "$prefix/abs$dir" "$dir") + done + printf '%s\n' "${argv[@]}" +} + +# The write manifest: what the run actually placed, with content hashes. +# +# This is the half of the change that satisfies "a silent stub is a trap". +# Redirecting a write without reporting it would recreate exactly the failure +# the doc warns about -- a dropped `cat > $HOME/.local/bin/foo.sh` would show +# up as nothing at all, and two traces would match while one lost a file. +# +# Content is normalized before hashing: the prefix path is replaced with a +# literal @PREFIX@. install_usage_monitoring.sh writes an UNQUOTED heredoc, so +# $HOME interpolates into the generated file's contents at write time; without +# normalization a mktemp-d prefix bakes a fresh random path into every artifact +# and no two traces ever match. +# +# What normalization does NOT remove is a timestamp the target itself embeds: +# nvidia_troubleshoot.sh writes `# Created by ... on $(date)`, so its hash +# changes every run and only its size is stable. That is a property of the +# traced script, not of the harness -- when diffing such a target, expect the +# sha to differ and compare the file LIST and sizes instead. Do not "fix" it by +# freezing the clock; that would hide a real content change too. +trace_prefix_manifest() { + local prefix="$1" rel abs hash size + if [[ ! -d $prefix ]]; then + return 0 + fi + while IFS= read -r rel; do + abs="$prefix/$rel" + size="$(stat -c '%s' "$abs")" + hash="$(sed "s#${prefix}#@PREFIX@#g" "$abs" | sha256sum | cut -c1-16)" + printf '%s size=%s sha=%s\n' "$rel" "$size" "$hash" + done < <(find "$prefix" -type f -printf '%P\n' | sort) +} diff --git a/meta/scripts/lib/trace_stubs.sh b/meta/scripts/lib/trace_stubs.sh new file mode 100755 index 00000000..2acc63e3 --- /dev/null +++ b/meta/scripts/lib/trace_stubs.sh @@ -0,0 +1,72 @@ +#!/usr/bin/env bash + +# ============================================================================ +# PATH stub generation for trace_shell_split.sh. +# +# Split out of trace_shell_split.sh when --prefix pushed that file past the +# 250-line cap. The harness is not exempt from the rule it exists to serve. +# +# These stubs shadow mutating BINARIES. Mutating REDIRECTIONS (`cat > /etc/...`) +# cannot be intercepted this way at all and are handled by lib/trace_prefix.sh. +# ============================================================================ + +# Commands that change the system, the phone, or the package set. Each becomes +# a stub that records its invocation and exits 0. Extend deliberately: a +# missing name here means the real binary runs. +# +# Deliberately NOT here: `git`, `curl`, `wget`, `makepkg`. Those are stubbed +# per-run via --stub, because plenty of scripts read git state harmlessly and a +# blanket git stub would change what they see rather than protect anything. +# Grep the target for network and build verbs before tracing it. +readonly DEFAULT_STUBBED_COMMANDS=( + sudo pacman yay paru systemctl systemd-run + adb fastboot + nft iptables ip6tables firewall-cmd + mount umount swapoff modprobe + useradd usermod visudo chpasswd + mkinitcpio grub-mkconfig bootctl + npm pip pip3 flutter gradle + reboot shutdown poweroff halt +) + +# One stub per mutating command: append the call to the trace, succeed. +# Stubs report success because the point is to reach later code paths, not to +# simulate failure. A script whose logic branches on a real exit status needs a +# hand-written stub instead -- note that in the split's evidence file. +# +# A stub may carry an output value as `name=text` (from --stub). This matters +# more than it looks: a stub that prints nothing makes every `size=$(du ...)` +# empty, so every `((size > 0))` is false and the trace walks straight past all +# the branches you wanted to compare. Two such traces are identical and +# meaningless. Give value-producing commands a plausible non-zero output. +# +# Extra stubs are passed as the trailing arguments rather than read from a +# global, so this stays callable independently of the caller's variables. +write_stubs() { + local bin_dir="$1" entry name value + shift + mkdir -p "$bin_dir" + for entry in "${DEFAULT_STUBBED_COMMANDS[@]}" "$@"; do + name="${entry%%=*}" + value="" + if [[ $entry == *=* ]]; then + value="${entry#*=}" + fi + # Quoted heredoc: everything is literal, so $* and $TRACE_FILE reach + # the generated stub instead of expanding here. Only $name and $value + # are interpolated, via the unquoted echo lines around it. + { + echo "#!/usr/bin/env bash" + echo "STUB_NAME=$(printf '%q' "$name")" + echo "STUB_VALUE=$(printf '%q' "$value")" + cat <<'STUB' +printf '%s %s\n' "$STUB_NAME" "$*" >>"$TRACE_FILE" +if [[ -n $STUB_VALUE ]]; then + printf '%s\n' "$STUB_VALUE" +fi +exit 0 +STUB + } >"$bin_dir/$name" + chmod +x "$bin_dir/$name" + done +} diff --git a/meta/scripts/trace_shell_split.sh b/meta/scripts/trace_shell_split.sh index dbd2524a..f7f6e6a5 100755 --- a/meta/scripts/trace_shell_split.sh +++ b/meta/scripts/trace_shell_split.sh @@ -19,10 +19,15 @@ # is never touched. That is what makes this safe for the enforcement and # installer scripts Decision 6 forbids executing for real. # +# Scripts whose job is PLACING FILES need --prefix: shell redirections cannot +# be stubbed via PATH, so without it the choice is an empty trace or a rewritten +# live system. See lib/trace_prefix.sh for the two layers that solves. +# # Usage: # trace_shell_split.sh