The jail recorded each case's exit code but reported only timeouts,
discarding every other non-zero status. A suite with a failing assertion
exited 0 through the push gate -- the same blind spot that let an earlier
session ship 2 failing assertions in the nc_php suite. Found by negative
control, not by reading: a deliberately failing assertion still exited 0.
Propagating that status unconditionally was wrong, and caught before landing:
the jail's own usage example passes a `bogus` case that exits non-zero to
cover an error path, so a clean 5/5 = 100.00% run started failing. That is
the mode the 105-lib campaign runs in.
So the behaviour sits behind --fail-on-case-error, default off, passed only
by ci_mirror.sh and shell-tests.yml. Verified in all four quadrants:
measurement + error case = 0, gate + error case = 1, gate + clean suite = 0,
gate + broken assertion = 1.
Both gates also pass --min 0: with no --measure the jail was measuring
run_all.sh itself against its default --min 100 and failing the push at
54.55%, a coverage verdict neither gate asked for. ci_mirror.sh now prints
the jail's output on failure, so a red gate can be diagnosed.
The added flag pushed the jail over the 250-line cap, so the in-namespace
case runner moved to lib/shell_coverage_case_runner.sh, following the
existing lib/shell_coverage_jail_setup.sh split. All four quadrants were
re-run after the extraction.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016sxxsDHWgdRhmdmzNMA6XM
nc_php.sh was the next-hardest library on the allowlist. It targets a
Debian Raspberry Pi -- it edits /etc/php/<ver>/apache2/php.ini, drives
occ as www-data and installs a crontab -- so none of its write targets
exist on this Arch host and the jail had nothing to mirror.
- --seed-dir / --seed-file create the paths a foreign-platform subject
expects. Verified with a probe: /etc/php/8.2/apache2 exists inside
the jail while /etc/php does not exist outside it.
- A failed bind is now FATAL, and missing mountpoints are created
first. A bind that fails silently is indistinguishable from
unreachable code in the report, which is the misdiagnosis this
runner exists to prevent.
- Mirroring skips unreadable targets (/root as a normal user) rather
than aborting under set -e.
nc_php.sh: 84/85 = 98.82%, 24 assertions. It STAYS on the allowlist,
because 98.82% is below the bar. Line 111 -- the crontab first-run
fallback -- hung the suite on both attempts to cover it, so it is left
uncovered and documented rather than tried a third time.
Two harness hazards found the hard way and written down: a top-level
stub function shadows the subject's own function for every later
assertion (this silently dropped coverage 83.53% -> 57.65% while the
tests still passed), and a stub for a piped-into command must drain
stdin or the writer takes SIGPIPE.
dot_resolver_install.sh still measures 39/39 = 100.00%.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DbD4nco3EgL1UFoZKmKsLE
The gate was a presence check -- 'a run_all.sh exists beside this lib' --
which an empty suite that asserts nothing satisfies. It now measures.
100% is reachable with no suppressions and no production source changes:
run the suite inside a user+mount namespace and the sudo-writes, systemd
units and nftables rules execute for real against a throwaway /etc.
- meta/scripts/shell_coverage_jail.sh: the runner, promoted out of a
session scratchpad. Two fixes it needed on the way: bind targets mirror
the real directory tree (an empty bind makes writes fail with ENOENT,
which reads as unreachable code), and sudo gets a pass-through stub
because real sudo cannot setresuid inside a userns.
- It fingerprints git status before and after and fails the run if a
write escapes the jail. Verified with a deliberate escapee.
- dot_resolver_install.sh: 39/39 = 100.00%, 28 assertions. Picked by
scoring every allowlisted lib on root ops, system writes and
destructive calls -- it scored highest in the repo. Allowlist 106 -> 105.
- jscpd was already over its own threshold at 2.45% and would have
rejected every commit staging a .sh file. Excluding vendored .venv/
shell alone gives 1.67%; also excluding lib/tests/ gives 1.47%.
The gate is deliberately NOT wired into the hook chain yet.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DbD4nco3EgL1UFoZKmKsLE
All three were found by review and reproduced before fixing. Each let a run
report success while proving nothing -- the shape this harness exists to
prevent.
A reused prefix defeated dropped-write detection entirely. Run 1's files were
still there when run 2's manifest was taken, so the manifest reported writes
that never happened and the fixture pair diffed CLEAN. The doc's own examples
demonstrated that workflow. A non-empty prefix is now refused -- not deleted,
since the path is user-supplied -- and the examples use distinct prefixes.
A missing bwrap was swallowed. trace_prefix_bwrap_argv returned 1, but the
caller reads it through `mapfile < <(...)` and a process substitution's status
is discarded, so runner ended up empty and the run continued with no sandbox
and no uid 0. Same swallow as the grep-status one fixed earlier, one layer up.
bwrap is now probed in validate_requirements, before anything runs.
The absolute-path scan only fired when no --bind-abs was given at all, so
binding one path and missing three produced no warning. It now runs whenever
--prefix is given and reports only what is not already covered. Fixing that
surfaced a second defect: --bind-abs on a FILE aborted with "Can't mkdir
/etc/profile: Not a directory", and binding its parent reintroduced the
wholesale /etc shadowing this change had already measured as harmful ($SUDO_USER
collapsing to /home//, the real /etc/profile unreadable). Files are now bound
file-over-file, seeded with a copy so a script that greps before appending sees
real content.
Also measured rather than argued: the XDG redirect. A new fixture run with a
real XDG_DATA_HOME pointing at a decoy shows all four writes landing in the
prefix and the decoy empty -- the rsync --delete data-loss path, proven closed
without running install_leechblock.sh. And no-prefix output was diffed against
cdf2a55 from a detached worktree: byte-identical, so existing baselines hold.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011w2Pj4Ek3ZNMQVYcFHgXDY
Shell redirections are builtins, so no PATH stub can intercept
`cat > /etc/modprobe.d/x.conf`. That left installer scripts unverifiable:
stub cp/chmod/ln and the trace is nearly empty, don't and the run rewrites
the live system.
--prefix redirects HOME and every XDG_* base into a temp tree. XDG_DATA_HOME
is exported explicitly, not left to default off HOME: install_leechblock.sh
reads it directly and then runs `rsync -a --delete` at it.
--bind-abs binds hardcoded absolutes via bwrap --unshare-user --uid 0, leaf
directories only. uid 0 is load-bearing beyond permissions: common.sh's
require_root does `exec sudo "$0"`, and under the sudo stub that truncated
the whole run to one recorded call -- a trace that diffs clean against any
split. Binding /etc wholesale was measured to shadow /etc/passwd, collapse
$SUDO_USER to /home//, and mask a nested bind, so the scanner never emits it.
The trace gains a manifest of every file written, with content hashes. That
section is the point: with it, a dropped write in the fixture pair is a
one-line diff; with it stripped, the two traces are identical, i.e. the old
harness passed that broken split. Hashes normalize the prefix to @PREFIX@
because install_usage_monitoring.sh writes an unquoted heredoc that bakes
$HOME into the artifact.
Verified: ~/.config/systemd/user, ~/.local/bin, ~/.local/share and
/etc/modprobe.d byte-identical before and after; usage-report units still 2.
The three prior regressions are now committed fixtures rather than ad-hoc --
one was silently neutered mid-change when a trailing echo added for SC2034
became the function's last statement and turned exit 1 into exit 0.
Stub generation moved to lib/ because --prefix pushed the harness to 252
lines, over the cap it exists to serve.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011w2Pj4Ek3ZNMQVYcFHgXDY
329 -> 230 + 124. The colour constants, human_readable, dir_size_kb,
report, confirm and try_clean move to lib/disk_cleanup_common.sh, along
with the report-only loop and the summary block as print_report_only and
print_summary.
The seam passes state -- report accumulates into TOTAL_RECLAIMABLE and
confirm reads CLEAN -- so both are set before the source line, and the
verification is a trace diff rather than a static check: identical exit
status, identical ordered du calls, identical stdout and stderr against a
detached worktree at the pre-split commit.
Also ran it for real from /tmp (not its own directory) to confirm
SCRIPT_DIR resolves the lib from any cwd: 303.7 GB reported reclaimable,
nothing deleted. The one shellcheck finding (SC2034 on RED) was fixed by
moving the function that uses it, not suppressed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011w2Pj4Ek3ZNMQVYcFHgXDY
