mirror of
https://github.com/kuhyx/testsAndMisc.git
synced 2026-11-11 14:27:49 +01:00
Krzysztof kuhy Rudnicki
0091608b08
Two defects found in review, both of which made checks PASS more easily, so neither ever went red. GATE: is_covered() hardcoded its jail flags and never read the jail_args marker that ci_mirror.sh and shell-tests.yml both consume. The features/lib suite needs /usr/local/share and /etc/pki and aborts under set -e without them -- but the coverage number still printed, because kcov had already recorded whatever the dead run reached. The gate would have graded 44 libs on a suite that never finished. It now reads the marker, keeps the historical defaults for suites without one, fails closed on an empty marker, and passes --fail-on-case-error. Negative control: stripping the binds makes the gate exit 1 and restoring them returns exit 0. Before this commit both cases exited 0 -- the gate could not fail on an aborting suite. HASH CACHE: the _t_unstub helper written earlier today for the transcribe harness was needed in features_harness.sh too. Two assertions were vacuous -- test_dwm_config's "verify still reports when dwm is absent", and test_rpi_nc_ca's uname removal, where a cached uname left every later branch running under a faked Darwin. All four `rm -f "$TEST_TMPDIR/bin/` sites in the repo were audited; the other two are sound (one probes with [[ -f ]], the other runs in a subshell). No measured figure moved: dot_resolver_install.sh still 39/39 = 100.00% with real absence enforced, so its off-allowlist clearance holds. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016sxxsDHWgdRhmdmzNMA6XM
23 lines
2.5 KiB
JSON
23 lines
2.5 KiB
JSON
{
|
|
"title": "Make the coverage gate read jail_args and refuse a suite that dies partway",
|
|
"objective": "Close the two review-found defects that let checks pass more easily than they should: is_covered() ignored the jail_args marker every other consumer reads, and bash's command hash cache made two 'tool is missing' assertions test the opposite of their names.",
|
|
"acceptance_criteria": [
|
|
"is_covered() reads the jail_args file beside a suite's run_all.sh, the same marker ci_mirror.sh and shell-tests.yml consume, so a suite declares its own mounts in exactly one place.",
|
|
"A suite with no jail_args still gets the historical default binds, so nothing that passed before starts failing for lack of a marker.",
|
|
"A jail_args that is present but empty is a hard error, never a silent skip.",
|
|
"is_covered() passes --fail-on-case-error, so a suite that aborts partway fails the gate rather than being graded on the lines it reached before dying.",
|
|
"NEGATIVE CONTROL: stripping the binds a suite needs must make the gate exit non-zero. Before this change both the working and the broken configuration exited 0.",
|
|
"dot_resolver_install.sh, which is already off the allowlist, still clears the real gate afterwards -- the campaign's proof that 100% is reachable must survive the stricter check.",
|
|
"Every 'tool is missing' assertion genuinely hides the tool: removal clears bash's hash table via a shared _t_unstub helper.",
|
|
"No measured coverage figure moves as a result of either fix, and every existing assertion still passes.",
|
|
"shellcheck clean with no suppressions; all files within the 250-line cap."
|
|
],
|
|
"out_of_scope": [
|
|
"test_cco_state.sh's removal of beeper.orig: get_real_binary probes with [[ -f ]], which the hash cache does not affect.",
|
|
"test_aw_autostart.sh's removal of aw-qt: the assertion runs in a subshell via $status_script, so the parent shell's cache is irrelevant.",
|
|
"Arming check_shell_coverage.sh in any hook; that waits for allowlist-zero.",
|
|
"The kcov under-report, which is tracked separately in docs/shell-split-verification.md."
|
|
],
|
|
"verifier": "bash meta/scripts/check_shell_coverage.sh on dot_resolver_install.sh must exit 0; replacing that directory's jail_args with `--bind /etc` alone must make the same command exit non-zero, and restoring it must return exit 0 with an empty git diff --stat. Plus the features/lib runner under --fail-on-case-error reporting zero case failures, and all four features libs re-measured at their pre-change figures."
|
|
}
|