{ "intent": "Make the non-vacuity of the session_autopsy hook tests re-verifiable and enforced, instead of a claim in a commit message. 63a5124 asserted '12 tests, 12 killed by at least one mutation', but the harness that proved it lived in a session scratchpad and would have vanished with the session -- leaving an unfalsifiable claim about tests whose whole purpose is to not be vacuous.", "scope": [ "meta/scripts/mutate_session_autopsy_hooks.py (new)", ".pre-commit-config.yaml (one new local hook)", "Non-goal: changing the hooks or the tests. Both are unmodified by this commit." ], "changes": [ "Promoted the scratchpad harness to a repo tool: 8 named mutations, each breaking ONE guard in a copy of the hooks, run against the suite through AUTOPSY_HOOKS_DIR. Fails if any test survives every mutation.", "Per-guard rather than whole-body mutation, because gutting a hook body kills only the 3 positive tests -- a gutted hook emits neither a launch nor output, which is exactly what the 9 negative tests expect. Whole-body gutting alone would falsely brand them vacuous.", "Added a staleness guard: if a mutation leaves the hook text unchanged it raises rather than silently reporting 'killed 0'. A behaviour-preserving edit to a hook (renaming a variable, say) would otherwise turn a mutation into a no-op and quietly weaken the proof.", "Exits 0 when the hooks are not deployed, so a CI runner with no ~/.claude skips rather than fails.", "Wired as a pre-commit hook scoped by files: regex to the suite and the mutation definitions -- the only two files that can invalidate the proof. It takes ~40s, so it must not run on unrelated commits.", "The tool is a gate, not a report: it returns non-zero on survivors, per the 'self-checks must gate, not warn' rule." ], "verification": [ { "command": "python3 meta/scripts/mutate_session_autopsy_hooks.py", "result": "pass", "evidence": "8 mutations; 'PASS: all 12 tests killed by at least one mutation'. Per-mutation kills: gut-end 2, gut-start 1, end-no-transcript-guard 4, start-count-ge-zero 1, start-no-numeric-guard 1, start-no-readable-guard 0, start-always-announce 3, start-unguarded-announce 5." }, { "command": "AUTOPSY_HOOKS_DIR=/nonexistent/hooks python3 meta/scripts/mutate_session_autopsy_hooks.py", "result": "pass", "evidence": "'session_autopsy hooks are not deployed under /nonexistent/hooks -- skipping', exit 0. This is the CI path." }, { "command": "staleness guard: copy the hooks, sed 's/\\bcount\\b/n/g' the start hook (behaviour-identical rename), run against the copy", "result": "pass", "evidence": "The drifted hook still printed the correct banner, confirming the rename preserved behaviour; the tool then failed with \"mutation 'start-count-ge-zero' no longer applies to session_autopsy_start.sh -- the hook text changed, so this mutation tests nothing.\" Text drift is detected without a behaviour break." }, { "command": "pre-commit run --files meta/scripts/mutate_session_autopsy_hooks.py .pre-commit-config.yaml linux_configuration/tests/test_session_autopsy_hooks.py", "result": "pass", "evidence": "All hooks pass, including the new 'session_autopsy hook tests are non-vacuous (mutation)'. Two real lint findings were fixed rather than suppressed: an import inside a function (ruff PLC0415 / pylint C0415) moved to module scope, and unsorted imports. No noqa or type: ignore added." } ], "risks": [ "The gate adds ~40s to any commit touching either of the two scoped files. Deliberate: it is the only thing that can catch the tests silently decaying into exit-status assertions.", "Mutations are string transforms over the hook text, so they are coupled to that text by construction. The staleness guard converts that coupling from a silent weakening into a loud failure, but it does mean editing a hook can require updating a mutation.", "start-no-readable-guard kills nothing on its own; this is a recorded finding, not a gap. The start hook's readability guard is redundant -- jq's stderr is suppressed and an empty count already fails the arithmetic -- so only the compound start-unguarded-announce mutation reaches those cases." ], "rollback": [ "Remove the session-autopsy-hook-mutations hook from .pre-commit-config.yaml and delete meta/scripts/mutate_session_autopsy_hooks.py. No production code or test behaviour is affected; the 12 tests keep running under the ordinary pytest gate.", "After rollback, confirm 'pre-commit run --files linux_configuration/tests/test_session_autopsy_hooks.py' still passes and the suite still reports 12 passed." ] }