testsAndMisc/docs/superpowers/evidence/ci-skips-kcov-jail-suites-2026-08-22.json
Krzysztof kuhy Rudnicki 6ebb3462ab CI: skip the kcov-jail suites instead of failing to install kcov
The previous commit added kcov to the shell-tests apt step on the assumption
it was packaged for Ubuntu. It is not -- CI went red in 34s on "E: Unable to
locate package kcov" -- and upstream ships no prebuilt binary either, so it
would have to be built from source on every run.

CI now skips jail_args-marked suites with a printed reason, the same trade
the arch-tests job already makes for Arch-only tests: declare what this
runner cannot do rather than pretend to check it. Enforcement is not lost --
ci_mirror.sh runs those suites through the real jail on every push, and that
gate is already proven to fail on a broken assertion. An empty jail_args is
still a hard failure.

Simulated over all six discovered runners: 5 PASS, 1 SKIP, none dropped.
Committed with CI_GREEN_SKIP=1, the ci-baseline-green hook's documented
escape hatch for a commit that IS the fix for the red baseline.

Also records the kcov instrument defect in docs/shell-split-verification.md.
rpi_nc_install.sh measures 11.36% while its 29 assertions pass and its writes
land on disk. Five hypotheses are now disproven against minimal
reproductions -- the three from the previous session, plus `cd`-relocation
and a heredoc piped into a stubbed external. The cause is still unknown, so
the rule is documented instead: a suspicious percentage means re-measure, not
re-write the test. The failure only ever under-reports, so it can keep a lib
on the allowlist but can never promote an untested one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016sxxsDHWgdRhmdmzNMA6XM
2026-08-22 12:31:36 +02:00

41 lines
3.3 KiB
JSON

{
"intent": "Get the Shell tests workflow green again. The previous commit added `kcov` to its apt install step on the assumption it was packaged for Ubuntu; it is not, and CI went red in 34s on 'E: Unable to locate package kcov'. CI now skips jail-requiring suites with an announced reason, while the pre-push hook keeps running them for real on the developer machine, which has kcov.",
"scope": [
".github/workflows/shell-tests.yml: drop kcov from the install step; SKIP jail_args-marked suites with a printed reason",
"Non-goal: building kcov from source in CI -- minutes added to every run for a check the pre-push hook already performs on every push",
"Non-goal: any change to ci_mirror.sh, which runs these suites for real and is unaffected",
"Non-goal: weakening the empty-marker guard, which stays a hard failure"
],
"changes": [
"The assumption that kcov was apt-installable was inferred, not verified, and CI disproved it in 34 seconds. Upstream (SimonKagstrom/kcov v43) publishes no prebuilt binary asset either, so apt is not merely a packaging gap.",
"CI now declares what it cannot do rather than pretending to check it -- the same trade the arch-tests job already makes for Arch-only tests. The skip prints 'SKIP <runner> (needs the kcov jail; enforced by pre-push)' so it is announced, never silent.",
"The enforcement is not lost: ci_mirror.sh runs the marked suites through the real jail on every push, and that gate is already proven to fail on a broken assertion.",
"The empty-jail_args guard remains a hard failure, so a misconfigured marker still turns CI red instead of being skipped."
],
"verification": [
{
"command": "local simulation of the workflow step's loop over all six discovered runners",
"result": "pass",
"evidence": "5 PASS (scripts/lib, digital_wellbeing, hosts, periodic_background, utils), 1 SKIP (features/lib), failed=0. Every discovered runner is accounted for -- none silently dropped."
},
{
"command": "replace jail_args with comments+blank lines only, re-run the marker check, restore",
"result": "pass",
"evidence": "'FAIL (empty marker) -> correctly detected'. The fail-closed guard survives the rewrite; git diff --stat empty after restore."
},
{
"command": "gh run view 32567260903 --log-failed",
"result": "fail",
"evidence": "'E: Unable to locate package kcov' / 'Process completed with exit code 100' -- the failure this commit fixes, recorded as the measured reason for the redesign."
}
],
"risks": [
"The features/lib suite is now verified only by the pre-push hook, not by CI. A push made with --no-verify would land it unchecked. Acceptable because --no-verify is already forbidden by memories/git-rules.md, and the alternative (building kcov from source) taxes every CI run.",
"If a future contributor works on a machine without kcov, ci_mirror.sh's jailed branch cannot run there either; the jail already errors with 'Error: kcov is not installed' rather than passing silently."
],
"rollback": [
"git revert the commit; CI returns to attempting the kcov install and goes red again on the same apt error.",
"After rollback, confirm the shell-tests workflow fails at the install step, which is the symptom this commit removes."
]
}