mirror of
https://github.com/kuhyx/testsAndMisc.git
synced 2026-11-11 17:34:04 +01:00
Krzysztof kuhy Rudnicki
a174b2a3c4
kcov works here once invoked correctly, so shell coverage is measurable after all. shell_coverage.sh wraps it as a gate; the XML parsing lives in shell_coverage_report.py per the rule against inlining another language's logic in a shell script. Two kcov traps are documented in the wrapper because both fail silently. It must be handed the test script DIRECTLY: given `bash script.sh` it instruments the bash binary, finds no shell source, and reports 0/0 at 0.00% -- indistinguishable from "nothing is covered". And per-line detail exists only in cov.xml; coverage.json carries the percentage alone, so it cannot say which lines are missing. First measurement put dns_iptables.sh at 68/74 = 91.89%, with all six uncovered lines in error and edge branches: chain creation failing, jump insertion failing, the stale-jump de-dupe log, and the v6 trusted-resolver ACCEPT. The stub grew failure injection and jump seeding to reach them, and four new cases take it to 74/74 = 100.00%. The gate fails closed: deleting the three ensure_chain failure cases drops it to 93.24%, names lines 23, 24, 37, 40, 41, and exits 1. Not wired into pre-commit -- kcov is absent on CI, and most shell files here have no test at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011w2Pj4Ek3ZNMQVYcFHgXDY
57 lines
4.3 KiB
JSON
57 lines
4.3 KiB
JSON
{
|
|
"intent": "Make shell line coverage measurable in this repo, and bring dns_iptables.sh to 100% now that it can be measured.",
|
|
"scope": [
|
|
"meta/scripts/shell_coverage.sh + shell_coverage_report.py — new coverage gate",
|
|
"phone_focus_mode/lib/tests/dns_iptables_harness.sh — failure injection and jump seeding added to the iptables stub",
|
|
"phone_focus_mode/lib/tests/test_dns_iptables.sh — 4 new cases, 20 -> 24",
|
|
"Non-goal: wiring coverage into pre-commit. kcov is not installed on CI and most shell files have no test at all."
|
|
],
|
|
"changes": [
|
|
"Added shell_coverage.sh, which runs a test under kcov and gates on a minimum percentage, defaulting to 100.",
|
|
"Two kcov behaviours are documented in the script because both are silent traps: it must be handed the test script DIRECTLY (given `bash script.sh` it instruments the bash binary and reports 0/0 at 0.00%, which reads like zero coverage rather than zero measurement), and per-line detail exists only in cov.xml, not in the coverage.json summary.",
|
|
"XML parsing lives in shell_coverage_report.py rather than inline, per the repo rule against embedding another language's logic in a shell script.",
|
|
"Extended the iptables stub with failure injection (_fail_op) and stale-jump seeding (_seed_jumps), which is what makes ensure_chain's error and de-dupe branches reachable.",
|
|
"Added 4 cases: chain creation failing, OUTPUT jump insertion failing, several stale jumps collapsing to one, and the v6 mirror of the ACCEPT-before-REJECT ordering rule."
|
|
],
|
|
"verification": [
|
|
{
|
|
"command": "bash meta/scripts/shell_coverage.sh phone_focus_mode/lib/tests/test_dns_iptables.sh dns_iptables.sh",
|
|
"result": "pass",
|
|
"evidence": "'dns_iptables.sh: 74/74 lines = 100.00%', exit 0. Before the 4 new cases it measured 68/74 = 91.89% with lines 23, 24, 37, 40, 41 and 99 uncovered — every one an error or edge branch."
|
|
},
|
|
{
|
|
"command": "gate check: delete the three ensure_chain failure cases and re-measure",
|
|
"result": "fail-as-expected",
|
|
"evidence": "'dns_iptables.sh: 69/74 lines = 93.24%', 'uncovered: 23, 24, 37, 40, 41', 'coverage 93.24% is below the required 100.00%', exit 1. Restoring the cases returns 74/74. The gate fails closed and names the missing lines."
|
|
},
|
|
{
|
|
"command": "bash phone_focus_mode/lib/tests/test_dns_iptables.sh",
|
|
"result": "pass",
|
|
"evidence": "24 passed, 0 failed."
|
|
},
|
|
{
|
|
"command": "bash meta/scripts/verify_shell_split.sh 728301c phone_focus_mode/dns_enforcer.sh phone_focus_mode/dns_enforcer.sh phone_focus_mode/dns_iptables.sh",
|
|
"result": "pass",
|
|
"evidence": "'IDENTICAL: all 14 top-level functions moved verbatim' — the subject is unchanged by the harness work; only the tests grew."
|
|
},
|
|
{
|
|
"command": "bash phone_focus_mode/lib/tests/test_monitor.sh and test_tether_enforcer.sh",
|
|
"result": "pass",
|
|
"evidence": "9 passed / 0 failed and 17 passed / 0 failed, unchanged."
|
|
},
|
|
{
|
|
"command": "pre-commit run --files <the four changed files>",
|
|
"result": "pass",
|
|
"evidence": "All hooks pass with zero suppressions. Four ruff findings were fixed at the source: S314 by switching to defusedxml (already a pinned dependency), N817 by dropping the ET acronym alias, INP001 by adding the shebang its sibling scripts carry, and a T201 autofix that had silently replaced the tool's own output with `pass` was repaired by using sys.stdout.write, matching extract_shell_functions.py."
|
|
}
|
|
],
|
|
"risks": [
|
|
"kcov measures lines, not branches. A line containing `[ -n \"$x\" ] && do_thing` counts as covered when only one side is taken, so 100% here is weaker than the 100% branch coverage the Python gate enforces. The mutation checks are what cover that gap.",
|
|
"The gate is not wired into pre-commit, so nothing forces coverage to stay at 100% for this file. Doing so would require kcov on CI and would fail immediately for every shell file that has no test."
|
|
],
|
|
"rollback": [
|
|
"git revert the commit. The two tooling scripts disappear and the test returns to 20 assertions; the subject under test is untouched either way.",
|
|
"After rollback, run bash phone_focus_mode/lib/tests/test_dns_iptables.sh and confirm it still passes, since the tests do not depend on the coverage tooling."
|
|
]
|
|
}
|