mirror of
https://github.com/kuhyx/testsAndMisc.git
synced 2026-11-12 07:51:00 +01:00
Krzysztof kuhy Rudnicki
81191f40e6
The instrument, not the tests, was the bottleneck. Coverage is now
(PS4-traced lines | kcov hits) & (kcov line set - continuation lines).
The trace deliberately does NOT supply the denominator. A trace reports only
lines that RAN, so using it for both halves would make every subject 100.00%
and arm a repo-wide gate that checks nothing.
Three measured findings, each of which yields a silently wrong trace:
1. kcov's ptrace and bash's xtrace cannot share a process. Under
SHELLOPTS=xtrace every line kcov recorded as hits="1" comes back hits="0"
-- reproduced A/B on one subject. Hence two passes, two namespaces.
2. SHELLOPTS is readonly inside bash, so `export SHELLOPTS=xtrace` fails and
tracing never turns on; and `set -x` does not reach child processes.
3. Decisive: bash under `unshare --map-root-user` runs PRIVILEGED and
DISCARDS an inherited PS4, falling back to "+ " while still honouring
xtrace. The trace loses its file:line prefix and every subject reads 0%.
Fixed by delivering PS4 and `set -x` through a BASH_ENV file, which also
solves child-process propagation.
A fourth trap, in the report: kcov records filename="dwm_config.sh", a bare
basename. The source path comes from the trace's ${BASH_SOURCE} or a unique
repo search; an ambiguous basename yields no exclusion rather than a guess.
Measured through run_all.sh, nothing down:
dot_resolver_install.sh 39/39 = 100.00% (acceptance, unchanged)
rpi_nc_ca.sh 73/73 = 100.00% (was 72/73 = 98.63%)
dwm_config.sh 65/66 = 98.48% (was 35/73 = 47.95%)
rpi_nc_install.sh 85/88 = 96.59% (was 10/88 = 11.36%)
rpi_nc_ca reaching 100% is the load-bearing check: its missing line 141 was
cross-process contagion, and the trace pass has no kcov in it to contaminate.
dwm_config's remaining line 39 is a `done < <(...)`. Left in the denominator
on purpose -- removing lines inflates coverage, the fail-open direction, so a
second exclusion rule needs per-line justification, not a pattern from one lib.
Both new Python modules hold 100% branch coverage (29 tests).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
19 lines
2.2 KiB
JSON
19 lines
2.2 KiB
JSON
{
|
|
"title": "Fix both kcov defects: correct denominator, and replace its numerator with a PS4 trace",
|
|
"objective": "Make the shell-coverage instrument report the truth, so the allowlist campaign measures tests rather than tooling. kcov counted continuation lines of multi-line quoted arguments as statements (wrong denominator) and failed to record ordinary statements that provably execute (wrong numerator); the two together held dwm_config.sh at 47.95% and rpi_nc_install.sh at 11.36% regardless of how many tests were written.",
|
|
"acceptance_criteria": [
|
|
"dot_resolver_install.sh still measures 39/39 = 100.00% through run_all.sh, gating at --min 100.",
|
|
"rpi_nc_ca.sh measures 73/73 = 100.00% -- it was 72/73 = 98.63% only because of cross-process contagion from test_dwm_config.sh, which a kcov-free trace pass cannot suffer.",
|
|
"No lib's percentage goes DOWN. A drop means a lost trace, not a stricter instrument.",
|
|
"The denominator is never derived from the trace. A PS4 trace reports only lines that ran, so trace-as-denominator would make every subject 100% and gate on nothing; kcov's line set is what keeps the gate meaningful.",
|
|
"meta/scripts/lib/shell_coverage_lines.py and meta/scripts/shell_coverage_report.py both hold 100% branch coverage.",
|
|
"pre-commit passes on every touched file with no suppression added."
|
|
],
|
|
"out_of_scope": [
|
|
"Adding a second denominator-exclusion rule for `done < <(...)` process substitution. Every line removed from a denominator inflates coverage -- the fail-open direction -- so it must be justified per line, not generalised from one lib.",
|
|
"Wiring check_shell_coverage.sh into pre-commit/pre-push/CI, which waits for the allowlist to reach zero.",
|
|
"The unrelated red `Python tests` CI job (pip resolution-too-deep since 2026-08-17)."
|
|
],
|
|
"verifier": "bash meta/scripts/shell_coverage_jail.sh --subject linux_configuration/scripts/single_use/features/lib/tests/run_all.sh $(jail_args) --measure dot_resolver_install.sh --min 100 --fail-on-case-error -- \"\" and the same for rpi_nc_ca.sh (both must exit 0), plus python3 -m pytest python_pkg/tests/test_shell_coverage_lines.py python_pkg/tests/test_shell_coverage_report.py (expects 29 passed), plus pre-commit run --files on the touched files."
|
|
}
|