mirror of
https://github.com/kuhyx/testsAndMisc.git
synced 2026-11-10 22:07:03 +01:00
The instrument, not the tests, was the bottleneck. Coverage is now
(PS4-traced lines | kcov hits) & (kcov line set - continuation lines).
The trace deliberately does NOT supply the denominator. A trace reports only
lines that RAN, so using it for both halves would make every subject 100.00%
and arm a repo-wide gate that checks nothing.
Three measured findings, each of which yields a silently wrong trace:
1. kcov's ptrace and bash's xtrace cannot share a process. Under
SHELLOPTS=xtrace every line kcov recorded as hits="1" comes back hits="0"
-- reproduced A/B on one subject. Hence two passes, two namespaces.
2. SHELLOPTS is readonly inside bash, so `export SHELLOPTS=xtrace` fails and
tracing never turns on; and `set -x` does not reach child processes.
3. Decisive: bash under `unshare --map-root-user` runs PRIVILEGED and
DISCARDS an inherited PS4, falling back to "+ " while still honouring
xtrace. The trace loses its file:line prefix and every subject reads 0%.
Fixed by delivering PS4 and `set -x` through a BASH_ENV file, which also
solves child-process propagation.
A fourth trap, in the report: kcov records filename="dwm_config.sh", a bare
basename. The source path comes from the trace's ${BASH_SOURCE} or a unique
repo search; an ambiguous basename yields no exclusion rather than a guess.
Measured through run_all.sh, nothing down:
dot_resolver_install.sh 39/39 = 100.00% (acceptance, unchanged)
rpi_nc_ca.sh 73/73 = 100.00% (was 72/73 = 98.63%)
dwm_config.sh 65/66 = 98.48% (was 35/73 = 47.95%)
rpi_nc_install.sh 85/88 = 96.59% (was 10/88 = 11.36%)
rpi_nc_ca reaching 100% is the load-bearing check: its missing line 141 was
cross-process contagion, and the trace pass has no kcov in it to contaminate.
dwm_config's remaining line 39 is a `done < <(...)`. Left in the denominator
on purpose -- removing lines inflates coverage, the fail-open direction, so a
second exclusion rule needs per-line justification, not a pattern from one lib.
Both new Python modules hold 100% branch coverage (29 tests).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
119 lines
5.5 KiB
Bash
Executable File
119 lines
5.5 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
|
|
# ============================================================================
|
|
# write_case_runner — emit the script that executes the cases INSIDE the jail.
|
|
#
|
|
# Split out of shell_coverage_jail.sh to hold that file under the 250-line cap
|
|
# after --fail-on-case-error was added. Sourced, never run: it needs $JAIL from
|
|
# its caller.
|
|
#
|
|
# The runner is written to a file rather than inlined into the `unshare`
|
|
# command because it executes in the child namespace: a quoted inline block
|
|
# would have to defer every expansion to the child shell, which reads as a
|
|
# quoting bug and trips SC2016.
|
|
#
|
|
# TWO PASSES, and they cannot be merged. kcov instruments via ptrace; bash's
|
|
# xtrace is a shell feature. Running both over one process was measured to
|
|
# destroy kcov's numerator outright -- with SHELLOPTS=xtrace set, every line
|
|
# kcov had recorded as hits="1" comes back hits="0". So:
|
|
#
|
|
# pass "kcov" -> the instrumentable LINE SET (the denominator) plus kcov's
|
|
# own hit counts, which are correct for some lines and
|
|
# missing for others (defect (b) in docs/kcov-under-report.md).
|
|
# pass "trace" -> the EXECUTED line set, from a PS4 xtrace. This is the
|
|
# authoritative numerator; kcov's hits are unioned in so a
|
|
# line either instrument saw counts as covered.
|
|
#
|
|
# Tracing is delivered via BASH_ENV, not SHELLOPTS/PS4, and the reason is
|
|
# measured, not stylistic. Three findings, each of which yields a silently
|
|
# WRONG trace rather than an error:
|
|
#
|
|
# 1. `set -x` does NOT reach child bash processes, and the suites drive
|
|
# their subjects several processes deep.
|
|
# 2. SHELLOPTS is a READONLY variable inside bash, so `export
|
|
# SHELLOPTS=xtrace` fails with "readonly variable" and tracing never
|
|
# turns on. It only works placed in the environment OF the process.
|
|
# 3. Decisive: bash started under `unshare --user --map-root-user` runs in
|
|
# PRIVILEGED mode (uid != euid) and DISCARDS an inherited PS4, falling
|
|
# back to the default "+ " -- while still honouring SHELLOPTS=xtrace.
|
|
# The trace then carries no file:line prefix at all and every subject
|
|
# reads as 0% covered.
|
|
#
|
|
# BASH_ENV survives all three: every non-interactive bash sources it at
|
|
# startup, so PS4 is ASSIGNED from inside the shell (where privileged mode
|
|
# cannot strip it) and `set -x` is re-applied per process, including children.
|
|
#
|
|
# BASH_XTRACEFD uses fd 199, not a low fd: subjects and harnesses open their
|
|
# own descriptors, and a subject doing `exec 9>...` would silently clobber the
|
|
# trace mid-run and truncate it.
|
|
# ============================================================================
|
|
|
|
write_case_runner() {
|
|
# The runner below executes INSIDE the namespace, so it is written out as
|
|
# its own file rather than inlined: a quoted inline block would have to
|
|
# defer every expansion to the child shell, which reads as a quoting bug
|
|
# and trips SC2016.
|
|
cat >"$JAIL/run_cases.sh" <<'INNER'
|
|
#!/usr/bin/env bash
|
|
set -u
|
|
jail="$1"
|
|
subject_dir="$2"
|
|
subject_base="$3"
|
|
measure_base="$4"
|
|
case_timeout="$5"
|
|
pass="$6"
|
|
. "$jail/mounts.sh"
|
|
export PATH="$jail/bin:/usr/bin:/bin"
|
|
export HOME="$jail/home"
|
|
USER="${USER:-root}"
|
|
export USER
|
|
export SUDO_USER="$USER"
|
|
cd "$subject_dir" || exit 1
|
|
case_index=0
|
|
while IFS= read -r line; do
|
|
read -r -a case_args <<<"$line"
|
|
case_index=$((case_index + 1))
|
|
# Bounded, and with stdin closed. Both matter: a subject that reads stdin
|
|
# blocks forever on the caller's terminal (a `crontab -` fallback did
|
|
# exactly this and hung a run for the full two minutes with no output),
|
|
# and a subject that spins never yields at all. A timed-out case is
|
|
# reported rather than swallowed, because a silent timeout looks
|
|
# identical to an unreachable line in the coverage report.
|
|
if [[ $pass == kcov ]]; then
|
|
timeout --kill-after=10s "$case_timeout" \
|
|
kcov --include-pattern="$measure_base" "$jail/cov" \
|
|
"./$subject_base" ${case_args[@]+"${case_args[@]}"} \
|
|
>/dev/null 2>&1 </dev/null
|
|
rc=$?
|
|
else
|
|
# PS4 carries the full BASH_SOURCE, not ${BASH_SOURCE##*/}: two libs
|
|
# in different directories can share a basename, and the report
|
|
# filters by basename itself.
|
|
#
|
|
# The trace fd is opened HERE and inherited, so the subject's own
|
|
# stdout/stderr redirections (the jail sends both to /dev/null)
|
|
# cannot swallow the trace -- a 2>/dev/null once cost a bogus
|
|
# "0 lines traced" reading.
|
|
timeout --kill-after=10s "$case_timeout" \
|
|
env "BASH_XTRACEFD=199" "BASH_ENV=$jail/xtrace_env.sh" \
|
|
"./$subject_base" ${case_args[@]+"${case_args[@]}"} \
|
|
>/dev/null 2>&1 </dev/null 199>>"$jail/trace/case.$case_index"
|
|
rc=$?
|
|
fi
|
|
if [[ $rc -eq 124 || $rc -eq 137 ]]; then
|
|
printf 'warn: case timed out after %s: %s\n' "$case_timeout" "$line" >&2
|
|
printf '%s\n' "$rc" >>"$jail/case_failures.$pass"
|
|
elif [[ $rc -ne 0 ]]; then
|
|
# A non-zero case is only noise when measuring (an installer that
|
|
# exits 1 on a bogus arg still yields coverage), but it is the whole
|
|
# signal when the subject is a TEST SUITE. Record it and let the
|
|
# caller decide; swallowing it made a deliberately-broken assertion
|
|
# exit 0 through the push gate.
|
|
printf 'warn: case exited %d: %s\n' "$rc" "$line" >&2
|
|
printf '%s\n' "$rc" >>"$jail/case_failures.$pass"
|
|
fi
|
|
done <"$jail/cases"
|
|
INNER
|
|
chmod +x "$JAIL/run_cases.sh"
|
|
}
|