mirror of
https://github.com/kuhyx/testsAndMisc.git
synced 2026-11-10 22:27:01 +01:00
Krzysztof kuhy Rudnicki
0e9629e110
75/75 lines through run_all.sh at --min 100, exit 0, and
check_shell_coverage.sh passes with the allowlist line already deleted
so the pass cannot come from the exemption.
55 assertions over 23 cases: all five profile repairs plus test_thorium,
including every dry-run arm, the partial singleton-lock case, a corrupt
database and the sqlite3-absent path. The lib needed NO production
change -- it already takes every path from $THORIUM_CONFIG_DIR, so
pointing that one variable at the tmpdir confines it completely.
The gate's per-case timeout moves from the jail's 120s default to 600s.
That default is a hang-catcher, not a performance budget, and this suite
has outgrown it: 76s bare at 25 test files, 93s at 27, and kcov roughly
doubles it. The symptom was a lib FAILING the gate while measuring
perfectly -- bt_audio.sh printed "127/127 lines = 100.00%" and then
"warn: case timed out after 120s", which --fail-on-case-error turned
into a non-zero exit. bt_audio.sh and arch_cpu.sh both pass now.
The cost, recorded in the evidence: a genuine hang takes 10 minutes per
pass to surface instead of 2. test_thorium's `read -r -p` is exactly
that shape, so every case here feeds it stdin explicitly.
This also settles the quarantine in b2779059. Re-adding the arch_sysctl
cases still yields "kcov instrumented no lines" for bt_audio at a 600s
timeout with no timeout warning, so runtime was never that file's
problem and the quarantine stands for the reason its header gives.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
84 lines
3.7 KiB
Bash
Executable File
84 lines
3.7 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
|
|
# ============================================================================
|
|
# The coverage bar for check_shell_coverage.sh.
|
|
#
|
|
# Split out of that script on 2026-08-22 to keep both files under the repo's
|
|
# 250-line cap. Sourced, not executed.
|
|
# ============================================================================
|
|
|
|
# A lib is covered when its directory has a suite AND that suite drives the
|
|
# lib to COVERAGE_BAR% line coverage.
|
|
#
|
|
# This was a presence check until 2026-08-22 -- "a run_all.sh exists beside
|
|
# it" -- which is satisfiable by an empty suite that sources the lib and
|
|
# asserts nothing. The bar is a measured percentage because that is the only
|
|
# version a suite cannot trivially pass.
|
|
#
|
|
# 100% is reachable without suppressions and without touching production
|
|
# source: run the suite inside the user+mount namespace of
|
|
# shell_coverage_jail.sh and the sudo-writes, systemd units and nftables rules
|
|
# execute for real against a throwaway /etc. Measured on the repo's hardest
|
|
# library, dot_resolver_install.sh (15 root ops, 5 system writes): 39/39.
|
|
readonly COVERAGE_BAR=100
|
|
|
|
is_covered() {
|
|
local path="$1"
|
|
local suite
|
|
suite="$(dirname "$path")/tests/run_all.sh"
|
|
[[ -f "$suite" ]] || return 1
|
|
|
|
# Presence alone is enough when the measurement cannot be made: kcov is a
|
|
# developer dependency, and a hook that hard-fails on a missing tool would
|
|
# block commits on any machine without it. The CI job carries the tool and
|
|
# enforces the number, so nothing merges on presence alone.
|
|
command -v kcov >/dev/null 2>&1 || return 0
|
|
command -v unshare >/dev/null 2>&1 || return 0
|
|
|
|
# A suite declares the mounts and seeds it needs in a `jail_args` file
|
|
# beside its run_all.sh -- the same marker ci_mirror.sh and
|
|
# shell-tests.yml read. Hardcoding flags here was a latent gate failure:
|
|
# the features/lib suite needs /usr/local/share and /etc/pki, and without
|
|
# them it ABORTS partway under `set -e`. The coverage number still printed
|
|
# (kcov had already recorded the lines the dead run reached), so the gate
|
|
# would have passed a lib whose suite never finished.
|
|
local jail_args=()
|
|
local args_file
|
|
args_file="$(dirname "$suite")/jail_args"
|
|
if [[ -f "$args_file" ]]; then
|
|
local arg
|
|
while IFS= read -r arg; do
|
|
jail_args+=("$arg")
|
|
done < <(grep -v '^#\|^$' "$args_file" || true)
|
|
if [[ ${#jail_args[@]} -eq 0 ]]; then
|
|
echo "Error: jail_args is present but empty: $args_file" >&2
|
|
return 1
|
|
fi
|
|
else
|
|
# No marker: the suite runs without needing anything mounted, so give
|
|
# it the historical defaults rather than nothing.
|
|
jail_args=(--bind /etc --bind /usr/local/bin --bind /var/lib)
|
|
fi
|
|
|
|
# --fail-on-case-error: a suite that dies partway must fail the gate, not
|
|
# be graded on however many lines it managed to reach first.
|
|
# --timeout: the jail defaults to 120s per case, which is a hang-catcher
|
|
# rather than a performance budget -- and this repo's suites have outgrown
|
|
# it. fixes/lib/tests/run_all.sh measures 76s bare with 25 test files and
|
|
# 93s with 27, and kcov instrumentation roughly doubles that, so the
|
|
# default started failing the gate on libs that measured a clean 100%
|
|
# (bt_audio.sh: "127/127 lines = 100.00%" followed by "case timed out").
|
|
# CI runners are slower than this workstation, so the ceiling is set well
|
|
# clear of the suite rather than just above it. The cost is that a genuine
|
|
# hang -- thorium_repairs.sh's `read -r -p` is exactly that shape -- now
|
|
# takes 10 minutes per pass to surface instead of 2.
|
|
"$REPO_ROOT/meta/scripts/shell_coverage_jail.sh" \
|
|
--subject "$suite" \
|
|
--timeout 600s \
|
|
"${jail_args[@]}" \
|
|
--measure "$(basename "$path")" \
|
|
--min "$COVERAGE_BAR" \
|
|
--fail-on-case-error \
|
|
-- "" >/dev/null 2>&1
|
|
}
|