mirror of
https://github.com/kuhyx/testsAndMisc.git
synced 2026-11-10 22:07:03 +01:00
Krzysztof kuhy Rudnicki
a174b2a3c4
kcov works here once invoked correctly, so shell coverage is measurable after all. shell_coverage.sh wraps it as a gate; the XML parsing lives in shell_coverage_report.py per the rule against inlining another language's logic in a shell script. Two kcov traps are documented in the wrapper because both fail silently. It must be handed the test script DIRECTLY: given `bash script.sh` it instruments the bash binary, finds no shell source, and reports 0/0 at 0.00% -- indistinguishable from "nothing is covered". And per-line detail exists only in cov.xml; coverage.json carries the percentage alone, so it cannot say which lines are missing. First measurement put dns_iptables.sh at 68/74 = 91.89%, with all six uncovered lines in error and edge branches: chain creation failing, jump insertion failing, the stale-jump de-dupe log, and the v6 trusted-resolver ACCEPT. The stub grew failure injection and jump seeding to reach them, and four new cases take it to 74/74 = 100.00%. The gate fails closed: deleting the three ensure_chain failure cases drops it to 93.24%, names lines 23, 24, 37, 40, 41, and exits 1. Not wired into pre-commit -- kcov is absent on CI, and most shell files here have no test at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011w2Pj4Ek3ZNMQVYcFHgXDY
72 lines
2.4 KiB
Bash
Executable File
72 lines
2.4 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
|
|
# ============================================================================
|
|
# Measure line coverage of a shell library from its test script, via kcov.
|
|
#
|
|
# shell_coverage.sh <test-script> <subject-basename> [min-percent]
|
|
#
|
|
# Reports covered/total and the uncovered line numbers, and exits 1 when the
|
|
# result is below min-percent (default 100).
|
|
#
|
|
# Two things about kcov that cost an hour to find, both worth keeping:
|
|
#
|
|
# 1. It must be given the test script DIRECTLY, not `bash <script>`. Handed
|
|
# `bash foo.sh` it instruments the bash binary, finds no shell source,
|
|
# and reports 0/0 at 0.00% -- which reads exactly like "nothing is
|
|
# covered" rather than "nothing was measured".
|
|
# 2. Per-line detail is only in cov.xml. The summary coverage.json carries
|
|
# just the percentage, so it cannot tell you WHICH lines are missing.
|
|
#
|
|
# It does follow into subprocesses, so a harness that stages the subject into
|
|
# a temp dir and runs each case as a child still gets measured -- which is why
|
|
# --include-pattern matches on basename rather than an absolute path.
|
|
# ============================================================================
|
|
|
|
set -euo pipefail
|
|
|
|
readonly SCRIPT_NAME="${0##*/}"
|
|
|
|
usage() {
|
|
echo "Usage: $SCRIPT_NAME <test-script> <subject-basename> [min-percent]"
|
|
echo
|
|
echo " test-script the test to run, e.g. phone_focus_mode/lib/tests/test_x.sh"
|
|
echo " subject-basename the file to measure, e.g. dns_iptables.sh"
|
|
echo " min-percent fail below this (default 100)"
|
|
exit 0
|
|
}
|
|
|
|
[[ $# -ge 1 && ( "$1" == "-h" || "$1" == "--help" ) ]] && usage
|
|
|
|
if [[ $# -lt 2 ]]; then
|
|
echo "Error: need a test script and a subject basename" >&2
|
|
exit 1
|
|
fi
|
|
|
|
readonly TEST_SCRIPT="$1"
|
|
readonly SUBJECT="$2"
|
|
readonly MIN_PERCENT="${3:-100}"
|
|
|
|
if ! command -v kcov >/dev/null 2>&1; then
|
|
echo "Error: kcov is not installed (pacman -S kcov)" >&2
|
|
exit 1
|
|
fi
|
|
|
|
if [[ ! -f "$TEST_SCRIPT" ]]; then
|
|
echo "Error: no such test script: $TEST_SCRIPT" >&2
|
|
exit 1
|
|
fi
|
|
|
|
OUT_DIR="$(mktemp -d)"
|
|
readonly OUT_DIR
|
|
cleanup() { rm -rf "$OUT_DIR"; }
|
|
trap cleanup EXIT
|
|
|
|
# Direct invocation, per note 1 above.
|
|
if ! kcov --include-pattern="$SUBJECT" "$OUT_DIR" "$TEST_SCRIPT" >"$OUT_DIR/run.log" 2>&1; then
|
|
echo "Error: the test script failed under kcov; its output follows" >&2
|
|
tail -20 "$OUT_DIR/run.log" >&2
|
|
exit 1
|
|
fi
|
|
|
|
python3 "$(dirname "$0")/shell_coverage_report.py" "$OUT_DIR" "$SUBJECT" "$MIN_PERCENT"
|