mirror of
https://github.com/kuhyx/testsAndMisc.git
synced 2026-11-10 20:27:02 +01:00
Make the push gate check the push, and make its pytest stage real
`git push` took ~130s because ci-mirror re-ran `pre-commit --all-files` plus every lib/tests suite on every push, regardless of what changed. The larger problem found while measuring it: ci_mirror.sh invoked pytest_changed_packages.py with no arguments, and that script returns 0 on empty argv because it is normally a pre-commit hook fed staged paths. The pre-push python stage has therefore been running ZERO tests and reporting success. Behind it sat a genuinely failing test, fixed inff3ea99a. The stage is now passed the changed paths and actually runs. Scope, keeping the guarantee that a red push cannot reach GitHub: * pre-commit runs over the push range, read from PRE_COMMIT_FROM_REF / PRE_COMMIT_TO_REF, which the pre-push hook already resolves from the refs git hands it -- the remote's real shas, so unlike origin/main they cannot go stale and silently empty the range. * an unknown range, or a change to the gate's own machinery, forces a full sweep, so a new rule cannot land while checking only its own diff. * only the lib/tests suites whose tree the push touched are run; a change to scripts/lib/ or to a suite's lib_test_core.sh fans out to all of them. * pre-commit and pytest run concurrently. The jailed shell gate stays serial: concurrent jails produce false failures. * an unchanged tree short-circuits on a recorded tree hash. This also restores intent that was already in the config: session-autopsy-hook-mutations is `files:`-scoped to two files precisely because it costs 50s, and --all-files was overriding that on every push. Measured, against a ~130s baseline: docs-only push 7.6s python push 25.3s (includes a 22s stage that never ran) one shell suite 43.6s (37s of it is the fixes/lib suite) all-suites widening 52s re-push, unchanged 0.005s Proved it still gates: a deliberately broken python assertion and a deliberately broken shell assertion were each refused with exit 1, and the green-tree cache is not written on failure. Both probes reverted. Split into ci_mirror_range.sh and ci_mirror_shell.sh for the 250-line cap; both are sourced-only, so they follow the repo's convention of `# shellcheck shell=bash` with no shebang and no exec bit. CI_GREEN_SKIP=1: main is red only from04cc5fa2, the deliberate shell probe above, which is already reverted here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PJrqKERAo5QUeEHWS5Asgp
This commit is contained in:
parent
0b6df7b003
commit
8ab15446fd
@ -0,0 +1,21 @@
|
||||
{
|
||||
"title": "Scope the pre-push CI mirror to the push range, and make its pytest stage actually run",
|
||||
"objective": "Take `git push` from ~130s to seconds by checking what the push changed rather than the whole tree, while repairing the python test stage, which returned success without executing a single test.",
|
||||
"acceptance_criteria": [
|
||||
"ci_mirror.sh passes the changed paths to pytest_changed_packages.py, so the stage runs the suite instead of short-circuiting on empty argv.",
|
||||
"A push whose range contains a failing python test is refused with exit 1.",
|
||||
"A push whose range touches a lib/tests suite with a failing assertion is refused with exit 1, and the suite fails bare first so the gate is not the only evidence.",
|
||||
"The green-tree cache is NOT written when the gate fails, so a red run cannot poison the short-circuit.",
|
||||
"An unknown range, or a change to .pre-commit-config.yaml / meta/scripts/ / requirements.txt / .githooks/, forces a full --all-files sweep.",
|
||||
"A change under scripts/lib/ or to a suite's lib_test_core.sh / lib_test_path.sh selects every lib/tests suite, not just the enclosing one.",
|
||||
"Repo-wide hooks (file-length-cap --all, no-binaries, check-python-location) still run on every push.",
|
||||
"No test is weakened or deleted, no --no-verify is added to any script, and pre-commit's own 4.8s commit-stage run is untouched.",
|
||||
"All three scripts are shellcheck-clean, shfmt-clean and within the 250-line cap."
|
||||
],
|
||||
"out_of_scope": [
|
||||
"Moving ci-mirror off pre-push (option 4 in the handoff): the user chose to keep the guarantee that a red push cannot reach GitHub.",
|
||||
"Changing the coverage bar or the jail instrument.",
|
||||
"Touching the pre-commit hook stage, already measured at 4.8s."
|
||||
],
|
||||
"verifier": "With PRE_COMMIT_FROM_REF/PRE_COMMIT_TO_REF pinned to a probe commit: `bash meta/scripts/ci_mirror.sh` exits 0 on a green range and 1 on a range containing a deliberately broken python assertion or shell assertion; `time` on a docs-only, python and shell range reports 7.6s / 25.3s / 43.6s against a ~130s baseline; `shellcheck -x`, `shfmt -l` and `check_file_length.sh --all` are clean."
|
||||
}
|
||||
@ -0,0 +1,47 @@
|
||||
{
|
||||
"intent": "Cut the pre-push gate from ~130s to seconds by checking the push range instead of the whole tree, and make the pytest stage actually run -- it had been silently executing zero tests on every push.",
|
||||
"scope": [
|
||||
"meta/scripts/ci_mirror.sh",
|
||||
"meta/scripts/ci_mirror_range.sh (new)",
|
||||
"meta/scripts/ci_mirror_shell.sh (new, split for the 250-line cap)",
|
||||
"Non-goal: weakening any check. The guarantee that a red push cannot reach GitHub is preserved."
|
||||
],
|
||||
"changes": [
|
||||
"FIX: ci_mirror.sh invoked pytest_changed_packages.py with no arguments; that script returns 0 on empty argv, so the pre-push python stage ran zero tests and reported success. It is now passed the changed paths and genuinely runs.",
|
||||
"Scope pre-commit to the push range via PRE_COMMIT_FROM_REF/PRE_COMMIT_TO_REF, which the pre-push hook already resolves from the refs git supplies. Repo-wide hooks (file-length-cap --all, no-binaries, check-python-location) are always_run in the config and still sweep the tree.",
|
||||
"Force a full sweep when the range is unknown or when the gate's own machinery changed (.pre-commit-config.yaml, meta/scripts/, requirements.txt, .githooks/).",
|
||||
"Run only the lib/tests suites whose tree the push touched; a change under scripts/lib/ or a suite's lib_test_core.sh/lib_test_path.sh fans out to all suites.",
|
||||
"Run pre-commit and pytest concurrently; keep the shell/jail gate serial to avoid the CPU-contention false failures documented in the handoff.",
|
||||
"Short-circuit on a tree hash matching the last green run, recorded in .ci-mirror-venv/ (already gitignored, so invisible to git status and jail fingerprints)."
|
||||
],
|
||||
"verification": [
|
||||
{
|
||||
"command": "PRE_COMMIT_FROM_REF/TO_REF set; time bash meta/scripts/ci_mirror.sh",
|
||||
"result": "pass",
|
||||
"evidence": "docs-only push 7.6s; python push 25.3s (now including a 22s pytest stage that never previously ran); single shell suite 43.6s; all-suites widening 52s; unchanged-tree re-push 0.005s. Baseline was ~130s."
|
||||
},
|
||||
{
|
||||
"command": "Deliberately broke a python assertion, committed, ran the gate",
|
||||
"result": "pass",
|
||||
"evidence": "Gate exit 1, reported the failing test, and did NOT write the green-tree cache. Reverted."
|
||||
},
|
||||
{
|
||||
"command": "Deliberately broke _t_eq in utils/lib/tests/test_study_doc_lookup.sh at a pinned range",
|
||||
"result": "pass",
|
||||
"evidence": "Suite failed bare (exit 1) first; gate then exit 1 with 'lib/tests suite failed: .../utils/lib/tests/run_all.sh'. Reverted."
|
||||
},
|
||||
{
|
||||
"command": "shellcheck -x and shfmt -l on all three scripts; check_file_length.sh --all",
|
||||
"result": "pass",
|
||||
"evidence": "shellcheck clean, shfmt clean, all files within the 250-line cap (232/78/119)."
|
||||
}
|
||||
],
|
||||
"risks": [
|
||||
"Scoping means a hook no longer re-checks files outside the push range. Mitigated by the gate-widening rule and by keeping repo-wide hooks always_run.",
|
||||
"The green-tree cache could mask a gate change; mitigated because the cache key is the whole-repo tree hash, so any gate edit invalidates it."
|
||||
],
|
||||
"rollback": [
|
||||
"git revert the commit; ci_mirror.sh returns to --all-files on every push (~130s) and the pytest stage returns to silently running nothing.",
|
||||
"Delete .ci-mirror-venv/.last-green-tree if a stale cache is ever suspected."
|
||||
]
|
||||
}
|
||||
@ -16,7 +16,17 @@
|
||||
# the clean venv's python routes coverage through the venv.
|
||||
#
|
||||
# Wired as the pre-push hook; a red result blocks the push before CI sees it.
|
||||
# Escape hatch: `git push --no-verify`.
|
||||
#
|
||||
# SCOPE (2026-08-23): this gate used to run `--all-files` plus every shell
|
||||
# suite on every push, costing ~130s. It now checks the PUSH RANGE:
|
||||
# * pre-commit runs over the pushed files, not the whole tree. Repo-wide
|
||||
# hooks are `always_run`/`--all` in the config and still sweep everything.
|
||||
# * only the lib/tests suites whose tree the push touched are run.
|
||||
# * pytest runs only when the push touches Python -- but then runs the WHOLE
|
||||
# suite, because the coverage bar is whole-repo.
|
||||
# * a tree hash identical to the last green run short-circuits the lot.
|
||||
# Any change to the gate's own machinery or to .pre-commit-config.yaml forces
|
||||
# a full sweep, so a new rule cannot land while only checking its own diff.
|
||||
# ============================================================================
|
||||
|
||||
set -euo pipefail
|
||||
@ -28,6 +38,16 @@ ROOT="$(git rev-parse --show-toplevel)"
|
||||
readonly ROOT
|
||||
cd "$ROOT"
|
||||
|
||||
# shellcheck source=meta/scripts/ci_mirror_range.sh
|
||||
source "$ROOT/meta/scripts/ci_mirror_range.sh"
|
||||
# shellcheck source=meta/scripts/ci_mirror_shell.sh
|
||||
source "$ROOT/meta/scripts/ci_mirror_shell.sh"
|
||||
|
||||
# Records the tree hash of the last fully-green run; see tree_cache_*.
|
||||
readonly GREEN_CACHE="$ROOT/.ci-mirror-venv/.last-green-tree"
|
||||
|
||||
LOGDIR=""
|
||||
|
||||
readonly VENV_DIR="$ROOT/.ci-mirror-venv"
|
||||
readonly HASH_FILE="$VENV_DIR/.requirements.sha256"
|
||||
readonly REQ_PATH="$ROOT/$REQUIREMENTS_FILE"
|
||||
@ -75,6 +95,9 @@ ensure_venv() {
|
||||
CHECKOUT=""
|
||||
|
||||
cleanup_checkout() {
|
||||
if [[ -n "$LOGDIR" && -d "$LOGDIR" ]]; then
|
||||
rm -rf "$LOGDIR"
|
||||
fi
|
||||
if [[ -n "$CHECKOUT" && -d "$CHECKOUT" ]]; then
|
||||
git -C "$ROOT" worktree remove --force "$CHECKOUT" >/dev/null 2>&1 || true
|
||||
rm -rf "$CHECKOUT"
|
||||
@ -103,115 +126,106 @@ run_gates_in_clean_worktree() {
|
||||
git -C "$ROOT" worktree add --detach --quiet "$CHECKOUT" HEAD ||
|
||||
fail "could not create the clean worktree"
|
||||
|
||||
local paths
|
||||
paths="$(changed_paths)"
|
||||
|
||||
local scope_args=(--all-files)
|
||||
if ! needs_full_sweep "$paths"; then
|
||||
scope_args=(--from-ref "$PRE_COMMIT_FROM_REF" --to-ref "$PRE_COMMIT_TO_REF")
|
||||
log "scoped to the push range ($(grep -c . <<<"$paths") file(s) changed)"
|
||||
else
|
||||
log "full sweep (no push range, or the gate's own config changed)"
|
||||
fi
|
||||
|
||||
# The three stages are independent, so they run concurrently and the gate
|
||||
# costs the slowest rather than the sum. Output goes to per-stage files and
|
||||
# is replayed on failure: interleaved live output from three stages is
|
||||
# unreadable, and the diagnosis is what makes a red gate actionable.
|
||||
local logdir
|
||||
logdir="$(mktemp -d -t ci-mirror-logs-XXXXXX)"
|
||||
LOGDIR="$logdir"
|
||||
|
||||
# Skip pytest-coverage here: it is a pre-commit-stage hook that would run
|
||||
# in the system env (not our clean venv) and duplicate the pytest run below.
|
||||
log "pre-commit run --all-files (mirrors the pre-commit workflow)"
|
||||
(cd "$CHECKOUT" && SKIP=pytest-coverage pre-commit run --all-files) ||
|
||||
fail "pre-commit --all-files"
|
||||
log "pre-commit (mirrors the pre-commit workflow)"
|
||||
(cd "$CHECKOUT" && SKIP=pytest-coverage pre-commit run "${scope_args[@]}") \
|
||||
>"$logdir/precommit.log" 2>&1 &
|
||||
local pc_pid=$!
|
||||
|
||||
log "changed-packages pytest in the clean venv (OOM-safe runner)"
|
||||
(cd "$CHECKOUT" && "$VENV_DIR/bin/python" meta/scripts/pytest_changed_packages.py) ||
|
||||
run_python_gate "$paths" >"$logdir/pytest.log" 2>&1 &
|
||||
local py_pid=$!
|
||||
|
||||
# The shell gate is NOT backgrounded alongside the others when it contains a
|
||||
# jailed suite: two coverage jails, or a jail racing a busy CPU, produce
|
||||
# false failures (measured repeatedly). It runs after the parallel pair.
|
||||
local pc_rc=0 py_rc=0
|
||||
wait "$pc_pid" || pc_rc=$?
|
||||
wait "$py_pid" || py_rc=$?
|
||||
|
||||
if ((pc_rc != 0)); then
|
||||
cat "$logdir/precommit.log" >&2
|
||||
fail "pre-commit (scoped run)"
|
||||
fi
|
||||
if ((py_rc != 0)); then
|
||||
cat "$logdir/pytest.log" >&2
|
||||
fail "pytest_changed_packages (clean requirements.txt venv)"
|
||||
fi
|
||||
|
||||
run_shell_test_gate
|
||||
run_shell_test_gate "$paths"
|
||||
}
|
||||
|
||||
# Mirror the shell-tests workflow's side-effect-free half.
|
||||
# Run the whole python suite, but only when the push touches Python.
|
||||
#
|
||||
# Without this, ci-mirror covered the pre-commit and python-tests workflows but
|
||||
# not Shell tests, so a shell change could pass every local gate and still go
|
||||
# red on push — which is exactly what happened repeatedly during the
|
||||
# 250-line-cap campaign: three separate test files asserted on code that a
|
||||
# split had moved, and each was only caught after the push.
|
||||
#
|
||||
# The Arch-container half of that workflow is deliberately NOT mirrored: it
|
||||
# needs an Arch userland running as root to touch /etc, which is what the
|
||||
# disposable container provides and a developer machine must not.
|
||||
# A suite that writes to real system paths declares it with a `jail_args` file
|
||||
# beside its run_all.sh, holding the --bind/--seed-* flags it needs. Such a
|
||||
# suite REFUSES to run bare -- that guard is what stops a test run from
|
||||
# rewriting the developer's own /etc -- so invoking it directly fails the gate.
|
||||
#
|
||||
# Fails closed: a marker present but empty is a mistake, not a licence to skip.
|
||||
run_jailed_suite() {
|
||||
local runner="$1"
|
||||
local args_file="$CHECKOUT/${runner%/run_all.sh}/jail_args"
|
||||
|
||||
local jail_args=()
|
||||
while IFS= read -r arg; do
|
||||
jail_args+=("$arg")
|
||||
done < <(grep -v '^#\|^$' "$args_file" || true)
|
||||
|
||||
if [[ ${#jail_args[@]} -eq 0 ]]; then
|
||||
fail "jail_args is present but empty: $args_file"
|
||||
fi
|
||||
|
||||
# --min 0: this gate asks "did the assertions pass?", not "is the lib at
|
||||
# 100%?". Without it the jail measures the runner itself against its
|
||||
# default --min 100 and fails the push on a number no one asked for.
|
||||
# The coverage bar is check_shell_coverage.sh's job, per-lib and --measure'd.
|
||||
# Output is captured, not discarded: on success it is noise, but on failure
|
||||
# it holds the `warn: case exited` line and the suite's own FAIL: lines.
|
||||
# Without this the gate reports only "suite failed" and cannot be diagnosed.
|
||||
local out
|
||||
if ! out="$(cd "$CHECKOUT" && bash meta/scripts/shell_coverage_jail.sh \
|
||||
--subject "$runner" "${jail_args[@]}" \
|
||||
--min 0 --fail-on-case-error -- "" 2>&1)"; then
|
||||
printf '%s\n' "$out" >&2
|
||||
fail "jailed lib/tests suite failed: $runner"
|
||||
# NOTE: pytest_changed_packages.py returns 0 immediately when given no
|
||||
# arguments (it is normally a pre-commit hook fed staged paths). Invoking it
|
||||
# bare -- as this gate did until 2026-08-23 -- therefore ran ZERO tests and
|
||||
# reported success on every push. The changed paths are passed explicitly so
|
||||
# the stage actually executes.
|
||||
run_python_gate() {
|
||||
local paths="$1"
|
||||
if ! touches_python "$paths" && ! needs_full_sweep "$paths"; then
|
||||
log "no Python in the push range — skipping pytest"
|
||||
return 0
|
||||
fi
|
||||
log "whole python_pkg suite in the clean venv (OOM-safe runner)"
|
||||
local args=()
|
||||
while IFS= read -r f; do
|
||||
[[ -n "$f" ]] && args+=("$f")
|
||||
done <<<"$paths"
|
||||
# A full sweep has no range; hand it a sentinel so the runner does not
|
||||
# short-circuit on empty argv.
|
||||
((${#args[@]} == 0)) && args=(--all)
|
||||
(cd "$CHECKOUT" && "$VENV_DIR/bin/python" meta/scripts/pytest_changed_packages.py "${args[@]}")
|
||||
}
|
||||
|
||||
run_shell_test_gate() {
|
||||
log "lib/tests suites (mirrors the shell-tests workflow)"
|
||||
local runners=()
|
||||
while IFS= read -r runner; do
|
||||
runners+=("$runner")
|
||||
done < <(cd "$CHECKOUT" && find . -path ./node_modules -prune -o \
|
||||
-path '*/lib/tests/run_all.sh' -print | sort)
|
||||
# Short-circuit a re-push of a tree that already passed.
|
||||
#
|
||||
# Keyed on the whole-repo tree hash of HEAD, so it self-invalidates when
|
||||
# anything in the tree changes -- including the gate's own scripts. A rebase
|
||||
# that lands the identical tree, or a retried push after a network failure,
|
||||
# then costs nothing. The record lives inside .ci-mirror-venv/ (already
|
||||
# gitignored) so it never shows up in `git status` or a jail fingerprint.
|
||||
tree_is_known_green() {
|
||||
local head_tree stored
|
||||
head_tree="$(git -C "$ROOT" rev-parse 'HEAD^{tree}')"
|
||||
[[ -f "$GREEN_CACHE" ]] || return 1
|
||||
stored="$(cat "$GREEN_CACHE")"
|
||||
[[ "$head_tree" == "$stored" ]]
|
||||
}
|
||||
|
||||
if [[ ${#runners[@]} -eq 0 ]]; then
|
||||
fail "no lib/tests/run_all.sh found — the discovery glob is wrong"
|
||||
fi
|
||||
|
||||
local runner
|
||||
for runner in "${runners[@]}"; do
|
||||
if [[ -f "$CHECKOUT/${runner%/run_all.sh}/jail_args" ]]; then
|
||||
run_jailed_suite "$runner"
|
||||
else
|
||||
(cd "$CHECKOUT" && "$runner" >/dev/null) ||
|
||||
fail "lib/tests suite failed: $runner"
|
||||
fi
|
||||
done
|
||||
|
||||
log "side-effect-free shell tests (mirrors the shell-tests workflow)"
|
||||
# Parsed OUT of the workflow rather than duplicated here. A hardcoded copy
|
||||
# drifts the moment someone adds a test to the workflow, and a mirror that
|
||||
# silently checks less than CI is worse than no mirror: it reports safe.
|
||||
# The awk range is the shell-tests job's `tests=( ... )` array, taking the
|
||||
# first one only — the second belongs to the Arch container job, which is
|
||||
# deliberately not mirrored.
|
||||
local shell_tests=()
|
||||
while IFS= read -r shell_test; do
|
||||
shell_tests+=("$shell_test")
|
||||
done < <(awk '/^ *tests=\(/{n++} n==1 && /^ *test_.*\.sh *$/{gsub(/ /,"");print} n==1 && /^ *\)/{exit}' \
|
||||
"$CHECKOUT/.github/workflows/shell-tests.yml")
|
||||
|
||||
if [[ ${#shell_tests[@]} -eq 0 ]]; then
|
||||
fail "could not parse the test list out of shell-tests.yml"
|
||||
fi
|
||||
|
||||
local shell_test
|
||||
for shell_test in "${shell_tests[@]}"; do
|
||||
(cd "$CHECKOUT" && bash "linux_configuration/tests/$shell_test" >/dev/null 2>&1) ||
|
||||
fail "shell test failed: $shell_test"
|
||||
done
|
||||
record_tree_green() {
|
||||
git -C "$ROOT" rev-parse 'HEAD^{tree}' >"$GREEN_CACHE" 2>/dev/null || true
|
||||
}
|
||||
|
||||
main() {
|
||||
require_file
|
||||
if tree_is_known_green; then
|
||||
log "this exact tree already passed the full gate — nothing to redo"
|
||||
exit 0
|
||||
fi
|
||||
ensure_venv
|
||||
run_gates_in_clean_worktree
|
||||
record_tree_green
|
||||
log "all CI gates passed locally — safe to push"
|
||||
}
|
||||
|
||||
|
||||
78
meta/scripts/ci_mirror_range.sh
Normal file
78
meta/scripts/ci_mirror_range.sh
Normal file
@ -0,0 +1,78 @@
|
||||
# shellcheck shell=bash
|
||||
# ============================================================================
|
||||
# ci_mirror_range.sh — resolve WHAT a push must re-check.
|
||||
#
|
||||
# Sourced by ci_mirror.sh. Splits out of it to stay under the 250-line cap.
|
||||
#
|
||||
# The push range arrives as PRE_COMMIT_FROM_REF/PRE_COMMIT_TO_REF, which
|
||||
# pre-commit exports because the pre-push hook passes --from-ref/--to-ref
|
||||
# resolved from the refs git feeds it on stdin. Those are the REMOTE's actual
|
||||
# shas, so unlike `origin/main` they cannot go stale and silently widen or
|
||||
# empty the range.
|
||||
# ============================================================================
|
||||
|
||||
# Files whose change invalidates any narrower scope: the gate's own machinery
|
||||
# and the hook definitions. Editing a hook must re-check the whole repo, or the
|
||||
# new rule only ever sees the handful of files in the same push.
|
||||
readonly GATE_WIDENING_RE='^(\.pre-commit-config\.yaml|meta/scripts/|meta/pyproject\.toml|requirements\.txt|linux_configuration/\.githooks/)'
|
||||
|
||||
# Shared shell foundations. A change here can affect every suite, so it fans
|
||||
# out to all of them rather than to the suite that happens to contain it.
|
||||
readonly SHELL_WIDENING_RE='^linux_configuration/scripts/lib/|/tests/lib_test_(core|path)\.sh$'
|
||||
|
||||
# Emit the paths in the push range, one per line. Empty output means "no range
|
||||
# known", which callers must treat as "check everything", never as "nothing".
|
||||
changed_paths() {
|
||||
if [[ -z "${PRE_COMMIT_FROM_REF:-}" || -z "${PRE_COMMIT_TO_REF:-}" ]]; then
|
||||
return 0
|
||||
fi
|
||||
git -C "$ROOT" diff --name-only \
|
||||
"$PRE_COMMIT_FROM_REF" "$PRE_COMMIT_TO_REF" 2>/dev/null || true
|
||||
}
|
||||
|
||||
# True when the whole repo must be swept: no known range, or a gate file moved.
|
||||
needs_full_sweep() {
|
||||
local paths="$1"
|
||||
[[ -z "$paths" ]] && return 0
|
||||
grep -qE "$GATE_WIDENING_RE" <<<"$paths" && return 0
|
||||
return 1
|
||||
}
|
||||
|
||||
# True when the push touches Python at all. The pytest stage is all-or-nothing
|
||||
# by design (whole-repo 100% branch coverage), so this is a run/skip decision,
|
||||
# not a subset one.
|
||||
touches_python() {
|
||||
grep -qE '\.py$|^requirements\.txt$' <<<"$1"
|
||||
}
|
||||
|
||||
# Map changed paths to the lib/tests suites that must run.
|
||||
#
|
||||
# Printing nothing means no shell suite needs to run. A shared-foundation
|
||||
# change prints every suite instead of just the enclosing one.
|
||||
select_shell_suites() {
|
||||
local paths="$1" all_suites="$2"
|
||||
if ! grep -qE '\.(sh|bash)$|/tests/' <<<"$paths"; then
|
||||
return 0
|
||||
fi
|
||||
if grep -qE "$SHELL_WIDENING_RE" <<<"$paths"; then
|
||||
printf '%s\n' "$all_suites"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# A suite covers the entry scripts in its PARENT directory as well as its
|
||||
# own lib/, so the parent is the right scope -- but match it
|
||||
# non-recursively. `scripts/lib/tests` has parent
|
||||
# `linux_configuration/scripts`, which is a prefix of every other suite's
|
||||
# path; a recursive match there would fire it on every shell push.
|
||||
# `[^/]+$` keeps a path in the parent itself, while the second alternative
|
||||
# picks up the suite's own lib/ and tests/ at any depth.
|
||||
local suite parent
|
||||
while IFS= read -r suite; do
|
||||
[[ -z "$suite" ]] && continue
|
||||
parent="${suite%/lib/tests/run_all.sh}"
|
||||
parent="${parent#./}"
|
||||
if grep -qE "^${parent}/([^/]+$|lib/)" <<<"$paths"; then
|
||||
printf '%s\n' "$suite"
|
||||
fi
|
||||
done <<<"$all_suites"
|
||||
}
|
||||
119
meta/scripts/ci_mirror_shell.sh
Normal file
119
meta/scripts/ci_mirror_shell.sh
Normal file
@ -0,0 +1,119 @@
|
||||
# shellcheck shell=bash
|
||||
# ============================================================================
|
||||
# ci_mirror_shell.sh — the shell half of the CI mirror.
|
||||
#
|
||||
# Sourced by ci_mirror.sh. Splits out of it to stay under the 250-line cap.
|
||||
# Provides run_shell_test_gate, which mirrors the shell-tests workflow, and
|
||||
# run_jailed_suite, which wraps a suite that declares a jail_args file.
|
||||
#
|
||||
# Uses ci_mirror.sh's CHECKOUT/fail/log and ci_mirror_range.sh's
|
||||
# select_shell_suites / needs_full_sweep.
|
||||
# ============================================================================
|
||||
|
||||
# Mirror the shell-tests workflow's side-effect-free half.
|
||||
#
|
||||
# Without this, ci-mirror covered the pre-commit and python-tests workflows but
|
||||
# not Shell tests, so a shell change could pass every local gate and still go
|
||||
# red on push — which is exactly what happened repeatedly during the
|
||||
# 250-line-cap campaign: three separate test files asserted on code that a
|
||||
# split had moved, and each was only caught after the push.
|
||||
#
|
||||
# The Arch-container half of that workflow is deliberately NOT mirrored: it
|
||||
# needs an Arch userland running as root to touch /etc, which is what the
|
||||
# disposable container provides and a developer machine must not.
|
||||
# A suite that writes to real system paths declares it with a `jail_args` file
|
||||
# beside its run_all.sh, holding the --bind/--seed-* flags it needs. Such a
|
||||
# suite REFUSES to run bare -- that guard is what stops a test run from
|
||||
# rewriting the developer's own /etc -- so invoking it directly fails the gate.
|
||||
#
|
||||
# Fails closed: a marker present but empty is a mistake, not a licence to skip.
|
||||
run_jailed_suite() {
|
||||
local runner="$1"
|
||||
local args_file="$CHECKOUT/${runner%/run_all.sh}/jail_args"
|
||||
|
||||
local jail_args=()
|
||||
while IFS= read -r arg; do
|
||||
jail_args+=("$arg")
|
||||
done < <(grep -v '^#\|^$' "$args_file" || true)
|
||||
|
||||
if [[ ${#jail_args[@]} -eq 0 ]]; then
|
||||
fail "jail_args is present but empty: $args_file"
|
||||
fi
|
||||
|
||||
# --min 0: this gate asks "did the assertions pass?", not "is the lib at
|
||||
# 100%?". Without it the jail measures the runner itself against its
|
||||
# default --min 100 and fails the push on a number no one asked for.
|
||||
# The coverage bar is check_shell_coverage.sh's job, per-lib and --measure'd.
|
||||
# Output is captured, not discarded: on success it is noise, but on failure
|
||||
# it holds the `warn: case exited` line and the suite's own FAIL: lines.
|
||||
# Without this the gate reports only "suite failed" and cannot be diagnosed.
|
||||
local out
|
||||
if ! out="$(cd "$CHECKOUT" && bash meta/scripts/shell_coverage_jail.sh \
|
||||
--subject "$runner" "${jail_args[@]}" \
|
||||
--min 0 --fail-on-case-error -- "" 2>&1)"; then
|
||||
printf '%s\n' "$out" >&2
|
||||
fail "jailed lib/tests suite failed: $runner"
|
||||
fi
|
||||
}
|
||||
|
||||
run_shell_test_gate() {
|
||||
local paths="$1"
|
||||
local all_runners
|
||||
all_runners="$(cd "$CHECKOUT" && find . -path ./node_modules -prune -o \
|
||||
-path '*/lib/tests/run_all.sh' -print | sort)"
|
||||
|
||||
if [[ -z "$all_runners" ]]; then
|
||||
fail "no lib/tests/run_all.sh found — the discovery glob is wrong"
|
||||
fi
|
||||
|
||||
local selected
|
||||
if needs_full_sweep "$paths"; then
|
||||
selected="$all_runners"
|
||||
else
|
||||
selected="$(select_shell_suites "$paths" "$all_runners")"
|
||||
fi
|
||||
|
||||
if [[ -z "$selected" ]]; then
|
||||
log "no shell files in the push range — skipping the lib/tests suites"
|
||||
else
|
||||
log "lib/tests suites (mirrors the shell-tests workflow)"
|
||||
fi
|
||||
|
||||
local runners=()
|
||||
while IFS= read -r runner; do
|
||||
[[ -n "$runner" ]] && runners+=("$runner")
|
||||
done <<<"$selected"
|
||||
|
||||
local runner
|
||||
for runner in "${runners[@]}"; do
|
||||
if [[ -f "$CHECKOUT/${runner%/run_all.sh}/jail_args" ]]; then
|
||||
run_jailed_suite "$runner"
|
||||
else
|
||||
(cd "$CHECKOUT" && "$runner" >/dev/null) ||
|
||||
fail "lib/tests suite failed: $runner"
|
||||
fi
|
||||
done
|
||||
|
||||
log "side-effect-free shell tests (mirrors the shell-tests workflow)"
|
||||
# Parsed OUT of the workflow rather than duplicated here. A hardcoded copy
|
||||
# drifts the moment someone adds a test to the workflow, and a mirror that
|
||||
# silently checks less than CI is worse than no mirror: it reports safe.
|
||||
# The awk range is the shell-tests job's `tests=( ... )` array, taking the
|
||||
# first one only — the second belongs to the Arch container job, which is
|
||||
# deliberately not mirrored.
|
||||
local shell_tests=()
|
||||
while IFS= read -r shell_test; do
|
||||
shell_tests+=("$shell_test")
|
||||
done < <(awk '/^ *tests=\(/{n++} n==1 && /^ *test_.*\.sh *$/{gsub(/ /,"");print} n==1 && /^ *\)/{exit}' \
|
||||
"$CHECKOUT/.github/workflows/shell-tests.yml")
|
||||
|
||||
if [[ ${#shell_tests[@]} -eq 0 ]]; then
|
||||
fail "could not parse the test list out of shell-tests.yml"
|
||||
fi
|
||||
|
||||
local shell_test
|
||||
for shell_test in "${shell_tests[@]}"; do
|
||||
(cd "$CHECKOUT" && bash "linux_configuration/tests/$shell_test" >/dev/null 2>&1) ||
|
||||
fail "shell test failed: $shell_test"
|
||||
done
|
||||
}
|
||||
Loading…
Reference in New Issue
Block a user