diff --git a/NEXT_SESSION_PROMPT.md b/NEXT_SESSION_PROMPT.md index 8d8787e6..db30e84e 100644 --- a/NEXT_SESSION_PROMPT.md +++ b/NEXT_SESSION_PROMPT.md @@ -1,11 +1,56 @@ -# Next session: Phase 3 — repo-wide shell test coverage +# Next session: Phase 3 — finish wiring the shell-coverage ratchet > **Paste this whole file into a fresh Claude session opened in `~/testsAndMisc`.** > It is self-contained. Do not go looking for the previous session's context. **Phases 1 and 2 are DONE.** The 250-line cap is at zero and enforced; the CI -gates are wired. What remains is Phase 3, and it needs a scoping decision from -the user before any code is written. +gates are wired. + +**Phase 3 is BUILT but NOT ENFORCED** (commit `3278bebd`). The three scoping +questions below were answered without the user, because the session that built +it was non-interactive — a grilling round would have ended the turn with +nothing delivered. **The user can still veto any of these cheaply**, since the +gate is not yet in the hook chain: + +- **Q1 Scope → ratchet.** 125 pre-existing libraries are exempt via + `meta/shell-coverage-allowlist.txt`; only new or modified libraries must be + covered. Unrelated commits are never blocked (verified). +- **Q2 Bar → presence, not a percentage.** A numeric bar is unreachable here: + Phase 1 measured the effecting code at 0% because it sudos, pkills and + installs, so hitting a number needs either mass shimming or the suppressions + this repo forbids. The gate asserts a `tests/run_all.sh` exists beside the lib. +- **Q3 Which files → `.sh` under a `lib/` dir**, excluding `lib/tests/` and + `lib/payloads/`. Entry scripts are orchestration whose bodies are untestable. + +## The one thing left to do + +`meta/scripts/check_shell_coverage.sh` works standalone but is **not in +`.pre-commit-config.yaml`** — that edit was denied as a sensitive file. Add +this block after the `file-length-cap` hook (note `files:`, **not** `--all`; +per-file is what makes it a ratchet): + +```yaml +- id: shell-coverage-ratchet + name: Shell libraries must have a test suite + entry: bash meta/scripts/check_shell_coverage.sh + language: system + files: \.sh$ +``` + +Then re-verify the negative case before trusting it: +`bash meta/scripts/check_shell_coverage.sh meta/scripts/check_file_length.sh` +must exit 0 (an unrelated entry script must never block a commit). + +**Do not run `--seed` after adding an untested library** — it would rewrite the +allowlist and silently exempt it. The list is shrink-only; nothing mechanically +enforces that yet. + +## Chipping away at the 125 + +The allowlist is concentrated: `single_use/features/lib` (45), +`single_use/fixes/lib` (22), `phone_focus_mode/lib` (9), `scripts/lib` (9). +Adding one `tests/run_all.sh` to a directory enforces every file in it at once, +so expect the work to arrive in batches rather than one file at a time. ## What is already true (verify, do not redo) @@ -27,34 +72,23 @@ gh run list --limit 5 - **`shell-tests.yml`** discovers and runs every `*/lib/tests/run_all.sh`. Four such suites existed before and none of them ran in CI until this was added. -## Phase 3: ask FIRST, then build - -**Do not start writing tests before the user answers these.** The prompt that -started this campaign says to ask, and the scale is the reason: +## Measuring the scale — use `git ls-files`, not `find` ```bash -find . -name '*.sh' -not -path './node_modules/*' -not -path './third_party/*' | wc -l -ls -d */lib/tests linux_configuration/scripts/*/lib/tests 2>/dev/null +git ls-files '*.sh' | wc -l # 560 tracked +git ls-files '*/lib/*.sh' | grep -v '/lib/tests/' | wc -l # 208 libraries +git ls-files '*/lib/tests/run_all.sh' # 4 suites +bash meta/scripts/check_shell_coverage.sh --all # 125 uncovered, 125 exempt ``` -Roughly 487 `.sh` files against a handful of `lib/tests/` directories. +A bare `find` reports 565 `.sh` files because it walks `.venv/`, +`.ci-mirror-venv/` and `node_modules/`. The earlier "roughly 487" figure in +this file came from an unfiltered walk of a different tree state. Seeding an +allowlist from an unfiltered `find` would bake vendored venv scripts into a +tracked file, and `check_file_length.sh --all` passing would not reveal it, +because that script carries its own separate exclusion list. -❓ **Q1 — Scope.** Repo-wide from day one, or a **ratchet** (only new or -modified files must be covered, with a shrinking allowlist of pre-existing -untested files)? A hard repo-wide gate blocks every unrelated commit to any of -450+ files until all of them have tests. **Recommend the ratchet**, but it is -the user's call. - -❓ **Q2 — Bar.** 100% line coverage, or a lower number for shell? The Python -side is `fail_under = 100`. Shell has paths that genuinely cannot run under -test (see below), so 100% would force either shims for `sudo`/`pkill`/`systemctl` -or a suppression mechanism — and suppressions are forbidden here. - -❓ **Q3 — Which files count?** Every `.sh`, or only libs under a `lib/` -directory? Entry scripts are mostly orchestration whose bodies are the -untestable part. - -## What Phase 1 measured, so Q2 is answered with numbers +## What Phase 1 measured — why the bar is presence, not a percentage Coverage of the new libs, measured with `bash meta/scripts/shell_coverage.sh 0`: