testsAndMisc/NEXT_SESSION_PROMPT.md
Krzysztof kuhy Rudnicki 0048293a2b
Some checks are pending
Pre-commit checks / pre-commit (push) Waiting to run
Shell tests / shell-tests (push) Waiting to run
Shell tests / arch-tests (push) Waiting to run
NEXT_SESSION_PROMPT.md: record Phase 3 as built-but-unenforced
Replaces the Q1/Q2/Q3 grilling block with the answers the ratchet was
built under, flagged as cheaply vetoable since the hook is not yet
wired. Documents the one remaining step (the .pre-commit-config.yaml
entry, denied as a sensitive file) and corrects the file-count method:
`find` reports 565 .sh because it walks .venv and node_modules, which
is how the stale "roughly 487" got in here.
2026-08-21 17:02:31 +02:00

175 lines
9.3 KiB
Markdown

# Next session: Phase 3 — finish wiring the shell-coverage ratchet
> **Paste this whole file into a fresh Claude session opened in `~/testsAndMisc`.**
> It is self-contained. Do not go looking for the previous session's context.
**Phases 1 and 2 are DONE.** The 250-line cap is at zero and enforced; the CI
gates are wired.
**Phase 3 is BUILT but NOT ENFORCED** (commit `3278bebd`). The three scoping
questions below were answered without the user, because the session that built
it was non-interactive — a grilling round would have ended the turn with
nothing delivered. **The user can still veto any of these cheaply**, since the
gate is not yet in the hook chain:
- **Q1 Scope → ratchet.** 125 pre-existing libraries are exempt via
`meta/shell-coverage-allowlist.txt`; only new or modified libraries must be
covered. Unrelated commits are never blocked (verified).
- **Q2 Bar → presence, not a percentage.** A numeric bar is unreachable here:
Phase 1 measured the effecting code at 0% because it sudos, pkills and
installs, so hitting a number needs either mass shimming or the suppressions
this repo forbids. The gate asserts a `tests/run_all.sh` exists beside the lib.
- **Q3 Which files → `.sh` under a `lib/` dir**, excluding `lib/tests/` and
`lib/payloads/`. Entry scripts are orchestration whose bodies are untestable.
## The one thing left to do
`meta/scripts/check_shell_coverage.sh` works standalone but is **not in
`.pre-commit-config.yaml`** — that edit was denied as a sensitive file. Add
this block after the `file-length-cap` hook (note `files:`, **not** `--all`;
per-file is what makes it a ratchet):
```yaml
- id: shell-coverage-ratchet
name: Shell libraries must have a test suite
entry: bash meta/scripts/check_shell_coverage.sh
language: system
files: \.sh$
```
Then re-verify the negative case before trusting it:
`bash meta/scripts/check_shell_coverage.sh meta/scripts/check_file_length.sh`
must exit 0 (an unrelated entry script must never block a commit).
**Do not run `--seed` after adding an untested library** — it would rewrite the
allowlist and silently exempt it. The list is shrink-only; nothing mechanically
enforces that yet.
## Chipping away at the 125
The allowlist is concentrated: `single_use/features/lib` (45),
`single_use/fixes/lib` (22), `phone_focus_mode/lib` (9), `scripts/lib` (9).
Adding one `tests/run_all.sh` to a directory enforces every file in it at once,
so expect the work to arrive in batches rather than one file at a time.
## What is already true (verify, do not redo)
```bash
bash meta/scripts/check_file_length.sh --all # "all checked files are within 250 lines"
bash meta/scripts/check_ci_green.sh # exit 0 while main is green
gh run list --limit 5
```
- **Every file is under 250 lines.** The five deployment-trap files were split:
`install_leechblock.sh` (485→119), `block_compulsive_opening.sh` (705→182),
`setup_night_lockdown.sh` (918→127), `pacman_wrapper.sh` (929→217),
`setup_midnight_shutdown.sh` (1734→109), plus `_glyph_art.py` (253→208).
- **`file-length-cap`** runs `check_file_length.sh --all` on every commit.
- **`ci-baseline-green`** refuses to commit onto a red `main` (fails closed on
red, warns and passes when it cannot tell). Bypass: `CI_GREEN_SKIP=1`.
- **`ci-mirror`** on pre-push now also runs every `*/lib/tests/run_all.sh` and
the side-effect-free half of `shell-tests.yml`.
- **`shell-tests.yml`** discovers and runs every `*/lib/tests/run_all.sh`. Four
such suites existed before and none of them ran in CI until this was added.
## Measuring the scale — use `git ls-files`, not `find`
```bash
git ls-files '*.sh' | wc -l # 560 tracked
git ls-files '*/lib/*.sh' | grep -v '/lib/tests/' | wc -l # 208 libraries
git ls-files '*/lib/tests/run_all.sh' # 4 suites
bash meta/scripts/check_shell_coverage.sh --all # 125 uncovered, 125 exempt
```
A bare `find` reports 565 `.sh` files because it walks `.venv/`,
`.ci-mirror-venv/` and `node_modules/`. The earlier "roughly 487" figure in
this file came from an unfiltered walk of a different tree state. Seeding an
allowlist from an unfiltered `find` would bake vendored venv scripts into a
tracked file, and `check_file_length.sh --all` passing would not reveal it,
because that script carries its own separate exclusion list.
## What Phase 1 measured — why the bar is presence, not a percentage
Coverage of the new libs, measured with
`bash meta/scripts/shell_coverage.sh <lib/tests/run_all.sh> <lib-basename> 0`:
| lib | coverage | what is uncovered |
| --------------------------------------------------------- | -------------- | ------------------------------------------------ |
| `cco_state.sh` | 93.94% (62/66) | only `kill_app`'s two `pkill` lines |
| `leechblock_fetch.sh` | 38.57% (27/70) | `download_extension`: curl/tar/unzip |
| `leechblock_browsers.sh` | 36.47% (31/85) | `replace_browser_in_place`: sudo binary patching |
| `cco_wrapper/install/report`, `leechblock_config/firefox` | 0% | every function pkills, sudo-writes or installs |
**The pattern: the decision logic reaches ~90%+, the effecting code reaches 0%.**
A 100% bar means shimming `sudo`, `pkill`, `systemctl`, `curl` and `npm` for
every installer in the repo. That is the real cost to put in front of the user.
## The harness that works — extend it, do not clone it
`linux_configuration/scripts/periodic_background/digital_wellbeing/lib/tests/`
holds `leechblock_harness.sh` and `cco_harness.sh` plus a `run_all.sh` whose
glob is `test_*.sh`. **One harness per directory, not per file** — `jscpd`
fails above 2% duplication and five near-identical harnesses is exactly how
that trips.
Two patterns worth copying:
- **`_t_isolate_path`** narrows `PATH` to a shim dir. It is the only way to
make `command -v jq` or `command -v chromium` fail on a machine that has
them. **Every external the code under test calls must be seeded into that
dir** — a missing `mkdir` aborted the suite under `set -e` with _no stderr_,
which under kcov looked like a coverage-tool bug for twenty minutes.
- **A fixture self-check.** `cco_harness.sh` asserts its own globals are
populated and calls `notify` for real. This is not linter appeasement: it is
what turns "a typo in a global name" from a silent skipped assertion into a
loud failure, and it happens to satisfy SC2034 honestly.
## Traps that cost real time in Phase 1 — all still live
- **`shfmt -w` corrupts `[hyphenated-keys]` in associative arrays**, turning
`${BROWSERS[google-chrome-stable]}` into a subtraction expression. **Quote the
subscripts.** Reproduced live in a test file during this campaign. Canary:
`grep -nE '\[[a-z0-9]+ - ' <file>` must be empty after every `shfmt` and
every `pre-commit run`.
- **A comment line starting with `# shellcheck` is parsed as a directive** and
fails with SC1072/SC1073. Write "the linter", not "shellcheck", at the start
of a comment line.
- **`verify_shell_split.sh` cannot see heredocs.** In
`setup_midnight_shutdown.sh` it reported a bare `-log`, because `log()` sits
at column 0 _inside a generated script_. For any file that emits scripts,
the real check is **hashing the emitted content**, not the function list.
- **New sourced libs need a shebang AND the exec bit**, staged with
`git add --chmod=+x`. A plain `git add` afterwards resets it. Payload files
under `lib/payloads/` are `100755` too — the shebang hook rejects `100644`.
- **`gh` and `git push` race.** Several pushes failed with "cannot lock ref";
`git fetch` then re-push. Not a divergence.
## Known-broken, NOT caused by the campaign
**`Python tests` has been failing since 2026-08-17** — before any of this work.
The cause is `pip` hitting `error: resolution-too-deep` while installing
`meta/requirements.txt`, on commits that touch no Python at all. It needs
dependency pinning, and it is why `python-tests.yml` is deliberately **not** in
`check_ci_green.sh`'s required list. Worth fixing on its own; do not fold it
into Phase 3.
## Rules that will bite you
- **No suppressions, ever.** No `# noqa`, no `shellcheck disable`, no
per-file-ignore without asking every single time.
- **Every commit touching code needs evidence** in
`docs/superpowers/evidence/<slug>-<date>.json`, **plus a contract** in
`docs/superpowers/contracts/` once ≥4 code files are staged. Put the
**measured** number in it, never a rounded or hoped-for one.
- **A test file broken by your commit is a same-session bug.** Three were
broken this way in Phase 1 (`security_apps.sh`, `test_shutdown_timer_monitor.sh`,
and the pacman pair) — each asserted on code a split had moved. Fix the
assertion to search the entry script _and_ its libs; do not weaken it.
- `pre-commit run --files <changed>` **after** `git add`. `prettier` and
`ci-mirror` run on **pre-push**. `npx prettier --write` any `.md`/`.json`.
- Work directly on `main`. `git stash` and branch creation are blocked.
- **Verify what actually got committed.** Re-grep the file after committing;
an autoformat pass stripped a load-bearing variable definition three times in
Phase 1, once landing a `set -u` abort in `setup_night_lockdown.sh` that had
to be fixed in a follow-up commit.