diff --git a/NEXT_SESSION_PROMPT.md b/NEXT_SESSION_PROMPT.md index 7312117e..be8e4ddd 100644 --- a/NEXT_SESSION_PROMPT.md +++ b/NEXT_SESSION_PROMPT.md @@ -1,201 +1,208 @@ -# Next session: the instrument is FIXED — go clear the allowlist +# Next session: test the 16 untested `fixes/lib` libs to 100% > **Paste this whole file into a fresh Claude session opened in `~/testsAndMisc`.** > It is self-contained. Do not go looking for the previous session's context. -## What changed: the bottleneck is gone +## The job -The previous handoff said "the instrument is the bottleneck, not the tests." -That is **no longer true**. Both kcov defects are fixed and pushed -(`81191f40`, `29b8c5a7`). The two libs that were pinned at absurd numbers were -never test-limited: +`linux_configuration/scripts/single_use/fixes/lib/` has 22 libs. Six are done +and off the allowlist. **Sixteen have no tests at all.** Write them, to 100% +line coverage, and take each one off `meta/shell-coverage-allowlist.txt`. -| lib | was | now | -| ------------------------- | --------------- | ------------------- | -| `dwm_config.sh` | 35/73 = 47.95% | **65/66 = 98.48%** | -| `rpi_nc_install.sh` | 10/88 = 11.36% | **85/88 = 96.59%** | -| `rpi_nc_ca.sh` | 72/73 = 98.63% | **73/73 = 100.00%** | -| `aw_autostart.sh` | 68/82 = 82.93% | **80/82 = 97.56%** | -| `clean_audio_filters.sh` | 83/93 = 89.25% | **85/93 = 91.40%** | -| `dot_resolver_install.sh` | 39/39 = 100.00% | 39/39 = 100.00% | +That is the largest remaining block of real test-writing work in the campaign. -Nothing went down. **Every allowlisted lib is worth re-measuring before you -write a single new test** — several are probably already at or near 100%. +| lines | lib | family | +| ----- | ------------------------- | -------------------------------------- | +| 215 | `ubuntu_perf_fixes.sh` | ubuntu perf | +| 214 | `bt_audio.sh` | bluetooth (`fix_bluetooth.sh`) | +| 202 | `arch_hardware.sh` | arch perf (`optimize_arch_desktop.sh`) | +| 180 | `nvidia_config.sh` | arch perf | +| 179 | `thorium_repairs.sh` | standalone | +| 166 | `arch_perf_probes.sh` | arch perf | +| 161 | `hosts_guard_migrate.sh` | hosts guard | +| 150 | `bt_adapter.sh` | bluetooth | +| 141 | `hosts_guard_rollback.sh` | hosts guard | +| 128 | `bt_pairing.sh` | bluetooth | +| 126 | `arch_cpu.sh` | arch perf | +| 122 | `arch_sysctl.sh` | arch perf | +| 121 | `arch_perf_fixes.sh` | arch perf | +| 103 | `arch_perf_report.sh` | arch perf | +| 80 | `ubuntu_perf_more.sh` | ubuntu perf | +| 76 | `bt_report.sh` | bluetooth | -## How the instrument works now (read before changing it) +**Suggested order: smallest first, by family.** `bt_report.sh` (76) then the +rest of the bluetooth four; the arch-perf six share `optimize_arch_desktop.sh` +and will share stubs. Do not start with `ubuntu_perf_fixes.sh`. + +## Everything you need already exists — do NOT build a new harness + +`fixes/lib/tests/` is a working suite. It has: + +- `run_all.sh` — the runner. **This is the subject the gate measures through.** +- `pacman_hook_stall_harness.sh` (205 lines) — lib-level harness: `mktemp -d` + sandbox, a `FAKE_BIN` PATH stub dir, `_t_eq` / `_t_pass` / `_t_fail`, + and `_t_unstub` (which also runs `hash -r`). +- `pacman_hook_stall_entry_harness.sh` (142 lines) — same for entry scripts. +- Eight `test_*.sh` files to copy the shape from. + +**jscpd fails the commit above 2% duplication**, so extend/parameterise these +rather than cloning a harness per family. If a family genuinely needs +different setup, add a small harness that _sources_ the existing one. + +No `jail_args` file sits beside `run_all.sh`, deliberately: the harness +confines every write to its own `mktemp -d` and stubs external commands on +PATH. Keep it that way — the new tests must not write to `/etc`, `/var` or +`/usr`. If a lib insists on writing an absolute path, stub the writer, do not +add a bind mount. + +## What these libs touch (drives your stubs) + +`systemctl` (most common), `bluetoothctl`, `pacman`, `sysctl`, `sudo`, +`nvidia-*`, `apt-get`. `systemctl`, `pacman` and `sudo` are already in the +jail's `DEFAULT_SHIMS`; the rest you stub in `FAKE_BIN`. + +Two entry scripts source these libs, and are the place to look for how each +lib is really called: + +- `linux_configuration/scripts/single_use/fixes/fix_bluetooth.sh` +- `linux_configuration/scripts/single_use/fixes/optimize_arch_desktop.sh` + +## The loop, per lib + +```bash +# 1. write tests/test_.sh, chmod +x it +# 2. run the suite bare first -- much faster than the jail +bash linux_configuration/scripts/single_use/fixes/lib/tests/run_all.sh + +# 3. then measure through the runner, in the jail +timeout 900s bash meta/scripts/shell_coverage_jail.sh \ + --subject linux_configuration/scripts/single_use/fixes/lib/tests/run_all.sh \ + --measure .sh --min 1 --fail-on-case-error -- "" + +# 4. close the gap, repeat until 100.00% +``` + +**Always measure through `run_all.sh`, never a single test file.** That is the +subject `is_covered()` builds, and the runner's number can be LOWER than a +suite's own. + +## The bar for taking a lib off the allowlist + +Do all three. This is the standard the five cleared libs met: + +1. Measure 100.00% through `run_all.sh`. +2. **Measure again at `--min 100`** so a shortfall is a non-zero exit rather + than a number you might misread. +3. Remove the line from `meta/shell-coverage-allowlist.txt`, then run the real + gate with no exemption and confirm exit 0: + ```bash + bash meta/scripts/check_shell_coverage.sh .sh + # expect: "every checked shell library meets the coverage bar" + ``` + +Two matching runs matters: this directory's suites drive timing loops, and a +timing-dependent suite can vary its line set between runs (`dwm_config.sh` was +observed doing exactly that). A flaky 100% becomes a red CI job whose cause is +invisible. + +Allowlist is at **98** entries. Each lib you finish takes it down by one. + +## The instrument is FIXED — trust its numbers now Coverage = `(PS4-traced lines | kcov hits) & (kcov line set - continuation lines)` -Two passes over the same cases, in two fresh namespaces. They **cannot** be -merged: kcov's ptrace and bash's xtrace are mutually exclusive — under -`SHELLOPTS=xtrace` every line kcov recorded as `hits="1"` comes back -`hits="0"`, measured A/B. +Two passes, two namespaces, because kcov's ptrace and bash's xtrace cannot +share a process. Full detail in `docs/kcov-under-report.md`. You should not +need to touch it. If you do, the acceptance test is +`dot_resolver_install.sh` = 39/39 and `rpi_nc_ca.sh` = 73/73, and **no lib's +percentage may go DOWN** — a drop means a lost trace, not a stricter +instrument. -**The trace must never supply the denominator.** A trace reports only lines -that RAN, so trace-as-denominator makes every subject 100.00% and gates on -nothing. kcov's line set is what keeps the gate meaningful. - -Full detail, including the three findings that each yield a silently wrong -trace, is in `docs/kcov-under-report.md`. The load-bearing one: **bash under -`unshare --map-root-user` runs privileged and DISCARDS an inherited `PS4`**, -so tracing is delivered through a `BASH_ENV` file instead. - -## The scoping answers are SETTLED. Do not re-ask them. - -Repo-wide gate (not a ratchet) · **100%** line coverage · **ALL** `.sh` · -pre-commit **AND** pre-push **AND** CI · tests first, arm the gate last · -hardest first. - -## START HERE — re-measure the allowlist - -98 libs are allowlisted on numbers produced by a broken instrument. Sweep -them with the fixed one before writing tests. The invocation: - -```bash -bash -c ' -ja=(); while IFS= read -r l; do ja+=("$l"); done \ - < <(grep -v "^#\|^$" /tests/jail_args) -timeout 900s bash meta/scripts/shell_coverage_jail.sh \ - --subject /tests/run_all.sh \ - "${ja[@]}" --measure .sh --min 1 --fail-on-case-error -- ""' -``` - -Omit the `ja` array entirely for suites with no `jail_args` (transcribe, -fixes) — those confine their writes to a `mktemp -d` and run bare. - -**Always measure through `run_all.sh`, never a single test file.** That is the -subject `is_covered()` builds. - -## Directories and their harness state +One bound to know: the report may print ``` -41 libs linux_configuration/scripts/single_use/features/lib/ harness + jail_args - 5 libs .../misc/testsAndMisc-bash/lib/ harness, no jail_args -22 libs .../single_use/fixes/lib/ run_all.sh ADDED this session; 6 of 22 tested - 9 libs phone_focus_mode/lib/ NO harness +outside kcov's line set (ran, but counted in neither half): 68, 70, ... ``` -`fixes/lib` was listed as "no harness at all" in the previous handoff. That -was wrong — 8 test files and 2 harnesses already existed; what was missing was -`run_all.sh`, which is what `is_covered()` measures through. It now exists and -all 8 test files pass through it. **16 of the 22 libs there still have no -tests at all** — that is the largest remaining block of real test-writing work. - -## Already measured this session — do NOT redo these - -All through `run_all.sh` on the fixed instrument. `fixes/lib` needs no -`jail_args`; the features libs use the one beside their `run_all.sh`. - -| lib | coverage | note | -| ------------------------------ | ------------------- | ------------------------- | -| `rpi_nc_ca.sh` | **73/73 = 100.00%** | measured 3x, solid | -| `dot_resolver_install.sh` | **39/39 = 100.00%** | acceptance baseline | -| `pacman_hook_stall_capture.sh` | **59/59 = 100.00%** | fixes/lib | -| `pacman_hook_stall_load.sh` | **25/25 = 100.00%** | fixes/lib | -| `pacman_hook_stall_summary.sh` | **18/18 = 100.00%** | fixes/lib; off-set: 36 | -| `pacman_hook_stall_watch.sh` | **24/24 = 100.00%** | fixes/lib | -| `pacman_hook_stall_usage.sh` | **1/1 = 100.00%** | body is one heredoc | -| `dwm_config.sh` | 65/66 = 98.48% | line 39 is `done < <(…)` | -| `pacman_hook_stall_setup.sh` | 26/27 = 96.30% | uncovered: 7 | -| `aw_autostart.sh` | 80/82 = 97.56% | 11 off-set lines | -| `rpi_nc_install.sh` | 85/88 = 96.59% | uncovered: 70, 202, 216 | -| `transcribe_pkgmgr.sh` | 49/50 = 98.00% | uncovered: 42 | -| `clean_audio_filters.sh` | 85/93 = 91.40% | uncovered incl. 52-58 | -| `transcribe_deps.sh` | 48/52 = 92.31% | uncovered: 19, 52, 53, 55 | - -**Five were CLEARED this session** (`83de41f4`): the pacman_hook_stall libs -`capture`, `load`, `summary`, `usage` and `watch`. Each was measured twice, -the second time gating at `--min 100`, then confirmed through the real -`check_shell_coverage.sh` with no exemption — all exit 0. **Allowlist: -103 -> 98.** - -`rpi_nc_ca.sh` and `dot_resolver_install.sh` are also at 100.00% and measured -3x / 2x, but they live in `features/lib`, where 39 other libs are still -allowlisted. Removing them is safe by the same standard whenever you want. - -The bar to reuse: **two matching runs, the second at `--min 100`, then the -real gate.** `fixes/lib`'s suites drive `watch_forever` timing loops, and a -timing-dependent suite can vary its line set run to run (`dwm_config.sh` was -seen to). A flaky 100% becomes a red CI job whose cause is invisible. - -Allowlist entries are **static per-file lines**, verified: adding -`fixes/lib/tests/run_all.sh` did NOT start failing the 16 untested libs in -that directory — `check_shell_coverage.sh ` still exits 0, -because each is listed explicitly. The allowlist header's "every file in that -directory is enforced" describes intent, not current mechanics. - -## Two instrument bounds you must know - -**Defect (c): kcov's LINE SET is incomplete too.** The report now prints, per -subject, any line the trace saw run that kcov never listed: - -``` -outside kcov's line set (ran, but counted in neither half): 68, 70, 73, ... -``` - -This **cannot inflate a percentage**. A line reaches that list only because -the trace saw it EXECUTE, so folding them in only ever raises the number -(`aw_autostart.sh` 80/82 = 97.56% -> 91/93 = 97.85%). It can never hide an -untested line. What it does mean: such a subject is measured over a SUBSET of -its statements. Read the list before clearing that lib. - -`aw_autostart.sh` is the known case — kcov lists nothing after the -`{ ... } >>"$file"` command group at lines 61-65. - -**`nc_php.sh` does not measure at all.** "kcov instrumented no lines", on the -fixed instrument AND on the pre-fix one (checked at `078d3463` in a throwaway -worktree). Not a regression. The 84/85 = 98.82% in older notes does not -reproduce — treat it as unverified. +That **cannot inflate a number** — such a line is by construction one the +trace saw execute, so folding it in only ever raises the figure. It does mean +the lib is measured over a subset of its statements. Read the list before +clearing that lib; if the lines are ordinary statements, say so in the +evidence. ## Traps that cost real time -- **`--fail-on-case-error` or you are flying blind.** The jail suppresses a +- **Never edit the working tree while a measurement runs.** The jail + fingerprints `git status --porcelain` and aborts with "a write escaped the + jail". Editing an unrelated file trips it. FALSE alarm; cost two runs last + session, the second time after it had already been diagnosed. Commit before + starting a sweep; use the scratchpad for notes. +- **A sweep's output file buffers.** `pgrep -x kcov` is the liveness signal, + not file growth. A measurement silent for minutes is usually still running; + this directory's suites take 30s+ per kcov pass, and there are two passes. +- **Do not `pgrep -f shell_coverage_jail` in a wait loop** — the loop's own + command line matches, so it never exits. +- **`--fail-on-case-error` or you are flying blind.** The jail suppresses the suite's stdout, so assertions fail invisibly without it. - **To find WHERE a suite dies, use an `exit ` sentinel.** The jail surfaces the code. The only way to tell a tracing failure from an abort. - **`rm -f "$TEST_TMPDIR/bin/foo"` does NOT hide foo.** bash caches executable - locations. Use the harnesses' `_t_unstub`, which also runs `hash -r`. + locations. Use `_t_unstub`, which also runs `hash -r`. - **A prepended stub dir cannot hide a real binary.** If the host genuinely - has the tool, REPLACE PATH or jail it. + has the tool, REPLACE PATH or jail it, or the "not installed" case tests + nothing. - **A stub must materialise what the real tool creates.** A record-only stub in a `[[ ! -s $file ]]` chain silently exercises the NEXT branch. - **A top-level stub function SHADOWS the subject's own** for every later assertion. Put redefinitions in a subshell. -- **`shfmt` and `prettier` reformat after your own check passes** — re-check - the 250-line cap after staging, and expect the pre-push prettier hook to - reject markdown you hand-formatted. +- **`shfmt` and `prettier` reformat after your check passes** — re-check the + 250-line cap after staging, and expect the pre-push prettier hook to reject + hand-formatted markdown. - **Commits and pushes exceed a 2-minute foreground timeout.** Background them. -- **Never edit the working tree while a measurement runs.** The jail - fingerprints `git status --porcelain` before each case and aborts with - "the subject modified the working tree; a write escaped the jail" if it - changed. Editing an unrelated file mid-run produces that error and it is a - FALSE alarm — cost two runs this session, twice. -- **A sweep's output file buffers.** `pgrep -f shell_coverage_jail` is the - liveness signal, not file growth. A measurement that has printed nothing for - minutes is usually still running; `fixes/lib`'s timing-loop suites take 30s+ - per kcov pass. - **Two-strike rule.** Two failed attempts at the same thing -> stop, document, keep the working state. ## Rules that will bite you - **No suppressions, ever.** Fix SC2016/SC2155/EXE001/PLR2004 at the source. +- **Never weaken an assertion to move a number.** If a percentage looks absurd + while assertions pass, suspect the instrument and re-measure. - **Every commit touching code needs evidence** in `docs/superpowers/evidence/-.json` (validate with `python3 meta/scripts/validate_evidence.py `), **plus a contract** in `docs/superpowers/contracts/` once **>=4 code files** are staged. - **Put the MEASURED number in it**, never a rounded or hoped-for one. -- **Stage narrowly.** Another session may be working in this repo at the same - time; `git restore --staged` anything you did not touch. - **New test files need the exec bit** — `git add --chmod=+x` only changes the INDEX; `chmod +x` the working tree too or the shebang hook fails. +- **Stage narrowly.** Another session may be working in this repo at the same + time; `git restore --staged` anything you did not touch. - Work directly on `main`. `git stash` and branch creation are blocked. -## When the allowlist reaches zero +## State as of 2026-08-22 -Wire `check_shell_coverage.sh` into pre-commit, pre-push and CI. The gate is -already correct. Note it now runs each suite TWICE per lib, so an `--all` -sweep is roughly double what it was and belongs in CI, not a commit hook. +Allowlist **98**. Instrument fixed and pushed (`81191f40`, `29b8c5a7`). Five +libs cleared (`83de41f4`). `fixes/lib/tests/run_all.sh` added (`347222b9`). + +Already measured — do NOT redo: + +| lib | coverage | +| ---------------------------- | ------------------- | +| `rpi_nc_ca.sh` | **73/73 = 100.00%** | +| `dot_resolver_install.sh` | **39/39 = 100.00%** | +| `dwm_config.sh` | 65/66 = 98.48% | +| `pacman_hook_stall_setup.sh` | 26/27 = 96.30% | +| `aw_autostart.sh` | 80/82 = 97.56% | +| `rpi_nc_install.sh` | 85/88 = 96.59% | +| `transcribe_pkgmgr.sh` | 49/50 = 98.00% | +| `clean_audio_filters.sh` | 85/93 = 91.40% | +| `transcribe_deps.sh` | 48/52 = 92.31% | + +`rpi_nc_ca.sh` and `dot_resolver_install.sh` are at 100% and could come off +the allowlist whenever you want, by the three-step bar above. + +**`nc_php.sh` does not measure at all** — "kcov instrumented no lines", on the +pre-fix instrument too (checked at `078d3463` in a throwaway worktree). Not a +regression. Its old 84/85 = 98.82% does not reproduce; treat as unverified. ## Not caused by this campaign