mirror of
https://github.com/kuhyx/testsAndMisc.git
synced 2026-11-22 00:22:19 +01:00
Phases 1 and 2 are done — the cap is at zero and enforced, and both CI gates are wired — so the handoff now carries what a Phase 3 session needs instead. Phase 3 opens with questions rather than work, because the scale (roughly 487 .sh files against a handful of lib/tests/ dirs) makes scope the user's call. The coverage numbers Phase 1 actually measured are included so the "what bar" question gets answered with evidence: decision logic reached 94%, while every function that pkills, sudo-writes or installs sat at 0%. A 100% bar means shimming sudo/pkill/systemctl/curl/npm across every installer, and that cost belongs in front of the user before anyone starts writing tests. Also records the traps that cost real time and are all still live: shfmt corrupting hyphenated array subscripts, `# shellcheck` at the start of a comment being parsed as a directive, verify_shell_split.sh being blind to heredoc-internal functions, and the exec-bit staging rule. Notes that Python tests has been red since 2026-08-17 from a pip resolution-too-deep failure that predates this campaign and is unrelated to it — which is why python-tests.yml is not in check_ci_green.sh's required list. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F3PzGc75gBcdxP3PkHXGcV
141 lines
7.7 KiB
Markdown
141 lines
7.7 KiB
Markdown
# Next session: Phase 3 — repo-wide shell test coverage
|
|
|
|
> **Paste this whole file into a fresh Claude session opened in `~/testsAndMisc`.**
|
|
> It is self-contained. Do not go looking for the previous session's context.
|
|
|
|
**Phases 1 and 2 are DONE.** The 250-line cap is at zero and enforced; the CI
|
|
gates are wired. What remains is Phase 3, and it needs a scoping decision from
|
|
the user before any code is written.
|
|
|
|
## What is already true (verify, do not redo)
|
|
|
|
```bash
|
|
bash meta/scripts/check_file_length.sh --all # "all checked files are within 250 lines"
|
|
bash meta/scripts/check_ci_green.sh # exit 0 while main is green
|
|
gh run list --limit 5
|
|
```
|
|
|
|
- **Every file is under 250 lines.** The five deployment-trap files were split:
|
|
`install_leechblock.sh` (485→119), `block_compulsive_opening.sh` (705→182),
|
|
`setup_night_lockdown.sh` (918→127), `pacman_wrapper.sh` (929→217),
|
|
`setup_midnight_shutdown.sh` (1734→109), plus `_glyph_art.py` (253→208).
|
|
- **`file-length-cap`** runs `check_file_length.sh --all` on every commit.
|
|
- **`ci-baseline-green`** refuses to commit onto a red `main` (fails closed on
|
|
red, warns and passes when it cannot tell). Bypass: `CI_GREEN_SKIP=1`.
|
|
- **`ci-mirror`** on pre-push now also runs every `*/lib/tests/run_all.sh` and
|
|
the side-effect-free half of `shell-tests.yml`.
|
|
- **`shell-tests.yml`** discovers and runs every `*/lib/tests/run_all.sh`. Four
|
|
such suites existed before and none of them ran in CI until this was added.
|
|
|
|
## Phase 3: ask FIRST, then build
|
|
|
|
**Do not start writing tests before the user answers these.** The prompt that
|
|
started this campaign says to ask, and the scale is the reason:
|
|
|
|
```bash
|
|
find . -name '*.sh' -not -path './node_modules/*' -not -path './third_party/*' | wc -l
|
|
ls -d */lib/tests linux_configuration/scripts/*/lib/tests 2>/dev/null
|
|
```
|
|
|
|
Roughly 487 `.sh` files against a handful of `lib/tests/` directories.
|
|
|
|
❓ **Q1 — Scope.** Repo-wide from day one, or a **ratchet** (only new or
|
|
modified files must be covered, with a shrinking allowlist of pre-existing
|
|
untested files)? A hard repo-wide gate blocks every unrelated commit to any of
|
|
450+ files until all of them have tests. **Recommend the ratchet**, but it is
|
|
the user's call.
|
|
|
|
❓ **Q2 — Bar.** 100% line coverage, or a lower number for shell? The Python
|
|
side is `fail_under = 100`. Shell has paths that genuinely cannot run under
|
|
test (see below), so 100% would force either shims for `sudo`/`pkill`/`systemctl`
|
|
or a suppression mechanism — and suppressions are forbidden here.
|
|
|
|
❓ **Q3 — Which files count?** Every `.sh`, or only libs under a `lib/`
|
|
directory? Entry scripts are mostly orchestration whose bodies are the
|
|
untestable part.
|
|
|
|
## What Phase 1 measured, so Q2 is answered with numbers
|
|
|
|
Coverage of the new libs, measured with
|
|
`bash meta/scripts/shell_coverage.sh <lib/tests/run_all.sh> <lib-basename> 0`:
|
|
|
|
| lib | coverage | what is uncovered |
|
|
| --------------------------------------------------------- | -------------- | ------------------------------------------------ |
|
|
| `cco_state.sh` | 93.94% (62/66) | only `kill_app`'s two `pkill` lines |
|
|
| `leechblock_fetch.sh` | 38.57% (27/70) | `download_extension`: curl/tar/unzip |
|
|
| `leechblock_browsers.sh` | 36.47% (31/85) | `replace_browser_in_place`: sudo binary patching |
|
|
| `cco_wrapper/install/report`, `leechblock_config/firefox` | 0% | every function pkills, sudo-writes or installs |
|
|
|
|
**The pattern: the decision logic reaches ~90%+, the effecting code reaches 0%.**
|
|
A 100% bar means shimming `sudo`, `pkill`, `systemctl`, `curl` and `npm` for
|
|
every installer in the repo. That is the real cost to put in front of the user.
|
|
|
|
## The harness that works — extend it, do not clone it
|
|
|
|
`linux_configuration/scripts/periodic_background/digital_wellbeing/lib/tests/`
|
|
holds `leechblock_harness.sh` and `cco_harness.sh` plus a `run_all.sh` whose
|
|
glob is `test_*.sh`. **One harness per directory, not per file** — `jscpd`
|
|
fails above 2% duplication and five near-identical harnesses is exactly how
|
|
that trips.
|
|
|
|
Two patterns worth copying:
|
|
|
|
- **`_t_isolate_path`** narrows `PATH` to a shim dir. It is the only way to
|
|
make `command -v jq` or `command -v chromium` fail on a machine that has
|
|
them. **Every external the code under test calls must be seeded into that
|
|
dir** — a missing `mkdir` aborted the suite under `set -e` with _no stderr_,
|
|
which under kcov looked like a coverage-tool bug for twenty minutes.
|
|
- **A fixture self-check.** `cco_harness.sh` asserts its own globals are
|
|
populated and calls `notify` for real. This is not linter appeasement: it is
|
|
what turns "a typo in a global name" from a silent skipped assertion into a
|
|
loud failure, and it happens to satisfy SC2034 honestly.
|
|
|
|
## Traps that cost real time in Phase 1 — all still live
|
|
|
|
- **`shfmt -w` corrupts `[hyphenated-keys]` in associative arrays**, turning
|
|
`${BROWSERS[google-chrome-stable]}` into a subtraction expression. **Quote the
|
|
subscripts.** Reproduced live in a test file during this campaign. Canary:
|
|
`grep -nE '\[[a-z0-9]+ - ' <file>` must be empty after every `shfmt` and
|
|
every `pre-commit run`.
|
|
- **A comment line starting with `# shellcheck` is parsed as a directive** and
|
|
fails with SC1072/SC1073. Write "the linter", not "shellcheck", at the start
|
|
of a comment line.
|
|
- **`verify_shell_split.sh` cannot see heredocs.** In
|
|
`setup_midnight_shutdown.sh` it reported a bare `-log`, because `log()` sits
|
|
at column 0 _inside a generated script_. For any file that emits scripts,
|
|
the real check is **hashing the emitted content**, not the function list.
|
|
- **New sourced libs need a shebang AND the exec bit**, staged with
|
|
`git add --chmod=+x`. A plain `git add` afterwards resets it. Payload files
|
|
under `lib/payloads/` are `100755` too — the shebang hook rejects `100644`.
|
|
- **`gh` and `git push` race.** Several pushes failed with "cannot lock ref";
|
|
`git fetch` then re-push. Not a divergence.
|
|
|
|
## Known-broken, NOT caused by the campaign
|
|
|
|
**`Python tests` has been failing since 2026-08-17** — before any of this work.
|
|
The cause is `pip` hitting `error: resolution-too-deep` while installing
|
|
`meta/requirements.txt`, on commits that touch no Python at all. It needs
|
|
dependency pinning, and it is why `python-tests.yml` is deliberately **not** in
|
|
`check_ci_green.sh`'s required list. Worth fixing on its own; do not fold it
|
|
into Phase 3.
|
|
|
|
## Rules that will bite you
|
|
|
|
- **No suppressions, ever.** No `# noqa`, no `shellcheck disable`, no
|
|
per-file-ignore without asking every single time.
|
|
- **Every commit touching code needs evidence** in
|
|
`docs/superpowers/evidence/<slug>-<date>.json`, **plus a contract** in
|
|
`docs/superpowers/contracts/` once ≥4 code files are staged. Put the
|
|
**measured** number in it, never a rounded or hoped-for one.
|
|
- **A test file broken by your commit is a same-session bug.** Three were
|
|
broken this way in Phase 1 (`security_apps.sh`, `test_shutdown_timer_monitor.sh`,
|
|
and the pacman pair) — each asserted on code a split had moved. Fix the
|
|
assertion to search the entry script _and_ its libs; do not weaken it.
|
|
- `pre-commit run --files <changed>` **after** `git add`. `prettier` and
|
|
`ci-mirror` run on **pre-push**. `npx prettier --write` any `.md`/`.json`.
|
|
- Work directly on `main`. `git stash` and branch creation are blocked.
|
|
- **Verify what actually got committed.** Re-grep the file after committing;
|
|
an autoformat pass stripped a load-bearing variable definition three times in
|
|
Phase 1, once landing a `set -u` abort in `setup_night_lockdown.sh` that had
|
|
to be fixed in a follow-up commit.
|