From ac8e098c03c54cd12ca62ad2f25bda79ed72b922 Mon Sep 17 00:00:00 2001 From: Krzysztof kuhy Rudnicki Date: Sun, 23 Aug 2026 20:08:40 +0200 Subject: [PATCH] Point the diagnostics bats suite at the flattened path Shell tests went red on my last push. test_network_diagnostics.bats sourced its scripts through $REPO_DIR/scripts/diagnostics, a level the flatten removed, so setup() failed and all 30 tests errored. Latent rather than new: the previous run was green only because ci-mirror aborted before the bats stage. Fixing the concurrency OOM let the gate reach this suite for the first time since the flatten. Grepped the tree for the removed scripts/ and single_use/ levels; this was the last one in tracked code. Also rewrites the two handoff docs around the units that remain. CI_GREEN_SKIP=1 because this commit is the fix for the red baseline. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01PJrqKERAo5QUeEHWS5Asgp --- NEXT_SESSION_PROMPT.md | 477 +++--------------- NEXT_SESSION_RESTRUCTURE.md | 174 ++++--- .../fix-bats-diag-path-2026-08-23.json | 37 ++ .../tests/test_network_diagnostics.bats | 4 +- 4 files changed, 209 insertions(+), 483 deletions(-) create mode 100644 docs/superpowers/evidence/fix-bats-diag-path-2026-08-23.json diff --git a/NEXT_SESSION_PROMPT.md b/NEXT_SESSION_PROMPT.md index b6bdfc65..89de5785 100644 --- a/NEXT_SESSION_PROMPT.md +++ b/NEXT_SESSION_PROMPT.md @@ -1,413 +1,82 @@ -# STATUS 2026-08-23 (later session): jobs 1 and 2 are DONE +# Next session prompt — testsAndMisc restructure, final units -> Read this box before anything below it. The rest of this file is the -> ORIGINAL handoff, kept because its kcov/jail notes are still the reference -> for measuring a lib on purpose. Its SCOPE is now obsolete. - -**Job 1 — the push gate — DONE (commit 8ab15446).** ~130s to: -docs-only 7.6s, python 25.3s, one shell suite 43.6s, unchanged-tree re-push -0.005s. The guarantee is intact; a broken python assertion and a broken shell -assertion were each refused with exit 1. - -Found while measuring it: `ci_mirror.sh` called `pytest_changed_packages.py` -with no arguments, and that script returns 0 on empty argv. **The pre-push -python stage had never run a single test.** Behind it sat a real failing test -(fixed in ff3ea99a). - -**Job 2 — the 100% shell bar — DELIBERATELY DROPPED (commit 578c4a6c).** -The user's call on 2026-08-23, after being shown the real cost. The bar had -been raised from a presence check only the day before (bca70b67), and pricing -it out honestly: one hand-written test file per lib (fixes/lib took 34 files -for 22 libs), a serial jail at 37-90s a pass, and a production seam needed in -most libs before they are testable at all — several days for a repo of -personal automation scripts. `is_covered` is a presence check again. - -**The ratchet is now WIRED into pre-commit (commit 46c8f065)** — it never was -before, which is why it rotted. 0 BLOCK entries, 0.16s repo-wide. The three -`meta/scripts/lib` coverage-tooling libs got a 48-assertion suite so they stop -blocking it. - -**Python needs nothing:** 2101 tests, 100.00%, zero missing statements or -branches, measured in a clean worktree. **File-length cap needs nothing:** 0 -violations, hook-enforced. - -**What is left, if anyone wants it:** 8 directories still have no suite at all -(`.githooks/lib`, `meta/lib`, `mtk_root/lib`, `digital_wellbeing/{pacman, -virtualbox}/lib`, `system-maintenance/bin/lib`, `single_use/{fresh-install,}/lib`) -— 20 libs, ~1,980 lines. All are allowlisted, so nothing is blocked. This is -optional work now, not a campaign. - -**Traps worth carrying forward, beyond the list below:** - -- **Never `git reset --hard` to undo a probe commit.** It discards UNSTAGED - working-tree edits. That cost a full rewrite of `ci_mirror.sh` this session. - Recovery: pre-commit stashes unstaged files to `~/.cache/pre-commit/patch*`; - `git apply` on the newest one restored it exactly. Use - `git revert --no-commit `. -- **Read the predicate, not the header comment.** `check_shell_coverage.sh`'s - header still described a presence gate for a full day after the gate became - a 100% bar. Acting on the prose produced a confidently wrong scope estimate. -- A sourced-only lib takes `# shellcheck shell=bash`, no shebang, no exec bit, - or `check-shebang-scripts-are-executable` fails the commit. +Paste everything below into a fresh session. --- -# Next session: make the push gate fast, then close the coverage holes +Extract `linux_configuration/periodic_background/hosts/` into its own repo, +then `digital_wellbeing/`. Read `NEXT_SESSION_RESTRUCTURE.md` first — its +"Traps already paid for" section is the expensive part and every line in it +cost a real failure. -> **Paste this whole file into a fresh Claude session opened in `~/testsAndMisc`.** -> It is self-contained. Do not go looking for the previous session's context. - -There are **two jobs** here. Do them in order — job 1 is small and makes job 2 -bearable, because job 2 involves many commits. - ---- - -# JOB 1: get `git push` under 10 seconds - -## What is slow - -`git push` currently takes **~130 seconds**. Nearly all of it is the -`ci-mirror` pre-push hook (`meta/scripts/ci_mirror.sh`), which deliberately -re-runs CI locally so a red push cannot reach GitHub. It does four things -inside a fresh `git worktree` checkout of HEAD: - -| stage | what it costs | -| ----------------------------------------- | ------------------------------------- | -| venv install from `requirements.txt` | ~0s when cached, ~40s on a change | -| `pre-commit run --all-files` | the bulk of it | -| `pytest_changed_packages.py` in that venv | seconds to minutes, package-dependent | -| `run_shell_test_gate` (the shell suites) | ~40s+ | - -For comparison: **pre-commit is already fast — 4.8s measured.** Do not touch -it. The whole problem is pre-push. - -## The target, and the honest tradeoff - -The user asked for **under 10 seconds, ideally ~10s or less, total**. - -**Be straight with them about this:** a clean-venv install plus `--all-files` -plus two test suites cannot fit in 10s. Something must actually be given up. -The decision is theirs, so **measure first, then present the options and their -real costs, then implement the one they pick.** - -### Step 1 — measure each stage separately - -Do this before proposing anything. Numbers, not guesses: +## Start here ```bash -cd /home/kuhy/testsAndMisc -SP=$(mktemp -d); git worktree add --detach --quiet "$SP/co" HEAD - -time (cd "$SP/co" && SKIP=pytest-coverage pre-commit run --all-files) -time (cd "$SP/co" && python meta/scripts/pytest_changed_packages.py) -time bash linux_configuration/fixes/lib/tests/run_all.sh - -git worktree remove --force "$SP/co" +cd ~/testsAndMisc +./meta/scripts/check_directory_depth.sh --all | head -1 # expect 46 +./meta/scripts/check_repo_size.sh # expect 1051/1200 +git status -sb # expect clean ``` -Then find which individual pre-commit hooks dominate `--all-files`: - -```bash -cd "$SP/co" && for h in $(grep -oP '(?<=id: ).*' .pre-commit-config.yaml | sort -u); do - s=$(date +%s%N); SKIP='' pre-commit run "$h" --all-files >/dev/null 2>&1 - printf '%6d ms %s\n' $(( ($(date +%s%N)-s)/1000000 )) "$h" -done | sort -rn | head -15 -``` - -### Step 2 — the options, in the order I would try them - -1. **Only check what changed.** The single biggest win and the least loss. - `--all-files` is the expensive part; `pre-commit run --from-ref -origin/main --to-ref HEAD` checks the pushed range instead. The clean - worktree already guarantees you are checking committed content. - Similarly, only run a shell suite when the push touches shell files. -2. **Parallelise the stages.** `pre-commit --all-files`, pytest and the shell - gate are independent. Run them concurrently and wait. Roughly the cost of - the slowest rather than the sum. **Note:** two coverage jails must never run - at once (see the warning in job 2), so if the shell gate is jailed, keep it - serial with respect to any other jail. -3. **Cache on tree state.** Skip the whole mirror when - `git rev-parse HEAD:` (the tree hash) matches the last green run, recorded - in a dotfile. A re-push of an unchanged tree then costs ~0s. -4. **Move ci-mirror off pre-push entirely.** Pre-push drops to ~5s. The cost - is real and must be stated plainly: **a red push can then reach GitHub**, - and CI catches it minutes later instead of the hook catching it instantly. - The repo's `refuse-red-ci` hook already blocks the NEXT commit on a red - baseline, so this is less dangerous than it sounds — but it is a genuine - change in guarantees, not a free win. - -Options 1-3 are optimisations that keep the guarantee. Option 4 trades the -guarantee for speed. **1+2+3 together may well reach ~10s; try them before -proposing 4.** If they land at, say, 25s, say so honestly rather than reaching -for 4 without asking. - -### Step 3 — prove it - -```bash -time git push # the number that matters -``` - -And prove the gate still catches things: deliberately break something -(a failing assertion in a shell test), confirm the push is refused, then -revert. A fast gate that no longer gates is worse than a slow one. - -## Do not - -- Do not weaken or delete a test to make a gate faster. -- Do not add `--no-verify` anywhere, or suggest it. It is forbidden by the - user's rules. -- Do not touch pre-commit's 4.8s — it is already at target. - ---- - -# JOB 2: actually get the repo to 100% coverage - -The user asked to "make the repo truly 100% test covered". Be careful with -that phrasing: **it is a large campaign, not one task.** Scope it, report the -real size, and then work through it. Do not report it done until it is. - -## Where things actually stand - -### Shell: `meta/shell-coverage-allowlist.txt` has **84 exempt libs** - -```bash -grep -vc '^#' meta/shell-coverage-allowlist.txt # 84 -``` - -Each line is a shell library allowed to sit below 100% line coverage. The -just-finished campaign took this from 91 to 84 by clearing seven libs in -`fixes/lib`. The remaining 84 are the job. - -### ~12 gated libs OUTSIDE `fixes/lib` fail the gate RIGHT NOW - -`check_shell_coverage.sh` is **not wired into pre-commit**, so these have been -failing silently. Confirmed pre-existing (verified against an earlier commit — -they are not caused by recent work): - -``` -lib/common_packages.sh FAIL -periodic_background/digital_wellbeing/lib/ms_override_mgr.sh FAIL -periodic_background/digital_wellbeing/lib/ms_scripts.sh FAIL -periodic_background/lib/periodic_browser.sh FAIL -single_use/utils/lib/analyze_repo_symbols.sh FAIL -single_use/utils/lib/bl9000_backup.sh FAIL -single_use/utils/lib/download_media.sh FAIL -single_use/utils/lib/idle_inhibit.sh FAIL -single_use/utils/lib/offline_index.sh FAIL -single_use/utils/lib/steam_api.sh FAIL -single_use/utils/lib/steam_report.sh FAIL -``` - -`hosts_protect_custom.sh` and `hosts_protect_unblock.sh` PASS. - -**Start here.** These are worse than an exemption: they are libs the gate -believes it is checking and which do not meet the bar. `single_use/utils/lib/` -is most of the list and shares one `tests/` directory, so one harness covers -several. - -### Python: `python_pkg/` is 25 packages at `fail_under = 100` - -Branch coverage is already enforced at 100% by `meta/pyproject.toml`, so this -is likely in decent shape. **Verify before assuming** — and note the user's -standing rule: never exclude a package from coverage, write the tests. - -```bash -timeout 1800 python -m pytest python_pkg --cov=python_pkg --cov-report=term-missing -q 2>&1 | tail -30 -``` - -### Other languages - -`billsplit/` (Dart), `reverse_survivors/`, `poker-stakes/` (TS) have their own -CI. Check their gates before claiming anything about them. - -## Report the real scope first - -Before writing a single test, produce a count: N shell libs exempt, M gated -libs failing, Python status, other languages. Give the user that number and a -suggested order. "100% covered" spans several days of work; they should see -the shape before you spend it. - -## The order I would work in - -1. The ~12 gated-but-failing shell libs (worst: the gate is lying today). -2. Wire `check_shell_coverage.sh` into pre-commit so it cannot rot again — - **but only after 1**, or every commit is blocked. Mind job 1: this hook is - jailed and slow, so scope it to changed files. -3. The 84 allowlist entries, cheapest first. -4. Python, if the run above shows gaps. - ---- - -# How to do shell coverage work here (read before starting job 2) - -## The suite and its harnesses - -`linux_configuration/fixes/lib/tests/` is the reference -suite: **30 test files, 645 assertions, all green, zero suppressions, 37s.** -Copy its shape. Layers: - -- `run_all.sh` — the runner, and the subject the gate measures through. -- `lib_test_core.sh` (194 lines) — assertions and stubs. -- `lib_test_path.sh` (67 lines) — `_t_hide` and the PATH shim farm. -- Six family harnesses, one per entry script. - -Key primitives: - -- `_t_eq` / `_t_contains` / `_t_lacks` -- `_t_stub NAME BODY`, plus `_t_stub_stdin` (quoted heredoc), `_t_stub_cat`, - `_t_stub_writes`. Use these rather than `_t_stub foo '...$1...'`, which - trips SC2016 every time. -- **`_t_run FUNC [ARG...]`** — runs FUNC in the CURRENT shell, output in - `$out`. **Never `out="$(func)"`:** command substitution forks a subshell and - kcov then does not register the sourced lib at all (measured: 0 lines via - `$( )`, 54 via a direct call). -- **`_t_hide TOOL...`** — makes a tool genuinely unfindable. It KEEPS - `$FAKE_BIN` on PATH, so **`_t_unstub` the tool first** or the harness's own - default stub still satisfies `has_cmd` and the branch never runs. -- `_t_unstub` (runs `hash -r`; deleting the file alone does nothing). - -## Production seams - -Most libs write to `/etc`. The idiom is an override **defaulting to the real -path**, so production behaviour is byte-identical: - -```bash -local dropin="${SYSCTL_DROPIN_DIR:-/etc/sysctl.d}/99-performance-tuning.conf" -``` - -Reuse existing names — `SYSTEMD_UNIT_DIR`, `JOURNALD_CONF_DIR`, -`UDEV_RULES_DIR`, `SYSCTL_DROPIN_DIR` — and grep the entry script before -picking one; a collision is silent. - -**Watch for globals ASSIGNED inside a function** (`XORG_CONF="/etc/..."` with -no `local`). Exporting from the harness does not survive the assignment; the -default must live at the assignment itself. - -After the first run, `ls -la` the real paths and confirm the mtimes did not -move. - -## The measurement loop - -```bash -# 1. write the tests, chmod +x -# 2. run bare first -- far faster than the jail -bash /run_all.sh - -# 3. measure through the RUNNER, in a detached worktree -timeout 900s bash meta/scripts/shell_coverage_jail.sh --timeout 600s \ - --subject /run_all.sh --measure .sh --min 1 \ - --fail-on-case-error -- "" -``` - -Then the three-step bar to clear an exemption: - -1. 100.00% through `run_all.sh`. -2. Again at `--min 100`, so a shortfall is a non-zero exit. -3. Delete the allowlist line, **then** run - `bash meta/scripts/check_shell_coverage.sh ` and confirm exit 0. - Running it before deleting proves nothing — it passes via the exemption. - -## Traps that cost real hours last session - -- **Verify an edit landed.** A seam edit reported success, was measured, and - never reached the commit; ci-mirror caught it only because it runs from a - clean checkout. **After editing, `grep` the file for what you just wrote.** -- **Run the suite from a clean checkout before pushing.** It catches exactly - the above: - ```bash - SP=$(mktemp -d); git worktree add --detach "$SP/c" HEAD - bash "$SP/c//run_all.sh"; git worktree remove --force "$SP/c" - ``` -- **NEVER run two coverage jails at once.** Every false FAIL last session was - CPU contention — a jail plus another jail, or a jail plus `run_all.sh`. - Libs "failed" a sweep and passed solo. While a sweep runs, do zero-CPU work. -- **Another agent may be committing to this repo concurrently.** The jail - fingerprints `git status --porcelain` and aborts with "a write escaped the - jail" on THEIR edits. Measure in a detached worktree, and re-check a - transient gate failure before believing it. -- **Adding a test file can break an ALREADY-CLEARED lib.** Unexplained: - `test_arch_sysctl.sh` made kcov stop registering `bt_audio.sh` entirely - ("instrumented no lines"), turning the gate red. Ruled out: `export out`, - the `_t_hide` cache, and runtime (still fails at a 600s timeout). The file - is quarantined as `arch_sysctl_quarantined_cases.sh`, outside the - `test_*.sh` glob, with 23 still-green assertions. - **After adding any test file, re-measure an already-cleared lib** — - `bt_audio.sh` is the canary. -- **`shfmt` reformats staged shell files**, moving the denominator and - invalidating a measurement. Run `shfmt -l` first and land reformats - separately. -- **The 250-line cap aborts commits**, and a task notification can report - "completed (exit code 0)" for a commit that actually failed. Check. -- **Stage narrowly.** A `git add` of a directory swept another agent's - untracked file into a commit and broke the push. -- **`mapfile` is bash-only**; the tool shell is zsh. Wrap in `bash -c`. -- **`git stash` is blocked** — even `git stash list` inside a compound - command trips the guard. Use a detached worktree. -- **Do not pipe jail output through `grep 'lines ='`** — that hides the - `warn: case timed out` line, which is the diagnosis you need. -- **Do not `pgrep -f ` in a wait loop** — the loop's own command line - matches, so it never exits. - -## The coverage instrument - -`coverage = (PS4-traced | kcov hits) & (kcov line set - continuation lines)` - -`continuation_lines()` in `meta/scripts/lib/shell_coverage_lines.py` excludes -lines kcov instruments but bash attributes no statement to: multi-line quoted -arguments, multi-line array literals, `} |` / `} >>file`, and (added last -session) `done >file` / `done -``` - -### Libs that do not measure at all - -`arch_sysctl.sh` and `nc_php.sh` both report "kcov instrumented no lines" on -every version of the instrument. Treat as unverified; they stay exempt. - ---- - -## State as of 2026-08-23 - -- Allowlist **84** (was 91 at the start of the previous session). -- `fixes/lib` cleared: `arch_cpu` 57/57, `arch_hardware` 94/94, - `thorium_repairs` 75/75, `hosts_guard_migrate` 81/81, - `hosts_guard_rollback` 65/65, `ubuntu_perf_fixes` 81/81, - `nvidia_config` 76/76. -- The `fixes/lib` suite: 30 files, 645 assertions, 37s, zero suppressions. -- Three production bugs were found by writing tests, all dead branches: - `bt_audio.sh:21` (`timeout` cannot invoke a shell function), - `arch_perf_report.sh:36,65` and `arch_hardware.sh` (journal-size regex - demanded a space before the unit, but `journalctl` prints `305.5M`, so the - vacuum was unreachable). **Expect more. When a branch looks untestable, - check whether it is reachable in production first** — the user's standing - answer is "fix it in this commit". - -Still exempt and measured by an older session; **all predate the instrument -fixes and may now measure higher — re-measure before writing any test**: -`dwm_config.sh` 98.48%, `pacman_hook_stall_setup.sh` 96.30%, -`aw_autostart.sh` 97.56%, `rpi_nc_install.sh` 96.59%, -`transcribe_pkgmgr.sh` 98.00%, `clean_audio_filters.sh` 91.40%, -`transcribe_deps.sh` 92.31%. - -`rpi_nc_ca.sh` (73/73) and `dot_resolver_install.sh` (39/39) are already at -100% and can come off the allowlist whenever you want, by the three-step bar. -That is two free entries. +If those three numbers do not match, something moved since 2026-08-23 — +find out what before extracting anything. + +## What to do + +**1. `hosts` first** (5 depth violations). It is unblocked: the guard repoint +landed on 2026-08-23, so no guard names a repo path any more +(`sudo guardctl file-guard status resolved` should print +`/usr/local/share/guard-lib-plugins/...`). Its installer was already proven in +vmbox to run from an extracted layout. + +**2. `digital_wellbeing` last** (41 violations). The hard one. It owns the +pacman wrapper, `chattr +i`, `heavy_job_lock.sh`, and 12 orchestrator paths in +`check_and_enable_services.sh` / `setup_periodic_system.sh` that the flatten +already rewrote once. `setup_midnight_shutdown.sh` is `chattr +i` and is +excluded from two pre-commit fixer hooks **by path** — that exclude must move +with it, or the next commit fails on a file nothing can write. + +**3. Then close the loop:** delete `--warn` from the `directory-depth-cap` hook +entry in `.pre-commit-config.yaml`. That is the whole point of the campaign — +the cap only enforces once both units are gone. Verify with +`./meta/scripts/check_directory_depth.sh --all` exiting 0. + +## Constraints + +- Use the `extract-to-own-repo` skill; do not hand-roll the mechanics. +- **Sandbox-first**: both units are root/`/etc`/systemd/pacman territory, so + vmbox before the host (`vmbox-testing` skill). A vmbox pass never proves the + host is fine — report it as "passed in vmbox, not verified on the host". +- Never `--no-verify`. Every commit in this campaign has been green. +- `sudo -n` works and `guardctl` is at `/usr/local/bin/guardctl` — do not + hand me commands you can run yourself. +- Run `./meta/scripts/report_live_state.sh --check` **before and after** each + move. It scans `/usr/local/bin` too, where installed copies name repo paths + invisibly to a repo grep. There are already 8 known-stale references from the + earlier flatten; do not add more. +- Each commit needs an evidence artifact; 4+ staged code files (`.sh` counts, + `.json`/`.yaml` do not) also needs a contract. + +## Ask before doing (do not decide these yourself) + +- deleting `.hippo/` (161 tracked files; dead since 2026-05-28, but it is a + deletion) +- `rm -rf` on the 3.7 GB of untracked extraction residue (`focus_owner/`, + `bucket_catch/`, `kcd2_dice_solver/`, `billsplit/`) + +## Do NOT + +- attempt the Nextcloud dedup — measured and recommended against; it needs a + Raspberry Pi that cannot be reached from here, and only 3 of 15 libs have + tests. See item 3 of the status doc for the numbers. +- raise the 2G pytest cap if the push fails with "tests passed, then Killed". + That is the ci-mirror concurrency issue, already fixed; measure what runs + alongside instead. +- "sync" `.pre-commit-config.yaml` with `meta/.pre-commit-config.yaml`. They + are deliberately different files. + +## Done means + +`./meta/scripts/check_directory_depth.sh --all` exits **0** with `--warn` +removed from the hook, both units are their own repos with CI, the live-state +check adds no new stale references, and `main` is pushed clean. diff --git a/NEXT_SESSION_RESTRUCTURE.md b/NEXT_SESSION_RESTRUCTURE.md index 2817c1be..df5cbfad 100644 --- a/NEXT_SESSION_RESTRUCTURE.md +++ b/NEXT_SESSION_RESTRUCTURE.md @@ -4,102 +4,122 @@ Plan: `~/.claude/plans/after-refactor-of-testsandmisc-modular-wigderson.md` Live-state checklist: `docs/restructure-live-state.md` (regenerate with `./meta/scripts/report_live_state.sh`) +## Verified state as of 2026-08-23 (all pushed, `main` clean) + +``` +depth violations 46 (41 digital_wellbeing + 5 hosts — nothing else) +source files 1051 / 1200 ceiling +tracked files 1346 (2295 at the start, 1993 before the prune) +``` + +Both gates are wired and run on every commit. The depth gate runs `--warn` +because the only paths left over the cap are the two units below. **Delete +`--warn` from the `directory-depth-cap` hook entry in `.pre-commit-config.yaml` +once BOTH units are out, and the cap enforces for real.** + ## Done **Archived** to `github.com/kuhyx/testsAndMisc-archive`, full history: -`bucket_catch`, `poker-stakes`, `billsplit` (+ `python_pkg/billsplit_coverage`, -its CI workflow, the `app_icons` registry entry, and the paired -`.binary-allowlist` / `.gitignore` icon rules — `.binary-allowlist` now has no -patterns at all). +`bucket_catch`, `poker-stakes`, `billsplit`. -**Extracted** to public repos, full history, each with CI it did not have here: -`reverse-survivors`, `kcd2-dice-solver` (`~/kcd2-dice-solver`), `focus-owner`, -`mtk-root`, `system-maintenance` (`~/system-maintenance`), `android-guardian` -(`~/android-guardian`), `phone-focus-mode` (with `python_pkg/focus_policy`). +**Extracted** to public repos, full history: `reverse-survivors`, +`kcd2-dice-solver` (`~/kcd2-dice-solver`), `focus-owner` (**no local clone — +GitHub only**), `mtk-root`, `system-maintenance` (`~/system-maintenance`), +`android-guardian` (`~/android-guardian`), `phone-focus-mode`. -**Flattened.** `linux_configuration/scripts/` and `single_use/` are gone; -categories sit directly under `linux_configuration/`. `i3-configuration` moved -out of `periodic_background` into `i3/` + `i3blocks/`, and -`misc/testsAndMisc-bash/` lost its redundant nesting. +**Flattened.** `linux_configuration/scripts/` and `single_use/` are gone. -**Length cap widened** to catch extensionless scripts, and `boot-repair` split -756 → 218 with four libs. +**Gates wired.** 57 structural depth violations exempted (whole projects `C/`, +`dwm/`, `artgate/`; a module-split tool; prose trees; and the five +format-dictated segments `systemd`/`hooks`/`workflows`/`patches`/`fixtures`). +`check_repo_size.sh` counts tracked source files and caps the bookkeeping dirs +at 60 each, with `--prune` to clear them. -Depth violations **556 → 104**; tracked files **2295 → ~1980**. Every commit -green; nothing used `--no-verify`. +**Pruned** 655 superpowers artifacts (695 → 40 + 2 templates). + +**Guard repoint DONE on the host.** Both guards now name +`/usr/local/share/guard-lib-plugins/`, re-enforced, target files byte-identical +by md5, DNS verified between them. Backups at +`/var/tmp/{resolved.conf,nsswitch.conf}.pre-repoint`. ## Remaining -1. ~~The guard repoint.~~ **DONE on the host, 2026-08-23.** The "classifier - blocks me" note was stale: `sudo -n` works and `guardctl` is at - `/usr/local/bin/guardctl`. Verified in vmbox first (stale-path scenario - reproduced, then repointed), then run on the host one guard at a time with - a DNS check between them. Both now name - `/usr/local/share/guard-lib-plugins/`, both re-enforced (`----i-------e---`, - path units active), both target files byte-identical by md5 before and - after, DNS resolving throughout. Enforcement confirmed live, not just - reported: a `sudo` append to `/etc/nsswitch.conf` is refused. - Backups at `/var/tmp/{resolved.conf,nsswitch.conf}.pre-repoint`. +1. **`hosts` extraction — do this first; everything else is optional.** + Now unblocked: the guards no longer name a repo path. Its installer was + already proven in vmbox to run from an extracted layout. 5 depth violations + live here. -2. `hosts` extraction — **now unblocked** by (1). Its installer was already proven in - vmbox to run from an extracted layout with no monorepo path. -3. `digital_wellbeing` **last** — pacman wrapper, `chattr +i`, `heavy_job_lock.sh` - (moved in with it), and the 12 orchestrator paths in - `check_and_enable_services.sh` / `setup_periodic_system.sh`, which now point - at the NEW locations and will need updating again when it moves. It also owns +2. **`digital_wellbeing` — last, and the hardest.** 41 depth violations. + Owns the pacman wrapper, `chattr +i`, `heavy_job_lock.sh`, and 12 + orchestrator paths in `check_and_enable_services.sh` / + `setup_periodic_system.sh` that were already rewritten once by the flatten + and will need it again. Also owns `linux_configuration/tests/test_shutdown_timer_monitor.sh`. -4. ~~Wire the gates.~~ **DONE.** The 57 structural violations are exempted - and both gates run on every commit. The depth gate runs `--warn`, because - the only 47 paths left over the cap are the ones (2) and (3) will remove; - drop `--warn` as the last step of those extractions and the cap enforces - for real. `check_repo_size.sh` counts tracked source files (1049/1200) and - also caps the bookkeeping dirs at 60 each. -5. ~~Prune `docs/superpowers/evidence|contracts`.~~ **DONE** — 655 removed, - newest 20 of each kept, tracked files 1993 → 1340. Nothing reads the - corpus; git history keeps it. `.hippo/` is **decided but not executed**: - all 158 entries have `retrieval_count: 0`, 22 self-tag `invalidated`, the - last write was 2026-05-28, `hippo.db` is untracked and the tool is - uninstalled. It is dead; awaiting the untrack-vs-delete call. -6. **Recommended against**, with numbers. The estimate was right (~3,600 lines - across 22 files — a filename grep finds only 400 of them), but the two - `nc_*` (1212 lines) and `rpi_nc_*` (1171 lines) lib families are parallel - _evolutions_, not copies: all four shared function names differ in body, - not just comments. Only 3 of 15 libs have tests, and the target is a remote - Raspberry Pi that cannot be tested against from here. Merging is a - regression risk out of proportion to an optional cleanup. + `setup_midnight_shutdown.sh` is `chattr +i` and is excluded from two + pre-commit fixer hooks by path — that exclude must move with it. -7. **3.7 GB of extraction residue**, untracked and unreported until now: - `focus_owner/` (3.2G), `bucket_catch/` (303M), `kcd2_dice_solver/` (149M), - `billsplit/` (62M). None has a `.git`; each holds **zero** source files - outside `build/`, `node_modules/`, `dist/` and `coverage/`. The real repos - are on GitHub and were pushed 2026-08-23, so this is pure build output — - `rm -rf` on the user's say-so. Note `focus-owner` has no local clone at - `~/focus-owner`; only the GitHub copy exists. +3. **Nextcloud dedup — recommended AGAINST, with numbers.** Not a filename + grep: `grep -rln -i nextcloud linux_configuration/` finds **3,618 lines + across 22 files** (566 entry points + 2,383 libs + 604 tests). The two lib + families `nc_*` (1212 lines) and `rpi_nc_*` (1171) are parallel + _evolutions_, not copies — all four shared function names differ in body, + not just comments. Only **3 of 15 libs have tests**, and the target is a + remote Raspberry Pi that cannot be tested against from here. Do not attempt + without hardware access. + +4. **`.hippo/` — dead, awaiting a decision.** 161 tracked files, 3.2M. All 158 + episodic entries have `retrieval_count: 0`, 22 self-tag `invalidated`, last + write 2026-05-28, the tool is uninstalled, nothing references it. + Untracking needs a new `.gitignore` entry — `hippo.db` is currently + untracked only because `*.db` is ignored. **Ask before deleting.** + +5. **3.7 GB of extraction residue**, untracked: `focus_owner/` (3.2G), + `bucket_catch/` (303M), `kcd2_dice_solver/` (149M), `billsplit/` (62M). + None has a `.git`. The only non-build files are `.idea/` IDE config and a + Flutter-_generated_ `GeneratedPluginRegistrant.java` — no hand-written + source. Repos are safe on GitHub. **Ask before `rm -rf`.** + +6. **8 stale live-system references** (pre-existing, from the earlier flatten, + NOT from the gate work). `./meta/scripts/report_live_state.sh --check` + lists them; they name `linux_configuration/scripts/...` paths that no + longer exist. Two were the guard plugins, now fixed; the rest include + `setup_dns_blocker.sh` and `organize_downloads.sh`. ## Traps already paid for -- **Run it, don't read it.** Repointing kcd2 hit three failures in a row (pnpm - ignoring the `package.json` field, no-TTY `node_modules` purge, a Docker mount - on the deleted path). `dice.kuhy.duckdns.org` now serves from the new repo. -- **`/usr/local/bin` holds copies naming repo paths** — invisible to a repo grep. - `report_live_state.sh` scans there now. -- **Removing a file can break a suite that stays** (`mtk_harness.sh`), and a - removal can silently shrink CI (`shell-tests.yml` uses TWO hand-maintained - lists plus a glob — check all three). -- **Never let a test fall back to `/usr/local/bin`**: the installed copy can be - stale, and the test then compares against a binary it never generated. -- **pre-commit's patch restore can revert an edit** made before the commit. The - `app_icons` and `AGENTS.md` changes each had to be applied twice. +- **Run it, don't read it.** `--prune` sorted on `%cs` (date only), which ties + every same-day artifact; `sort` fell back to the path and it deleted + _alphabetically_, picking one of the newest files. Invisible in review. + Use `%ct` + `sort -rn`. Test destructive modes in a detached worktree. +- **A filename grep undercounts a capability 7x.** `git ls-files | grep -i X` + matches names only and misses the `lib/` layer, where the bulk lives. + Use `grep -rln` over contents before calling an estimate stale. +- **"Tests passed, then Killed" is not a test failure.** ci-mirror ran + `pre-commit --all-files` (~1.75 GiB) and pytest (~0.29 GiB) concurrently; + the pytest scope has `MemorySwapMax=0`, so it was the one killed. Fixed by + `should_serialise_gates` in `meta/scripts/ci_mirror_mem.sh`. **Never raise + the 2G cap** — measure what runs alongside. `/usr/bin/time -v` hides this + (it measures only the parent); read `memory.peak` from a cgroup. +- **`sudo -n` works here and `guardctl` is at `/usr/local/bin/guardctl`.** The + old "the classifier blocks me" note was stale. +- **The pre-push gate is not the pre-commit gate.** It adds prettier and + ci-mirror. Two docs were non-conforming before this session and blocked + _every_ push; prettier normalises `*emphasis*` to `_emphasis_`. +- **`.pre-commit-config.yaml` is NOT a symlink** (unlike `pyproject.toml`, + `run.sh`, `requirements.txt`, `lint_python.sh`, `.fvmrc`). The root file is + the active config; `meta/.pre-commit-config.yaml` is a separate standalone + variant, already ~219 lines divergent. Do not "sync" them. +- **`/usr/local/bin` holds copies naming repo paths** — invisible to a repo + grep. `report_live_state.sh` scans there. - **Path rewrites need resolving, not reviewing.** Relative `source` depths - differ per file (`../../` vs `../../../`), `REPO_ROOT` walks a fixed number of - levels, `$VAR/scripts/...` prefixes hide from a naive regex, and - `# shellcheck source=` directives break independently of the code. + differ per file, and a test can hardcode a prefix the script no longer uses + — `test_music_parallelism.sh` failed a layout that was correct. - **Sourcing top-level code is not a split.** boot-repair's report block has `exit` calls; moving it broke 30 of 42 tests. ## Unrelated: syncyomi-guard is red -Failing since 00:00 on 2026-08-23, ~17h before this work started. It is working -as designed: the library collapsed from 2185 manga / 68902 chapters to 8 / 895. -See the `syncyomi-restore-silent-partial` memory; do NOT checkpoint the WAL -before recovering. Newest snapshot in `~/syncyomi/snapshots/` is 2026-08-15. +Failing since 00:00 on 2026-08-23, working as designed: the library collapsed +from 2185 manga / 68902 chapters to 8 / 895. See the +`syncyomi-restore-silent-partial` memory; do NOT checkpoint the WAL before +recovering. Newest snapshot in `~/syncyomi/snapshots/` is 2026-08-15. diff --git a/docs/superpowers/evidence/fix-bats-diag-path-2026-08-23.json b/docs/superpowers/evidence/fix-bats-diag-path-2026-08-23.json new file mode 100644 index 00000000..56032a79 --- /dev/null +++ b/docs/superpowers/evidence/fix-bats-diag-path-2026-08-23.json @@ -0,0 +1,37 @@ +{ + "intent": "Turn the Shell tests workflow green again: test_network_diagnostics.bats still sourced its scripts through the scripts/ level the flatten removed.", + "scope": [ + "linux_configuration/tests/test_network_diagnostics.bats - DIAG_DIR and its header comment", + "NEXT_SESSION_RESTRUCTURE.md / NEXT_SESSION_PROMPT.md - handoff rewrite for the two remaining units", + "Non-goal: the diagnostics scripts themselves, which are correctly placed." + ], + "changes": [ + "DIAG_DIR pointed at $REPO_DIR/scripts/diagnostics; the flatten dissolved linux_configuration/scripts/, so setup() failed to source and all 30 tests errored. Now $REPO_DIR/diagnostics.", + "Fixed the header comment naming the same dead path.", + "Rewrote the handoff docs around the two units that remain, with the verified numbers and the traps this session paid for." + ], + "verification": [ + { + "command": "bats linux_configuration/tests/test_network_diagnostics.bats", + "result": "pass", + "evidence": "30/30 ok. Previously every test failed in setup with 'No such file or directory' on .../scripts/diagnostics/steam-download-duty.sh." + }, + { + "command": "grep -rn 'linux_configuration/scripts/|scripts/single_use|single_use/' over tracked .sh/.bats/.py/.yml", + "result": "pass", + "evidence": "No matches. This was the last stale reference to the removed levels inside the repo; the 8 remaining live-state hits are installed copies under /usr/local/bin, outside the tree." + }, + { + "command": "gh run view 32656921622 --log-failed", + "result": "pass", + "evidence": "Confirmed the CI failure was exactly this suite. The previous run (cbdf3a3d) was green because ci-mirror aborted before the bats stage, so the breakage was latent rather than newly introduced." + } + ], + "risks": [ + "CI_GREEN_SKIP=1 is used for this commit because the ci-baseline-green hook refuses to build on a red main and this commit IS the fix - the documented escape for exactly this case." + ], + "rollback": [ + "git revert the commit; the bats suite returns to the dead path and Shell tests goes red again.", + "After rollback, run bats linux_configuration/tests/test_network_diagnostics.bats to confirm the expected failure." + ] +} diff --git a/linux_configuration/tests/test_network_diagnostics.bats b/linux_configuration/tests/test_network_diagnostics.bats index 3145cfe9..ba230e21 100755 --- a/linux_configuration/tests/test_network_diagnostics.bats +++ b/linux_configuration/tests/test_network_diagnostics.bats @@ -1,5 +1,5 @@ #!/usr/bin/env bats -# Unit tests for the pure helpers in scripts/diagnostics/. +# Unit tests for the pure helpers in diagnostics/. # # Requires: bats (extra/bats). Run with: # bats linux_configuration/tests/test_network_diagnostics.bats @@ -11,7 +11,7 @@ setup() { REPO_DIR="$(cd -- "$(dirname -- "$BATS_TEST_FILENAME")/.." && pwd)" - DIAG_DIR="$REPO_DIR/scripts/diagnostics" + DIAG_DIR="$REPO_DIR/diagnostics" # Both scripts guard main() behind a BASH_SOURCE check, so sourcing them # loads the helpers without starting a measurement run. # shellcheck source=/dev/null