kuhyx/testsAndMisc
1
0
Fork 0
mirror of https://github.com/kuhyx/testsAndMisc.git synced 2026-11-10 20:27:02 +01:00
Code Issues Actions 6 Packages Projects Releases Wiki Activity

Point the diagnostics bats suite at the flattened path

Browse Source 
Shell tests went red on my last push. test_network_diagnostics.bats sourced
its scripts through $REPO_DIR/scripts/diagnostics, a level the flatten
removed, so setup() failed and all 30 tests errored.

Latent rather than new: the previous run was green only because ci-mirror
aborted before the bats stage. Fixing the concurrency OOM let the gate reach
this suite for the first time since the flatten.

Grepped the tree for the removed scripts/ and single_use/ levels; this was
the last one in tracked code.

Also rewrites the two handoff docs around the units that remain.

CI_GREEN_SKIP=1 because this commit is the fix for the red baseline.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PJrqKERAo5QUeEHWS5Asgp
This commit is contained in:
Krzysztof Rudnicki 2026-08-23 20:08:40 +02:00
parent 2a1bb60d9f
commit ac8e098c03
4 changed files with 209 additions and 483 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

477
NEXT_SESSION_PROMPT.md
View File

@ -1,413 +1,82 @@
# STATUS 2026-08-23 (later session): jobs 1 and 2 are DONE
# Next session prompt — testsAndMisc restructure, final units
> Read this box before anything below it. The rest of this file is the
> ORIGINAL handoff, kept because its kcov/jail notes are still the reference
> for measuring a lib on purpose. Its SCOPE is now obsolete.
**Job 1 — the push gate — DONE (commit 8ab15446).** ~130s to:
docs-only 7.6s, python 25.3s, one shell suite 43.6s, unchanged-tree re-push
0.005s. The guarantee is intact; a broken python assertion and a broken shell
assertion were each refused with exit 1.
Found while measuring it: `ci_mirror.sh` called `pytest_changed_packages.py`
with no arguments, and that script returns 0 on empty argv. **The pre-push
python stage had never run a single test.** Behind it sat a real failing test
(fixed in ff3ea99a).
**Job 2 — the 100% shell bar — DELIBERATELY DROPPED (commit 578c4a6c).**
The user's call on 2026-08-23, after being shown the real cost. The bar had
been raised from a presence check only the day before (bca70b67), and pricing
it out honestly: one hand-written test file per lib (fixes/lib took 34 files
for 22 libs), a serial jail at 37-90s a pass, and a production seam needed in
most libs before they are testable at all — several days for a repo of
personal automation scripts. `is_covered` is a presence check again.
**The ratchet is now WIRED into pre-commit (commit 46c8f065)** — it never was
before, which is why it rotted. 0 BLOCK entries, 0.16s repo-wide. The three
`meta/scripts/lib` coverage-tooling libs got a 48-assertion suite so they stop
blocking it.
**Python needs nothing:** 2101 tests, 100.00%, zero missing statements or
branches, measured in a clean worktree. **File-length cap needs nothing:** 0
violations, hook-enforced.
**What is left, if anyone wants it:** 8 directories still have no suite at all
(`.githooks/lib`, `meta/lib`, `mtk_root/lib`, `digital_wellbeing/{pacman,
virtualbox}/lib`, `system-maintenance/bin/lib`, `single_use/{fresh-install,}/lib`)
— 20 libs, ~1,980 lines. All are allowlisted, so nothing is blocked. This is
optional work now, not a campaign.
**Traps worth carrying forward, beyond the list below:**
- **Never `git reset --hard` to undo a probe commit.** It discards UNSTAGED
working-tree edits. That cost a full rewrite of `ci_mirror.sh` this session.
Recovery: pre-commit stashes unstaged files to `~/.cache/pre-commit/patch*`;
`git apply` on the newest one restored it exactly. Use
`git revert --no-commit <sha>`.
- **Read the predicate, not the header comment.** `check_shell_coverage.sh`'s
header still described a presence gate for a full day after the gate became
a 100% bar. Acting on the prose produced a confidently wrong scope estimate.
- A sourced-only lib takes `# shellcheck shell=bash`, no shebang, no exec bit,
or `check-shebang-scripts-are-executable` fails the commit.
Paste everything below into a fresh session.
---
# Next session: make the push gate fast, then close the coverage holes
Extract `linux_configuration/periodic_background/hosts/` into its own repo,
then `digital_wellbeing/`. Read `NEXT_SESSION_RESTRUCTURE.md` first — its
"Traps already paid for" section is the expensive part and every line in it
cost a real failure.
> **Paste this whole file into a fresh Claude session opened in `~/testsAndMisc`.**
> It is self-contained. Do not go looking for the previous session's context.
There are **two jobs** here. Do them in order — job 1 is small and makes job 2
bearable, because job 2 involves many commits.
---
# JOB 1: get `git push` under 10 seconds
## What is slow
`git push` currently takes **~130 seconds**. Nearly all of it is the
`ci-mirror` pre-push hook (`meta/scripts/ci_mirror.sh`), which deliberately
re-runs CI locally so a red push cannot reach GitHub. It does four things
inside a fresh `git worktree` checkout of HEAD:
| stage | what it costs |
| ----------------------------------------- | ------------------------------------- |
| venv install from `requirements.txt` | ~0s when cached, ~40s on a change |
| `pre-commit run --all-files` | the bulk of it |
| `pytest_changed_packages.py` in that venv | seconds to minutes, package-dependent |
| `run_shell_test_gate` (the shell suites) | ~40s+ |
For comparison: **pre-commit is already fast — 4.8s measured.** Do not touch
it. The whole problem is pre-push.
## The target, and the honest tradeoff
The user asked for **under 10 seconds, ideally ~10s or less, total**.
**Be straight with them about this:** a clean-venv install plus `--all-files`
plus two test suites cannot fit in 10s. Something must actually be given up.
The decision is theirs, so **measure first, then present the options and their
real costs, then implement the one they pick.**
### Step 1 — measure each stage separately
Do this before proposing anything. Numbers, not guesses:
## Start here
```bash
cd /home/kuhy/testsAndMisc
SP=$(mktemp -d); git worktree add --detach --quiet "$SP/co" HEAD
time (cd "$SP/co" && SKIP=pytest-coverage pre-commit run --all-files)
time (cd "$SP/co" && python meta/scripts/pytest_changed_packages.py)
time bash linux_configuration/fixes/lib/tests/run_all.sh
git worktree remove --force "$SP/co"
cd ~/testsAndMisc
./meta/scripts/check_directory_depth.sh --all | head -1 # expect 46
./meta/scripts/check_repo_size.sh # expect 1051/1200
git status -sb # expect clean
```
Then find which individual pre-commit hooks dominate `--all-files`:
```bash
cd "$SP/co" && for h in $(grep -oP '(?<=id: ).*' .pre-commit-config.yaml | sort -u); do
s=$(date +%s%N); SKIP='' pre-commit run "$h" --all-files >/dev/null 2>&1
printf '%6d ms %s\n' $(( ($(date +%s%N)-s)/1000000 )) "$h"
done | sort -rn | head -15
```
### Step 2 — the options, in the order I would try them
1. **Only check what changed.** The single biggest win and the least loss.
`--all-files` is the expensive part; `pre-commit run --from-ref
origin/main --to-ref HEAD` checks the pushed range instead. The clean
worktree already guarantees you are checking committed content.
Similarly, only run a shell suite when the push touches shell files.
2. **Parallelise the stages.** `pre-commit --all-files`, pytest and the shell
gate are independent. Run them concurrently and wait. Roughly the cost of
the slowest rather than the sum. **Note:** two coverage jails must never run
at once (see the warning in job 2), so if the shell gate is jailed, keep it
serial with respect to any other jail.
3. **Cache on tree state.** Skip the whole mirror when
`git rev-parse HEAD:` (the tree hash) matches the last green run, recorded
in a dotfile. A re-push of an unchanged tree then costs ~0s.
4. **Move ci-mirror off pre-push entirely.** Pre-push drops to ~5s. The cost
is real and must be stated plainly: **a red push can then reach GitHub**,
and CI catches it minutes later instead of the hook catching it instantly.
The repo's `refuse-red-ci` hook already blocks the NEXT commit on a red
baseline, so this is less dangerous than it sounds — but it is a genuine
change in guarantees, not a free win.
Options 1-3 are optimisations that keep the guarantee. Option 4 trades the
guarantee for speed. **1+2+3 together may well reach ~10s; try them before
proposing 4.** If they land at, say, 25s, say so honestly rather than reaching
for 4 without asking.
### Step 3 — prove it
```bash
time git push # the number that matters
```
And prove the gate still catches things: deliberately break something
(a failing assertion in a shell test), confirm the push is refused, then
revert. A fast gate that no longer gates is worse than a slow one.
## Do not
- Do not weaken or delete a test to make a gate faster.
- Do not add `--no-verify` anywhere, or suggest it. It is forbidden by the
user's rules.
- Do not touch pre-commit's 4.8s — it is already at target.
---
# JOB 2: actually get the repo to 100% coverage
The user asked to "make the repo truly 100% test covered". Be careful with
that phrasing: **it is a large campaign, not one task.** Scope it, report the
real size, and then work through it. Do not report it done until it is.
## Where things actually stand
### Shell: `meta/shell-coverage-allowlist.txt` has **84 exempt libs**
```bash
grep -vc '^#' meta/shell-coverage-allowlist.txt # 84
```
Each line is a shell library allowed to sit below 100% line coverage. The
just-finished campaign took this from 91 to 84 by clearing seven libs in
`fixes/lib`. The remaining 84 are the job.
### ~12 gated libs OUTSIDE `fixes/lib` fail the gate RIGHT NOW
`check_shell_coverage.sh` is **not wired into pre-commit**, so these have been
failing silently. Confirmed pre-existing (verified against an earlier commit —
they are not caused by recent work):
```
lib/common_packages.sh FAIL
periodic_background/digital_wellbeing/lib/ms_override_mgr.sh FAIL
periodic_background/digital_wellbeing/lib/ms_scripts.sh FAIL
periodic_background/lib/periodic_browser.sh FAIL
single_use/utils/lib/analyze_repo_symbols.sh FAIL
single_use/utils/lib/bl9000_backup.sh FAIL
single_use/utils/lib/download_media.sh FAIL
single_use/utils/lib/idle_inhibit.sh FAIL
single_use/utils/lib/offline_index.sh FAIL
single_use/utils/lib/steam_api.sh FAIL
single_use/utils/lib/steam_report.sh FAIL
```
`hosts_protect_custom.sh` and `hosts_protect_unblock.sh` PASS.
**Start here.** These are worse than an exemption: they are libs the gate
believes it is checking and which do not meet the bar. `single_use/utils/lib/`
is most of the list and shares one `tests/` directory, so one harness covers
several.
### Python: `python_pkg/` is 25 packages at `fail_under = 100`
Branch coverage is already enforced at 100% by `meta/pyproject.toml`, so this
is likely in decent shape. **Verify before assuming** — and note the user's
standing rule: never exclude a package from coverage, write the tests.
```bash
timeout 1800 python -m pytest python_pkg --cov=python_pkg --cov-report=term-missing -q 2>&1 | tail -30
```
### Other languages
`billsplit/` (Dart), `reverse_survivors/`, `poker-stakes/` (TS) have their own
CI. Check their gates before claiming anything about them.
## Report the real scope first
Before writing a single test, produce a count: N shell libs exempt, M gated
libs failing, Python status, other languages. Give the user that number and a
suggested order. "100% covered" spans several days of work; they should see
the shape before you spend it.
## The order I would work in
1. The ~12 gated-but-failing shell libs (worst: the gate is lying today).
2. Wire `check_shell_coverage.sh` into pre-commit so it cannot rot again —
**but only after 1**, or every commit is blocked. Mind job 1: this hook is
jailed and slow, so scope it to changed files.
3. The 84 allowlist entries, cheapest first.
4. Python, if the run above shows gaps.
---
# How to do shell coverage work here (read before starting job 2)
## The suite and its harnesses
`linux_configuration/fixes/lib/tests/` is the reference
suite: **30 test files, 645 assertions, all green, zero suppressions, 37s.**
Copy its shape. Layers:
- `run_all.sh` — the runner, and the subject the gate measures through.
- `lib_test_core.sh` (194 lines) — assertions and stubs.
- `lib_test_path.sh` (67 lines) — `_t_hide` and the PATH shim farm.
- Six family harnesses, one per entry script.
Key primitives:
- `_t_eq` / `_t_contains` / `_t_lacks`
- `_t_stub NAME BODY`, plus `_t_stub_stdin` (quoted heredoc), `_t_stub_cat`,
`_t_stub_writes`. Use these rather than `_t_stub foo '...$1...'`, which
trips SC2016 every time.
- **`_t_run FUNC [ARG...]`** — runs FUNC in the CURRENT shell, output in
`$out`. **Never `out="$(func)"`:** command substitution forks a subshell and
kcov then does not register the sourced lib at all (measured: 0 lines via
`$( )`, 54 via a direct call).
- **`_t_hide TOOL...`** — makes a tool genuinely unfindable. It KEEPS
`$FAKE_BIN` on PATH, so **`_t_unstub` the tool first** or the harness's own
default stub still satisfies `has_cmd` and the branch never runs.
- `_t_unstub` (runs `hash -r`; deleting the file alone does nothing).
## Production seams
Most libs write to `/etc`. The idiom is an override **defaulting to the real
path**, so production behaviour is byte-identical:
```bash
local dropin="${SYSCTL_DROPIN_DIR:-/etc/sysctl.d}/99-performance-tuning.conf"
```
Reuse existing names — `SYSTEMD_UNIT_DIR`, `JOURNALD_CONF_DIR`,
`UDEV_RULES_DIR`, `SYSCTL_DROPIN_DIR` — and grep the entry script before
picking one; a collision is silent.
**Watch for globals ASSIGNED inside a function** (`XORG_CONF="/etc/..."` with
no `local`). Exporting from the harness does not survive the assignment; the
default must live at the assignment itself.
After the first run, `ls -la` the real paths and confirm the mtimes did not
move.
## The measurement loop
```bash
# 1. write the tests, chmod +x
# 2. run bare first -- far faster than the jail
bash <suite>/run_all.sh
# 3. measure through the RUNNER, in a detached worktree
timeout 900s bash meta/scripts/shell_coverage_jail.sh --timeout 600s \
--subject <suite>/run_all.sh --measure <lib>.sh --min 1 \
--fail-on-case-error -- ""
```
Then the three-step bar to clear an exemption:
1. 100.00% through `run_all.sh`.
2. Again at `--min 100`, so a shortfall is a non-zero exit.
3. Delete the allowlist line, **then** run
`bash meta/scripts/check_shell_coverage.sh <lib>` and confirm exit 0.
Running it before deleting proves nothing — it passes via the exemption.
## Traps that cost real hours last session
- **Verify an edit landed.** A seam edit reported success, was measured, and
never reached the commit; ci-mirror caught it only because it runs from a
clean checkout. **After editing, `grep` the file for what you just wrote.**
- **Run the suite from a clean checkout before pushing.** It catches exactly
the above:
```bash
SP=$(mktemp -d); git worktree add --detach "$SP/c" HEAD
bash "$SP/c/<suite>/run_all.sh"; git worktree remove --force "$SP/c"
```
- **NEVER run two coverage jails at once.** Every false FAIL last session was
CPU contention — a jail plus another jail, or a jail plus `run_all.sh`.
Libs "failed" a sweep and passed solo. While a sweep runs, do zero-CPU work.
- **Another agent may be committing to this repo concurrently.** The jail
fingerprints `git status --porcelain` and aborts with "a write escaped the
jail" on THEIR edits. Measure in a detached worktree, and re-check a
transient gate failure before believing it.
- **Adding a test file can break an ALREADY-CLEARED lib.** Unexplained:
`test_arch_sysctl.sh` made kcov stop registering `bt_audio.sh` entirely
("instrumented no lines"), turning the gate red. Ruled out: `export out`,
the `_t_hide` cache, and runtime (still fails at a 600s timeout). The file
is quarantined as `arch_sysctl_quarantined_cases.sh`, outside the
`test_*.sh` glob, with 23 still-green assertions.
**After adding any test file, re-measure an already-cleared lib** —
`bt_audio.sh` is the canary.
- **`shfmt` reformats staged shell files**, moving the denominator and
invalidating a measurement. Run `shfmt -l` first and land reformats
separately.
- **The 250-line cap aborts commits**, and a task notification can report
"completed (exit code 0)" for a commit that actually failed. Check.
- **Stage narrowly.** A `git add` of a directory swept another agent's
untracked file into a commit and broke the push.
- **`mapfile` is bash-only**; the tool shell is zsh. Wrap in `bash -c`.
- **`git stash` is blocked** — even `git stash list` inside a compound
command trips the guard. Use a detached worktree.
- **Do not pipe jail output through `grep 'lines ='`** — that hides the
`warn: case timed out` line, which is the diagnosis you need.
- **Do not `pgrep -f <thing>` in a wait loop** — the loop's own command line
matches, so it never exits.
## The coverage instrument
`coverage = (PS4-traced | kcov hits) & (kcov line set - continuation lines)`
`continuation_lines()` in `meta/scripts/lib/shell_coverage_lines.py` excludes
lines kcov instruments but bash attributes no statement to: multi-line quoted
arguments, multi-line array literals, `} |` / `} >>file`, and (added last
session) `done >file` / `done <file`.
If you touch it: the acceptance pair is `dot_resolver_install.sh` = 39/39 and
`rpi_nc_ca.sh` = 73/73 through `features/lib/tests/run_all.sh` **with its
`jail_args` file**. And audit any new predicate by printing every newly
excluded line **with its text, across the WHOLE repo** — scoping that audit to
one directory last session undercounted the blast radius by 35x (2 lines vs
~75).
A lib already at 100% cannot regress from excluding more lines. Over-exclusion
is the only real risk.
### Known-unfixed artifact
`bt_adapter.sh` is capped at 98.55% (68/69) by line 97: the opening line of a
backslash-continued **assignment**, which bash attributes to the LAST line of
the run. A backslash-continued plain **command** is attributed to its first
line and covers normally. Find them with:
```bash
grep -nE '^[[:space:]]*(local[[:space:]]+)?[A-Za-z_][A-Za-z0-9_]*=.*\\$' <lib>
```
### Libs that do not measure at all
`arch_sysctl.sh` and `nc_php.sh` both report "kcov instrumented no lines" on
every version of the instrument. Treat as unverified; they stay exempt.
---
## State as of 2026-08-23
- Allowlist **84** (was 91 at the start of the previous session).
- `fixes/lib` cleared: `arch_cpu` 57/57, `arch_hardware` 94/94,
`thorium_repairs` 75/75, `hosts_guard_migrate` 81/81,
`hosts_guard_rollback` 65/65, `ubuntu_perf_fixes` 81/81,
`nvidia_config` 76/76.
- The `fixes/lib` suite: 30 files, 645 assertions, 37s, zero suppressions.
- Three production bugs were found by writing tests, all dead branches:
`bt_audio.sh:21` (`timeout` cannot invoke a shell function),
`arch_perf_report.sh:36,65` and `arch_hardware.sh` (journal-size regex
demanded a space before the unit, but `journalctl` prints `305.5M`, so the
vacuum was unreachable). **Expect more. When a branch looks untestable,
check whether it is reachable in production first** — the user's standing
answer is "fix it in this commit".
Still exempt and measured by an older session; **all predate the instrument
fixes and may now measure higher — re-measure before writing any test**:
`dwm_config.sh` 98.48%, `pacman_hook_stall_setup.sh` 96.30%,
`aw_autostart.sh` 97.56%, `rpi_nc_install.sh` 96.59%,
`transcribe_pkgmgr.sh` 98.00%, `clean_audio_filters.sh` 91.40%,
`transcribe_deps.sh` 92.31%.
`rpi_nc_ca.sh` (73/73) and `dot_resolver_install.sh` (39/39) are already at
100% and can come off the allowlist whenever you want, by the three-step bar.
That is two free entries.
If those three numbers do not match, something moved since 2026-08-23 —
find out what before extracting anything.
## What to do
**1. `hosts` first** (5 depth violations). It is unblocked: the guard repoint
landed on 2026-08-23, so no guard names a repo path any more
(`sudo guardctl file-guard status resolved` should print
`/usr/local/share/guard-lib-plugins/...`). Its installer was already proven in
vmbox to run from an extracted layout.
**2. `digital_wellbeing` last** (41 violations). The hard one. It owns the
pacman wrapper, `chattr +i`, `heavy_job_lock.sh`, and 12 orchestrator paths in
`check_and_enable_services.sh` / `setup_periodic_system.sh` that the flatten
already rewrote once. `setup_midnight_shutdown.sh` is `chattr +i` and is
excluded from two pre-commit fixer hooks **by path** — that exclude must move
with it, or the next commit fails on a file nothing can write.
**3. Then close the loop:** delete `--warn` from the `directory-depth-cap` hook
entry in `.pre-commit-config.yaml`. That is the whole point of the campaign —
the cap only enforces once both units are gone. Verify with
`./meta/scripts/check_directory_depth.sh --all` exiting 0.
## Constraints
- Use the `extract-to-own-repo` skill; do not hand-roll the mechanics.
- **Sandbox-first**: both units are root/`/etc`/systemd/pacman territory, so
vmbox before the host (`vmbox-testing` skill). A vmbox pass never proves the
host is fine — report it as "passed in vmbox, not verified on the host".
- Never `--no-verify`. Every commit in this campaign has been green.
- `sudo -n` works and `guardctl` is at `/usr/local/bin/guardctl` — do not
hand me commands you can run yourself.
- Run `./meta/scripts/report_live_state.sh --check` **before and after** each
move. It scans `/usr/local/bin` too, where installed copies name repo paths
invisibly to a repo grep. There are already 8 known-stale references from the
earlier flatten; do not add more.
- Each commit needs an evidence artifact; 4+ staged code files (`.sh` counts,
`.json`/`.yaml` do not) also needs a contract.
## Ask before doing (do not decide these yourself)
- deleting `.hippo/` (161 tracked files; dead since 2026-05-28, but it is a
deletion)
- `rm -rf` on the 3.7 GB of untracked extraction residue (`focus_owner/`,
`bucket_catch/`, `kcd2_dice_solver/`, `billsplit/`)
## Do NOT
- attempt the Nextcloud dedup — measured and recommended against; it needs a
Raspberry Pi that cannot be reached from here, and only 3 of 15 libs have
tests. See item 3 of the status doc for the numbers.
- raise the 2G pytest cap if the push fails with "tests passed, then Killed".
That is the ci-mirror concurrency issue, already fixed; measure what runs
alongside instead.
- "sync" `.pre-commit-config.yaml` with `meta/.pre-commit-config.yaml`. They
are deliberately different files.
## Done means
`./meta/scripts/check_directory_depth.sh --all` exits **0** with `--warn`
removed from the hook, both units are their own repos with CI, the live-state
check adds no new stale references, and `main` is pushed clean.

174
NEXT_SESSION_RESTRUCTURE.md
View File

@ -4,102 +4,122 @@ Plan: `~/.claude/plans/after-refactor-of-testsandmisc-modular-wigderson.md`
Live-state checklist: `docs/restructure-live-state.md`
(regenerate with `./meta/scripts/report_live_state.sh`)
## Verified state as of 2026-08-23 (all pushed, `main` clean)
```
depth violations 46 (41 digital_wellbeing + 5 hosts — nothing else)
source files 1051 / 1200 ceiling
tracked files 1346 (2295 at the start, 1993 before the prune)
```
Both gates are wired and run on every commit. The depth gate runs `--warn`
because the only paths left over the cap are the two units below. **Delete
`--warn` from the `directory-depth-cap` hook entry in `.pre-commit-config.yaml`
once BOTH units are out, and the cap enforces for real.**
## Done
**Archived** to `github.com/kuhyx/testsAndMisc-archive`, full history:
`bucket_catch`, `poker-stakes`, `billsplit` (+ `python_pkg/billsplit_coverage`,
its CI workflow, the `app_icons` registry entry, and the paired
`.binary-allowlist` / `.gitignore` icon rules — `.binary-allowlist` now has no
patterns at all).
`bucket_catch`, `poker-stakes`, `billsplit`.
**Extracted** to public repos, full history, each with CI it did not have here:
`reverse-survivors`, `kcd2-dice-solver` (`~/kcd2-dice-solver`), `focus-owner`,
`mtk-root`, `system-maintenance` (`~/system-maintenance`), `android-guardian`
(`~/android-guardian`), `phone-focus-mode` (with `python_pkg/focus_policy`).
**Extracted** to public repos, full history: `reverse-survivors`,
`kcd2-dice-solver` (`~/kcd2-dice-solver`), `focus-owner` (**no local clone —
GitHub only**), `mtk-root`, `system-maintenance` (`~/system-maintenance`),
`android-guardian` (`~/android-guardian`), `phone-focus-mode`.
**Flattened.** `linux_configuration/scripts/` and `single_use/` are gone;
categories sit directly under `linux_configuration/`. `i3-configuration` moved
out of `periodic_background` into `i3/` + `i3blocks/`, and
`misc/testsAndMisc-bash/` lost its redundant nesting.
**Flattened.** `linux_configuration/scripts/` and `single_use/` are gone.
**Length cap widened** to catch extensionless scripts, and `boot-repair` split
756 → 218 with four libs.
**Gates wired.** 57 structural depth violations exempted (whole projects `C/`,
`dwm/`, `artgate/`; a module-split tool; prose trees; and the five
format-dictated segments `systemd`/`hooks`/`workflows`/`patches`/`fixtures`).
`check_repo_size.sh` counts tracked source files and caps the bookkeeping dirs
at 60 each, with `--prune` to clear them.
Depth violations **556 → 104**; tracked files **2295 → ~1980**. Every commit
green; nothing used `--no-verify`.
**Pruned** 655 superpowers artifacts (695 → 40 + 2 templates).
**Guard repoint DONE on the host.** Both guards now name
`/usr/local/share/guard-lib-plugins/`, re-enforced, target files byte-identical
by md5, DNS verified between them. Backups at
`/var/tmp/{resolved.conf,nsswitch.conf}.pre-repoint`.
## Remaining
1. ~~The guard repoint.~~ **DONE on the host, 2026-08-23.** The "classifier
blocks me" note was stale: `sudo -n` works and `guardctl` is at
`/usr/local/bin/guardctl`. Verified in vmbox first (stale-path scenario
reproduced, then repointed), then run on the host one guard at a time with
a DNS check between them. Both now name
`/usr/local/share/guard-lib-plugins/`, both re-enforced (`----i-------e---`,
path units active), both target files byte-identical by md5 before and
after, DNS resolving throughout. Enforcement confirmed live, not just
reported: a `sudo` append to `/etc/nsswitch.conf` is refused.
Backups at `/var/tmp/{resolved.conf,nsswitch.conf}.pre-repoint`.
1. **`hosts` extraction — do this first; everything else is optional.**
Now unblocked: the guards no longer name a repo path. Its installer was
already proven in vmbox to run from an extracted layout. 5 depth violations
live here.
2. `hosts` extraction — **now unblocked** by (1). Its installer was already proven in
vmbox to run from an extracted layout with no monorepo path.
3. `digital_wellbeing` **last** — pacman wrapper, `chattr +i`, `heavy_job_lock.sh`
(moved in with it), and the 12 orchestrator paths in
`check_and_enable_services.sh` / `setup_periodic_system.sh`, which now point
at the NEW locations and will need updating again when it moves. It also owns
2. **`digital_wellbeing` — last, and the hardest.** 41 depth violations.
Owns the pacman wrapper, `chattr +i`, `heavy_job_lock.sh`, and 12
orchestrator paths in `check_and_enable_services.sh` /
`setup_periodic_system.sh` that were already rewritten once by the flatten
and will need it again. Also owns
`linux_configuration/tests/test_shutdown_timer_monitor.sh`.
4. ~~Wire the gates.~~ **DONE.** The 57 structural violations are exempted
and both gates run on every commit. The depth gate runs `--warn`, because
the only 47 paths left over the cap are the ones (2) and (3) will remove;
drop `--warn` as the last step of those extractions and the cap enforces
for real. `check_repo_size.sh` counts tracked source files (1049/1200) and
also caps the bookkeeping dirs at 60 each.
5. ~~Prune `docs/superpowers/evidence|contracts`.~~ **DONE** — 655 removed,
newest 20 of each kept, tracked files 1993 → 1340. Nothing reads the
corpus; git history keeps it. `.hippo/` is **decided but not executed**:
all 158 entries have `retrieval_count: 0`, 22 self-tag `invalidated`, the
last write was 2026-05-28, `hippo.db` is untracked and the tool is
uninstalled. It is dead; awaiting the untrack-vs-delete call.
6. **Recommended against**, with numbers. The estimate was right (~3,600 lines
across 22 files — a filename grep finds only 400 of them), but the two
`nc_*` (1212 lines) and `rpi_nc_*` (1171 lines) lib families are parallel
_evolutions_, not copies: all four shared function names differ in body,
not just comments. Only 3 of 15 libs have tests, and the target is a remote
Raspberry Pi that cannot be tested against from here. Merging is a
regression risk out of proportion to an optional cleanup.
`setup_midnight_shutdown.sh` is `chattr +i` and is excluded from two
pre-commit fixer hooks by path — that exclude must move with it.
7. **3.7 GB of extraction residue**, untracked and unreported until now:
`focus_owner/` (3.2G), `bucket_catch/` (303M), `kcd2_dice_solver/` (149M),
`billsplit/` (62M). None has a `.git`; each holds **zero** source files
outside `build/`, `node_modules/`, `dist/` and `coverage/`. The real repos
are on GitHub and were pushed 2026-08-23, so this is pure build output —
`rm -rf` on the user's say-so. Note `focus-owner` has no local clone at
`~/focus-owner`; only the GitHub copy exists.
3. **Nextcloud dedup — recommended AGAINST, with numbers.** Not a filename
grep: `grep -rln -i nextcloud linux_configuration/` finds **3,618 lines
across 22 files** (566 entry points + 2,383 libs + 604 tests). The two lib
families `nc_*` (1212 lines) and `rpi_nc_*` (1171) are parallel
_evolutions_, not copies — all four shared function names differ in body,
not just comments. Only **3 of 15 libs have tests**, and the target is a
remote Raspberry Pi that cannot be tested against from here. Do not attempt
without hardware access.
4. **`.hippo/` — dead, awaiting a decision.** 161 tracked files, 3.2M. All 158
episodic entries have `retrieval_count: 0`, 22 self-tag `invalidated`, last
write 2026-05-28, the tool is uninstalled, nothing references it.
Untracking needs a new `.gitignore` entry — `hippo.db` is currently
untracked only because `*.db` is ignored. **Ask before deleting.**
5. **3.7 GB of extraction residue**, untracked: `focus_owner/` (3.2G),
`bucket_catch/` (303M), `kcd2_dice_solver/` (149M), `billsplit/` (62M).
None has a `.git`. The only non-build files are `.idea/` IDE config and a
Flutter-_generated_ `GeneratedPluginRegistrant.java` — no hand-written
source. Repos are safe on GitHub. **Ask before `rm -rf`.**
6. **8 stale live-system references** (pre-existing, from the earlier flatten,
NOT from the gate work). `./meta/scripts/report_live_state.sh --check`
lists them; they name `linux_configuration/scripts/...` paths that no
longer exist. Two were the guard plugins, now fixed; the rest include
`setup_dns_blocker.sh` and `organize_downloads.sh`.
## Traps already paid for
- **Run it, don't read it.** Repointing kcd2 hit three failures in a row (pnpm
ignoring the `package.json` field, no-TTY `node_modules` purge, a Docker mount
on the deleted path). `dice.kuhy.duckdns.org` now serves from the new repo.
- **`/usr/local/bin` holds copies naming repo paths** — invisible to a repo grep.
`report_live_state.sh` scans there now.
- **Removing a file can break a suite that stays** (`mtk_harness.sh`), and a
removal can silently shrink CI (`shell-tests.yml` uses TWO hand-maintained
lists plus a glob — check all three).
- **Never let a test fall back to `/usr/local/bin`**: the installed copy can be
stale, and the test then compares against a binary it never generated.
- **pre-commit's patch restore can revert an edit** made before the commit. The
`app_icons` and `AGENTS.md` changes each had to be applied twice.
- **Run it, don't read it.** `--prune` sorted on `%cs` (date only), which ties
every same-day artifact; `sort` fell back to the path and it deleted
_alphabetically_, picking one of the newest files. Invisible in review.
Use `%ct` + `sort -rn`. Test destructive modes in a detached worktree.
- **A filename grep undercounts a capability 7x.** `git ls-files | grep -i X`
matches names only and misses the `lib/` layer, where the bulk lives.
Use `grep -rln` over contents before calling an estimate stale.
- **"Tests passed, then Killed" is not a test failure.** ci-mirror ran
`pre-commit --all-files` (~1.75 GiB) and pytest (~0.29 GiB) concurrently;
the pytest scope has `MemorySwapMax=0`, so it was the one killed. Fixed by
`should_serialise_gates` in `meta/scripts/ci_mirror_mem.sh`. **Never raise
the 2G cap** — measure what runs alongside. `/usr/bin/time -v` hides this
(it measures only the parent); read `memory.peak` from a cgroup.
- **`sudo -n` works here and `guardctl` is at `/usr/local/bin/guardctl`.** The
old "the classifier blocks me" note was stale.
- **The pre-push gate is not the pre-commit gate.** It adds prettier and
ci-mirror. Two docs were non-conforming before this session and blocked
_every_ push; prettier normalises `*emphasis*` to `_emphasis_`.
- **`.pre-commit-config.yaml` is NOT a symlink** (unlike `pyproject.toml`,
`run.sh`, `requirements.txt`, `lint_python.sh`, `.fvmrc`). The root file is
the active config; `meta/.pre-commit-config.yaml` is a separate standalone
variant, already ~219 lines divergent. Do not "sync" them.
- **`/usr/local/bin` holds copies naming repo paths** — invisible to a repo
grep. `report_live_state.sh` scans there.
- **Path rewrites need resolving, not reviewing.** Relative `source` depths
differ per file (`../../` vs `../../../`), `REPO_ROOT` walks a fixed number of
levels, `$VAR/scripts/...` prefixes hide from a naive regex, and
`# shellcheck source=` directives break independently of the code.
differ per file, and a test can hardcode a prefix the script no longer uses
— `test_music_parallelism.sh` failed a layout that was correct.
- **Sourcing top-level code is not a split.** boot-repair's report block has
`exit` calls; moving it broke 30 of 42 tests.
## Unrelated: syncyomi-guard is red
Failing since 00:00 on 2026-08-23, ~17h before this work started. It is working
as designed: the library collapsed from 2185 manga / 68902 chapters to 8 / 895.
See the `syncyomi-restore-silent-partial` memory; do NOT checkpoint the WAL
before recovering. Newest snapshot in `~/syncyomi/snapshots/` is 2026-08-15.
Failing since 00:00 on 2026-08-23, working as designed: the library collapsed
from 2185 manga / 68902 chapters to 8 / 895. See the
`syncyomi-restore-silent-partial` memory; do NOT checkpoint the WAL before
recovering. Newest snapshot in `~/syncyomi/snapshots/` is 2026-08-15.

37
docs/superpowers/evidence/fix-bats-diag-path-2026-08-23.json Normal file
View File

@ -0,0 +1,37 @@
{
"intent": "Turn the Shell tests workflow green again: test_network_diagnostics.bats still sourced its scripts through the scripts/ level the flatten removed.",
"scope": [
"linux_configuration/tests/test_network_diagnostics.bats - DIAG_DIR and its header comment",
"NEXT_SESSION_RESTRUCTURE.md / NEXT_SESSION_PROMPT.md - handoff rewrite for the two remaining units",
"Non-goal: the diagnostics scripts themselves, which are correctly placed."
],
"changes": [
"DIAG_DIR pointed at $REPO_DIR/scripts/diagnostics; the flatten dissolved linux_configuration/scripts/, so setup() failed to source and all 30 tests errored. Now $REPO_DIR/diagnostics.",
"Fixed the header comment naming the same dead path.",
"Rewrote the handoff docs around the two units that remain, with the verified numbers and the traps this session paid for."
],
"verification": [
{
"command": "bats linux_configuration/tests/test_network_diagnostics.bats",
"result": "pass",
"evidence": "30/30 ok. Previously every test failed in setup with 'No such file or directory' on .../scripts/diagnostics/steam-download-duty.sh."
},
{
"command": "grep -rn 'linux_configuration/scripts/|scripts/single_use|single_use/' over tracked .sh/.bats/.py/.yml",
"result": "pass",
"evidence": "No matches. This was the last stale reference to the removed levels inside the repo; the 8 remaining live-state hits are installed copies under /usr/local/bin, outside the tree."
},
{
"command": "gh run view 32656921622 --log-failed",
"result": "pass",
"evidence": "Confirmed the CI failure was exactly this suite. The previous run (cbdf3a3d) was green because ci-mirror aborted before the bats stage, so the breakage was latent rather than newly introduced."
}
],
"risks": [
"CI_GREEN_SKIP=1 is used for this commit because the ci-baseline-green hook refuses to build on a red main and this commit IS the fix - the documented escape for exactly this case."
],
"rollback": [
"git revert the commit; the bats suite returns to the dead path and Shell tests goes red again.",
"After rollback, run bats linux_configuration/tests/test_network_diagnostics.bats to confirm the expected failure."
]
}

4
linux_configuration/tests/test_network_diagnostics.bats
View File

@ -1,5 +1,5 @@
#!/usr/bin/env bats
# Unit tests for the pure helpers in scripts/diagnostics/.
# Unit tests for the pure helpers in diagnostics/.
#
# Requires: bats (extra/bats). Run with:
# bats linux_configuration/tests/test_network_diagnostics.bats
@ -11,7 +11,7 @@
setup() {
REPO_DIR="$(cd -- "$(dirname -- "$BATS_TEST_FILENAME")/.." && pwd)"
DIAG_DIR="$REPO_DIR/scripts/diagnostics"
DIAG_DIR="$REPO_DIR/diagnostics"
# Both scripts guard main() behind a BASH_SOURCE check, so sourcing them
# loads the helpers without starting a measurement run.
# shellcheck source=/dev/null