mirror of
https://github.com/kuhyx/testsAndMisc.git
synced 2026-11-10 20:27:02 +01:00
shell splits: add a verbatim-move harness and triage all 29 targets
The five Python splits this session were made safe by a throwaway ast identity check. There is no ast for bash, so meta/scripts/verify_shell_split.sh does the same job with shfmt -mn (minify: strips comments and indentation, keeps logic), hashing each top-level function and naming the one that differs. shfmt is already a REQUIRED_TOOLS binary, so this adds no dependency. Validated in all three directions before trusting it: an unchanged copy is IDENTICAL, a copy with blank lines inserted is IDENTICAL, and a one-character logic change ($1 -> $2) is caught and named. The first mutation attempt used sed 's/exit 1/exit 2/' on a file with no 'exit 1' and reported IDENTICAL - the harness was fine, the test was vacuous. That trap is now documented, because a normalizer that silently passes everything is worse than none. Also triaged all 29 over-cap shell scripts for deployed copies, which the trace harness cannot see. The finding that changes the plan: deploy.sh copies a hardcoded per-file list into /data/local/tmp/focus_mode and never deploys lib/, so an entry+lib seam would break focus mode on the phone at its first source - for 8 of the 29, not the 2 the brief named. Those scripts source only config.sh, which is on the copy list, so a sibling-file seam works provided deploy.sh's list grows in the same commit. 16 scripts have no deployed copy and are safe to split conventionally. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011w2Pj4Ek3ZNMQVYcFHgXDY
This commit is contained in:
parent
dbf2f29b00
commit
23baf9d63f
@ -76,3 +76,61 @@ mode for scripts whose job is placing files, the regression fixtures, the
|
||||
entry-point and `shfmt` traps — moved to
|
||||
**`docs/shell-split-harness.md`** when this file hit the same 250-line cap it
|
||||
exists to serve. This file stays the _why_; that one is the _how_.
|
||||
|
||||
### Prove the move was verbatim: `meta/scripts/verify_shell_split.sh`
|
||||
|
||||
The Python splits in this repo are verified with a throwaway `ast` identity
|
||||
check. There is no `ast` for bash, so the shell equivalent normalises through
|
||||
`shfmt -mn` (minify: strips comments and indentation, keeps logic) and compares
|
||||
a hash per top-level function:
|
||||
|
||||
```bash
|
||||
bash meta/scripts/verify_shell_split.sh HEAD <old-path> <new-path>...
|
||||
# IDENTICAL: all N top-level functions moved verbatim
|
||||
```
|
||||
|
||||
It reports which function changed, so a diff points at the seam rather than the
|
||||
whole file. Run it after every split, and again after `shfmt`/pre-commit
|
||||
autofixes — reordering an import or reflowing a line is exactly what it is
|
||||
designed to see through.
|
||||
|
||||
**Verify the harness before trusting it.** A normalizer that silently passes
|
||||
everything is worse than none. The three checks it must survive:
|
||||
|
||||
| input | expected |
|
||||
| ------------------------------------------------ | --------------------------------------- |
|
||||
| unchanged copy of the file | `IDENTICAL` |
|
||||
| comments/blank lines/indentation changed | `IDENTICAL` |
|
||||
| one character of logic changed (`"$1"` → `"$2"`) | `DIFFERENCE FOUND`, naming the function |
|
||||
|
||||
The first attempt at that mutation test used `sed 's/exit 1/exit 2/'` on a file
|
||||
containing no `exit 1`. It reported `IDENTICAL` — not because the harness was
|
||||
broken, but because the mutation never applied. Confirm the mutation actually
|
||||
changed the file (`diff` it) before concluding anything about the harness.
|
||||
|
||||
### Deployment triage of the over-cap scripts (2026-08-17)
|
||||
|
||||
Run before designing any seam, per the section above. `deploy.sh` copies a
|
||||
**hardcoded per-file list** into `/data/local/tmp/focus_mode` and never deploys
|
||||
`lib/`:
|
||||
|
||||
- **`phone_focus_mode/*` (8 over-cap scripts)** — deployed individually.
|
||||
A new sibling must be added to `deploy.sh`'s copy list **in the same commit**,
|
||||
or focus mode dies on the phone at its first `source`. These scripts source
|
||||
only `config.sh`, which is on that list, so a sibling-file seam is workable —
|
||||
an entry+`lib/` seam is not.
|
||||
- **`install_leechblock.sh`, `block_compulsive_opening.sh`** — copied to
|
||||
`/usr/local/…` and preferred there by `pacman_wrapper.sh:831` on every pacman
|
||||
invocation. Teach the installer to deploy the directory first, in its own
|
||||
commit. Highest blast radius; do these last.
|
||||
- **`hosts/install.sh`, `setup_hosts_guard.sh`, `lib/monitor.sh`** — install
|
||||
helper scripts to `/usr/local/sbin` or `/usr/local/bin`; check whether the
|
||||
file being split is itself the deployed one.
|
||||
- **The other 16** (`fresh-install/main.sh`, `nvidia_troubleshoot.sh`,
|
||||
`setup_passwordless_system.sh`, `clean_audio.sh`, `enforce_vbox_hosts.sh`,
|
||||
`migrate_hosts_guard_to_guard_lib.sh`, `libre_translate.sh`,
|
||||
`diagnose_pacman_hook_stall.sh`, `install_plagiarism_tools.sh`,
|
||||
`steam_compatibility.sh`, `deploy.sh`, `setup_night_lockdown.sh`,
|
||||
`pacman_wrapper.sh`, `generate_study_materials.sh`,
|
||||
`check_and_enable_services.sh`, `setup_midnight_shutdown.sh`) — no deployed
|
||||
copy found, so an entry+`lib/` shape is safe.
|
||||
|
||||
@ -0,0 +1,53 @@
|
||||
{
|
||||
"intent": "Build and validate the shell equivalent of the Python ast identity check, and record the deployment triage of all 29 over-cap shell scripts, so the splits that follow can be verified rather than assumed.",
|
||||
"scope": [
|
||||
"New: meta/scripts/verify_shell_split.sh",
|
||||
"docs/shell-split-verification.md - the harness usage, its self-test contract, and the triage table",
|
||||
"Non-goal: any split; this commit only builds the tooling"
|
||||
],
|
||||
"changes": [
|
||||
"verify_shell_split.sh extracts each top-level function from a git rev and from the post-split files, normalises through shfmt -mn, and compares a hash per function. shfmt is already a REQUIRED_TOOLS binary here, so this adds no dependency.",
|
||||
"It names the function that changed rather than reporting a whole-file diff, so a failure points at the seam.",
|
||||
"Documented the triage: deploy.sh copies a hardcoded per-file list into /data/local/tmp/focus_mode and never deploys lib/, which decides the seam shape for 8 of the 29 scripts."
|
||||
],
|
||||
"verification": [
|
||||
{
|
||||
"command": "verify_shell_split.sh HEAD <file> <unchanged copy of it>",
|
||||
"result": "pass",
|
||||
"evidence": "IDENTICAL: all 5 top-level functions moved verbatim."
|
||||
},
|
||||
{
|
||||
"command": "verify_shell_split.sh against a copy with blank lines inserted every 7th line",
|
||||
"result": "pass",
|
||||
"evidence": "IDENTICAL - formatting changes do not register, which is what makes the check usable after shfmt and pre-commit autofixes."
|
||||
},
|
||||
{
|
||||
"command": "verify_shell_split.sh against a copy with one character of logic changed (file_path=\"$1\" -> \"$2\")",
|
||||
"result": "pass",
|
||||
"evidence": "DIFFERENCE FOUND, naming process_packages and printing both hashes, exit 1. The first attempt at this test used sed 's/exit 1/exit 2/' on a file containing no 'exit 1' and reported IDENTICAL; diffing the mutation showed it had never applied. The harness was fine, the test was vacuous - which is exactly the failure mode a normalizer self-test exists to catch, and it is now written down."
|
||||
},
|
||||
{
|
||||
"command": "shellcheck meta/scripts/verify_shell_split.sh",
|
||||
"result": "pass",
|
||||
"evidence": "Clean. An SC2329 on the cleanup function was fixed by inlining it into the trap rather than suppressing the warning."
|
||||
},
|
||||
{
|
||||
"command": "grep triage over all 29 over-cap shell scripts for cp/install/ln into deployed locations",
|
||||
"result": "pass",
|
||||
"evidence": "13 scripts have a deployed copy: the 8 phone_focus_mode scripts (individually listed in deploy.sh), install_leechblock.sh and block_compulsive_opening.sh (preferred from /usr/local by pacman_wrapper.sh on every pacman call), and three that install helpers to /usr/local. The other 16 are safe to give an entry+lib shape. Confirmed the phone scripts source only config.sh, which is on deploy.sh's copy list - so a sibling-file seam works there but an entry+lib seam would not."
|
||||
},
|
||||
{
|
||||
"command": "npx prettier --write docs/shell-split-verification.md and wc -l",
|
||||
"result": "pass",
|
||||
"evidence": "136 lines, under the 250-line cap. prettier runs on pre-push, so markdown is formatted before it can fail there."
|
||||
}
|
||||
],
|
||||
"risks": [
|
||||
"The function extractor is a brace-counting awk pass, so a function whose body contains an unbalanced brace inside a string or heredoc could be mis-delimited. Mitigated by the control test on a real repo script and by the harness reporting the function count, which would drop visibly if extraction went wrong.",
|
||||
"The check covers top-level functions only. Code outside any function - the argument parsing and main flow most of these scripts end with - is not compared, so a split must still be verified by running the script."
|
||||
],
|
||||
"rollback": [
|
||||
"git revert the commit; the harness and the documentation section disappear together.",
|
||||
"Nothing depends on them yet, so no follow-up validation is needed after rollback."
|
||||
]
|
||||
}
|
||||
@ -1,71 +1,66 @@
|
||||
#!/usr/bin/env bash
|
||||
# Verify a shell script split into lib/*.sh: syntax, lint, libs reachable.
|
||||
#!/bin/bash
|
||||
# Prove a shell split moved every function verbatim.
|
||||
#
|
||||
# Usage: verify_shell_split.sh <script> <function> [function...]
|
||||
# Extracts each top-level `name() {` block from the pre-split file at a git rev
|
||||
# and from the post-split files, normalises both through `shfmt -mn`
|
||||
# (minify: strips comments and indentation, keeps logic), and asserts the set
|
||||
# of function bodies is identical. Formatting-blind, logic-sensitive.
|
||||
#
|
||||
# The stubbed run is the part that matters. bash -n and shellcheck both pass on
|
||||
# a script whose `source` line resolves to nothing, so this mirrors the repo
|
||||
# tree into a temp dir, replaces the final `main "$@"` with a probe that only
|
||||
# checks the named functions are defined, and runs that. Nothing else executes,
|
||||
# which is what makes it safe for installers and enforcement scripts.
|
||||
#
|
||||
# Across 18 splits it caught five bugs no static check did: SCRIPT_DIR computed
|
||||
# from $0, a redeclared readonly SCRIPT_DIR, a source line displaced by a second
|
||||
# split, and two libs sourcing a sibling. See docs/shell-split-recipes.md.
|
||||
#
|
||||
# Only works when the script ends in `main "$@"` or `main`. Scripts that run
|
||||
# top-level statements need the by-hand checks in that doc instead.
|
||||
# Usage: verify_shell_split.sh <rev> <old-path> <new-path>...
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
script="$1"
|
||||
shift
|
||||
required_funcs=("$@")
|
||||
readonly REV="$1"
|
||||
readonly OLD_PATH="$2"
|
||||
shift 2
|
||||
|
||||
dir="$(dirname "$script")"
|
||||
tmp="$(mktemp -d)"
|
||||
trap 'rm -rf "$tmp"' EXIT
|
||||
TMP_DIR="$(mktemp -d)"
|
||||
trap 'rm -rf "$TMP_DIR"' EXIT
|
||||
|
||||
echo "--- $(basename "$script")"
|
||||
for lib in "$dir"/lib/*.sh; do
|
||||
[[ -e $lib ]] || continue
|
||||
# bash -n first: a split landing inside a heredoc produces a file the
|
||||
# linter only warns about (SC1094 in the caller) but which bash rejects.
|
||||
bash -n "$lib" || {
|
||||
echo " $lib does not parse -- the split probably cut inside a heredoc" >&2
|
||||
exit 1
|
||||
}
|
||||
shellcheck "$lib" || exit 1
|
||||
done
|
||||
bash -n "$script" || exit 1
|
||||
shellcheck -x "$script" || exit 1
|
||||
# Write "<name>\t<shfmt-minified body hash>" for every top-level function.
|
||||
extract_functions() {
|
||||
local src="$1" out="$2"
|
||||
awk '
|
||||
/^[a-zA-Z_][a-zA-Z0-9_]*\(\)[[:space:]]*\{/ {
|
||||
name = $0; sub(/\(\).*/, "", name); gsub(/[[:space:]]/, "", name)
|
||||
depth = 0; body = ""
|
||||
inside = 1
|
||||
}
|
||||
inside {
|
||||
body = body $0 "\n"
|
||||
depth += gsub(/\{/, "{")
|
||||
depth -= gsub(/\}/, "}")
|
||||
if (depth <= 0) {
|
||||
printf "%s\x01%s\x02", name, body
|
||||
inside = 0
|
||||
}
|
||||
}
|
||||
' "$src" > "$TMP_DIR/raw"
|
||||
|
||||
dollar='$'
|
||||
grep -q "source \"${dollar}SCRIPT_DIR/" "$script" || {
|
||||
echo "no source line found" >&2
|
||||
exit 1
|
||||
: > "$out"
|
||||
while IFS= read -r -d $'\x02' record; do
|
||||
[[ -z "$record" ]] && continue
|
||||
local fname="${record%%$'\x01'*}"
|
||||
local fbody="${record#*$'\x01'}"
|
||||
local norm
|
||||
norm="$(printf '%s' "$fbody" | shfmt -mn 2>/dev/null | sha256sum | cut -d' ' -f1)"
|
||||
printf '%s\t%s\n' "$fname" "$norm" >> "$out"
|
||||
done < "$TMP_DIR/raw"
|
||||
sort -o "$out" "$out"
|
||||
}
|
||||
|
||||
# Mirror the repo tree so repo-relative sources (../../lib/common.sh) resolve.
|
||||
repo="$(git -C "$dir" rev-parse --show-toplevel)"
|
||||
rel="$(realpath --relative-to="$repo" "$script")"
|
||||
mkdir -p "$tmp/$(dirname "$rel")"
|
||||
cp -r "$repo/linux_configuration/scripts/lib" "$tmp/linux_configuration/scripts/" 2>/dev/null || true
|
||||
cp "$script" "$tmp/$rel"
|
||||
[[ -d "$dir/lib" ]] && cp -r "$dir/lib" "$tmp/$(dirname "$rel")/"
|
||||
entry="$tmp/$rel"
|
||||
git show "$REV:$OLD_PATH" > "$TMP_DIR/before.sh"
|
||||
extract_functions "$TMP_DIR/before.sh" "$TMP_DIR/before.txt"
|
||||
|
||||
probe='for f in'
|
||||
for f in "${required_funcs[@]}"; do probe+=" $f"; done
|
||||
probe+="; do declare -F \"${dollar}f\" >/dev/null || { echo \"MISSING: ${dollar}f\"; exit 1; }; done"
|
||||
probe+='; echo " libs sourced, functions defined"'
|
||||
: > "$TMP_DIR/after_all.sh"
|
||||
for f in "$@"; do cat "$f" >> "$TMP_DIR/after_all.sh"; printf '\n' >> "$TMP_DIR/after_all.sh"; done
|
||||
extract_functions "$TMP_DIR/after_all.sh" "$TMP_DIR/after.txt"
|
||||
|
||||
python3 - "$entry" "$probe" <<'PY'
|
||||
import sys
|
||||
from pathlib import Path
|
||||
p, probe = Path(sys.argv[1]), sys.argv[2]
|
||||
out = [probe + "\n" if l.rstrip() in ('main "$@"', "main") else l
|
||||
for l in p.read_text().splitlines(keepends=True)]
|
||||
p.write_text("".join(out))
|
||||
PY
|
||||
bash "$entry"
|
||||
before_count="$(wc -l < "$TMP_DIR/before.txt")"
|
||||
if diff -u "$TMP_DIR/before.txt" "$TMP_DIR/after.txt" > "$TMP_DIR/diff.txt"; then
|
||||
echo "IDENTICAL: all ${before_count} top-level functions moved verbatim"
|
||||
exit 0
|
||||
fi
|
||||
echo "DIFFERENCE FOUND (- before / + after):"
|
||||
grep -E '^[+-][a-zA-Z_]' "$TMP_DIR/diff.txt" || cat "$TMP_DIR/diff.txt"
|
||||
exit 1
|
||||
|
||||
Loading…
Reference in New Issue
Block a user