testsAndMisc/.github/skills/efficient-polling-scripts/SKILL.md
Krzysztof kuhy Rudnicki fc4b504093 Harden phone_focus_mode CPU use; whitelist wake_alarm_sync
The on-device focus enforcer was overheating the phone by rebuilding
firewall rules every tick. State-gate the work instead:

- dns_enforcer.sh: only rebuild the DNS chain when it's missing, detached
  from OUTPUT, or the wrong rule count (chain_intact/expected_rule_count),
  instead of flushing+reinserting ~50-100 rules every tick (~50% netd CPU).
- config.sh: throttle CURFEW_NET_REASSERT_INTERVAL 1s -> 15s (1s forked
  iptables every second all night, overheating); also whitelist
  com.kuhy.wake_alarm_sync (general + curfew-night lists).
- focus_daemon.sh: build the blocked-system-apps file once at startup and
  memoize is_allowed() per tick.
- deploy.sh: drop `su --mount-master` (removed in Magisk v26+).
- SKILL.md: document the state-change-gate polling pattern.

Verification: shellcheck clean; no-polling-antipatterns + evidence/contract
hooks pass. Evidence + contract artifacts included.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017RRCdR4D8SaVrN2xraMQxM
2026-07-11 00:16:11 +02:00

217 lines
11 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

---
name: efficient-polling-scripts
description: Use BEFORE writing any shell or Python script that runs on a timer, per-tick status bar (i3blocks/waybar/polybar), cron-like loop, or any repeated invocation. Prevents fork-storm anti-patterns that can consume many CPU-hours per day from tiny polling scripts.
---
# Efficient Polling & Status-Bar Scripts
## When this applies
Any script that runs **frequently** — per second or per few seconds — especially:
- i3blocks / waybar / polybar / xmobar / tmux status-line scripts
- cron / systemd-timer jobs with intervals < 1 min
- watcher loops invoked by another process every tick
- Python CLIs invoked from a shell hot loop
A single fork pipeline running once per second will consume ~30–50 CPU-minutes per day per forked helper. Five such scripts with 3–8 helpers each turn into **days of CPU-time lost per day** and tens of thousands of forked processes showing up in `atop`.
## The rules
### R1. Zero forks in the hot path when possible
Every `$(...)`, backtick, and `|` in a shell script forks a process. Favor bash builtins:
| Instead of | Use |
| ------------------------------- | ----------------------------------------------------------------------------------- |
| `$(cat /proc/loadavg)` | `$(</proc/loadavg)` or `read -r one _ < /proc/loadavg` |
| `echo "$x" \| awk '{print $1}'` | `read -r first _ <<< "$x"` or `arr=($x); first=${arr[0]}` |
| `echo "$x" \| tr -d '%'` | `${x//%/}` |
| `echo "$x" \| grep -Po '\d+%'` | `[[ $x =~ ([0-9]+)% ]] && vol=${BASH_REMATCH[1]}` |
| `echo "$a < $b" \| bc -l` | `(( a_times_100 < b_times_100 ))` (scale decimals to ints) |
| `sensors \| awk ...` | `read -r milli < /sys/class/hwmon/hwmonN/temp1_input` |
| `acpi -b \| awk ...` | `read -r cap < /sys/class/power_supply/BAT0/capacity` |
| `free -h \| awk ...` | parse `/proc/meminfo` with `while read -r` |
| `df -h / \| awk ...` | `stat -f` builtin? No: use a long-lived reader, or accept one fork at low frequency |
| `lspci \| grep -i nvidia` | check `/sys/bus/pci/devices/*/vendor` (0x10de == NVIDIA) |
### R2. Read from /sys and /proc directly
The kernel exposes structured data without forking anything. Useful paths:
- CPU load: `/proc/loadavg`
- CPU per-core stat: `/proc/stat`
- Memory: `/proc/meminfo`
- Temps / fans / voltages: `/sys/class/hwmon/hwmon*/`
- CPU on AMD: `name=k10temp`, `temp1_input` = Tctl (milli-°C, divide by 1000)
- CPU on Intel: `name=coretemp`
- Motherboard Super-I/O: `name=nct*` / `it87*` / `f71*`
- AMD GPU: `name=amdgpu`, plus `/sys/class/drm/card*/device/gpu_busy_percent`
- Battery: `/sys/class/power_supply/BAT*/` (`capacity`, `status`, `energy_now`, `power_now`)
- Backlight: `/sys/class/backlight/*/brightness`
- Network link: `/sys/class/net/*/operstate`, `/sys/class/net/*/statistics/*_bytes`
NVIDIA is the unfortunate exception — there is no sysfs utilization interface, so `nvidia-smi` is required. Mitigate with **R4** (long-lived producer).
### R3. Integer arithmetic, never `bc` in a hot loop
`bc` forks a process. For decimal comparisons, multiply out:
```bash
# "1.23" → 123, "0.45" → 45; compare against threshold ×100.
load_x100=$((10#${one//./}))
(( load_x100 < 150 )) && echo 'normal'
```
Bash's `((…))` and `[[ … ]]` are builtins — free.
### R4. Prefer event-driven / long-lived producers over polling + sleep
When an update needs to happen often, replace "poll + sleep + exit" with one of:
- **i3blocks `interval=persist`**: script runs forever, prints one block per update. Block on an event stream with `read` — no sleep, no busy-wait.
- **`pactl subscribe`**: event stream for PulseAudio/PipeWire volume/mute changes.
- **`udevadm monitor`**: hardware / power-supply / backlight events.
- **`inotifywait -m`**: file/dir changes.
- **`dbus-monitor`**: session-wide events (network, media keys, NetworkManager).
- **`journalctl -f`**: new log lines.
- **`nvidia-smi --loop=N`** / **`nvidia-smi dmon -d N`**: one long-lived nvidia-smi emitting rows instead of forking every N seconds. Tail its stdout with `while read`.
- **`mpstat N`**, **`iostat N`**, **`vmstat N`**: same pattern for CPU/IO.
Canonical persist skeleton:
```bash
#!/bin/bash
set -u
emit() { printf '%s\n' "$1"; }
emit "$(initial_value)"
producer_command | while read -r line; do
# `read` blocks on I/O — no CPU, no sleep, no poll.
[[ $line matches relevant event ]] || continue
emit "$(compute_new_value)"
done
```
### R5. One-shot scripts must still be cheap
Even with `interval=5`, 1728 invocations/day × 3 forks = 5k forks/day. Make the single-invocation path fork-free when possible. Profile with:
```bash
strace -f -e trace=%process -c ./myscript.sh
```
The `clone` / `execve` counts are your fork count.
### R6. Python called from a hot loop is an anti-pattern
CPython startup is ~50–80 ms on modern hardware. Invoking `python my_helper.py` once per second = ~5–8% of one core doing nothing but importing stdlib.
If a status-bar value needs Python logic:
- **Inline it in bash** when possible (the rules above almost always suffice).
- **Run a persistent Python daemon** that writes to a FIFO / Unix socket / tmpfile; the bash hot-path reads from it with `read` / `$(<file)`.
- **Use a compiled helper** (Go/Rust/C) if Python startup is the only issue — a static binary startup is sub-millisecond.
### R7. Cap risk with a systemd slice
Even a correct script can regress. Put status-bar / monitoring work in a resource-capped user slice so the blast radius is bounded:
```ini
# ~/.config/systemd/user/monitors.slice
[Slice]
CPUQuota=50%
MemoryMax=512M
MemorySwapMax=0 # REQUIRED on zram systems — see oom-prevention skill
TasksMax=256
```
Launch i3blocks (or individual persist scripts) under that slice, e.g. via a user service with `Slice=monitors.slice`, so every child inherits the cap.
### R8. Measure before and after
For any "fast" shell script, time 10k invocations:
```bash
time for _ in {1..10000}; do ./script.sh >/dev/null; done
```
Target: a 1-Hz script should take < 2 ms per invocation on a modern desktop.
A 5-second-interval script can afford ~20 ms.
If you're over budget, count the `execve` with `strace -c` and remove forks.
### R9. Never flush+rebuild firewall rules unconditionally in a poll loop
`iptables -F chain` followed by dozens of `iptables -A` calls every tick is
one of the most expensive fork patterns possible — each rule insert is its
own fork, plus it contends the xtables lock with the kernel's own netfilter
subsystem (netd on Android, firewalld/nftables on Linux). This is a specific
enough, and costly enough, instance of R6-style unconditional recomputation
to call out on its own: a real incident (`phone_focus_mode/dns_enforcer.sh`,
see `feedback-iptables-hash-gate-before-rebuild` in this user's memory)
rebuilt ~100 rules every 20s with no state check at all, and measured out
to pegging `netd` at a sustained ~50% CPU, contributing to a phone
overheating.
Fix: check whether the current state is already correct before touching
iptables at all —
```bash
# Cheap check (2-3 forks): is the chain already right?
chain_intact() {
"$1" -C OUTPUT -j "$2" >/dev/null 2>&1 || return 1
[ "$("$1" -S "$2" 2>/dev/null | grep -c '^-A')" = "$3" ]
}
# Only pay for the expensive flush+rebuild (50-100+ forks) when it's wrong.
chain_intact iptables MY_CHAIN "$expected_count" || rebuild_my_chain
```
This preserves self-healing (a flush by the kernel or another process is
still detected and repaired on the next tick) while skipping the ~95% of
ticks where nothing changed. If diagnosing whether this pattern is already
live in a running system, `kill -STOP`/`kill -CONT` the suspect script's PID
and compare `/proc/<pid>/stat` utime+stime jiffy deltas of the netfilter
daemon before/after — don't trust `dumpsys cpuinfo` (Android) or similar
cached snapshot tools, some builds don't refresh them between calls.
## Python-specific rules (for daemons, not hot-loop callees)
- Use `pathlib.Path.read_text()` / `read_bytes()` — one syscall, no subprocess.
- Open `/sys` / `/proc` files with the builtin `open()`; they're tiny reads.
- For event loops, use `asyncio` / `selectors` to block on fds (same idea as `read` in bash) instead of `time.sleep()` in a polling loop.
- Don't shell out with `subprocess.run("sensors")` when `/sys/class/hwmon` exists.
- Cache `psutil` objects across ticks — `psutil.cpu_percent(interval=None)` uses deltas and is O(1) after the first call.
## Common red flags (search for these in review)
- `while true` / `while :` with a `sleep` and no event source
- `$(…|…|…)` chains with three or more pipes in a status-bar script
- `| awk`, `| grep`, `| tr`, `| cut`, `| sed`, `| head`, `| tail` where bash builtins would do
- `$(cat foo)` anywhere — always replaceable with `$(<foo)`
- `echo … | bc` — replaceable with bash integer math
- `sensors`, `acpi`, `free`, `lspci`, `iwgetid` in a per-second script
- `python …` / `node …` invoked per tick
- No `set -u` (silent typo bugs compound over thousands of ticks)
- `iptables -F` / `ip6tables -F` followed by unconditional `-A` rebuilds with
no preceding "is this already correct?" check (see R9)
## Verification checklist before shipping
1. `shellcheck script.sh` — clean.
2. `strace -c -f script.sh 2>&1 | grep -E 'execve|clone'` — fork count matches expectation.
3. `time for _ in {1..10000}; do script.sh >/dev/null; done` — under budget.
4. For persist scripts: run for 60 s under `perf stat -p $PID` — CPU time near zero when idle.
5. Running under the `monitors.slice` unit — verify with `systemctl --user status monitors.slice`.
## Reference implementations in this repo
- `linux_configuration/i3-configuration/i3blocks/volume.sh` — persist mode with `pactl subscribe`.
- `linux_configuration/i3-configuration/i3blocks/gpu_monitor.sh` — persist mode with `nvidia-smi --loop`.
- `linux_configuration/i3-configuration/i3blocks/battery_status.sh` — zero-fork via `/sys/class/power_supply`.
- `linux_configuration/i3-configuration/i3blocks/cpu_monitor.sh` — zero-fork via `/proc/loadavg` + `/sys/class/hwmon`.
- `linux_configuration/i3-configuration/i3blocks/motherboard_temp.sh` — zero-fork via `/sys/class/hwmon`.
- `linux_configuration/scripts/system-maintenance/systemd/monitors.slice` — resource-cap slice.
- `phone_focus_mode/dns_enforcer.sh::chain_intact()`/`expected_rule_count()` — R9's hash/count-gate pattern, fixed 2026-07-03 after it was found pegging Android's `netd` at ~50% CPU.
- `phone_focus_mode/hosts_enforcer.sh::sync_magisk_module` — the original hash-before-write pattern R9 mirrors.