mirror of
https://github.com/kuhyx/testsAndMisc.git
synced 2026-11-10 20:27:02 +01:00
Run Arch-only shell tests in a container; harden worktree stubbing
Adds an arch-tests job on archlinux:base-devel so the pacman wrapper, makepkg wrapper and hosts-guard tests finally run somewhere. They need an Arch userland and root, which a disposable container provides. Four more tests still referenced pre-reorganisation paths and could never find their targets; repairing them took the container set from two passing to five. All five were verified with --network=none, so the job installs nothing beyond git (base-devel ships no git, and actions/checkout needs it). Also removes the silent-failure mode behind the earlier music_parallelism confusion. The test now derives its temp-worktree path from the script's real position instead of hardcoding the depth, checks up front that ../../lib/common.sh resolves to the stub, and has the stub write a marker asserted after every run. Previously a reorganisation made the script source the real /usr/local/lib/common.sh, so the test measured the live system and failed as though the daemon were broken. Three tests stay out, each for a different reason, documented in the workflow: test_i3blocks_efficiency.sh needs strace and could not be verified locally (containers here have no DNS); test_hosts_guard_pacman_integration.sh fails a real assertion worth investigating; test_security_hardening.sh asserts live host state and can never pass in a fresh container.
This commit is contained in:
parent
e7c3300e80
commit
896d7b1d15
55
.github/workflows/shell-tests.yml
vendored
55
.github/workflows/shell-tests.yml
vendored
@ -47,3 +47,58 @@ jobs:
|
||||
printf '\n%d shell test(s) failed\n' "$failed"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Tests that assume Arch: they exercise the pacman wrapper, makepkg wrapper
|
||||
# and hosts-guard hooks, so they need an Arch userland rather than Ubuntu.
|
||||
# They run as root here, which is what lets them touch /etc safely — the
|
||||
# container is disposable.
|
||||
arch-tests:
|
||||
runs-on: ubuntu-latest
|
||||
container: archlinux:base-devel
|
||||
steps:
|
||||
# base-devel ships no git, and actions/checkout needs it to clone.
|
||||
- name: Install git
|
||||
run: pacman -Sy --noconfirm --needed git
|
||||
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
# Verified to pass in `docker run --network=none archlinux:base-devel`,
|
||||
# so this list needs no package installs beyond git.
|
||||
#
|
||||
# Deliberately excluded:
|
||||
# test_i3blocks_efficiency.sh
|
||||
# needs strace (and SYS_PTRACE) to count execve calls. Could not be
|
||||
# validated locally because this machine's containers have no DNS,
|
||||
# so pacman cannot install strace. Add it once someone can confirm
|
||||
# it passes in a container that can reach the mirrors.
|
||||
# test_hosts_guard_pacman_integration.sh
|
||||
# fails on a real assertion ('wrapper detects when pacman hooks
|
||||
# already manage hosts guard'), not a path or environment problem.
|
||||
# test_security_hardening.sh
|
||||
# asserts live machine state (e.g. '/etc/hosts is immutable'). It
|
||||
# validates a configured host, not the repo, so it can never pass
|
||||
# in a fresh container.
|
||||
- name: Run Arch-specific shell tests
|
||||
run: |
|
||||
set -euo pipefail
|
||||
tests=(
|
||||
test_hosts_file_monitor.sh
|
||||
test_makepkg_capped.sh
|
||||
test_makepkg_wrapper.sh
|
||||
test_pacman_wrapper_security.sh
|
||||
test_shutdown_timer_monitor.sh
|
||||
)
|
||||
failed=0
|
||||
for t in "${tests[@]}"; do
|
||||
printf '\n=== %s ===\n' "$t"
|
||||
if bash "linux_configuration/tests/$t"; then
|
||||
printf 'PASS %s\n' "$t"
|
||||
else
|
||||
printf 'FAIL %s\n' "$t"
|
||||
failed=$((failed + 1))
|
||||
fi
|
||||
done
|
||||
if [ "$failed" -gt 0 ]; then
|
||||
printf '\n%d shell test(s) failed\n' "$failed"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
@ -0,0 +1,20 @@
|
||||
{
|
||||
"title": "Run Arch-only shell tests in CI and harden worktree stubbing",
|
||||
"objective": "The Arch-dependent shell tests (pacman wrapper, makepkg wrapper, hosts-guard hooks) have never run anywhere, so they rot unnoticed. Give them an archlinux:base-devel container job, repair the ones still pointing at pre-reorganisation paths, and remove the failure mode where test_music_parallelism.sh could silently source the real installed common.sh instead of its stub — which previously made a healthy daemon look broken. Success means CI runs every shell test that can meaningfully run, and any future path drift fails loudly with an explanatory message instead of masquerading as a behavioural bug.",
|
||||
"acceptance_criteria": [
|
||||
"A new arch-tests job runs test_hosts_file_monitor.sh, test_makepkg_capped.sh, test_makepkg_wrapper.sh, test_pacman_wrapper_security.sh and test_shutdown_timer_monitor.sh in archlinux:base-devel, and all five pass.",
|
||||
"Those five pass with --network=none locally, proving the job needs no package installs beyond git.",
|
||||
"test_music_parallelism.sh derives its temp-worktree path from the script's real location, so a future reorganisation cannot silently change the depth.",
|
||||
"test_music_parallelism.sh fails with an explanatory message, not a wrong number, if the stub is unreachable: a pre-flight resolution check plus a marker the stub writes and every run asserts.",
|
||||
"The existing ubuntu shell-tests job (6 tests + 30 bats assertions) still passes unchanged.",
|
||||
"Every excluded test carries a comment in the workflow naming the specific reason it is excluded."
|
||||
],
|
||||
"out_of_scope": [
|
||||
"test_i3blocks_efficiency.sh — needs strace and SYS_PTRACE; could not be validated locally because containers on this machine have no DNS, so it is excluded rather than shipped unverified.",
|
||||
"test_hosts_guard_pacman_integration.sh — after the path repair it fails a genuine assertion about pacman-hook detection; that is a real finding to investigate, not CI plumbing.",
|
||||
"test_security_hardening.sh — asserts live host state such as an immutable /etc/hosts, so it can never pass in a fresh container.",
|
||||
"Any change to the scripts under test. Every failure addressed here was in the tests or the workflow.",
|
||||
"Fixing this machine's container DNS, which is what blocked validating the strace-dependent test."
|
||||
],
|
||||
"verifier": "docker run --rm --network=none -v $PWD:/repo:ro -w /repo archlinux:base-devel bash -c '<five tests>' for the container set; bats linux_configuration/tests/*.bats plus the six ubuntu-set tests on the host; pre-commit run --all-files and the pre-push ci-mirror gate; finally the Shell tests workflow itself on GitHub Actions."
|
||||
}
|
||||
@ -0,0 +1,43 @@
|
||||
{
|
||||
"intent": "Bring the Arch-only shell tests into CI via a container job, and make test_music_parallelism.sh incapable of silently testing the wrong library after a future directory reorganisation.",
|
||||
"scope": [
|
||||
".github/workflows/shell-tests.yml: new arch-tests job running archlinux:base-devel",
|
||||
"linux_configuration/tests/test_music_parallelism.sh: derive the worktree path and assert the stub was sourced",
|
||||
"linux_configuration/tests/{test_hosts_file_monitor,test_hosts_guard_pacman_integration,test_makepkg_capped,test_shutdown_timer_monitor}.sh: stale path repairs",
|
||||
"Non-goal: three tests that cannot pass in a fresh container — see risks"
|
||||
],
|
||||
"changes": [
|
||||
"test_music_parallelism.sh now derives its temp-worktree path from TARGET_SCRIPT's position relative to REPO_DIR, so the copy automatically mirrors the real directory depth. Previously the depth was hardcoded and a reorganisation broke it silently.",
|
||||
"Added two guards to the same test: a pre-flight check that ../../lib/common.sh from the copied script actually resolves to the stub, and a marker file the stub writes when sourced, asserted after every run. Either fires loudly instead of letting the script fall back to the real /usr/local/lib/common.sh and report a misleading behavioural failure.",
|
||||
"Four more tests referenced pre-reorganisation paths (missing the periodic_background/ segment) and now resolve: hosts-file-monitor, hosts-guard pacman hooks, makepkg_capped, shutdown-timer-monitor.",
|
||||
"New arch-tests job runs five Arch-dependent tests in archlinux:base-devel. git is installed first because base-devel does not ship it and actions/checkout needs it."
|
||||
],
|
||||
"verification": [
|
||||
{
|
||||
"command": "docker run --rm --network=none -v $PWD:/repo:ro -w /repo archlinux:base-devel <5 tests>",
|
||||
"result": "pass",
|
||||
"evidence": "All five PASS with no network, so the job needs no package installs beyond git. Before the path repairs only two of these passed (test_makepkg_wrapper.sh, test_pacman_wrapper_security.sh)."
|
||||
},
|
||||
{
|
||||
"command": "bash linux_configuration/tests/<6 ubuntu-set tests> and bats linux_configuration/tests/*.bats",
|
||||
"result": "pass",
|
||||
"evidence": "0 failures and 30/30 bats — the ubuntu job is unaffected by these edits."
|
||||
},
|
||||
{
|
||||
"command": "drift-guard logic checked against three worktree layouts",
|
||||
"result": "pass",
|
||||
"evidence": "Correct depth reaches the stub; the old shallow depth that caused the original bug fires the guard; a hypothetical three-level layout also fires it."
|
||||
}
|
||||
],
|
||||
"risks": [
|
||||
"test_i3blocks_efficiency.sh is excluded: it needs strace plus SYS_PTRACE to count execve calls. This could NOT be validated locally because containers on this machine have no DNS (pacman -Sy fails to resolve mirrors), so strace could not be installed. It may well pass on a GitHub runner, but shipping it unverified risked a red job, so it is left out with a comment.",
|
||||
"test_hosts_guard_pacman_integration.sh is excluded: after the path repair it fails on a genuine assertion ('wrapper detects when pacman hooks already manage hosts guard'), which is a real finding needing investigation rather than a CI plumbing problem.",
|
||||
"test_security_hardening.sh is excluded permanently: it asserts live machine state such as '/etc/hosts is immutable'. It validates a configured host, not the repository, so it can never pass in a fresh container.",
|
||||
"The arch-tests job runs as root in a disposable container, which is what makes the pacman and /etc tests safe. Anything added to that list inherits root — worth checking a new test cannot reach outside the container before adding it.",
|
||||
"The offline validation used --network=none; the real job has network, so a test that quietly depends on the network would pass locally and could behave differently in CI."
|
||||
],
|
||||
"rollback": [
|
||||
"git revert this commit: the arch-tests job disappears, the four path repairs revert, and test_music_parallelism.sh returns to hardcoded worktree paths without the stub guards.",
|
||||
"Validate after rollback: the four reverted tests fail with 'cannot stat'/'No such file' again, and .github/workflows/shell-tests.yml contains only the shell-tests job."
|
||||
]
|
||||
}
|
||||
@ -5,7 +5,7 @@ set -euo pipefail
|
||||
|
||||
SCRIPT_DIR=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)
|
||||
REPO_DIR=$(cd -- "$SCRIPT_DIR/.." && pwd)
|
||||
TARGET_SCRIPT="$REPO_DIR/scripts/system-maintenance/bin/hosts-file-monitor.sh"
|
||||
TARGET_SCRIPT="$REPO_DIR/scripts/periodic_background/system-maintenance/bin/hosts-file-monitor.sh"
|
||||
|
||||
fail() {
|
||||
printf 'FAIL: %s\n' "$1" >&2
|
||||
|
||||
@ -6,10 +6,10 @@ set -euo pipefail
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
REPO_DIR="$(dirname "$SCRIPT_DIR")"
|
||||
WRAPPER_FILE="$REPO_DIR/scripts/periodic_background/digital_wellbeing/pacman/pacman_wrapper.sh"
|
||||
PRE_HOOK_FILE="$REPO_DIR/hosts/guard/pacman-hooks/pacman-pre-unlock-hosts.sh"
|
||||
POST_HOOK_FILE="$REPO_DIR/hosts/guard/pacman-hooks/pacman-post-relock-hosts.sh"
|
||||
COMMON_FILE="$REPO_DIR/hosts/guard/pacman-hooks/hosts-guard-common.sh"
|
||||
INSTALLER_FILE="$REPO_DIR/hosts/guard/install_pacman_hooks.sh"
|
||||
PRE_HOOK_FILE="$REPO_DIR/scripts/periodic_background/hosts/guard/pacman-hooks/pacman-pre-unlock-hosts.sh"
|
||||
POST_HOOK_FILE="$REPO_DIR/scripts/periodic_background/hosts/guard/pacman-hooks/pacman-post-relock-hosts.sh"
|
||||
COMMON_FILE="$REPO_DIR/scripts/periodic_background/hosts/guard/pacman-hooks/hosts-guard-common.sh"
|
||||
INSTALLER_FILE="$REPO_DIR/scripts/periodic_background/hosts/guard/install_pacman_hooks.sh"
|
||||
|
||||
assert_contains() {
|
||||
local file_path="$1"
|
||||
|
||||
@ -5,7 +5,7 @@ set -euo pipefail
|
||||
|
||||
SCRIPT_DIR=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)
|
||||
REPO_DIR=$(cd -- "$SCRIPT_DIR/.." && pwd)
|
||||
WRAPPER="$REPO_DIR/scripts/digital_wellbeing/pacman/makepkg_capped.sh"
|
||||
WRAPPER="$REPO_DIR/scripts/periodic_background/digital_wellbeing/pacman/makepkg_capped.sh"
|
||||
|
||||
fail() {
|
||||
printf 'FAIL: %s\n' "$1" >&2
|
||||
|
||||
@ -29,12 +29,41 @@ trap cleanup EXIT
|
||||
|
||||
WORKTREE="$TMP_DIR/worktree"
|
||||
BIN_DIR="$TMP_DIR/bin"
|
||||
mkdir -p "$WORKTREE/scripts/periodic_background/digital_wellbeing" "$WORKTREE/scripts/lib" "$BIN_DIR"
|
||||
cp "$TARGET_SCRIPT" "$WORKTREE/scripts/periodic_background/digital_wellbeing/music_parallelism.sh"
|
||||
|
||||
cat >"$WORKTREE/scripts/lib/common.sh" <<'EOF'
|
||||
# Mirror the script's real position inside the worktree instead of hardcoding
|
||||
# it. music_parallelism.sh sources its library as $SCRIPT_DIR/../../lib/common.sh,
|
||||
# so the copy must sit at the same depth below the repo root or that path
|
||||
# resolves outside the worktree. When it did, the script silently fell back to
|
||||
# the real /usr/local/lib/common.sh and the stub below was never used — the
|
||||
# tests then measured the live system and failed as if the daemon were wrong.
|
||||
# Deriving the path keeps this correct across future reorganisations.
|
||||
TARGET_REL="${TARGET_SCRIPT#"$REPO_DIR"/}"
|
||||
if [[ "$TARGET_REL" == "$TARGET_SCRIPT" ]]; then
|
||||
fail "TARGET_SCRIPT ($TARGET_SCRIPT) is not below REPO_DIR ($REPO_DIR)"
|
||||
fi
|
||||
WORKTREE_SCRIPT="$WORKTREE/$TARGET_REL"
|
||||
# The library lives at <repo>/scripts/lib/common.sh; keep it relative to the
|
||||
# same root rather than assuming how deep the script itself is nested.
|
||||
WORKTREE_LIB="$WORKTREE/scripts/lib/common.sh"
|
||||
|
||||
mkdir -p "$(dirname "$WORKTREE_SCRIPT")" "$(dirname "$WORKTREE_LIB")" "$BIN_DIR"
|
||||
cp "$TARGET_SCRIPT" "$WORKTREE_SCRIPT"
|
||||
|
||||
# Fail loudly if the copy cannot reach the stub, rather than letting the script
|
||||
# fall through to the real library and produce a misleading behavioural failure.
|
||||
STUB_FROM_SCRIPT="$(dirname "$WORKTREE_SCRIPT")/../../lib/common.sh"
|
||||
if [[ "$(realpath -m "$STUB_FROM_SCRIPT")" != "$(realpath -m "$WORKTREE_LIB")" ]]; then
|
||||
fail "worktree layout drift: script at $WORKTREE_SCRIPT resolves ../../lib/common.sh to $(realpath -m "$STUB_FROM_SCRIPT"), not the stub at $WORKTREE_LIB"
|
||||
fi
|
||||
|
||||
cat >"$WORKTREE_LIB" <<'EOF'
|
||||
#!/bin/bash
|
||||
|
||||
# Records that the stub — not the real installed common.sh — was sourced.
|
||||
if [[ -n ${MUSIC_PARALLELISM_STUB_MARKER:-} ]]; then
|
||||
printf 'sourced\n' >"$MUSIC_PARALLELISM_STUB_MARKER"
|
||||
fi
|
||||
|
||||
FOCUS_APPS_WINDOWS=("Mock Focus App")
|
||||
FOCUS_APPS_PROCESSES=("mock-focus-proc")
|
||||
|
||||
@ -55,7 +84,7 @@ log_message() {
|
||||
:
|
||||
}
|
||||
EOF
|
||||
chmod +x "$WORKTREE/scripts/lib/common.sh"
|
||||
chmod +x "$WORKTREE_LIB"
|
||||
|
||||
create_fake_proc_process() {
|
||||
local proc_root="$1"
|
||||
@ -72,8 +101,10 @@ run_case() {
|
||||
local mode="${4:-instant}"
|
||||
local wait_log="$TMP_DIR/wait.log"
|
||||
local proc_root="$TMP_DIR/proc"
|
||||
local stub_marker="$TMP_DIR/stub.marker"
|
||||
|
||||
: >"$wait_log"
|
||||
rm -f "$stub_marker"
|
||||
rm -rf "$proc_root"
|
||||
mkdir -p "$proc_root"
|
||||
|
||||
@ -84,12 +115,18 @@ run_case() {
|
||||
PATH="$BIN_DIR:$PATH" \
|
||||
MUSIC_PARALLELISM_TEST_WAIT_LOG="$wait_log" \
|
||||
MUSIC_PARALLELISM_TEST_EXIT_AFTER_WAIT=1 \
|
||||
MUSIC_PARALLELISM_STUB_MARKER="$stub_marker" \
|
||||
XDOTOOL_LOG="${XDOTOOL_LOG:-}" \
|
||||
PROC_ROOT="$proc_root" \
|
||||
MOCK_FOCUS_ACTIVE="$focus_active" \
|
||||
bash "$WORKTREE/scripts/periodic_background/digital_wellbeing/music_parallelism.sh" "$mode" \
|
||||
bash "$WORKTREE_SCRIPT" "$mode" \
|
||||
>/dev/null 2>&1 || true
|
||||
|
||||
# Without this the script can source the real /usr/local/lib/common.sh and
|
||||
# still "pass" or fail for reasons that have nothing to do with the daemon.
|
||||
[[ -f "$stub_marker" ]] \
|
||||
|| fail 'stubbed common.sh was not sourced — the script used a different library, so this run measured the live system rather than the mock'
|
||||
|
||||
assert_equals "$expected_wait" "$(<"$wait_log")" "music_parallelism.sh should pick the expected wait interval"
|
||||
}
|
||||
|
||||
|
||||
@ -5,8 +5,8 @@ set -euo pipefail
|
||||
|
||||
SCRIPT_DIR=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)
|
||||
REPO_DIR=$(cd -- "$SCRIPT_DIR/.." && pwd)
|
||||
TARGET_SCRIPT="$REPO_DIR/scripts/system-maintenance/bin/shutdown-timer-monitor.sh"
|
||||
SETUP_SCRIPT="$REPO_DIR/scripts/digital_wellbeing/setup_midnight_shutdown.sh"
|
||||
TARGET_SCRIPT="$REPO_DIR/scripts/periodic_background/system-maintenance/bin/shutdown-timer-monitor.sh"
|
||||
SETUP_SCRIPT="$REPO_DIR/scripts/periodic_background/digital_wellbeing/setup_midnight_shutdown.sh"
|
||||
|
||||
fail() {
|
||||
printf 'FAIL: %s\n' "$1" >&2
|
||||
|
||||
Loading…
Reference in New Issue
Block a user