Run all 14 shell tests in CI
Some checks are pending
Pre-commit checks / pre-commit (push) Waiting to run
Python tests / test (push) Waiting to run
Shell tests / shell-tests (push) Waiting to run
Shell tests / arch-tests (push) Waiting to run

The last three exclusions are gone.

test_i3blocks_efficiency.sh only ever needed strace and ptrace. It went
unverified earlier because Docker's bridge network has no DNS on this
machine, so pacman could not install strace; --network=host works, and
the test passes. The job installs strace and adds SYS_PTRACE.

test_hosts_guard_pacman_integration.sh asserted three symbols the
guard-lib refactor renamed (pacman_hooks_manage_hosts_guard ->
pacman_hooks_manage_guard_lib, and similar). The feature was intact;
only the assertions were stale.

test_security_hardening.sh now separates live-host checks from
repository checks. Off-host the live ones skip, so it is green in a
container while still running its repository checks — and on the real
host it still fails loudly when a guard unit is down.

That last part needed care: the first detection attempt used
`systemctl is-system-running`, which exits non-zero on a degraded
system — precisely the state a failed guard unit produces. It masked
all four real failures. Detection now tests for /run/systemd/system
plus the installed guard payload, which unit health does not affect.

Verified: all 8 arch tests pass in a container; the same
security-hardening file still reports 4 failures on this host.
This commit is contained in:
Krzysztof Rudnicki 2026-07-19 19:39:28 +02:00
parent 896d7b1d15
commit ee740bf92e
4 changed files with 95 additions and 23 deletions

View File

@ -54,38 +54,37 @@ jobs:
# container is disposable.
arch-tests:
runs-on: ubuntu-latest
container: archlinux:base-devel
container:
image: archlinux:base-devel
# test_i3blocks_efficiency.sh counts execve calls with strace, which needs
# ptrace. Without these the trace file is never produced.
options: --cap-add=SYS_PTRACE --security-opt seccomp=unconfined
steps:
# base-devel ships no git, and actions/checkout needs it to clone.
- name: Install git
run: pacman -Sy --noconfirm --needed git
# base-devel ships neither git (actions/checkout needs it) nor strace.
- name: Install dependencies
run: pacman -Sy --noconfirm --needed git strace
- uses: actions/checkout@v4
# Verified to pass in `docker run --network=none archlinux:base-devel`,
# so this list needs no package installs beyond git.
# The complete Arch-dependent set. All eight verified locally in
# `docker run --cap-add=SYS_PTRACE archlinux:base-devel`.
#
# Deliberately excluded:
# test_i3blocks_efficiency.sh
# needs strace (and SYS_PTRACE) to count execve calls. Could not be
# validated locally because this machine's containers have no DNS,
# so pacman cannot install strace. Add it once someone can confirm
# it passes in a container that can reach the mirrors.
# test_hosts_guard_pacman_integration.sh
# fails on a real assertion ('wrapper detects when pacman hooks
# already manage hosts guard'), not a path or environment problem.
# test_security_hardening.sh
# asserts live machine state (e.g. '/etc/hosts is immutable'). It
# validates a configured host, not the repo, so it can never pass
# in a fresh container.
# test_security_hardening.sh inspects a configured machine (chattr flags,
# systemd guard units, /usr/local payloads). It detects that it is not on
# one and reports those checks as skipped, while still running its
# repository checks — so it is meaningful here AND on the real host,
# where it continues to fail loudly if a guard unit is down.
- name: Run Arch-specific shell tests
run: |
set -euo pipefail
tests=(
test_hosts_file_monitor.sh
test_hosts_guard_pacman_integration.sh
test_i3blocks_efficiency.sh
test_makepkg_capped.sh
test_makepkg_wrapper.sh
test_pacman_wrapper_security.sh
test_security_hardening.sh
test_shutdown_timer_monitor.sh
)
failed=0

View File

@ -0,0 +1,41 @@
{
"intent": "Bring the last three excluded shell tests into CI so all 14 run, without weakening what test_security_hardening.sh detects on the real host.",
"scope": [
".github/workflows/shell-tests.yml: arch-tests gains strace, SYS_PTRACE, and the final three tests",
"linux_configuration/tests/test_hosts_guard_pacman_integration.sh: renamed-symbol assertions",
"linux_configuration/tests/test_security_hardening.sh: environment-aware live checks",
"Non-goal: repairing the four guard units found down on the host — reported separately, not silently fixed"
],
"changes": [
"test_i3blocks_efficiency.sh needed strace plus ptrace. Earlier this could not be verified because Docker's bridge network has no DNS on this machine; --network=host works, so it was validated after all. The job installs strace and the container gains --cap-add=SYS_PTRACE.",
"test_hosts_guard_pacman_integration.sh asserted three symbols that the guard-lib refactor renamed: pacman_hooks_manage_hosts_guard -> pacman_hooks_manage_guard_lib, should_use_wrapper_hosts_guard_fallback -> should_use_wrapper_guard_lib_fallback, manual_hosts_guard=1 -> manual_guard_lib_fallback=1. The feature exists under the new names; only the assertions were stale.",
"test_security_hardening.sh now distinguishes live-host checks from repository checks via a LIVE_SECTION flag. Off-host the live ones report skip instead of fail, so the suite is green in a container while its repository checks still run. Two checks outside the contiguous live sections (/usr/bin/pacman symlink, /var/lib/pacman-wrapper/policy.sha256) are individually marked live."
],
"verification": [
{
"command": "docker run --rm --network=host --cap-add=SYS_PTRACE -v $PWD:/repo:ro archlinux:base-devel <all 8 arch tests>",
"result": "pass",
"evidence": "All eight PASS, up from five."
},
{
"command": "bash linux_configuration/tests/test_security_hardening.sh (on the real host)",
"result": "fail",
"evidence": "Correctly still reports 15 passed / 4 failed / 5 skipped, naming hosts-guard.path, hosts-bind-mount.service, nsswitch-guard.path and shutdown-schedule-guard.path as inactive. The container run of the same file exits 0 with 7 passed / 0 failed / 17 skipped."
},
{
"command": "coverage check: every test_*.sh listed in the workflow",
"result": "pass",
"evidence": "14 tests on disk, 14 listed in CI, none unlisted."
}
],
"risks": [
"The first attempt at host detection used `systemctl is-system-running`, which exits non-zero when the system is degraded — exactly what a failed guard unit causes. That version silently downgraded all four real host failures to skips, i.e. it hid the problem the suite exists to find. It now tests for /run/systemd/system plus the installed guard payload, which are unaffected by unit health. Any future change to this detection must be re-checked against a degraded host, not just a healthy one.",
"LIVE_SECTION is positional: a new check added between the markers is treated as live and will be skipped off-host. Anyone adding a repository check inside the live sections must move it or toggle the flag.",
"The arch-tests container runs as root with SYS_PTRACE and seccomp unconfined. That is acceptable for a disposable CI container but is a wider capability set than the other job, so new tests added there should be reviewed for what they touch.",
"test_security_hardening.sh remains a host-health suite by nature; in CI it meaningfully verifies only its 7 repository checks."
],
"rollback": [
"git revert this commit: the arch job drops back to five tests, the renamed assertions revert to the old symbols, and test_security_hardening.sh fails everywhere except a fully configured host.",
"Validate after rollback: the container run of test_security_hardening.sh exits non-zero again, and test_hosts_guard_pacman_integration.sh fails on 'wrapper detects when pacman hooks already manage hosts guard'."
]
}

View File

@ -78,13 +78,13 @@ echo "PASS: shell syntax is valid"
assert_not_regex "$PRE_HOOK_FILE" '(^|[[:space:]])(sudo[[:space:]]+)?rm[[:space:]]+/etc/hosts([[:space:]]|$)' \
"pre-transaction hook must not delete /etc/hosts"
assert_contains "$WRAPPER_FILE" 'pacman_hooks_manage_hosts_guard()' \
assert_contains "$WRAPPER_FILE" 'pacman_hooks_manage_guard_lib()' \
"wrapper detects when pacman hooks already manage hosts guard"
assert_contains "$WRAPPER_FILE" 'should_use_wrapper_hosts_guard_fallback()' \
assert_contains "$WRAPPER_FILE" 'should_use_wrapper_guard_lib_fallback()' \
"wrapper exposes a dedicated fallback path for hosts guard"
assert_order "$WRAPPER_FILE" 'if ! check_and_handle_db_lock "$@"; then' 'if should_use_wrapper_hosts_guard_fallback "$@"; then' \
assert_order "$WRAPPER_FILE" 'if ! check_and_handle_db_lock "$@"; then' 'if should_use_wrapper_guard_lib_fallback "$@"; then' \
"wrapper checks pacman db lock before any manual hosts unlock fallback"
assert_contains "$WRAPPER_FILE" 'manual_hosts_guard=1' \
assert_contains "$WRAPPER_FILE" 'manual_guard_lib_fallback=1' \
"wrapper tracks whether manual hosts guard fallback was used"
assert_contains "$INSTALLER_FILE" 'install -m 755 "$SCRIPT_DIR/pacman-hooks/hosts-guard-common.sh" /usr/local/share/hosts-guard/' \

View File

@ -12,6 +12,24 @@ PASS=0
FAIL=0
SKIP=0
# Roughly half this suite inspects a *configured machine* (chattr flags, systemd
# guard units, /usr/local payloads) rather than the repository. Those checks
# cannot pass in CI or a container, where none of it is installed — but the
# repo-file checks further down can and should still run. So live checks are
# reported as skipped off-host instead of failed, which keeps the suite
# meaningful in both places rather than permanently red in one.
# Detection must not consult `systemctl is-system-running`: it exits non-zero
# when the system is *degraded*, which is exactly the state a failed guard unit
# produces. Using it would make this suite skip the very failures it exists to
# catch. Test instead for systemd being init plus the guard payload being
# installed — both true on a configured host regardless of unit health.
IS_CONFIGURED_HOST=0
if [[ -d /run/systemd/system ]] && [[ -d /usr/local/share/hosts-guard ]]; then
IS_CONFIGURED_HOST=1
fi
# Set to 1 while inside the live-host sections, 0 for repository checks.
LIVE_SECTION=0
# Colors
RED='\033[0;31m'
GREEN='\033[0;32m'
@ -29,6 +47,13 @@ test_result() {
((PASS++))
;;
fail)
if ((LIVE_SECTION == 1)) && ((IS_CONFIGURED_HOST == 0)); then
# Off-host: the mechanism being absent is expected, not a defect.
echo -e "${YELLOW}⏭️ SKIP${NC}: $name"
echo -e " ${YELLOW}Reason: not a configured host (CI/container)${NC}"
((SKIP++))
return
fi
echo -e "${RED}❌ FAIL${NC}: $name"
[[ -n "$reason" ]] && echo -e " ${RED}Reason: $reason${NC}"
((FAIL++))
@ -54,6 +79,7 @@ echo ""
# ==================================================================
# HOSTS GUARD TESTS
# ==================================================================
LIVE_SECTION=1 # live-host checks begin
echo "--- HOSTS GUARD ---"
# Test 1: /etc/hosts is immutable
@ -179,6 +205,8 @@ else
fi
fi
LIVE_SECTION=0 # repository checks from here on
# Test 13: google-chrome is blocked
blocked_file="$REPO_DIR/scripts/periodic_background/digital_wellbeing/pacman/pacman_blocked_keywords.txt"
if [[ -f "$blocked_file" ]]; then
@ -203,11 +231,15 @@ else
fi
# Test 15: Policy integrity file exists
# Lives under /var on the configured host, so it is a live check despite
# sitting among the repository ones.
LIVE_SECTION=1
if [[ -f /var/lib/pacman-wrapper/policy.sha256 ]]; then
test_result "Pacman policy integrity file exists" "pass"
else
test_result "Pacman policy integrity file exists" "fail" "Not found"
fi
LIVE_SECTION=0
# Test 16: LeechBlock auto-install function exists in wrapper
wrapper_file="$REPO_DIR/scripts/periodic_background/digital_wellbeing/pacman/pacman_wrapper.sh"