signal-bot/scripts/e2e_proofs.sh
Krzysztof kuhy Rudnicki 72e90acb47
Some checks are pending
CI / check (push) Waiting to run
CI / web (push) Waiting to run
Let the operator add aliases from the Commands tab
A custom alias is data the command table is built from, so it cannot be a
live switch like the ones already on the tab. Instead the management page
writes aliases_custom.json in the bot's state directory and then asks the
bot to reload, in one step: whenever the answer is anything but `reloaded`
-- a word another command has, a name no command has, a bot still
connecting -- the file is restored byte for byte and the bot's own reason is
shown next to the field. What is on disk is therefore always what the bot
serves, and a typo cannot sit in the file poisoning every later reload.

The registry appends the operator's words to the shipped list at
registration, which keeps `!pomoc` on the shipped spelling and routes the
new words through the same clash check as aliases.json. An alias for a name
no command has fails the whole build (ConfigError -> `kept`), because a
silently ignored entry is how a file drifts. The tagger treats custom
aliases as reserved words too, so a person cannot be given one.

registry.py crossed the 250-line cap and now re-exports the record and the
error types from command.py; the bot's state directory lookup, previously
duplicated in pack.py and the suggestions export, is state_dir.py.

Proven end to end: !e2ekot is unknown, the file is written and reload.sh
runs, !e2ekot posts a cat, a file naming a command that does not exist is
refused with 409 and the previous table still answers.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-04 16:20:05 +02:00

135 lines
6.2 KiB
Bash
Executable File

#!/bin/bash
# ============================================================================
# The assertions scripts/e2e.sh makes after the scenarios have replayed.
#
# Split out of e2e.sh at the 250-line cap. A filing change, not an interface
# change: these are sourced, so they still run against the same fake server
# and the same bot process the scenarios used.
# ============================================================================
# The gate that did not exist on 2026-08-29, proven against the real process:
# with signal-cli unable to reach Signal, /healthz must go red rather than
# keep answering 200 while nothing arrives.
prove_the_deafness_gate() {
local health="http://127.0.0.1:${BOT_PORT}/healthz"
echo "== cutting the fake's connection to Signal"
curl -sf -X POST "${FAKE_URL}/_control/connection" \
-H 'Content-Type: application/json' -d '{"alive": false}' >/dev/null
local code="" waited=0
while ((waited < 150)); do
code="$(curl -s -o /dev/null -w '%{http_code}' "$health")"
[[ "$code" == "503" ]] && break
sleep 1
waited=$((waited + 1))
done
[[ "$code" == "503" ]] || die "/healthz still answers ${code} with the connection cut"
curl -s "$health" | grep -q "Closed unexpectedly" ||
die "/healthz went red without saying why"
echo "== /healthz turned 503 and named the reason after ${waited}s"
echo "== restoring the connection"
curl -sf -X POST "${FAKE_URL}/_control/connection" \
-H 'Content-Type: application/json' -d '{"alive": true}' >/dev/null
waited=0
while ((waited < 150)); do
code="$(curl -s -o /dev/null -w '%{http_code}' "$health")"
[[ "$code" == "200" ]] && break
sleep 1
waited=$((waited + 1))
done
[[ "$code" == "200" ]] || die "/healthz stayed ${code} after the connection came back"
echo "== /healthz recovered on its own after ${waited}s"
}
# A person tagged while the bot is up becomes a command after POST /_reload,
# with no restart: the same process that just answered "Nie znam !delta"
# posts delta's photo a moment later. The scenarios live next to the main
# set but outside its glob, because the second one is only true after the
# reload that this proof performs between them.
prove_the_table_reloads_without_a_restart() {
local proofs="${REPO_ROOT}/fake_signal/proofs"
echo "== before the reload, !delta is nobody"
replay "${proofs}/reload_before.yaml"
echo "== tagging a fifth person into the pack, then scripts/reload.sh"
"$PYTHON" "$REPO_ROOT/tool/e2e_pack.py" \
--dir "${TEMP_DIR}/pack" --group "$WATCHED_GROUP" --extra delta >/dev/null
# The operator's script, not a bare curl: this is the one place it runs
# against a real process in CI, so its exit code is the proof.
BIND_PORT="$BOT_PORT" "$REPO_ROOT/scripts/reload.sh" ||
die "scripts/reload.sh did not get the new table live"
echo "== after the reload, !delta posts a photo"
replay "${proofs}/reload_after.yaml"
}
# A switch thrown from the management page is read on the very next attempt:
# no reload, no restart. The PUT here is what the page's Save button sends,
# aimed at the bot directly; the two scenarios bracket it the same way the
# reload pair does. Not a timed switch, because a proof that depends on the
# clock has to wait on it, and the unit tests already cover the deadline.
prove_a_switch_is_live_without_a_reload() {
local proofs="${REPO_ROOT}/fake_signal/proofs"
local switch="http://127.0.0.1:${BOT_PORT}/_commands/cat"
echo "== switching !kot off"
curl -sf -X PUT "$switch" -H 'Content-Type: application/json' \
-d '{"enabled": false}' | grep -q '"enabled":false' ||
die "PUT /_commands/cat did not report the command off"
replay "${proofs}/switch_off.yaml"
echo "== switching !kot back on"
curl -sf -X PUT "$switch" -H 'Content-Type: application/json' \
-d '{"enabled": true}' | grep -q '"enabled":true' ||
die "PUT /_commands/cat did not report the command on"
replay "${proofs}/switch_on.yaml"
}
# The operator's alias file is data the table is built from, so it takes
# the same path as a tagged person: written, then scripts/reload.sh. The
# file is the one CUSTOM_ALIASES_FILE points the bot at, never the
# operator's own; a name the table does not hold is what the second half
# proves the bot refuses, with the old table still answering.
prove_an_alias_is_live_after_a_reload() {
local proofs="${REPO_ROOT}/fake_signal/proofs"
local file="${TEMP_DIR}/aliases_custom.json"
echo "== before the alias, !e2ekot is nobody"
replay "${proofs}/alias_before.yaml"
echo "== writing the operator's alias file, then scripts/reload.sh"
printf '{"cat": ["!e2ekot"]}\n' > "$file"
BIND_PORT="$BOT_PORT" "$REPO_ROOT/scripts/reload.sh" ||
die "scripts/reload.sh did not get the alias live"
replay "${proofs}/alias_after.yaml"
echo "== an alias for a command that does not exist keeps the old table"
printf '{"cat": ["!e2ekot"], "nosuch": ["!nic"]}\n' > "$file"
local answer
answer="$(curl -s -w '%{http_code}' -X POST "http://127.0.0.1:${BOT_PORT}/_reload")"
[[ "$answer" == *nosuch*409 ]] ||
die "a bad alias file was not kept out with 409 and its name: ${answer}"
replay "${proofs}/alias_kept.yaml"
printf '{"cat": ["!e2ekot"]}\n' > "$file"
}
# A budget, not a benchmark. This run was four minutes until the liveness
# tick became configurable, and nothing announced the cost -- slower is the
# one regression a green suite never reports. Exceeding it fails the run;
# whoever exceeded it decides whether the check earned its seconds or the
# budget moves. E2E_BUDGET_SECONDS raises it for a loaded machine.
enforce_the_time_budget() {
local started="$1"
local budget="${E2E_BUDGET_SECONDS:-40}"
local elapsed=$(( $(date +%s) - started ))
echo "== end to end took ${elapsed}s (budget ${budget}s)"
if ((elapsed > budget)); then
echo "Error: the offline run took ${elapsed}s, over its ${budget}s budget." >&2
echo " Either the new check earns its seconds and the budget moves," >&2
echo " or something got slower and this is the report of it." >&2
exit 1
fi
}