signal-bot/scripts/e2e.sh
Krzysztof kuhy Rudnicki 08599e6056 feat: two commands that answer with somebody else's work
!sztuka posts a work of art, skewed 60% Polish; !wiersz posts one short
public-domain Polish poem, whole. Both follow the rule the rest of the bot
already keeps: the repository commits identifiers, and the words arrive over
the wire when somebody asks.

!wiersz was written the other way first, and the gate caught it. A pack of
short public-domain poems was harvested, name-filtered and committed, and
check_personal_data.sh refused it: three lines carried a real group member's
given name, and two still did after a filter that had discarded 31% of the
corpus. That is structural, not a tuning problem -- Polish verse capitalises
every line, so a line-initial capital carries no signal, and that is exactly
where a name sits. The pack now holds Wolne Lektury slugs; the poem, its
title and its author are fetched at post time and appear nowhere in git.
It is also the better version: with no name filter there are no false
positives, so the pool is larger and the attribution is a full name again.

The gate then refused the art category list too, over a painter whose given
name is somebody's. That is the popes.json case exactly, so it takes the
same answer: a file-scoped allowlist entry, still refused everywhere else.

The Commons harvest is now three modules by what they are, not by where the
line count fell: commons_api is the conversation with Commons, commons_harvest
turns categories into a pool of licensed page ids, commons_cache makes an
interrupted run resumable. Along the way it gained licences in the same
request that lists a category (the second pass was 1,500 requests at 2.3s
each), maxlag instead of a guessed sleep, 5xx retries because one 504 used to
destroy a whole run, and Wikidata for works by creators of Polish
citizenship: the Polish pool grew 15,221 -> 20,092.

Batching categories through CirrusSearch looked like a further 6x and was
measured out again: 2,404 files where the authoritative enumeration returns
2,429, a different ~1% missing on every run.

The offline run is 19s where it was four minutes, almost all of it the
deafness gate waiting out three thirty-second liveness probes to prove
something provable in three; the tick is configuration now, and e2e enforces
a 40s budget on itself so that never silently comes back.
2026-08-31 21:32:07 +02:00

179 lines
7.1 KiB
Bash
Executable File

#!/bin/bash
# ============================================================================
# End-to-end test: the real bot against the fake Signal server.
#
# Boots the protocol-faithful fake, boots the unmodified bot pointed at it,
# replays every scenario and exits non-zero on the first mismatch. No Signal
# account, no network, no container.
# ============================================================================
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
readonly SCRIPT_DIR
REPO_ROOT="$(dirname "$SCRIPT_DIR")"
readonly REPO_ROOT
readonly PYTHON="${PYTHON:-$REPO_ROOT/.venv/bin/python}"
readonly FAKE_PORT="${FAKE_PORT:-8099}"
readonly BOT_PORT="${BOT_PORT:-8098}"
readonly FAKE_URL="http://127.0.0.1:${FAKE_PORT}"
# Both spellings of one group, exactly as the real API reports them:
# /v1/groups returns the `group.<base64>` form (what .env holds), while an
# incoming envelope carries the id that base64 decodes to. Modelling only
# one is why the offline suite once passed against a bot that ignored every
# message in its own group.
readonly WATCHED_GROUP="group.WlRKbExYZGhkR05vWldRdFozSnZkWEF0YVdRdE16SXRZbmwwWlhNaElTRT0="
readonly WATCHED_GROUP_INTERNAL="ZTJlLXdhdGNoZWQtZ3JvdXAtaWQtMzItYnl0ZXMhISE="
# A second watched group, so the suite proves replies and rate limits stay
# in the group that asked instead of leaking into the other one.
readonly SECOND_GROUP="group.WlRKbExYTmxZMjl1WkMxbmNtOTFjQzFwWkMwek1pMWllWFJsY3lFaElTRT0="
readonly SECOND_GROUP_INTERNAL="ZTJlLXNlY29uZC1ncm91cC1pZC0zMi1ieXRlcyEhISE="
# Not seeded at boot: these arrive mid-run, which is the point.
readonly THIRD_GROUP="group.WlRKbExYUm9hWEprTFdkeWIzVndMV2xrTFRNeUxXSjVkR1Z6SVNFaElRPT0="
readonly THIRD_GROUP_INTERNAL="ZTJlLXRoaXJkLWdyb3VwLWlkLTMyLWJ5dGVzISEhIQ=="
readonly FOURTH_GROUP="group.WlRKbExXWnZkWEowYUMxbmNtOTFjQzFwWkMwek1pMWllWFJsY3lFaElTRT0="
readonly FOURTH_GROUP_INTERNAL="ZTJlLWZvdXJ0aC1ncm91cC1pZC0zMi1ieXRlcyEhISE="
readonly OTHER_GROUP="group.e2e.other=="
FAKE_PID=""
BOT_PID=""
TEMP_DIR=""
cleanup() {
# Plain ifs, not `A && B || C`: that is not if-then-else, and C runs when
# A is true but B fails -- which is exactly the case here (kill on an
# already-dead pid).
if [[ -n "$BOT_PID" ]]; then
kill "$BOT_PID" 2>/dev/null || true
fi
if [[ -n "$FAKE_PID" ]]; then
kill "$FAKE_PID" 2>/dev/null || true
fi
wait 2>/dev/null || true
if [[ -n "$TEMP_DIR" && -d "$TEMP_DIR" ]]; then
rm -rf "$TEMP_DIR"
fi
}
trap cleanup EXIT
die() {
echo "Error: $1" >&2
exit 1
}
wait_for_url() {
local url="$1" name="$2" attempt=0
until curl -sf "$url" >/dev/null 2>&1; do
attempt=$((attempt + 1))
if [[ $attempt -ge 100 ]]; then
echo "Error: $name did not come up at $url" >&2
exit 1
fi
sleep 0.1
done
}
wait_for_subscription() {
local attempt=0
until [[ "$(curl -sf "${FAKE_URL}/_control/ready" | tr -dc '0-9')" != "0" ]]; do
attempt=$((attempt + 1))
if [[ $attempt -ge 100 ]]; then
echo "Error: the bot never subscribed to the receive websocket" >&2
exit 1
fi
sleep 0.1
done
}
validate_requirements() {
if [[ ! -x "$PYTHON" ]]; then
echo "Error: no interpreter at $PYTHON. Run: uv venv .venv && uv pip install -e '.[dev]'" >&2
exit 1
fi
command -v curl >/dev/null || { echo "Error: curl is required" >&2; exit 1; }
}
main() {
validate_requirements
TEMP_DIR="$(mktemp -d)"
STARTED_AT="$(date +%s)"
echo "== booting the fake Signal server on :${FAKE_PORT}"
export FAKE_PORT
"$PYTHON" -m uvicorn fake_signal.server:app \
--host 127.0.0.1 --port "$FAKE_PORT" --log-level warning &
FAKE_PID=$!
wait_for_url "${FAKE_URL}/v1/about" "fake signal server"
echo "== seeding the fake with a registered, joined account"
curl -sf -X POST "${FAKE_URL}/_control/accounts" \
-H 'Content-Type: application/json' \
-d '["+15550000000"]' >/dev/null
curl -sf -X POST "${FAKE_URL}/_control/groups" \
-H 'Content-Type: application/json' \
-d "[{\"id\":\"${WATCHED_GROUP}\",\"internal_id\":\"${WATCHED_GROUP_INTERNAL}\",\"name\":\"e2e\",\"member\":true,\"joinable\":true},{\"id\":\"${SECOND_GROUP}\",\"internal_id\":\"${SECOND_GROUP_INTERNAL}\",\"name\":\"e2e-second\",\"member\":true,\"joinable\":true}]" >/dev/null
# GROUP_IDS and TRUSTED_INVITERS deliberately empty: the bot discovers
# its groups from the account, which is the configuration real
# deployments use. Set rather than merely unset, because load_dotenv()
# would otherwise let the developer's own .env decide what this run
# watches -- and a TRUSTED_INVITERS there would refuse every fixture
# group. The gate itself is covered in tests/test_trust.py.
# The album is private and lives outside this repository, so the run
# builds a synthetic one: three invented people, a combination and a
# video, scoped to the watched group so the scoping scenario can prove
# it does not exist in the second one.
echo "== writing a synthetic photo pack"
"$PYTHON" "$REPO_ROOT/tool/e2e_pack.py" --dir "${TEMP_DIR}/pack" --group "$WATCHED_GROUP"
echo "== booting the bot on :${BOT_PORT}, pointed at the fake"
env -u CAT_API \
PHONE_NUMBER="+15550000000" \
GROUP_IDS="" \
TRUSTED_INVITERS="" \
API_BASE_URL="$FAKE_URL" \
DATABASE_PATH="${TEMP_DIR}/e2e.sqlite3" \
CAT_API_URL="${FAKE_URL}/_fixtures/cat-api" \
DOG_API_URL="${FAKE_URL}/_fixtures/dog-api" \
GECKO_API_URL="${FAKE_URL}/_fixtures/gecko-api" \
COMMONS_API_URL="${FAKE_URL}/_fixtures/commons-api" \
POEM_TEXT_URL="${FAKE_URL}/_fixtures/poem-text?slug={slug}" \
PORTRAIT_CACHE_DIR="${TEMP_DIR}/portraits" \
PHOTO_PACK_DIR="${TEMP_DIR}/pack" \
BIND_PORT="$BOT_PORT" \
SILENCE_SECONDS=0 \
LIVENESS_TICK_SECONDS=1 \
"$PYTHON" -m signal_bot &
BOT_PID=$!
wait_for_url "http://127.0.0.1:${BOT_PORT}/healthz" "bot"
wait_for_subscription
echo "== generating media for !add_photo (real exif, never committed)"
"$PYTHON" "$REPO_ROOT/tool/e2e_media.py" --upload "$FAKE_URL"
echo "== replaying scenarios"
"$PYTHON" -m fake_signal.cli \
"$REPO_ROOT"/fake_signal/scenarios/*.yaml \
--settle "${E2E_SETTLE:-0.15}" \
--base-url "$FAKE_URL" \
--group "watched=${WATCHED_GROUP_INTERNAL}" \
--group "reply=${WATCHED_GROUP}" \
--group "second=${SECOND_GROUP_INTERNAL}" \
--group "second_reply=${SECOND_GROUP}" \
--group "third=${THIRD_GROUP_INTERNAL}" \
--group "third_reply=${THIRD_GROUP}" \
--group "fourth=${FOURTH_GROUP_INTERNAL}" \
--group "fourth_reply=${FOURTH_GROUP}" \
--group "other=${OTHER_GROUP}"
prove_the_add_photo_invariants
prove_the_deafness_gate
enforce_the_time_budget "$STARTED_AT"
}
# The three claims a reply cannot make on its own: nothing was written when
# nothing was stored, nothing kept its metadata, and no second copy survived.
main "$@"