signal-bot/tests/test_discovery.py
Krzysztof kuhy Rudnicki d5c63b423b feat: refuse to look healthy while deaf, and heal the container by itself
Today the bot was up, healthy and answering nothing for three hours. A host
firewall change (fixed in testsAndMisc 334c0cd9) cut every Docker container
off the internet; signal-cli lost Signal and reconnected in a loop from
15:32:50 onward. Nothing noticed:

  - the container's own healthcheck answered 204
  - /v1/groups answered 200 out of local state, so discovery saw no change
  - the bot's WebSocket to the container stayed OPEN, so the transport's
    reconnect logic never fired
  - /healthz answered {"status": "ok"} the entire time

The only symptom was silence, which is also what a quiet evening looks like.

signal_bot/liveness.py asks the one question that cannot be answered from a
cache -- GET /v1/devices, which needs a live round trip -- but only after
SILENCE_SECONDS (120) with no frame of any kind, so a busy group costs
nothing and an idle one costs one request per window. Three consecutive
failures and /healthz answers 503 with the reason. Verified end to end in
scripts/e2e.sh against the real process: 503 after 84s with the fake's
connection cut, back to 200 on its own 30s after restoring it.

scripts/signal_api_watchdog.sh + a two-minute user timer do the recovery the
bot cannot do for itself, since a wedged bot cannot restart the container it
depends on. Two tiers, so the steady state costs no Signal round trip at
all: the bot's own /healthz first, the authenticated probe only when that is
not 200. It stops after 3 restarts in 6 hours and escalates instead -- a
deregistered account or a blocked network is not fixable by restarting, and
a restart loop would hide it. Its own bash suite stubs curl and docker and
drives every branch including the give-up cap.

signal_bot/trust.py answers the other half: TRUSTED_INVITERS decides who may
put this bot in a group. The gate sits in GroupDirectory._permitted, which
both _join_pending and _adopt already route through -- gating only invites
would gate the door nobody uses, since prime_profile.sh exists precisely so
that an add becomes a direct join. Evidence is either the group-update
envelope, which is the only frame naming an inviter, or a trusted identity
being a full member in the listing. A refusal is logged once, with that
group's members, so the operator can see which string to add.

verify_allowlist moved to membership.py to keep discovery.py under the
250-line gate; it was already the module that owns MembershipError.

Verified live, by stopping the container by hand and touching nothing else:

  18:54:35  docker stop signal-api
  18:56:36  liveness: failure 1 of 3 after 123s of silence
  18:57:36  failure 3 of 3 -> /healthz answers 503 from 18:57:45
  18:58:18  watchdog: probe failed (1/2)
  19:00:28  watchdog: probe failed (2/2) -> docker restart signal-api
  19:00:36  liveness: receive path recovered after 8 failed probe(s)
  19:01:13  watchdog: signal-api is back

Six minutes, unattended, against three hours of silence this afternoon. All
four real groups are still adopted with TRUSTED_INVITERS=+48123456789 set,
and no group was refused.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KbjfWdAyHbCubac7wJKJZW
2026-08-29 19:01:55 +02:00

216 lines
7.1 KiB
Python

# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""The live group set: discovery, the allowlist, and never narrowing on failure."""
from __future__ import annotations
from typing import TYPE_CHECKING, Any
import httpx
import pytest
from signal_bot.discovery import GroupDirectory
from signal_bot.trust import InviteTrust
from tests.conftest import (
OTHER_GROUP,
SECOND_GROUP,
WATCHED_GROUP,
WATCHED_INTERNAL,
group_entry,
listing_client,
)
if TYPE_CHECKING:
from signal_bot.config import Config
from signal_bot.store import Store
Listing = list[dict[str, Any]]
def _directory(config: Config, store: Store, handler: object) -> GroupDirectory:
return GroupDirectory(
config,
httpx.AsyncClient(transport=httpx.MockTransport(handler)),
store,
InviteTrust(()),
)
async def test_discovers_every_group_the_account_is_in(config: Config, store: Store) -> None:
"""No GROUP_IDS at all: whatever the account is a member of is watched."""
found = GroupDirectory(
config,
listing_client([group_entry(WATCHED_GROUP), group_entry(SECOND_GROUP)]),
store,
InviteTrust(()),
)
await found.refresh()
assert found.groups == (WATCHED_GROUP, SECOND_GROUP)
assert found.resolve(WATCHED_INTERNAL) == WATCHED_GROUP
assert found.resolve(OTHER_GROUP) is None
assert found.resolve(None) is None
async def test_a_group_that_is_only_an_invite_is_not_watched(config: Config, store: Store) -> None:
"""Joining is attempted, but the listing decides whether it worked."""
found = _directory(
config,
store,
lambda r: httpx.Response(200, json=[group_entry(WATCHED_GROUP, member=False)]),
)
await found.refresh()
assert found.groups == ()
async def test_a_pending_invitation_is_joined(config: Config, store: Store) -> None:
# Two GETs per refresh when a join is attempted: read, join, re-read.
# The re-read is the authority -- the join endpoint answers 201 either way.
listings = iter(
[
[group_entry(WATCHED_GROUP, member=False)],
[group_entry(WATCHED_GROUP)],
]
)
def handler(request: httpx.Request) -> httpx.Response:
if request.method == "POST":
return httpx.Response(201, text="")
return httpx.Response(200, json=next(listings))
found = _directory(config, store, handler)
await found.refresh()
assert found.groups == (WATCHED_GROUP,)
async def test_an_impossible_invitation_is_not_retried_forever(
config: Config, store: Store
) -> None:
"""signal-cli cannot accept an invite addressed to a phone number."""
posts: list[str] = []
def handler(request: httpx.Request) -> httpx.Response:
if request.method == "POST":
posts.append(str(request.url))
return httpx.Response(400, text="Cannot find service ID for self")
return httpx.Response(200, json=[group_entry(WATCHED_GROUP, member=False)])
found = _directory(config, store, handler)
await found.refresh()
await found.refresh()
await found.refresh()
assert len(posts) == 1
async def test_a_failed_listing_never_narrows_the_watched_set(config: Config, store: Store) -> None:
"""The worst bug available: conclude you were removed from everything."""
responses = iter(
[
httpx.Response(200, json=[group_entry(WATCHED_GROUP)]),
httpx.Response(500, text="upstream is down"),
]
)
found = _directory(config, store, lambda r: next(responses))
await found.refresh()
assert found.groups == (WATCHED_GROUP,)
await found.refresh()
assert found.groups == (WATCHED_GROUP,)
assert found.resolve(WATCHED_INTERNAL) == WATCHED_GROUP
async def test_a_transport_error_never_narrows_the_watched_set(
config: Config, store: Store
) -> None:
calls = {"n": 0}
def handler(request: httpx.Request) -> httpx.Response:
calls["n"] += 1
if calls["n"] == 1:
return httpx.Response(200, json=[group_entry(WATCHED_GROUP)])
msg = "signal-cli went away"
raise httpx.ConnectError(msg)
found = _directory(config, store, handler)
await found.refresh()
await found.refresh()
assert found.groups == (WATCHED_GROUP,)
@pytest.mark.parametrize("body", [{"error": "nope"}, "a string"])
async def test_a_listing_that_is_not_a_list_is_ignored(
config: Config, store: Store, body: object
) -> None:
responses = iter(
[httpx.Response(200, json=[group_entry(WATCHED_GROUP)]), httpx.Response(200, json=body)]
)
found = _directory(config, store, lambda r: next(responses))
await found.refresh()
await found.refresh()
assert found.groups == (WATCHED_GROUP,)
async def test_rows_that_are_not_objects_are_skipped(config: Config, store: Store) -> None:
found = _directory(
config, store, lambda r: httpx.Response(200, json=["nonsense", group_entry(WATCHED_GROUP)])
)
await found.refresh()
assert found.groups == (WATCHED_GROUP,)
@pytest.mark.parametrize("entry", [{"member": True}, {"id": "", "member": True}, {"id": 7}])
async def test_a_row_without_a_usable_id_is_skipped(
config: Config, store: Store, entry: dict[str, Any]
) -> None:
found = _directory(config, store, lambda r: httpx.Response(200, json=[entry]))
await found.refresh()
assert found.groups == ()
async def test_a_removed_group_is_dropped_and_its_counts_purged(
config: Config, store: Store
) -> None:
"""A re-add should start a fresh tally, not resurrect an old one."""
responses = iter(
[
httpx.Response(200, json=[group_entry(WATCHED_GROUP), group_entry(SECOND_GROUP)]),
httpx.Response(200, json=[group_entry(SECOND_GROUP)]),
]
)
found = _directory(config, store, lambda r: next(responses))
await found.refresh()
store.increment(WATCHED_GROUP, "u1", "Alice")
store.increment(SECOND_GROUP, "u2", "Bob")
await found.refresh()
assert found.groups == (SECOND_GROUP,)
assert found.resolve(WATCHED_INTERNAL) is None
assert store.tallies(WATCHED_GROUP) == []
assert store.tallies(SECOND_GROUP)[0].count == 1
async def test_refresh_for_looks_again_once_per_unknown_group(config: Config, store: Store) -> None:
"""The first command after an add must work, without hammering the API."""
listings: list[Any] = [
[],
[group_entry(WATCHED_GROUP)],
[group_entry(WATCHED_GROUP)],
]
calls = {"n": 0}
def handler(request: httpx.Request) -> httpx.Response:
index = min(calls["n"], len(listings) - 1)
calls["n"] += 1
return httpx.Response(200, json=listings[index])
found = _directory(config, store, handler)
await found.refresh()
assert found.groups == ()
assert await found.refresh_for(WATCHED_INTERNAL) == WATCHED_GROUP
before = calls["n"]
# A second unknown id refreshes; the same one again does not.
assert await found.refresh_for(OTHER_GROUP) is None
assert calls["n"] > before
after = calls["n"]
assert await found.refresh_for(OTHER_GROUP) is None
assert calls["n"] == after