signal-bot/signal_bot/discovery.py
Krzysztof kuhy Rudnicki d5c63b423b feat: refuse to look healthy while deaf, and heal the container by itself
Today the bot was up, healthy and answering nothing for three hours. A host
firewall change (fixed in testsAndMisc 334c0cd9) cut every Docker container
off the internet; signal-cli lost Signal and reconnected in a loop from
15:32:50 onward. Nothing noticed:

  - the container's own healthcheck answered 204
  - /v1/groups answered 200 out of local state, so discovery saw no change
  - the bot's WebSocket to the container stayed OPEN, so the transport's
    reconnect logic never fired
  - /healthz answered {"status": "ok"} the entire time

The only symptom was silence, which is also what a quiet evening looks like.

signal_bot/liveness.py asks the one question that cannot be answered from a
cache -- GET /v1/devices, which needs a live round trip -- but only after
SILENCE_SECONDS (120) with no frame of any kind, so a busy group costs
nothing and an idle one costs one request per window. Three consecutive
failures and /healthz answers 503 with the reason. Verified end to end in
scripts/e2e.sh against the real process: 503 after 84s with the fake's
connection cut, back to 200 on its own 30s after restoring it.

scripts/signal_api_watchdog.sh + a two-minute user timer do the recovery the
bot cannot do for itself, since a wedged bot cannot restart the container it
depends on. Two tiers, so the steady state costs no Signal round trip at
all: the bot's own /healthz first, the authenticated probe only when that is
not 200. It stops after 3 restarts in 6 hours and escalates instead -- a
deregistered account or a blocked network is not fixable by restarting, and
a restart loop would hide it. Its own bash suite stubs curl and docker and
drives every branch including the give-up cap.

signal_bot/trust.py answers the other half: TRUSTED_INVITERS decides who may
put this bot in a group. The gate sits in GroupDirectory._permitted, which
both _join_pending and _adopt already route through -- gating only invites
would gate the door nobody uses, since prime_profile.sh exists precisely so
that an add becomes a direct join. Evidence is either the group-update
envelope, which is the only frame naming an inviter, or a trusted identity
being a full member in the listing. A refusal is logged once, with that
group's members, so the operator can see which string to add.

verify_allowlist moved to membership.py to keep discovery.py under the
250-line gate; it was already the module that owns MembershipError.

Verified live, by stopping the container by hand and touching nothing else:

  18:54:35  docker stop signal-api
  18:56:36  liveness: failure 1 of 3 after 123s of silence
  18:57:36  failure 3 of 3 -> /healthz answers 503 from 18:57:45
  18:58:18  watchdog: probe failed (1/2)
  19:00:28  watchdog: probe failed (2/2) -> docker restart signal-api
  19:00:36  liveness: receive path recovered after 8 failed probe(s)
  19:01:13  watchdog: signal-api is back

Six minutes, unattended, against three hours of silence this afternoon. All
four real groups are still adopted with TRUSTED_INVITERS=+48123456789 set,
and no group was refused.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KbjfWdAyHbCubac7wJKJZW
2026-08-29 19:01:55 +02:00

247 lines
9.6 KiB
Python

# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Which groups the bot is in, decided at runtime rather than in .env.
The bot is its own registered Signal account, so the groups it can see are
exactly the groups somebody put it in. Making the operator also write those
ids into .env and restart added a second step to "add the bot" that could
only ever be forgotten. This module keeps the watched set in sync with the
account itself.
The one rule that matters: a *failed* refresh never narrows the set. If the
listing cannot be fetched, or comes back malformed, the previous set stands.
Rebuilding from a partial answer would make the bot conclude it had been
removed from everything -- deaf, and with every group's counts purged.
"""
from __future__ import annotations
import asyncio
import logging
from collections.abc import Awaitable, Callable
from typing import TYPE_CHECKING, Any, Final
import httpx
from signal_bot.groups import build_alias_map, canonical_group_id
from signal_bot.membership import JoinAttempts, attempt_join, verify_allowlist
if TYPE_CHECKING:
from signal_bot.config import Config
from signal_bot.models import IncomingMessage
from signal_bot.store import Store
from signal_bot.trust import InviteTrust
logger = logging.getLogger(__name__)
REFRESH_SECONDS: Final = 30.0
Sleeper = Callable[[float], Awaitable[None]]
class GroupDirectory:
"""The live set of groups the bot answers in."""
def __init__(
self,
config: Config,
client: httpx.AsyncClient,
store: Store,
trust: InviteTrust,
*,
sleeper: Sleeper | None = None,
) -> None:
"""Wire the directory to its config, HTTP client and store.
Args:
config: Resolved runtime configuration.
client: HTTP client for the REST API.
store: Where per-group state lives, so a group the bot is
removed from can have its rows dropped.
trust: Decides whether a discovered group is one the operator
asked for.
sleeper: Async sleep used by run(). Injected by the tests.
"""
self._config = config
self._client = client
self._store = store
self._trust = trust
self._sleep = sleeper if sleeper is not None else asyncio.sleep
self._aliases: dict[str, str] = {}
self._groups: tuple[str, ...] = ()
self._joins = JoinAttempts()
# Group ids seen in an envelope that were not in the listing yet. One
# refresh per *id*, not per unit of time: a global cooldown would
# make a stranger's message a second earlier swallow the first
# command in a group that was just added.
self._refreshed_for: set[str] = set()
@property
def groups(self) -> tuple[str, ...]:
"""Every group currently watched, canonical and in listing order."""
return self._groups
def resolve(self, group_id: str | None) -> str | None:
"""Return the canonical id of a watched group, or None.
This is the single place an envelope's group id becomes a group the
bot acts on: the scoping gate, the reply address and the per-group
state key all come from this one answer.
Args:
group_id: The id as it arrived, or None for a direct message.
Returns:
The canonical id, or None for a direct message or a group the
bot does not watch.
"""
if group_id is None:
return None
return self._aliases.get(group_id)
def note_group_update(self, message: IncomingMessage) -> None:
"""Record who changed a group's membership.
Called before the scoping gate on purpose: the frame that names the
inviter arrives while the group it describes is still unknown, and
it is the only authoritative answer to "who added this bot?".
Args:
message: Any parsed inbound message.
"""
if message.group_id is None or not message.is_group_update:
return
self._trust.note_update(message.group_id, message.source_number, message.source_uuid)
async def refresh(self) -> None:
"""Rebuild the watched set from the account's own group listing.
Joins anything the bot was merely invited to, then re-reads the
listing so membership is decided by the API rather than by the join
endpoint's own optimistic answer.
"""
entries = await self._fetch()
if entries is None:
return
if await self._join_pending(entries):
entries = await self._fetch()
if entries is None:
return
self._adopt(entries)
async def refresh_for(self, group_id: str) -> str | None:
"""Refresh once because a message arrived from an unknown group.
Args:
group_id: The unrecognised id from the envelope.
Returns:
The canonical id if the refresh adopted that group, else None.
"""
if group_id in self._refreshed_for:
return None
self._refreshed_for.add(group_id)
logger.info("message from unknown group %r; refreshing the listing", group_id)
await self.refresh()
return self.resolve(group_id)
async def verify_allowlist(self) -> None:
"""Refuse to start when GROUP_IDS names a group the bot cannot use."""
verify_allowlist(self._config, self._groups, self._joins)
async def run(self, iterations: int | None = None) -> None:
"""Refresh forever, so a group added later needs no restart.
Args:
iterations: Stop after this many refreshes. None means forever.
"""
completed = 0
while iterations is None or completed < iterations:
await self._sleep(REFRESH_SECONDS)
await self.refresh()
completed += 1
async def _fetch(self) -> list[dict[str, Any]] | None:
"""Return the group listing, or None when it could not be read.
None is the "keep what we have" signal. Every failure mode of this
call -- timeout, 500, a body that is not a list -- must leave the
watched set alone rather than emptying it.
"""
try:
response = await self._client.get(self._config.groups_url)
response.raise_for_status()
body = response.json()
except (httpx.HTTPError, ValueError) as exc:
logger.warning(
"group listing unavailable, keeping the %d group(s) already known: %s",
len(self._groups),
exc,
)
return None
if not isinstance(body, list):
logger.warning("group listing was %s, not a list; keeping what we have", type(body))
return None
return [entry for entry in body if isinstance(entry, dict)]
def _permitted(self, entry: dict[str, Any]) -> str | None:
"""Return the canonical id of a listing row the bot may watch."""
raw = entry.get("id") or entry.get("internal_id")
if not isinstance(raw, str) or not raw:
return None
group_id = canonical_group_id(raw)
if not self._config.allows(group_id):
return None
# A pinned id was named by the operator in .env, which is already the
# strongest possible vouch; running it past the trust gate as well
# would make verify_allowlist fail with a message about joins that
# says nothing about trust.
if self._config.is_pinned or self._trust.allows(group_id, entry):
return group_id
return None
async def _join_pending(self, entries: list[dict[str, Any]]) -> bool:
"""Try to accept every invitation worth retrying.
Args:
entries: The current group listing.
Returns:
True when at least one join was attempted, so the caller knows
the listing needs re-reading.
"""
attempted = False
for entry in entries:
group_id = self._permitted(entry)
if group_id is None or entry.get("member") or self._joins.has_failed(group_id):
continue
logger.warning("invited to %r but not a member; trying to join", entry.get("name"))
detail = await attempt_join(self._client, self._config, group_id)
self._joins.record_failure(group_id, detail)
attempted = True
return attempted
def _adopt(self, entries: list[dict[str, Any]]) -> None:
"""Replace the watched set from a listing known to be good."""
groups = tuple(
group_id
for entry in entries
if entry.get("member") and (group_id := self._permitted(entry)) is not None
)
if groups == self._groups:
return
added = [group_id for group_id in groups if group_id not in self._groups]
removed = [group_id for group_id in self._groups if group_id not in groups]
# Built whole and assigned in one statement: the dispatcher reads
# this map between awaits, and must never see it half-built.
self._aliases = build_alias_map(groups)
self._groups = groups
for group_id in added:
self._joins.forget(group_id)
self._refreshed_for.discard(group_id)
logger.info("now watching %s", group_id)
for group_id in removed:
# Dropped, not kept: a re-add should start a fresh tally rather
# than resurrect counts from whenever the bot was last in there.
self._store.forget_group(group_id)
logger.info("no longer a member of %s; its counts were dropped", group_id)