mirror of
https://github.com/kuhyx/signal-bot.git
synced 2026-11-10 19:17:02 +01:00
Today the bot was up, healthy and answering nothing for three hours. A host
firewall change (fixed in testsAndMisc 334c0cd9) cut every Docker container
off the internet; signal-cli lost Signal and reconnected in a loop from
15:32:50 onward. Nothing noticed:
- the container's own healthcheck answered 204
- /v1/groups answered 200 out of local state, so discovery saw no change
- the bot's WebSocket to the container stayed OPEN, so the transport's
reconnect logic never fired
- /healthz answered {"status": "ok"} the entire time
The only symptom was silence, which is also what a quiet evening looks like.
signal_bot/liveness.py asks the one question that cannot be answered from a
cache -- GET /v1/devices, which needs a live round trip -- but only after
SILENCE_SECONDS (120) with no frame of any kind, so a busy group costs
nothing and an idle one costs one request per window. Three consecutive
failures and /healthz answers 503 with the reason. Verified end to end in
scripts/e2e.sh against the real process: 503 after 84s with the fake's
connection cut, back to 200 on its own 30s after restoring it.
scripts/signal_api_watchdog.sh + a two-minute user timer do the recovery the
bot cannot do for itself, since a wedged bot cannot restart the container it
depends on. Two tiers, so the steady state costs no Signal round trip at
all: the bot's own /healthz first, the authenticated probe only when that is
not 200. It stops after 3 restarts in 6 hours and escalates instead -- a
deregistered account or a blocked network is not fixable by restarting, and
a restart loop would hide it. Its own bash suite stubs curl and docker and
drives every branch including the give-up cap.
signal_bot/trust.py answers the other half: TRUSTED_INVITERS decides who may
put this bot in a group. The gate sits in GroupDirectory._permitted, which
both _join_pending and _adopt already route through -- gating only invites
would gate the door nobody uses, since prime_profile.sh exists precisely so
that an add becomes a direct join. Evidence is either the group-update
envelope, which is the only frame naming an inviter, or a trusted identity
being a full member in the listing. A refusal is logged once, with that
group's members, so the operator can see which string to add.
verify_allowlist moved to membership.py to keep discovery.py under the
250-line gate; it was already the module that owns MembershipError.
Verified live, by stopping the container by hand and touching nothing else:
18:54:35 docker stop signal-api
18:56:36 liveness: failure 1 of 3 after 123s of silence
18:57:36 failure 3 of 3 -> /healthz answers 503 from 18:57:45
18:58:18 watchdog: probe failed (1/2)
19:00:28 watchdog: probe failed (2/2) -> docker restart signal-api
19:00:36 liveness: receive path recovered after 8 failed probe(s)
19:01:13 watchdog: signal-api is back
Six minutes, unattended, against three hours of silence this afternoon. All
four real groups are still adopted with TRUSTED_INVITERS=+48123456789 set,
and no group was refused.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KbjfWdAyHbCubac7wJKJZW
225 lines
7.6 KiB
Python
225 lines
7.6 KiB
Python
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
|
|
"""Application wiring and the ASGI entrypoint."""
|
|
|
|
from __future__ import annotations
|
|
|
|
import asyncio
|
|
import contextlib
|
|
import datetime as dt
|
|
import logging
|
|
from dataclasses import dataclass, field
|
|
from typing import TYPE_CHECKING
|
|
|
|
if TYPE_CHECKING:
|
|
from collections.abc import AsyncIterator
|
|
|
|
from signal_bot.transport_base import Transport
|
|
|
|
import httpx
|
|
from fastapi import FastAPI, Response, status
|
|
|
|
from signal_bot.animals import register_animal_commands
|
|
from signal_bot.config import Config, load_config
|
|
from signal_bot.counting import format_tally
|
|
from signal_bot.decorators import cooldown
|
|
from signal_bot.discovery import GroupDirectory
|
|
from signal_bot.dispatcher import Dispatcher
|
|
from signal_bot.http_client import ImageFetcher
|
|
from signal_bot.liveness import Liveness
|
|
from signal_bot.registry import CommandRegistry
|
|
from signal_bot.scheduler import DailyScheduler
|
|
from signal_bot.store import Store
|
|
from signal_bot.transport_rest import RestTransport
|
|
from signal_bot.trust import InviteTrust
|
|
|
|
logger = logging.getLogger(__name__)
|
|
|
|
COOLDOWN_MESSAGE = "BEEP BOOP POCZEKAJ {seconds} SEKUND."
|
|
HTTP_TIMEOUT_SECONDS = 30.0
|
|
|
|
|
|
def now_local() -> dt.datetime:
|
|
"""Current local time, timezone-aware."""
|
|
return dt.datetime.now().astimezone()
|
|
|
|
|
|
@dataclass
|
|
class Bot:
|
|
"""The wired-up bot and its two long-running loops."""
|
|
|
|
config: Config
|
|
store: Store
|
|
transport: Transport
|
|
directory: GroupDirectory
|
|
dispatcher: Dispatcher
|
|
liveness: Liveness
|
|
|
|
async def publish_report(self) -> None:
|
|
"""Post each group's own tally into that group, and reset it.
|
|
|
|
Each group is isolated on purpose. One group refusing the report --
|
|
the bot removed from it, a transient send failure -- must not cost
|
|
every other group its report, and a tally is cleared only after its
|
|
report has actually gone out, so a failed send loses nothing. A
|
|
group where nobody spoke is skipped entirely.
|
|
"""
|
|
for group_id in self.directory.groups:
|
|
rows = [(t.display_name or t.uuid, t.count) for t in self.store.tallies(group_id)]
|
|
if not rows:
|
|
# Logged, not posted. The bot is in every group it was ever
|
|
# added to, so posting "nobody spoke" would mean a nightly
|
|
# ping into every quiet group, forever.
|
|
logger.info("no messages in %s today; not posting a report", group_id)
|
|
continue
|
|
try:
|
|
await self.transport.send_text(format_tally(rows), group_id)
|
|
except Exception:
|
|
logger.exception("daily report failed for group %s", group_id)
|
|
continue
|
|
self.store.reset_tallies(group_id)
|
|
|
|
async def listen(self) -> None:
|
|
"""Consume frames forever, handing each to the dispatcher."""
|
|
async for frame in self.transport.receive():
|
|
# Every frame counts, including receipts and typing indicators:
|
|
# the question this answers is "can anything still reach us?",
|
|
# not "did anybody address the bot?".
|
|
self.liveness.note_envelope()
|
|
await self.dispatcher.handle(frame)
|
|
|
|
|
|
def build_bot(
|
|
config: Config,
|
|
client: httpx.AsyncClient,
|
|
store: Store,
|
|
transport: Transport,
|
|
directory: GroupDirectory,
|
|
liveness: Liveness,
|
|
) -> Bot:
|
|
"""Assemble the object graph from its injected dependencies."""
|
|
registry = CommandRegistry()
|
|
register_animal_commands(
|
|
registry,
|
|
ImageFetcher(client),
|
|
decorate=cooldown(config.cooldown_seconds, COOLDOWN_MESSAGE),
|
|
endpoints={"cat": config.cat_api_url, "dog": config.dog_api_url},
|
|
)
|
|
dispatcher = Dispatcher(
|
|
config=config,
|
|
directory=directory,
|
|
registry=registry,
|
|
store=store,
|
|
transport=transport,
|
|
clock=now_local,
|
|
)
|
|
return Bot(
|
|
config=config,
|
|
store=store,
|
|
transport=transport,
|
|
directory=directory,
|
|
dispatcher=dispatcher,
|
|
liveness=liveness,
|
|
)
|
|
|
|
|
|
@dataclass
|
|
class Runtime:
|
|
"""Owns everything that must be torn down when the app stops."""
|
|
|
|
bot: Bot
|
|
client: httpx.AsyncClient
|
|
store: Store
|
|
tasks: list[asyncio.Task[None]] = field(default_factory=list)
|
|
|
|
def start(self) -> None:
|
|
"""Spawn the listener and the daily scheduler as background tasks."""
|
|
scheduler = DailyScheduler(
|
|
self.bot.config.report_hour,
|
|
self.bot.config.report_minute,
|
|
self.bot.publish_report,
|
|
now_local,
|
|
asyncio.sleep,
|
|
)
|
|
self.tasks = [
|
|
asyncio.create_task(self.bot.listen()),
|
|
asyncio.create_task(scheduler.run()),
|
|
# Third loop: without it, a group added later is only noticed
|
|
# when somebody happens to speak in it.
|
|
asyncio.create_task(self.bot.directory.run()),
|
|
# Fourth loop: without it, a dead connection to Signal is
|
|
# indistinguishable from a quiet evening, for hours.
|
|
asyncio.create_task(self.bot.liveness.run()),
|
|
]
|
|
|
|
async def stop(self) -> None:
|
|
"""Cancel the background tasks and close every resource."""
|
|
for task in self.tasks:
|
|
task.cancel()
|
|
for task in self.tasks:
|
|
with contextlib.suppress(asyncio.CancelledError):
|
|
await task
|
|
await self.client.aclose()
|
|
self.store.close()
|
|
|
|
|
|
async def create_runtime(config: Config, client: httpx.AsyncClient) -> Runtime:
|
|
"""Build a Runtime, verifying the container exposes a WebSocket first."""
|
|
store = Store(config.database_path)
|
|
transport = RestTransport(config, client)
|
|
await transport.verify_mode()
|
|
directory = GroupDirectory(config, client, store, InviteTrust(config.trusted_inviters))
|
|
await directory.refresh()
|
|
await directory.verify_allowlist()
|
|
# No probe here: /v1/devices has been seen to hang past 20s, and a
|
|
# startup that can hang is a startup that can burn the unit's restart
|
|
# limit. The gate starts healthy and earns its verdict at runtime.
|
|
liveness = Liveness(config, client)
|
|
return Runtime(
|
|
bot=build_bot(config, client, store, transport, directory, liveness),
|
|
client=client,
|
|
store=store,
|
|
)
|
|
|
|
|
|
@contextlib.asynccontextmanager
|
|
async def lifespan(app: FastAPI) -> AsyncIterator[None]:
|
|
"""Start the background tasks without blocking ASGI startup.
|
|
|
|
The previous version awaited asyncio.gather() on two infinite tasks inside
|
|
an on_event("startup") handler, so uvicorn never finished starting up and
|
|
the HTTP app never began serving.
|
|
"""
|
|
config = load_config()
|
|
client = httpx.AsyncClient(timeout=HTTP_TIMEOUT_SECONDS)
|
|
runtime = await create_runtime(config, client)
|
|
runtime.start()
|
|
logger.info(
|
|
"listening as %s, watching %d group(s): %r",
|
|
config.phone_number,
|
|
len(runtime.bot.directory.groups),
|
|
list(runtime.bot.directory.groups),
|
|
)
|
|
app.state.runtime = runtime
|
|
try:
|
|
yield
|
|
finally:
|
|
await runtime.stop()
|
|
|
|
|
|
app = FastAPI(lifespan=lifespan)
|
|
|
|
|
|
@app.get("/healthz")
|
|
async def healthz(response: Response) -> dict[str, str]:
|
|
"""Liveness probe, and the only place the bot admits it has gone deaf.
|
|
|
|
Answering 200 while nothing can reach Signal is what let a three-hour
|
|
outage look like a quiet evening, so this must go red before anybody
|
|
thinks to look at the logs.
|
|
"""
|
|
liveness: Liveness = app.state.runtime.bot.liveness
|
|
if liveness.healthy:
|
|
return {"status": "ok"}
|
|
response.status_code = status.HTTP_503_SERVICE_UNAVAILABLE
|
|
return {"status": "deaf", "reason": liveness.reason}
|