signal-bot/DOCS-signal-envelopes.md
Krzysztof kuhy Rudnicki d659ff5e9c feat: refuse to look healthy while deaf, and heal the container by itself
Today the bot was up, healthy and answering nothing for three hours. A host
firewall change (fixed in testsAndMisc 334c0cd9) cut every Docker container
off the internet; signal-cli lost Signal and reconnected in a loop from
15:32:50 onward. Nothing noticed:

  - the container's own healthcheck answered 204
  - /v1/groups answered 200 out of local state, so discovery saw no change
  - the bot's WebSocket to the container stayed OPEN, so the transport's
    reconnect logic never fired
  - /healthz answered {"status": "ok"} the entire time

The only symptom was silence, which is also what a quiet evening looks like.

signal_bot/liveness.py asks the one question that cannot be answered from a
cache -- GET /v1/devices, which needs a live round trip -- but only after
SILENCE_SECONDS (120) with no frame of any kind, so a busy group costs
nothing and an idle one costs one request per window. Three consecutive
failures and /healthz answers 503 with the reason. Verified end to end in
scripts/e2e.sh against the real process: 503 after 84s with the fake's
connection cut, back to 200 on its own 30s after restoring it.

scripts/signal_api_watchdog.sh + a two-minute user timer do the recovery the
bot cannot do for itself, since a wedged bot cannot restart the container it
depends on. Two tiers, so the steady state costs no Signal round trip at
all: the bot's own /healthz first, the authenticated probe only when that is
not 200. It stops after 3 restarts in 6 hours and escalates instead -- a
deregistered account or a blocked network is not fixable by restarting, and
a restart loop would hide it. Its own bash suite stubs curl and docker and
drives every branch including the give-up cap.

signal_bot/trust.py answers the other half: TRUSTED_INVITERS decides who may
put this bot in a group. The gate sits in GroupDirectory._permitted, which
both _join_pending and _adopt already route through -- gating only invites
would gate the door nobody uses, since prime_profile.sh exists precisely so
that an add becomes a direct join. Evidence is either the group-update
envelope, which is the only frame naming an inviter, or a trusted identity
being a full member in the listing. A refusal is logged once, with that
group's members, so the operator can see which string to add.

verify_allowlist moved to membership.py to keep discovery.py under the
250-line gate; it was already the module that owns MembershipError.

Verified live, by stopping the container by hand and touching nothing else:

  18:54:35  docker stop signal-api
  18:56:36  liveness: failure 1 of 3 after 123s of silence
  18:57:36  failure 3 of 3 -> /healthz answers 503 from 18:57:45
  18:58:18  watchdog: probe failed (1/2)
  19:00:28  watchdog: probe failed (2/2) -> docker restart signal-api
  19:00:36  liveness: receive path recovered after 8 failed probe(s)
  19:01:13  watchdog: signal-api is back

Six minutes, unattended, against three hours of silence this afternoon. All
four real groups are still adopted with TRUSTED_INVITERS=+48516501330 set,
and no group was refused.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KbjfWdAyHbCubac7wJKJZW
2026-08-29 19:01:55 +02:00

110 lines
3.5 KiB
Markdown

# Signal envelope shapes
Reference for the two JSON frames `signal-cli-rest-api` pushes down
`ws://…/v1/receive/{number}`. `signal_bot/models.py` flattens both into one
`IncomingMessage`; these are the shapes it flattens.
## From another account: `dataMessage`
```json
{
"envelope": {
"source": "NAME",
"sourceNumber": "PHONE_NUMBER",
"sourceUuid": "SOURCE_ID",
"sourceName": "USER_DEFINED_NAME",
"sourceDevice": 69,
"timestamp": 1719177917717,
"dataMessage": {
"timestamp": 1719177917717,
"message": "MESSAGE_CONTENT",
"expiresInSeconds": 0,
"viewOnce": false,
"sticker": { "packId": "166dd1b5…", "stickerId": 6 },
"groupInfo": { "groupId": "ID_OF_GROUP", "type": "DELIVER" }
}
},
"account": "BOT_ACCOUNT_PHONE_NUMBER"
}
```
## Somebody changed the membership: `groupInfo.type` is `UPDATE`
The only frame that names *who* added the bot to a group, which is what
`signal_bot/trust.py` needs. It carries no `message` at all, so it counts
toward nothing and answers nothing; `source`/`sourceNumber` is the person
who made the change. Captured live on 2026-08-29:
```json
{
"envelope": {
"source": "+48516501330",
"sourceNumber": "+48516501330",
"sourceUuid": "ffcae71a-…",
"sourceName": "kuhy",
"sourceDevice": 1,
"timestamp": 1788006851503,
"dataMessage": {
"timestamp": 1788006851503,
"message": null,
"groupInfo": { "groupId": "6/gEwRkqs/…", "groupName": "testbot", "type": "UPDATE" }
}
},
"account": "+420720355763"
}
```
Note `source` here is a **phone number**, not the display name this document
used to show for `dataMessage`. Both spellings have been observed, which is
why `models.py` reads the sender's number from `sourceNumber` and nothing
trusts `source`.
## From the bot's own account: `syncMessage.sentMessage`
Because the bot runs as a *linked device*, the account's own outgoing messages
come back as sync messages. Same fields, one level deeper.
```json
{
"envelope": {
"sourceUuid": "SOURCE_ID",
"sourceName": "USER_DEFINED_NAME",
"timestamp": 1719177728639,
"syncMessage": {
"sentMessage": {
"destination": "DEST_NAME",
"destinationNumber": "DEST_PHONE_NUMBER",
"timestamp": 1719177728639,
"message": "MESSAGE_CONTENT",
"sticker": { "packId": "166dd1b5…", "stickerId": 6 },
"groupInfo": { "groupId": "ID_OF_GROUP", "type": "DELIVER" }
}
}
},
"account": "BOT_ACCOUNT_PHONE_NUMBER"
}
```
## The trap this caused
The original bot read `envelope.dataMessage.sticker` even when the frame was a
`syncMessage`. Its own stickers therefore never matched the sticker filter and
were counted as messages. Four consecutive commits (`f1931b8`, `b325126`,
`62352e5`, `d35aa59`) fought the symptom.
`models.py` resolves the payload *first* — `dataMessage` or
`syncMessage.sentMessage` — and reads `sticker`, `reaction`, `message` and
`groupInfo` off whichever it found. `tests/test_models.py` asserts both shapes.
## No `groupInfo` means a direct message
A frame without `groupInfo` arrived in a one-to-one conversation. A linked
device receives every one of those on the account, so `counting.watched_group`
rejects them explicitly rather than relying on a falsy comparison. See the
`scoping` scenario in `fake_signal/scenarios/`.
## Frames the bot ignores
Receipts, typing indicators and anything else with no `dataMessage` and no
`syncMessage.sentMessage` parse to `None` and are dropped.