Krzysztof kuhy Rudnicki
d5c63b423b
Today the bot was up, healthy and answering nothing for three hours. A host
firewall change (fixed in testsAndMisc 334c0cd9) cut every Docker container
off the internet; signal-cli lost Signal and reconnected in a loop from
15:32:50 onward. Nothing noticed:
- the container's own healthcheck answered 204
- /v1/groups answered 200 out of local state, so discovery saw no change
- the bot's WebSocket to the container stayed OPEN, so the transport's
reconnect logic never fired
- /healthz answered {"status": "ok"} the entire time
The only symptom was silence, which is also what a quiet evening looks like.
signal_bot/liveness.py asks the one question that cannot be answered from a
cache -- GET /v1/devices, which needs a live round trip -- but only after
SILENCE_SECONDS (120) with no frame of any kind, so a busy group costs
nothing and an idle one costs one request per window. Three consecutive
failures and /healthz answers 503 with the reason. Verified end to end in
scripts/e2e.sh against the real process: 503 after 84s with the fake's
connection cut, back to 200 on its own 30s after restoring it.
scripts/signal_api_watchdog.sh + a two-minute user timer do the recovery the
bot cannot do for itself, since a wedged bot cannot restart the container it
depends on. Two tiers, so the steady state costs no Signal round trip at
all: the bot's own /healthz first, the authenticated probe only when that is
not 200. It stops after 3 restarts in 6 hours and escalates instead -- a
deregistered account or a blocked network is not fixable by restarting, and
a restart loop would hide it. Its own bash suite stubs curl and docker and
drives every branch including the give-up cap.
signal_bot/trust.py answers the other half: TRUSTED_INVITERS decides who may
put this bot in a group. The gate sits in GroupDirectory._permitted, which
both _join_pending and _adopt already route through -- gating only invites
would gate the door nobody uses, since prime_profile.sh exists precisely so
that an add becomes a direct join. Evidence is either the group-update
envelope, which is the only frame naming an inviter, or a trusted identity
being a full member in the listing. A refusal is logged once, with that
group's members, so the operator can see which string to add.
verify_allowlist moved to membership.py to keep discovery.py under the
250-line gate; it was already the module that owns MembershipError.
Verified live, by stopping the container by hand and touching nothing else:
18:54:35 docker stop signal-api
18:56:36 liveness: failure 1 of 3 after 123s of silence
18:57:36 failure 3 of 3 -> /healthz answers 503 from 18:57:45
18:58:18 watchdog: probe failed (1/2)
19:00:28 watchdog: probe failed (2/2) -> docker restart signal-api
19:00:36 liveness: receive path recovered after 8 failed probe(s)
19:01:13 watchdog: signal-api is back
Six minutes, unattended, against three hours of silence this afternoon. All
four real groups are still adopted with TRUSTED_INVITERS=+48123456789 set,
and no group was refused.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KbjfWdAyHbCubac7wJKJZW
3.5 KiB
Signal envelope shapes
Reference for the two JSON frames signal-cli-rest-api pushes down
ws://…/v1/receive/{number}. signal_bot/models.py flattens both into one
IncomingMessage; these are the shapes it flattens.
From another account: dataMessage
{
"envelope": {
"source": "NAME",
"sourceNumber": "PHONE_NUMBER",
"sourceUuid": "SOURCE_ID",
"sourceName": "USER_DEFINED_NAME",
"sourceDevice": 69,
"timestamp": 1719177917717,
"dataMessage": {
"timestamp": 1719177917717,
"message": "MESSAGE_CONTENT",
"expiresInSeconds": 0,
"viewOnce": false,
"sticker": { "packId": "166dd1b5…", "stickerId": 6 },
"groupInfo": { "groupId": "ID_OF_GROUP", "type": "DELIVER" }
}
},
"account": "BOT_ACCOUNT_PHONE_NUMBER"
}
Somebody changed the membership: groupInfo.type is UPDATE
The only frame that names who added the bot to a group, which is what
signal_bot/trust.py needs. It carries no message at all, so it counts
toward nothing and answers nothing; source/sourceNumber is the person
who made the change. Captured live on 2026-08-29:
{
"envelope": {
"source": "+48123456789",
"sourceNumber": "+48123456789",
"sourceUuid": "b1c2d3e4-…",
"sourceName": "kuhy",
"sourceDevice": 1,
"timestamp": 1788006851503,
"dataMessage": {
"timestamp": 1788006851503,
"message": null,
"groupInfo": { "groupId": "Y29uZmlnLWdy/…", "groupName": "testbot", "type": "UPDATE" }
}
},
"account": "+420111222333"
}
Note source here is a phone number, not the display name this document
used to show for dataMessage. Both spellings have been observed, which is
why models.py reads the sender's number from sourceNumber and nothing
trusts source.
From the bot's own account: syncMessage.sentMessage
Because the bot runs as a linked device, the account's own outgoing messages come back as sync messages. Same fields, one level deeper.
{
"envelope": {
"sourceUuid": "SOURCE_ID",
"sourceName": "USER_DEFINED_NAME",
"timestamp": 1719177728639,
"syncMessage": {
"sentMessage": {
"destination": "DEST_NAME",
"destinationNumber": "DEST_PHONE_NUMBER",
"timestamp": 1719177728639,
"message": "MESSAGE_CONTENT",
"sticker": { "packId": "166dd1b5…", "stickerId": 6 },
"groupInfo": { "groupId": "ID_OF_GROUP", "type": "DELIVER" }
}
}
},
"account": "BOT_ACCOUNT_PHONE_NUMBER"
}
The trap this caused
The original bot read envelope.dataMessage.sticker even when the frame was a
syncMessage. Its own stickers therefore never matched the sticker filter and
were counted as messages. Four consecutive commits (f1931b8, b325126,
62352e5, d35aa59) fought the symptom.
models.py resolves the payload first — dataMessage or
syncMessage.sentMessage — and reads sticker, reaction, message and
groupInfo off whichever it found. tests/test_models.py asserts both shapes.
No groupInfo means a direct message
A frame without groupInfo arrived in a one-to-one conversation. A linked
device receives every one of those on the account, so counting.watched_group
rejects them explicitly rather than relying on a falsy comparison. See the
scoping scenario in fake_signal/scenarios/.
Frames the bot ignores
Receipts, typing indicators and anything else with no dataMessage and no
syncMessage.sentMessage parse to None and are dropped.