diff --git a/DOCS-statistics.md b/DOCS-statistics.md new file mode 100644 index 0000000..4d2dba4 --- /dev/null +++ b/DOCS-statistics.md @@ -0,0 +1,90 @@ +# Statistics + +## What is recorded + +One row per message in `message_events`: sender uuid and the display name they +had at the time, the Signal send timestamp, the Warsaw day and hour, the media +kind and count, the character length, and whether it was a command or a reply. + +**Never the message body.** `tests/test_stats_events.py` asserts the column set +exactly, so a later edit cannot quietly add one. Reactions are not messages and +stay out. The bot's own posts never count — decided by which envelope branch +carried the payload, not by comparing identities, because the account is +spelled as a uuid on some frames and a +E.164 number on others. + +A photo posted without a caption counts. Before 0.1.0 it was worth zero, which +is why everyone's numbers jumped on release. + +## Days + +A day is a calendar day in **Europe/Warsaw**, computed when the event is +recorded and stored as a string. Every query groups on that string rather than +on a range of timestamps, because Warsaw days are not all 24 hours long — +2026-03-29 is 23 and 2026-10-25 is 25 — and a timestamp range moves the +boundary twice a year. + +Two consequences worth knowing before editing anything here: + +- "Yesterday" is `date - timedelta(days=1)` on the *date*. Subtracting 86400 + seconds lands at 23:00 the previous evening on a short day. +- Elapsed-time arithmetic must convert to UTC first. Subtracting two aware + datetimes that share a tzinfo gives the *wall-clock* difference, so the naive + spelling reports every day as exactly 24 hours and hides the bug. + +## The nightly report + +Posted at `REPORT_HOUR:REPORT_MINUTE` for the previous full day: the table, +members who said nothing at 0, a summary line, and up to three highlights +chosen from around twenty candidates — one per family, so a record, a share +and a crown streak about the same person do not become three lines. + +Capped at twelve lines. Names are not padded: Signal renders a proportional +font, so `ljust` lines up in a terminal and on nobody's phone. + +Only people whose display name has been seen are named. Somebody who has never +written is counted, not named — the group listing knows them only as a raw +uuid or a phone number, and printing either into a chat is a leak. + +## Outages + +Nothing is ever deleted, and the last day reported is recorded per group. + +- **Missed counts** come back on their own. An event carries Signal's own send + timestamp, signal-cli keeps receiving in json-rpc mode while the bot is + down, and the replay on reconnect is absorbed by the `(group, uuid, + sent_at_ms)` primary key. Messages sent during an outage are filed under the + day they were actually sent. +- **Missed reports** are posted when the bot returns, oldest first, bounded at + a week. A day nobody spoke on is marked settled rather than reconsidered + forever; a day whose *send* failed is left unmarked so the next run retries + it. + +## The backfilled day + +2026-08-30 in the real group is carried over from the pre-0.1.0 counter, which +had no timestamps. Its totals are real; its hours and lengths are not, and its +window was 22 hours rather than a day. It is marked in `group_meta`, claims no +average message length, and is excluded from every record and average — a +merged day would otherwise hand somebody a personal best they never set. + +The counter could not be split across the two days it spanned. Four places +were checked for a real inbound timestamp: journald (commands only), +signal-cli's `account.db` (no inbound table), Signal Desktop (unlinked, and it +does not backfill on link), and `signal-api/attachments` (dated, but media +only and with no sender). The attachment mtimes put 12 files on 29 August +against 79 on the 30th inside that window, which is why the whole counter sits +on the 30th. + +## Debugging the wire format + +`LOG_LEVEL=DEBUG` makes the dispatcher log each frame's *shape* — keys and +value types, never values, with a small allowlist for MIME and groupInfo types +that are format rather than content. Libraries stay at WARNING whatever the +bot is set to. + + journalctl --user -u signal-bot -f | grep 'envelope shape' + +Captured shapes live in `DOCS-signal-envelopes.md`. Read that before changing +anything that parses an envelope: `quote` carries its own nested `attachments` +array belonging to the *quoted* message, and counting it would credit a +replier with somebody else's photo. diff --git a/README.md b/README.md index 3509915..e359931 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,7 @@ # signal-bot -A Signal group bot: `!cat` / `!dog` / `!gekon` image commands and a daily tally. +A Signal group bot: `!cat` / `!dog` / `!gekon` image commands, `!papież`, and +group message statistics. Runs on its **own registered Signal number** (or, if you prefer, as a linked device on an existing account) via @@ -221,15 +222,14 @@ quietly posting the day's report into one group's chat. The `per_group_isolation` scenario is the gate: one person asks in both groups inside a single cooldown window and must get two images, one in each. -## Daily tally +## Statistics -Every non-sticker, non-reaction message in a watched group increments its -sender's count **for that group**. At `REPORT_HOUR:REPORT_MINUTE` (default -21:37) each group is posted its own tally, in its own chat, and reset. A group -where nobody spoke is logged and **not** posted into — otherwise every quiet -group the bot has ever been added to would get a nightly ping. A group that -refuses the report keeps its counts and does not cost the others theirs. -Counts live in SQLite and survive a restart. +One row of metadata per message — sender, Warsaw day, hour, media, length — +and **never the message body**. At `REPORT_HOUR:REPORT_MINUTE` (default 21:37) +each group gets its report for the previous full calendar day: silent members +at 0, up to three highlights. Sundays close a week, month ends a month. Ask it +things: `!staty` `!wczoraj` `!tydzień` `!miesiąc` `!rok` `!rekordy` `!korona` +`!dni` `!ja`. Full behaviour: `DOCS-statistics.md`. ## Develop