docs: describe what the bot actually does now

The README still described a counter that increments and resets, which
stopped being true two commits ago. The behaviour that replaced it does
not fit in a README section, so it moves to DOCS-statistics.md: what is
recorded and what deliberately is not, why a day is a stored Warsaw date
string and what breaks if it stops being one, what happens to counts and
to reports across an outage, why 2026-08-30 is marked and excluded from
every average, and how to read a wire format with LOG_LEVEL=DEBUG.

README keeps eight lines and a pointer, which puts it at exactly 250.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015d48t4qsQLBJnDwDeo2Nom
This commit is contained in:
Krzysztof Rudnicki 2026-08-30 19:58:25 +02:00
parent 8df2646377
commit dedc153cb4
2 changed files with 99 additions and 9 deletions

90
DOCS-statistics.md Normal file
View File

@ -0,0 +1,90 @@
# Statistics
## What is recorded
One row per message in `message_events`: sender uuid and the display name they
had at the time, the Signal send timestamp, the Warsaw day and hour, the media
kind and count, the character length, and whether it was a command or a reply.
**Never the message body.** `tests/test_stats_events.py` asserts the column set
exactly, so a later edit cannot quietly add one. Reactions are not messages and
stay out. The bot's own posts never count — decided by which envelope branch
carried the payload, not by comparing identities, because the account is
spelled as a uuid on some frames and a +E.164 number on others.
A photo posted without a caption counts. Before 0.1.0 it was worth zero, which
is why everyone's numbers jumped on release.
## Days
A day is a calendar day in **Europe/Warsaw**, computed when the event is
recorded and stored as a string. Every query groups on that string rather than
on a range of timestamps, because Warsaw days are not all 24 hours long —
2026-03-29 is 23 and 2026-10-25 is 25 — and a timestamp range moves the
boundary twice a year.
Two consequences worth knowing before editing anything here:
- "Yesterday" is `date - timedelta(days=1)` on the *date*. Subtracting 86400
seconds lands at 23:00 the previous evening on a short day.
- Elapsed-time arithmetic must convert to UTC first. Subtracting two aware
datetimes that share a tzinfo gives the *wall-clock* difference, so the naive
spelling reports every day as exactly 24 hours and hides the bug.
## The nightly report
Posted at `REPORT_HOUR:REPORT_MINUTE` for the previous full day: the table,
members who said nothing at 0, a summary line, and up to three highlights
chosen from around twenty candidates — one per family, so a record, a share
and a crown streak about the same person do not become three lines.
Capped at twelve lines. Names are not padded: Signal renders a proportional
font, so `ljust` lines up in a terminal and on nobody's phone.
Only people whose display name has been seen are named. Somebody who has never
written is counted, not named — the group listing knows them only as a raw
uuid or a phone number, and printing either into a chat is a leak.
## Outages
Nothing is ever deleted, and the last day reported is recorded per group.
- **Missed counts** come back on their own. An event carries Signal's own send
timestamp, signal-cli keeps receiving in json-rpc mode while the bot is
down, and the replay on reconnect is absorbed by the `(group, uuid,
sent_at_ms)` primary key. Messages sent during an outage are filed under the
day they were actually sent.
- **Missed reports** are posted when the bot returns, oldest first, bounded at
a week. A day nobody spoke on is marked settled rather than reconsidered
forever; a day whose *send* failed is left unmarked so the next run retries
it.
## The backfilled day
2026-08-30 in the real group is carried over from the pre-0.1.0 counter, which
had no timestamps. Its totals are real; its hours and lengths are not, and its
window was 22 hours rather than a day. It is marked in `group_meta`, claims no
average message length, and is excluded from every record and average — a
merged day would otherwise hand somebody a personal best they never set.
The counter could not be split across the two days it spanned. Four places
were checked for a real inbound timestamp: journald (commands only),
signal-cli's `account.db` (no inbound table), Signal Desktop (unlinked, and it
does not backfill on link), and `signal-api/attachments` (dated, but media
only and with no sender). The attachment mtimes put 12 files on 29 August
against 79 on the 30th inside that window, which is why the whole counter sits
on the 30th.
## Debugging the wire format
`LOG_LEVEL=DEBUG` makes the dispatcher log each frame's *shape* — keys and
value types, never values, with a small allowlist for MIME and groupInfo types
that are format rather than content. Libraries stay at WARNING whatever the
bot is set to.
journalctl --user -u signal-bot -f | grep 'envelope shape'
Captured shapes live in `DOCS-signal-envelopes.md`. Read that before changing
anything that parses an envelope: `quote` carries its own nested `attachments`
array belonging to the *quoted* message, and counting it would credit a
replier with somebody else's photo.

View File

@ -1,6 +1,7 @@
# signal-bot
A Signal group bot: `!cat` / `!dog` / `!gekon` image commands and a daily tally.
A Signal group bot: `!cat` / `!dog` / `!gekon` image commands, `!papież`, and
group message statistics.
Runs on its **own registered Signal number** (or, if you prefer, as a linked
device on an existing account) via
@ -221,15 +222,14 @@ quietly posting the day's report into one group's chat.
The `per_group_isolation` scenario is the gate: one person asks in both groups
inside a single cooldown window and must get two images, one in each.
## Daily tally
## Statistics
Every non-sticker, non-reaction message in a watched group increments its
sender's count **for that group**. At `REPORT_HOUR:REPORT_MINUTE` (default
21:37) each group is posted its own tally, in its own chat, and reset. A group
where nobody spoke is logged and **not** posted into — otherwise every quiet
group the bot has ever been added to would get a nightly ping. A group that
refuses the report keeps its counts and does not cost the others theirs.
Counts live in SQLite and survive a restart.
One row of metadata per message — sender, Warsaw day, hour, media, length —
and **never the message body**. At `REPORT_HOUR:REPORT_MINUTE` (default 21:37)
each group gets its report for the previous full calendar day: silent members
at 0, up to three highlights. Sundays close a week, month ends a month. Ask it
things: `!staty` `!wczoraj` `!tydzień` `!miesiąc` `!rok` `!rekordy` `!korona`
`!dni` `!ja`. Full behaviour: `DOCS-statistics.md`.
## Develop