Krzysztof kuhy Rudnicki
dedc153cb4
The README still described a counter that increments and resets, which stopped being true two commits ago. The behaviour that replaced it does not fit in a README section, so it moves to DOCS-statistics.md: what is recorded and what deliberately is not, why a day is a stored Warsaw date string and what breaks if it stops being one, what happens to counts and to reports across an outage, why 2026-08-30 is marked and excluded from every average, and how to read a wire format with LOG_LEVEL=DEBUG. README keeps eight lines and a pointer, which puts it at exactly 250. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015d48t4qsQLBJnDwDeo2Nom
4.3 KiB
Statistics
What is recorded
One row per message in message_events: sender uuid and the display name they
had at the time, the Signal send timestamp, the Warsaw day and hour, the media
kind and count, the character length, and whether it was a command or a reply.
Never the message body. tests/test_stats_events.py asserts the column set
exactly, so a later edit cannot quietly add one. Reactions are not messages and
stay out. The bot's own posts never count — decided by which envelope branch
carried the payload, not by comparing identities, because the account is
spelled as a uuid on some frames and a +E.164 number on others.
A photo posted without a caption counts. Before 0.1.0 it was worth zero, which is why everyone's numbers jumped on release.
Days
A day is a calendar day in Europe/Warsaw, computed when the event is recorded and stored as a string. Every query groups on that string rather than on a range of timestamps, because Warsaw days are not all 24 hours long — 2026-03-29 is 23 and 2026-10-25 is 25 — and a timestamp range moves the boundary twice a year.
Two consequences worth knowing before editing anything here:
- "Yesterday" is
date - timedelta(days=1)on the date. Subtracting 86400 seconds lands at 23:00 the previous evening on a short day. - Elapsed-time arithmetic must convert to UTC first. Subtracting two aware datetimes that share a tzinfo gives the wall-clock difference, so the naive spelling reports every day as exactly 24 hours and hides the bug.
The nightly report
Posted at REPORT_HOUR:REPORT_MINUTE for the previous full day: the table,
members who said nothing at 0, a summary line, and up to three highlights
chosen from around twenty candidates — one per family, so a record, a share
and a crown streak about the same person do not become three lines.
Capped at twelve lines. Names are not padded: Signal renders a proportional
font, so ljust lines up in a terminal and on nobody's phone.
Only people whose display name has been seen are named. Somebody who has never written is counted, not named — the group listing knows them only as a raw uuid or a phone number, and printing either into a chat is a leak.
Outages
Nothing is ever deleted, and the last day reported is recorded per group.
- Missed counts come back on their own. An event carries Signal's own send
timestamp, signal-cli keeps receiving in json-rpc mode while the bot is
down, and the replay on reconnect is absorbed by the
(group, uuid, sent_at_ms)primary key. Messages sent during an outage are filed under the day they were actually sent. - Missed reports are posted when the bot returns, oldest first, bounded at a week. A day nobody spoke on is marked settled rather than reconsidered forever; a day whose send failed is left unmarked so the next run retries it.
The backfilled day
2026-08-30 in the real group is carried over from the pre-0.1.0 counter, which
had no timestamps. Its totals are real; its hours and lengths are not, and its
window was 22 hours rather than a day. It is marked in group_meta, claims no
average message length, and is excluded from every record and average — a
merged day would otherwise hand somebody a personal best they never set.
The counter could not be split across the two days it spanned. Four places
were checked for a real inbound timestamp: journald (commands only),
signal-cli's account.db (no inbound table), Signal Desktop (unlinked, and it
does not backfill on link), and signal-api/attachments (dated, but media
only and with no sender). The attachment mtimes put 12 files on 29 August
against 79 on the 30th inside that window, which is why the whole counter sits
on the 30th.
Debugging the wire format
LOG_LEVEL=DEBUG makes the dispatcher log each frame's shape — keys and
value types, never values, with a small allowlist for MIME and groupInfo types
that are format rather than content. Libraries stay at WARNING whatever the
bot is set to.
journalctl --user -u signal-bot -f | grep 'envelope shape'
Captured shapes live in DOCS-signal-envelopes.md. Read that before changing
anything that parses an envelope: quote carries its own nested attachments
array belonging to the quoted message, and counting it would credit a
replier with somebody else's photo.