mirror of
https://github.com/kuhyx/signal-bot.git
synced 2026-11-10 20:17:02 +01:00
Krzysztof kuhy Rudnicki
dedc153cb4
The README still described a counter that increments and resets, which stopped being true two commits ago. The behaviour that replaced it does not fit in a README section, so it moves to DOCS-statistics.md: what is recorded and what deliberately is not, why a day is a stored Warsaw date string and what breaks if it stops being one, what happens to counts and to reports across an outage, why 2026-08-30 is marked and excluded from every average, and how to read a wire format with LOG_LEVEL=DEBUG. README keeps eight lines and a pointer, which puts it at exactly 250. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015d48t4qsQLBJnDwDeo2Nom
91 lines
4.3 KiB
Markdown
91 lines
4.3 KiB
Markdown
# Statistics
|
|
|
|
## What is recorded
|
|
|
|
One row per message in `message_events`: sender uuid and the display name they
|
|
had at the time, the Signal send timestamp, the Warsaw day and hour, the media
|
|
kind and count, the character length, and whether it was a command or a reply.
|
|
|
|
**Never the message body.** `tests/test_stats_events.py` asserts the column set
|
|
exactly, so a later edit cannot quietly add one. Reactions are not messages and
|
|
stay out. The bot's own posts never count — decided by which envelope branch
|
|
carried the payload, not by comparing identities, because the account is
|
|
spelled as a uuid on some frames and a +E.164 number on others.
|
|
|
|
A photo posted without a caption counts. Before 0.1.0 it was worth zero, which
|
|
is why everyone's numbers jumped on release.
|
|
|
|
## Days
|
|
|
|
A day is a calendar day in **Europe/Warsaw**, computed when the event is
|
|
recorded and stored as a string. Every query groups on that string rather than
|
|
on a range of timestamps, because Warsaw days are not all 24 hours long —
|
|
2026-03-29 is 23 and 2026-10-25 is 25 — and a timestamp range moves the
|
|
boundary twice a year.
|
|
|
|
Two consequences worth knowing before editing anything here:
|
|
|
|
- "Yesterday" is `date - timedelta(days=1)` on the *date*. Subtracting 86400
|
|
seconds lands at 23:00 the previous evening on a short day.
|
|
- Elapsed-time arithmetic must convert to UTC first. Subtracting two aware
|
|
datetimes that share a tzinfo gives the *wall-clock* difference, so the naive
|
|
spelling reports every day as exactly 24 hours and hides the bug.
|
|
|
|
## The nightly report
|
|
|
|
Posted at `REPORT_HOUR:REPORT_MINUTE` for the previous full day: the table,
|
|
members who said nothing at 0, a summary line, and up to three highlights
|
|
chosen from around twenty candidates — one per family, so a record, a share
|
|
and a crown streak about the same person do not become three lines.
|
|
|
|
Capped at twelve lines. Names are not padded: Signal renders a proportional
|
|
font, so `ljust` lines up in a terminal and on nobody's phone.
|
|
|
|
Only people whose display name has been seen are named. Somebody who has never
|
|
written is counted, not named — the group listing knows them only as a raw
|
|
uuid or a phone number, and printing either into a chat is a leak.
|
|
|
|
## Outages
|
|
|
|
Nothing is ever deleted, and the last day reported is recorded per group.
|
|
|
|
- **Missed counts** come back on their own. An event carries Signal's own send
|
|
timestamp, signal-cli keeps receiving in json-rpc mode while the bot is
|
|
down, and the replay on reconnect is absorbed by the `(group, uuid,
|
|
sent_at_ms)` primary key. Messages sent during an outage are filed under the
|
|
day they were actually sent.
|
|
- **Missed reports** are posted when the bot returns, oldest first, bounded at
|
|
a week. A day nobody spoke on is marked settled rather than reconsidered
|
|
forever; a day whose *send* failed is left unmarked so the next run retries
|
|
it.
|
|
|
|
## The backfilled day
|
|
|
|
2026-08-30 in the real group is carried over from the pre-0.1.0 counter, which
|
|
had no timestamps. Its totals are real; its hours and lengths are not, and its
|
|
window was 22 hours rather than a day. It is marked in `group_meta`, claims no
|
|
average message length, and is excluded from every record and average — a
|
|
merged day would otherwise hand somebody a personal best they never set.
|
|
|
|
The counter could not be split across the two days it spanned. Four places
|
|
were checked for a real inbound timestamp: journald (commands only),
|
|
signal-cli's `account.db` (no inbound table), Signal Desktop (unlinked, and it
|
|
does not backfill on link), and `signal-api/attachments` (dated, but media
|
|
only and with no sender). The attachment mtimes put 12 files on 29 August
|
|
against 79 on the 30th inside that window, which is why the whole counter sits
|
|
on the 30th.
|
|
|
|
## Debugging the wire format
|
|
|
|
`LOG_LEVEL=DEBUG` makes the dispatcher log each frame's *shape* — keys and
|
|
value types, never values, with a small allowlist for MIME and groupInfo types
|
|
that are format rather than content. Libraries stay at WARNING whatever the
|
|
bot is set to.
|
|
|
|
journalctl --user -u signal-bot -f | grep 'envelope shape'
|
|
|
|
Captured shapes live in `DOCS-signal-envelopes.md`. Read that before changing
|
|
anything that parses an envelope: `quote` carries its own nested `attachments`
|
|
array belonging to the *quoted* message, and counting it would credit a
|
|
replier with somebody else's photo.
|