mirror of
https://github.com/kuhyx/signal-bot.git
synced 2026-11-10 19:17:02 +01:00
Krzysztof kuhy Rudnicki
c195e6b4c1
Seven numbers the group finds funny get a marker wherever they show up as
a count: the nightly table, every !stats_* view, !ja, the records and the
highlights. Percentages and shares stay bare (a 69% share is not a 69), and
so does the (N 📷) aside, so one line never carries two jokes. One table,
FUNNY in stats/text.py, and one helper, marked(), so adding a number is one
line. Documented in DOCS-statistics.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
103 lines
5.0 KiB
Markdown
103 lines
5.0 KiB
Markdown
# Statistics
|
|
|
|
This is the **message** history: who said how much, when. The **command**
|
|
history -- what people asked the bot for, whether it worked and how long it
|
|
took -- is a different table and a different package; see
|
|
`DOCS-analytics.md`.
|
|
|
|
## What is recorded
|
|
|
|
One row per message in `message_events`: sender uuid and the display name they
|
|
had at the time, the Signal send timestamp, the Warsaw day and hour, the media
|
|
kind and count, the character length, and whether it was a command or a reply.
|
|
|
|
**Never the message body.** `tests/test_stats_events.py` asserts the column set
|
|
exactly, so a later edit cannot quietly add one. Reactions are not messages and
|
|
stay out. The bot's own posts never count — decided by which envelope branch
|
|
carried the payload, not by comparing identities, because the account is
|
|
spelled as a uuid on some frames and a +E.164 number on others.
|
|
|
|
A photo posted without a caption counts. Before 0.1.0 it was worth zero, which
|
|
is why everyone's numbers jumped on release.
|
|
|
|
## Days
|
|
|
|
A day is a calendar day in **Europe/Warsaw**, computed when the event is
|
|
recorded and stored as a string. Every query groups on that string rather than
|
|
on a range of timestamps, because Warsaw days are not all 24 hours long —
|
|
2026-03-29 is 23 and 2026-10-25 is 25 — and a timestamp range moves the
|
|
boundary twice a year.
|
|
|
|
Two consequences worth knowing before editing anything here:
|
|
|
|
- "Yesterday" is `date - timedelta(days=1)` on the *date*. Subtracting 86400
|
|
seconds lands at 23:00 the previous evening on a short day.
|
|
- Elapsed-time arithmetic must convert to UTC first. Subtracting two aware
|
|
datetimes that share a tzinfo gives the *wall-clock* difference, so the naive
|
|
spelling reports every day as exactly 24 hours and hides the bug.
|
|
|
|
## The nightly report
|
|
|
|
Posted at `REPORT_HOUR:REPORT_MINUTE` for the previous full day: the table,
|
|
members who said nothing at 0, a summary line, and up to three highlights
|
|
chosen from around twenty candidates — one per family, so a record, a share
|
|
and a crown streak about the same person do not become three lines.
|
|
|
|
Capped at twelve lines. Names are not padded: Signal renders a proportional
|
|
font, so `ljust` lines up in a terminal and on nobody's phone.
|
|
|
|
Only people whose display name has been seen are named. Somebody who has never
|
|
written is counted, not named — the group listing knows them only as a raw
|
|
uuid or a phone number, and printing either into a chat is a leak.
|
|
|
|
Seven numbers are funny to this group and get a marker wherever they appear
|
|
as a *count* — `69 (nice)`, `2137 🙏`, `1410 ⚔️`, `1916 🇵🇱`, `420 🌿`,
|
|
`314 🥧`, `67 🤷` — in the nightly report, every `!stats_*` and `!ja`, and the
|
|
highlights. Percentages, shares and the `(N 📷)` aside stay bare: a 69% share
|
|
is not a 69, and marking the photo count next to an already-marked message
|
|
count would be two jokes on one line. The table is `FUNNY` in `stats/text.py`.
|
|
|
|
## Outages
|
|
|
|
Nothing is ever deleted, and the last day reported is recorded per group.
|
|
|
|
- **Missed counts** come back on their own. An event carries Signal's own send
|
|
timestamp, signal-cli keeps receiving in json-rpc mode while the bot is
|
|
down, and the replay on reconnect is absorbed by the `(group, uuid,
|
|
sent_at_ms)` primary key. Messages sent during an outage are filed under the
|
|
day they were actually sent.
|
|
- **Missed reports** are posted when the bot returns, oldest first, bounded at
|
|
a week. A day nobody spoke on is marked settled rather than reconsidered
|
|
forever; a day whose *send* failed is left unmarked so the next run retries
|
|
it.
|
|
|
|
## The backfilled day
|
|
|
|
2026-08-30 in the real group is carried over from the pre-0.1.0 counter, which
|
|
had no timestamps. Its totals are real; its hours and lengths are not, and its
|
|
window was 22 hours rather than a day. It is marked in `group_meta`, claims no
|
|
average message length, and is excluded from every record and average — a
|
|
merged day would otherwise hand somebody a personal best they never set.
|
|
|
|
The counter could not be split across the two days it spanned. Four places
|
|
were checked for a real inbound timestamp: journald (commands only),
|
|
signal-cli's `account.db` (no inbound table), Signal Desktop (unlinked, and it
|
|
does not backfill on link), and `signal-api/attachments` (dated, but media
|
|
only and with no sender). The attachment mtimes put 12 files on 29 August
|
|
against 79 on the 30th inside that window, which is why the whole counter sits
|
|
on the 30th.
|
|
|
|
## Debugging the wire format
|
|
|
|
`LOG_LEVEL=DEBUG` makes the dispatcher log each frame's *shape* — keys and
|
|
value types, never values, with a small allowlist for MIME and groupInfo types
|
|
that are format rather than content. Libraries stay at WARNING whatever the
|
|
bot is set to.
|
|
|
|
journalctl --user -u signal-bot -f | grep 'envelope shape'
|
|
|
|
Captured shapes live in `DOCS-signal-envelopes.md`. Read that before changing
|
|
anything that parses an envelope: `quote` carries its own nested `attachments`
|
|
array belonging to the *quoted* message, and counting it would credit a
|
|
replier with somebody else's photo.
|