signal-bot/signal_bot/stats/recorder.py
Krzysztof kuhy Rudnicki dc1995b558 feat: the message event log, and the nightly report built on it
The tally was one integer per person, deleted after each post. Nothing
dated survived it, so weeks, months, records, streaks and weekday
averages were all unanswerable. This replaces it with an append-only
per-message log holding metadata only -- never a message body -- and
rebuilds the nightly report on top.

What changes for the group: a photo posted without a caption now counts
(it was worth zero), stickers count, the bot's own messages no longer
do, people who said nothing appear with a 0, and the report covers the
previous full calendar day in Europe/Warsaw instead of "since the last
post" -- which used to merge two days whenever a group went quiet.

Notable decisions, each with the trap it avoids:

- Events and people are PERMANENT_TABLES: neither forget_group nor a
  future SCHEMA_VERSION bump may drop them. Both paths used to cost a
  day's counter and would now cost the entire history, and forget_group
  fires on a listing that merely omits members.
- is_own comes from which envelope branch carried the payload, not from
  comparing identities. The account is spelled as a uuid on some frames
  and a +E.164 number on others, so any comparison misses one of them.
- The roster is stashed in refresh(), not _adopt(), which returns early
  once the watched set stops changing -- i.e. always, in the steady
  state.
- The roster matches on uuid OR number because /v1/groups spells kuhy as
  +48123456789 while his messages carry an ACI uuid. Joining on uuid
  alone would have listed him twice every night, once with 0.
- Members whose display name we have never seen are counted, not named:
  their only spellings are a uuid or a phone number.
- (group, uuid, sent_at_ms) is the primary key, so a websocket
  redelivery after a reconnect cannot inflate the counts.
- Days come from a stored Warsaw date string, never a timestamp range:
  2026-03-29 is 23 hours long and 2026-10-25 is 25.
- Polish number agreement is a real rule with a 12-14 exception, so
  "44 znaki" and "45 znaków" are both spelled correctly.

config.py and startup.py were two lines under the 250-line cap and both
had to grow, so config_env and startup_retry come out of them first, as
pure moves.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015d48t4qsQLBJnDwDeo2Nom
2026-08-30 19:14:43 +02:00

68 lines
2.0 KiB
Python

# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Turning one parsed message into one row of history."""
from __future__ import annotations
from typing import TYPE_CHECKING
from signal_bot.counting import media_kind
from signal_bot.stats.events import MessageEvent
from signal_bot.stats.periods import MS_PER_SECOND, day_of, hour_of
if TYPE_CHECKING:
import datetime as dt
from signal_bot.models import IncomingMessage
from signal_bot.stats.events import EventStore
def to_event(
group_id: str,
message: IncomingMessage,
now: dt.datetime,
*,
is_command: bool,
) -> MessageEvent:
"""Reduce a message to the metadata the statistics keep.
The timestamp falls back to the clock because ``models._timestamp``
returns 0 when neither the payload nor the envelope carried one. Zero is
not merely inaccurate: it is part of the primary key, so every
zero-timestamp frame from one sender would collide and the whole run
would be recorded as a single message.
"""
sent_at_ms = message.timestamp or int(now.timestamp() * MS_PER_SECOND)
return MessageEvent(
group_id=group_id,
uuid=message.source_uuid,
display_name=message.source_name or message.source_uuid,
sent_at_ms=sent_at_ms,
day=day_of(sent_at_ms),
local_hour=hour_of(sent_at_ms),
media_kind=media_kind(message),
media_count=len(message.attachment_types) or int(message.has_sticker),
char_len=len(message.text or ""),
is_command=is_command,
is_reply=message.is_reply,
)
def record_message(
events: EventStore,
group_id: str,
message: IncomingMessage,
now: dt.datetime,
*,
is_command: bool,
) -> None:
"""Append a message to the history and refresh who its sender is."""
event = to_event(group_id, message, now, is_command=is_command)
events.record(event)
events.note_person(
group_id,
event.uuid,
event.display_name,
message.source_number,
event.day,
)