signal-bot/signal_bot/analytics/hours.py
Krzysztof kuhy Rudnicki 405541be9b
Some checks are pending
CI / check (push) Waiting to run
CI / web (push) Waiting to run
release: 0.2.6 «Stephanus I» — dodano !analityka
One row per command attempt, not two aggregates written only on success.

command_uses holds last-use only and daily_uses counts successes; both are
rate-limiter state and both are written by store.record_use, which the
dispatcher calls only when a handler returns cleanly. A gate refusal and a
crash returned before it, so a command that failed every time looked exactly
like one nobody ran, and nothing knew which command a counted message was.

command_events now takes every attempt with its outcome, its duration and
the item it served; upstream_calls takes every outbound request made while
serving one. The two writes stay separate on purpose -- recording a refusal
as a use is what slid the cooldown window forward every time somebody was
told to wait -- and test_dispatcher_outcomes asserts last_used does not move.

Backfilled from journald joined against message_events, with daily_uses
filling the residual so the two sources cannot double-count. Two things were
wrong on the first attempt and both were found by measuring rather than by
reasoning: the window has to be symmetric, because sent_at_ms comes from the
sender's phone and the journal stamp from this host (194 of 299 lines landed
*before* their own message, by up to 3.2 s); and matching has to be a global
closest-pair assignment, because per-row greedy cannot separate two commands
seconds apart. Together those took the dropped rows from 137 to zero, and
recovered 418 invocations back to 29 August.

Also: !analityka in the group, twelve Polish lines, naming at most three
people and only ones whose display name has been seen; an Analytics tab with
Nivo charts; and continuous latency measurement, which answers the open
question in DOCS-latency.md -- a text send is 235 ms and an image send about
2.2 s, so the upload really is where the time goes.

EPHEMERAL_TABLES was doing two unrelated jobs and is now two tuples: what a
schema bump rebuilds, and what leaving a group clears. daily_uses left the
first (it is a backfill source now) and stayed in the second, and
_drop_reshaped took over the repair a bump used to do for it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T4QTp3h5Q3MvfrwGV8xqPf
2026-08-31 23:19:51 +02:00

150 lines
5.0 KiB
Python

# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Family B, the time-of-day half: when the bot is actually used.
Every query here filters ``local_hour >= 0``. That is the whole point of the
filter: ``daily_uses`` rows carry ``UNKNOWN_HOUR`` because their source knew
only a date, and letting them through would file a third of the history
under hour -1 -- or, worse, under midnight if anybody later "tidied" the
sentinel to 0.
``journald`` rows *are* included. They were matched to a real message, so
their hour was observed even though their duration was not. Excluding them
would throw away most of the history for no gain; that distinction is why
``scope`` names two predicates rather than one.
A Warsaw day is not always 24 hours -- 2026-03-29 has 23 and 2026-10-25 has
25 -- so callers must not assume the buckets sum to a day, and nothing here
divides by 24.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import TYPE_CHECKING, Final
if TYPE_CHECKING:
import sqlite3
from signal_bot.analytics.scope import Scope
HOURS_IN_DAY: Final = 24
DAYS_IN_WEEK: Final = 7
@dataclass(frozen=True, slots=True)
class Peak:
"""The busiest and quietest hours, when there is enough to say."""
busiest: int = -1
busiest_uses: int = 0
quietest: int = -1
quietest_uses: int = 0
BY_HOUR_SQL: Final = """
SELECT local_hour AS hour, COUNT(*) AS uses
FROM command_events
WHERE (:group IS NULL OR group_id = :group)
AND local_hour >= 0
AND outcome != 'unknown_word'
GROUP BY local_hour
ORDER BY local_hour
"""
BY_WEEKDAY_HOUR_SQL: Final = """
SELECT weekday, local_hour AS hour, COUNT(*) AS uses
FROM command_events
WHERE (:group IS NULL OR group_id = :group)
AND local_hour >= 0
AND weekday >= 0
AND outcome != 'unknown_word'
GROUP BY weekday, local_hour
ORDER BY weekday, local_hour
"""
BY_COMMAND_HOUR_SQL: Final = """
SELECT command, local_hour AS hour, COUNT(*) AS uses
FROM command_events
WHERE (:group IS NULL OR group_id = :group)
AND local_hour >= 0
AND outcome != 'unknown_word'
GROUP BY command, local_hour
ORDER BY command, local_hour
"""
BY_WEEKDAY_SQL: Final = """
SELECT weekday, COUNT(*) AS uses, COUNT(DISTINCT day) AS days
FROM command_events
WHERE (:group IS NULL OR group_id = :group)
AND weekday >= 0
AND outcome != 'unknown_word'
GROUP BY weekday
ORDER BY weekday
"""
def by_hour(conn: sqlite3.Connection, scope: Scope) -> list[int]:
"""Uses per hour of the day, as 24 buckets with the empty ones present.
A dense list rather than the rows sqlite returns: a bar chart with a
missing 04:00 draws the wrong shape, and "nobody uses it at four in the
morning" is one of the things the panel is for.
"""
# Indexed without a bounds check: local_hour is written by hour_of(),
# which cannot return anything outside 0-23, and the query already
# excludes the UNKNOWN_HOUR sentinel. A guard here would be dead code
# that hides a corrupt row instead of failing on one.
buckets = [0] * HOURS_IN_DAY
for row in conn.execute(BY_HOUR_SQL, scope.params()):
buckets[int(row["hour"])] = int(row["uses"])
return buckets
def by_weekday_hour(conn: sqlite3.Connection, scope: Scope) -> list[list[int]]:
"""A 7x24 grid, Monday first, matching date.weekday()."""
grid = [[0] * HOURS_IN_DAY for _ in range(DAYS_IN_WEEK)]
for row in conn.execute(BY_WEEKDAY_HOUR_SQL, scope.params()):
grid[int(row["weekday"])][int(row["hour"])] = int(row["uses"])
return grid
def by_command_hour(conn: sqlite3.Connection, scope: Scope) -> dict[str, list[int]]:
"""One 24-bucket row per command: which commands belong to which hours."""
grid: dict[str, list[int]] = {}
for row in conn.execute(BY_COMMAND_HOUR_SQL, scope.params()):
buckets = grid.setdefault(str(row["command"]), [0] * HOURS_IN_DAY)
buckets[int(row["hour"])] = int(row["uses"])
return grid
def by_weekday(conn: sqlite3.Connection, scope: Scope) -> list[tuple[int, int, int]]:
"""(weekday, uses, days seen) so an average can be taken honestly.
Dividing by the number of weeks would be wrong on a history this short:
a Tuesday that has happened once is not a weekly average.
"""
return [
(int(row["weekday"]), int(row["uses"]), int(row["days"]))
for row in conn.execute(BY_WEEKDAY_SQL, scope.params())
]
def peak(buckets: list[int]) -> Peak:
"""The busiest hour, and the quietest hour that anybody ever used.
"Quietest" skips hours with nothing at all: at three days old most of
the night is empty, and "the deadest hour is 04:00 with zero" would be
true of about nine different hours and interesting about none of them.
"""
if not any(buckets):
return Peak()
used = [(hour, uses) for hour, uses in enumerate(buckets) if uses]
busiest = max(used, key=lambda pair: (pair[1], -pair[0]))
quietest = min(used, key=lambda pair: (pair[1], pair[0]))
return Peak(
busiest=busiest[0],
busiest_uses=busiest[1],
quietest=quietest[0],
quietest_uses=quietest[1],
)