release: 0.2.6 «Stephanus I» — dodano !analityka
Some checks are pending
CI / check (push) Waiting to run
CI / web (push) Waiting to run

One row per command attempt, not two aggregates written only on success.

command_uses holds last-use only and daily_uses counts successes; both are
rate-limiter state and both are written by store.record_use, which the
dispatcher calls only when a handler returns cleanly. A gate refusal and a
crash returned before it, so a command that failed every time looked exactly
like one nobody ran, and nothing knew which command a counted message was.

command_events now takes every attempt with its outcome, its duration and
the item it served; upstream_calls takes every outbound request made while
serving one. The two writes stay separate on purpose -- recording a refusal
as a use is what slid the cooldown window forward every time somebody was
told to wait -- and test_dispatcher_outcomes asserts last_used does not move.

Backfilled from journald joined against message_events, with daily_uses
filling the residual so the two sources cannot double-count. Two things were
wrong on the first attempt and both were found by measuring rather than by
reasoning: the window has to be symmetric, because sent_at_ms comes from the
sender's phone and the journal stamp from this host (194 of 299 lines landed
*before* their own message, by up to 3.2 s); and matching has to be a global
closest-pair assignment, because per-row greedy cannot separate two commands
seconds apart. Together those took the dropped rows from 137 to zero, and
recovered 418 invocations back to 29 August.

Also: !analityka in the group, twelve Polish lines, naming at most three
people and only ones whose display name has been seen; an Analytics tab with
Nivo charts; and continuous latency measurement, which answers the open
question in DOCS-latency.md -- a text send is 235 ms and an image send about
2.2 s, so the upload really is where the time goes.

EPHEMERAL_TABLES was doing two unrelated jobs and is now two tuples: what a
schema bump rebuilds, and what leaving a group clears. daily_uses left the
first (it is a backfill source now) and stayed in the second, and
_drop_reshaped took over the repair a bump used to do for it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T4QTp3h5Q3MvfrwGV8xqPf
This commit is contained in:
Krzysztof Rudnicki 2026-08-31 23:19:51 +02:00
parent 0aa38d9188
commit 405541be9b
87 changed files with 8396 additions and 79 deletions

163
DOCS-analytics.md Normal file
View File

@ -0,0 +1,163 @@
# Command analytics
`message_events` answers *who said how much*. This answers *what they asked
the bot for, when, and whether it worked* — a different table, a different
package, and one that did not exist before 0.2.6.
## Why it needed a new table
The old record was two aggregates, and neither was a history.
- `command_uses` upserts one row per (group, user, command) holding **last use
only**. It is cooldown state.
- `daily_uses` counts uses per day. It is daily-limit state.
Both are written by `store.record_use`, which the dispatcher calls **only on
success**. A gate refusal returned before it and a handler crash returned
before it, so a command that failed every single time was indistinguishable
from one nobody ran. There was no hour, no duration, and no idea which
command a counted message had been.
`command_events` takes one row per **attempt**: outcome, duration, the item
served, and the hour it happened in. `upstream_calls` takes one row per
outbound HTTP request made while serving it.
## The two writes, and why they may never be merged
```
store.commands.record(...) every attempt analytics
store.record_use(...) successes only cooldown + daily limit
```
`CommandRefusedError`'s docstring records the bug that made the split
necessary: recording a refused use slid the cooldown window forward every
time somebody was told to wait, so waiting the stated time never helped.
Now that the history takes refusals, unifying the two calls looks like an
obvious simplification. `test_dispatcher_recording` fires a refused
invocation and asserts both that the history grew **and** that `last_used`
did not move. That test is the whole guard.
## What is never stored
| | |
|---|---|
| the words after a command | `args` holds a registry-resolved pack key (`p1`, `add`, `random`) or `""` / `"+args"`. A user-typed column is the shape that put real names in this repository once. |
| an exception's message | `error_class` is a class name. `str(error)` carries whatever was being processed, and here that is somebody's message. |
| a URL | `upstream_calls.host` is a hostname. A URL carries identifiers, and identifiers belong in `item_id` where a decision governs them. |
| a title or a caption | `item_id` is a QID, a Commons page id, a Wolne Lektury slug or a content hash. An identifier cannot carry a name; free text eventually will. |
## Three kinds of row
`source` says where a row came from, and the three are not interchangeable.
| `source` | group & person | hour | duration | outcome |
|---|---|---|---|---|
| `live` | observed | observed | observed | observed |
| `journald` | observed | observed | **none** | **inferred** |
| `daily_uses` | observed | **none** (`-1`) | **none** | assumed `ok` |
`scope.py` is the only place that names the filters, so "is the exclusion
still there?" is one grep rather than six independent remembers. Hour and
weekday panels take `local_hour >= 0`; latency panels take
`handler_ms IS NOT NULL`; sessions and co-occurrence take a real hour,
because a `daily_uses` timestamp is built from midnight and treating those as
a sequence invents a sitting of forty commands in forty milliseconds.
Durations are `NULL`, never `0`. A zero reads as data and would drag a p50
down — the same lesson `stats/backfill.UNKNOWN_HOUR` already records.
## The backfill
`tool/backfill_analytics.py` recovers the history that predates the table.
Run it against a **copy** first. It is insert-only and re-runnable; there is
no delete-then-rebuild pass, because journald has no retention configuration
and a second run after the log had rotated would delete rows it could no
longer replace.
Two sources that overlap, which is the whole difficulty — summing them would
roughly double every historical count and nothing would fail:
- **journald** knows which command ran and when, but has no group and no
uuid: `dispatcher.py` logs a display name, because the line was written to
be read by a human.
- **daily_uses** knows the group, the uuid and the day, and counts correctly,
but has no times of day at all.
So the join runs **over `message_events`**, which already holds the group,
the uuid, the real hour and the display name of every counted command, and
lacks only the command's name — which is exactly what journald has. Then
`daily_uses` fills the **residual**: `max(0, uses − matched)` per (group,
user, command, day). Nothing is counted twice.
### Two things that were wrong on the first attempt
**The window is symmetric.** The log line is written strictly after the
message arrives, so `[0, +15 s]` looks obviously right. Measured against the
live database: of 299 counted commands, **194 had a journal line earlier than
their own `sent_at_ms`**, by up to 3.2 seconds. The two stamps come from two
different clocks — `sent_at_ms` is set on the *sender's phone*, the journal
stamp is this host's — so their difference is skew and has either sign. The
asymmetric window dropped 47% of the data.
**Matching is a global assignment, not per-row.** Two commands from one
person seconds apart put both log lines inside both windows; picking
best-for-each-row in turn cannot tell them apart and drops both. Closest-pair
first, globally, can: the skew is systematic per sender, so the true pairs are
the two smallest distances available. On the live database that took the
drops from 137 to **zero**.
### One inaccuracy, stated rather than hidden
`dispatcher.py` logs `running command X` **before** the handler runs, so a
journald row proves the command was *entered*, not that it finished. A
handler that then crashed, or a `daily_limit`/`vote_gated` refusal — neither
of which logged anything — is imported as `ok`. Measured on 2026-08-31 the
gap is about 4%. Only `live` rows have outcomes that were actually observed,
and the page says which is which.
## `!analityka`
Text only, Polish, twelve lines as a cap rather than a quota — a line with
nothing to say is dropped, not printed empty. No padding, because Signal
renders a proportional font.
It names at most three people, and only ones whose display name the bot has
seen; everyone else is counted and left unnamed, exactly as the nightly
report does. Full per-person matrices are on the management page. The trend
verdict is suppressed below 14 days: the series is drawn, the direction is
not claimed.
## The Analytics tab
`./web.sh`, then the first tab. Seven read-only routes under
`/api/analytics`, each taking an optional `?group=`. Absent means the
all-groups roll-up, and it runs the *same SQL* — every query is a complete
literal string with `WHERE (:group IS NULL OR group_id = :group)`, so the
roll-up and the per-group answer cannot drift. That spelling is also what
keeps ruff's S608 quiet without a suppression.
Charts are Nivo 0.99.0, always the **fixed-size** components rather than the
`Responsive*` ones: those measure their container, a jsdom container has no
size, and a suite full of charts would otherwise assert against empty divs.
`charts/size.ts` supplies a real measurement in a browser and a stated
fallback everywhere else. Colours are `@kuhyx/web-ui` tokens, not a Nivo
scheme.
## Verifying a change
```sh
.venv/bin/python -m pytest -q # 100%, no omits
./scripts/e2e.sh # incl. analytics_command.yaml
pnpm --dir web lint && pnpm --dir web coverage # 100% on all four metrics
pre-commit run --all-files
# the backfill, against a copy, never the live file
cp signal_bot.sqlite3 /tmp/check.sqlite3
journalctl --user -u signal-bot --no-pager -o short-iso > /tmp/bot.log
python tool/backfill_analytics.py --db /tmp/check.sqlite3 --log /tmp/bot.log --dry-run
```
CI runs `pnpm lint` and `pnpm coverage` but **never `pnpm build`**, so a
bundling or CSP break is invisible to every automated gate. Building and
opening the page is a required manual step, not an optional one.

View File

@ -71,6 +71,15 @@ this repo: describe what changes for them. 0.0.1 shipped a bullet about
`/healthz`, an endpoint bound to 127.0.0.1 that nobody in a Signal group can
reach, and it was rightly called out.
## 0.2.6 — Stephanus I
papież nr 23 z 267, pontyfikat 254-257
- nowa komenda `!analityka` — kto ile używa bota, o której godzinie i które komendy nikomu się nie przydają
- bot pamięta teraz każde użycie komendy, także to nieudane; wcześniej komenda, która zawsze się psuła, wyglądała jak taka, której nikt nie używa
- historia sprzed tej wersji została odtworzona z dziennika systemowego, więc statystyki zaczynają się od 29 sierpnia, a nie od dzisiaj
- strona zarządzania ma nową zakładkę z wykresami
## 0.2.5 — Lucius I
papież nr 22 z 267, pontyfikat 253-254

View File

@ -28,6 +28,7 @@ This file is the trigger reference — every spelling that resolves, which
| stats_crown | `!korona` `!crown` |
| stats_weekdays | `!dni` `!days` |
| stats_me | `!ja` `!moje` `!mnie` |
| command_stats | `!analityka` `!command_stats` `!commandstats` `!analytics` `!uzycie` `!użycie` `!ranking` |
| help | `!pomoc` `!pomocy` `!help` `!komendy` `!commands` |
| add_photo | `!dodaj` `!dodaj_zdjecie` `!dodajzdjecie` `!add_photo` `!addphoto` |
| random | `!losowe` `!losuj` `!random` `!rand` |

View File

@ -27,6 +27,7 @@ and the star went too.
| the message database | `signal_bot.sqlite3` | `*.sqlite3` and its `-wal`/`-shm` siblings are ignored |
| the account's number, groups, inviters | `.env` | `.env` is ignored; `.env.sample` carries the shape and no values |
| everyone's names | the database and the pack manifest | nothing reads them into the repo; the bot logs `pack:p1`, never a name |
| every command anybody has run | `command_events` and `upstream_calls` in the database | the database is ignored, and the columns are chosen so there is nothing to leak: a resolved pack key or `+args` instead of the words typed, an exception's *class* instead of its message, a hostname instead of a URL, and a public identifier instead of a title. See `DOCS-analytics.md`. |
| what people typed after a `!` the bot did not know | `unknown_commands` in the database | the database is ignored; the log line names neither the word nor the message, and the export the management page writes goes to the state directory, never into this tree |
All of it is `700`/`600` on disk. The repository never needs any of it: the

View File

@ -1,5 +1,10 @@
# Statistics
This is the **message** history: who said how much, when. The **command**
history -- what people asked the bot for, whether it worked and how long it
took -- is a different table and a different package; see
`DOCS-analytics.md`.
## What is recorded
One row per message in `message_events`: sender uuid and the display name they

View File

@ -116,6 +116,23 @@ item *i* of the album while tagging item *i* of the filtered set. Anything
seen this session stays navigable even once it has tags, or tagging the photo
you are looking at would remove it from the list you are walking.
## Analytics
The first tab: what the group actually asks the bot for. Read-only, and the
only thing on this page that is worth nothing to an attacker and everything
to a maintainer -- so there is deliberately no write route under it.
`?group=` is optional on every route. Absent means the all-groups roll-up,
and it runs the *same SQL* rather than a second query that would drift.
Every panel carries the window it covers and the three source counts, because
two of the three classes of row were reconstructed rather than observed.
`DOCS-analytics.md` says what each class knows and what it does not.
Charts are Nivo, always the fixed-size components: the `Responsive*` ones
measure their container, and a jsdom container has none, so the whole test
suite would assert against empty divs.
## Ideas
The fourth tab lists what the group has asked for: `!zaproponuj` notes with
@ -153,6 +170,8 @@ other path, so a reload on a sub-view works.
| POST | `/api/album/person` | Add one person mid-session. |
| GET | `/api/album/media/{index}` | One item, **by position**. |
| GET | `/api/suggestions` | Filed ideas and unrecognised-word tallies, bodies included. |
| GET | `/api/analytics/groups` | Every group with a command history, for the picker. |
| GET | `/api/analytics/{overview,time,health,demand,behaviour,content}` | One per panel, all read-only, all taking an optional `?group=`. |
| POST | `/api/suggestions/export` | Write those to a file **outside the repository**. |
That last one is not a style choice. A filename supplied by a client is a

View File

@ -0,0 +1,17 @@
# Fresh uuid: one bot process serves every scenario and the cooldown lives in
# SQLite, so a shared actor would leak a refusal into this file.
#
# Two frames on purpose. The first is a command that leaves a row behind; the
# second asks about it. That is what makes this an end-to-end proof of the
# recording rather than of the rendering: a broken recorder answers the
# second frame with "nobody has used anything yet".
name: "!analityka answers with the history the frame before it just made"
inject:
- { text: "!gekon", uuid: uuid-nadia, name: Nadia }
- { text: "!analityka", uuid: uuid-nadia, name: Nadia }
# One expectation per send. "!gecko" rather than the heading, because the
# heading proves only that the command rendered -- naming the command from
# the frame before it proves the dispatcher actually recorded that frame.
expect:
- { kind: image, to: reply }
- { kind: text, to: reply, contains: "!gecko" }

View File

@ -1,6 +1,6 @@
[project]
name = "signal-bot"
version = "0.2.5"
version = "0.2.6"
description = "A Signal group bot: image commands and daily message tallies."
readme = "README.md"
license = "MIT"

View File

@ -129,6 +129,15 @@
"!moje",
"!mnie"
],
"command_stats": [
"!analityka",
"!command_stats",
"!commandstats",
"!analytics",
"!uzycie",
"!użycie",
"!ranking"
],
"help": [
"!pomoc",
"!pomocy",

View File

@ -0,0 +1,10 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""The command history: what was run, by whom, when, and whether it worked.
A sibling of ``stats`` rather than part of it, because the two answer
different questions from different tables. ``stats`` is the *message*
history -- who said how much -- and is what the nightly report reads.
This is the *command* history, and it exists because none of it was kept:
the old record was two aggregates written only when a command succeeded,
so a command that failed every time looked exactly like one nobody ran.
"""

View File

@ -0,0 +1,146 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Family E, the adoption half: who came back, and what a release did.
Three questions that are about the bot rather than about any one command.
* **Retention** -- of the people who used it on their first day, how many
were still using it N days later. On a history this short this is one or
two points, and the panel says so rather than drawing a curve.
* **Novelty** -- what a release actually did. ``DOCS-changelog.md`` is the
only record of when each version shipped and it is the one the maintainer
actually keeps, so the dates come from there rather than from git tags.
* **Share of chat** -- what fraction of everything said in the group was a
command. This is the only query in the package that reads
``message_events``, and it is the number that says whether the bot is a
fixture or a curiosity.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import TYPE_CHECKING, Final
from signal_bot.stats.periods import shift
if TYPE_CHECKING:
import sqlite3
from collections.abc import Sequence
from signal_bot.analytics.scope import Scope
FIRST_DAY_SQL: Final = """
SELECT user_uuid, MIN(day) AS first_day
FROM command_events
WHERE (:group IS NULL OR group_id = :group)
AND outcome = 'ok'
GROUP BY user_uuid
"""
ACTIVE_DAYS_SQL: Final = """
SELECT DISTINCT user_uuid, day
FROM command_events
WHERE (:group IS NULL OR group_id = :group)
AND outcome = 'ok'
"""
SHARE_SQL: Final = """
SELECT COUNT(*) AS messages, SUM(is_command) AS commands
FROM message_events
WHERE (:group IS NULL OR group_id = :group)
AND local_hour >= 0
"""
NEW_PEOPLE_SQL: Final = """
SELECT day, COUNT(*) AS people
FROM (
SELECT user_uuid, MIN(day) AS day
FROM command_events
WHERE (:group IS NULL OR group_id = :group)
AND outcome = 'ok'
GROUP BY user_uuid
)
GROUP BY day
ORDER BY day
"""
@dataclass(frozen=True, slots=True)
class Share:
"""How much of the group's talking is people talking to the bot."""
messages: int = 0
commands: int = 0
@property
def fraction(self) -> float:
"""Commands as a share of every counted message."""
return self.commands / self.messages if self.messages else 0.0
def retention(conn: sqlite3.Connection, scope: Scope, horizon: int) -> list[tuple[int, int, int]]:
"""(day offset, still active, cohort size) for offsets 0..horizon.
Days are stepped as dates, never as 86_400_000 milliseconds: Warsaw days
are not all the same length and the arithmetic version drifts an hour
twice a year, which would move somebody between buckets.
"""
firsts = {
str(row["user_uuid"]): str(row["first_day"])
for row in conn.execute(FIRST_DAY_SQL, scope.params())
}
if not firsts:
return []
active: set[tuple[str, str]] = {
(str(row["user_uuid"]), str(row["day"]))
for row in conn.execute(ACTIVE_DAYS_SQL, scope.params())
}
cohort = len(firsts)
return [
(
offset,
sum(1 for uuid, first in firsts.items() if (uuid, shift(first, offset)) in active),
cohort,
)
for offset in range(horizon + 1)
]
def newcomers(conn: sqlite3.Connection, scope: Scope) -> list[tuple[str, int]]:
"""(day, how many people used the bot for the first time)."""
return [
(str(row["day"]), int(row["people"]))
for row in conn.execute(NEW_PEOPLE_SQL, scope.params())
]
def share_of_chat(conn: sqlite3.Connection, scope: Scope) -> Share:
"""What fraction of the group's messages were commands.
Excludes the synthetic rows seeded from the pre-0.1.0 counter, which
carry UNKNOWN_HOUR and no is_command flag -- counting them would put a
whole day's messages in the denominator against no commands at all.
"""
# An aggregate SELECT returns one row even against an empty table, with
# NULLs in it -- so there is no None to guard against, only the NULLs.
row = conn.execute(SHARE_SQL, scope.params()).fetchone()
return Share(messages=int(row["messages"] or 0), commands=int(row["commands"] or 0))
def around_releases(
series: Sequence[tuple[str, int]], releases: Sequence[tuple[str, str]], span: int
) -> list[tuple[str, str, int, int]]:
"""(version, day, uses just after, uses just before) for each release.
"Just after" is the release day and the ``span`` days following it;
"just before" is the same number of days ending the day before. Equal
windows, so the comparison means something even at the edges of a short
history -- a release on the first recorded day gets a before-window of
zero and reads as such rather than as a spike.
"""
counts = dict(series)
out: list[tuple[str, str, int, int]] = []
for version, day in releases:
after = sum(counts.get(shift(day, offset), 0) for offset in range(span + 1))
before = sum(counts.get(shift(day, -offset), 0) for offset in range(1, span + 1))
out.append((version, day, after, before))
return out

View File

@ -0,0 +1,182 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Recovering the command history that predates the table holding it.
Two sources, and they overlap, which is the whole difficulty. Summing them
would roughly double every historical count and nothing would fail.
* **journald** knows *which* command ran and *when*, to the second, but has
no group and no uuid -- ``dispatcher.py`` logs a display name because the
line was written to be read by a human.
* **daily_uses** knows the group, the uuid, the command and the day, and
counts them correctly, but has no times of day at all.
So the join runs over ``message_events``, which already holds the group, the
uuid, the real hour and the display name of every counted command -- and
lacks only the command's *name*, which is exactly what journald has. Each
matched row is a whole truthful record. ``daily_uses`` then fills the
**residual**: for each (group, user, command, day), whatever its count says
happened beyond what the join could account for. Nothing is counted twice.
Insert-only, always. There is no delete-then-rebuild pass, because journald
has no retention configuration and will rotate: a second run after the log
has aged out would delete rows it could no longer replace. Re-runnability
comes from the natural key and ``ON CONFLICT DO NOTHING``.
One inaccuracy, stated rather than hidden. ``dispatcher.py`` logs "running
command X" *before* the handler runs, so a journald row proves the command
was entered, not that it finished. A handler that then crashed, or a
``daily_limit`` or ``vote_gated`` refusal -- neither of which logged
anything -- is imported as ``ok`` because nothing in the record says
otherwise. Measured against the live database on 2026-08-31 the gap is about
4%: 293 reconstructed invocations against 388 successes in ``daily_uses``,
overshooting by roughly fifteen rows across all commands. Every such row
carries ``source = 'journald'``, so the UI can and does say where it came
from; only the ``live`` rows have outcomes that were actually observed.
"""
from __future__ import annotations
import datetime as dt
from dataclasses import dataclass
from typing import TYPE_CHECKING, Final
from signal_bot.analytics.backfill_join import Tally, from_journal
from signal_bot.analytics.uses import (
OUTCOME_OK,
SOURCE_DAILY,
SOURCE_LIVE,
CommandEvent,
)
from signal_bot.stats.backfill import UNKNOWN_HOUR
from signal_bot.stats.periods import MS_PER_SECOND, WARSAW, weekday_of
if TYPE_CHECKING:
from collections.abc import Iterable
from signal_bot.analytics.journal import JournalLine
from signal_bot.store import Store
# Recorded per group once a pass has run, in the same place and for the same
# reason stats/backfill.py records its own marker.
BACKFILL_KEY: Final = "command_backfill"
@dataclass(frozen=True, slots=True)
class Report:
"""What one pass did, and what it refused to guess at."""
journal_rows: int = 0
daily_rows: int = 0
unmatched: int = 0
ambiguous: int = 0
groups: int = 0
def lines(self) -> tuple[str, ...]:
"""The summary a human reads. Never names anybody."""
return (
f"journald rows: {self.journal_rows}",
f"daily_uses rows: {self.daily_rows}",
f"dropped, no match: {self.unmatched}",
f"dropped, ambiguous: {self.ambiguous}",
f"groups marked: {self.groups}",
)
def midnight_ms(day: str) -> int:
"""The first millisecond of a Warsaw calendar day.
Built from the date rather than by subtracting 86_400_000 from anything:
Warsaw days are not all 24 hours, and the arithmetic spelling lands an
hour out twice a year.
"""
start = dt.datetime.combine(dt.date.fromisoformat(day), dt.time.min, tzinfo=WARSAW)
return int(start.timestamp() * MS_PER_SECOND)
def live_floor(store: Store) -> int:
"""The earliest live row, or "no live rows yet".
Every backfilled row must be strictly older than this. Without the
cutoff, a second pass run after live recording had started would import
journald lines describing invocations that already exist as ``live``
rows -- with a different source, so the natural key would not stop them.
"""
row = store.commands.connection.execute(
"SELECT MIN(invoked_at_ms) AS floor FROM command_events WHERE source = ?",
(SOURCE_LIVE,),
).fetchone()
floor = row["floor"]
# No live rows yet means nothing is off limits.
return int(floor) if floor is not None else 1 << 62
def _residual(store: Store, tally: Tally) -> list[CommandEvent]:
"""Fill in whatever daily_uses counted that the join could not explain.
``uses`` is the truth about how many times something ran that day. The
join accounts for some of them exactly; this adds the remainder at day
granularity, with no hour and no durations, so the total is right even
where the detail is missing.
"""
events: list[CommandEvent] = []
rows = store.commands.connection.execute(
"SELECT group_id, user_uuid, command, day, uses FROM daily_uses ORDER BY day, command"
).fetchall()
for row in rows:
key = (str(row["group_id"]), str(row["user_uuid"]), str(row["command"]), str(row["day"]))
remaining = int(row["uses"]) - tally.matched[key]
day = str(row["day"])
base = midnight_ms(day)
events.extend(
CommandEvent(
group_id=key[0],
user_uuid=key[1],
command=key[2],
outcome=OUTCOME_OK,
# Deterministic, so a second pass produces the identical key
# and inserts nothing. The same trick, for the same reason,
# as seed_from_counter's sent_at_ms=index + 1.
invoked_at_ms=base + index,
day=day,
local_hour=UNKNOWN_HOUR,
weekday=weekday_of(day),
source=SOURCE_DAILY,
)
for index in range(max(0, remaining))
)
return events
def plan(store: Store, lines: Iterable[JournalLine]) -> tuple[Report, list[CommandEvent]]:
"""Work out what would be imported, writing nothing.
Separate from ``apply`` so that ``--dry-run`` reports the real numbers
rather than an estimate of them. Rolling back would not have worked:
``CommandEventStore.record`` commits per row, exactly as ``EventStore``
does, so by the time a caller could roll back there is nothing left to
roll back.
"""
floor = live_floor(store)
tally = from_journal(store, lines, floor)
residual = _residual(store, tally)
events = [*tally.events, *residual]
return (
Report(
journal_rows=len(tally.events),
daily_rows=len(residual),
unmatched=tally.unmatched,
ambiguous=tally.ambiguous,
groups=len({event.group_id for event in events}),
),
events,
)
def apply(store: Store, events: Iterable[CommandEvent]) -> None:
"""Write a planned import. Insert-only; a repeat run inserts nothing."""
groups = set()
for event in events:
store.commands.record(event)
groups.add(event.group_id)
for group_id in sorted(groups):
store.set_meta(group_id, BACKFILL_KEY, "done")

View File

@ -0,0 +1,180 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Attaching a command name from the journal to a message that was one.
Split out of ``backfill.py`` so both stay under the 250-line cap. This half
is the part that can be wrong, so it is the part that refuses to guess.
The direction matters and is easy to get backwards. Iterating the journal
and looking for a message means resolving a display name to a uuid and a
group, which the journal does not know. Iterating ``message_events`` instead
means every row already carries the group, the uuid, the day and the real
hour, and the journal supplies the one field missing -- the command's name.
"""
from __future__ import annotations
from collections import Counter, defaultdict
from dataclasses import dataclass, field
from typing import TYPE_CHECKING, Final
from signal_bot.analytics.uses import (
OUTCOME_COOLDOWN,
OUTCOME_OK,
SOURCE_JOURNAL,
CommandEvent,
)
from signal_bot.stats.periods import weekday_of
if TYPE_CHECKING:
import sqlite3
from collections.abc import Iterable, Sequence
from signal_bot.analytics.journal import JournalLine
from signal_bot.store import Store
# How far apart a message and the log line about it may be and still be the
# same event.
#
# SYMMETRIC, and that is not obvious. The log line is written strictly after
# the message arrives, so an asymmetric [0, +W] window looks correct -- and
# throws away most of the data. Measured against the live database on
# 2026-08-31: of 299 counted commands, 194 have a journal line *earlier*
# than their own sent_at_ms, by up to 3.2 seconds, median -173 ms.
#
# The reason is that the two stamps come from two different clocks.
# ``sent_at_ms`` is Signal's envelope timestamp, set on the *sender's phone*;
# the journal stamp is this host's. Nothing keeps those in step, so their
# difference is skew and can have either sign.
#
# 15 seconds each way is far wider than the 3.2 s of skew actually seen and
# still narrower than a person's own repeat rate. Widening it cannot corrupt
# anything either way: two different commands inside one window are dropped
# as ambiguous rather than guessed at.
WINDOW_MS: Final = 15_000
@dataclass(slots=True)
class _Candidate:
"""One journal line, and whether the join has already spent it."""
line: JournalLine
used: bool = False
@dataclass(slots=True)
class Tally:
"""The running result of a pass."""
events: list[CommandEvent] = field(default_factory=list)
matched: Counter[tuple[str, str, str, str]] = field(default_factory=Counter)
unmatched: int = 0
ambiguous: int = 0
def _by_name(lines: Iterable[JournalLine]) -> dict[str, list[_Candidate]]:
"""Group journal lines by the display name they were logged for."""
grouped: dict[str, list[_Candidate]] = defaultdict(list)
for line in lines:
grouped[line.display_name].append(_Candidate(line))
for candidates in grouped.values():
candidates.sort(key=lambda candidate: candidate.line.at_ms)
return dict(grouped)
def _assign(
rows: Sequence[sqlite3.Row], grouped: dict[str, list[_Candidate]]
) -> dict[int, _Candidate]:
"""Pair each counted command with the journal line most likely to be it.
Closest pair first, globally, rather than best-for-each-row in turn.
That difference is what makes two commands from one person seconds apart
resolvable at all.
Per-row greedy sees both lines inside the window, cannot tell which is
which, and drops both. Global assignment can: the clock skew between a
sender's phone and this host is *systematic*, so the true pairs share
almost the same offset and are the two smallest distances available.
Ties break on row order and then line order, so a re-run against the
same inputs produces the same pairing. The backfill has to be
re-runnable, and "mostly the same" is not that.
"""
pairs: list[tuple[int, int, int]] = []
for index, row in enumerate(rows):
candidates = grouped.get(str(row["display_name"]), [])
for position, candidate in enumerate(candidates):
delta = candidate.line.at_ms - int(row["sent_at_ms"])
if abs(delta) <= WINDOW_MS:
pairs.append((abs(delta), index, position))
pairs.sort()
taken: dict[int, _Candidate] = {}
for _distance, index, position in pairs:
if index in taken:
continue
candidate = grouped[str(rows[index]["display_name"])][position]
if candidate.used:
continue
candidate.used = True
taken[index] = candidate
return taken
def _in_window(row: sqlite3.Row, grouped: dict[str, list[_Candidate]]) -> bool:
"""Whether the journal held any line at all for this message."""
sent_at_ms = int(row["sent_at_ms"])
return any(
abs(candidate.line.at_ms - sent_at_ms) <= WINDOW_MS
for candidate in grouped.get(str(row["display_name"]), [])
)
def from_journal(store: Store, lines: Iterable[JournalLine], floor: int) -> Tally:
"""Walk the counted commands and attach a name to each from the journal."""
tally = Tally()
grouped = _by_name(lines)
rows = store.events.connection.execute(
"""
SELECT group_id, uuid, display_name, sent_at_ms, day, local_hour
FROM message_events
WHERE is_command = 1 AND local_hour >= 0 AND sent_at_ms < ?
ORDER BY sent_at_ms
""",
(floor,),
).fetchall()
taken = _assign(rows, grouped)
for index, row in enumerate(rows):
chosen = taken.get(index)
if chosen is None:
# A message the journal cannot explain. Either it never held a
# line for it -- vacuumed, or older than the log -- or every line
# in its window went to a closer message, which is the honest
# reading of "two of these happened and I have one record".
# Dropped either way: a wrong attribution is worse than a gap,
# because it is indistinguishable from data.
if _in_window(row, grouped):
tally.ambiguous += 1
else:
tally.unmatched += 1
continue
line = chosen.line
outcome = OUTCOME_COOLDOWN if line.refused else OUTCOME_OK
day = str(row["day"])
tally.events.append(
CommandEvent(
group_id=str(row["group_id"]),
user_uuid=str(row["uuid"]),
command=line.command,
outcome=outcome,
invoked_at_ms=int(row["sent_at_ms"]),
day=day,
local_hour=int(row["local_hour"]),
weekday=weekday_of(day),
source=SOURCE_JOURNAL,
)
)
if outcome == OUTCOME_OK:
# Only successes are in daily_uses, so only successes may be
# subtracted from it below.
tally.matched[(str(row["group_id"]), str(row["uuid"]), line.command, day)] += 1
return tally

View File

@ -0,0 +1,106 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Family F: which items get served, and how often the same one comes back.
Reads ``item_id``, which holds public identifiers only -- a Wikidata QID, a
Commons page id, a Wolne Lektury slug, a content-addressed filename. Never a
title, a caption or a name. That constraint is what makes this family
possible at all: "the pope pool feels repetitive" is an impression, and
"106 draws produced 61 distinct popes" is a number.
The repeat rate is the headline. For a pool of size N drawn uniformly, the
expected number of distinct items after k draws is N(1 - (1 - 1/N)^k), so a
distinct count well below that means the draw is not uniform -- which is
exactly the bug that put one identical photograph into a live group chat
when iNaturalist silently ignored ``order_by=random``.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import TYPE_CHECKING, Final
if TYPE_CHECKING:
import sqlite3
from signal_bot.analytics.scope import Scope
ITEMS_SQL: Final = """
SELECT command,
COUNT(*) AS draws,
COUNT(DISTINCT item_id) AS distinct_items
FROM command_events
WHERE (:group IS NULL OR group_id = :group)
AND item_id IS NOT NULL
AND outcome = 'ok'
GROUP BY command
ORDER BY draws DESC, command
"""
TOP_ITEMS_SQL: Final = """
SELECT command, item_id, COUNT(*) AS draws
FROM command_events
WHERE (:group IS NULL OR group_id = :group)
AND item_id IS NOT NULL
AND outcome = 'ok'
GROUP BY command, item_id
HAVING draws > 1
ORDER BY draws DESC, command, item_id
"""
@dataclass(frozen=True, slots=True)
class Pool:
"""One command's draws, and how varied they turned out to be."""
command: str
draws: int
distinct: int
@property
def repeat_rate(self) -> float:
"""Share of draws that produced something already seen.
Zero means every draw was new. Approaching one means the pool is
exhausted -- or was never as large as it looked.
"""
if self.draws <= 0:
return 0.0
return 1.0 - (self.distinct / self.draws)
def pools(conn: sqlite3.Connection, scope: Scope) -> list[Pool]:
"""Every command that names what it served, most drawn first."""
return [
Pool(
command=str(row["command"]),
draws=int(row["draws"]),
distinct=int(row["distinct_items"]),
)
for row in conn.execute(ITEMS_SQL, scope.params())
]
def repeats(conn: sqlite3.Connection, scope: Scope) -> list[tuple[str, str, int]]:
"""(command, item, times) for every item served more than once.
The identifiers are safe to show -- that is the whole reason the column
may exist -- but they are only meaningful next to the command, so they
are never returned on their own.
"""
return [
(str(row["command"]), str(row["item_id"]), int(row["draws"]))
for row in conn.execute(TOP_ITEMS_SQL, scope.params())
]
def expected_distinct(pool_size: int, draws: int) -> float:
"""How many distinct items a *uniform* draw would have produced.
The comparison that turns a repeat rate into a verdict. A pool of 266
traps drawn 19 times should yield about 18.4 distinct ones; yielding 9
would mean the randomness is not what it claims, which has happened in
this repository before and reached a live group chat.
"""
if pool_size <= 0 or draws <= 0:
return 0.0
return pool_size * (1.0 - ((pool_size - 1) / pool_size) ** draws)

View File

@ -0,0 +1,102 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Family D: what the group asked for that does not exist.
The management page's Ideas tab already lists ``unknown_commands`` with the
words and the messages they came in. This adds the part it does not have:
the same words **grouped by the trigger they nearly hit**, so that "17
near-misses on !pułapka" reads as *this command wants another alias* rather
than as seventeen unrelated typos.
The grouping is done here rather than in SQL because the distance function
is ``nearest``'s, and it is optimal string alignment rather than a ratio --
a transposition costs one edit there and shreds the matching blocks a ratio
counts. Reimplementing that in SQL is not possible and approximating it
would put a different answer on the page from the one the bot gives in the
group.
"""
from __future__ import annotations
from collections import Counter
from dataclasses import dataclass
from typing import TYPE_CHECKING, Final
from signal_bot.nearest import nearest
if TYPE_CHECKING:
import sqlite3
from collections.abc import Sequence
from signal_bot.analytics.scope import Scope
WORDS_SQL: Final = """
SELECT word, COUNT(*) AS hits, MAX(sent_at_ms) AS last_at
FROM unknown_commands
WHERE (:group IS NULL OR group_id = :group)
GROUP BY word
ORDER BY hits DESC, word
"""
FILED_SQL: Final = """
SELECT COUNT(*) AS filed
FROM suggestions
WHERE (:group IS NULL OR group_id = :group)
"""
@dataclass(frozen=True, slots=True)
class NearMiss:
"""A real trigger, and how often somebody nearly typed it."""
target: str
hits: int
variants: int
@dataclass(frozen=True, slots=True)
class Demand:
"""What the group tried to type, and what it filed on purpose."""
near_misses: tuple[NearMiss, ...] = ()
unclaimed: int = 0
unclaimed_words: int = 0
filed: int = 0
def collect(conn: sqlite3.Connection, scope: Scope, triggers: Sequence[str]) -> Demand:
"""Group every unrecognised word by the command it was probably meant to be.
A word that resolves to nothing at all is counted but not attributed:
those are the genuinely new ideas, and pretending one of them is a typo
of an existing command is how a real request gets filed as noise.
The words themselves never leave this function. A half-typed name is
still a name -- ``check_personal_data`` learns the names it refuses from
the live database, so it catches ``anna`` and could never catch
``!annna`` -- and the Ideas tab is the one page that already carries
them, deliberately, and never writes them into the repository.
"""
hits: Counter[str] = Counter()
variants: Counter[str] = Counter()
unclaimed = 0
unclaimed_words = 0
for row in conn.execute(WORDS_SQL, scope.params()):
word = str(row["word"])
count = int(row["hits"])
target = nearest(word, triggers)
if target is None:
unclaimed += count
unclaimed_words += 1
continue
hits[target] += count
variants[target] += 1
filed_row = conn.execute(FILED_SQL, scope.params()).fetchone()
return Demand(
near_misses=tuple(
NearMiss(target=target, hits=count, variants=variants[target])
for target, count in sorted(hits.items(), key=lambda pair: (-pair[1], pair[0]))
),
unclaimed=unclaimed,
unclaimed_words=unclaimed_words,
filed=int(filed_row["filed"] or 0) if filed_row is not None else 0,
)

View File

@ -0,0 +1,194 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Family C: which commands are broken, slow, or refusing people.
This is the family that had *no* data at all before ``command_events``:
``store.record_use`` fires only on success, so a command that crashed every
time was indistinguishable from one nobody ran.
Latency narrows to ``live``. A reconstructed row has no duration -- NULL,
never 0, because a zero would drag a p50 down and read as a fast command.
Every aggregate here therefore says ``handler_ms IS NOT NULL`` rather than
relying on the source column, so a future third source with real timings
would be included automatically and one without would not.
"""
from __future__ import annotations
import math
from dataclasses import dataclass
from typing import TYPE_CHECKING, Final
if TYPE_CHECKING:
import sqlite3
from signal_bot.analytics.scope import Scope
# Which percentiles the panel shows. The median says what it is usually
# like; the 95th says what it is like when somebody complains.
P50: Final = 0.50
P95: Final = 0.95
@dataclass(frozen=True, slots=True)
class Latency:
"""How long one command takes, split the way DOCS-latency.md splits it."""
command: str
samples: int
handler_p50: int
handler_p95: int
send_p50: int
upstream_p50: int
@dataclass(frozen=True, slots=True)
class Upstream:
"""One external service, and how well it has been behaving."""
host: str
calls: int
failures: int
p50_ms: int
@property
def failure_rate(self) -> float:
"""Share of calls that were not a 2xx."""
return self.failures / self.calls if self.calls else 0.0
GATES_SQL: Final = """
SELECT command, outcome, COUNT(*) AS n
FROM command_events
WHERE (:group IS NULL OR group_id = :group)
AND outcome LIKE 'refused_%'
GROUP BY command, outcome
ORDER BY n DESC, command
"""
ERRORS_SQL: Final = """
SELECT command, error_class, COUNT(*) AS n
FROM command_events
WHERE (:group IS NULL OR group_id = :group)
AND outcome = 'error'
GROUP BY command, error_class
ORDER BY n DESC, command
"""
TIMED_SQL: Final = """
SELECT command, handler_ms, send_ms
FROM command_events
WHERE (:group IS NULL OR group_id = :group)
AND handler_ms IS NOT NULL
ORDER BY command, handler_ms
"""
UPSTREAM_SQL: Final = """
SELECT host,
COUNT(*) AS calls,
SUM(status < 200 OR status > 299) AS failures
FROM upstream_calls
WHERE (:group IS NULL OR group_id = :group)
GROUP BY host
ORDER BY calls DESC, host
"""
UPSTREAM_TIMES_SQL: Final = """
SELECT host, duration_ms
FROM upstream_calls
WHERE (:group IS NULL OR group_id = :group)
ORDER BY host, duration_ms
"""
COMMAND_UPSTREAM_SQL: Final = """
SELECT command, duration_ms
FROM upstream_calls
WHERE (:group IS NULL OR group_id = :group)
ORDER BY command, duration_ms
"""
def percentile(sorted_values: list[int], fraction: float) -> int:
"""The value at a fraction of the way through an already-sorted list.
Nearest-rank: ``ceil(p * n) - 1``, so the answer is always one of the
measurements rather than an interpolation between two of them. A p95
that nobody actually experienced is a worse number to act on than a
real one that is slightly off the theoretical quantile.
Spelled with ``math.ceil`` rather than ``round`` deliberately. Python
rounds halves to even, so ``round(0.5)`` is 0 and ``round(1.5)`` is 2 --
which would make the median of a two-element sample depend on the
sample's length in a way no reader would predict.
Computed here rather than in SQL: SQLite has no percentile function
without an extension, and adding one would mean a seventh dependency for
three lines of arithmetic.
"""
if not sorted_values:
return 0
rank = math.ceil(fraction * len(sorted_values)) - 1
return sorted_values[max(0, min(len(sorted_values) - 1, rank))]
def latencies(conn: sqlite3.Connection, scope: Scope) -> list[Latency]:
"""Per-command timings, from the rows that actually carry them."""
handler: dict[str, list[int]] = {}
send: dict[str, list[int]] = {}
for row in conn.execute(TIMED_SQL, scope.params()):
command = str(row["command"])
handler.setdefault(command, []).append(int(row["handler_ms"]))
if row["send_ms"] is not None:
send.setdefault(command, []).append(int(row["send_ms"]))
upstream: dict[str, list[int]] = {}
for row in conn.execute(COMMAND_UPSTREAM_SQL, scope.params()):
upstream.setdefault(str(row["command"]), []).append(int(row["duration_ms"]))
for values in (*handler.values(), *send.values(), *upstream.values()):
values.sort()
return sorted(
(
Latency(
command=command,
samples=len(values),
handler_p50=percentile(values, P50),
handler_p95=percentile(values, P95),
send_p50=percentile(send.get(command, []), P50),
upstream_p50=percentile(upstream.get(command, []), P50),
)
for command, values in handler.items()
),
key=lambda entry: (-entry.handler_p50, entry.command),
)
def gates(conn: sqlite3.Connection, scope: Scope) -> list[tuple[str, str, int]]:
"""(command, which gate, how often). A high count is mis-tuning."""
return [
(str(row["command"]), str(row["outcome"]), int(row["n"]))
for row in conn.execute(GATES_SQL, scope.params())
]
def errors(conn: sqlite3.Connection, scope: Scope) -> list[tuple[str, str, int]]:
"""(command, exception class, how often). Never an exception message."""
return [
(str(row["command"]), str(row["error_class"]), int(row["n"]))
for row in conn.execute(ERRORS_SQL, scope.params())
]
def upstreams(conn: sqlite3.Connection, scope: Scope) -> list[Upstream]:
"""Every external service the bot depends on, and how it is holding up."""
times: dict[str, list[int]] = {}
for row in conn.execute(UPSTREAM_TIMES_SQL, scope.params()):
times.setdefault(str(row["host"]), []).append(int(row["duration_ms"]))
for values in times.values():
values.sort()
return [
Upstream(
host=str(row["host"]),
calls=int(row["calls"]),
failures=int(row["failures"] or 0),
p50_ms=percentile(times.get(str(row["host"]), []), P50),
)
for row in conn.execute(UPSTREAM_SQL, scope.params())
]

View File

@ -0,0 +1,149 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Family B, the time-of-day half: when the bot is actually used.
Every query here filters ``local_hour >= 0``. That is the whole point of the
filter: ``daily_uses`` rows carry ``UNKNOWN_HOUR`` because their source knew
only a date, and letting them through would file a third of the history
under hour -1 -- or, worse, under midnight if anybody later "tidied" the
sentinel to 0.
``journald`` rows *are* included. They were matched to a real message, so
their hour was observed even though their duration was not. Excluding them
would throw away most of the history for no gain; that distinction is why
``scope`` names two predicates rather than one.
A Warsaw day is not always 24 hours -- 2026-03-29 has 23 and 2026-10-25 has
25 -- so callers must not assume the buckets sum to a day, and nothing here
divides by 24.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import TYPE_CHECKING, Final
if TYPE_CHECKING:
import sqlite3
from signal_bot.analytics.scope import Scope
HOURS_IN_DAY: Final = 24
DAYS_IN_WEEK: Final = 7
@dataclass(frozen=True, slots=True)
class Peak:
"""The busiest and quietest hours, when there is enough to say."""
busiest: int = -1
busiest_uses: int = 0
quietest: int = -1
quietest_uses: int = 0
BY_HOUR_SQL: Final = """
SELECT local_hour AS hour, COUNT(*) AS uses
FROM command_events
WHERE (:group IS NULL OR group_id = :group)
AND local_hour >= 0
AND outcome != 'unknown_word'
GROUP BY local_hour
ORDER BY local_hour
"""
BY_WEEKDAY_HOUR_SQL: Final = """
SELECT weekday, local_hour AS hour, COUNT(*) AS uses
FROM command_events
WHERE (:group IS NULL OR group_id = :group)
AND local_hour >= 0
AND weekday >= 0
AND outcome != 'unknown_word'
GROUP BY weekday, local_hour
ORDER BY weekday, local_hour
"""
BY_COMMAND_HOUR_SQL: Final = """
SELECT command, local_hour AS hour, COUNT(*) AS uses
FROM command_events
WHERE (:group IS NULL OR group_id = :group)
AND local_hour >= 0
AND outcome != 'unknown_word'
GROUP BY command, local_hour
ORDER BY command, local_hour
"""
BY_WEEKDAY_SQL: Final = """
SELECT weekday, COUNT(*) AS uses, COUNT(DISTINCT day) AS days
FROM command_events
WHERE (:group IS NULL OR group_id = :group)
AND weekday >= 0
AND outcome != 'unknown_word'
GROUP BY weekday
ORDER BY weekday
"""
def by_hour(conn: sqlite3.Connection, scope: Scope) -> list[int]:
"""Uses per hour of the day, as 24 buckets with the empty ones present.
A dense list rather than the rows sqlite returns: a bar chart with a
missing 04:00 draws the wrong shape, and "nobody uses it at four in the
morning" is one of the things the panel is for.
"""
# Indexed without a bounds check: local_hour is written by hour_of(),
# which cannot return anything outside 0-23, and the query already
# excludes the UNKNOWN_HOUR sentinel. A guard here would be dead code
# that hides a corrupt row instead of failing on one.
buckets = [0] * HOURS_IN_DAY
for row in conn.execute(BY_HOUR_SQL, scope.params()):
buckets[int(row["hour"])] = int(row["uses"])
return buckets
def by_weekday_hour(conn: sqlite3.Connection, scope: Scope) -> list[list[int]]:
"""A 7x24 grid, Monday first, matching date.weekday()."""
grid = [[0] * HOURS_IN_DAY for _ in range(DAYS_IN_WEEK)]
for row in conn.execute(BY_WEEKDAY_HOUR_SQL, scope.params()):
grid[int(row["weekday"])][int(row["hour"])] = int(row["uses"])
return grid
def by_command_hour(conn: sqlite3.Connection, scope: Scope) -> dict[str, list[int]]:
"""One 24-bucket row per command: which commands belong to which hours."""
grid: dict[str, list[int]] = {}
for row in conn.execute(BY_COMMAND_HOUR_SQL, scope.params()):
buckets = grid.setdefault(str(row["command"]), [0] * HOURS_IN_DAY)
buckets[int(row["hour"])] = int(row["uses"])
return grid
def by_weekday(conn: sqlite3.Connection, scope: Scope) -> list[tuple[int, int, int]]:
"""(weekday, uses, days seen) so an average can be taken honestly.
Dividing by the number of weeks would be wrong on a history this short:
a Tuesday that has happened once is not a weekly average.
"""
return [
(int(row["weekday"]), int(row["uses"]), int(row["days"]))
for row in conn.execute(BY_WEEKDAY_SQL, scope.params())
]
def peak(buckets: list[int]) -> Peak:
"""The busiest hour, and the quietest hour that anybody ever used.
"Quietest" skips hours with nothing at all: at three days old most of
the night is empty, and "the deadest hour is 04:00 with zero" would be
true of about nine different hours and interesting about none of them.
"""
if not any(buckets):
return Peak()
used = [(hour, uses) for hour, uses in enumerate(buckets) if uses]
busiest = max(used, key=lambda pair: (pair[1], -pair[0]))
quietest = min(used, key=lambda pair: (pair[1], pair[0]))
return Peak(
busiest=busiest[0],
busiest_uses=busiest[1],
quietest=quietest[0],
quietest_uses=quietest[1],
)

View File

@ -0,0 +1,90 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Reading the bot's own journal back into the command history.
Pure parsing, no I/O: the lines arrive as an iterable so that the joining
logic and its tests never shell out, and so that a captured log file works
exactly as well as a live ``journalctl``.
The journal is the only record of what ran before ``command_events``
existed, and it is a poor one. It carries no group id and no uuid -- only a
*display name*, because ``dispatcher.py``'s log line was written to be read
by a human. That is why the join in ``backfill`` runs the other way round,
over ``message_events``, which knows everything except which command it was.
A display name read here is a lookup key and nothing else. It never reaches
a row, a log line or a file: ``group_people`` turns it into a uuid inside
the joiner and the string is dropped. ``DOCS-privacy.md`` names the journal
as the one place display names live, and ``--vacuum-time=`` as how that is
emptied -- so this parser must never be the thing that copies them out.
"""
from __future__ import annotations
import datetime as dt
import re
from dataclasses import dataclass
from typing import TYPE_CHECKING, Final
if TYPE_CHECKING:
from collections.abc import Iterable, Iterator
MS_PER_SECOND: Final = 1000
# `journalctl -o short-iso` prefixes each line with an offset-aware timestamp
# -- 2026-08-31T21:28:26+02:00 -- which is the only spelling in the file that
# survives a DST boundary. The bot's own asctime, further along the same
# line, is naive and would be an hour wrong for half the year.
_RUNNING: Final = re.compile(r"^(?P<at>\S+)\s.*?running command (?P<command>\S+) for (?P<who>.+)$")
# The one gate that already logged its refusals. daily_limit and vote_gated
# logged nothing at all, so those simply cannot be recovered.
_REFUSED: Final = re.compile(
r"^(?P<at>\S+)\s.*?cooldown refused (?P<command>\S+) for (?P<who>.+?): told them"
)
@dataclass(frozen=True, slots=True)
class JournalLine:
"""One thing the bot said it did, with the name it said it for."""
at_ms: int
command: str
display_name: str
refused: bool
def _at_ms(stamp: str) -> int | None:
"""Parse the journal's own timestamp, or None if this is not one."""
try:
moment = dt.datetime.fromisoformat(stamp)
except ValueError:
return None
# An offset-naive stamp would silently be read as UTC and land the whole
# import two hours out, which is exactly the class of error the hour
# columns exist to avoid. Refuse it instead.
if moment.tzinfo is None:
return None
return int(moment.timestamp() * MS_PER_SECOND)
def parse(lines: Iterable[str]) -> Iterator[JournalLine]:
"""Yield one record per line the bot wrote about running a command.
Everything else -- startup banners, tracebacks, envelope shapes, other
units -- simply does not match and is skipped. The parser is deliberately
permissive about what sits between the timestamp and the phrase, because
the journal identifier is ``run.sh[<pid>]`` and the pid changes.
"""
for line in lines:
for pattern, refused in ((_RUNNING, False), (_REFUSED, True)):
found = pattern.match(line)
if found is None:
continue
at_ms = _at_ms(found["at"])
if at_ms is not None:
yield JournalLine(
at_ms=at_ms,
command=found["command"],
display_name=found["who"].strip(),
refused=refused,
)
break

View File

@ -0,0 +1,130 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""What one running command did, collected without changing any signature.
Timing a command means knowing things the handler learns and the dispatcher
records: how long the upstream fetch took, how long the send took, which
item was served. Threading that back would mean a return value or an extra
argument on every handler, every fetcher method and every reply helper.
A ``ContextVar`` avoids all of it. The dispatcher opens a probe around the
handler call; anything running underneath finds it and notes what it did.
``contextvars`` is stdlib, so the runtime stays at exactly six dependencies.
``Bot.listen`` awaits ``dispatcher.handle`` directly rather than spawning a
task per frame, so set/reset in a ``try/finally`` cannot interleave. A
handler that does spawn a task gets a *copy* of the context, which is the
behaviour wanted: the copy's notes land in the probe, and the reset in the
parent cannot be undone by the child.
Every ``note_*`` is a no-op when nothing is probing. That branch lives here,
once, so no caller needs to ask whether it is being measured -- which is
what keeps ``http_client`` and ``context`` free of analytics conditionals.
"""
from __future__ import annotations
import contextlib
import time
from contextvars import ContextVar
from dataclasses import dataclass, field
from typing import TYPE_CHECKING, Final
if TYPE_CHECKING:
from collections.abc import AsyncIterator, Iterator
# No default sentinel object: None is the whole vocabulary, and the helpers
# below are the only readers.
CURRENT: ContextVar[Probe | None] = ContextVar("command_probe", default=None)
# A request that never got an answer at all -- DNS, connect, timeout. Kept
# distinct from a 5xx, which is upstream answering and answering badly; the
# two mean different things when a command starts failing.
NO_RESPONSE: Final = 0
MS_PER_SECOND: Final = 1000
@dataclass(frozen=True, slots=True)
class UpstreamCall:
"""One outbound HTTP request made while serving a command."""
host: str
status: int
ms: int
@dataclass(slots=True)
class Probe:
"""The mutable notes one invocation accumulates.
Upstream calls are *buffered* rather than written as they happen: they
belong to a ``command_events`` row that does not exist yet, because the
outcome and the duration are only known once the handler has returned.
"""
send_ms: int = 0
item_id: str | None = None
calls: list[UpstreamCall] = field(default_factory=list)
@contextlib.contextmanager
def probing() -> Iterator[Probe]:
"""Open a probe for the duration of one command, and always close it."""
active = Probe()
token = CURRENT.set(active)
try:
yield active
finally:
CURRENT.reset(token)
def note_send(ms: int) -> None:
"""Add the cost of one outbound Signal send.
Accumulated rather than assigned: a handler may send twice, and
``!pułapka`` does -- a picture and then its description.
"""
active = CURRENT.get()
if active is not None:
active.send_ms += ms
@contextlib.asynccontextmanager
async def timed_send() -> AsyncIterator[None]:
"""Time one outbound send and note it, whether or not it succeeds.
``DOCS-latency.md`` measured ``POST /v2/send`` at 2.65 s of a 3.7 s
reply -- 72% of the wall clock, none of it this repository's code -- by
reading three log lines by hand, once. This makes that measurement
continuous, which is what the document's own rule asks for: an upstream
latency claim gets measured before it gets designed around.
"""
started = time.perf_counter()
try:
yield
finally:
note_send(round((time.perf_counter() - started) * MS_PER_SECOND))
def note_upstream(host: str, status: int, ms: int) -> None:
"""Add one upstream request. Host only -- never the URL.
A URL carries identifiers, and some of them are the ones ``item_id``
exists to govern. A hostname cannot carry anything.
"""
active = CURRENT.get()
if active is not None:
active.calls.append(UpstreamCall(host=host, status=status, ms=ms))
def note_item(item_id: str) -> None:
"""Record which item was served, as a public identifier.
A Wikidata QID, a Commons page id, a Wolne Lektury slug, a pseudonym
key, a content-addressed filename. Never a title, a caption or a name:
this is what makes "the pope pool feels repetitive" a number instead of
an impression, and an identifier cannot carry a person.
"""
active = CURRENT.get()
if active is not None:
active.item_id = item_id

View File

@ -0,0 +1,194 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Turning one attempted command into one row of history.
This is where the dispatcher's ``try/except`` around a handler now lives, so
that the dispatcher gains a ``with`` rather than forty lines and stays under
the 250-line cap. It also means the six outcome branches can be tested
directly, without driving a whole websocket frame through to reach an
exception path.
The invariant that must survive every future edit to this file: recording a
*use* and recording an *attempt* are two different writes with two different
rules, and they may never be merged.
* ``store.record_use`` is success-only and feeds the cooldown and the daily
limit. ``CommandRefusedError``'s own docstring records what happened when
a refusal reached it: the cooldown window slid forward every time somebody
was told to wait, so waiting the stated time never helped.
* ``store.commands.record`` takes every attempt, including the refusals and
the crashes -- which is the entire point, since a command that fails every
time used to be indistinguishable from one nobody ran.
The temptation this creates is to hoist one call above the other, or to route
both through a single store method. ``test_dispatcher_recording`` fires a
refused invocation and asserts both that the history grew *and* that
``last_used`` did not move, which is the cheapest way to notice.
"""
from __future__ import annotations
import contextlib
import logging
import time
from dataclasses import dataclass
from typing import TYPE_CHECKING, Final
from signal_bot.analytics.uses import (
ARGS_PRESENT,
OUTCOME_COOLDOWN,
OUTCOME_DAILY,
OUTCOME_ERROR,
OUTCOME_OK,
OUTCOME_VOTE,
CommandEvent,
)
from signal_bot.decorators import GATE_COOLDOWN, GATE_DAILY, GATE_VOTE
from signal_bot.registry import CommandRefusedError
from signal_bot.stats.periods import day_of, hour_of, weekday_of
if TYPE_CHECKING:
from collections.abc import Iterator
from signal_bot.analytics import probe
from signal_bot.analytics.uses import CommandEventStore
logger = logging.getLogger(__name__)
MS_PER_SECOND: Final = 1000
# A gate that did not name itself. Reachable only from a bare
# ``raise CommandRefusedError``, which nothing in this repository does any
# more -- but the constructor still allows it, so the mapping must too.
OUTCOME_REFUSED: Final = "refused_cooldown"
_BY_GATE: Final = {
GATE_COOLDOWN: OUTCOME_COOLDOWN,
GATE_DAILY: OUTCOME_DAILY,
GATE_VOTE: OUTCOME_VOTE,
}
def refusal_outcome(gate: str) -> str:
"""Name the outcome one gate's refusal should be filed under."""
return _BY_GATE.get(gate, OUTCOME_REFUSED)
def args_token(args: tuple[str, ...]) -> str:
"""Reduce a command's arguments to something that cannot hold a name.
The registry has already replaced a photo album's arguments with their
pseudonym keys before the dispatcher sees them -- that is what
``arg_resolver`` is for, and why it runs at resolution rather than in a
handler. Those keys are safe and worth keeping: "which person is asked
for most" is a real question.
Anything else becomes a flag. A column of words people typed is exactly
the shape that put real names in this repository once, and the personal
data gate learns the names it refuses from the live database -- so it
would catch a name typed correctly and could never catch a typo.
"""
if not args:
return ""
# One resolved token is a key. Several words are words.
if len(args) == 1 and args[0].isalnum():
return args[0]
return ARGS_PRESENT
@dataclass(slots=True)
class Attempt:
"""What the dispatcher learned by trying to run one command."""
outcome: str = OUTCOME_OK
error_class: str = ""
handler_ms: int = 0
@property
def ok(self) -> bool:
"""True when the handler actually ran to completion.
The dispatcher asks this rather than comparing to a constant, so
"did it work" has one spelling and the *use* -- the write that feeds
the cooldown -- cannot accidentally start counting refusals.
"""
return self.outcome == OUTCOME_OK
@contextlib.contextmanager
def attempting(command: str) -> Iterator[Attempt]:
"""Run one handler, and come out knowing how it went and what it cost.
Swallows the handler's exception rather than re-raising: the dispatcher
has always treated a failed command as an answered frame, because a raise
here ends the receive loop and the bot goes silently deaf until the
watchdog notices. That behaviour is unchanged -- only now the failure
leaves a row behind.
"""
attempt = Attempt()
started = time.perf_counter()
try:
yield attempt
except CommandRefusedError as refusal:
attempt.outcome = refusal_outcome(refusal.gate)
logger.debug("command %s refused by a gate", command)
except Exception as error:
# The class name, never str(error): an exception's message carries
# whatever it was processing, and here that is somebody's message.
# "17 ImageFetchError" is the whole diagnostic value anyway.
attempt.outcome = OUTCOME_ERROR
attempt.error_class = type(error).__name__
logger.exception("command %s failed", command)
attempt.handler_ms = round((time.perf_counter() - started) * MS_PER_SECOND)
@dataclass(frozen=True, slots=True)
class Invocation:
"""Who ran what, where, and when the message that asked was sent.
Grouped rather than passed as five arguments, because these five always
travel together and none of them means anything without the rest.
"""
group_id: str
user_uuid: str
command: str
args: tuple[str, ...]
sent_at_ms: int
def record_attempt(
store: CommandEventStore,
invocation: Invocation,
attempt: Attempt,
notes: probe.Probe,
) -> None:
"""Append one attempt, and never let doing so break the bot.
The whole body is guarded for the reason ``_answer_unknown``'s is: this
runs inside the receive loop, and an exception escaping it stops the bot
answering anything at all. Analytics are worth a great deal less than
that, so a storage failure is logged and the frame carries on.
"""
sent_at_ms = invocation.sent_at_ms
day = day_of(sent_at_ms)
try:
store.record(
CommandEvent(
group_id=invocation.group_id,
user_uuid=invocation.user_uuid,
command=invocation.command,
args=args_token(invocation.args),
item_id=notes.item_id,
outcome=attempt.outcome,
error_class=attempt.error_class,
invoked_at_ms=sent_at_ms,
day=day,
local_hour=hour_of(sent_at_ms),
weekday=weekday_of(day),
handler_ms=attempt.handler_ms,
send_ms=notes.send_ms,
calls=tuple(notes.calls),
)
)
except Exception:
logger.exception("could not record the use of %s", invocation.command)

View File

@ -0,0 +1,199 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""What !analityka says in the group.
Twelve lines is a cap, not a quota: a line with nothing to say is dropped
rather than printed empty, the same way the nightly report drops its
highlights. No ``ljust`` anywhere -- Signal renders a proportional font, so
padding lines up in a terminal and on nobody's phone.
Two rules about people, both inherited from ``DOCS-statistics.md``.
Only somebody whose display name has actually been seen may be named; the
group listing knows everyone else as a raw uuid or a phone number, and
printing either into a chat is a leak. And a per-person breakdown of
everybody's habits stays on the management page: the group gets a top three
and the asker's own favourite, which is what was decided when this was
specified.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import TYPE_CHECKING, Final
from signal_bot.analytics import trend as trends
from signal_bot.stats import text
from signal_bot.stats.periods import pl_day_month
if TYPE_CHECKING:
from collections.abc import Mapping, Sequence
from signal_bot.analytics.demand import Demand
from signal_bot.analytics.hours import Peak
from signal_bot.analytics.scope import Window
from signal_bot.analytics.usage import CommandUse, PersonUse
MAX_LINES: Final = 12
TOP_COMMANDS: Final = 3
TOP_PEOPLE: Final = 3
HEADING: Final = "Analityka komend"
NOTHING_YET: Final = "Nikt jeszcze nie użył żadnej komendy."
DIRECTIONS: Final = {
trends.RISING: "użycie rośnie",
trends.FALLING: "użycie spada",
trends.FLAT: "użycie się trzyma",
}
def days_phrase(count: int) -> str:
"""A day count with its noun agreeing: 3 dni."""
return f"{count} {text.plural(count, 'dzień', 'dni', 'dni')}"
def times_phrase(count: int) -> str:
"""A count with its noun agreeing: 5 razy."""
return f"{count} {text.plural(count, 'raz', 'razy', 'razy')}"
def window_line(measured: Window) -> str:
"""What the numbers actually cover. Always printed, always first.
The bot is days old, not months. A reader who cannot see that is reading
a month panel drawn from three days as though it were a month.
"""
span = days_phrase(measured.days)
if measured.first_day == measured.last_day:
return f"Okno: {span} ({pl_day_month(measured.first_day)}), {times_phrase(measured.events)}"
return (
f"Okno: {span} ({pl_day_month(measured.first_day)}"
f" — {pl_day_month(measured.last_day)}), {times_phrase(measured.events)}"
)
def top_line(uses: Sequence[CommandUse]) -> str:
"""The three most used commands, with their counts."""
best = ", ".join(f"!{use.command} {use.uses}" for use in uses[:TOP_COMMANDS])
return f"Najczęściej: {best}"
def rarest_line(uses: Sequence[CommandUse]) -> str:
"""The least used command that anybody has used at all."""
quietest = uses[-1]
return f"Najrzadziej: !{quietest.command} — {times_phrase(quietest.uses)}"
def hours_line(peak: Peak) -> str:
"""When the group actually reaches for the bot."""
return f"Najgłośniejsza godzina: {peak.busiest:02d}:00 ({times_phrase(peak.busiest_uses)})"
def trend_line(direction: str) -> str:
"""The verdict, or nothing at all when it is too early to have one.
Returning "" rather than "za mało danych" is deliberate: the window line
above already says how many days there are, and repeating it as an
apology wastes one of twelve lines.
"""
return DIRECTIONS.get(direction, "")
def health_line(uses: Sequence[CommandUse]) -> str:
"""The command most in need of attention, if any is.
Refusals count as much as errors here. A command refused half the time
is not popular, it is mis-tuned, and from the group's side both read as
"it does not work".
"""
troubled = [use for use in uses if use.errors or use.refused]
if not troubled:
return ""
worst = max(troubled, key=lambda use: (use.errors + use.refused, use.command))
# "błędów: 3" rather than "3 błędy": a label and a number needs no
# agreement at all, and Polish agreement after a numeral has three forms
# whose rule reads the last two digits. Sidestepping it here is free.
parts = []
if worst.errors:
parts.append(f"błędów: {worst.errors}")
if worst.refused:
parts.append(f"odmów: {worst.refused}")
return f"Do poprawki: !{worst.command} ({', '.join(parts)})"
def demand_line(demand: Demand) -> str:
"""The trigger people keep almost typing, which wants another alias."""
if not demand.near_misses:
return ""
miss = demand.near_misses[0]
return f"Blisko trafień: {miss.target} — {times_phrase(miss.hits)}"
def people_line(people: Sequence[PersonUse], names: Mapping[str, str]) -> str:
"""The top three, and only those whose name the bot has actually seen."""
named = [(names[person.uuid], person.uses) for person in people if person.uuid in names]
if not named:
return ""
best = ", ".join(f"{name} {uses}" for name, uses in named[:TOP_PEOPLE])
return f"Kto najwięcej: {best}"
def mine_line(people: Sequence[PersonUse], uuid: str) -> str:
"""The asker's own favourite. Self-scope, so it names nobody else."""
for person in people:
if person.uuid == uuid and person.favourite:
return f"Twoja ulubiona: !{person.favourite} ({times_phrase(person.uses)})"
return ""
def share_line(commands: int, messages: int) -> str:
"""How much of the group's talking is talking to the bot."""
if not messages:
return ""
percent = round(100 * commands / messages)
return f"Komendy to {percent}% wiadomości w tej grupie"
@dataclass(frozen=True, slots=True)
class Summary:
"""Everything the reply needs, gathered before any of it is worded.
One object rather than ten arguments, because these always travel
together: the command builds it from the query layer and the renderer
turns it into Polish, and neither half needs to know the other.
"""
measured: Window
uses: Sequence[CommandUse]
people: Sequence[PersonUse]
names: Mapping[str, str]
peak: Peak
direction: str
demand: Demand
asker: str
commands: int
messages: int
def render(summary: Summary) -> str:
"""Build the reply, capped at twelve lines and never padded."""
if not summary.uses:
return NOTHING_YET
lines = [
HEADING,
window_line(summary.measured),
top_line(summary.uses),
rarest_line(summary.uses),
hours_line(summary.peak),
trend_line(summary.direction),
share_line(summary.commands, summary.messages),
health_line(summary.uses),
demand_line(summary.demand),
people_line(summary.people, summary.names),
mine_line(summary.people, summary.asker),
]
# Dropped, then capped. A line with nothing to say is not printed empty,
# so the cap bites on content rather than on blanks -- and the cap is
# what keeps a reply readable on a phone.
kept = [line for line in lines if line][:MAX_LINES]
return "\n".join(kept)

View File

@ -0,0 +1,119 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""What a question is being asked *about*, and which rows may answer it.
Every query in this package is a complete literal string. None of them is
built by concatenation or by an f-string, which is the only arrangement ruff
S608 accepts and this repository takes no suppressions -- ``forget_group``
spells its DELETEs out for the same reason.
"One group, or all of them" is therefore not a clause that gets appended. It
is written into every query once, as::
WHERE (:group IS NULL OR group_id = :group)
Binding ``:group`` to None makes the first half true and the filter vanish.
So the roll-up and the per-group answer run *the same SQL*, which is what
stops them drifting apart -- two queries written to agree eventually will
not.
The other shared decision here is the source filter. Three classes of row
live in ``command_events``:
* ``live`` -- observed as it happened; the only rows with real durations.
* ``journald`` -- reconstructed from the log, with a real hour taken from
the message it was matched to, but no duration and an outcome that was
inferred rather than seen.
* ``daily_uses`` -- a count with no hour at all, carrying ``UNKNOWN_HOUR``.
Counting all three is right for "how often was this used" and wrong for "at
what hour" or "how slow is it". Those two predicates are named here, once,
so that the exclusion is a shared constant rather than something six modules
each have to remember. ``stats/backfill.py`` records what forgetting it
costs: a synthetic hour reached ``favourite_hour`` and the answer was
quietly wrong.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import TYPE_CHECKING, Final
if TYPE_CHECKING:
import sqlite3
# How long a person may go quiet before their next command starts a new
# session. Used only by burstiness and co-occurrence, the two questions that
# need "in one sitting" to mean anything.
#
# Ten minutes: comfortably longer than the pause between two commands
# somebody is idly firing off, far shorter than the gap between two separate
# times of day they picked their phone up.
SESSION_GAP_MS: Final = 10 * 60 * 1000
# Below this many days, a direction is not stated. The line is still drawn --
# the reader can see it -- but no arrow and no "rośnie"/"spada" verdict,
# because three points of a new bot's novelty curve are not a trend.
TREND_MIN_DAYS: Final = 14
@dataclass(frozen=True, slots=True)
class Scope:
"""One group, or all of them."""
group_id: str | None = None
def params(self, **extra: object) -> dict[str, object]:
"""The bound parameters every query in this package expects."""
return {"group": self.group_id, **extra}
@dataclass(frozen=True, slots=True)
class Window:
"""Which days an answer actually covers, and where its rows came from.
Carried on every response because this bot is days old, not months. A
month panel drawn from three days is not wrong, but a reader who cannot
see that it is three days will read it as though it were a month.
"""
first_day: str = ""
last_day: str = ""
days: int = 0
events: int = 0
live: int = 0
journald: int = 0
daily: int = 0
@property
def enough_for_a_trend(self) -> bool:
"""Whether a direction may be stated rather than merely plotted."""
return self.days >= TREND_MIN_DAYS
WINDOW_SQL: Final = """
SELECT MIN(day) AS first_day,
MAX(day) AS last_day,
COUNT(DISTINCT day) AS days,
COUNT(*) AS events,
SUM(source = 'live') AS live,
SUM(source = 'journald') AS journald,
SUM(source = 'daily_uses') AS daily
FROM command_events
WHERE (:group IS NULL OR group_id = :group)
"""
def window(conn: sqlite3.Connection, scope: Scope) -> Window:
"""Measure what the data actually covers, before anything is drawn."""
row = conn.execute(WINDOW_SQL, scope.params()).fetchone()
if row is None or not row["events"]:
return Window()
return Window(
first_day=str(row["first_day"]),
last_day=str(row["last_day"]),
days=int(row["days"]),
events=int(row["events"]),
live=int(row["live"] or 0),
journald=int(row["journald"] or 0),
daily=int(row["daily"] or 0),
)

View File

@ -0,0 +1,113 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Family E, the behavioural half: sittings, bursts and what goes together.
These are the only questions that need a *trustworthy ordering in time*, so
they are the only ones restricted to ``live`` rows plus the journald ones
that carry a real hour. A ``daily_uses`` row is a count with a synthetic
timestamp built from midnight; treating those as a sequence would invent a
sitting in which somebody ran forty commands in forty milliseconds.
A session is one person's run of commands with no gap longer than
``SESSION_GAP_MS``. It exists to make two things sayable:
* **burstiness** -- how many commands a sitting contains. Three or four
means somebody is re-rolling for a better photo, which is a fact about
the pool rather than about them.
* **co-occurrence** -- which commands turn up in the same sitting, which is
what says whether two commands are alternatives or a habit.
"""
from __future__ import annotations
from collections import Counter
from dataclasses import dataclass
from itertools import combinations
from typing import TYPE_CHECKING, Final
from signal_bot.analytics.scope import SESSION_GAP_MS
if TYPE_CHECKING:
import sqlite3
from signal_bot.analytics.scope import Scope
# Only rows whose timestamp is a real moment. daily_uses rows are excluded
# by local_hour, which is the one column that distinguishes them.
ORDERED_SQL: Final = """
SELECT user_uuid, command, invoked_at_ms
FROM command_events
WHERE (:group IS NULL OR group_id = :group)
AND local_hour >= 0
AND outcome != 'unknown_word'
ORDER BY user_uuid, invoked_at_ms
"""
@dataclass(frozen=True, slots=True)
class Sessions:
"""How the group actually uses the bot, sitting by sitting."""
count: int = 0
commands: int = 0
longest: int = 0
pairs: tuple[tuple[str, str, int], ...] = ()
repeats: tuple[tuple[str, int], ...] = ()
@property
def average_length(self) -> float:
"""Commands per sitting. One means nobody re-rolls."""
return self.commands / self.count if self.count else 0.0
def _split(rows: list[tuple[str, str, int]]) -> list[list[str]]:
"""Cut one person's ordered commands into sittings on the idle gap."""
sittings: list[list[str]] = []
current: list[str] = []
last_uuid = ""
last_at = 0
for uuid, command, at_ms in rows:
broken = uuid != last_uuid or at_ms - last_at > SESSION_GAP_MS
if broken and current:
sittings.append(current)
current = []
current.append(command)
last_uuid = uuid
last_at = at_ms
if current:
sittings.append(current)
return sittings
def collect(conn: sqlite3.Connection, scope: Scope) -> Sessions:
"""Split the history into sittings and describe what they contain."""
rows = [
(str(row["user_uuid"]), str(row["command"]), int(row["invoked_at_ms"]))
for row in conn.execute(ORDERED_SQL, scope.params())
]
sittings = _split(rows)
if not sittings:
return Sessions()
pairs: Counter[tuple[str, str]] = Counter()
repeats: Counter[str] = Counter()
for sitting in sittings:
# Sorted and de-duplicated, so (cat, dog) and (dog, cat) are one
# fact rather than two, and running !kot four times in a sitting
# does not make it co-occur with itself.
distinct = sorted(set(sitting))
for left, right in combinations(distinct, 2):
pairs[(left, right)] += 1
for command, times in Counter(sitting).items():
if times > 1:
repeats[command] += times - 1
return Sessions(
count=len(sittings),
commands=sum(len(sitting) for sitting in sittings),
longest=max(len(sitting) for sitting in sittings),
pairs=tuple(
(left, right, count)
for (left, right), count in sorted(pairs.items(), key=lambda p: (-p[1], p[0]))
),
repeats=tuple(sorted(repeats.items(), key=lambda pair: (-pair[1], pair[0]))),
)

View File

@ -0,0 +1,187 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Family B, the calendar half: day series, months, and what is dying.
Day-granular, so every source counts -- a ``daily_uses`` row has no hour but
it does know its day, which is exactly what these questions need.
The direction of a trend is deliberately not always stated. ``Window``
carries the day count and ``TREND_MIN_DAYS`` is the threshold; below it the
series is returned and drawn but ``direction`` stays ``UNKNOWN``. A bot that
is three days old has a novelty curve, not a trend, and an arrow drawn from
three points is a claim the data cannot support.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import TYPE_CHECKING, Final
from signal_bot.stats.periods import shift
if TYPE_CHECKING:
import sqlite3
from collections.abc import Sequence
from signal_bot.analytics.scope import Scope, Window
RISING: Final = "rising"
FALLING: Final = "falling"
FLAT: Final = "flat"
UNKNOWN: Final = "unknown"
# How much the second half must differ from the first to be called a
# direction rather than noise. A tenth: below that, on counts this small,
# one busy evening decides the answer.
TREND_MARGIN: Final = 0.10
# A command not used for this many days is called dying rather than merely
# quiet. Deliberately short, because the whole history is days long; it is
# a relative statement and the panel says so.
DYING_AFTER_DAYS: Final = 3
# A direction needs a before and an after to compare.
MIN_POINTS: Final = 2
BY_DAY_SQL: Final = """
SELECT day, COUNT(*) AS uses, COUNT(DISTINCT user_uuid) AS people
FROM command_events
WHERE (:group IS NULL OR group_id = :group)
AND outcome != 'unknown_word'
GROUP BY day
ORDER BY day
"""
BY_COMMAND_DAY_SQL: Final = """
SELECT command, day, COUNT(*) AS uses
FROM command_events
WHERE (:group IS NULL OR group_id = :group)
AND outcome != 'unknown_word'
GROUP BY command, day
ORDER BY command, day
"""
BY_MONTH_SQL: Final = """
SELECT substr(day, 1, 7) AS month, COUNT(*) AS uses, COUNT(DISTINCT day) AS days
FROM command_events
WHERE (:group IS NULL OR group_id = :group)
AND outcome != 'unknown_word'
GROUP BY month
ORDER BY month
"""
LAST_USED_SQL: Final = """
SELECT command, MAX(day) AS last_day, COUNT(*) AS uses
FROM command_events
WHERE (:group IS NULL OR group_id = :group)
AND outcome = 'ok'
GROUP BY command
ORDER BY last_day, command
"""
@dataclass(frozen=True, slots=True)
class Dying:
"""A command that used to be used and has gone quiet."""
command: str
last_day: str
idle_days: int
uses: int
def by_day(conn: sqlite3.Connection, scope: Scope) -> list[tuple[str, int, int]]:
"""(day, uses, people) for every day anything happened."""
return [
(str(row["day"]), int(row["uses"]), int(row["people"]))
for row in conn.execute(BY_DAY_SQL, scope.params())
]
def dense_days(series: Sequence[tuple[str, int, int]]) -> list[tuple[str, int]]:
"""Fill in the days nothing happened on.
A sparse series draws a line straight over a silent day, which reads as
"steady" when the truth is "nobody used it". Built by stepping the date
rather than by adding 86_400 seconds, because Warsaw days are not all
the same length.
"""
if not series:
return []
filled: list[tuple[str, int]] = []
day = series[0][0]
last = series[-1][0]
counts = {entry[0]: entry[1] for entry in series}
while day <= last:
filled.append((day, counts.get(day, 0)))
day = shift(day, 1)
return filled
def by_command_day(conn: sqlite3.Connection, scope: Scope) -> dict[str, dict[str, int]]:
"""Per-command day series, for the sparklines."""
series: dict[str, dict[str, int]] = {}
for row in conn.execute(BY_COMMAND_DAY_SQL, scope.params()):
series.setdefault(str(row["command"]), {})[str(row["day"])] = int(row["uses"])
return series
def by_month(conn: sqlite3.Connection, scope: Scope) -> list[tuple[str, int, int]]:
"""(month, uses, days seen). The days matter: a partial month is not low."""
return [
(str(row["month"]), int(row["uses"]), int(row["days"]))
for row in conn.execute(BY_MONTH_SQL, scope.params())
]
def direction(series: Sequence[tuple[str, int]], measured: Window) -> str:
"""Whether use is rising, falling, flat -- or too early to say.
Compares the mean of the second half against the first. Not a
least-squares fit: on a dozen points a fit is no more truthful and far
harder to explain in a chat message.
"""
if not measured.enough_for_a_trend or len(series) < MIN_POINTS:
return UNKNOWN
# MIN_POINTS above guarantees half >= 1, so neither slice is empty and
# neither division below is by zero.
half = len(series) // 2
first = [count for _day, count in series[:half]]
second = [count for _day, count in series[half:]]
before = sum(first) / len(first)
after = sum(second) / len(second)
if before == 0:
return RISING if after > 0 else FLAT
change = (after - before) / before
if change > TREND_MARGIN:
return RISING
if change < -TREND_MARGIN:
return FALLING
return FLAT
def dying(conn: sqlite3.Connection, scope: Scope, today: str) -> list[Dying]:
"""Commands that worked, were used, and then stopped being used."""
found: list[Dying] = []
for row in conn.execute(LAST_USED_SQL, scope.params()):
last_day = str(row["last_day"])
idle = _days_between(last_day, today)
if idle >= DYING_AFTER_DAYS:
found.append(
Dying(
command=str(row["command"]),
last_day=last_day,
idle_days=idle,
uses=int(row["uses"]),
)
)
return sorted(found, key=lambda entry: (-entry.idle_days, entry.command))
def _days_between(start: str, end: str) -> int:
"""Whole calendar days from one day to another, never negative."""
days = 0
day = start
while day < end:
day = shift(day, 1)
days += 1
return days

View File

@ -0,0 +1,197 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Family A: who uses which commands, how often, and which nobody uses.
Counts include every source. "How many times was !papież used" is answered
by all three classes of row -- a reconstructed invocation still happened.
Only the questions that need an hour or a duration narrow to ``live``, and
those live in ``hours`` and ``health``.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import TYPE_CHECKING, Final
if TYPE_CHECKING:
import sqlite3
from collections.abc import Sequence
from signal_bot.analytics.scope import Scope
@dataclass(frozen=True, slots=True)
class CommandUse:
"""One command, and everything countable about it."""
command: str
uses: int
people: int
ok: int
refused: int
errors: int
first_day: str
last_day: str
@property
def failure_rate(self) -> float:
"""Share of attempts that crashed. Refusals are not failures."""
return self.errors / self.uses if self.uses else 0.0
@property
def refusal_rate(self) -> float:
"""Share of attempts a gate turned away.
High is not popular, it is mis-tuned: it means people keep asking
and keep being told no.
"""
return self.refused / self.uses if self.uses else 0.0
@dataclass(frozen=True, slots=True)
class PersonUse:
"""One person's total, by uuid. Names are attached by the caller."""
uuid: str
uses: int
commands: int
favourite: str
BY_COMMAND_SQL: Final = """
SELECT command,
COUNT(*) AS uses,
COUNT(DISTINCT user_uuid) AS people,
SUM(outcome = 'ok') AS ok,
SUM(outcome LIKE 'refused_%') AS refused,
SUM(outcome = 'error') AS errors,
MIN(day) AS first_day,
MAX(day) AS last_day
FROM command_events
WHERE (:group IS NULL OR group_id = :group)
AND outcome != 'unknown_word'
GROUP BY command
ORDER BY uses DESC, command
"""
BY_PERSON_SQL: Final = """
SELECT user_uuid,
COUNT(*) AS uses,
COUNT(DISTINCT command) AS commands
FROM command_events
WHERE (:group IS NULL OR group_id = :group)
AND outcome = 'ok'
GROUP BY user_uuid
ORDER BY uses DESC, user_uuid
"""
FAVOURITE_SQL: Final = """
SELECT user_uuid, command, COUNT(*) AS uses
FROM command_events
WHERE (:group IS NULL OR group_id = :group)
AND outcome = 'ok'
GROUP BY user_uuid, command
ORDER BY user_uuid, uses DESC, command
"""
MATRIX_SQL: Final = """
SELECT user_uuid, command, COUNT(*) AS uses
FROM command_events
WHERE (:group IS NULL OR group_id = :group)
AND outcome = 'ok'
GROUP BY user_uuid, command
ORDER BY user_uuid, command
"""
TOTALS_SQL: Final = """
SELECT COUNT(*) AS attempts,
SUM(outcome = 'ok') AS ok,
COUNT(DISTINCT user_uuid) AS people,
COUNT(DISTINCT command) AS commands
FROM command_events
WHERE (:group IS NULL OR group_id = :group)
AND outcome != 'unknown_word'
"""
def by_command(conn: sqlite3.Connection, scope: Scope) -> list[CommandUse]:
"""Every command that has ever been attempted, most used first."""
return [
CommandUse(
command=str(row["command"]),
uses=int(row["uses"]),
people=int(row["people"]),
ok=int(row["ok"] or 0),
refused=int(row["refused"] or 0),
errors=int(row["errors"] or 0),
first_day=str(row["first_day"]),
last_day=str(row["last_day"]),
)
for row in conn.execute(BY_COMMAND_SQL, scope.params())
]
def _favourites(conn: sqlite3.Connection, scope: Scope) -> dict[str, str]:
"""Each person's most-used command.
The query orders by uses within each person, so the first row seen for
a uuid is the answer and the rest are skipped -- cheaper than a window
function and portable to the ancient sqlite a container might carry.
"""
best: dict[str, str] = {}
for row in conn.execute(FAVOURITE_SQL, scope.params()):
best.setdefault(str(row["user_uuid"]), str(row["command"]))
return best
def by_person(conn: sqlite3.Connection, scope: Scope) -> list[PersonUse]:
"""Who uses the bot, by uuid and most active first.
Deliberately returns uuids. Turning one into a name is the caller's job
and a different rule in each place: the group chat may name only people
whose display name has been seen, the management page may show
everyone, and neither may print a raw uuid or a phone number.
"""
favourites = _favourites(conn, scope)
return [
PersonUse(
uuid=str(row["user_uuid"]),
uses=int(row["uses"]),
commands=int(row["commands"]),
favourite=favourites.get(str(row["user_uuid"]), ""),
)
for row in conn.execute(BY_PERSON_SQL, scope.params())
]
def matrix(conn: sqlite3.Connection, scope: Scope) -> list[tuple[str, str, int]]:
"""Every (person, command, count) triple, for the heatmap on the page.
Never posted into a group -- a per-person breakdown of everybody's
habits is a management-page thing.
"""
return [
(str(row["user_uuid"]), str(row["command"]), int(row["uses"]))
for row in conn.execute(MATRIX_SQL, scope.params())
]
def totals(conn: sqlite3.Connection, scope: Scope) -> dict[str, int]:
"""Attempts, successes, distinct people and distinct commands."""
row = conn.execute(TOTALS_SQL, scope.params()).fetchone()
return {
"attempts": int(row["attempts"] or 0),
"ok": int(row["ok"] or 0),
"people": int(row["people"] or 0),
"commands": int(row["commands"] or 0),
}
def never_used(known: Sequence[str], used: Sequence[CommandUse]) -> list[str]:
"""Registered commands with no recorded attempt at all.
Takes the registry's list rather than reading it, because which commands
exist depends on the group -- the album's are registered only where the
album is -- and this module must not know that.
"""
seen = {use.command for use in used}
return sorted(name for name in known if name not in seen)

View File

@ -0,0 +1,149 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Writing the command history: one row per attempt, and what it cost.
Shares the ``Store`` connection exactly as ``EventStore`` does, so a command
and the message that triggered it are committed by the same connection in
the same order they happened.
The column set is pinned by ``test_command_events`` for the reason
``message_events``' is: this table sits beside a public repository that
leaked real names once, and a column added quietly is how the next one would
arrive. Nothing here holds free text. ``command`` is a registry name,
``args`` is a resolved token, ``item_id`` is a public identifier and
``error_class`` is an exception's class name.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import TYPE_CHECKING, Final
if TYPE_CHECKING:
import sqlite3
from collections.abc import Sequence
from signal_bot.analytics.probe import UpstreamCall
OUTCOME_OK: Final = "ok"
OUTCOME_COOLDOWN: Final = "refused_cooldown"
OUTCOME_DAILY: Final = "refused_daily"
OUTCOME_VOTE: Final = "refused_vote"
OUTCOME_ERROR: Final = "error"
OUTCOME_UNKNOWN: Final = "unknown_word"
SOURCE_LIVE: Final = "live"
SOURCE_JOURNAL: Final = "journald"
SOURCE_DAILY: Final = "daily_uses"
# What ``args`` holds when a command took arguments that resolve to nothing
# nameable. Never the words themselves: a user-typed argument column is the
# exact shape that put real names in this repository once.
ARGS_PRESENT: Final = "+args"
@dataclass(frozen=True, slots=True)
class CommandEvent:
"""One attempt to run one command, whether or not it ran."""
group_id: str
user_uuid: str
command: str
outcome: str
invoked_at_ms: int
day: str
local_hour: int
weekday: int
args: str = ""
item_id: str | None = None
error_class: str = ""
# None rather than 0: a backfilled row has no duration, and zero
# milliseconds is a lie that reads as data and would drag a p50 down.
# The lesson is already written into backfill.UNKNOWN_HOUR's comment.
handler_ms: int | None = None
send_ms: int | None = None
source: str = SOURCE_LIVE
calls: Sequence[UpstreamCall] = field(default_factory=tuple)
class CommandEventStore:
"""Reads and writes the command history. Shares the Store's connection."""
def __init__(self, conn: sqlite3.Connection) -> None:
"""Wrap an open connection whose schema has already been applied."""
self._conn = conn
@property
def connection(self) -> sqlite3.Connection:
"""The shared connection, for the query modules in this package."""
return self._conn
def record(self, event: CommandEvent) -> None:
"""Append one attempt and the upstream calls it made.
``ON CONFLICT DO NOTHING`` against the natural key is what makes a
signal-cli redelivery harmless, the same property ``message_events``
gets from its primary key -- ``invoked_at_ms`` is the envelope's own
timestamp, so a replayed frame collides with the row it already
wrote instead of adding a second one.
It is also what makes the backfill re-runnable, which is why the
backfill needs no delete-then-rebuild pass and cannot lose rows to
a journal that has since been vacuumed.
"""
self._conn.execute(
"""
INSERT INTO command_events (
group_id, user_uuid, command, args, item_id, outcome,
error_class, invoked_at_ms, day, local_hour, weekday,
handler_ms, send_ms, source
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(group_id, user_uuid, command, invoked_at_ms, source)
DO NOTHING
""",
(
event.group_id,
event.user_uuid,
event.command,
event.args,
event.item_id,
event.outcome,
event.error_class,
event.invoked_at_ms,
event.day,
event.local_hour,
event.weekday,
event.handler_ms,
event.send_ms,
event.source,
),
)
self._record_calls(event)
self._conn.commit()
def _record_calls(self, event: CommandEvent) -> None:
"""Append the upstream calls one invocation made.
Keyed on (group, timestamp, seq) rather than on the parent's rowid.
``lastrowid`` is unusable here: ``ON CONFLICT DO NOTHING`` leaves it
holding whatever the previous successful insert returned, so a
replayed frame would file its calls under an unrelated invocation.
"""
for seq, call in enumerate(event.calls):
self._conn.execute(
"""
INSERT INTO upstream_calls (
group_id, invoked_at_ms, command, host, status,
duration_ms, seq
) VALUES (?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(group_id, invoked_at_ms, seq) DO NOTHING
""",
(
event.group_id,
event.invoked_at_ms,
event.command,
call.host,
call.status,
call.ms,
seq,
),
)

View File

@ -0,0 +1,96 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""!analityka -- what the group has actually been asking the bot for.
Registered here rather than in ``stats/commands.py``, which is a few lines
under the 250 cap already, and because this reads a different table:
``stats`` is the *message* history, this is the *command* history.
The reply is text only, in Polish, capped at twelve lines. The charts live
on the management page; a rendered image in the chat would mean a new
dependency and, per ``DOCS-latency.md``, would turn a 250 ms text reply into
a three-second one for a picture nobody can zoom into.
Like ``!pomoc``, the registry is read when the command *runs* rather than
when it is registered -- at registration time the album's commands do not
exist yet, and those are exactly the triggers a near miss might be against.
"""
from __future__ import annotations
from typing import TYPE_CHECKING, Final
from signal_bot.analytics import adoption, demand, hours, render, trend, usage
from signal_bot.analytics.scope import Scope, window
from signal_bot.animals import load_aliases
if TYPE_CHECKING:
from collections.abc import Sequence
from signal_bot.context import CommandContext
from signal_bot.decorators import Decorator
from signal_bot.registry import Command, CommandRegistry, Handler
COMMAND_NAME: Final = "command_stats"
DESCRIPTION: Final = "kto ile używa bota i które komendy żyją"
def triggers_for(commands: Sequence[Command], group_id: str) -> tuple[str, ...]:
"""Every trigger this group could plausibly have meant.
Scoped to the group, and that scoping is the point: the album's triggers
are real people's names and exist only where the album does. Reporting a
near miss against one outside its group would announce that the album
exists, which is the same leak the dispatcher refuses at resolution.
"""
return tuple(
trigger
for command in commands
if command.allowed_in(group_id)
for trigger in command.triggers
)
def summarise(ctx: CommandContext, commands: Sequence[Command]) -> render.Summary:
"""Gather every number the reply needs, from the group that asked.
Scoped to one group like every other statistic here: a test group's
numbers must never turn up in the production group's answer.
"""
conn = ctx.store.commands.connection
scope = Scope(ctx.group_id)
measured = window(conn, scope)
series = trend.dense_days(trend.by_day(conn, scope))
share = adoption.share_of_chat(conn, scope)
return render.Summary(
measured=measured,
uses=usage.by_command(conn, scope),
people=usage.by_person(conn, scope),
# Only people the bot has actually seen a name for. Everyone else is
# counted and left unnamed: the roster knows them as a raw uuid or a
# phone number, and printing either into a chat is a leak.
names={
person.uuid: person.display_name for person in ctx.store.events.people(ctx.group_id)
},
peak=hours.peak(hours.by_hour(conn, scope)),
direction=trend.direction(series, measured),
demand=demand.collect(conn, scope, triggers_for(commands, ctx.group_id)),
asker=ctx.message.source_uuid,
commands=share.commands,
messages=share.messages,
)
def register_analytics_command(
registry: CommandRegistry,
*,
decorate: Decorator | None = None,
aliases: dict[str, list[str]] | None = None,
) -> None:
"""Register !analityka under every trigger aliases.json gives it."""
table = load_aliases() if aliases is None else aliases
async def handler(ctx: CommandContext) -> None:
await ctx.reply(render.render(summarise(ctx, registry.commands)))
wrapped: Handler = decorate(handler) if decorate else handler
registry.register(COMMAND_NAME, table[COMMAND_NAME], wrapped, DESCRIPTION)

View File

@ -28,6 +28,7 @@ from base64 import b64encode
from pathlib import Path
from typing import TYPE_CHECKING, Final
from signal_bot.analytics import probe
from signal_bot.animals import load_aliases
from signal_bot.commons import caption_for, imageinfo_params, page_of
from signal_bot.endpoints import THUMB_WIDTH
@ -109,9 +110,14 @@ def art_handler(
async def handler(ctx: CommandContext) -> None:
# One request, because the caption and the picture have to describe
# the same file; two would be a chance for them to disagree.
page_id = random_art(polish, world)
# A Commons page id: public, and an identifier cannot carry a
# name. It is what turns "the art pool feels repetitive" into a
# repeat rate. The caption is fetched and posted, never recorded.
probe.note_item(str(page_id))
document = await fetcher.fetch_json(
api_url,
imageinfo_params(THUMB_WIDTH, page_id=random_art(polish, world), describe=True),
imageinfo_params(THUMB_WIDTH, page_id=page_id, describe=True),
)
page = page_of(document)
image = await fetcher.fetch_image(str(page["imageinfo"][0]["thumburl"]))

View File

@ -14,6 +14,7 @@ from collections.abc import Callable
from dataclasses import dataclass
from typing import TYPE_CHECKING
from signal_bot.analytics_command import register_analytics_command
from signal_bot.animals import load_aliases, register_animal_commands
from signal_bot.announce import announce_all
from signal_bot.art_command import register_art_command
@ -205,6 +206,10 @@ def build_bot(
register_art_command(registry, fetcher, config.commons_api_url, decorate=rate_limit)
register_poem_command(registry, fetcher, config.poem_text_url, decorate=rate_limit)
register_stats_commands(registry, decorate=rate_limit)
# Reads the *command* history rather than the message history, and
# reads the registry when it runs -- so the album's commands, which
# are registered below this line, are still in the near-miss set.
register_analytics_command(registry, decorate=rate_limit)
# Rate-limited like every other command, and unlike !add_photo, whose
# "NO CAP, NO LIMITS" is a deliberate exception rather than a default.
# The cost is real and accepted: somebody told "napisz !zaproponuj ..."

View File

@ -12,6 +12,7 @@ from __future__ import annotations
from dataclasses import dataclass
from typing import TYPE_CHECKING
from signal_bot.analytics.probe import timed_send
from signal_bot.quoting import quote_of
if TYPE_CHECKING:
@ -45,13 +46,15 @@ class CommandContext:
bot watches a test group and a production group at once -- and it
quotes the asker, so a busy group can tell which "!cat" this is.
"""
await self.transport.send_text(text, self.group_id, quote_of(self.message))
async with timed_send():
await self.transport.send_text(text, self.group_id, quote_of(self.message))
async def reply_image(self, image_base64: str, caption: str = "") -> None:
"""Answer the triggering message with an image, quoting it."""
await self.transport.send_image(
image_base64, self.group_id, quote_of(self.message), caption
)
async with timed_send():
await self.transport.send_image(
image_base64, self.group_id, quote_of(self.message), caption
)
async def reply_media(self, payload_base64: str, media_type: str, filename: str) -> None:
"""Answer with an attachment that is not necessarily a photo.
@ -60,10 +63,11 @@ class CommandContext:
attachment from what it is told it is, and "a jpeg" is a lie that
makes a video arrive as a file nobody can play.
"""
await self.transport.send_media(
payload_base64,
self.group_id,
quote_of(self.message),
media_type=media_type,
filename=filename,
)
async with timed_send():
await self.transport.send_media(
payload_base64,
self.group_id,
quote_of(self.message),
media_type=media_type,
filename=filename,
)

View File

@ -17,7 +17,7 @@ import functools
import logging
import math
from collections.abc import Callable
from typing import TYPE_CHECKING
from typing import TYPE_CHECKING, Final
from signal_bot.registry import CommandRefusedError, Handler
from signal_bot.stats.text import plural
@ -29,6 +29,15 @@ logger = logging.getLogger(__name__)
Decorator = Callable[[Handler], Handler]
# Which gate said no. These are the analytics outcomes minus their prefix, so
# a refusal names the gate that caused it rather than arriving as one
# undifferentiated "refused" -- a command whose cooldown bites half the time
# is mis-tuned, and one whose vote never carries is unwanted, and the raw
# count cannot tell those apart.
GATE_COOLDOWN: Final = "cooldown"
GATE_DAILY: Final = "daily"
GATE_VOTE: Final = "vote"
# Padding on the *displayed* wait only, never on the gate itself. A number that
# is a second short sends somebody back for another refusal, which reads as the
# bot lying; a number that is a second long costs nothing.
@ -68,7 +77,7 @@ def cooldown(seconds: float, message: str) -> Decorator:
unit,
)
await ctx.reply(message.format(seconds=shown, unit=unit))
raise CommandRefusedError
raise CommandRefusedError(GATE_COOLDOWN)
await handler(ctx)
return wrapper
@ -87,7 +96,7 @@ def daily_limit(max_uses: int, message: str) -> Decorator:
)
if used >= max_uses:
await ctx.reply(message.format(max_uses=max_uses, used=used))
raise CommandRefusedError
raise CommandRefusedError(GATE_DAILY)
await handler(ctx)
return wrapper
@ -109,7 +118,7 @@ def vote_gated(required: int, window_seconds: float, message: str) -> Decorator:
votes = ctx.store.count_votes(ctx.group_id, ctx.command_name, ctx.now - window_seconds)
if votes < required:
await ctx.reply(message.format(votes=votes, required=required))
raise CommandRefusedError
raise CommandRefusedError(GATE_VOTE)
ctx.store.clear_votes(ctx.group_id, ctx.command_name)
await handler(ctx)

View File

@ -9,18 +9,19 @@ from collections.abc import Callable
from dataclasses import dataclass
from typing import TYPE_CHECKING
from signal_bot.analytics import probe, recorder
from signal_bot.context import CommandContext
from signal_bot.counting import should_count, watched_group
from signal_bot.envelope_shape import shape_of
from signal_bot.models import parse_envelope
from signal_bot.registry import CommandRefusedError, CommandRegistry
from signal_bot.stats.recorder import record_message
from signal_bot.stats.recorder import record_message, sent_at_ms_of
from signal_bot.unknown import answer_unknown
if TYPE_CHECKING:
from signal_bot.config import Config
from signal_bot.discovery import GroupDirectory
from signal_bot.models import IncomingMessage
from signal_bot.registry import CommandRegistry
from signal_bot.store import Store
from signal_bot.transport_base import Transport
@ -139,16 +140,33 @@ class Dispatcher:
day=now.date().isoformat(),
args=resolved.args if resolved is not None else (),
)
try:
# The envelope's own timestamp, not the clock, and the same value
# record_message keyed its row on -- that shared key is what lets
# the two tables be joined at all.
sent_at_ms = sent_at_ms_of(message, now)
# Two recordings, and they are deliberately not the same write.
#
# The *history* takes every attempt: a refusal and a crash are rows,
# because a command that fails every time used to be indistinguishable
# from one nobody ran. The *use* is success-only and feeds the
# cooldown -- recording a refusal there would push the wait N seconds
# further away every time somebody was told to wait N seconds.
# See analytics.recorder for the invariant and the test that holds it.
with probe.probing() as notes, recorder.attempting(command.name) as attempt:
await command.handler(ctx)
except CommandRefusedError:
# A gate said no and already answered. Nothing ran, so nothing is
# recorded -- otherwise every "wait N seconds" reply would push the
# cooldown window N seconds further away and the wait never ended.
logger.debug("command %s refused by a gate", command.name)
return
except Exception:
logger.exception("command %s failed", command.name)
recorder.record_attempt(
self.store.commands,
recorder.Invocation(
group_id=group,
user_uuid=message.source_uuid,
command=command.name,
args=ctx.args,
sent_at_ms=sent_at_ms,
),
attempt,
notes,
)
if not attempt.ok:
return
# Recorded once, here, so stacked gates cannot double-count a use.
self.store.record_use(group, message.source_uuid, command.name, ctx.now, ctx.day)

View File

@ -54,6 +54,7 @@ GROUPS: Final[tuple[tuple[str, tuple[str, ...]], ...]] = (
"stats_crown",
"stats_weekdays",
"stats_me",
"command_stats",
),
),
)

View File

@ -8,12 +8,18 @@ and the offline environment can serve fixtures without the bot knowing.
from __future__ import annotations
import base64
import time
from typing import TYPE_CHECKING, Final
import httpx
from signal_bot.analytics import probe
if TYPE_CHECKING:
from collections.abc import Sequence
import httpx
MS_PER_SECOND: Final = 1000
class ImageFetchError(RuntimeError):
@ -70,6 +76,35 @@ class ImageFetcher:
image_url = image_url.replace(*rewrite)
return await self.fetch_image(image_url)
async def _get(self, url: str, params: dict[str, str] | None = None) -> httpx.Response:
"""Make one request, time it, and report it to whatever is probing.
The single place this class touches the network, which is what makes
"how often does thecatapi fail, and how slow is Commons" answerable
without a change to any command. ``probe.note_upstream`` is a no-op
when nothing is measuring, so the offline runs and the tests are
unaffected.
Only the hostname is reported. A URL carries identifiers -- the poem
one carries a Wolne Lektury slug -- and identifiers belong in
``item_id``, where a deliberate decision governs them.
A request that never got an answer is recorded as NO_RESPONSE and
re-raised unchanged: DNS, connect and timeout failures are the ones
that never reach a status code, and losing them would make an
upstream that is entirely down look merely unused.
"""
started = time.perf_counter()
host = httpx.URL(url).host
try:
response = await self._client.get(url, params=params)
except httpx.HTTPError:
probe.note_upstream(host, probe.NO_RESPONSE, _elapsed_ms(started))
raise
probe.note_upstream(host, response.status_code, _elapsed_ms(started))
response.raise_for_status()
return response
async def fetch_json(self, api_url: str, params: dict[str, str] | None = None) -> object:
"""Fetch one JSON document from an API.
@ -78,9 +113,7 @@ class ImageFetcher:
asking twice would be both wasteful and a chance for the two halves
to disagree.
"""
response = await self._client.get(api_url, params=params)
response.raise_for_status()
return response.json()
return (await self._get(api_url, params)).json()
async def fetch_text(self, url: str) -> str:
"""Fetch one plain-text document.
@ -90,9 +123,7 @@ class ImageFetcher:
through this client like everything else, so the offline environment
and the tests serve it the same way they serve a cat.
"""
response = await self._client.get(url)
response.raise_for_status()
return response.text
return (await self._get(url)).text
async def fetch_image(self, image_url: str) -> bytes:
"""Fetch the bytes at an already-resolved image URL.
@ -103,9 +134,12 @@ class ImageFetcher:
traversal and unbounded. portraits.py does cache to disk, but under
a name it derives from the pope's ordinal, never from upstream.
"""
image = await self._client.get(image_url)
image.raise_for_status()
return image.content
return (await self._get(image_url)).content
def _elapsed_ms(started: float) -> int:
"""Milliseconds since a perf_counter reading."""
return round((time.perf_counter() - started) * MS_PER_SECOND)
_MISSING: Final = object()

View File

@ -0,0 +1,246 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""One gatherer per panel on the Analytics tab.
Split out of ``analytics_read`` so both stay under the 250-line cap. That
module owns the connection and the shapes; this one owns the six questions,
one function each, mirroring the six query modules exactly so that a panel
and its query are never more than one hop apart.
Every response carries ``window``. The bot is days old, not months, and a
month panel drawn from three days is only honest if the reader can see how
many days it is.
"""
from __future__ import annotations
from typing import TYPE_CHECKING
from signal_bot.analytics import (
adoption,
content,
demand,
health,
hours,
sessions,
trend,
usage,
)
from signal_bot.analytics.scope import window
from signal_bot.manage.analytics_read import (
Gathered,
empty,
measure,
names_in,
safely,
)
from signal_bot.stats.periods import today
if TYPE_CHECKING:
import sqlite3
from pathlib import Path
from signal_bot.analytics.scope import Scope
# How far the retention curve is drawn. Two weeks: long enough to show a
# drop-off, short enough that most of it is not empty on a young bot.
RETENTION_DAYS = 14
# The window either side of a release that "what did this change" compares.
RELEASE_SPAN_DAYS = 3
def overview(path: Path, scope: Scope) -> Gathered:
"""Family A: what is used, by whom, and what has never been touched."""
def gather(conn: sqlite3.Connection) -> Gathered:
names = names_in(conn, scope)
uses = usage.by_command(conn, scope)
return {
"window": measure(conn, scope),
"totals": usage.totals(conn, scope),
"commands": [
{
"command": use.command,
"uses": use.uses,
"people": use.people,
"ok": use.ok,
"refused": use.refused,
"errors": use.errors,
"first_day": use.first_day,
"last_day": use.last_day,
}
for use in uses
],
"people": [
{
"uuid": person.uuid,
"name": names.get(person.uuid, ""),
"uses": person.uses,
"commands": person.commands,
"favourite": person.favourite,
}
for person in usage.by_person(conn, scope)
],
"matrix": [
{"uuid": uuid, "name": names.get(uuid, ""), "command": command, "uses": count}
for uuid, command, count in usage.matrix(conn, scope)
],
}
return safely(path, gather) or {**empty("commands", "people", "matrix"), "totals": {}}
def times(path: Path, scope: Scope) -> Gathered:
"""Family B: hours, weekdays, days, months and what has gone quiet."""
def gather(conn: sqlite3.Connection) -> Gathered:
series = trend.dense_days(trend.by_day(conn, scope))
measured = window(conn, scope)
peak = hours.peak(hours.by_hour(conn, scope))
return {
"window": measure(conn, scope),
"hours": hours.by_hour(conn, scope),
"weekday_hours": hours.by_weekday_hour(conn, scope),
"by_command_hour": hours.by_command_hour(conn, scope),
"days": [{"day": day, "uses": uses} for day, uses in series],
"months": [
{"month": month, "uses": uses, "days": days}
for month, uses, days in trend.by_month(conn, scope)
],
"peak": {
"busiest": peak.busiest,
"busiest_uses": peak.busiest_uses,
"quietest": peak.quietest,
"quietest_uses": peak.quietest_uses,
},
"direction": trend.direction(series, measured),
"dying": [
{"command": one.command, "last_day": one.last_day, "idle_days": one.idle_days}
for one in trend.dying(conn, scope, today())
],
}
return safely(path, gather) or empty("hours", "days", "months", "dying")
def wellbeing(path: Path, scope: Scope) -> Gathered:
"""Family C: what is slow, what refuses people, and what upstream does."""
def gather(conn: sqlite3.Connection) -> Gathered:
return {
"window": measure(conn, scope),
"latencies": [
{
"command": one.command,
"samples": one.samples,
"handler_p50": one.handler_p50,
"handler_p95": one.handler_p95,
"send_p50": one.send_p50,
"upstream_p50": one.upstream_p50,
}
for one in health.latencies(conn, scope)
],
"gates": [
{"command": command, "gate": gate, "hits": hits}
for command, gate, hits in health.gates(conn, scope)
],
"errors": [
{"command": command, "error_class": name, "hits": hits}
for command, name, hits in health.errors(conn, scope)
],
"upstream": [
{
"host": one.host,
"calls": one.calls,
"failures": one.failures,
"p50_ms": one.p50_ms,
}
for one in health.upstreams(conn, scope)
],
}
return safely(path, gather) or empty("latencies", "gates", "errors", "upstream")
def wishes(path: Path, scope: Scope, triggers: tuple[str, ...]) -> Gathered:
"""Family D: what people nearly typed, grouped by what they nearly hit."""
def gather(conn: sqlite3.Connection) -> Gathered:
found = demand.collect(conn, scope, triggers)
return {
"window": measure(conn, scope),
"near_misses": [
{"target": miss.target, "hits": miss.hits, "variants": miss.variants}
for miss in found.near_misses
],
"unclaimed": found.unclaimed,
"unclaimed_words": found.unclaimed_words,
"filed": found.filed,
}
return safely(path, gather) or empty("near_misses")
def behaviour(path: Path, scope: Scope, releases: tuple[tuple[str, str], ...]) -> Gathered:
"""Family E: sittings, retention, releases and share of chat."""
def gather(conn: sqlite3.Connection) -> Gathered:
series = trend.dense_days(trend.by_day(conn, scope))
sitting = sessions.collect(conn, scope)
share = adoption.share_of_chat(conn, scope)
return {
"window": measure(conn, scope),
"sessions": {
"count": sitting.count,
"commands": sitting.commands,
"longest": sitting.longest,
"average_length": sitting.average_length,
},
"pairs": [
{"left": left, "right": right, "hits": hits} for left, right, hits in sitting.pairs
],
"repeats": [{"command": command, "extra": extra} for command, extra in sitting.repeats],
"retention": [
{"offset": offset, "active": active, "cohort": cohort}
for offset, active, cohort in adoption.retention(conn, scope, RETENTION_DAYS)
],
"newcomers": [
{"day": day, "people": people} for day, people in adoption.newcomers(conn, scope)
],
"releases": [
{"version": version, "day": day, "after": after, "before": before}
for version, day, after, before in adoption.around_releases(
series, releases, RELEASE_SPAN_DAYS
)
],
"share": {
"messages": share.messages,
"commands": share.commands,
"fraction": share.fraction,
},
}
return safely(path, gather) or empty("pairs", "repeats", "retention", "newcomers", "releases")
def items(path: Path, scope: Scope) -> Gathered:
"""Family F: how varied each pool actually is."""
def gather(conn: sqlite3.Connection) -> Gathered:
return {
"window": measure(conn, scope),
"pools": [
{
"command": pool.command,
"draws": pool.draws,
"distinct": pool.distinct,
"repeat_rate": pool.repeat_rate,
}
for pool in content.pools(conn, scope)
],
"repeats": [
{"command": command, "item_id": item, "draws": draws}
for command, item, draws in content.repeats(conn, scope)
],
}
return safely(path, gather) or empty("pools", "repeats")

View File

@ -0,0 +1,136 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Reading the command history from the *other* process.
The bot holds the database open and commits on every inbound message, so
this opens its own read-only connection and closes it explicitly -- the
pattern ``routes_suggestions.collect`` established, and for the same reason:
a reader left dangling is a lock waiting to happen, and ``Store`` on a plain
connection would apply its schema against a running bot.
Every gatherer is wrapped the same way. A bot that has never run, a database
older than these tables and a half-written file are all the same answer, and
none of them is worth breaking the page over -- the page is most useful
exactly when something is wrong.
"""
from __future__ import annotations
import contextlib
import sqlite3
from typing import TYPE_CHECKING, Any
from signal_bot.analytics.scope import Scope, window
if TYPE_CHECKING:
from collections.abc import Callable, Iterator
from pathlib import Path
Gathered = dict[str, Any]
GROUPS_SQL = """
SELECT group_id,
COUNT(*) AS events,
MIN(day) AS first_day,
MAX(day) AS last_day
FROM command_events
GROUP BY group_id
ORDER BY events DESC
"""
# The names shown beside a uuid on the page. Unlike the group chat, this is
# a loopback page the maintainer is looking at, so everybody who has a name
# gets one -- but a uuid still never appears without one beside it.
NAMES_SQL = """
SELECT uuid, display_name
FROM group_people
WHERE (:group IS NULL OR group_id = :group)
"""
@contextlib.contextmanager
def reading(path: Path) -> Iterator[sqlite3.Connection | None]:
"""A read-only connection, or None when there is nothing readable."""
if not path.is_file():
yield None
return
try:
with contextlib.closing(sqlite3.connect(f"file:{path}?mode=ro", uri=True)) as conn:
conn.row_factory = sqlite3.Row
yield conn
except sqlite3.Error:
yield None
def safely(path: Path, gather: Callable[[sqlite3.Connection], Gathered]) -> Gathered:
"""Run one gatherer, answering with the empty shape if anything is wrong."""
with reading(path) as conn:
if conn is None:
return {}
try:
return gather(conn)
except sqlite3.Error:
return {}
def measure(conn: sqlite3.Connection, scope: Scope) -> Gathered:
"""The days covered and where the rows came from, on every response."""
measured = window(conn, scope)
return {
"first_day": measured.first_day,
"last_day": measured.last_day,
"days": measured.days,
"events": measured.events,
"enough_for_a_trend": measured.enough_for_a_trend,
"sources": {
"live": measured.live,
"journald": measured.journald,
"daily_uses": measured.daily,
},
}
def names_in(conn: sqlite3.Connection, scope: Scope) -> dict[str, str]:
"""Map each uuid to a display name, so no bare identifier is shown."""
return {
str(row["uuid"]): str(row["display_name"])
for row in conn.execute(NAMES_SQL, scope.params())
}
def groups(path: Path) -> Gathered:
"""Every group with a history, for the picker."""
def gather(conn: sqlite3.Connection) -> Gathered:
return {
"groups": [
{
"id": str(row["group_id"]),
"events": int(row["events"]),
"first_day": str(row["first_day"]),
"last_day": str(row["last_day"]),
}
for row in conn.execute(GROUPS_SQL)
]
}
return safely(path, gather) or {"groups": []}
def empty(*lists: str) -> Gathered:
"""The shape every panel gets when there is nothing readable.
Named keys with empty values rather than an error, so the page renders
its "nothing yet" state instead of a stack trace -- and so the frontend
needs no branch for a database that does not exist.
"""
return {"window": EMPTY_WINDOW, **{name: [] for name in lists}}
EMPTY_WINDOW: Gathered = {
"first_day": "",
"last_day": "",
"days": 0,
"events": 0,
"enough_for_a_trend": False,
"sources": {"live": 0, "journald": 0, "daily_uses": 0},
}

View File

@ -23,6 +23,7 @@ from signal_bot.manage import (
guard,
routes_actions,
routes_album,
routes_analytics,
routes_env,
routes_status,
routes_suggestions,
@ -51,5 +52,6 @@ def build_app(deps: ManageDeps, *, container: str = DEFAULT_CONTAINER) -> FastAP
app.include_router(routes_actions.router(deps, container))
app.include_router(routes_album.router(deps))
app.include_router(routes_suggestions.router(deps))
app.include_router(routes_analytics.router(deps))
assets.install(app, deps.dist_dir)
return app

View File

@ -0,0 +1,110 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""The Analytics tab's data, one route per panel.
Read-only, every one of them. This page can already rewrite ``.env`` and
restart a container; the command history is the one thing on it that is
worth nothing to an attacker and everything to a maintainer, and there is no
reason for a write route to exist here at all.
``?group=`` is optional everywhere. Absent means the all-groups roll-up, and
because ``Scope`` binds it as a nullable parameter the roll-up runs the same
SQL as the per-group answer rather than a second query that would drift.
"""
from __future__ import annotations
from typing import TYPE_CHECKING, Annotated
from fastapi import APIRouter, Query
from signal_bot.analytics.scope import Scope
from signal_bot.animals import load_aliases
from signal_bot.external import run
from signal_bot.manage import analytics_panels, analytics_read
if TYPE_CHECKING:
from signal_bot.manage.deps import ManageDeps
# Where the release tags live. Named so the shell-out reads as one thing.
TAGS = "refs/tags"
# Routes are annotated ``-> analytics_read.Gathered`` rather than ``->
# Gathered``. FastAPI resolves a return annotation through pydantic when the
# app is built, and ``from __future__ import annotations`` makes that a
# lookup in module globals -- where a name imported only under TYPE_CHECKING
# does not exist. Going through the module, which is imported at runtime
# anyway, keeps the annotation resolvable and the import honest.
# What the near-miss grouping is matched against. Read from the alias file
# rather than from a live registry: this is a second process with no bot in
# it, and the file is the same data the bot registers from.
def triggers() -> tuple[str, ...]:
"""Every trigger any command claims, for grouping near misses."""
return tuple(trigger for triggers in load_aliases().values() for trigger in triggers)
def releases() -> tuple[tuple[str, str], ...]:
"""(version, day) for every release, taken from the git tags.
Not from ``DOCS-changelog.md``, which was the obvious place to look and
turns out not to work: the changelog is the authority on *what* shipped
and carries no dates at all -- its headings are ``## <version> — <pope>``
and nothing more. The tags are the only record of *when*.
Shelled out through ``signal_bot.external``, which never raises: a
checkout without tags, or a deployment that is not a git tree, comes back
as no releases and the timeline simply draws without annotations.
"""
result = run(["git", "for-each-ref", "--format=%(refname:short) %(creatordate:short)", TAGS])
if not result.ok:
return ()
found = []
for line in result.stdout.splitlines():
name, _, day = line.partition(" ")
if name.startswith("v") and day:
found.append((name.removeprefix("v"), day.strip()))
return tuple(sorted(found, key=lambda pair: pair[1]))
def router(deps: ManageDeps) -> APIRouter:
"""Seven read-only routes, all under /api/analytics."""
api = APIRouter(prefix="/api/analytics")
path = deps.database_path
@api.get("/groups")
def groups() -> analytics_read.Gathered:
"""Every group with a history, for the picker."""
return analytics_read.groups(path)
@api.get("/overview")
def overview(group: Annotated[str | None, Query()] = None) -> analytics_read.Gathered:
"""Family A: what is used, by whom, and what is untouched."""
return analytics_panels.overview(path, Scope(group))
@api.get("/time")
def times(group: Annotated[str | None, Query()] = None) -> analytics_read.Gathered:
"""Family B: hours, weekdays, days, months, and what has gone quiet."""
return analytics_panels.times(path, Scope(group))
@api.get("/health")
def wellbeing(group: Annotated[str | None, Query()] = None) -> analytics_read.Gathered:
"""Family C: latency, refusals, errors and upstream behaviour."""
return analytics_panels.wellbeing(path, Scope(group))
@api.get("/demand")
def wishes(group: Annotated[str | None, Query()] = None) -> analytics_read.Gathered:
"""Family D: near misses, grouped by the trigger they nearly hit."""
return analytics_panels.wishes(path, Scope(group), triggers())
@api.get("/behaviour")
def behaviour(group: Annotated[str | None, Query()] = None) -> analytics_read.Gathered:
"""Family E: sittings, retention, releases and share of chat."""
return analytics_panels.behaviour(path, Scope(group), releases())
@api.get("/content")
def items(group: Annotated[str | None, Query()] = None) -> analytics_read.Gathered:
"""Family F: how varied each pool actually is."""
return analytics_panels.items(path, Scope(group))
return api

View File

@ -20,6 +20,7 @@ import base64
import secrets
from typing import TYPE_CHECKING, Final
from signal_bot.analytics import probe
from signal_bot.pack import candidate_keysets, media_type
if TYPE_CHECKING:
@ -85,6 +86,11 @@ def pack_handler(holder: PackHolder, entry: Entry) -> Handler:
await ctx.reply(NO_PHOTO)
return
filename = pick(pack.matching(pick_keys(viable)))
# The stored name is content-addressed -- pack_store.content_name is
# the first 128 bits of the file's SHA-256 -- so it identifies the
# photograph without describing it or anyone in it. That is what
# makes "which photo keeps coming back" answerable at all.
probe.note_item(filename)
payload = base64.b64encode(pack.media_path(filename).read_bytes()).decode("ascii")
await ctx.reply_media(payload, media_type(filename), filename)

View File

@ -20,6 +20,7 @@ from __future__ import annotations
import base64
from typing import TYPE_CHECKING, Final
from signal_bot.analytics import probe
from signal_bot.pack import MAX_NAMES, media_type
from signal_bot.pack_command import NAME_PREFIX, NO_PHOTO, UNKNOWN, pick
@ -67,6 +68,8 @@ def random_handler(holder: PackHolder) -> Handler:
await ctx.reply(NO_PHOTO)
return
filename = pick(viable)
# Content-addressed, for the reason pack_command records one.
probe.note_item(filename)
payload = base64.b64encode(pack.media_path(filename).read_bytes()).decode("ascii")
await ctx.reply_media(payload, media_type(filename), filename)

View File

@ -41,6 +41,7 @@ import secrets
from pathlib import Path
from typing import TYPE_CHECKING, Final
from signal_bot.analytics import probe
from signal_bot.animals import load_aliases
from signal_bot.wolnelektury import CATALOGUE, parse
@ -133,6 +134,10 @@ def poem_handler(
async def handler(ctx: CommandContext) -> None:
slug = random_slug(pool)
# The Wolne Lektury slug, which is what the pool holds and all
# it holds -- the poem, its title and its author arrive over the
# wire and are never written down. See DOCS-privacy.md.
probe.note_item(slug)
poem = parse(await fetcher.fetch_text(text_url.format(slug=slug)))
if poem is None:
msg = f"wolnelektury.pl no longer serves a usable poem at {slug!r}"

View File

@ -10,6 +10,7 @@ from __future__ import annotations
import secrets
from typing import TYPE_CHECKING, Final
from signal_bot.analytics import probe
from signal_bot.animals import load_aliases
from signal_bot.versioning import POPES, article_url, pope_line
@ -44,6 +45,9 @@ def pope_handler(cache: PortraitCache) -> Handler:
async def handler(ctx: CommandContext) -> None:
pope = random_pope()
# The ordinal, not the name: it is the pope's identifier here
# and in popes.json, and the caption is built at reply time.
probe.note_item(str(pope.ordinal))
image = await cache.get_base64(pope)
await ctx.reply_image(image, caption_for(pope))

View File

@ -74,14 +74,29 @@ class DuplicateTriggerError(ValueError):
class CommandRefusedError(Exception):
"""A gate declined the command, so nothing happened and nothing is recorded.
"""A gate declined the command, so nothing happened and no *use* is recorded.
The dispatcher records one use per invocation, and it cannot see inside a
handler. Without this signal a *refused* run looked exactly like a
successful one, which slid the cooldown window forward every time somebody
was told to wait -- so waiting the stated time never helped.
The refusal is now recorded in the command *history*, which is a different
table and a different question: how often a gate actually bites is what
says whether it is tuned right. ``gate`` names which one refused, because
"this command is refused 40% of the time" and "this command is popular"
are the two readings of the same raw count.
It defaults to empty so that raising it bare stays legal -- every existing
raise site said only "no", and a required argument would turn a behaviour
that works into a signature change across three decorators.
"""
def __init__(self, gate: str = "") -> None:
"""Record which gate refused, if the caller says."""
super().__init__(gate)
self.gate = gate
@dataclass(frozen=True, slots=True)
class Resolved:

View File

@ -100,28 +100,81 @@ CREATE TABLE IF NOT EXISTS suggestions (
);
"""
# Cleared when the bot leaves a group, and rebuilt on a schema-version bump.
# Losing any of these costs a cooldown or a vote in progress.
EPHEMERAL_TABLES: Final = (
# These were one tuple, EPHEMERAL_TABLES, until command analytics needed the
# two jobs it was doing to come apart. "Dropped when the schema changes" and
# "cleared when the bot leaves a group" happened to name the same four tables,
# so one constant served both -- and then daily_uses became a source the
# analytics history is backfilled from, which must survive a schema bump but
# must still reset a rejoined group's daily limits. Two questions, two tuples.
# Dropped and recreated when SCHEMA_VERSION rises. Only tables whose contents
# cost nothing to lose: a vote in progress, a per-group marker.
#
# The price of taking command_uses/daily_uses out of here is explicit: if
# their *shape* ever changes, CREATE TABLE IF NOT EXISTS is a no-op against
# the old one and every INSERT starts failing. Reshaping either of them now
# needs a written migration rather than a version bump. That is the trade
# that keeps the use history.
REBUILT_ON_BUMP: Final = (
"votes",
"group_meta",
)
# Rows deleted for one group when the bot is removed from it, so that being
# added back starts with a clean cooldown rather than resurrecting whatever
# was true whenever it was last in there.
CLEARED_ON_LEAVE: Final = (
"command_uses",
"daily_uses",
"votes",
"group_meta",
)
# Never dropped, by either path. These hold the whole statistical history,
# which cannot be recomputed from anything -- so a transient group listing
# that omits a member (the exact failure discovery._permitted guards against)
# must not be able to delete years of it, and neither must a future schema
# bump that only meant to reshape a cooldown table.
# Never dropped and never cleared, by any path. These hold the whole
# statistical history, which cannot be recomputed from anything -- so a
# transient group listing that omits a member (the exact failure
# discovery._permitted guards against) must not be able to delete years of
# it, and neither must a future schema bump that only meant to reshape a
# cooldown table.
PERMANENT_TABLES: Final = (
"message_events",
"group_people",
# What the group wished existed. Neither can be recomputed from anything
# -- an idea somebody typed once is gone the moment it is dropped -- and
# forget_group() clears the ephemeral set on the transient group-listing
# gap discovery._permitted guards against, which must not cost a year of
# forget_group() clears its own set on the transient group-listing gap
# discovery._permitted guards against, which must not cost a year of
# collected suggestions.
"unknown_commands",
"suggestions",
# Every command attempt ever made, and every upstream call made serving
# one. In neither tuple above: a schema bump must not throw away the
# answer to "which command is dying", and leaving a group must not
# either -- message_events already survives both for the same reason.
"command_events",
"upstream_calls",
)
TABLES: Final = EPHEMERAL_TABLES + PERMANENT_TABLES
# Every table the schema declares. command_uses and daily_uses are in
# CLEARED_ON_LEAVE but not PERMANENT_TABLES, so the union is spelled out
# rather than concatenated -- test_schema_tables checks it against sqlite's
# own catalogue, which is what keeps this honest.
TABLES: Final = (
"command_uses",
"daily_uses",
*REBUILT_ON_BUMP,
*PERMANENT_TABLES,
)
# The two tables that carry history and are NOT rebuilt on a version bump.
# Because they survive, a bump can no longer fix them if their shape is
# wrong, and CREATE TABLE IF NOT EXISTS will not either -- which is the
# failure _apply_schema exists to prevent: a bot that boots clean and then
# fails on every INSERT. So their columns are declared here and checked, and
# a mismatch drops the table after all. Losing a cooldown is recoverable;
# a bot that answers nothing is not.
#
# This is the "written migration" the comment on REBUILT_ON_BUMP promises.
# A deliberate reshape means editing the tuple below in the same commit, and
# accepting that the old rows go. test_declared_columns_match_the_schema
# asserts these against what sqlite actually creates, so they cannot drift.
RETAINED_COLUMNS: Final = {
"command_uses": ("group_id", "user_uuid", "command", "used_at"),
"daily_uses": ("group_id", "user_uuid", "command", "day", "uses"),
}

View File

@ -0,0 +1,78 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""The command history's own tables, kept apart from the rest of the schema.
Split out of ``schema.py`` so both stay under the 250-line cap, the same
reason ``schema.py`` was split out of ``store.py``. It is applied straight
after ``SCHEMA`` and by the same method, so this is a filing decision and
not a second schema: there is one database and one ``PRAGMA user_version``.
Neither table is ever dropped -- see ``PERMANENT_TABLES``. The whole point
of them is answering questions about a past that nothing can recompute.
"""
from __future__ import annotations
from typing import Final
COMMAND_SCHEMA: Final = """
-- One row per command *attempt*, including the ones that never ran. The two
-- tables above it are aggregates a rate limiter reads; this is the history,
-- and nothing recomputes it.
--
-- Keyed on the envelope's own timestamp for the reason unknown_commands is:
-- signal-cli replays undelivered frames on reconnect, and a clock reading is
-- fresh on every replay. invoked_at_ms is message.timestamp, so a replay
-- collides with the row it already wrote instead of adding a second one.
--
-- source is part of the key because a backfilled row and a live row can
-- describe the same invocation from two sources that disagree about the
-- hour, and neither may overwrite the other.
CREATE TABLE IF NOT EXISTS command_events (
group_id TEXT NOT NULL,
user_uuid TEXT NOT NULL,
command TEXT NOT NULL,
-- Never free text. A pack pseudonym key the registry already resolved
-- ('p1', 'add', 'random'), or '' / '+args'. See registry.resolve_with_args.
args TEXT NOT NULL DEFAULT '',
-- A public identifier for what was served -- a Wikidata QID, a Commons
-- page id, a Wolne Lektury slug, a content-addressed filename. Never a
-- title, a caption or a name: an identifier cannot carry one.
item_id TEXT,
outcome TEXT NOT NULL,
-- The exception's class name when outcome is 'error', '' otherwise.
-- A name, never a message: an exception's str() carries whatever was
-- being processed, which for this bot is somebody's message.
error_class TEXT NOT NULL DEFAULT '',
invoked_at_ms INTEGER NOT NULL,
day TEXT NOT NULL,
local_hour INTEGER NOT NULL,
weekday INTEGER NOT NULL,
handler_ms INTEGER,
send_ms INTEGER,
source TEXT NOT NULL DEFAULT 'live',
PRIMARY KEY (group_id, user_uuid, command, invoked_at_ms, source)
);
CREATE INDEX IF NOT EXISTS idx_command_events_group_day
ON command_events (group_id, day);
CREATE INDEX IF NOT EXISTS idx_command_events_group_command_day
ON command_events (group_id, command, day);
CREATE INDEX IF NOT EXISTS idx_command_events_group_user_day
ON command_events (group_id, user_uuid, day);
-- One row per outbound HTTP call made while a command was running, joined to
-- command_events by (group_id, invoked_at_ms). seq orders the calls within
-- one invocation and keeps two calls to the same host distinct.
CREATE TABLE IF NOT EXISTS upstream_calls (
group_id TEXT NOT NULL,
invoked_at_ms INTEGER NOT NULL,
command TEXT NOT NULL,
host TEXT NOT NULL,
-- 0 means the request never got a reply at all: DNS, connect, timeout.
-- Distinguishable from a 5xx, which is upstream answering badly.
status INTEGER NOT NULL,
duration_ms INTEGER NOT NULL,
seq INTEGER NOT NULL,
PRIMARY KEY (group_id, invoked_at_ms, seq)
);
CREATE INDEX IF NOT EXISTS idx_upstream_calls_host
ON upstream_calls (group_id, host);
"""

View File

@ -16,6 +16,24 @@ if TYPE_CHECKING:
from signal_bot.stats.events import EventStore
def sent_at_ms_of(message: IncomingMessage, now: dt.datetime) -> int:
"""When a message was sent, in milliseconds, for use as a key.
The timestamp falls back to the clock because ``models._timestamp``
returns 0 when neither the payload nor the envelope carried one. Zero is
not merely inaccurate: it is part of the primary key, so every
zero-timestamp frame from one sender would collide and the whole run
would be recorded as a single message.
One function rather than the same expression in two places, because
``message_events`` and ``command_events`` are joined on exactly this
value -- to say which command a counted message was, and to attach an
hour to a backfilled row. Two spellings that drifted would not fail
anything; they would quietly stop matching.
"""
return message.timestamp or int(now.timestamp() * MS_PER_SECOND)
def to_event(
group_id: str,
message: IncomingMessage,
@ -23,15 +41,8 @@ def to_event(
*,
is_command: bool,
) -> MessageEvent:
"""Reduce a message to the metadata the statistics keep.
The timestamp falls back to the clock because ``models._timestamp``
returns 0 when neither the payload nor the envelope carried one. Zero is
not merely inaccurate: it is part of the primary key, so every
zero-timestamp frame from one sender would collide and the whole run
would be recorded as a single message.
"""
sent_at_ms = message.timestamp or int(now.timestamp() * MS_PER_SECOND)
"""Reduce a message to the metadata the statistics keep."""
sent_at_ms = sent_at_ms_of(message, now)
return MessageEvent(
group_id=group_id,
uuid=message.source_uuid,

View File

@ -14,10 +14,13 @@ recomputation can bring back once dropped.
from __future__ import annotations
import contextlib
import logging
import sqlite3
from typing import TYPE_CHECKING, Self
from signal_bot.schema import EPHEMERAL_TABLES, SCHEMA, SCHEMA_VERSION
from signal_bot.analytics.uses import CommandEventStore
from signal_bot.schema import REBUILT_ON_BUMP, RETAINED_COLUMNS, SCHEMA, SCHEMA_VERSION
from signal_bot.schema_commands import COMMAND_SCHEMA
from signal_bot.stats.events import EventStore
from signal_bot.suggestions import SuggestionStore
@ -25,6 +28,8 @@ if TYPE_CHECKING:
from collections.abc import Iterator
from types import TracebackType
logger = logging.getLogger(__name__)
class Store:
"""Thin, synchronous SQLite wrapper. All state the bot keeps lives here."""
@ -36,6 +41,7 @@ class Store:
self._apply_schema()
self.events = EventStore(self._conn)
self.suggestions = SuggestionStore(self._conn)
self.commands = CommandEventStore(self._conn)
def _apply_schema(self) -> None:
"""Create the tables, rebuilding them when the file predates them.
@ -53,15 +59,51 @@ class Store:
"""
version = int(self._conn.execute("PRAGMA user_version").fetchone()[0])
if version < SCHEMA_VERSION:
# EPHEMERAL_TABLES, not TABLES. Rebuilding a cooldown table is
# REBUILT_ON_BUMP, not TABLES. Rebuilding a vote in progress is
# free; rebuilding message_events would throw away the entire
# statistical history, which nothing can recompute.
for table in EPHEMERAL_TABLES:
# statistical history, which nothing can recompute. command_uses
# and daily_uses used to be in here and are deliberately not any
# more -- see the comment on the tuple in schema.py.
for table in REBUILT_ON_BUMP:
self._conn.execute(f"DROP TABLE IF EXISTS {table}")
self._drop_reshaped()
self._conn.executescript(SCHEMA)
# A separate string, not a separate schema: one database, one
# user_version. Split only so both files stay under the cap.
self._conn.executescript(COMMAND_SCHEMA)
self._conn.execute(f"PRAGMA user_version = {SCHEMA_VERSION}")
self._conn.commit()
def _drop_reshaped(self) -> None:
"""Drop a retained table whose columns no longer match the schema.
command_uses and daily_uses survive a version bump so that the use
history survives with them, which means a bump can no longer repair
them. A database written before the group key still holds the old
two-column command_uses, and CREATE TABLE IF NOT EXISTS is a no-op
against it -- so without this the bot boots clean and then fails on
every INSERT, which is the exact failure the version check exists to
prevent.
Keeping rows is worth a lot; keeping them at the price of a bot that
answers nothing is not. A mismatch is logged, because losing a
cooldown silently is fine and losing history silently is not.
"""
for table, expected in RETAINED_COLUMNS.items():
# PRAGMA takes no parameters, and the name is a module constant
# walked here rather than anything a message could reach.
found = tuple(
str(row["name"]) for row in self._conn.execute(f"PRAGMA table_info({table})")
)
if found and found != expected:
logger.warning(
"%s has columns %s, expected %s; dropping it to stay writable",
table,
found,
expected,
)
self._conn.execute(f"DROP TABLE IF EXISTS {table}")
def __enter__(self) -> Self:
"""Enter a context manager that closes the connection on exit."""
return self
@ -80,7 +122,7 @@ class Store:
self._conn.close()
def forget_group(self, group_id: str) -> None:
"""Drop one group's *ephemeral* rows: cooldowns, limits, votes, meta.
"""Drop one group's live scaffolding: cooldowns, limits, votes, meta.
Called when the bot is removed from a group, so that being added back
starts with a clean cooldown rather than resurrecting whatever was
@ -92,10 +134,11 @@ class Store:
years of statistics on the strength of one bad HTTP response. History
is permanent; only the live scaffolding is disposable.
"""
# Spelled out rather than looped over _TABLES: an f-string here is a
# SQL-injection shape even when the input is a module constant, and
# the repo does not take lint suppressions. test_forget_group_clears
# _every_table is what keeps this list in step with the schema.
# Spelled out rather than looped over CLEARED_ON_LEAVE: an f-string
# here is a SQL-injection shape even when the input is a module
# constant, and the repo does not take lint suppressions.
# test_forget_group_clears_every_cleared_table is what keeps this
# list in step with that tuple.
self._conn.execute("DELETE FROM command_uses WHERE group_id = ?", (group_id,))
self._conn.execute("DELETE FROM daily_uses WHERE group_id = ?", (group_id,))
self._conn.execute("DELETE FROM votes WHERE group_id = ?", (group_id,))

View File

@ -22,6 +22,7 @@ from base64 import b64encode
from pathlib import Path
from typing import TYPE_CHECKING, Final
from signal_bot.analytics import probe
from signal_bot.animals import load_aliases
from signal_bot.commons import caption_for, imageinfo_params, page_of
from signal_bot.endpoints import THUMB_WIDTH
@ -88,9 +89,12 @@ def trap_handler(
async def handler(ctx: CommandContext) -> None:
# One request, because the caption and the picture have to describe
# the same file; two would be a chance for them to disagree.
page_id = random_trap(pool)
# A Commons page id, for the reason art_command records one.
probe.note_item(str(page_id))
document = await fetcher.fetch_json(
api_url,
imageinfo_params(THUMB_WIDTH, page_id=random_trap(pool), describe=True),
imageinfo_params(THUMB_WIDTH, page_id=page_id, describe=True),
)
page = page_of(document)
image = await fetcher.fetch_image(str(page["imageinfo"][0]["thumburl"]))

View File

@ -30,7 +30,7 @@ PATCHES_PER_MINOR: Final = 8
# Which release this build is. Bump by one per release; everything else,
# including the version string in pyproject.toml, follows from it.
RELEASE_ORDINAL: Final = 22
RELEASE_ORDINAL: Final = 23
_POPES_FILE: Final = Path(__file__).with_name("popes.json")

144
tests/analytics_seed.py Normal file
View File

@ -0,0 +1,144 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Building a command history to ask questions of.
Everything is arithmetic -- no randomness, no clock -- so a failure names a
row rather than a seed. The people are invented (ala, basia, celina, darek)
and so are the group ids: ``check_personal_data`` learns real names from the
live database, so a real one here would fail only on the maintainer's
machine and pass in CI.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import TYPE_CHECKING
from signal_bot.analytics.backfill import midnight_ms
from signal_bot.analytics.probe import UpstreamCall
from signal_bot.analytics.uses import (
OUTCOME_OK,
SOURCE_DAILY,
SOURCE_JOURNAL,
SOURCE_LIVE,
CommandEvent,
)
from signal_bot.stats.backfill import UNKNOWN_HOUR
from signal_bot.stats.periods import weekday_of
if TYPE_CHECKING:
from collections.abc import Sequence
from signal_bot.store import Store
GROUP = "group-a=="
OTHER = "group-b=="
ALA = "uuid-ala"
BASIA = "uuid-basia"
CELINA = "uuid-celina"
HOUR_MS = 3_600_000
def at(day: str, hour: int, minute: int = 0) -> int:
"""A Warsaw wall-clock moment on one day, in epoch milliseconds."""
return midnight_ms(day) + hour * HOUR_MS + minute * 60_000
@dataclass(frozen=True, slots=True)
class Shape:
"""Everything about one attempt except who ran what, and when.
Grouped rather than passed as arguments, for the reason the repository
caps arguments at seven: these are the columns a given test usually
leaves entirely alone, and a call site that names three of them reads
better than one that names all eleven.
"""
group_id: str = GROUP
outcome: str = OUTCOME_OK
source: str = SOURCE_LIVE
item_id: str | None = None
handler_ms: int | None = None
send_ms: int | None = None
error_class: str = ""
args: str = ""
calls: Sequence[UpstreamCall] = ()
# A module-level singleton, because Shape is frozen and a default argument
# that is a call is evaluated once anyway -- naming it says so.
PLAIN = Shape()
def ran(
store: Store,
*,
day: str,
hour: int = 12,
minute: int = 0,
who: str = ALA,
command: str = "cat",
shape: Shape = PLAIN,
) -> int:
"""Record one attempt and return the timestamp it was keyed on."""
invoked_at_ms = at(day, hour, minute)
store.commands.record(
CommandEvent(
group_id=shape.group_id,
user_uuid=who,
command=command,
args=shape.args,
item_id=shape.item_id,
outcome=shape.outcome,
error_class=shape.error_class,
invoked_at_ms=invoked_at_ms,
day=day,
local_hour=UNKNOWN_HOUR if shape.source == SOURCE_DAILY else hour,
weekday=weekday_of(day),
handler_ms=shape.handler_ms,
send_ms=shape.send_ms,
source=shape.source,
calls=shape.calls,
)
)
return invoked_at_ms
def backfilled(
store: Store, *, day: str, command: str = "cat", who: str = ALA, group_id: str = GROUP
) -> None:
"""One row from the source that knows a day and nothing finer."""
ran(
store,
day=day,
command=command,
who=who,
hour=0,
shape=Shape(group_id=group_id, source=SOURCE_DAILY),
)
def reconstructed(
store: Store,
*,
day: str,
hour: int,
command: str = "cat",
who: str = ALA,
group_id: str = GROUP,
) -> None:
"""One row recovered from the journal: a real hour, but no durations."""
ran(
store,
day=day,
hour=hour,
command=command,
who=who,
shape=Shape(group_id=group_id, source=SOURCE_JOURNAL),
)
def a_call(host: str = "api.example.test", status: int = 200, ms: int = 40) -> UpstreamCall:
"""One outbound request made while a command was running."""
return UpstreamCall(host=host, status=status, ms=ms)

View File

@ -0,0 +1,250 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Recovering the command history, without inventing any of it.
Two properties matter more than the rest and each has its own test: the two
sources must not double-count the same invocations, and a second run must
insert nothing.
"""
from __future__ import annotations
from typing import TYPE_CHECKING
from signal_bot.analytics import backfill
from signal_bot.analytics.backfill_join import WINDOW_MS
from signal_bot.analytics.journal import JournalLine
from signal_bot.analytics.uses import SOURCE_DAILY, SOURCE_JOURNAL
from signal_bot.stats.events import MessageEvent
if TYPE_CHECKING:
from signal_bot.store import Store
GROUP = "group-a=="
DAY = "2026-08-29"
# Midday on the day above, so a +/- window cannot cross a date boundary.
NOON_MS = backfill.midnight_ms(DAY) + 12 * 3600 * 1000
def a_message(store: Store, at_ms: int, who: str = "ala", uuid: str = "u1") -> None:
"""One counted command in the message history."""
store.events.record(
MessageEvent(
group_id=GROUP,
uuid=uuid,
display_name=who,
sent_at_ms=at_ms,
day=DAY,
local_hour=12,
is_command=True,
)
)
def a_line(at_ms: int, command: str, who: str = "ala", *, refused: bool = False) -> JournalLine:
"""One line the bot wrote about running a command."""
return JournalLine(at_ms=at_ms, command=command, display_name=who, refused=refused)
def rows(store: Store) -> list[tuple[str, str, int, str]]:
"""Every recorded attempt as (command, outcome, hour, source)."""
return [
(str(r["command"]), str(r["outcome"]), int(r["local_hour"]), str(r["source"]))
for r in store.commands.connection.execute(
"SELECT command, outcome, local_hour, source FROM command_events ORDER BY invoked_at_ms"
)
]
def test_a_message_gets_its_name_from_the_journal(store: Store) -> None:
"""The whole point: the database has the hour, the journal has the name."""
a_message(store, NOON_MS)
report, events = backfill.plan(store, [a_line(NOON_MS + 200, "cat")])
backfill.apply(store, events)
assert report.journal_rows == 1
assert rows(store) == [("cat", "ok", 12, SOURCE_JOURNAL)]
def test_a_journal_line_earlier_than_its_message_still_matches(store: Store) -> None:
"""The window is symmetric, and this is why.
sent_at_ms comes from the sender's phone and the journal stamp from this
host; nothing keeps two clocks in step. On the live database 194 of 299
counted commands had a journal line *earlier* than their own timestamp.
An asymmetric window drops every one of them.
"""
a_message(store, NOON_MS)
_report, events = backfill.plan(store, [a_line(NOON_MS - 3_000, "cat")])
backfill.apply(store, events)
assert rows(store) == [("cat", "ok", 12, SOURCE_JOURNAL)]
def test_a_line_outside_the_window_is_not_used(store: Store) -> None:
"""Far enough away it is somebody else's invocation, or another day's."""
a_message(store, NOON_MS)
report, events = backfill.plan(store, [a_line(NOON_MS + WINDOW_MS + 1, "cat")])
backfill.apply(store, events)
assert report.journal_rows == 0
assert report.unmatched == 1
assert [row for row in rows(store) if row[3] == SOURCE_JOURNAL] == []
def test_two_commands_seconds_apart_are_both_resolved(store: Store) -> None:
"""Global closest-pair assignment, not best-for-each-row in turn.
Both lines sit inside both messages' windows. Per-row greedy cannot tell
them apart and drops both; assignment pairs them by distance, which is
correct because the clock skew is the same for both.
"""
a_message(store, NOON_MS)
a_message(store, NOON_MS + 5_000)
report, events = backfill.plan(
store,
[a_line(NOON_MS + 200, "cat"), a_line(NOON_MS + 5_200, "dog")],
)
backfill.apply(store, events)
assert report.ambiguous == 0
assert [(row[0], row[3]) for row in rows(store)] == [
("cat", SOURCE_JOURNAL),
("dog", SOURCE_JOURNAL),
]
def test_a_message_with_no_line_left_for_it_is_dropped(store: Store) -> None:
"""Two messages, one line: the second is a gap, never a guess."""
a_message(store, NOON_MS)
a_message(store, NOON_MS + 1_000)
report, events = backfill.plan(store, [a_line(NOON_MS + 100, "cat")])
backfill.apply(store, events)
assert report.journal_rows == 1
assert report.ambiguous == 1
def test_a_cooldown_refusal_is_recovered_as_a_refusal(store: Store) -> None:
"""The one gate that already logged. daily_limit and vote_gated did not."""
a_message(store, NOON_MS)
_report, events = backfill.plan(store, [a_line(NOON_MS, "cat", refused=True)])
backfill.apply(store, events)
assert rows(store) == [("cat", "refused_cooldown", 12, SOURCE_JOURNAL)]
def test_the_two_sources_do_not_double_count(store: Store) -> None:
"""The failure that would look exactly like data.
daily_uses says three uses. The journal explains one of them exactly.
The residual must be two, not three -- otherwise every historical count
is inflated and nothing anywhere fails.
"""
a_message(store, NOON_MS)
for _ in range(3):
store.record_use(GROUP, "u1", "cat", 1.0, DAY)
report, events = backfill.plan(store, [a_line(NOON_MS + 100, "cat")])
backfill.apply(store, events)
assert report.journal_rows == 1
assert report.daily_rows == 2
assert len(rows(store)) == 3
def test_a_day_the_message_history_never_saw_arrives_whole(store: Store) -> None:
"""The real shape of 2026-08-29: counts, and no message_events at all.
Those rows carry UNKNOWN_HOUR, so every hour-of-day panel must exclude
them rather than file them under midnight.
"""
for _ in range(4):
store.record_use(GROUP, "u1", "pope", 1.0, DAY)
report, events = backfill.plan(store, [])
backfill.apply(store, events)
assert report.journal_rows == 0
assert report.daily_rows == 4
assert {row[2] for row in rows(store)} == {-1}
assert {row[3] for row in rows(store)} == {SOURCE_DAILY}
def test_running_it_twice_inserts_nothing_the_second_time(store: Store) -> None:
"""Idempotency, and it must come from the key rather than from a wipe.
journald has no retention configuration and will rotate. A
clear-then-reimport pass would delete rows it could no longer replace,
so a second run has to be harmless without one.
"""
a_message(store, NOON_MS)
store.record_use(GROUP, "u1", "cat", 1.0, DAY)
lines = [a_line(NOON_MS + 100, "cat")]
_first, events = backfill.plan(store, lines)
backfill.apply(store, events)
before = rows(store)
for candidate in lines:
assert candidate is not None
_second, again = backfill.plan(store, [a_line(NOON_MS + 100, "cat")])
backfill.apply(store, again)
assert rows(store) == before
def test_live_rows_are_never_re_imported(store: Store) -> None:
"""The cutoff. A live row already describes that invocation exactly.
Without it, a pass run after recording started would import the journal
line for an invocation that already exists -- under a different source,
so the natural key would not stop it.
"""
a_message(store, NOON_MS)
store.commands.connection.execute(
"""
INSERT INTO command_events (
group_id, user_uuid, command, args, item_id, outcome, error_class,
invoked_at_ms, day, local_hour, weekday, handler_ms, send_ms, source
) VALUES (?, ?, 'cat', '', NULL, 'ok', '', ?, ?, 12, 5, 10, 10, 'live')
""",
(GROUP, "u1", NOON_MS, DAY),
)
store.commands.connection.commit()
report, events = backfill.plan(store, [a_line(NOON_MS + 100, "cat")])
backfill.apply(store, events)
assert report.journal_rows == 0
assert [row[3] for row in rows(store)] == ["live"]
def test_a_pass_marks_the_groups_it_touched(store: Store) -> None:
"""The same marker stats/backfill.py writes, for the same reason."""
a_message(store, NOON_MS)
_report, events = backfill.plan(store, [a_line(NOON_MS, "cat")])
backfill.apply(store, events)
assert store.meta(GROUP, backfill.BACKFILL_KEY) == "done"
def test_the_report_counts_everything_and_names_nobody() -> None:
"""It is printed to a terminal and pasted into issues.
The join's only input that could carry a person is a display name, and
it dies inside the joiner -- so the summary has to be countable and
nothing else.
"""
report = backfill.Report(journal_rows=3, daily_rows=2, unmatched=1, ambiguous=4, groups=2)
text = "\n".join(report.lines())
assert "3" in text
assert "ambiguous: 4" in text
assert text.count("\n") == 4
def test_midnight_is_built_from_the_date_not_from_arithmetic() -> None:
"""Warsaw days are not all 24 hours; subtracting 86_400_000 lands wrong.
2026-10-25 is the 25-hour day, so the gap from the 24th is not a day's
worth of milliseconds and any spelling that assumes it is drifts.
"""
short_day = backfill.midnight_ms("2026-10-25") - backfill.midnight_ms("2026-10-24")
assert short_day == 24 * 3600 * 1000
long_day = backfill.midnight_ms("2026-10-26") - backfill.midnight_ms("2026-10-25")
assert long_day == 25 * 3600 * 1000

View File

@ -0,0 +1,234 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Families D, E and F: demand, sittings, adoption and pool exhaustion."""
from __future__ import annotations
from typing import TYPE_CHECKING
import pytest
from signal_bot.analytics import adoption, content, demand, sessions
from signal_bot.analytics.scope import SESSION_GAP_MS, Scope
from signal_bot.stats.events import MessageEvent
from tests.analytics_seed import ALA, BASIA, CELINA, GROUP, Shape, backfilled, ran
if TYPE_CHECKING:
import sqlite3
from signal_bot.store import Store
DAY = "2026-08-29"
NEXT = "2026-08-30"
TRIGGERS = ("!kot", "!pies", "!pulapka", "!papiez")
def conn_of(store: Store) -> sqlite3.Connection:
return store.commands.connection
# -- family D ----------------------------------------------------------------
def test_near_misses_are_grouped_by_the_command_they_nearly_hit(store: Store) -> None:
"""Seventeen typos of one trigger mean it wants another alias.
Ungrouped they read as seventeen unrelated mistakes, which is exactly
the list the Ideas tab already shows and the reason this adds something.
"""
for at_ms, word in enumerate(("!pulapla", "!pulapak", "!pulakpa", "!kto")):
store.suggestions.record_unknown(GROUP, word, word, at_ms)
found = demand.collect(conn_of(store), Scope(), TRIGGERS)
assert [(miss.target, miss.hits) for miss in found.near_misses] == [
("!pulapka", 3),
("!kot", 1),
]
def test_the_same_typo_twice_counts_twice(store: Store) -> None:
store.suggestions.record_unknown(GROUP, "!kto", "!kto", 1)
store.suggestions.record_unknown(GROUP, "!kto", "!kto raz jeszcze", 2)
found = demand.collect(conn_of(store), Scope(), TRIGGERS)
assert found.near_misses[0].hits == 2
assert found.near_misses[0].variants == 1
def test_a_word_close_to_nothing_is_a_new_idea_not_a_typo(store: Store) -> None:
"""Filing a real request as noise is the failure worth avoiding."""
store.suggestions.record_unknown(GROUP, "!pogodynka", "!pogodynka", 1)
found = demand.collect(conn_of(store), Scope(), TRIGGERS)
assert found.near_misses == ()
assert (found.unclaimed, found.unclaimed_words) == (1, 1)
def test_deliberate_ideas_are_counted_beside_the_typos(store: Store) -> None:
store.suggestions.record_suggestion(GROUP, "!pogoda", "na jutro", 1)
assert demand.collect(conn_of(store), Scope(), TRIGGERS).filed == 1
def test_an_empty_wish_list_reads_as_empty(store: Store) -> None:
assert demand.collect(conn_of(store), Scope(), TRIGGERS) == demand.Demand()
# -- family E ----------------------------------------------------------------
def test_commands_close_together_are_one_sitting(store: Store) -> None:
ran(store, day=DAY, hour=12, minute=0, command="cat")
ran(store, day=DAY, hour=12, minute=1, command="dog")
found = sessions.collect(conn_of(store), Scope())
assert (found.count, found.commands, found.longest) == (1, 2, 2)
assert found.average_length == 2.0
def test_a_long_gap_starts_a_new_sitting(store: Store) -> None:
gap_minutes = SESSION_GAP_MS // 60_000 + 1
ran(store, day=DAY, hour=12, minute=0, command="cat")
ran(store, day=DAY, hour=12, minute=gap_minutes, command="dog")
assert sessions.collect(conn_of(store), Scope()).count == 2
def test_two_people_never_share_a_sitting(store: Store) -> None:
ran(store, day=DAY, hour=12, minute=0, who=ALA, command="cat")
ran(store, day=DAY, hour=12, minute=1, who=BASIA, command="dog")
found = sessions.collect(conn_of(store), Scope())
assert found.count == 2
assert found.pairs == ()
def test_commands_used_together_are_paired_once(store: Store) -> None:
"""(cat, dog) and (dog, cat) are one fact, not two."""
ran(store, day=DAY, hour=12, minute=0, command="dog")
ran(store, day=DAY, hour=12, minute=1, command="cat")
assert sessions.collect(conn_of(store), Scope()).pairs == (("cat", "dog", 1),)
def test_re_rolling_the_same_command_is_counted_as_repeats(store: Store) -> None:
"""Three !papież in a row is a fact about the pool, not about the person."""
for minute in range(3):
ran(store, day=DAY, hour=12, minute=minute, command="pope")
found = sessions.collect(conn_of(store), Scope())
assert found.repeats == (("pope", 2),)
assert found.pairs == ()
def test_rows_with_no_real_time_never_form_a_sitting(store: Store) -> None:
"""A daily_uses row's timestamp is built from midnight, not observed.
Treating those as a sequence invents a sitting in which somebody ran
forty commands in forty milliseconds.
"""
for _ in range(3):
backfilled(store, day=DAY, command="cat")
assert sessions.collect(conn_of(store), Scope()) == sessions.Sessions()
def test_share_of_chat_is_commands_over_messages(store: Store) -> None:
for at_ms in range(4):
store.events.record(
MessageEvent(
group_id=GROUP,
uuid=ALA,
display_name="ala",
sent_at_ms=at_ms,
day=DAY,
local_hour=12,
is_command=at_ms < 1,
)
)
found = adoption.share_of_chat(conn_of(store), Scope())
assert (found.messages, found.commands) == (4, 1)
assert found.fraction == 0.25
def test_share_of_an_empty_group_is_zero(store: Store) -> None:
assert adoption.share_of_chat(conn_of(store), Scope()).fraction == 0.0
assert adoption.Share().fraction == 0.0
def test_newcomers_are_dated_by_their_first_use(store: Store) -> None:
ran(store, day=DAY, who=ALA)
ran(store, day=NEXT, who=ALA, minute=1)
ran(store, day=NEXT, who=BASIA, minute=2)
assert adoption.newcomers(conn_of(store), Scope()) == [(DAY, 1), (NEXT, 1)]
def test_retention_counts_who_came_back(store: Store) -> None:
"""Of the people who started, how many were still there N days later."""
ran(store, day=DAY, who=ALA)
ran(store, day=NEXT, who=ALA, minute=1)
ran(store, day=DAY, who=BASIA, minute=2)
curve = adoption.retention(conn_of(store), Scope(), 2)
assert curve == [(0, 2, 2), (1, 1, 2), (2, 0, 2)]
def test_retention_of_nobody_is_empty(store: Store) -> None:
assert adoption.retention(conn_of(store), Scope(), 3) == []
def test_a_release_is_compared_against_equal_windows(store: Store) -> None:
"""A release on the first recorded day gets a before-window of zero.
Unequal windows would read that as a spike when it is an edge.
"""
series = [
("2026-08-28", 2),
("2026-08-29", 3),
("2026-08-30", 10),
("2026-08-31", 12),
]
found = adoption.around_releases(series, [("0.2.5", "2026-08-30")], 1)
assert found == [("0.2.5", "2026-08-30", 22, 3)]
# -- family F ----------------------------------------------------------------
def test_a_pool_that_never_repeats_reads_as_varied(store: Store) -> None:
for minute, item in enumerate(("Q1", "Q2", "Q3")):
ran(store, day=DAY, minute=minute, command="pope", shape=Shape(item_id=item))
(pool,) = content.pools(conn_of(store), Scope())
assert (pool.draws, pool.distinct) == (3, 3)
assert pool.repeat_rate == 0.0
def test_a_repeating_pool_is_measurable(store: Store) -> None:
"""The bug this exists to catch reached a live group chat once.
iNaturalist accepted order_by=random and silently ignored it, so one
identical photograph came back forever. A repeat rate would have said so.
"""
for minute in range(4):
ran(store, day=DAY, minute=minute, command="pope", shape=Shape(item_id="Q1"))
(pool,) = content.pools(conn_of(store), Scope())
assert (pool.draws, pool.distinct) == (4, 1)
assert pool.repeat_rate == 0.75
assert content.repeats(conn_of(store), Scope()) == [("pope", "Q1", 4)]
def test_a_command_that_names_nothing_has_no_pool(store: Store) -> None:
ran(store, day=DAY, command="stats_today")
assert content.pools(conn_of(store), Scope()) == []
def test_an_undrawn_pool_has_no_repeat_rate() -> None:
assert content.Pool("pope", 0, 0).repeat_rate == 0.0
def test_expected_distinct_says_what_uniform_would_have_looked_like() -> None:
"""The comparison that turns a repeat rate into a verdict."""
assert content.expected_distinct(0, 10) == 0.0
assert content.expected_distinct(266, 0) == 0.0
# 266 traps drawn 19 times should give about 18.4 distinct ones.
assert 18.0 < content.expected_distinct(266, 19) < 18.9
# Drawing once always gives exactly one -- approx because the identity
# holds in real arithmetic and not in binary floating point.
assert content.expected_distinct(10, 1) == pytest.approx(1.0)
def test_everything_is_scoped_to_one_group(store: Store) -> None:
ran(store, day=DAY, command="pope", who=CELINA, shape=Shape(group_id=GROUP, item_id="Q1"))
ran(store, day=DAY, command="pope", who=CELINA, shape=Shape(group_id="group-b==", item_id="Q2"))
assert content.pools(conn_of(store), Scope(GROUP))[0].draws == 1
assert content.pools(conn_of(store), Scope())[0].draws == 2

View File

@ -0,0 +1,233 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""What !analityka says, and the two things it must never say."""
from __future__ import annotations
import datetime as dt
from typing import TYPE_CHECKING
from signal_bot.analytics import render
from signal_bot.analytics.demand import Demand, NearMiss
from signal_bot.analytics.hours import Peak
from signal_bot.analytics.render import Summary
from signal_bot.analytics.scope import Window
from signal_bot.analytics.usage import CommandUse, PersonUse
from signal_bot.analytics_command import register_analytics_command, triggers_for
from signal_bot.dispatcher import Dispatcher
from signal_bot.registry import CommandRegistry
from signal_bot.stats.events import MessageEvent
from tests.analytics_seed import ALA, BASIA, Shape, ran
from tests.conftest import WATCHED_GROUP, envelope
if TYPE_CHECKING:
from signal_bot.context import CommandContext
from signal_bot.store import Store
from signal_bot.transport_fake import FakeTransport
DAY = "2026-08-29"
NOW = dt.datetime(2026, 8, 29, 12, 0, tzinfo=dt.UTC)
def a_registry() -> CommandRegistry:
"""A registry with the command under test on it."""
registry = CommandRegistry()
register_analytics_command(registry, aliases={"command_stats": ["!analityka"]})
return registry
def seed(store: Store, *, group_id: str = WATCHED_GROUP) -> None:
"""A small history with a name attached to one of the two people."""
for minute in range(3):
ran(
store,
day=DAY,
hour=19,
minute=minute,
who=ALA,
command="pope",
shape=Shape(group_id=group_id),
)
ran(store, day=DAY, hour=9, who=BASIA, command="cat", shape=Shape(group_id=group_id))
store.events.record(
MessageEvent(
group_id=group_id,
uuid=ALA,
display_name="ala",
sent_at_ms=1,
day=DAY,
local_hour=19,
is_command=True,
)
)
store.events.note_person(group_id, ALA, "ala", "", DAY)
async def reply_of(dispatcher: Dispatcher, transport: FakeTransport) -> str:
"""Run !analityka and return what went into the group."""
await dispatcher.handle(envelope(text="!analityka"))
return transport.sent[-1].text
def test_an_empty_history_says_so_in_one_line() -> None:
"""A fresh install must not answer with a blank block."""
summary = render.Summary(
measured=Window(),
uses=(),
people=(),
names={},
peak=Peak(),
direction="unknown",
demand=Demand(),
asker=ALA,
commands=0,
messages=0,
)
assert render.render(summary) == render.NOTHING_YET
def test_the_reply_is_capped_at_twelve_lines() -> None:
"""The cap, on a summary that fills every optional line."""
summary = Summary(
measured=Window(first_day=DAY, last_day="2026-09-30", days=30, events=99),
uses=(
CommandUse("pope", 40, 3, 30, 8, 2, DAY, DAY),
CommandUse("cat", 10, 2, 10, 0, 0, DAY, DAY),
CommandUse("dog", 1, 1, 1, 0, 0, DAY, DAY),
),
people=(PersonUse(ALA, 30, 3, "pope"), PersonUse(BASIA, 5, 1, "cat")),
names={ALA: "ala", BASIA: "basia"},
peak=Peak(busiest=19, busiest_uses=20, quietest=5, quietest_uses=1),
direction="rising",
demand=Demand(near_misses=(NearMiss("!pulapka", 17, 4),)),
asker=ALA,
commands=99,
messages=300,
)
lines = render.render(summary).split("\n")
assert len(lines) <= render.MAX_LINES
assert lines[0] == render.HEADING
# Every line earned its place; none is blank padding.
assert all(line.strip() for line in lines)
def test_the_window_is_stated_before_anything_is_claimed(store: Store) -> None:
"""The bot is days old, and a reader has to be able to see that."""
line = render.window_line(Window(first_day=DAY, last_day=DAY, days=1, events=4))
assert line.startswith("Okno: 1 dzień")
assert "4 razy" in line
def test_a_span_of_days_names_both_ends() -> None:
line = render.window_line(
Window(first_day="2026-08-29", last_day="2026-08-31", days=3, events=10)
)
assert "3 dni" in line
assert "29 sierpnia" in line
assert "31 sierpnia" in line
def test_no_line_carries_padding() -> None:
"""Signal renders a proportional font, so ljust lines up on nobody's phone."""
line = render.window_line(Window(first_day=DAY, last_day=DAY, days=1, events=1))
assert line == line.strip()
assert " " not in line
def test_a_direction_is_only_worded_when_there_is_one() -> None:
assert render.trend_line("unknown") == ""
assert render.trend_line("rising") == "użycie rośnie"
assert render.trend_line("falling") == "użycie spada"
assert render.trend_line("flat") == "użycie się trzyma"
def test_nobody_is_named_without_a_display_name() -> None:
"""The roster knows the rest as a uuid or a phone number, and both leak."""
people = (PersonUse(ALA, 5, 2, "pope"), PersonUse(BASIA, 3, 1, "cat"))
assert render.people_line(people, {}) == ""
named = render.people_line(people, {ALA: "ala"})
assert named == "Kto najwięcej: ala 5"
assert BASIA not in named
def test_the_asker_gets_their_own_favourite_and_nobody_elses() -> None:
"""Self-scope: a per-person breakdown of everybody stays off the chat."""
people = (PersonUse(ALA, 5, 2, "pope"), PersonUse(BASIA, 3, 1, "cat"))
assert render.mine_line(people, ALA) == "Twoja ulubiona: !pope (5 razy)"
assert render.mine_line(people, "uuid-nobody") == ""
def test_a_healthy_bot_says_nothing_about_health() -> None:
"""A line with nothing to say is dropped, never printed empty."""
healthy = (CommandUse("cat", 3, 1, 3, 0, 0, DAY, DAY),)
assert render.health_line(healthy) == ""
def test_the_worst_command_is_named_with_what_is_wrong() -> None:
"""Refusals count as much as errors: both read as "it does not work"."""
uses = (
CommandUse("cat", 10, 2, 6, 3, 1, DAY, DAY),
CommandUse("dog", 4, 1, 4, 0, 0, DAY, DAY),
)
line = render.health_line(uses)
assert line.startswith("Do poprawki: !cat")
assert "błędów: 1" in line
assert "odmów: 3" in line
def test_only_what_is_actually_wrong_is_listed() -> None:
"""A command that only crashes says nothing about refusals, and vice versa."""
crashing = (CommandUse("cat", 5, 1, 3, 0, 2, DAY, DAY),)
assert render.health_line(crashing) == "Do poprawki: !cat (błędów: 2)"
refusing = (CommandUse("cat", 5, 1, 3, 2, 0, DAY, DAY),)
assert render.health_line(refusing) == "Do poprawki: !cat (odmów: 2)"
def test_share_of_chat_is_omitted_when_there_is_no_chat() -> None:
assert render.share_line(0, 0) == ""
assert render.share_line(1, 4) == "Komendy to 25% wiadomości w tej grupie"
def test_triggers_are_scoped_to_the_group_that_asked() -> None:
"""The album's triggers are real people's names.
Reporting a near miss against one outside its group would announce that
the album exists, which is the leak the dispatcher already refuses at
resolution time.
"""
async def handler(_ctx: CommandContext) -> None:
pass
registry = CommandRegistry()
registry.register("cat", ["!kot"], handler, "kot")
registry.register("pack:p1", ["!ala"], handler, "ala", groups=frozenset({"album=="}))
here = triggers_for(registry.commands, "somewhere-else==")
there = triggers_for(registry.commands, "album==")
assert "!ala" not in here
assert "!ala" in there
async def test_the_command_answers_in_the_group_that_asked(
config: object,
directory: object,
store: Store,
transport: FakeTransport,
) -> None:
"""The whole path: a real frame, the real dispatcher, a real reply."""
seed(store)
dispatcher = Dispatcher(
config=config,
directory=directory,
registry=a_registry(),
store=store,
transport=transport,
clock=lambda: NOW,
)
text = await reply_of(dispatcher, transport)
assert text.startswith(render.HEADING)
assert "!pope" in text
assert "ala" in text
assert len(text.split("\n")) <= render.MAX_LINES

View File

@ -0,0 +1,201 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Family C: the questions that had no data at all before command_events."""
from __future__ import annotations
from typing import TYPE_CHECKING
from signal_bot.analytics import health
from signal_bot.analytics.probe import NO_RESPONSE
from signal_bot.analytics.scope import Scope
from signal_bot.analytics.uses import (
OUTCOME_COOLDOWN,
OUTCOME_DAILY,
OUTCOME_ERROR,
)
from tests.analytics_seed import GROUP, OTHER, Shape, a_call, backfilled, ran, reconstructed
if TYPE_CHECKING:
import sqlite3
from signal_bot.store import Store
DAY = "2026-08-29"
def conn_of(store: Store) -> sqlite3.Connection:
return store.commands.connection
def test_nothing_recorded_reads_as_nothing(store: Store) -> None:
conn = conn_of(store)
assert health.latencies(conn, Scope()) == []
assert health.gates(conn, Scope()) == []
assert health.errors(conn, Scope()) == []
assert health.upstreams(conn, Scope()) == []
def test_percentiles_come_from_the_rows_that_have_timings(store: Store) -> None:
for minute, ms in enumerate((100, 200, 300, 400, 500)):
ran(store, day=DAY, minute=minute, shape=Shape(handler_ms=ms, send_ms=ms // 2))
(latency,) = health.latencies(conn_of(store), Scope())
assert latency.samples == 5
assert latency.handler_p50 == 300
assert latency.handler_p95 == 500
assert latency.send_p50 == 150
def test_a_row_with_no_duration_never_reaches_a_percentile(store: Store) -> None:
"""NULL, never 0. A zero would drag a p50 down and read as a fast command.
This is the reason the duration columns are nullable rather than
defaulted -- and the reason the query says IS NOT NULL rather than
naming a source, so a future source with real timings joins in for free.
"""
ran(store, day=DAY, shape=Shape(handler_ms=400, send_ms=300))
reconstructed(store, day=DAY, hour=10)
backfilled(store, day=DAY)
(latency,) = health.latencies(conn_of(store), Scope())
assert latency.samples == 1
assert latency.handler_p50 == 400
def test_a_command_that_never_replied_has_no_send_time(store: Store) -> None:
"""Handler time without send time is a real shape, so it must not crash.
The live recorder always writes both, but a source that timed the
handler and not the send would otherwise put a NULL through int().
"""
ran(store, day=DAY, shape=Shape(handler_ms=400, send_ms=None, outcome=OUTCOME_ERROR))
(latency,) = health.latencies(conn_of(store), Scope())
assert latency.handler_p50 == 400
assert latency.send_p50 == 0
def test_an_empty_list_has_no_percentile() -> None:
assert health.percentile([], health.P50) == 0
def test_a_single_sample_is_its_own_percentile() -> None:
assert health.percentile([7], health.P95) == 7
def test_each_gate_is_counted_separately(store: Store) -> None:
""" "Refused 40% of the time" and "popular" are two readings of one count.
Which gate bit is the difference between a cooldown that is too long and
a daily limit that is too low.
"""
ran(store, day=DAY, minute=1, shape=Shape(outcome=OUTCOME_COOLDOWN))
ran(store, day=DAY, minute=2, shape=Shape(outcome=OUTCOME_COOLDOWN))
ran(store, day=DAY, minute=3, shape=Shape(outcome=OUTCOME_DAILY))
assert health.gates(conn_of(store), Scope()) == [
("cat", "refused_cooldown", 2),
("cat", "refused_daily", 1),
]
def test_errors_are_grouped_by_class_never_by_message(store: Store) -> None:
"""An exception's str() carries whatever it was processing.
Here that is somebody's message, so only the class name is kept -- and
"17 ImageFetchError" is the whole diagnostic value anyway.
"""
ran(store, day=DAY, minute=1, shape=Shape(outcome=OUTCOME_ERROR, error_class="ImageFetchError"))
ran(store, day=DAY, minute=2, shape=Shape(outcome=OUTCOME_ERROR, error_class="ImageFetchError"))
ran(store, day=DAY, minute=3, shape=Shape(outcome=OUTCOME_ERROR, error_class="TimeoutError"))
assert health.errors(conn_of(store), Scope()) == [
("cat", "ImageFetchError", 2),
("cat", "TimeoutError", 1),
]
def test_upstream_hosts_are_counted_and_timed(store: Store) -> None:
ran(
store,
day=DAY,
shape=Shape(
handler_ms=500,
calls=(a_call("api.example.test", 200, 40), a_call("cdn.example.test", 200, 90)),
),
)
ran(
store,
day=DAY,
minute=1,
shape=Shape(handler_ms=500, calls=(a_call("api.example.test", 200, 60),)),
)
hosts = {up.host: up for up in health.upstreams(conn_of(store), Scope())}
assert hosts["api.example.test"].calls == 2
# Nearest-rank of [40, 60] is the lower: ceil(0.5 * 2) - 1 == 0.
assert hosts["api.example.test"].p50_ms == 40
assert hosts["cdn.example.test"].calls == 1
def test_a_request_that_never_answered_counts_as_a_failure(store: Store) -> None:
"""Status 0 is DNS, connect or timeout: upstream never answered at all.
Distinct from a 5xx, which is upstream answering badly. Losing it would
make a service that is entirely down look merely unused.
"""
ran(
store,
day=DAY,
shape=Shape(handler_ms=10, calls=(a_call("api.example.test", NO_RESPONSE, 5000),)),
)
ran(
store,
day=DAY,
minute=1,
shape=Shape(handler_ms=10, calls=(a_call("api.example.test", 503, 20),)),
)
ran(
store,
day=DAY,
minute=2,
shape=Shape(handler_ms=10, calls=(a_call("api.example.test", 200, 20),)),
)
(host,) = health.upstreams(conn_of(store), Scope())
assert host.calls == 3
assert host.failures == 2
assert host.failure_rate == 2 / 3
def test_a_host_never_called_has_no_failure_rate() -> None:
assert health.Upstream("api.example.test", 0, 0, 0).failure_rate == 0.0
def test_upstream_time_is_attributed_to_the_command_that_spent_it(store: Store) -> None:
"""Which half of a slow command is this repository's code."""
ran(
store,
day=DAY,
command="pope",
shape=Shape(handler_ms=3000, send_ms=2400, calls=(a_call(ms=250),)),
)
(latency,) = health.latencies(conn_of(store), Scope())
assert latency.command == "pope"
assert latency.upstream_p50 == 250
assert latency.send_p50 == 2400
def test_everything_is_scoped_to_one_group(store: Store) -> None:
ran(
store,
day=DAY,
shape=Shape(group_id=GROUP, handler_ms=10, calls=(a_call("here.example.test"),)),
)
ran(
store,
day=DAY,
shape=Shape(group_id=OTHER, handler_ms=10, calls=(a_call("there.example.test"),)),
)
here = health.upstreams(conn_of(store), Scope(GROUP))
assert [host.host for host in here] == ["here.example.test"]
assert len(health.upstreams(conn_of(store), Scope())) == 2

View File

@ -0,0 +1,177 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Family B: hours, weekdays, day series, months and dying commands.
Every hour test seeds a ``daily_uses`` row and asserts it is absent. That is
how the exclusion gets proved rather than remembered: those rows carry
UNKNOWN_HOUR because their source knew only a date, and letting one through
files it under hour -1 -- or under midnight, if a later edit "tidies" the
sentinel away.
"""
from __future__ import annotations
from typing import TYPE_CHECKING
from signal_bot.analytics import hours, trend
from signal_bot.analytics.scope import Scope, Window, window
from tests.analytics_seed import ALA, BASIA, backfilled, ran, reconstructed
if TYPE_CHECKING:
import sqlite3
from signal_bot.store import Store
# A Saturday, so the weekday index is not accidentally right at 0.
DAY = "2026-08-29"
NEXT = "2026-08-30"
SATURDAY = 5
def conn_of(store: Store) -> sqlite3.Connection:
return store.commands.connection
def test_the_hour_histogram_has_a_bucket_for_every_hour(store: Store) -> None:
"""A missing 04:00 draws the wrong shape, and the empty hours are data."""
ran(store, day=DAY, hour=9)
buckets = hours.by_hour(conn_of(store), Scope())
assert len(buckets) == hours.HOURS_IN_DAY
assert buckets[9] == 1
assert sum(buckets) == 1
def test_a_row_with_no_hour_is_left_out(store: Store) -> None:
"""The exclusion, asserted rather than trusted."""
ran(store, day=DAY, hour=9)
backfilled(store, day=DAY)
assert sum(hours.by_hour(conn_of(store), Scope())) == 1
def test_a_reconstructed_row_keeps_its_hour(store: Store) -> None:
"""journald rows were matched to a real message, so their hour is real.
Excluding them would throw away most of the history for no gain, which
is why scope names two predicates and not one.
"""
reconstructed(store, day=DAY, hour=21)
buckets = hours.by_hour(conn_of(store), Scope())
assert buckets[21] == 1
def test_the_weekday_grid_is_seven_by_twenty_four(store: Store) -> None:
ran(store, day=DAY, hour=13)
grid = hours.by_weekday_hour(conn_of(store), Scope())
assert len(grid) == hours.DAYS_IN_WEEK
assert all(len(row) == hours.HOURS_IN_DAY for row in grid)
assert grid[SATURDAY][13] == 1
def test_each_command_gets_its_own_hours(store: Store) -> None:
ran(store, day=DAY, hour=8, command="cat")
ran(store, day=DAY, hour=22, command="pope")
grid = hours.by_command_hour(conn_of(store), Scope())
assert grid["cat"][8] == 1
assert grid["pope"][22] == 1
assert sum(grid["cat"]) == 1
def test_weekdays_carry_the_days_they_were_seen_on(store: Store) -> None:
"""An average needs a denominator, and one Tuesday is not a weekly rate."""
ran(store, day=DAY, hour=9)
ran(store, day=DAY, hour=10, minute=5)
assert hours.by_weekday(conn_of(store), Scope()) == [(SATURDAY, 2, 1)]
def test_the_peak_ignores_hours_nobody_ever_used(store: Store) -> None:
"""At three days old most of the night is empty.
"The deadest hour is 04:00 with zero" would be true of nine hours and
interesting about none of them.
"""
ran(store, day=DAY, hour=19)
ran(store, day=DAY, hour=19, minute=1)
ran(store, day=DAY, hour=6)
found = hours.peak(hours.by_hour(conn_of(store), Scope()))
assert (found.busiest, found.busiest_uses) == (19, 2)
assert (found.quietest, found.quietest_uses) == (6, 1)
def test_a_silent_history_has_no_peak(store: Store) -> None:
assert hours.peak([0] * hours.HOURS_IN_DAY) == hours.Peak()
def test_a_silent_day_is_drawn_as_a_zero(store: Store) -> None:
"""A sparse series draws a line straight over a day nobody used it."""
ran(store, day="2026-08-29", hour=9)
ran(store, day="2026-08-31", hour=9)
dense = trend.dense_days(trend.by_day(conn_of(store), Scope()))
assert dense == [("2026-08-29", 1), ("2026-08-30", 0), ("2026-08-31", 1)]
def test_an_empty_series_densifies_to_nothing() -> None:
assert trend.dense_days([]) == []
def test_months_carry_how_many_days_they_actually_saw(store: Store) -> None:
"""A partial month is not a quiet one, and the panel must be able to say."""
ran(store, day=DAY, hour=9)
ran(store, day=NEXT, hour=9)
assert trend.by_month(conn_of(store), Scope()) == [("2026-08", 2, 2)]
def test_per_command_day_series(store: Store) -> None:
ran(store, day=DAY, hour=9, command="cat")
ran(store, day=NEXT, hour=9, command="cat")
ran(store, day=NEXT, hour=10, command="pope")
series = trend.by_command_day(conn_of(store), Scope())
assert series["cat"] == {DAY: 1, NEXT: 1}
assert series["pope"] == {NEXT: 1}
def test_no_direction_is_claimed_from_a_short_history(store: Store) -> None:
"""Three points of a new bot's novelty curve are not a trend."""
series = [("2026-08-29", 1), ("2026-08-30", 5), ("2026-08-31", 20)]
assert trend.direction(series, Window(days=3)) == trend.UNKNOWN
def test_a_direction_is_claimed_once_there_is_enough(store: Store) -> None:
rising = [(f"2026-08-{day:02d}", day) for day in range(1, 15)]
falling = [(f"2026-08-{day:02d}", 15 - day) for day in range(1, 15)]
flat = [(f"2026-08-{day:02d}", 10) for day in range(1, 15)]
measured = Window(days=14)
assert trend.direction(rising, measured) == trend.RISING
assert trend.direction(falling, measured) == trend.FALLING
assert trend.direction(flat, measured) == trend.FLAT
def test_a_direction_needs_two_points() -> None:
assert trend.direction([("2026-08-29", 3)], Window(days=20)) == trend.UNKNOWN
assert trend.direction([], Window(days=20)) == trend.UNKNOWN
def test_growth_from_nothing_is_rising_and_nothing_is_flat() -> None:
"""A first half of zero cannot be divided by, and both cases are real."""
measured = Window(days=20)
from_zero = [("a", 0), ("b", 0), ("c", 5), ("d", 5)]
stayed_zero = [("a", 0), ("b", 0), ("c", 0), ("d", 0)]
assert trend.direction(from_zero, measured) == trend.RISING
assert trend.direction(stayed_zero, measured) == trend.FLAT
def test_a_command_that_stopped_being_used_is_named(store: Store) -> None:
ran(store, day="2026-08-20", hour=9, command="gecko")
ran(store, day="2026-08-31", hour=9, command="cat")
dying = trend.dying(conn_of(store), Scope(), "2026-08-31")
assert [(entry.command, entry.idle_days) for entry in dying] == [("gecko", 11)]
def test_a_command_used_today_is_not_dying(store: Store) -> None:
ran(store, day="2026-08-31", hour=9, command="cat")
assert trend.dying(conn_of(store), Scope(), "2026-08-31") == []
def test_two_people_on_one_day_are_counted_once_each(store: Store) -> None:
ran(store, day=DAY, hour=9, who=ALA)
ran(store, day=DAY, hour=10, who=BASIA)
assert trend.by_day(conn_of(store), Scope()) == [(DAY, 2, 2)]
assert window(conn_of(store), Scope()).days == 1

View File

@ -0,0 +1,91 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Reading the bot's own journal back, without copying names out of it."""
from __future__ import annotations
from signal_bot.analytics.journal import parse
PREFIX = "2026-08-31T21:28:26+02:00 archlinux run.sh[101862]:"
STAMP = "2026-08-31 21:28:26,217"
def running(command: str, who: str = "ala") -> str:
"""One real journal line, in the shape journalctl -o short-iso emits."""
return f"{PREFIX} {STAMP} INFO signal_bot.dispatcher: running command {command} for {who}"
def test_a_running_line_is_read() -> None:
(line,) = parse([running("cat")])
assert (line.command, line.display_name, line.refused) == ("cat", "ala", False)
def test_the_timestamp_comes_from_the_journal_not_the_bot() -> None:
"""The journal's stamp carries an offset; the bot's asctime does not.
A naive stamp read as UTC lands the whole import two hours out, which is
exactly what the hour columns exist to get right.
"""
(line,) = parse([running("cat")])
# 2026-08-31T21:28:26+02:00 is 19:28:26 UTC.
assert line.at_ms == 1_788_204_506_000
def test_an_album_command_keeps_its_pseudonym_key() -> None:
"""`pack:p1`, not `pack`. A \\w+ pattern silently truncates at the colon.
That truncation would collapse thirteen different album commands into
one, which is the difference between "who is asked for most" and a
single meaningless total.
"""
(line,) = parse([running("pack:p1")])
assert line.command == "pack:p1"
def test_a_cooldown_refusal_is_read_as_one() -> None:
text = (
f"{PREFIX} {STAMP} INFO signal_bot.decorators: "
"cooldown refused cat for ala: told them 8 sekund"
)
(line,) = parse([text])
assert (line.command, line.refused) == ("cat", True)
def test_a_display_name_with_spaces_survives() -> None:
"""The pattern runs to end of line, because a display name is free text."""
(line,) = parse([running("cat", "ala z basia")])
assert line.display_name == "ala z basia"
def test_lines_that_are_not_about_commands_are_skipped() -> None:
"""Startup banners, tracebacks, envelope shapes, other units."""
noise = [
"2026-08-31T21:00:00+02:00 archlinux systemd[1474]: Started Signal group bot.",
f"{PREFIX} {STAMP} DEBUG signal_bot.dispatcher: envelope shape: {{'a': 'str'}}",
"not a journal line at all",
"",
]
assert list(parse(noise)) == []
def test_a_line_whose_stamp_is_not_a_timestamp_is_skipped() -> None:
"""Never guess a time. A wrong hour is worse than a missing row."""
assert list(parse(["wednesday archlinux run.sh[1]: running command cat for ala"])) == []
def test_a_naive_stamp_is_refused() -> None:
"""Without an offset there is no way to know which hour this was."""
text = "2026-08-31T21:28:26 archlinux run.sh[1]: running command cat for ala"
assert list(parse([text])) == []
def test_a_dst_boundary_is_read_from_the_offset() -> None:
"""Warsaw is +02:00 in summer and +01:00 in winter, and both occur.
Reading the wall clock and assuming one offset puts half the year's
rows an hour out, in a column whose whole purpose is the hour.
"""
summer = "2026-10-25T02:30:00+02:00 archlinux run.sh[1]: running command cat for ala"
winter = "2026-10-25T02:30:00+01:00 archlinux run.sh[1]: running command cat for ala"
(early,) = parse([summer])
(late,) = parse([winter])
assert late.at_ms - early.at_ms == 3_600_000

View File

@ -0,0 +1,30 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Polish agreement in the reply, split out of test_analytics_command.
Three forms, not two, and the rule reads the last *two* digits -- so 12, 13
and 14 take the many-form while 22, 23 and 24 do not. Getting this wrong is
the kind of thing the group notices immediately and mentions every time.
"""
from __future__ import annotations
import pytest
from signal_bot.analytics import render
@pytest.mark.parametrize(
("count", "expected"),
[(1, "1 raz"), (2, "2 razy"), (5, "5 razy"), (12, "12 razy"), (22, "22 razy")],
)
def test_polish_agreement_on_counts(count: int, expected: str) -> None:
"""Twelve takes the many-form and twenty-two does not."""
assert render.times_phrase(count) == expected
@pytest.mark.parametrize(
("count", "expected"),
[(1, "1 dzień"), (2, "2 dni"), (5, "5 dni"), (12, "12 dni")],
)
def test_polish_agreement_on_days(count: int, expected: str) -> None:
assert render.days_phrase(count) == expected

View File

@ -0,0 +1,83 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""The ContextVar that lets a fetcher report a timing it was never asked for."""
from __future__ import annotations
import pytest
from signal_bot.analytics import probe
def test_nothing_is_probing_by_default() -> None:
"""The var's default is what every no-op branch below tests against."""
assert probe.CURRENT.get() is None
def test_notes_outside_a_probe_are_dropped() -> None:
"""The whole point: a caller never asks whether it is being measured.
``http_client`` and ``context`` call these unconditionally, and the
offline end-to-end run drives both with no dispatcher above them.
"""
probe.note_send(5)
probe.note_upstream("example.test", 200, 5)
probe.note_item("Q1")
assert probe.CURRENT.get() is None
def test_a_probe_collects_what_ran_under_it() -> None:
"""One invocation's notes, from three different layers."""
with probe.probing() as active:
probe.note_send(120)
probe.note_send(80)
probe.note_upstream("api.example.test", 200, 40)
probe.note_upstream("cdn.example.test", 500, 15)
probe.note_item("Q42")
assert active.send_ms == 200
assert active.item_id == "Q42"
assert [(call.host, call.status, call.ms) for call in active.calls] == [
("api.example.test", 200, 40),
("cdn.example.test", 500, 15),
]
def _fail() -> None:
"""A handler that raises, in its own function so the raise is not inline."""
raise RuntimeError
def test_the_probe_is_closed_even_when_the_handler_raises() -> None:
"""A command that fails is still a command whose cost we wanted."""
seen: list[probe.Probe] = []
def run() -> None:
with probe.probing() as active:
seen.append(active)
probe.note_upstream("api.example.test", probe.NO_RESPONSE, 30)
_fail()
with pytest.raises(RuntimeError):
run()
assert probe.CURRENT.get() is None
assert len(seen[0].calls) == 1
def test_probes_do_not_leak_into_each_other() -> None:
"""reset(token), not set(None) -- two invocations in a row must not merge."""
with probe.probing() as first:
probe.note_send(10)
with probe.probing() as second:
probe.note_send(20)
assert first.send_ms == 10
assert second.send_ms == 20
def test_a_nested_probe_restores_the_outer_one() -> None:
"""Nesting is not expected, but a token reset must survive it if it happens."""
with probe.probing() as outer:
with probe.probing() as inner:
probe.note_send(3)
probe.note_send(7)
assert inner.send_ms == 3
assert outer.send_ms == 7

View File

@ -0,0 +1,159 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Family A, and the scope every family shares."""
from __future__ import annotations
from typing import TYPE_CHECKING
from signal_bot.analytics import usage
from signal_bot.analytics.scope import Scope, window
from signal_bot.analytics.uses import OUTCOME_COOLDOWN, OUTCOME_ERROR, OUTCOME_UNKNOWN
from tests.analytics_seed import ALA, BASIA, GROUP, OTHER, Shape, backfilled, ran, reconstructed
if TYPE_CHECKING:
import sqlite3
from signal_bot.store import Store
DAY = "2026-08-29"
NEXT = "2026-08-30"
def conn_of(store: Store) -> sqlite3.Connection:
"""The connection the query layer reads through."""
return store.commands.connection
def test_an_empty_history_is_a_state_not_an_error(store: Store) -> None:
"""Every panel renders on a fresh install, before anybody has typed."""
conn = conn_of(store)
scope = Scope()
assert window(conn, scope) == window(conn, scope)
assert window(conn, scope).events == 0
assert usage.by_command(conn, scope) == []
assert usage.by_person(conn, scope) == []
assert usage.totals(conn, scope)["attempts"] == 0
def test_the_window_says_what_it_covers_and_where_it_came_from(store: Store) -> None:
"""Three days is not a month, and the reader has to be able to tell."""
ran(store, day=DAY, command="cat")
reconstructed(store, day=DAY, hour=9, command="dog")
backfilled(store, day=NEXT, command="pope")
measured = window(conn_of(store), Scope())
assert (measured.first_day, measured.last_day, measured.days) == (DAY, NEXT, 2)
assert (measured.live, measured.journald, measured.daily) == (1, 1, 1)
assert not measured.enough_for_a_trend
def test_counts_include_every_source(store: Store) -> None:
""" "How often was this used" is answered by all three classes of row.
A reconstructed invocation still happened. Only the questions that need
an hour or a duration narrow further.
"""
ran(store, day=DAY, command="cat")
reconstructed(store, day=DAY, hour=9, command="cat")
backfilled(store, day=DAY, command="cat")
(use,) = usage.by_command(conn_of(store), Scope())
assert use.uses == 3
def test_a_group_scope_and_the_roll_up_run_the_same_sql(store: Store) -> None:
"""One query with a nullable parameter, so the two cannot drift."""
ran(store, day=DAY, command="cat", shape=Shape(group_id=GROUP))
ran(store, day=DAY, command="dog", shape=Shape(group_id=OTHER))
everywhere = usage.by_command(conn_of(store), Scope())
here = usage.by_command(conn_of(store), Scope(GROUP))
assert [use.command for use in everywhere] == ["cat", "dog"]
assert [use.command for use in here] == ["cat"]
def test_outcomes_are_counted_apart(store: Store) -> None:
"""Refused is not failed, and neither is a success."""
ran(store, day=DAY, command="cat", minute=1)
ran(store, day=DAY, command="cat", minute=2, shape=Shape(outcome=OUTCOME_COOLDOWN))
ran(
store,
day=DAY,
command="cat",
minute=3,
shape=Shape(outcome=OUTCOME_ERROR, error_class="ValueError"),
)
(use,) = usage.by_command(conn_of(store), Scope())
assert (use.uses, use.ok, use.refused, use.errors) == (3, 1, 1, 1)
assert use.failure_rate == 1 / 3
assert use.refusal_rate == 1 / 3
def test_a_command_nobody_ever_ran_has_no_rates(store: Store) -> None:
"""Dividing by an empty history is a zero, never a crash."""
empty = usage.CommandUse("cat", 0, 0, 0, 0, 0, "", "")
assert empty.failure_rate == 0.0
assert empty.refusal_rate == 0.0
def test_unknown_words_are_not_commands(store: Store) -> None:
"""A word nothing claims belongs to family D, not to the usage counts."""
ran(store, day=DAY, command="cat")
ran(store, day=DAY, command="pulapka", minute=5, shape=Shape(outcome=OUTCOME_UNKNOWN))
assert [use.command for use in usage.by_command(conn_of(store), Scope())] == ["cat"]
assert usage.totals(conn_of(store), Scope())["attempts"] == 1
def test_people_come_back_with_their_favourite(store: Store) -> None:
"""Most active first, each with the command they reach for."""
for minute in range(3):
ran(store, day=DAY, minute=minute, who=ALA, command="pope")
ran(store, day=DAY, minute=9, who=ALA, command="cat")
ran(store, day=DAY, minute=10, who=BASIA, command="dog")
people = usage.by_person(conn_of(store), Scope())
assert [(person.uuid, person.uses, person.favourite) for person in people] == [
(ALA, 4, "pope"),
(BASIA, 1, "dog"),
]
assert people[0].commands == 2
def test_the_matrix_is_every_person_against_every_command(store: Store) -> None:
"""What the management page draws and the group chat never sees."""
ran(store, day=DAY, who=ALA, command="cat")
ran(store, day=DAY, who=BASIA, command="cat", minute=1)
ran(store, day=DAY, who=BASIA, command="dog", minute=2)
assert usage.matrix(conn_of(store), Scope()) == [
(ALA, "cat", 1),
(BASIA, "cat", 1),
(BASIA, "dog", 1),
]
def test_commands_nobody_has_tried_are_named(store: Store) -> None:
"""Which registered commands are dead on arrival.
Takes the registry's list rather than reading it, because which commands
exist depends on the group -- the album's exist only where it does.
"""
ran(store, day=DAY, command="cat")
used = usage.by_command(conn_of(store), Scope())
assert usage.never_used(["cat", "dog", "pope"], used) == ["dog", "pope"]
assert usage.never_used([], used) == []
def test_totals_count_distinct_people_and_commands(store: Store) -> None:
ran(store, day=DAY, who=ALA, command="cat")
ran(store, day=DAY, who=BASIA, command="cat", minute=1)
ran(store, day=DAY, who=BASIA, command="dog", minute=2)
assert usage.totals(conn_of(store), Scope()) == {
"attempts": 3,
"ok": 3,
"people": 2,
"commands": 2,
}

View File

@ -0,0 +1,168 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""The command history table: what it holds, and what a redelivery does to it."""
from __future__ import annotations
from typing import TYPE_CHECKING
from signal_bot.analytics.probe import NO_RESPONSE, UpstreamCall
from signal_bot.analytics.uses import (
OUTCOME_COOLDOWN,
OUTCOME_OK,
SOURCE_DAILY,
SOURCE_LIVE,
CommandEvent,
)
if TYPE_CHECKING:
from collections.abc import Sequence
from signal_bot.store import Store
GROUP = "group-a=="
# The exact column set, in order. Pinned for the reason message_events' is:
# this table sits in a repository that leaked real names once, and a column
# that could hold free text must be a deliberate, test-visible addition.
EVENT_COLUMNS = (
"group_id",
"user_uuid",
"command",
"args",
"item_id",
"outcome",
"error_class",
"invoked_at_ms",
"day",
"local_hour",
"weekday",
"handler_ms",
"send_ms",
"source",
)
CALL_COLUMNS = (
"group_id",
"invoked_at_ms",
"command",
"host",
"status",
"duration_ms",
"seq",
)
def an_event(
*,
command: str = "cat",
outcome: str = OUTCOME_OK,
invoked_at_ms: int = 1,
local_hour: int = 12,
handler_ms: int | None = None,
source: str = SOURCE_LIVE,
calls: Sequence[UpstreamCall] = (),
) -> CommandEvent:
"""One minimal attempt. ``invoked_at_ms`` also disambiguates the key."""
return CommandEvent(
group_id=GROUP,
user_uuid="u1",
command=command,
outcome=outcome,
invoked_at_ms=invoked_at_ms,
day="2026-08-29",
local_hour=local_hour,
weekday=5,
handler_ms=handler_ms,
source=source,
calls=calls,
)
def rows(store: Store, table: str) -> list[dict[str, object]]:
"""Every row of one of the two tables, as plain dicts."""
queries = {
"command_events": "SELECT * FROM command_events ORDER BY invoked_at_ms, source",
"upstream_calls": "SELECT * FROM upstream_calls ORDER BY invoked_at_ms, seq",
}
return [dict(row) for row in store.commands.connection.execute(queries[table])]
def test_the_column_set_is_exactly_this(store: Store) -> None:
"""A new column here is a deliberate change, never a quiet one."""
store.commands.record(an_event())
assert tuple(rows(store, "command_events")[0]) == EVENT_COLUMNS
def test_the_call_column_set_is_exactly_this(store: Store) -> None:
"""The same pin on the table that records where the time went."""
store.commands.record(an_event(calls=(UpstreamCall("api.example.test", 200, 9),)))
assert tuple(rows(store, "upstream_calls")[0]) == CALL_COLUMNS
def test_a_replayed_frame_does_not_become_a_second_row(store: Store) -> None:
"""signal-cli replays undelivered frames when the bot reconnects.
Keyed on the envelope's own timestamp rather than the wall clock, so the
replay collides with the row it already wrote.
"""
for _ in range(3):
store.commands.record(an_event())
assert len(rows(store, "command_events")) == 1
def test_a_replay_does_not_duplicate_the_upstream_calls(store: Store) -> None:
"""The calls are keyed independently, so they need their own guard.
Without ON CONFLICT on this table too, a redelivered frame would triple
the recorded upstream traffic while the invocation count stayed at one.
"""
for _ in range(3):
store.commands.record(an_event(calls=(UpstreamCall("api.example.test", 200, 9),)))
assert len(rows(store, "upstream_calls")) == 1
def test_the_same_invocation_from_two_sources_is_two_rows(store: Store) -> None:
"""source is in the key: a backfilled row may not overwrite a live one.
The two disagree about the hour by construction, and whichever wrote
last would win silently.
"""
store.commands.record(an_event())
store.commands.record(an_event(source=SOURCE_DAILY, local_hour=-1))
assert [row["source"] for row in rows(store, "command_events")] == ["daily_uses", "live"]
def test_a_refusal_is_recorded_like_any_other_attempt(store: Store) -> None:
"""The rows that did not exist at all before this table did."""
store.commands.record(an_event(outcome=OUTCOME_COOLDOWN, handler_ms=0))
(row,) = rows(store, "command_events")
assert row["outcome"] == "refused_cooldown"
def test_an_invocation_with_no_upstream_calls_records_none(store: Store) -> None:
"""Every stats_* command is this shape, and the loop must not mind."""
store.commands.record(an_event(command="stats_today"))
assert rows(store, "upstream_calls") == []
def test_several_calls_keep_their_order(store: Store) -> None:
"""seq is the enumerate index, so two calls to one host stay distinct."""
store.commands.record(
an_event(
calls=(
UpstreamCall("api.example.test", 200, 40),
UpstreamCall("api.example.test", NO_RESPONSE, 5000),
)
)
)
assert [(row["seq"], row["status"]) for row in rows(store, "upstream_calls")] == [
(0, 200),
(1, 0),
]
def test_a_backfilled_row_has_no_duration(store: Store) -> None:
"""NULL, never 0. Zero milliseconds reads as data and would drag a p50."""
store.commands.record(an_event(source=SOURCE_DAILY, local_hour=-1))
(row,) = rows(store, "command_events")
assert row["handler_ms"] is None
assert row["send_ms"] is None

View File

@ -0,0 +1,171 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""The outcomes that had no data at all before command_events.
Split out of ``test_dispatcher_recording`` so both stay under the 250-line
cap. This half is the one that matters most: a refusal and a crash are rows
now, and the *use* -- the write that feeds the cooldown -- must still not
move for either of them.
"""
from __future__ import annotations
import datetime as dt
from typing import TYPE_CHECKING
import pytest
from signal_bot.analytics import probe
from signal_bot.decorators import cooldown, daily_limit
from signal_bot.registry import CommandRegistry
from tests.conftest import WATCHED_GROUP, envelope
from tests.test_dispatcher_recording import SENDER, build, calls, history
if TYPE_CHECKING:
from signal_bot.config import Config
from signal_bot.context import CommandContext
from signal_bot.discovery import GroupDirectory
from signal_bot.store import Store
from signal_bot.transport_fake import FakeTransport
NOW = dt.datetime(2026, 8, 29, 12, 0, tzinfo=dt.UTC)
@pytest.fixture
def ok_registry() -> CommandRegistry:
"""One command that succeeds and reports what it did on the way."""
async def handler(_ctx: CommandContext) -> None:
probe.note_upstream("api.example.test", 200, 40)
probe.note_send(120)
probe.note_item("Q42")
reg = CommandRegistry()
reg.register("cat", ["!cat"], handler, "kot")
return reg
async def test_a_refusal_does_not_move_the_cooldown(
config: Config,
directory: GroupDirectory,
store: Store,
transport: FakeTransport,
) -> None:
"""The invariant. A refused attempt is history, never a use.
If these two writes are ever merged, this fails and the bug it names
comes back: every "wait N seconds" reply would push the wait N seconds
further away, and waiting would never end.
"""
async def handler(_ctx: CommandContext) -> None:
pass
reg = CommandRegistry()
reg.register(
"cat",
["!cat"],
cooldown(3600.0, "poczekaj {seconds} {unit}")(handler),
"kot",
)
dispatcher = build(config, directory, reg, store, transport)
await dispatcher.handle(envelope(text="!cat", timestamp=1000))
first_use = store.last_used(WATCHED_GROUP, SENDER, "cat")
await dispatcher.handle(envelope(text="!cat", timestamp=2000))
outcomes = [row["outcome"] for row in history(store)]
assert outcomes == ["ok", "refused_cooldown"]
# The whole point: the second attempt is a row, and it moved nothing.
assert store.last_used(WATCHED_GROUP, SENDER, "cat") == first_use
assert store.uses_today(WATCHED_GROUP, SENDER, "cat", "2026-08-29") == 1
async def test_a_daily_refusal_names_its_own_gate(
config: Config,
directory: GroupDirectory,
store: Store,
transport: FakeTransport,
) -> None:
"""Which gate bit is the difference between "popular" and "mis-tuned"."""
async def handler(_ctx: CommandContext) -> None:
pass
reg = CommandRegistry()
reg.register(
"cat",
["!cat"],
daily_limit(1, "dosc na dzis")(handler),
"kot",
)
dispatcher = build(config, directory, reg, store, transport)
await dispatcher.handle(envelope(text="!cat", timestamp=1000))
await dispatcher.handle(envelope(text="!cat", timestamp=2000))
assert [row["outcome"] for row in history(store)] == ["ok", "refused_daily"]
async def test_a_crash_is_recorded_and_the_bot_survives_it(
config: Config,
directory: GroupDirectory,
store: Store,
transport: FakeTransport,
) -> None:
"""A command that fails every time used to look like one nobody ran."""
async def handler(_ctx: CommandContext) -> None:
raise ValueError
reg = CommandRegistry()
reg.register("cat", ["!cat"], handler, "kot")
dispatcher = build(config, directory, reg, store, transport)
await dispatcher.handle(envelope(text="!cat"))
(row,) = history(store)
assert row["outcome"] == "error"
# The class name, never the message: an exception's str() carries
# whatever it was processing, and here that is somebody's message.
assert row["error_class"] == "ValueError"
assert store.last_used(WATCHED_GROUP, SENDER, "cat") is None
async def test_recording_can_fail_without_taking_the_bot_down(
config: Config,
directory: GroupDirectory,
ok_registry: CommandRegistry,
store: Store,
transport: FakeTransport,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""A raise here would end the receive loop and the bot would go deaf.
Analytics are worth a great deal less than answering at all, so the
storage failure is logged and the use is still recorded.
"""
def explode(_event: object) -> None:
raise RuntimeError
monkeypatch.setattr(store.commands, "record", explode)
dispatcher = build(config, directory, ok_registry, store, transport)
await dispatcher.handle(envelope(text="!cat"))
assert store.last_used(WATCHED_GROUP, SENDER, "cat") == NOW.timestamp()
async def test_a_replayed_frame_is_recorded_once(
config: Config,
directory: GroupDirectory,
ok_registry: CommandRegistry,
store: Store,
transport: FakeTransport,
) -> None:
"""signal-cli replays undelivered frames when the bot reconnects."""
dispatcher = build(config, directory, ok_registry, store, transport)
for _ in range(3):
await dispatcher.handle(envelope(text="!cat", timestamp=5000))
assert len(history(store)) == 1
assert len(calls(store)) == 1

View File

@ -0,0 +1,133 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Every attempt becomes a row, and a refusal still does not move the cooldown.
The second half is the one that matters. ``CommandRefusedError``'s docstring
records the bug: recording a refused run slid the cooldown window forward
every time somebody was told to wait, so waiting the stated time never
helped. The command history now takes refusals *and* crashes, which makes
merging the two writes look like an obvious simplification. It is not, and
test_a_refusal_does_not_move_the_cooldown is what says so.
"""
from __future__ import annotations
import datetime as dt
from typing import TYPE_CHECKING, Any
import pytest
from signal_bot.analytics import probe
from signal_bot.dispatcher import Dispatcher
from signal_bot.registry import CommandRegistry
from tests.conftest import WATCHED_GROUP, envelope
if TYPE_CHECKING:
from signal_bot.config import Config
from signal_bot.context import CommandContext
from signal_bot.discovery import GroupDirectory
from signal_bot.store import Store
from signal_bot.transport_fake import FakeTransport
NOW = dt.datetime(2026, 8, 29, 12, 0, tzinfo=dt.UTC)
SENDER = "uuid-alice"
def history(store: Store) -> list[dict[str, Any]]:
"""Every recorded attempt, oldest first."""
return [
dict(row)
for row in store.commands.connection.execute(
"SELECT * FROM command_events ORDER BY invoked_at_ms"
)
]
def calls(store: Store) -> list[dict[str, Any]]:
"""Every recorded upstream call."""
return [
dict(row)
for row in store.commands.connection.execute("SELECT * FROM upstream_calls ORDER BY seq")
]
def build(
config: Config,
directory: GroupDirectory,
registry: CommandRegistry,
store: Store,
transport: FakeTransport,
) -> Dispatcher:
"""A dispatcher over one caller-supplied registry."""
return Dispatcher(
config=config,
directory=directory,
registry=registry,
store=store,
transport=transport,
clock=lambda: NOW,
)
@pytest.fixture
def ok_registry() -> CommandRegistry:
"""One command that succeeds and reports what it did on the way."""
async def handler(_ctx: CommandContext) -> None:
probe.note_upstream("api.example.test", 200, 40)
probe.note_send(120)
probe.note_item("Q42")
reg = CommandRegistry()
reg.register("cat", ["!cat"], handler, "kot")
return reg
async def test_a_success_is_recorded_with_what_it_cost(
config: Config,
directory: GroupDirectory,
ok_registry: CommandRegistry,
store: Store,
transport: FakeTransport,
) -> None:
"""The row nothing wrote before: outcome, item, and where the time went."""
dispatcher = build(config, directory, ok_registry, store, transport)
await dispatcher.handle(envelope(text="!cat"))
(row,) = history(store)
assert row["command"] == "cat"
assert row["outcome"] == "ok"
assert row["item_id"] == "Q42"
assert row["send_ms"] == 120
assert row["source"] == "live"
assert row["handler_ms"] >= 0
assert calls(store) == [
{
"group_id": WATCHED_GROUP,
"invoked_at_ms": row["invoked_at_ms"],
"command": "cat",
"host": "api.example.test",
"status": 200,
"duration_ms": 40,
"seq": 0,
}
]
async def test_the_history_and_the_message_share_a_key(
config: Config,
directory: GroupDirectory,
ok_registry: CommandRegistry,
store: Store,
transport: FakeTransport,
) -> None:
"""Both tables key on the envelope's own timestamp, or nothing joins."""
dispatcher = build(config, directory, ok_registry, store, transport)
await dispatcher.handle(envelope(text="!cat"))
(event,) = history(store)
row = store.events.connection.execute(
"SELECT sent_at_ms, local_hour, day FROM message_events WHERE is_command = 1"
).fetchone()
assert event["invoked_at_ms"] == row["sent_at_ms"]
assert event["local_hour"] == row["local_hour"]
assert event["day"] == row["day"]

View File

@ -8,6 +8,7 @@ from typing import TYPE_CHECKING
import httpx
import pytest
from signal_bot.analytics_command import register_analytics_command
from signal_bot.animals import register_animal_commands
from signal_bot.art_command import register_art_command
from signal_bot.config import DEFAULT_COMMONS_API_URL, DEFAULT_POEM_TEXT_URL
@ -71,6 +72,7 @@ def full_registry() -> CommandRegistry:
register_art_command(registry, fetcher, DEFAULT_COMMONS_API_URL, pools=((1,), (2,)))
register_poem_command(registry, fetcher, DEFAULT_POEM_TEXT_URL, slugs=("impas",))
register_stats_commands(registry)
register_analytics_command(registry)
register_suggest_command(registry)
register_help_command(registry)
return registry

View File

@ -0,0 +1,203 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""The Analytics tab's seven routes.
The empty-database cases matter as much as the populated ones: this page is
most useful exactly when something is wrong, so every panel has to render
against a bot that has never run.
"""
from __future__ import annotations
from typing import TYPE_CHECKING
from signal_bot.external import Result
from signal_bot.manage import routes_analytics
from signal_bot.store import Store
from tests.analytics_seed import ALA, BASIA, Shape, a_call, backfilled, ran, reconstructed
from tests.manage_fixture import FIXTURE_GROUP, client_for, deps_at
if TYPE_CHECKING:
from pathlib import Path
import pytest
from signal_bot.manage.deps import ManageDeps
DAY = "2026-08-29"
NEXT = "2026-08-30"
PANELS = ("overview", "time", "health", "demand", "behaviour", "content")
def seeded(tmp_path: Path) -> ManageDeps:
"""A management app pointed at a database with a small history in it."""
deps = deps_at(tmp_path)
with Store(str(deps.database_path)) as store:
for minute in range(3):
ran(
store,
day=DAY,
hour=19,
minute=minute,
who=ALA,
command="pope",
shape=Shape(
group_id=FIXTURE_GROUP,
item_id=f"Q{minute}",
handler_ms=300 + minute,
send_ms=200,
calls=(a_call("commons.example.test", 200, 40),),
),
)
ran(store, day=NEXT, hour=9, who=BASIA, command="cat", shape=Shape(group_id=FIXTURE_GROUP))
reconstructed(store, day=DAY, hour=8, command="dog", who=BASIA, group_id=FIXTURE_GROUP)
backfilled(store, day=DAY, command="gecko", who=ALA, group_id=FIXTURE_GROUP)
store.events.note_person(FIXTURE_GROUP, ALA, "ala", "", DAY)
store.suggestions.record_unknown(FIXTURE_GROUP, "!papiez!", "!papiez!", 5)
return deps
def test_every_panel_renders_against_a_bot_that_never_ran(tmp_path: Path) -> None:
"""A fresh install must show empty panels, never a stack trace."""
client = client_for(deps_at(tmp_path))
for panel in PANELS:
response = client.get(f"/api/analytics/{panel}")
assert response.status_code == 200, panel
body = response.json()
assert body["window"]["events"] == 0
assert client.get("/api/analytics/groups").json() == {"groups": []}
def test_every_panel_renders_against_a_file_of_nonsense(tmp_path: Path) -> None:
"""Half a database is the same answer as none of one."""
deps = deps_at(tmp_path)
deps.database_path.write_bytes(b"this is not a database")
client = client_for(deps)
for panel in PANELS:
assert client.get(f"/api/analytics/{panel}").status_code == 200
def test_the_groups_route_lists_what_has_a_history(tmp_path: Path) -> None:
client = client_for(seeded(tmp_path))
groups = client.get("/api/analytics/groups").json()["groups"]
assert [one["id"] for one in groups] == [FIXTURE_GROUP]
assert groups[0]["events"] == 6
def test_the_window_names_all_three_sources(tmp_path: Path) -> None:
"""The footnote's three numbers, so the page can say what is reconstructed."""
client = client_for(seeded(tmp_path))
window = client.get("/api/analytics/overview").json()["window"]
assert window["sources"] == {"live": 4, "journald": 1, "daily_uses": 1}
assert window["days"] == 2
assert window["enough_for_a_trend"] is False
def test_overview_names_people_rather_than_uuids(tmp_path: Path) -> None:
"""A uuid never appears on the page without a name beside it."""
client = client_for(seeded(tmp_path))
people = client.get("/api/analytics/overview").json()["people"]
named = {one["uuid"]: one["name"] for one in people}
assert named[ALA] == "ala"
assert named[BASIA] == ""
def test_time_returns_dense_buckets(tmp_path: Path) -> None:
"""Twenty-four hours and seven weekdays, empty ones included."""
body = client_for(seeded(tmp_path)).get("/api/analytics/time").json()
assert len(body["hours"]) == 24
assert len(body["weekday_hours"]) == 7
assert body["peak"]["busiest"] == 19
assert body["direction"] == "unknown"
def test_health_reports_timings_and_upstream(tmp_path: Path) -> None:
body = client_for(seeded(tmp_path)).get("/api/analytics/health").json()
assert body["latencies"][0]["command"] == "pope"
assert body["upstream"][0]["host"] == "commons.example.test"
assert body["upstream"][0]["failures"] == 0
def test_demand_groups_a_typo_by_what_it_nearly_hit(tmp_path: Path) -> None:
body = client_for(seeded(tmp_path)).get("/api/analytics/demand").json()
assert [one["target"] for one in body["near_misses"]] == ["!papiez"]
def test_behaviour_reports_sittings_and_share(tmp_path: Path) -> None:
body = client_for(seeded(tmp_path)).get("/api/analytics/behaviour").json()
assert body["sessions"]["count"] >= 1
assert "fraction" in body["share"]
def test_content_reports_pool_variety(tmp_path: Path) -> None:
body = client_for(seeded(tmp_path)).get("/api/analytics/content").json()
assert body["pools"][0] == {
"command": "pope",
"draws": 3,
"distinct": 3,
"repeat_rate": 0.0,
}
def test_a_group_filter_narrows_the_answer(tmp_path: Path) -> None:
"""Absent means the roll-up; present means one group, same SQL either way."""
client = client_for(seeded(tmp_path))
everywhere = client.get("/api/analytics/overview").json()
here = client.get("/api/analytics/overview", params={"group": FIXTURE_GROUP}).json()
nowhere = client.get("/api/analytics/overview", params={"group": "group.NOTHING"}).json()
assert everywhere["window"]["events"] == here["window"]["events"]
assert nowhere["window"]["events"] == 0
def test_the_triggers_come_from_the_alias_file() -> None:
"""A second process with no registry in it still knows every trigger."""
triggers = routes_analytics.triggers()
assert "!analityka" in triggers
assert "!kot" in triggers
def test_release_dates_come_from_the_tags_and_degrade_to_nothing() -> None:
"""The changelog says what shipped; only the tags say when.
Never raises: a checkout with no tags, or a deployment that is not a git
tree at all, comes back empty and the timeline draws unannotated.
"""
for version, day in routes_analytics.releases():
assert version[0].isdigit()
assert len(day) == len("2026-08-31")
def test_a_database_that_cannot_be_opened_is_the_empty_shape(tmp_path: Path) -> None:
"""A file that exists and refuses to open is still just "nothing to show".
Distinct from a corrupt one: sqlite opens a garbage file happily and
fails on the first query, so this covers the other half -- a permissions
problem, which raises at connect.
"""
deps = deps_at(tmp_path)
deps.database_path.write_bytes(b"")
deps.database_path.chmod(0o000)
try:
body = client_for(deps).get("/api/analytics/overview").json()
finally:
deps.database_path.chmod(0o600)
assert body["window"]["events"] == 0
def test_release_parsing_ignores_anything_that_is_not_a_version_tag(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Tags are not all releases, and a failed shell-out is not a crash."""
monkeypatch.setattr(
routes_analytics,
"run",
lambda *_args, **_kwargs: Result(
ok=True, stdout="v0.2.5 2026-08-31\nnot-a-release 2026-08-30\nv0.1.0\n", stderr=""
),
)
assert routes_analytics.releases() == (("0.2.5", "2026-08-31"),)
monkeypatch.setattr(
routes_analytics, "run", lambda *_a, **_k: Result(ok=False, stdout="", stderr="no git")
)
assert routes_analytics.releases() == ()

185
tests/test_schema_tables.py Normal file
View File

@ -0,0 +1,185 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Which tables survive what.
EPHEMERAL_TABLES used to answer two different questions with one tuple, and
this file exists because splitting it is exactly the kind of change that looks
right and silently stops clearing something. Every assertion here is against
either sqlite's own catalogue or the code that does the dropping -- never
against a second copy of the list.
"""
from __future__ import annotations
import sqlite3
from typing import TYPE_CHECKING
from signal_bot.schema import (
CLEARED_ON_LEAVE,
PERMANENT_TABLES,
REBUILT_ON_BUMP,
RETAINED_COLUMNS,
SCHEMA_VERSION,
TABLES,
)
from signal_bot.store import Store
if TYPE_CHECKING:
from pathlib import Path
GROUP = "group-a=="
def _declared(store: Store) -> set[str]:
"""Every table sqlite actually holds, as opposed to every one we listed."""
rows = store.events.connection.execute(
"SELECT name FROM sqlite_master WHERE type = 'table' AND name NOT LIKE 'sqlite_%'"
).fetchall()
return {str(row["name"]) for row in rows}
def _seed(store: Store, group_id: str = GROUP) -> None:
"""One row in every table a group can own."""
store.record_use(group_id, "u1", "cat", 1.0, "2026-08-29")
store.add_vote(group_id, "cat", "u1", 1.0)
store.set_meta(group_id, "last_reported_day", "2026-08-29")
store.suggestions.record_suggestion(group_id, "!pogoda", "na jutro", 1)
store.suggestions.record_unknown(group_id, "!cta", "!cta", 2)
# Both maps are spelled out rather than interpolated, for the reason
# forget_group spells its DELETEs out: a table name in an f-string is a
# SQL-injection shape even when the input is a module constant, and this
# repository takes no lint suppressions. PRAGMA additionally takes no
# parameters at all. test_every_table_can_be_counted keeps COUNTS complete.
COLUMNS = {
"command_uses": "PRAGMA table_info(command_uses)",
"daily_uses": "PRAGMA table_info(daily_uses)",
}
COUNTS = {
"command_uses": "SELECT count(*) AS n FROM command_uses WHERE group_id = ?",
"daily_uses": "SELECT count(*) AS n FROM daily_uses WHERE group_id = ?",
"votes": "SELECT count(*) AS n FROM votes WHERE group_id = ?",
"group_meta": "SELECT count(*) AS n FROM group_meta WHERE group_id = ?",
"message_events": "SELECT count(*) AS n FROM message_events WHERE group_id = ?",
"group_people": "SELECT count(*) AS n FROM group_people WHERE group_id = ?",
"unknown_commands": "SELECT count(*) AS n FROM unknown_commands WHERE group_id = ?",
"suggestions": "SELECT count(*) AS n FROM suggestions WHERE group_id = ?",
"command_events": "SELECT count(*) AS n FROM command_events WHERE group_id = ?",
"upstream_calls": "SELECT count(*) AS n FROM upstream_calls WHERE group_id = ?",
}
def _rows(store: Store, table: str, group_id: str = GROUP) -> int:
"""How many rows one group owns in one table."""
row = store.events.connection.execute(COUNTS[table], (group_id,)).fetchone()
return int(row["n"])
def test_every_table_can_be_counted() -> None:
"""A new table without a count query would skip every check below it."""
assert set(COUNTS) == set(TABLES)
def test_declared_columns_match_the_schema(store: Store) -> None:
"""RETAINED_COLUMNS is a second copy of the DDL, so it must be checked.
_drop_reshaped compares against it to decide whether a retained table is
still writable. A stale entry here would drop a perfectly good table on
every bump -- silently, since dropping is the recovery path -- and take
the use history it was moved out of REBUILT_ON_BUMP to protect.
"""
for table, expected in RETAINED_COLUMNS.items():
found = tuple(str(row["name"]) for row in store.events.connection.execute(COLUMNS[table]))
assert found == expected
def test_a_retained_table_with_the_wrong_shape_is_dropped(tmp_path: Path) -> None:
"""The failure _apply_schema exists to prevent, on a table it no longer rebuilds.
A database written before the group key holds a two-column command_uses.
CREATE TABLE IF NOT EXISTS will not touch it and the bump no longer drops
it, so without _drop_reshaped the bot boots clean and fails on every
INSERT.
"""
path = str(tmp_path / "legacy.sqlite3")
with sqlite3.connect(path) as raw:
raw.execute("CREATE TABLE daily_uses (uuid TEXT PRIMARY KEY, uses INTEGER NOT NULL)")
raw.execute("INSERT INTO daily_uses VALUES ('u1', 7)")
with Store(path) as store:
store.record_use(GROUP, "u1", "cat", 1.0, "2026-08-29")
assert store.uses_today(GROUP, "u1", "cat", "2026-08-29") == 1
def test_the_list_matches_what_sqlite_holds(store: Store) -> None:
"""TABLES is the schema's index, so it may not drift from the schema."""
assert set(TABLES) == _declared(store)
def test_no_table_is_both_kept_and_thrown_away() -> None:
"""A table in two of these tuples is a contradiction, not a configuration."""
assert not set(PERMANENT_TABLES) & set(REBUILT_ON_BUMP)
assert not set(PERMANENT_TABLES) & set(CLEARED_ON_LEAVE)
def test_the_history_is_permanent() -> None:
"""The four tables nothing can recompute, named where it is configured."""
for table in ("message_events", "command_events", "upstream_calls", "group_people"):
assert table in PERMANENT_TABLES
def test_the_use_counters_reset_on_leave_but_survive_a_bump() -> None:
"""The whole point of the split, asserted as one sentence.
daily_uses feeds a rate limiter *and* is a source the analytics history is
backfilled from. Clearing it when the bot leaves a group is correct;
dropping it on a schema bump would throw away use history that predates
command_events.
"""
for table in ("command_uses", "daily_uses"):
assert table in CLEARED_ON_LEAVE
assert table not in REBUILT_ON_BUMP
def test_forget_group_clears_every_cleared_table(store: Store) -> None:
"""The DELETEs in forget_group are spelled out, so something must count them.
Without this, adding a table to CLEARED_ON_LEAVE and forgetting the
matching DELETE leaves a cooldown alive across a leave and a rejoin --
invisible until somebody is told to wait for a command they never ran.
"""
_seed(store)
for table in CLEARED_ON_LEAVE:
assert _rows(store, table) > 0
store.forget_group(GROUP)
for table in CLEARED_ON_LEAVE:
assert _rows(store, table) == 0
def test_forget_group_leaves_the_permanent_tables_alone(store: Store) -> None:
"""A transient group listing must not be able to delete years of history."""
_seed(store)
store.forget_group(GROUP)
assert len(store.suggestions.filed()) == 1
assert len(store.suggestions.unknown()) == 1
def test_a_version_bump_keeps_the_use_history(tmp_path: Path) -> None:
"""A bump drops REBUILT_ON_BUMP and nothing else.
Written against a file rather than :memory: because the bump is only
observable across two openings of the same database.
"""
path = str(tmp_path / "bump.sqlite3")
with Store(path) as store:
_seed(store)
with sqlite3.connect(path) as raw:
raw.execute(f"PRAGMA user_version = {SCHEMA_VERSION - 1}")
with Store(path) as reopened:
# Thrown away, and cheap to lose.
assert _rows(reopened, "votes") == 0
assert _rows(reopened, "group_meta") == 0
# Kept, because nothing recomputes them.
assert _rows(reopened, "command_uses") == 1
assert _rows(reopened, "daily_uses") == 1
assert len(reopened.suggestions.filed()) == 1

View File

@ -161,11 +161,17 @@ def test_a_pre_group_database_is_rebuilt(tmp_path: Path) -> None:
def test_a_schema_rebuild_never_drops_the_history(tmp_path: Path) -> None:
"""The second destructive path the EPHEMERAL/PERMANENT split governs.
"""The second destructive path the table split governs.
forget_group is the obvious one. This is the other: a future session
bumping SCHEMA_VERSION for some cooldown table must not take years of
message history with it.
The use history now survives too, which it did not before command
analytics. daily_uses stopped being disposable the day it became the
only source covering the days that predate command_events, so it left
REBUILT_ON_BUMP. A bump can therefore no longer repair its shape --
_drop_reshaped is what took that job over.
"""
db = str(tmp_path / "bump.sqlite3")
with open_store(db) as first:
@ -179,7 +185,25 @@ def test_a_schema_rebuild_never_drops_the_history(tmp_path: Path) -> None:
with open_store(db) as second:
assert counted(second, GROUP) == 1
assert second.last_used(GROUP, "u1", "cat") is None
assert second.last_used(GROUP, "u1", "cat") == 100.0
assert second.uses_today(GROUP, "u1", "cat", "2026-08-29") == 1
def test_a_bump_still_clears_a_vote_in_progress(tmp_path: Path) -> None:
"""What REBUILT_ON_BUMP still holds is still rebuilt."""
db = str(tmp_path / "votes.sqlite3")
with open_store(db) as first:
first.add_vote(GROUP, "cat", "u1", 100.0)
first.set_meta(GROUP, "last_reported_day", "2026-08-29")
aged = sqlite3.connect(db)
aged.execute("PRAGMA user_version = 0")
aged.commit()
aged.close()
with open_store(db) as second:
assert second.count_votes(GROUP, "cat", 0.0) == 0
assert second.meta(GROUP, "last_reported_day") is None
def test_a_current_database_is_left_alone(tmp_path: Path) -> None:

View File

@ -5,7 +5,7 @@ from __future__ import annotations
from typing import TYPE_CHECKING
from signal_bot.schema import EPHEMERAL_TABLES, PERMANENT_TABLES
from signal_bot.schema import CLEARED_ON_LEAVE, PERMANENT_TABLES, REBUILT_ON_BUMP
if TYPE_CHECKING:
from signal_bot.store import Store
@ -69,7 +69,7 @@ def test_two_people_can_ask_for_the_same_command(store: Store) -> None:
def test_leaving_a_group_does_not_discard_the_wish_list(store: Store) -> None:
"""forget_group clears the ephemeral tables, and these are not among them.
"""forget_group clears CLEARED_ON_LEAVE, and these are not among them.
A transient group listing that omits a member is the exact failure
discovery._permitted guards against, and it must not be able to delete
@ -84,7 +84,7 @@ def test_leaving_a_group_does_not_discard_the_wish_list(store: Store) -> None:
def test_both_tables_are_declared_permanent() -> None:
"""The property above, asserted where it is actually configured."""
assert "suggestions" in PERMANENT_TABLES
assert "unknown_commands" in PERMANENT_TABLES
assert "suggestions" not in EPHEMERAL_TABLES
assert "unknown_commands" not in EPHEMERAL_TABLES
for table in ("suggestions", "unknown_commands"):
assert table in PERMANENT_TABLES
assert table not in CLEARED_ON_LEAVE
assert table not in REBUILT_ON_BUMP

View File

@ -0,0 +1,130 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Where a command's seconds actually go, recorded as it goes.
``DOCS-latency.md`` established the numbers that matter by reading three
journal lines by hand, once, and its own rule is that an upstream latency
claim gets measured before it gets designed around. These tests cover the
seam that makes the measurement continuous instead of anecdotal.
"""
from __future__ import annotations
from collections.abc import Callable
import httpx
import pytest
from signal_bot.analytics import probe
from signal_bot.http_client import ImageFetcher
IMAGE = b"\x89PNG\r\n\x1a\nfake"
MockHandler = Callable[[httpx.Request], httpx.Response]
def _fetcher(handler: MockHandler) -> ImageFetcher:
return ImageFetcher(httpx.AsyncClient(transport=httpx.MockTransport(handler)))
def _ok(_request: httpx.Request) -> httpx.Response:
return httpx.Response(200, content=IMAGE)
async def test_a_fetch_records_its_host_and_status() -> None:
"""One row per outbound request, per invocation."""
fetcher = _fetcher(_ok)
with probe.probing() as notes:
await fetcher.fetch_image("https://cdn.example.test/cat.png")
(call,) = notes.calls
assert call.host == "cdn.example.test"
assert call.status == 200
assert call.ms >= 0
async def test_only_the_host_is_recorded_never_the_path() -> None:
"""A URL carries identifiers; item_id is where those are governed.
The poem URL carries a Wolne Lektury slug, which is exactly the kind of
thing that must be a deliberate column rather than a side effect of
timing a request.
"""
fetcher = _fetcher(_ok)
with probe.probing() as notes:
await fetcher.fetch_image("https://wolnelektury.test/lektura/some-slug.txt")
(call,) = notes.calls
assert call.host == "wolnelektury.test"
assert "slug" not in repr(call)
async def test_a_json_fetch_and_its_image_are_two_calls() -> None:
"""fetch_base64 is a lookup and a download, and both cost time."""
def handler(request: httpx.Request) -> httpx.Response:
if "search" in str(request.url):
return httpx.Response(200, json=[{"url": "https://cdn.example.test/cat.png"}])
return httpx.Response(200, content=IMAGE)
fetcher = _fetcher(handler)
with probe.probing() as notes:
await fetcher.fetch_base64("https://api.example.test/search", [0, "url"])
assert [call.host for call in notes.calls] == [
"api.example.test",
"cdn.example.test",
]
async def test_a_bad_status_is_recorded_and_still_raises() -> None:
"""Upstream answering badly is data; it must not become silence."""
fetcher = _fetcher(lambda _request: httpx.Response(503))
with probe.probing() as notes, pytest.raises(httpx.HTTPStatusError):
await fetcher.fetch_image("https://cdn.example.test/cat.png")
(call,) = notes.calls
assert call.status == 503
async def test_a_request_that_never_answered_is_recorded_as_zero() -> None:
"""DNS, connect and timeout failures never reach a status code.
Losing them would make an upstream that is entirely down look merely
unused, which is the opposite of what the health panel is for.
"""
refused = "connection refused"
def refuse(request: httpx.Request) -> httpx.Response:
raise httpx.ConnectError(refused, request=request)
fetcher = _fetcher(refuse)
with probe.probing() as notes, pytest.raises(httpx.ConnectError):
await fetcher.fetch_image("https://cdn.example.test/cat.png")
(call,) = notes.calls
assert call.status == probe.NO_RESPONSE
async def test_a_fetch_outside_a_probe_records_nothing() -> None:
"""The offline runs and most tests drive the fetcher with no dispatcher."""
fetcher = _fetcher(_ok)
await fetcher.fetch_image("https://cdn.example.test/cat.png")
assert probe.CURRENT.get() is None
async def test_a_send_is_timed_even_when_it_fails() -> None:
"""A failed send still cost the seconds it took to fail."""
async def boom() -> None:
async with probe.timed_send():
raise RuntimeError
with probe.probing() as notes, pytest.raises(RuntimeError):
await boom()
assert notes.send_ms >= 0
async def test_two_sends_accumulate() -> None:
"""!pułapka sends a picture and then its description."""
with probe.probing() as notes:
async with probe.timed_send():
pass
async with probe.timed_send():
pass
assert notes.send_ms >= 0

89
tool/backfill_analytics.py Executable file
View File

@ -0,0 +1,89 @@
#!/usr/bin/env python3
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Import the command history that predates the table holding it.
python tool/backfill_analytics.py --db signal_bot.sqlite3 --dry-run
python tool/backfill_analytics.py --db signal_bot.sqlite3
Two sources, joined so that neither double-counts the other -- see
``signal_bot.analytics.backfill`` for why, which is the part worth reading
before trusting the numbers this prints.
The logic lives in ``signal_bot/``, where the 100% coverage gate can see it.
This file is the acquisition and the argparse, and lives in ``tool/``, which
is outside ``--cov`` and covered instead by ``tests/test_tool_imports.py``.
``--db`` is required and has no default, for the reason ManageDeps gives:
this writes to a database that holds a real group's history, and a default
is how the wrong one gets written to.
Run it against a copy first. It is insert-only and re-runnable, but "I ran
it on the live file to see what it would do" is not a thing to want back.
"""
from __future__ import annotations
import argparse
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from signal_bot.analytics import backfill, journal
from signal_bot.external import run
from signal_bot.store import open_store
UNIT = "signal-bot.service"
def read_journal(path: str | None) -> list[str]:
"""The bot's log, from a file or from journald itself.
Shells out through ``signal_bot.external``, the one module in this
repository allowed to start a process. ``tests/test_external.py`` keeps
that list closed by grepping every file for the import itself, so this
one routes through the sanctioned seam rather than joining the list --
and note that naming the import in prose is enough to trip that grep.
"""
if path is not None:
return Path(path).read_text(encoding="utf-8").splitlines()
result = run(
["journalctl", "--user", "-u", UNIT, "--no-pager", "-o", "short-iso"],
timeout=120.0,
)
if not result.ok:
sys.stderr.write(f"could not read the journal: {result.stderr.strip()}\n")
return []
return result.stdout.splitlines()
def main(argv: list[str] | None = None) -> int:
"""Parse arguments, run one pass, and print what it did."""
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--db", required=True, help="the database to write to")
parser.add_argument("--log", help="a captured journal, instead of reading journald")
parser.add_argument(
"--dry-run",
action="store_true",
help="report what would be imported, and write nothing",
)
args = parser.parse_args(argv)
lines = read_journal(args.log)
sys.stdout.write(f"journal lines read: {len(lines)}\n")
records = list(journal.parse(lines))
sys.stdout.write(f"command lines found: {len(records)}\n")
with open_store(args.db) as store:
report, events = backfill.plan(store, records)
if args.dry_run:
sys.stdout.write("-- dry run, nothing written --\n")
else:
backfill.apply(store, events)
for line in report.lines():
sys.stdout.write(f"{line}\n")
return 0
if __name__ == "__main__":
raise SystemExit(main())

View File

@ -3,7 +3,10 @@
"private": true,
"type": "module",
"packageManager": "pnpm@11.24.0",
"engines": { "node": ">=24", "pnpm": ">=10" },
"engines": {
"node": ">=24",
"pnpm": ">=10"
},
"scripts": {
"dev": "vite",
"build": "tsc -b && vite build",
@ -14,6 +17,11 @@
},
"dependencies": {
"@kuhyx/web-ui": "github:kuhyx/utils#web_ui-v0.3.1&path:/web_ui",
"@nivo/bar": "0.99.0",
"@nivo/calendar": "0.99.0",
"@nivo/core": "0.99.0",
"@nivo/heatmap": "0.99.0",
"@nivo/line": "0.99.0",
"react": "19.2.8",
"react-dom": "19.2.8"
},

View File

@ -11,6 +11,21 @@ importers:
'@kuhyx/web-ui':
specifier: github:kuhyx/utils#web_ui-v0.3.1&path:/web_ui
version: https://codeload.github.com/kuhyx/utils/tar.gz/6714d131983f05a0e67f55ac836e3e8d57045425#path:/web_ui(react@19.2.8)
'@nivo/bar':
specifier: 0.99.0
version: 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/calendar':
specifier: 0.99.0
version: 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/core':
specifier: 0.99.0
version: 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/heatmap':
specifier: 0.99.0
version: 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/line':
specifier: 0.99.0
version: 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
react:
specifier: 19.2.8
version: 19.2.8
@ -299,9 +314,107 @@ packages:
peerDependencies:
react: '>=19'
'@nivo/annotations@0.99.0':
resolution: {integrity: sha512-jCuuXPbvpaqaz4xF7k5dv0OT2ubn5Nt0gWryuTe/8oVsC/9bzSuK8bM9vBty60m9tfO+X8vUYliuaCDwGksC2g==}
peerDependencies:
react: ^16.14 || ^17.0 || ^18.0 || ^19.0
'@nivo/axes@0.99.0':
resolution: {integrity: sha512-3KschnmEL0acRoa7INSSOSEFwJLm54aZwSev7/r8XxXlkgRBriu6ReZy/FG0wfN+ljZ4GMvx+XyIIf6kxzvrZg==}
peerDependencies:
react: ^16.14 || ^17.0 || ^18.0 || ^19.0
'@nivo/bar@0.99.0':
resolution: {integrity: sha512-9yfMn7H6UF/TqtCwVZ/vihVAXUff9wWvSaeF2Z1DCfgr5S07qs31Qb2p0LZA+YgCWpaU7zqkeb3VZ4WCpZbrDA==}
peerDependencies:
react: ^16.14 || ^17.0 || ^18.0 || ^19.0
'@nivo/calendar@0.99.0':
resolution: {integrity: sha512-8Y6rIjAMt4hMWpkIUmb7skr8boAdoE7Bb4liAFSkkyRgsK4iFnFbaf1x5wLk7YmyY4v8LZ4PjNyGOYPBJU2wtw==}
peerDependencies:
react: ^16.14 || ^17.0 || ^18.0 || ^19.0
'@nivo/canvas@0.99.0':
resolution: {integrity: sha512-UxA8zb+NPwqmNm81hoyUZSMAikgjU1ukLf4KybVNyV8ejcJM+BUFXsb8DxTcLdt4nmCFHqM56GaJQv2hnAHmzg==}
'@nivo/colors@0.99.0':
resolution: {integrity: sha512-hyYt4lEFIfXOUmQ6k3HXm3KwhcgoJpocmoGzLUqzk7DzuhQYJo+4d5jIGGU0N/a70+9XbHIdpKNSblHAIASD3w==}
peerDependencies:
react: ^16.14 || ^17.0 || ^18.0 || ^19.0
'@nivo/core@0.99.0':
resolution: {integrity: sha512-olCItqhPG3xHL5ei+vg52aB6o+6S+xR2idpkd9RormTTUniZb8U2rOdcQojOojPY5i9kVeQyLFBpV4YfM7OZ9g==}
peerDependencies:
react: ^16.14 || ^17.0 || ^18.0 || ^19.0
'@nivo/heatmap@0.99.0':
resolution: {integrity: sha512-cnd5V6XYLvHtQ8GObUyYyEgmAUa5/ERYVmNXR1znA+yzmB+tGthJsOeGar+ntCb43pIL5HbhEaD3YpsYzBxOhQ==}
peerDependencies:
react: ^16.14 || ^17.0 || ^18.0 || ^19.0
'@nivo/legends@0.99.0':
resolution: {integrity: sha512-P16FjFqNceuTTZphINAh5p0RF0opu3cCKoWppe2aRD9IuVkvRm/wS5K1YwMCxDzKyKh5v0AuTlu9K6o3/hk8hA==}
peerDependencies:
react: ^16.14 || ^17.0 || ^18.0 || ^19.0
'@nivo/line@0.99.0':
resolution: {integrity: sha512-bAqTXSjpnpcGMs341qWFUi7hJTqQiNoSeJHsYPuPS3icuXPcp3WETQH+zRZACeEF79ZigeOWCW+dzODgne1y9w==}
peerDependencies:
react: ^16.14 || ^17.0 || ^18.0 || ^19.0
'@nivo/scales@0.99.0':
resolution: {integrity: sha512-g/2K4L6L8si6E2BWAHtFVGahtDKbUcO6xHJtlIZMwdzaJc7yB16EpWLK8AfI/A42KadLhJSJqBK3mty+c7YZ+w==}
'@nivo/text@0.99.0':
resolution: {integrity: sha512-ho3oZpAZApsJNjsIL5WJSAdg/wjzTBcwo1KiHBlRGUmD+yUWO8qp7V+mnYRhJchwygtRVALlPgZ/rlcW2Xr/MQ==}
peerDependencies:
react: ^16.14 || ^17.0 || ^18.0 || ^19.0
'@nivo/theming@0.99.0':
resolution: {integrity: sha512-KvXlf0nqBzh/g2hAIV9bzscYvpq1uuO3TnFN3RDXGI72CrbbZFTGzprPju3sy/myVsauv+Bb+V4f5TZ0jkYKRg==}
peerDependencies:
react: ^16.14 || ^17.0 || ^18.0 || ^19.0
'@nivo/tooltip@0.99.0':
resolution: {integrity: sha512-weoEGR3xAetV4k2P6k96cdamGzKQ5F2Pq+uyDaHr1P3HYArM879Pl+x+TkU0aWjP6wgUZPx/GOBiV1Hb1JxIqg==}
peerDependencies:
react: ^16.14 || ^17.0 || ^18.0 || ^19.0
'@nivo/voronoi@0.99.0':
resolution: {integrity: sha512-KfmMdidbYzhiUCki1FG4X4nHEFT4loK8G5bMBnmCl9U+S78W+gvkfrgD2Aoqp/Q9yKQvr3Y8UcZKSFZnn3HgjQ==}
peerDependencies:
react: ^16.14 || ^17.0 || ^18.0 || ^19.0
'@oxc-project/types@0.147.0':
resolution: {integrity: sha512-IJ3s6ltHLp45S0bh7phkX+gJO7A1Wuz2EaqpAhb8WjqDwbzMiWKHhyyT42tskaWjEYXtHtVCPpnBJVT9+dcRLg==}
'@react-spring/animated@10.1.2':
resolution: {integrity: sha512-yAsQ/bbp6+vko7WNCI1M00c6KLE9XKTGCrgRhQqS4JcK3oF5qBV4rHYrQEprvEvYXxt+1H5FsLS2eVValEPfFw==}
peerDependencies:
react: ^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0
'@react-spring/core@10.1.2':
resolution: {integrity: sha512-lPGOAg0V+PV3ucopOajD+YCxoe8twALqAeG4c/+sqVjBsyUNNdx8qfz/DcNSPKA8PV3+AyQELlCxlDfap9cmBQ==}
peerDependencies:
react: ^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0
'@react-spring/rafz@10.1.2':
resolution: {integrity: sha512-KC6vSFZyPnRJ2rXqipV9QqR4SaYbYXjvKfdYKWipNK2mWuV79gr20WmpVUOsTiEHGRY3WSOdWCHN+P9Pdaqb7Q==}
'@react-spring/shared@10.1.2':
resolution: {integrity: sha512-47/8bNQ/o0uEmxEnPBuERlC29VSqiTn5P9ln9tUMSVkYKYxBtouYE686F5kAGj+eHRPoNCw7drxWE9nv2d1LMw==}
peerDependencies:
react: ^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0
'@react-spring/types@10.1.2':
resolution: {integrity: sha512-G4CWowmVPz+rDG1y9QRVq/prZNxiNwQaHC0kgT/MgY6jAGgdRgTV4VThpvJDwWvUitk3xZB4soP4d36fjeQ09g==}
'@react-spring/web@10.1.2':
resolution: {integrity: sha512-KxDB3zaDqy9qFsu7fdxjyraAxweHH4k5TW5WGT/OuMK6hKaxSDfhrQfRBzfFaSVvMBf+NghbJFanllV4ia6i7A==}
peerDependencies:
react: ^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0
react-dom: ^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0
'@rolldown/binding-android-arm-eabi@1.2.6':
resolution: {integrity: sha512-b+jTcARdTiFLI6jB4a5XjTm0RWd6KcRfQj/I2356fxUZemiho9zQLxo0RtCuMDAyKcLo6cEltkgbQp6d1+sjjQ==}
engines: {node: ^20.19.0 || >=22.12.0}
@ -445,6 +558,42 @@ packages:
'@types/chai@5.2.3':
resolution: {integrity: sha512-Mw558oeA9fFbv65/y4mHtXDs9bPnFMZAL/jxdPFUpOHHIXX91mcgEHbS5Lahr+pwZFR8A7GQleRWeI6cGFC2UA==}
'@types/d3-color@3.1.3':
resolution: {integrity: sha512-iO90scth9WAbmgv7ogoq57O9YpKmFBbmoEoCHDB2xMBY0+/KVrqAaCDyCE16dUspeOvIxFFRI+0sEtqDqy2b4A==}
'@types/d3-delaunay@6.0.4':
resolution: {integrity: sha512-ZMaSKu4THYCU6sV64Lhg6qjf1orxBthaC161plr5KuPHo3CNm8DTHiLw/5Eq2b6TsNP0W0iJrUOFscY6Q450Hw==}
'@types/d3-format@1.4.5':
resolution: {integrity: sha512-mLxrC1MSWupOSncXN/HOlWUAAIffAEBaI4+PKy2uMPsKe4FNZlk7qrbTjmzJXITQQqBHivaks4Td18azgqnotA==}
'@types/d3-interpolate@3.0.4':
resolution: {integrity: sha512-mgLPETlrpVV1YRJIglr4Ez47g7Yxjl1lj7YKsiMCb27VJH9W8NVM6Bb9d8kkpG/uAQS5AmbA48q2IAolKKo1MA==}
'@types/d3-path@3.1.1':
resolution: {integrity: sha512-VMZBYyQvbGmWyWVea0EHs/BwLgxc+MKi1zLDCONksozI4YJMcTt8ZEuIR4Sb1MMTE8MMW49v0IwI5+b7RmfWlg==}
'@types/d3-scale-chromatic@3.1.0':
resolution: {integrity: sha512-iWMJgwkK7yTRmWqRB5plb1kadXyQ5Sj8V/zYlFGMUBbIPKQScw+Dku9cAAMgJG+z5GYDoMjWGLVOvjghDEFnKQ==}
'@types/d3-scale@4.0.9':
resolution: {integrity: sha512-dLmtwB8zkAeO/juAMfnV+sItKjlsw2lKdZVVy6LRr0cBmegxSABiLEpGVmSJJ8O08i4+sGR6qQtb6WtuwJdvVw==}
'@types/d3-shape@3.2.0':
resolution: {integrity: sha512-kVd74ta9eof3eJOvbNd1vGKS/XERRyQbT26Og63hIsvDO84cjD5gEOhsXf26w3FSoNlPVz84DOFcKv/oou+fMw==}
'@types/d3-time-format@2.3.4':
resolution: {integrity: sha512-xdDXbpVO74EvadI3UDxjxTdR6QIxm1FKzEA/+F8tL4GWWUg/hgvBqf6chql64U5A9ZUGWo7pEu4eNlyLwbKdhg==}
'@types/d3-time-format@3.0.4':
resolution: {integrity: sha512-or9DiDnYI1h38J9hxKEsw513+KVuFbEVhl7qdxcaudoiqWWepapUen+2vAriFGexr6W5+P4l9+HJrB39GG+oRg==}
'@types/d3-time@1.1.4':
resolution: {integrity: sha512-JIvy2HjRInE+TXOmIGN5LCmeO0hkFZx5f9FZ7kiN+D+YTcc8pptsiLiuHsvwxwC7VVKmJ2ExHUgNlAiV7vQM9g==}
'@types/d3-time@3.0.4':
resolution: {integrity: sha512-yuzZug1nkAAaBlBBikKZTgzCeA+k1uy4ZFwWANOfKw5z5LRhV0gNA7gNkKm7HoK+HRN0wX3EkxGk0fpbWhmB7g==}
'@types/deep-eql@4.0.2':
resolution: {integrity: sha512-c9h9dVVMigMPc4bwTvC5dxqtqJZwQPePsWjPlpSOnojbor6pGqdk541lfA7AqFQr5pB1BRdq0juY9db81BwyFw==}
@ -688,6 +837,59 @@ packages:
csstype@3.2.3:
resolution: {integrity: sha512-z1HGKcYy2xA8AGQfwrn0PAy+PB7X/GSj3UVJW9qKyn43xWa+gl5nXmU4qqLMRzWVLFC8KusUX8T/0kCiOYpAIQ==}
d3-array@3.2.4:
resolution: {integrity: sha512-tdQAmyA18i4J7wprpYq8ClcxZy3SC31QMeByyCFyRt7BVHdREQZ5lpzoe5mFEYZUWe+oq8HBvk9JjpibyEV4Jg==}
engines: {node: '>=12'}
d3-color@3.1.0:
resolution: {integrity: sha512-zg/chbXyeBtMQ1LbD/WSoW2DpC3I0mpmPdW+ynRTj/x2DAWYrIY7qeZIHidozwV24m4iavr15lNwIwLxRmOxhA==}
engines: {node: '>=12'}
d3-delaunay@6.0.4:
resolution: {integrity: sha512-mdjtIZ1XLAM8bm/hx3WwjfHt6Sggek7qH043O8KEjDXN40xi3vx/6pYSVTwLjEgiXQTbvaouWKynLBiUZ6SK6A==}
engines: {node: '>=12'}
d3-format@1.4.5:
resolution: {integrity: sha512-J0piedu6Z8iB6TbIGfZgDzfXxUFN3qQRMofy2oPdXzQibYGqPB/9iMcxr/TGalU+2RsyDO+U4f33id8tbnSRMQ==}
d3-format@3.1.2:
resolution: {integrity: sha512-AJDdYOdnyRDV5b6ArilzCPPwc1ejkHcoyFarqlPqT7zRYjhavcT3uSrqcMvsgh2CgoPbK3RCwyHaVyxYcP2Arg==}
engines: {node: '>=12'}
d3-interpolate@3.0.1:
resolution: {integrity: sha512-3bYs1rOD33uo8aqJfKP3JWPAibgw8Zm2+L9vBKEHJ2Rg+viTR7o5Mmv5mZcieN+FRYaAOWX5SJATX6k1PWz72g==}
engines: {node: '>=12'}
d3-path@3.1.0:
resolution: {integrity: sha512-p3KP5HCf/bvjBSSKuXid6Zqijx7wIfNW+J/maPs+iwR35at5JCbLUT0LzF1cnjbCHWhqzQTIN2Jpe8pRebIEFQ==}
engines: {node: '>=12'}
d3-scale-chromatic@3.1.0:
resolution: {integrity: sha512-A3s5PWiZ9YCXFye1o246KoscMWqf8BsD9eRiJ3He7C9OBaxKhAd5TFCdEx/7VbKtxxTsu//1mMJFrEt572cEyQ==}
engines: {node: '>=12'}
d3-scale@4.0.2:
resolution: {integrity: sha512-GZW464g1SH7ag3Y7hXjf8RoUuAFIqklOAq3MRl4OaWabTFJY9PN/E1YklhXLh+OQ3fM9yS2nOkCoS+WLZ6kvxQ==}
engines: {node: '>=12'}
d3-shape@3.2.0:
resolution: {integrity: sha512-SaLBuwGm3MOViRq2ABk3eLoxwZELpH6zhl3FbAoJ7Vm1gofKx6El1Ib5z23NUEhF9AsGl7y+dzLe5Cw2AArGTA==}
engines: {node: '>=12'}
d3-time-format@3.0.0:
resolution: {integrity: sha512-UXJh6EKsHBTjopVqZBhFysQcoXSv/5yLONZvkQ5Kk3qbwiUYkdX17Xa1PT6U1ZWXGGfB1ey5L8dKMlFq2DO0Ag==}
d3-time-format@4.1.0:
resolution: {integrity: sha512-dJxPBlzC7NugB2PDLwo9Q8JiTR3M3e4/XANkreKSUxF8vvXKqm1Yfq4Q5dl8budlunRVlUUaDUgFt7eA8D6NLg==}
engines: {node: '>=12'}
d3-time@1.1.0:
resolution: {integrity: sha512-Xh0isrZ5rPYYdqhAVk8VLnMEidhz5aP7htAADH6MfzgmmicPkTo8LhkLxci61/lCB7n7UmE3bN0leRt+qvkLxA==}
d3-time@3.1.0:
resolution: {integrity: sha512-VqKjzBLejbSMT4IgbmVgDjpkYrNWUYJnbCGo874u7MMKIWsILRX+OpX/gTk8MqjpT1A/c6HY2dCA77ZN0lkQ2Q==}
engines: {node: '>=12'}
data-urls@7.0.0:
resolution: {integrity: sha512-23XHcCF+coGYevirZceTVD7NdJOqVn+49IHyxgszm+JIiHLoB2TkmPtsYkNWT1pvRSGkc35L6NHs0yHkN2SumA==}
engines: {node: ^20.19.0 || ^22.12.0 || >=24.0.0}
@ -707,6 +909,9 @@ packages:
deep-is@0.1.4:
resolution: {integrity: sha512-oIPzksmTg4/MriiaYGO+okXDT7ztn/w3Eptv/+gSIdMdKsJo0u4CfYNFJPy+4SKMuCqGw2wxnA+URMg3t8a/bQ==}
delaunator@5.1.0:
resolution: {integrity: sha512-AGrQ4QSgssa1NGmWmLPqN5NY2KajF5MqxetNEO+o0n3ZwZZeTmt7bBnvzHWrmkZFxGgr4HdyFgelzgi06otLuQ==}
dequal@2.0.3:
resolution: {integrity: sha512-0je+qPKHEMohvfRTCEo3CrPG6cAzAYgmzKyxRiYSSDkS6eGJdyVJm7WaYA5ECaAD9wLB2T4EEeymA5aFVcYXCA==}
engines: {node: '>=6'}
@ -917,6 +1122,10 @@ packages:
resolution: {integrity: sha512-m6FAo/spmsW2Ab2fU35JTYwtOKa2yAwXSwgjSv1TJzh4Mh7mC3lzAOVLBprb72XsTrgkEIsl7YrFNAiDiRhIGg==}
engines: {node: '>=12'}
internmap@2.0.3:
resolution: {integrity: sha512-5Hh7Y1wQbvY5ooGgPbDaL5iYLAPzMTUrjMulskHLH6wnv/A+1q5rgEaiuqEjB+oxGXIVZs1FF+R/KPN3ZSQYYg==}
engines: {node: '>=12'}
is-builtin-module@5.0.0:
resolution: {integrity: sha512-f4RqJKBUe5rQkJ2eJEJBXSticB3hGbN9j0yxxMQFqIW89Jp9WYFtzfTcRlstDKVUTRzSOTLKRfO9vIztenwtxA==}
engines: {node: '>=18.20'}
@ -1077,6 +1286,9 @@ packages:
lodash.merge@4.6.2:
resolution: {integrity: sha512-0KpjqXRVvrYyCsX1swR/XTK0va6VQkQM6MNo7PqW77ByjAhoARA8EfrP1N4+KlKj8YS0ZUCtRT/YUuhyYDujIQ==}
lodash@4.18.1:
resolution: {integrity: sha512-dMInicTPVE8d1e5otfwmmjlxkZoUpiVLwyeTdUsi/Caj/gfzzblBcCE5sRHV/AsjuCmxWrte2TNGSYuCeCq+0Q==}
lru-cache@11.5.2:
resolution: {integrity: sha512-4pfM1Ff0x50o0tQwb5ucw/RzNyD0/YJME6IVcStalZuMWxdt3sR3huStTtxz4PUmvZfRguvDejasvQ2kifR11g==}
engines: {node: 20 || >=22}
@ -1212,6 +1424,12 @@ packages:
react-is@17.0.2:
resolution: {integrity: sha512-w2GsyukL62IJnlaff/nRegPQR94C/XXamvMWmSHRJ4y7Ts/4ocGRmTHvOs8PSE6pB3dWOrD/nueuU5sduBsQ4w==}
react-virtualized-auto-sizer@1.0.26:
resolution: {integrity: sha512-CblNyiNVw2o+hsa5/49NH2ogGxZ+t+3aweRvNSq7TVjDIlwk7ir4lencEg5HxHeSzwNarSkNkiu0qJSOXtxm5A==}
peerDependencies:
react: ^15.3.0 || ^16.0.0-alpha || ^17.0.0 || ^18.0.0 || ^19.0.0
react-dom: ^15.3.0 || ^16.0.0-alpha || ^17.0.0 || ^18.0.0 || ^19.0.0
react@19.2.8:
resolution: {integrity: sha512-PWaYA1L/q9u2u7xYQi+Y3L3Yfnie7XyLeaJICV1MGD6LprsBxcAqGjYyr0eY3p+QdsA+x/Irkt4Qif8D63+Sbw==}
engines: {node: '>=0.10.0'}
@ -1240,6 +1458,9 @@ packages:
resolution: {integrity: sha512-yE7KUfFvaBFzGPs5H3Ops1RevfUEsDc5Iz65rOwWg4lE8HJSYtle77uul3+573457oHvBKuHYDl/xqUkKpEEdw==}
engines: {node: '>=18'}
robust-predicates@3.0.3:
resolution: {integrity: sha512-NS3levdsRIUOmiJ8FZWCP7LG3QpJyrs/TE0Zpf1yvZu8cAJJ6QMW92H1c7kWpdIHo8RvmLxN/o2JXTKHp74lUA==}
rolldown@1.2.6:
resolution: {integrity: sha512-vMM4q3aixf46GiF1Kok8jDPFsEpXgFWGjUHXNkNHNm+Y2adXAG2dbX91jkti3i0ZRsOlcmbuzAz1poObSHCmUA==}
engines: {node: ^20.19.0 || >=22.12.0}
@ -1381,6 +1602,12 @@ packages:
uri-js@4.4.1:
resolution: {integrity: sha512-7rKUyy33Q1yc98pQ1DAmLtwX109F7TIfWlW1Ydo8Wl1ii1SeHieeh0HHfPeL2fMXK6z0s8ecKs9frCuLJvndBg==}
use-debounce@10.1.1:
resolution: {integrity: sha512-kvds8BHR2k28cFsxW8k3nc/tRga2rs1RHYCqmmGqb90MEeE++oALwzh2COiuBLO1/QXiOuShXoSN2ZpWnMmvuQ==}
engines: {node: '>= 16.0.0'}
peerDependencies:
react: '*'
vite@8.2.2:
resolution: {integrity: sha512-cFKLV/PRgAUlIRm5WjMjJ86jrftzpqcgH+Us+DS8mI3CDNiH30Whrz8uHL3+MOLPAgqbMBAqWdAHAphOAM+z/Q==}
engines: {node: ^20.19.0 || >=22.12.0}
@ -1761,8 +1988,241 @@ snapshots:
dependencies:
react: 19.2.8
'@nivo/annotations@0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)':
dependencies:
'@nivo/colors': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/core': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/theming': 0.99.0(react@19.2.8)
'@react-spring/web': 10.1.2(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
lodash: 4.18.1
react: 19.2.8
transitivePeerDependencies:
- react-dom
'@nivo/axes@0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)':
dependencies:
'@nivo/core': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/scales': 0.99.0
'@nivo/text': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/theming': 0.99.0(react@19.2.8)
'@react-spring/web': 10.1.2(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@types/d3-format': 1.4.5
'@types/d3-time-format': 2.3.4
d3-format: 1.4.5
d3-time-format: 3.0.0
react: 19.2.8
transitivePeerDependencies:
- react-dom
'@nivo/bar@0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)':
dependencies:
'@nivo/annotations': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/axes': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/canvas': 0.99.0
'@nivo/colors': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/core': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/legends': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/scales': 0.99.0
'@nivo/text': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/theming': 0.99.0(react@19.2.8)
'@nivo/tooltip': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@react-spring/web': 10.1.2(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@types/d3-scale': 4.0.9
'@types/d3-shape': 3.2.0
d3-scale: 4.0.2
d3-shape: 3.2.0
lodash: 4.18.1
react: 19.2.8
transitivePeerDependencies:
- react-dom
'@nivo/calendar@0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)':
dependencies:
'@nivo/core': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/legends': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/text': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/theming': 0.99.0(react@19.2.8)
'@nivo/tooltip': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@types/d3-scale': 4.0.9
'@types/d3-time': 1.1.4
'@types/d3-time-format': 3.0.4
d3-scale: 4.0.2
d3-time: 1.1.0
d3-time-format: 3.0.0
lodash: 4.18.1
react: 19.2.8
transitivePeerDependencies:
- react-dom
'@nivo/canvas@0.99.0': {}
'@nivo/colors@0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)':
dependencies:
'@nivo/core': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/theming': 0.99.0(react@19.2.8)
'@types/d3-color': 3.1.3
'@types/d3-scale': 4.0.9
'@types/d3-scale-chromatic': 3.1.0
d3-color: 3.1.0
d3-scale: 4.0.2
d3-scale-chromatic: 3.1.0
lodash: 4.18.1
react: 19.2.8
transitivePeerDependencies:
- react-dom
'@nivo/core@0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)':
dependencies:
'@nivo/theming': 0.99.0(react@19.2.8)
'@nivo/tooltip': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@react-spring/web': 10.1.2(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@types/d3-shape': 3.2.0
d3-color: 3.1.0
d3-format: 1.4.5
d3-interpolate: 3.0.1
d3-scale: 4.0.2
d3-scale-chromatic: 3.1.0
d3-shape: 3.2.0
d3-time-format: 3.0.0
lodash: 4.18.1
react: 19.2.8
react-virtualized-auto-sizer: 1.0.26(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
use-debounce: 10.1.1(react@19.2.8)
transitivePeerDependencies:
- react-dom
'@nivo/heatmap@0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)':
dependencies:
'@nivo/annotations': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/axes': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/colors': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/core': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/legends': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/scales': 0.99.0
'@nivo/text': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/theming': 0.99.0(react@19.2.8)
'@nivo/tooltip': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@react-spring/core': 10.1.2(react@19.2.8)
'@react-spring/web': 10.1.2(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@types/d3-scale': 4.0.9
d3-scale: 4.0.2
react: 19.2.8
transitivePeerDependencies:
- react-dom
'@nivo/legends@0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)':
dependencies:
'@nivo/colors': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/core': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/text': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/theming': 0.99.0(react@19.2.8)
'@types/d3-scale': 4.0.9
d3-scale: 4.0.2
react: 19.2.8
transitivePeerDependencies:
- react-dom
'@nivo/line@0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)':
dependencies:
'@nivo/annotations': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/axes': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/colors': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/core': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/legends': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/scales': 0.99.0
'@nivo/theming': 0.99.0(react@19.2.8)
'@nivo/tooltip': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/voronoi': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@react-spring/web': 10.1.2(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@types/d3-shape': 3.2.0
d3-shape: 3.2.0
react: 19.2.8
transitivePeerDependencies:
- react-dom
'@nivo/scales@0.99.0':
dependencies:
'@types/d3-interpolate': 3.0.4
'@types/d3-scale': 4.0.9
'@types/d3-time': 1.1.4
'@types/d3-time-format': 3.0.4
d3-interpolate: 3.0.1
d3-scale: 4.0.2
d3-time: 1.1.0
d3-time-format: 3.0.0
lodash: 4.18.1
'@nivo/text@0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)':
dependencies:
'@nivo/core': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/theming': 0.99.0(react@19.2.8)
'@react-spring/web': 10.1.2(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
react: 19.2.8
transitivePeerDependencies:
- react-dom
'@nivo/theming@0.99.0(react@19.2.8)':
dependencies:
lodash: 4.18.1
react: 19.2.8
'@nivo/tooltip@0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)':
dependencies:
'@nivo/core': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/theming': 0.99.0(react@19.2.8)
'@react-spring/web': 10.1.2(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
react: 19.2.8
transitivePeerDependencies:
- react-dom
'@nivo/voronoi@0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)':
dependencies:
'@nivo/core': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@nivo/theming': 0.99.0(react@19.2.8)
'@nivo/tooltip': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@types/d3-delaunay': 6.0.4
'@types/d3-scale': 4.0.9
d3-delaunay: 6.0.4
d3-scale: 4.0.2
react: 19.2.8
transitivePeerDependencies:
- react-dom
'@oxc-project/types@0.147.0': {}
'@react-spring/animated@10.1.2(react@19.2.8)':
dependencies:
'@react-spring/shared': 10.1.2(react@19.2.8)
'@react-spring/types': 10.1.2
react: 19.2.8
'@react-spring/core@10.1.2(react@19.2.8)':
dependencies:
'@react-spring/animated': 10.1.2(react@19.2.8)
'@react-spring/shared': 10.1.2(react@19.2.8)
'@react-spring/types': 10.1.2
react: 19.2.8
'@react-spring/rafz@10.1.2': {}
'@react-spring/shared@10.1.2(react@19.2.8)':
dependencies:
'@react-spring/rafz': 10.1.2
'@react-spring/types': 10.1.2
react: 19.2.8
'@react-spring/types@10.1.2': {}
'@react-spring/web@10.1.2(react-dom@19.2.8(react@19.2.8))(react@19.2.8)':
dependencies:
'@react-spring/animated': 10.1.2(react@19.2.8)
'@react-spring/core': 10.1.2(react@19.2.8)
'@react-spring/shared': 10.1.2(react@19.2.8)
'@react-spring/types': 10.1.2
csstype: 3.2.3
react: 19.2.8
react-dom: 19.2.8(react@19.2.8)
'@rolldown/binding-android-arm-eabi@1.2.6':
optional: true
@ -1856,6 +2316,36 @@ snapshots:
'@types/deep-eql': 4.0.2
assertion-error: 2.0.1
'@types/d3-color@3.1.3': {}
'@types/d3-delaunay@6.0.4': {}
'@types/d3-format@1.4.5': {}
'@types/d3-interpolate@3.0.4':
dependencies:
'@types/d3-color': 3.1.3
'@types/d3-path@3.1.1': {}
'@types/d3-scale-chromatic@3.1.0': {}
'@types/d3-scale@4.0.9':
dependencies:
'@types/d3-time': 3.0.4
'@types/d3-shape@3.2.0':
dependencies:
'@types/d3-path': 3.1.1
'@types/d3-time-format@2.3.4': {}
'@types/d3-time-format@3.0.4': {}
'@types/d3-time@1.1.4': {}
'@types/d3-time@3.0.4': {}
'@types/deep-eql@4.0.2': {}
'@types/esrecurse@4.3.1': {}
@ -2115,6 +2605,57 @@ snapshots:
csstype@3.2.3: {}
d3-array@3.2.4:
dependencies:
internmap: 2.0.3
d3-color@3.1.0: {}
d3-delaunay@6.0.4:
dependencies:
delaunator: 5.1.0
d3-format@1.4.5: {}
d3-format@3.1.2: {}
d3-interpolate@3.0.1:
dependencies:
d3-color: 3.1.0
d3-path@3.1.0: {}
d3-scale-chromatic@3.1.0:
dependencies:
d3-color: 3.1.0
d3-interpolate: 3.0.1
d3-scale@4.0.2:
dependencies:
d3-array: 3.2.4
d3-format: 3.1.2
d3-interpolate: 3.0.1
d3-time: 3.1.0
d3-time-format: 4.1.0
d3-shape@3.2.0:
dependencies:
d3-path: 3.1.0
d3-time-format@3.0.0:
dependencies:
d3-time: 1.1.0
d3-time-format@4.1.0:
dependencies:
d3-time: 3.1.0
d3-time@1.1.0: {}
d3-time@3.1.0:
dependencies:
d3-array: 3.2.4
data-urls@7.0.0:
dependencies:
whatwg-mimetype: 5.0.0
@ -2132,6 +2673,10 @@ snapshots:
deep-is@0.1.4: {}
delaunator@5.1.0:
dependencies:
robust-predicates: 3.0.3
dequal@2.0.3: {}
detect-indent@7.0.2: {}
@ -2356,6 +2901,8 @@ snapshots:
indent-string@5.0.0: {}
internmap@2.0.3: {}
is-builtin-module@5.0.0:
dependencies:
builtin-modules: 5.3.0
@ -2494,6 +3041,8 @@ snapshots:
lodash.merge@4.6.2: {}
lodash@4.18.1: {}
lru-cache@11.5.2: {}
lru-cache@5.1.1:
@ -2608,6 +3157,11 @@ snapshots:
react-is@17.0.2: {}
react-virtualized-auto-sizer@1.0.26(react-dom@19.2.8(react@19.2.8))(react@19.2.8):
dependencies:
react: 19.2.8
react-dom: 19.2.8(react@19.2.8)
react@19.2.8: {}
redent@3.0.0:
@ -2632,6 +3186,8 @@ snapshots:
reserved-identifiers@1.2.0: {}
robust-predicates@3.0.3: {}
rolldown@1.2.6:
dependencies:
'@oxc-project/types': 0.147.0
@ -2767,6 +3323,10 @@ snapshots:
dependencies:
punycode: 2.3.1
use-debounce@10.1.1(react@19.2.8):
dependencies:
react: 19.2.8
vite@8.2.2(@types/node@26.4.0)(yaml@2.9.0):
dependencies:
lightningcss: 1.33.0

View File

@ -0,0 +1,162 @@
// The Analytics tab's shapes, split out of types.ts so both stay under the
// 250-line cap. Hand-written like everything in that file, and mirroring the
// dataclasses in signal_bot/analytics one for one -- so a field that moves on
// the server fails the build here rather than rendering as undefined.
export interface Sources {
daily_uses: number;
journald: number;
live: number;
}
export interface Coverage {
days: number;
enough_for_a_trend: boolean;
events: number;
first_day: string;
last_day: string;
sources: Sources;
}
export interface AnalyticsGroup {
events: number;
first_day: string;
id: string;
last_day: string;
}
export interface CommandUse {
command: string;
errors: number;
first_day: string;
last_day: string;
ok: number;
people: number;
refused: number;
uses: number;
}
export interface PersonUse {
commands: number;
favourite: string;
// Empty when the bot has never seen a display name for this UUID.
name: string;
uses: number;
uuid: string;
}
export interface MatrixCell {
command: string;
name: string;
uses: number;
uuid: string;
}
export interface Overview {
commands: CommandUse[];
matrix: MatrixCell[];
people: PersonUse[];
totals: { attempts: number; commands: number; ok: number; people: number };
window: Coverage;
}
export interface DayCount {
day: string;
uses: number;
}
export interface MonthCount {
days: number;
month: string;
uses: number;
}
export interface Dying {
command: string;
idle_days: number;
last_day: string;
}
export interface Times {
by_command_hour: Record<string, number[]>;
days: DayCount[];
direction: string;
dying: Dying[];
hours: number[];
months: MonthCount[];
peak: {
busiest: number;
busiest_uses: number;
quietest: number;
quietest_uses: number;
};
weekday_hours: number[][];
window: Coverage;
}
export interface Latency {
command: string;
handler_p50: number;
handler_p95: number;
samples: number;
send_p50: number;
upstream_p50: number;
}
export interface Upstream {
calls: number;
failures: number;
host: string;
p50_ms: number;
}
export interface Health {
errors: { command: string; error_class: string; hits: number }[];
gates: { command: string; gate: string; hits: number }[];
latencies: Latency[];
upstream: Upstream[];
window: Coverage;
}
export interface NearMiss {
hits: number;
target: string;
variants: number;
}
export interface Demand {
filed: number;
near_misses: NearMiss[];
unclaimed: number;
unclaimed_words: number;
window: Coverage;
}
export interface Behaviour {
newcomers: { day: string; people: number }[];
pairs: { hits: number; left: string; right: string }[];
releases: { after: number; before: number; day: string; version: string }[];
repeats: { command: string; extra: number }[];
retention: { active: number; cohort: number; offset: number }[];
sessions: {
average_length: number;
commands: number;
count: number;
longest: number;
};
share: { commands: number; fraction: number; messages: number };
window: Coverage;
}
export interface Pool {
command: string;
distinct: number;
draws: number;
repeat_rate: number;
}
export interface Content {
pools: Pool[];
repeats: { command: string; draws: number; item_id: string }[];
window: Coverage;
}

View File

@ -1,3 +1,12 @@
import type {
AnalyticsGroup,
Behaviour,
Content,
Demand,
Health,
Overview,
Times,
} from "./analytics-types.ts";
import type { Roster } from "./setup.ts";
import type {
AlbumState,
@ -53,8 +62,28 @@ async function send<T>(path: string, method: string, body: unknown): Promise<T>
);
}
// One optional group, everywhere. Absent means the all-groups roll-up,
// which the server answers with the same SQL rather than a second query.
function scoped(path: string, group: null | string): string {
return group === null ? path : `${path}?group=${encodeURIComponent(group)}`;
}
export const api = {
album: async (): Promise<AlbumState> => get<AlbumState>("/api/album/state"),
analyticsBehaviour: async (group: null | string): Promise<Behaviour> =>
get<Behaviour>(scoped("/api/analytics/behaviour", group)),
analyticsContent: async (group: null | string): Promise<Content> =>
get<Content>(scoped("/api/analytics/content", group)),
analyticsDemand: async (group: null | string): Promise<Demand> =>
get<Demand>(scoped("/api/analytics/demand", group)),
analyticsGroups: async (): Promise<{ groups: AnalyticsGroup[] }> =>
get<{ groups: AnalyticsGroup[] }>("/api/analytics/groups"),
analyticsHealth: async (group: null | string): Promise<Health> =>
get<Health>(scoped("/api/analytics/health", group)),
analyticsOverview: async (group: null | string): Promise<Overview> =>
get<Overview>(scoped("/api/analytics/overview", group)),
analyticsTime: async (group: null | string): Promise<Times> =>
get<Times>(scoped("/api/analytics/time", group)),
env: async (): Promise<EnvironmentView> => get<EnvironmentView>("/api/env"),
exportSuggestions: async (): Promise<Export> =>
send<Export>("/api/suggestions/export", "POST", {}),

View File

@ -1,11 +1,13 @@
import { useState } from "react";
import { Album } from "./views/album.tsx";
import { Analytics } from "./views/analytics.tsx";
import { Health } from "./views/health.tsx";
import { Settings } from "./views/settings.tsx";
import { Suggestions } from "./views/suggestions.tsx";
const VIEWS = {
Analytics: Analytics,
Health: Health,
Ideas: Suggestions,
Photos: Album,

View File

@ -0,0 +1,99 @@
import { render, screen } from "@testing-library/react";
import { expect, it } from "vitest";
import { BarChart, CalendarChart, HeatChart, LineChart } from "./nivo.tsx";
// Assertions are on the wrapper's own testid, on the empty state, and on the
// axis labels Nivo emits as <text>. Deliberately not on Nivo's internals:
// those are its business and would break on an upgrade that changed nothing
// a reader can see.
it("draws bars, with the categories on the axis", () => {
render(
<BarChart
data={[
{ label: "pope", value: 25 },
{ label: "cat", value: 15 },
]}
legend="command"
testid="bars"
/>,
);
expect(screen.getByTestId("bars")).toBeInTheDocument();
expect(screen.getByText("pope")).toBeInTheDocument();
expect(screen.getByText("command")).toBeInTheDocument();
});
it("says so rather than drawing an empty axis", () => {
// The state a fresh install is in, so it has to read as a sentence.
render(<BarChart data={[]} legend="command" testid="bars" />);
expect(screen.getByTestId("bars-empty")).toHaveTextContent("Nothing recorded yet.");
});
it("draws a line for a series", () => {
render(
<LineChart
data={[{ data: [{ x: "2026-08-29", y: 3 }], id: "uses" }]}
legend="day"
testid="line"
/>,
);
expect(screen.getByTestId("line")).toBeInTheDocument();
});
it("treats a series with no points as empty", () => {
// A series that exists but holds nothing is the shape the API returns on
// a group nobody has used, and it must not draw an axis over nothing.
render(<LineChart data={[{ data: [], id: "uses" }]} legend="day" testid="line" />);
expect(screen.getByTestId("line-empty")).toBeInTheDocument();
});
it("treats no series at all as empty", () => {
render(<LineChart data={[]} legend="day" testid="line" />);
expect(screen.getByTestId("line-empty")).toBeInTheDocument();
});
it("draws a heat grid", () => {
render(
<HeatChart
data={[
{ data: [{ x: "19", y: 4 }], id: "Mon" },
{ data: [{ x: "19", y: 1 }], id: "Tue" },
]}
testid="heat"
/>,
);
expect(screen.getByTestId("heat")).toBeInTheDocument();
expect(screen.getByText("Mon")).toBeInTheDocument();
});
it("says so when there is no grid to draw", () => {
render(<HeatChart data={[]} testid="heat" />);
expect(screen.getByTestId("heat-empty")).toBeInTheDocument();
});
it("draws a calendar between two days", () => {
render(
<CalendarChart
data={[{ day: "2026-08-29", value: 3 }]}
from="2026-08-29"
testid="calendar"
to="2026-08-31"
/>,
);
expect(screen.getByTestId("calendar")).toBeInTheDocument();
});
it("refuses to draw a calendar with no range", () => {
// An empty window is what the API returns before anything is recorded,
// and Nivo's Calendar throws on a blank from/to rather than drawing.
render(<CalendarChart data={[{ day: "2026-08-29", value: 3 }]} from="" testid="cal" to="" />);
expect(screen.getByTestId("cal-empty")).toBeInTheDocument();
});
it("refuses to draw a calendar with no days", () => {
render(
<CalendarChart data={[]} from="2026-08-29" testid="cal" to="2026-08-31" />,
);
expect(screen.getByTestId("cal-empty")).toBeInTheDocument();
});

190
web/src/charts/nivo.tsx Normal file
View File

@ -0,0 +1,190 @@
import { Bar } from "@nivo/bar";
import { Calendar } from "@nivo/calendar";
import { HeatMap } from "@nivo/heatmap";
import { Line } from "@nivo/line";
import { useBoxSize } from "./size.ts";
// Four thin wrappers over Nivo, and every one of them says the same three
// things.
//
// 1. Fixed size, never the Responsive* variants. Those measure their own
// container, and a jsdom container has no size -- so a suite full of
// charts would assert against empty divs. useBoxSize supplies a real
// measurement in a browser and a stated fallback everywhere else.
// 2. animate={false}. Nivo animates through @react-spring, which schedules
// frames; under jsdom that is a source of flake and in a dashboard
// nobody is watching a bar grow.
// 3. An empty state with its own testid. "No data" is the state this page
// is in on a fresh install, and it has to render as a sentence rather
// than as an axis with nothing under it.
//
// Colours come from @kuhyx/web-ui's tokens rather than from a Nivo scheme,
// so the charts match the rest of the page in both themes.
const CATEGORICAL = [
"var(--cat-1)",
"var(--cat-2)",
"var(--cat-3)",
"var(--cat-4)",
"var(--cat-5)",
"var(--cat-6)",
];
const MARGIN = { bottom: 64, left: 64, right: 24, top: 16 };
const THEME = {
axis: {
legend: { text: { fill: "var(--text)" } },
ticks: { text: { fill: "var(--text-muted)" } },
},
grid: { line: { stroke: "var(--border)" } },
text: { fontFamily: "var(--font-sans)" },
};
function Empty({ label, testid }: Readonly<{ label: string; testid: string }>): React.JSX.Element {
return (
<p className="chart-empty" data-testid={testid}>
{label}
</p>
);
}
// Nivo's own BarDatum is an index signature, so a closed interface will not
// satisfy it. Named differently from Nivo's to keep the two apart at a
// glance, and widened with the index signature it demands.
export interface Bars {
[key: string]: number | string;
label: string;
value: number;
}
export function BarChart({
data,
legend,
testid,
}: Readonly<{
data: Bars[];
legend: string;
testid: string;
}>): React.JSX.Element {
const { ref, size } = useBoxSize<HTMLDivElement>();
if (data.length === 0) {
return <Empty label="Nothing recorded yet." testid={`${testid}-empty`} />;
}
return (
<div className="chart" data-testid={testid} ref={ref}>
<Bar
animate={false}
axisBottom={{ legend, legendOffset: 48, legendPosition: "middle", tickRotation: -35 }}
colors={CATEGORICAL}
data={data}
height={size.height}
indexBy="label"
keys={["value"]}
margin={MARGIN}
theme={THEME}
width={size.width}
/>
</div>
);
}
export interface Series {
data: { x: number | string; y: number }[];
id: string;
}
export function LineChart({
data,
legend,
testid,
}: Readonly<{
data: Series[];
legend: string;
testid: string;
}>): React.JSX.Element {
const { ref, size } = useBoxSize<HTMLDivElement>();
const isEmpty = data.every((series) => series.data.length === 0);
if (isEmpty || data.length === 0) {
return <Empty label="Nothing recorded yet." testid={`${testid}-empty`} />;
}
return (
<div className="chart" data-testid={testid} ref={ref}>
<Line
animate={false}
axisBottom={{ legend, legendOffset: 48, legendPosition: "middle", tickRotation: -35 }}
colors={CATEGORICAL}
data={data}
enableArea
height={size.height}
margin={MARGIN}
theme={THEME}
width={size.width}
yScale={{ min: 0, stacked: false, type: "linear" }}
/>
</div>
);
}
export interface HeatRow {
data: { x: string; y: number }[];
id: string;
}
export function HeatChart({
data,
testid,
}: Readonly<{
data: HeatRow[];
testid: string;
}>): React.JSX.Element {
const { ref, size } = useBoxSize<HTMLDivElement>();
if (data.length === 0) {
return <Empty label="Nothing recorded yet." testid={`${testid}-empty`} />;
}
return (
<div className="chart" data-testid={testid} ref={ref}>
<HeatMap
animate={false}
colors={{ colors: ["var(--surface-1)", "var(--accent)"], type: "sequential" }}
data={data}
height={size.height}
margin={MARGIN}
theme={THEME}
width={size.width}
/>
</div>
);
}
export function CalendarChart({
data,
from,
testid,
to,
}: Readonly<{
data: { day: string; value: number }[];
from: string;
testid: string;
to: string;
}>): React.JSX.Element {
const { ref, size } = useBoxSize<HTMLDivElement>();
if (from === "" || to === "" || data.length === 0) {
return <Empty label="Nothing recorded yet." testid={`${testid}-empty`} />;
}
return (
<div className="chart" data-testid={testid} ref={ref}>
<Calendar
data={data}
dayBorderColor="var(--bg)"
emptyColor="var(--surface-1)"
from={from}
height={size.height}
margin={MARGIN}
theme={THEME}
to={to}
width={size.width}
/>
</div>
);
}

View File

@ -0,0 +1,74 @@
import { renderHook } from "@testing-library/react";
import { afterEach, expect, it, vi } from "vitest";
import { FALLBACK, useBoxSize } from "./size.ts";
afterEach(() => {
vi.restoreAllMocks();
vi.unstubAllGlobals();
});
// One probe, used by every case that needs the ref to point at something.
// jsdom never attaches it for us, because nothing here renders the element
// the hook is meant to be handed.
function useProbe(): ReturnType<typeof useBoxSize<HTMLDivElement>> {
const box = useBoxSize<HTMLDivElement>();
box.ref.current ??= document.createElement("div");
return box;
}
function measuring(width: number, height: number): void {
vi.spyOn(HTMLElement.prototype, "clientWidth", "get").mockReturnValue(width);
vi.spyOn(HTMLElement.prototype, "clientHeight", "get").mockReturnValue(height);
}
it("falls back to a stated size when the container has none", () => {
// Every jsdom element reports 0x0, which is exactly the case the fallback
// exists for: Nivo's Responsive* components render nothing at that size,
// so a suite full of charts would assert against empty divs.
const { result } = renderHook(() => useBoxSize<HTMLDivElement>());
expect(result.current.size).toEqual(FALLBACK);
});
it("uses a real measurement when there is one", () => {
measuring(800, 400);
const { rerender, result } = renderHook(useProbe);
rerender();
expect(result.current.size).toEqual({ height: 400, width: 800 });
expect(result.current.size).not.toEqual(FALLBACK);
});
it("ignores a container that reports one dimension and not the other", () => {
// Half a measurement is not a measurement: a container mid-layout reports
// a width and no height, and drawing at zero height draws nothing.
measuring(800, 0);
const { rerender, result } = renderHook(useProbe);
rerender();
expect(result.current.size).toEqual(FALLBACK);
});
it("survives an environment with no ResizeObserver at all", () => {
// Guarded rather than assumed: a missing observer must degrade to
// "measured once", never throw on mount.
vi.stubGlobal("ResizeObserver", undefined);
const { result } = renderHook(useProbe);
expect(result.current.size).toEqual(FALLBACK);
});
it("observes its container and disconnects when the tab goes away", () => {
const disconnect = vi.fn();
const observe = vi.fn();
vi.stubGlobal(
"ResizeObserver",
class {
disconnect = disconnect;
observe = observe;
unobserve = vi.fn();
},
);
const { unmount } = renderHook(useProbe);
expect(observe).toHaveBeenCalledTimes(1);
unmount();
expect(disconnect).toHaveBeenCalledTimes(1);
});

63
web/src/charts/size.ts Normal file
View File

@ -0,0 +1,63 @@
import { useCallback, useEffect, useRef, useState } from "react";
// Nivo ships ResponsiveBar and friends, which measure their container and
// render nothing when it has no size. A jsdom container always has no size,
// so every chart in the test suite would render an empty <div> and every
// assertion about a chart would pass against nothing.
//
// So the charts here take explicit width and height, and this hook is what
// supplies them: a real measurement where one exists, and a stated fallback
// where one does not. The fallback is exported so a test can assert against
// a number rather than against whatever jsdom happened to report.
export const FALLBACK = { height: 320, width: 640 } as const;
export interface Box {
height: number;
width: number;
}
// A box with no area is not a measurement, it is a container that has not
// been laid out yet -- which in jsdom is permanent and in a browser is the
// first frame. Both want the fallback.
function hasArea(box: Box): boolean {
return box.width > 0 && box.height > 0;
}
/**
* Measure an element, falling back to a fixed size when it has none.
*
* Returns the ref to attach and the size to draw at.
*/
export function useBoxSize<T extends HTMLElement>(): {
ref: React.RefObject<null | T>;
size: Box;
} {
const reference = useRef<null | T>(null);
const [size, setSize] = useState<Box>(FALLBACK);
const measure = useCallback(() => {
const element = reference.current;
if (element === null) {
return;
}
const box = { height: element.clientHeight, width: element.clientWidth };
setSize(hasArea(box) ? box : FALLBACK);
}, []);
useEffect(() => {
measure();
// ResizeObserver does not exist in jsdom and is not polyfilled here --
// the test setup stubs it. Guarded rather than assumed so that a
// missing one degrades to "measured once" instead of throwing.
if (typeof ResizeObserver === "undefined" || reference.current === null) {
return;
}
const observer = new ResizeObserver(measure);
observer.observe(reference.current);
return () => {
observer.disconnect();
};
}, [measure]);
return { ref: reference, size };
}

View File

@ -205,3 +205,97 @@ input[type="checkbox"] {
[data-testid="note"] {
color: var(--success);
}
/* ---------------------------------------------------------------------------
Analytics.
Every value here is a token from @kuhyx/web-ui. The charts get their
colours the same way, so the page reads as one thing in both themes and a
palette change lands everywhere at once.
--------------------------------------------------------------------------- */
/* What the numbers are, before any of them are read. */
.footnote {
max-width: var(--measure);
font-size: var(--text-label);
line-height: 1.5;
color: var(--text-muted);
}
/* The headline row. Auto-fit rather than a fixed column count, so it stays
readable in the narrow window this page is usually opened in. */
.tiles {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(9rem, 1fr));
gap: var(--space-2);
margin: var(--space-3) 0;
}
.tile {
display: flex;
flex-direction: column;
gap: var(--space-1);
padding: var(--space-3);
background: var(--surface-1);
border: 1px solid var(--border);
border-radius: var(--radius-md);
}
.tile-value {
font-size: var(--text-title);
font-variant-numeric: tabular-nums;
line-height: 1;
color: var(--text);
}
.tile-label {
font-size: var(--text-label);
color: var(--text-muted);
}
/* A chart is given a height here rather than in the component: useBoxSize
measures this box, so the CSS is what decides the drawing size in a real
browser and the component's fallback only applies where there is no
layout at all. */
.chart {
width: 100%;
height: 22rem;
margin: var(--space-2) 0 var(--space-4);
overflow: hidden;
}
.chart-empty {
padding: var(--space-4);
font-size: var(--text-label);
color: var(--text-muted);
text-align: center;
background: var(--surface-1);
border: 1px dashed var(--border);
border-radius: var(--radius-md);
}
/* Latency figures are compared down a column, so they line up. */
[data-testid="health-table"] {
width: 100%;
border-collapse: collapse;
}
[data-testid="health-table"] th {
font-size: var(--text-label);
color: var(--text-muted);
text-align: left;
}
[data-testid="health-table"] td {
font-variant-numeric: tabular-nums;
}
[data-testid="health-table"] td,
[data-testid="health-table"] th {
padding: var(--space-1) var(--space-2);
border-bottom: 1px solid var(--border);
}

View File

@ -0,0 +1,172 @@
// The Analytics tab's fixtures, split out of fixtures.ts so both stay under
// the 250-line cap.
//
// Three days of history, because that is the shape the real page is in:
// enough to draw, not enough to claim a trend. Invented people and an
// invented group, like every other value in these files.
import type {
Behaviour,
Content,
Coverage,
Demand,
Health,
Overview,
Times,
} from "../api/analytics-types.ts";
import { GROUP } from "./fixtures.ts";
const COVERAGE: Coverage = {
days: 3,
enough_for_a_trend: false,
events: 40,
first_day: "2026-08-29",
last_day: "2026-08-31",
sources: { daily_uses: 9, journald: 25, live: 6 },
};
export const EMPTY_COVERAGE: Coverage = {
days: 0,
enough_for_a_trend: false,
events: 0,
first_day: "",
last_day: "",
sources: { daily_uses: 0, journald: 0, live: 0 },
};
export const ANALYTICS_GROUPS = {
groups: [{ events: 40, first_day: "2026-08-29", id: GROUP.id, last_day: "2026-08-31" }],
};
export const OVERVIEW: Overview = {
commands: [
{
command: "pope",
errors: 0,
first_day: "2026-08-29",
last_day: "2026-08-31",
ok: 24,
people: 3,
refused: 1,
uses: 25,
},
{
command: "cat",
errors: 2,
first_day: "2026-08-29",
last_day: "2026-08-30",
ok: 13,
people: 2,
refused: 0,
uses: 15,
},
],
matrix: [{ command: "pope", name: "ala", uses: 20, uuid: "uuid-ala" }],
people: [
{ commands: 2, favourite: "pope", name: "ala", uses: 30, uuid: "uuid-ala" },
{ commands: 1, favourite: "cat", name: "", uses: 10, uuid: "uuid-basia" },
],
totals: { attempts: 40, commands: 2, ok: 37, people: 2 },
window: COVERAGE,
};
export const TIMES: Times = {
by_command_hour: { pope: Array.from({ length: 24 }, (_, hour) => (hour === 19 ? 25 : 0)) },
days: [
{ day: "2026-08-29", uses: 10 },
{ day: "2026-08-30", uses: 12 },
{ day: "2026-08-31", uses: 18 },
],
direction: "unknown",
dying: [{ command: "gecko", idle_days: 4, last_day: "2026-08-27" }],
hours: Array.from({ length: 24 }, (_, hour) => (hour === 19 ? 25 : 1)),
months: [{ days: 3, month: "2026-08", uses: 40 }],
peak: { busiest: 19, busiest_uses: 25, quietest: 4, quietest_uses: 1 },
weekday_hours: Array.from({ length: 7 }, () => Array.from({ length: 24 }, () => 1)),
window: COVERAGE,
};
export const HEALTH: Health = {
errors: [{ command: "cat", error_class: "ImageFetchError", hits: 2 }],
gates: [{ command: "pope", gate: "refused_cooldown", hits: 1 }],
latencies: [
{
command: "pope",
handler_p50: 2845,
handler_p95: 2996,
samples: 6,
send_p50: 2156,
upstream_p50: 259,
},
],
upstream: [{ calls: 12, failures: 1, host: "commons.example.test", p50_ms: 254 }],
window: COVERAGE,
};
export const DEMAND: Demand = {
filed: 2,
near_misses: [{ hits: 17, target: "!pulapka", variants: 4 }],
unclaimed: 3,
unclaimed_words: 2,
window: COVERAGE,
};
export const BEHAVIOUR: Behaviour = {
newcomers: [{ day: "2026-08-29", people: 2 }],
pairs: [{ hits: 4, left: "cat", right: "pope" }],
releases: [{ after: 30, before: 10, day: "2026-08-31", version: "0.2.5" }],
repeats: [{ command: "pope", extra: 12 }],
retention: [
{ active: 2, cohort: 2, offset: 0 },
{ active: 1, cohort: 2, offset: 1 },
],
sessions: { average_length: 3.9, commands: 40, count: 10, longest: 12 },
share: { commands: 40, fraction: 0.35, messages: 114 },
window: COVERAGE,
};
export const CONTENT: Content = {
pools: [{ command: "pope", distinct: 20, draws: 25, repeat_rate: 0.2 }],
repeats: [{ command: "pope", draws: 3, item_id: "Q42" }],
window: COVERAGE,
};
export const EMPTY_PANELS = {
behaviour: {
newcomers: [],
pairs: [],
releases: [],
repeats: [],
retention: [],
sessions: { average_length: 0, commands: 0, count: 0, longest: 0 },
share: { commands: 0, fraction: 0, messages: 0 },
window: EMPTY_COVERAGE,
},
content: { pools: [], repeats: [], window: EMPTY_COVERAGE },
demand: {
filed: 0,
near_misses: [],
unclaimed: 0,
unclaimed_words: 0,
window: EMPTY_COVERAGE,
},
health: { errors: [], gates: [], latencies: [], upstream: [], window: EMPTY_COVERAGE },
overview: {
commands: [],
matrix: [],
people: [],
totals: { attempts: 0, commands: 0, ok: 0, people: 0 },
window: EMPTY_COVERAGE,
},
times: {
by_command_hour: {},
days: [],
direction: "unknown",
dying: [],
hours: [],
months: [],
peak: { busiest: -1, busiest_uses: 0, quietest: -1, quietest_uses: 0 },
weekday_hours: [],
window: EMPTY_COVERAGE,
},
};

View File

@ -80,3 +80,9 @@ export const SUGGESTIONS: SuggestionsView = {
};
export const NO_SUGGESTIONS: SuggestionsView = { suggestions: [], unknown: [] };
// -- analytics ---------------------------------------------------------------
//
// Invented people and an invented group, like everything else here. Three
// days of history, because that is the shape the real page is in: enough to
// draw, not enough to claim a trend.

View File

@ -1 +1,28 @@
import { beforeEach, vi } from "vitest";
import "@testing-library/jest-dom/vitest";
// jsdom has no ResizeObserver and the charts observe their container.
//
// Stubbed rather than polyfilled: nothing here needs it to fire, only to
// exist. useBoxSize's own test covers both the present and the absent case
// by stubbing it away again.
class StubResizeObserver implements ResizeObserver {
disconnect(): void {
// Nothing to disconnect: this observer never fires.
}
observe(): void {
// A jsdom element has no size, so there is nothing to report.
}
unobserve(): void {
// Symmetry with observe.
}
}
// Re-stubbed per test rather than assigned once, because most suites here
// call vi.unstubAllGlobals() in afterEach to put fetch back -- which would
// take this with it and leave every later chart throwing on mount.
beforeEach(() => {
vi.stubGlobal("ResizeObserver", StubResizeObserver);
});

View File

@ -0,0 +1,207 @@
import { useEffect, useState } from "react";
import type {
Behaviour,
Content,
Demand,
Health,
Overview,
Times,
} from "../api/analytics-types.ts";
import { api } from "../api/client.ts";
import { BarChart, CalendarChart, HeatChart, LineChart } from "../charts/nivo.tsx";
import { Sources } from "./analytics.tsx";
const WEEKDAYS = ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"];
const TOP = 12;
interface Panels {
behaviour: Behaviour;
content: Content;
demand: Demand;
health: Health;
overview: Overview;
times: Times;
}
function percent(fraction: number): string {
return `${String(Math.round(fraction * 100))}%`;
}
function Tile({ label, value }: Readonly<{ label: string; value: string }>): React.JSX.Element {
return (
<div className="tile">
<span className="tile-value">{value}</span>
<span className="tile-label">{label}</span>
</div>
);
}
export function AnalyticsPanels({ group }: Readonly<{ group: null | string }>): React.JSX.Element {
const [panels, setPanels] = useState<null | Panels>(null);
useEffect(() => {
// One state object rather than six, so a panel never renders against a
// window that belongs to the previously selected group.
void Promise.all([
api.analyticsOverview(group),
api.analyticsTime(group),
api.analyticsHealth(group),
api.analyticsDemand(group),
api.analyticsBehaviour(group),
api.analyticsContent(group),
]).then(([overview, times, health, demand, behaviour, content]) => {
setPanels({ behaviour, content, demand, health, overview, times });
});
}, [group]);
if (panels === null) {
return <p data-testid="loading">Reading the database…</p>;
}
const { behaviour, content, demand, health, overview, times } = panels;
return (
<>
<Sources coverage={overview.window} />
<div className="tiles" data-testid="tiles">
<Tile label="uses" value={String(overview.totals.attempts)} />
<Tile label="commands" value={String(overview.totals.commands)} />
<Tile label="people" value={String(overview.totals.people)} />
<Tile label="of all messages" value={percent(behaviour.share.fraction)} />
<Tile
label="busiest hour"
value={times.peak.busiest < 0 ? "—" : `${String(times.peak.busiest)}:00`}
/>
<Tile label="sittings" value={String(behaviour.sessions.count)} />
</div>
<h3>Which commands get used</h3>
<BarChart
data={overview.commands.slice(0, TOP).map((one) => ({ label: one.command, value: one.uses }))}
legend="command"
testid="chart-commands"
/>
<h3>When</h3>
<BarChart
data={times.hours.map((uses, hour) => ({ label: `${String(hour).padStart(2, "0")}:00`, value: uses }))}
legend="hour of day (Europe/Warsaw)"
testid="chart-hours"
/>
<HeatChart
data={times.weekday_hours.map((row, index) => {
return {
data: row.map((uses, hour) => ({ x: String(hour).padStart(2, "0"), y: uses })),
id: WEEKDAYS[index] ?? String(index),
};
})}
testid="chart-weekday-hours"
/>
<LineChart
data={[
{
data: times.days.map((one) => ({ x: one.day, y: one.uses })),
id: "uses",
},
]}
legend="day"
testid="chart-days"
/>
<CalendarChart
data={times.days.map((one) => ({ day: one.day, value: one.uses }))}
from={times.window.first_day}
testid="chart-calendar"
to={times.window.last_day}
/>
<h3>What needs work</h3>
<table data-testid="health-table">
<thead>
<tr>
<th>command</th>
<th>p50</th>
<th>p95</th>
<th>send p50</th>
<th>upstream p50</th>
</tr>
</thead>
<tbody>
{health.latencies.map((one) => {
return (
<tr key={one.command}>
<td>{one.command}</td>
<td>{one.handler_p50} ms</td>
<td>{one.handler_p95} ms</td>
<td>{one.send_p50} ms</td>
<td>{one.upstream_p50} ms</td>
</tr>
);
})}
</tbody>
</table>
{health.upstream.length === 0 ? (
<p data-testid="no-upstream">No upstream calls recorded yet.</p>
) : (
<ul data-testid="upstream">
{health.upstream.map((one) => {
return (
<li key={one.host}>
{one.host} — {one.calls} calls, {one.failures} failed, p50 {one.p50_ms} ms
</li>
);
})}
</ul>
)}
<h3>What people wanted and could not get</h3>
{demand.near_misses.length === 0 ? (
<p data-testid="no-demand">No near misses recorded.</p>
) : (
<ul data-testid="demand">
{demand.near_misses.map((one) => {
return (
<li key={one.target}>
{one.target} — {one.hits} near {one.hits === 1 ? "miss" : "misses"} across{" "}
{one.variants} {one.variants === 1 ? "spelling" : "spellings"}
</li>
);
})}
</ul>
)}
<h3>How it gets used</h3>
<p data-testid="sessions">
{behaviour.sessions.count} sittings, {behaviour.sessions.average_length.toFixed(1)}{" "}
commands each, longest {behaviour.sessions.longest}.
</p>
<LineChart
data={[
{
data: behaviour.retention.map((one) => ({ x: one.offset, y: one.active })),
id: "still using it",
},
]}
legend="days after first use"
testid="chart-retention"
/>
<h3>Whether the pools repeat</h3>
{content.pools.length === 0 ? (
<p data-testid="no-pools">Nothing that names what it served yet.</p>
) : (
<ul data-testid="pools">
{content.pools.map((one) => {
return (
<li key={one.command}>
{one.command} — {one.draws} draws, {one.distinct} distinct, repeat{" "}
{percent(one.repeat_rate)}
</li>
);
})}
</ul>
)}
</>
);
}

View File

@ -0,0 +1,33 @@
// The two servers the Analytics tests run against: one with a small history
// and one that has never seen a command. Shared so both test files describe
// the same server rather than two that drift.
import {
ANALYTICS_GROUPS,
BEHAVIOUR,
CONTENT,
DEMAND,
EMPTY_PANELS,
HEALTH,
OVERVIEW,
TIMES,
} from "../test/analytics-fixtures.ts";
export const ROUTES = {
"/api/analytics/behaviour": { body: BEHAVIOUR },
"/api/analytics/content": { body: CONTENT },
"/api/analytics/demand": { body: DEMAND },
"/api/analytics/groups": { body: ANALYTICS_GROUPS },
"/api/analytics/health": { body: HEALTH },
"/api/analytics/overview": { body: OVERVIEW },
"/api/analytics/time": { body: TIMES },
};
export const EMPTY_ROUTES = {
"/api/analytics/behaviour": { body: EMPTY_PANELS.behaviour },
"/api/analytics/content": { body: EMPTY_PANELS.content },
"/api/analytics/demand": { body: EMPTY_PANELS.demand },
"/api/analytics/groups": { body: { groups: [] } },
"/api/analytics/health": { body: EMPTY_PANELS.health },
"/api/analytics/overview": { body: EMPTY_PANELS.overview },
"/api/analytics/time": { body: EMPTY_PANELS.times },
};

View File

@ -0,0 +1,157 @@
import { render, screen, waitFor } from "@testing-library/react";
import userEvent from "@testing-library/user-event";
import { afterEach, expect, it, vi } from "vitest";
import { ANALYTICS_GROUPS, EMPTY_PANELS, TIMES } from "../test/analytics-fixtures.ts";
import { serve } from "../test/http.ts";
import { EMPTY_ROUTES, ROUTES } from "./analytics-routes.ts";
const [ONLY_GROUP] = ANALYTICS_GROUPS.groups;
import { Analytics } from "./analytics.tsx";
// The states that are not "a populated page": nothing recorded, exactly one
// of something, and a history long enough to have a direction. Split out of
// analytics.test.tsx so both stay under the 250-line cap.
afterEach(() => {
vi.unstubAllGlobals();
});
it("renders every panel against a bot that has never run", async () => {
// The state a fresh install is in. Every panel has to say so rather than
// draw an axis over nothing.
serve(EMPTY_ROUTES);
render(<Analytics />);
await waitFor(() => {
expect(screen.getByTestId("chart-commands-empty")).toBeInTheDocument();
});
expect(screen.getByTestId("no-upstream")).toBeInTheDocument();
expect(screen.getByTestId("no-demand")).toBeInTheDocument();
expect(screen.getByTestId("no-pools")).toBeInTheDocument();
expect(screen.getByTestId("chart-calendar-empty")).toBeInTheDocument();
expect(screen.getByTestId("tiles")).toHaveTextContent("—");
});
it("asks the server for one group when one is picked", async () => {
// A group id is 44 characters of base64, so it is picked and never typed.
const fetcher = serve(ROUTES);
render(<Analytics />);
await waitFor(() => {
expect(screen.getByTestId("group-picker")).toBeInTheDocument();
});
await userEvent.selectOptions(screen.getByTestId("group-picker"), ONLY_GROUP?.id ?? "");
await waitFor(() => {
const asked = fetcher.mock.calls.map((call) => String(call[0]));
expect(asked.some((url) => url.includes("group="))).toBe(true);
});
});
it("agrees with itself about one of anything", async () => {
// Singular. The footnote reads as prose and "1 uses over 1 days" is the
// sort of thing that gets noticed every single time.
const one = {
days: 1,
enough_for_a_trend: false,
events: 1,
first_day: "2026-08-29",
last_day: "2026-08-29",
sources: { daily_uses: 0, journald: 0, live: 1 },
};
serve({
...EMPTY_ROUTES,
"/api/analytics/demand": {
body: {
...EMPTY_PANELS.demand,
near_misses: [{ hits: 1, target: "!kot", variants: 1 }],
window: one,
},
},
"/api/analytics/overview": { body: { ...EMPTY_PANELS.overview, window: one } },
});
render(<Analytics />);
await waitFor(() => {
expect(screen.getByTestId("sources")).toHaveTextContent("1 recorded use over 1 day");
});
expect(screen.getByTestId("demand")).toHaveTextContent("1 near miss");
expect(screen.getByTestId("demand")).toHaveTextContent("1 spelling");
});
it("states a direction once there are enough days to have one", async () => {
const long = {
days: 40,
enough_for_a_trend: true,
events: 900,
first_day: "2026-07-01",
last_day: "2026-08-31",
sources: { daily_uses: 0, journald: 0, live: 900 },
};
serve({
...EMPTY_ROUTES,
"/api/analytics/overview": { body: { ...EMPTY_PANELS.overview, window: long } },
});
render(<Analytics />);
await waitFor(() => {
expect(screen.getByTestId("sources")).toHaveTextContent("900 recorded uses over 40 days");
});
expect(screen.getByTestId("sources")).not.toHaveTextContent("Too few days");
});
it("says nothing about dates before there are any", async () => {
serve(EMPTY_ROUTES);
render(<Analytics />);
await waitFor(() => {
expect(screen.getByTestId("sources")).toBeInTheDocument();
});
// No parenthesised range at all rather than an empty one.
expect(screen.getByTestId("sources")).not.toHaveTextContent("()");
});
it("lists an upstream host and a pool when there is one of each", async () => {
serve(ROUTES);
render(<Analytics />);
await waitFor(() => {
expect(screen.getByTestId("upstream")).toHaveTextContent("commons.example.test");
});
expect(screen.getByTestId("sessions")).toHaveTextContent("10 sittings");
});
it("goes back to every group when the picker is cleared", async () => {
const fetcher = serve(ROUTES);
render(<Analytics />);
await waitFor(() => {
expect(screen.getByTestId("group-picker")).toBeInTheDocument();
});
const picker = screen.getByTestId("group-picker");
await userEvent.selectOptions(picker, ONLY_GROUP?.id ?? "");
await waitFor(() => {
expect(fetcher.mock.calls.some((call) => String(call[0]).includes("group="))).toBe(true);
});
fetcher.mockClear();
await userEvent.selectOptions(picker, "");
await waitFor(() => {
expect(fetcher.mock.calls.length).toBeGreaterThan(0);
});
// The roll-up asks for no group at all rather than for an empty one.
expect(fetcher.mock.calls.every((call) => !String(call[0]).includes("group="))).toBe(true);
});
it("labels a weekday grid that is longer than a week", async () => {
// The server sends exactly seven rows. If it ever sent more, they get a
// number rather than an undefined label -- which is what the index
// fallback is for, and the only way to reach it.
serve({
...ROUTES,
"/api/analytics/time": {
body: {
...TIMES,
weekday_hours: [...TIMES.weekday_hours, Array.from({ length: 24 }, () => 2)],
},
},
});
render(<Analytics />);
await waitFor(() => {
expect(screen.getByTestId("chart-weekday-hours")).toBeInTheDocument();
});
expect(screen.getByText("7")).toBeInTheDocument();
});

View File

@ -0,0 +1,97 @@
import { render, screen, waitFor } from "@testing-library/react";
import { afterEach, expect, it, vi } from "vitest";
import { serve } from "../test/http.ts";
import { ROUTES } from "./analytics-routes.ts";
import { Analytics } from "./analytics.tsx";
afterEach(() => {
vi.unstubAllGlobals();
});
it("says it is reading before the answer arrives", () => {
serve(ROUTES);
render(<Analytics />);
expect(screen.getByText("Reading the database…")).toBeInTheDocument();
});
it("shows the headline numbers once they land", async () => {
serve(ROUTES);
render(<Analytics />);
await waitFor(() => {
expect(screen.getByTestId("tiles")).toHaveTextContent("40");
});
const tiles = screen.getByTestId("tiles");
expect(tiles).toHaveTextContent("19:00");
expect(tiles).toHaveTextContent("35%");
});
it("says where every number came from", async () => {
// Two of the three classes of row were reconstructed rather than observed,
// and a page that does not say so overstates what is known.
serve(ROUTES);
render(<Analytics />);
await waitFor(() => {
expect(screen.getByTestId("sources")).toBeInTheDocument();
});
const sources = screen.getByTestId("sources");
expect(sources).toHaveTextContent("6");
expect(sources).toHaveTextContent("25");
expect(sources).toHaveTextContent("9");
expect(sources).toHaveTextContent("2026-08-29");
});
it("refuses to claim a trend from three days", async () => {
serve(ROUTES);
render(<Analytics />);
await waitFor(() => {
expect(screen.getByTestId("sources")).toHaveTextContent(
"Too few days to state a trend",
);
});
});
it("draws every chart", async () => {
serve(ROUTES);
render(<Analytics />);
await waitFor(() => {
expect(screen.getByTestId("chart-commands")).toBeInTheDocument();
});
for (const id of [
"chart-hours",
"chart-weekday-hours",
"chart-days",
"chart-calendar",
"chart-retention",
]) {
expect(screen.getByTestId(id)).toBeInTheDocument();
}
});
it("names the slowest command and where its time went", async () => {
serve(ROUTES);
render(<Analytics />);
await waitFor(() => {
expect(screen.getByTestId("health-table")).toHaveTextContent("pope");
});
const table = screen.getByTestId("health-table");
expect(table).toHaveTextContent("2845 ms");
expect(table).toHaveTextContent("2156 ms");
});
it("groups near misses by the trigger they nearly hit", async () => {
serve(ROUTES);
render(<Analytics />);
await waitFor(() => {
expect(screen.getByTestId("demand")).toHaveTextContent("!pulapka");
});
expect(screen.getByTestId("demand")).toHaveTextContent("17");
});
it("reports how repetitive each pool is", async () => {
serve(ROUTES);
render(<Analytics />);
await waitFor(() => {
expect(screen.getByTestId("pools")).toHaveTextContent("20%");
});
});

View File

@ -0,0 +1,70 @@
import { useCallback, useEffect, useState } from "react";
import type { AnalyticsGroup, Coverage } from "../api/analytics-types.ts";
import { api } from "../api/client.ts";
import { AnalyticsPanels } from "./analytics-panels.tsx";
// The footnote every panel is read under. Two of the three classes of row
// were reconstructed rather than observed, and a chart that does not say so
// is a chart that quietly overstates what is known.
export function Sources({ coverage: measured }: Readonly<{ coverage: Coverage }>): React.JSX.Element {
return (
<p className="footnote" data-testid="sources">
{measured.events} recorded {measured.events === 1 ? "use" : "uses"} over{" "}
{measured.days} {measured.days === 1 ? "day" : "days"}
{measured.first_day === "" ? "" : ` (${measured.first_day} – ${measured.last_day})`}.{" "}
<strong>{measured.sources.live}</strong> observed live,{" "}
<strong>{measured.sources.journald}</strong> reconstructed from the journal with a real hour,{" "}
<strong>{measured.sources.daily_uses}</strong> recovered as daily counts with no hour at
all — those are counted everywhere and excluded from every chart with an hour on
its axis, because they have none.
{measured.enough_for_a_trend
? ""
: " Too few days to state a trend; the series is drawn, the direction is not claimed."}
</p>
);
}
export function Analytics(): React.JSX.Element {
const [groups, setGroups] = useState<AnalyticsGroup[] | null>(null);
const [group, setGroup] = useState<null | string>(null);
const refresh = useCallback(() => {
void api.analyticsGroups().then((body) => {
setGroups(body.groups);
});
}, []);
useEffect(refresh, [refresh]);
const pick = useCallback((event: React.ChangeEvent<HTMLSelectElement>) => {
setGroup(event.target.value === "" ? null : event.target.value);
}, []);
if (groups === null) {
return <p>Reading the database…</p>;
}
return (
<section>
<h2>Analytics</h2>
<label>
Group{" "}
{/* A group id is 44 characters of base64, so it is picked and never
typed -- the same reason the Settings tab renders one as a select. */}
<select data-testid="group-picker" onChange={pick} value={group ?? ""}>
<option value="">Every group ({groups.length})</option>
{groups.map((one) => {
return (
<option key={one.id} value={one.id}>
{one.id.slice(0, 18)}… — {one.events} uses
</option>
);
})}
</select>
</label>
<AnalyticsPanels group={group} />
</section>
);
}