diff --git a/DOCS-analytics.md b/DOCS-analytics.md new file mode 100644 index 0000000..1f5496f --- /dev/null +++ b/DOCS-analytics.md @@ -0,0 +1,163 @@ +# Command analytics + +`message_events` answers *who said how much*. This answers *what they asked +the bot for, when, and whether it worked* — a different table, a different +package, and one that did not exist before 0.2.6. + +## Why it needed a new table + +The old record was two aggregates, and neither was a history. + +- `command_uses` upserts one row per (group, user, command) holding **last use + only**. It is cooldown state. +- `daily_uses` counts uses per day. It is daily-limit state. + +Both are written by `store.record_use`, which the dispatcher calls **only on +success**. A gate refusal returned before it and a handler crash returned +before it, so a command that failed every single time was indistinguishable +from one nobody ran. There was no hour, no duration, and no idea which +command a counted message had been. + +`command_events` takes one row per **attempt**: outcome, duration, the item +served, and the hour it happened in. `upstream_calls` takes one row per +outbound HTTP request made while serving it. + +## The two writes, and why they may never be merged + +``` +store.commands.record(...) every attempt analytics +store.record_use(...) successes only cooldown + daily limit +``` + +`CommandRefusedError`'s docstring records the bug that made the split +necessary: recording a refused use slid the cooldown window forward every +time somebody was told to wait, so waiting the stated time never helped. + +Now that the history takes refusals, unifying the two calls looks like an +obvious simplification. `test_dispatcher_recording` fires a refused +invocation and asserts both that the history grew **and** that `last_used` +did not move. That test is the whole guard. + +## What is never stored + +| | | +|---|---| +| the words after a command | `args` holds a registry-resolved pack key (`p1`, `add`, `random`) or `""` / `"+args"`. A user-typed column is the shape that put real names in this repository once. | +| an exception's message | `error_class` is a class name. `str(error)` carries whatever was being processed, and here that is somebody's message. | +| a URL | `upstream_calls.host` is a hostname. A URL carries identifiers, and identifiers belong in `item_id` where a decision governs them. | +| a title or a caption | `item_id` is a QID, a Commons page id, a Wolne Lektury slug or a content hash. An identifier cannot carry a name; free text eventually will. | + +## Three kinds of row + +`source` says where a row came from, and the three are not interchangeable. + +| `source` | group & person | hour | duration | outcome | +|---|---|---|---|---| +| `live` | observed | observed | observed | observed | +| `journald` | observed | observed | **none** | **inferred** | +| `daily_uses` | observed | **none** (`-1`) | **none** | assumed `ok` | + +`scope.py` is the only place that names the filters, so "is the exclusion +still there?" is one grep rather than six independent remembers. Hour and +weekday panels take `local_hour >= 0`; latency panels take +`handler_ms IS NOT NULL`; sessions and co-occurrence take a real hour, +because a `daily_uses` timestamp is built from midnight and treating those as +a sequence invents a sitting of forty commands in forty milliseconds. + +Durations are `NULL`, never `0`. A zero reads as data and would drag a p50 +down — the same lesson `stats/backfill.UNKNOWN_HOUR` already records. + +## The backfill + +`tool/backfill_analytics.py` recovers the history that predates the table. +Run it against a **copy** first. It is insert-only and re-runnable; there is +no delete-then-rebuild pass, because journald has no retention configuration +and a second run after the log had rotated would delete rows it could no +longer replace. + +Two sources that overlap, which is the whole difficulty — summing them would +roughly double every historical count and nothing would fail: + +- **journald** knows which command ran and when, but has no group and no + uuid: `dispatcher.py` logs a display name, because the line was written to + be read by a human. +- **daily_uses** knows the group, the uuid and the day, and counts correctly, + but has no times of day at all. + +So the join runs **over `message_events`**, which already holds the group, +the uuid, the real hour and the display name of every counted command, and +lacks only the command's name — which is exactly what journald has. Then +`daily_uses` fills the **residual**: `max(0, uses − matched)` per (group, +user, command, day). Nothing is counted twice. + +### Two things that were wrong on the first attempt + +**The window is symmetric.** The log line is written strictly after the +message arrives, so `[0, +15 s]` looks obviously right. Measured against the +live database: of 299 counted commands, **194 had a journal line earlier than +their own `sent_at_ms`**, by up to 3.2 seconds. The two stamps come from two +different clocks — `sent_at_ms` is set on the *sender's phone*, the journal +stamp is this host's — so their difference is skew and has either sign. The +asymmetric window dropped 47% of the data. + +**Matching is a global assignment, not per-row.** Two commands from one +person seconds apart put both log lines inside both windows; picking +best-for-each-row in turn cannot tell them apart and drops both. Closest-pair +first, globally, can: the skew is systematic per sender, so the true pairs are +the two smallest distances available. On the live database that took the +drops from 137 to **zero**. + +### One inaccuracy, stated rather than hidden + +`dispatcher.py` logs `running command X` **before** the handler runs, so a +journald row proves the command was *entered*, not that it finished. A +handler that then crashed, or a `daily_limit`/`vote_gated` refusal — neither +of which logged anything — is imported as `ok`. Measured on 2026-08-31 the +gap is about 4%. Only `live` rows have outcomes that were actually observed, +and the page says which is which. + +## `!analityka` + +Text only, Polish, twelve lines as a cap rather than a quota — a line with +nothing to say is dropped, not printed empty. No padding, because Signal +renders a proportional font. + +It names at most three people, and only ones whose display name the bot has +seen; everyone else is counted and left unnamed, exactly as the nightly +report does. Full per-person matrices are on the management page. The trend +verdict is suppressed below 14 days: the series is drawn, the direction is +not claimed. + +## The Analytics tab + +`./web.sh`, then the first tab. Seven read-only routes under +`/api/analytics`, each taking an optional `?group=`. Absent means the +all-groups roll-up, and it runs the *same SQL* — every query is a complete +literal string with `WHERE (:group IS NULL OR group_id = :group)`, so the +roll-up and the per-group answer cannot drift. That spelling is also what +keeps ruff's S608 quiet without a suppression. + +Charts are Nivo 0.99.0, always the **fixed-size** components rather than the +`Responsive*` ones: those measure their container, a jsdom container has no +size, and a suite full of charts would otherwise assert against empty divs. +`charts/size.ts` supplies a real measurement in a browser and a stated +fallback everywhere else. Colours are `@kuhyx/web-ui` tokens, not a Nivo +scheme. + +## Verifying a change + +```sh +.venv/bin/python -m pytest -q # 100%, no omits +./scripts/e2e.sh # incl. analytics_command.yaml +pnpm --dir web lint && pnpm --dir web coverage # 100% on all four metrics +pre-commit run --all-files + +# the backfill, against a copy, never the live file +cp signal_bot.sqlite3 /tmp/check.sqlite3 +journalctl --user -u signal-bot --no-pager -o short-iso > /tmp/bot.log +python tool/backfill_analytics.py --db /tmp/check.sqlite3 --log /tmp/bot.log --dry-run +``` + +CI runs `pnpm lint` and `pnpm coverage` but **never `pnpm build`**, so a +bundling or CSP break is invisible to every automated gate. Building and +opening the page is a required manual step, not an optional one. diff --git a/DOCS-changelog.md b/DOCS-changelog.md index 33b66aa..b2cc002 100644 --- a/DOCS-changelog.md +++ b/DOCS-changelog.md @@ -71,6 +71,15 @@ this repo: describe what changes for them. 0.0.1 shipped a bullet about `/healthz`, an endpoint bound to 127.0.0.1 that nobody in a Signal group can reach, and it was rightly called out. +## 0.2.6 — Stephanus I + +papież nr 23 z 267, pontyfikat 254-257 + +- nowa komenda `!analityka` — kto ile używa bota, o której godzinie i które komendy nikomu się nie przydają +- bot pamięta teraz każde użycie komendy, także to nieudane; wcześniej komenda, która zawsze się psuła, wyglądała jak taka, której nikt nie używa +- historia sprzed tej wersji została odtworzona z dziennika systemowego, więc statystyki zaczynają się od 29 sierpnia, a nie od dzisiaj +- strona zarządzania ma nową zakładkę z wykresami + ## 0.2.5 — Lucius I papież nr 22 z 267, pontyfikat 253-254 diff --git a/DOCS-commands.md b/DOCS-commands.md index 7ea4e77..85d7ec9 100644 --- a/DOCS-commands.md +++ b/DOCS-commands.md @@ -28,6 +28,7 @@ This file is the trigger reference — every spelling that resolves, which | stats_crown | `!korona` `!crown` | | stats_weekdays | `!dni` `!days` | | stats_me | `!ja` `!moje` `!mnie` | +| command_stats | `!analityka` `!command_stats` `!commandstats` `!analytics` `!uzycie` `!użycie` `!ranking` | | help | `!pomoc` `!pomocy` `!help` `!komendy` `!commands` | | add_photo | `!dodaj` `!dodaj_zdjecie` `!dodajzdjecie` `!add_photo` `!addphoto` | | random | `!losowe` `!losuj` `!random` `!rand` | diff --git a/DOCS-privacy.md b/DOCS-privacy.md index 4a1a2c7..0845946 100644 --- a/DOCS-privacy.md +++ b/DOCS-privacy.md @@ -27,6 +27,7 @@ and the star went too. | the message database | `signal_bot.sqlite3` | `*.sqlite3` and its `-wal`/`-shm` siblings are ignored | | the account's number, groups, inviters | `.env` | `.env` is ignored; `.env.sample` carries the shape and no values | | everyone's names | the database and the pack manifest | nothing reads them into the repo; the bot logs `pack:p1`, never a name | +| every command anybody has run | `command_events` and `upstream_calls` in the database | the database is ignored, and the columns are chosen so there is nothing to leak: a resolved pack key or `+args` instead of the words typed, an exception's *class* instead of its message, a hostname instead of a URL, and a public identifier instead of a title. See `DOCS-analytics.md`. | | what people typed after a `!` the bot did not know | `unknown_commands` in the database | the database is ignored; the log line names neither the word nor the message, and the export the management page writes goes to the state directory, never into this tree | All of it is `700`/`600` on disk. The repository never needs any of it: the diff --git a/DOCS-statistics.md b/DOCS-statistics.md index 4d2dba4..d37e06b 100644 --- a/DOCS-statistics.md +++ b/DOCS-statistics.md @@ -1,5 +1,10 @@ # Statistics +This is the **message** history: who said how much, when. The **command** +history -- what people asked the bot for, whether it worked and how long it +took -- is a different table and a different package; see +`DOCS-analytics.md`. + ## What is recorded One row per message in `message_events`: sender uuid and the display name they diff --git a/DOCS-web-ui.md b/DOCS-web-ui.md index f42aa73..08621d1 100644 --- a/DOCS-web-ui.md +++ b/DOCS-web-ui.md @@ -116,6 +116,23 @@ item *i* of the album while tagging item *i* of the filtered set. Anything seen this session stays navigable even once it has tags, or tagging the photo you are looking at would remove it from the list you are walking. +## Analytics + +The first tab: what the group actually asks the bot for. Read-only, and the +only thing on this page that is worth nothing to an attacker and everything +to a maintainer -- so there is deliberately no write route under it. + +`?group=` is optional on every route. Absent means the all-groups roll-up, +and it runs the *same SQL* rather than a second query that would drift. + +Every panel carries the window it covers and the three source counts, because +two of the three classes of row were reconstructed rather than observed. +`DOCS-analytics.md` says what each class knows and what it does not. + +Charts are Nivo, always the fixed-size components: the `Responsive*` ones +measure their container, and a jsdom container has none, so the whole test +suite would assert against empty divs. + ## Ideas The fourth tab lists what the group has asked for: `!zaproponuj` notes with @@ -153,6 +170,8 @@ other path, so a reload on a sub-view works. | POST | `/api/album/person` | Add one person mid-session. | | GET | `/api/album/media/{index}` | One item, **by position**. | | GET | `/api/suggestions` | Filed ideas and unrecognised-word tallies, bodies included. | +| GET | `/api/analytics/groups` | Every group with a command history, for the picker. | +| GET | `/api/analytics/{overview,time,health,demand,behaviour,content}` | One per panel, all read-only, all taking an optional `?group=`. | | POST | `/api/suggestions/export` | Write those to a file **outside the repository**. | That last one is not a style choice. A filename supplied by a client is a diff --git a/fake_signal/scenarios/analytics_command.yaml b/fake_signal/scenarios/analytics_command.yaml new file mode 100644 index 0000000..9bb5c8c --- /dev/null +++ b/fake_signal/scenarios/analytics_command.yaml @@ -0,0 +1,17 @@ +# Fresh uuid: one bot process serves every scenario and the cooldown lives in +# SQLite, so a shared actor would leak a refusal into this file. +# +# Two frames on purpose. The first is a command that leaves a row behind; the +# second asks about it. That is what makes this an end-to-end proof of the +# recording rather than of the rendering: a broken recorder answers the +# second frame with "nobody has used anything yet". +name: "!analityka answers with the history the frame before it just made" +inject: + - { text: "!gekon", uuid: uuid-nadia, name: Nadia } + - { text: "!analityka", uuid: uuid-nadia, name: Nadia } +# One expectation per send. "!gecko" rather than the heading, because the +# heading proves only that the command rendered -- naming the command from +# the frame before it proves the dispatcher actually recorded that frame. +expect: + - { kind: image, to: reply } + - { kind: text, to: reply, contains: "!gecko" } diff --git a/pyproject.toml b/pyproject.toml index 62e4489..58ed40f 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "signal-bot" -version = "0.2.5" +version = "0.2.6" description = "A Signal group bot: image commands and daily message tallies." readme = "README.md" license = "MIT" diff --git a/signal_bot/aliases.json b/signal_bot/aliases.json index abb2f93..18892b3 100644 --- a/signal_bot/aliases.json +++ b/signal_bot/aliases.json @@ -129,6 +129,15 @@ "!moje", "!mnie" ], + "command_stats": [ + "!analityka", + "!command_stats", + "!commandstats", + "!analytics", + "!uzycie", + "!użycie", + "!ranking" + ], "help": [ "!pomoc", "!pomocy", diff --git a/signal_bot/analytics/__init__.py b/signal_bot/analytics/__init__.py new file mode 100644 index 0000000..5b39eb6 --- /dev/null +++ b/signal_bot/analytics/__init__.py @@ -0,0 +1,10 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""The command history: what was run, by whom, when, and whether it worked. + +A sibling of ``stats`` rather than part of it, because the two answer +different questions from different tables. ``stats`` is the *message* +history -- who said how much -- and is what the nightly report reads. +This is the *command* history, and it exists because none of it was kept: +the old record was two aggregates written only when a command succeeded, +so a command that failed every time looked exactly like one nobody ran. +""" diff --git a/signal_bot/analytics/adoption.py b/signal_bot/analytics/adoption.py new file mode 100644 index 0000000..f5c9879 --- /dev/null +++ b/signal_bot/analytics/adoption.py @@ -0,0 +1,146 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""Family E, the adoption half: who came back, and what a release did. + +Three questions that are about the bot rather than about any one command. + +* **Retention** -- of the people who used it on their first day, how many + were still using it N days later. On a history this short this is one or + two points, and the panel says so rather than drawing a curve. +* **Novelty** -- what a release actually did. ``DOCS-changelog.md`` is the + only record of when each version shipped and it is the one the maintainer + actually keeps, so the dates come from there rather than from git tags. +* **Share of chat** -- what fraction of everything said in the group was a + command. This is the only query in the package that reads + ``message_events``, and it is the number that says whether the bot is a + fixture or a curiosity. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from typing import TYPE_CHECKING, Final + +from signal_bot.stats.periods import shift + +if TYPE_CHECKING: + import sqlite3 + from collections.abc import Sequence + + from signal_bot.analytics.scope import Scope + +FIRST_DAY_SQL: Final = """ +SELECT user_uuid, MIN(day) AS first_day +FROM command_events +WHERE (:group IS NULL OR group_id = :group) + AND outcome = 'ok' +GROUP BY user_uuid +""" + +ACTIVE_DAYS_SQL: Final = """ +SELECT DISTINCT user_uuid, day +FROM command_events +WHERE (:group IS NULL OR group_id = :group) + AND outcome = 'ok' +""" + +SHARE_SQL: Final = """ +SELECT COUNT(*) AS messages, SUM(is_command) AS commands +FROM message_events +WHERE (:group IS NULL OR group_id = :group) + AND local_hour >= 0 +""" + +NEW_PEOPLE_SQL: Final = """ +SELECT day, COUNT(*) AS people +FROM ( + SELECT user_uuid, MIN(day) AS day + FROM command_events + WHERE (:group IS NULL OR group_id = :group) + AND outcome = 'ok' + GROUP BY user_uuid +) +GROUP BY day +ORDER BY day +""" + + +@dataclass(frozen=True, slots=True) +class Share: + """How much of the group's talking is people talking to the bot.""" + + messages: int = 0 + commands: int = 0 + + @property + def fraction(self) -> float: + """Commands as a share of every counted message.""" + return self.commands / self.messages if self.messages else 0.0 + + +def retention(conn: sqlite3.Connection, scope: Scope, horizon: int) -> list[tuple[int, int, int]]: + """(day offset, still active, cohort size) for offsets 0..horizon. + + Days are stepped as dates, never as 86_400_000 milliseconds: Warsaw days + are not all the same length and the arithmetic version drifts an hour + twice a year, which would move somebody between buckets. + """ + firsts = { + str(row["user_uuid"]): str(row["first_day"]) + for row in conn.execute(FIRST_DAY_SQL, scope.params()) + } + if not firsts: + return [] + active: set[tuple[str, str]] = { + (str(row["user_uuid"]), str(row["day"])) + for row in conn.execute(ACTIVE_DAYS_SQL, scope.params()) + } + cohort = len(firsts) + return [ + ( + offset, + sum(1 for uuid, first in firsts.items() if (uuid, shift(first, offset)) in active), + cohort, + ) + for offset in range(horizon + 1) + ] + + +def newcomers(conn: sqlite3.Connection, scope: Scope) -> list[tuple[str, int]]: + """(day, how many people used the bot for the first time).""" + return [ + (str(row["day"]), int(row["people"])) + for row in conn.execute(NEW_PEOPLE_SQL, scope.params()) + ] + + +def share_of_chat(conn: sqlite3.Connection, scope: Scope) -> Share: + """What fraction of the group's messages were commands. + + Excludes the synthetic rows seeded from the pre-0.1.0 counter, which + carry UNKNOWN_HOUR and no is_command flag -- counting them would put a + whole day's messages in the denominator against no commands at all. + """ + # An aggregate SELECT returns one row even against an empty table, with + # NULLs in it -- so there is no None to guard against, only the NULLs. + row = conn.execute(SHARE_SQL, scope.params()).fetchone() + return Share(messages=int(row["messages"] or 0), commands=int(row["commands"] or 0)) + + +def around_releases( + series: Sequence[tuple[str, int]], releases: Sequence[tuple[str, str]], span: int +) -> list[tuple[str, str, int, int]]: + """(version, day, uses just after, uses just before) for each release. + + "Just after" is the release day and the ``span`` days following it; + "just before" is the same number of days ending the day before. Equal + windows, so the comparison means something even at the edges of a short + history -- a release on the first recorded day gets a before-window of + zero and reads as such rather than as a spike. + """ + counts = dict(series) + out: list[tuple[str, str, int, int]] = [] + for version, day in releases: + after = sum(counts.get(shift(day, offset), 0) for offset in range(span + 1)) + before = sum(counts.get(shift(day, -offset), 0) for offset in range(1, span + 1)) + out.append((version, day, after, before)) + return out diff --git a/signal_bot/analytics/backfill.py b/signal_bot/analytics/backfill.py new file mode 100644 index 0000000..9f23166 --- /dev/null +++ b/signal_bot/analytics/backfill.py @@ -0,0 +1,182 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""Recovering the command history that predates the table holding it. + +Two sources, and they overlap, which is the whole difficulty. Summing them +would roughly double every historical count and nothing would fail. + +* **journald** knows *which* command ran and *when*, to the second, but has + no group and no uuid -- ``dispatcher.py`` logs a display name because the + line was written to be read by a human. +* **daily_uses** knows the group, the uuid, the command and the day, and + counts them correctly, but has no times of day at all. + +So the join runs over ``message_events``, which already holds the group, the +uuid, the real hour and the display name of every counted command -- and +lacks only the command's *name*, which is exactly what journald has. Each +matched row is a whole truthful record. ``daily_uses`` then fills the +**residual**: for each (group, user, command, day), whatever its count says +happened beyond what the join could account for. Nothing is counted twice. + +Insert-only, always. There is no delete-then-rebuild pass, because journald +has no retention configuration and will rotate: a second run after the log +has aged out would delete rows it could no longer replace. Re-runnability +comes from the natural key and ``ON CONFLICT DO NOTHING``. + +One inaccuracy, stated rather than hidden. ``dispatcher.py`` logs "running +command X" *before* the handler runs, so a journald row proves the command +was entered, not that it finished. A handler that then crashed, or a +``daily_limit`` or ``vote_gated`` refusal -- neither of which logged +anything -- is imported as ``ok`` because nothing in the record says +otherwise. Measured against the live database on 2026-08-31 the gap is about +4%: 293 reconstructed invocations against 388 successes in ``daily_uses``, +overshooting by roughly fifteen rows across all commands. Every such row +carries ``source = 'journald'``, so the UI can and does say where it came +from; only the ``live`` rows have outcomes that were actually observed. +""" + +from __future__ import annotations + +import datetime as dt +from dataclasses import dataclass +from typing import TYPE_CHECKING, Final + +from signal_bot.analytics.backfill_join import Tally, from_journal +from signal_bot.analytics.uses import ( + OUTCOME_OK, + SOURCE_DAILY, + SOURCE_LIVE, + CommandEvent, +) +from signal_bot.stats.backfill import UNKNOWN_HOUR +from signal_bot.stats.periods import MS_PER_SECOND, WARSAW, weekday_of + +if TYPE_CHECKING: + from collections.abc import Iterable + + from signal_bot.analytics.journal import JournalLine + from signal_bot.store import Store + +# Recorded per group once a pass has run, in the same place and for the same +# reason stats/backfill.py records its own marker. +BACKFILL_KEY: Final = "command_backfill" + + +@dataclass(frozen=True, slots=True) +class Report: + """What one pass did, and what it refused to guess at.""" + + journal_rows: int = 0 + daily_rows: int = 0 + unmatched: int = 0 + ambiguous: int = 0 + groups: int = 0 + + def lines(self) -> tuple[str, ...]: + """The summary a human reads. Never names anybody.""" + return ( + f"journald rows: {self.journal_rows}", + f"daily_uses rows: {self.daily_rows}", + f"dropped, no match: {self.unmatched}", + f"dropped, ambiguous: {self.ambiguous}", + f"groups marked: {self.groups}", + ) + + +def midnight_ms(day: str) -> int: + """The first millisecond of a Warsaw calendar day. + + Built from the date rather than by subtracting 86_400_000 from anything: + Warsaw days are not all 24 hours, and the arithmetic spelling lands an + hour out twice a year. + """ + start = dt.datetime.combine(dt.date.fromisoformat(day), dt.time.min, tzinfo=WARSAW) + return int(start.timestamp() * MS_PER_SECOND) + + +def live_floor(store: Store) -> int: + """The earliest live row, or "no live rows yet". + + Every backfilled row must be strictly older than this. Without the + cutoff, a second pass run after live recording had started would import + journald lines describing invocations that already exist as ``live`` + rows -- with a different source, so the natural key would not stop them. + """ + row = store.commands.connection.execute( + "SELECT MIN(invoked_at_ms) AS floor FROM command_events WHERE source = ?", + (SOURCE_LIVE,), + ).fetchone() + floor = row["floor"] + # No live rows yet means nothing is off limits. + return int(floor) if floor is not None else 1 << 62 + + +def _residual(store: Store, tally: Tally) -> list[CommandEvent]: + """Fill in whatever daily_uses counted that the join could not explain. + + ``uses`` is the truth about how many times something ran that day. The + join accounts for some of them exactly; this adds the remainder at day + granularity, with no hour and no durations, so the total is right even + where the detail is missing. + """ + events: list[CommandEvent] = [] + rows = store.commands.connection.execute( + "SELECT group_id, user_uuid, command, day, uses FROM daily_uses ORDER BY day, command" + ).fetchall() + for row in rows: + key = (str(row["group_id"]), str(row["user_uuid"]), str(row["command"]), str(row["day"])) + remaining = int(row["uses"]) - tally.matched[key] + day = str(row["day"]) + base = midnight_ms(day) + events.extend( + CommandEvent( + group_id=key[0], + user_uuid=key[1], + command=key[2], + outcome=OUTCOME_OK, + # Deterministic, so a second pass produces the identical key + # and inserts nothing. The same trick, for the same reason, + # as seed_from_counter's sent_at_ms=index + 1. + invoked_at_ms=base + index, + day=day, + local_hour=UNKNOWN_HOUR, + weekday=weekday_of(day), + source=SOURCE_DAILY, + ) + for index in range(max(0, remaining)) + ) + return events + + +def plan(store: Store, lines: Iterable[JournalLine]) -> tuple[Report, list[CommandEvent]]: + """Work out what would be imported, writing nothing. + + Separate from ``apply`` so that ``--dry-run`` reports the real numbers + rather than an estimate of them. Rolling back would not have worked: + ``CommandEventStore.record`` commits per row, exactly as ``EventStore`` + does, so by the time a caller could roll back there is nothing left to + roll back. + """ + floor = live_floor(store) + tally = from_journal(store, lines, floor) + residual = _residual(store, tally) + events = [*tally.events, *residual] + return ( + Report( + journal_rows=len(tally.events), + daily_rows=len(residual), + unmatched=tally.unmatched, + ambiguous=tally.ambiguous, + groups=len({event.group_id for event in events}), + ), + events, + ) + + +def apply(store: Store, events: Iterable[CommandEvent]) -> None: + """Write a planned import. Insert-only; a repeat run inserts nothing.""" + groups = set() + for event in events: + store.commands.record(event) + groups.add(event.group_id) + for group_id in sorted(groups): + store.set_meta(group_id, BACKFILL_KEY, "done") diff --git a/signal_bot/analytics/backfill_join.py b/signal_bot/analytics/backfill_join.py new file mode 100644 index 0000000..4c79496 --- /dev/null +++ b/signal_bot/analytics/backfill_join.py @@ -0,0 +1,180 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""Attaching a command name from the journal to a message that was one. + +Split out of ``backfill.py`` so both stay under the 250-line cap. This half +is the part that can be wrong, so it is the part that refuses to guess. + +The direction matters and is easy to get backwards. Iterating the journal +and looking for a message means resolving a display name to a uuid and a +group, which the journal does not know. Iterating ``message_events`` instead +means every row already carries the group, the uuid, the day and the real +hour, and the journal supplies the one field missing -- the command's name. +""" + +from __future__ import annotations + +from collections import Counter, defaultdict +from dataclasses import dataclass, field +from typing import TYPE_CHECKING, Final + +from signal_bot.analytics.uses import ( + OUTCOME_COOLDOWN, + OUTCOME_OK, + SOURCE_JOURNAL, + CommandEvent, +) +from signal_bot.stats.periods import weekday_of + +if TYPE_CHECKING: + import sqlite3 + from collections.abc import Iterable, Sequence + + from signal_bot.analytics.journal import JournalLine + from signal_bot.store import Store + +# How far apart a message and the log line about it may be and still be the +# same event. +# +# SYMMETRIC, and that is not obvious. The log line is written strictly after +# the message arrives, so an asymmetric [0, +W] window looks correct -- and +# throws away most of the data. Measured against the live database on +# 2026-08-31: of 299 counted commands, 194 have a journal line *earlier* +# than their own sent_at_ms, by up to 3.2 seconds, median -173 ms. +# +# The reason is that the two stamps come from two different clocks. +# ``sent_at_ms`` is Signal's envelope timestamp, set on the *sender's phone*; +# the journal stamp is this host's. Nothing keeps those in step, so their +# difference is skew and can have either sign. +# +# 15 seconds each way is far wider than the 3.2 s of skew actually seen and +# still narrower than a person's own repeat rate. Widening it cannot corrupt +# anything either way: two different commands inside one window are dropped +# as ambiguous rather than guessed at. +WINDOW_MS: Final = 15_000 + + +@dataclass(slots=True) +class _Candidate: + """One journal line, and whether the join has already spent it.""" + + line: JournalLine + used: bool = False + + +@dataclass(slots=True) +class Tally: + """The running result of a pass.""" + + events: list[CommandEvent] = field(default_factory=list) + matched: Counter[tuple[str, str, str, str]] = field(default_factory=Counter) + unmatched: int = 0 + ambiguous: int = 0 + + +def _by_name(lines: Iterable[JournalLine]) -> dict[str, list[_Candidate]]: + """Group journal lines by the display name they were logged for.""" + grouped: dict[str, list[_Candidate]] = defaultdict(list) + for line in lines: + grouped[line.display_name].append(_Candidate(line)) + for candidates in grouped.values(): + candidates.sort(key=lambda candidate: candidate.line.at_ms) + return dict(grouped) + + +def _assign( + rows: Sequence[sqlite3.Row], grouped: dict[str, list[_Candidate]] +) -> dict[int, _Candidate]: + """Pair each counted command with the journal line most likely to be it. + + Closest pair first, globally, rather than best-for-each-row in turn. + That difference is what makes two commands from one person seconds apart + resolvable at all. + + Per-row greedy sees both lines inside the window, cannot tell which is + which, and drops both. Global assignment can: the clock skew between a + sender's phone and this host is *systematic*, so the true pairs share + almost the same offset and are the two smallest distances available. + + Ties break on row order and then line order, so a re-run against the + same inputs produces the same pairing. The backfill has to be + re-runnable, and "mostly the same" is not that. + """ + pairs: list[tuple[int, int, int]] = [] + for index, row in enumerate(rows): + candidates = grouped.get(str(row["display_name"]), []) + for position, candidate in enumerate(candidates): + delta = candidate.line.at_ms - int(row["sent_at_ms"]) + if abs(delta) <= WINDOW_MS: + pairs.append((abs(delta), index, position)) + pairs.sort() + + taken: dict[int, _Candidate] = {} + for _distance, index, position in pairs: + if index in taken: + continue + candidate = grouped[str(rows[index]["display_name"])][position] + if candidate.used: + continue + candidate.used = True + taken[index] = candidate + return taken + + +def _in_window(row: sqlite3.Row, grouped: dict[str, list[_Candidate]]) -> bool: + """Whether the journal held any line at all for this message.""" + sent_at_ms = int(row["sent_at_ms"]) + return any( + abs(candidate.line.at_ms - sent_at_ms) <= WINDOW_MS + for candidate in grouped.get(str(row["display_name"]), []) + ) + + +def from_journal(store: Store, lines: Iterable[JournalLine], floor: int) -> Tally: + """Walk the counted commands and attach a name to each from the journal.""" + tally = Tally() + grouped = _by_name(lines) + rows = store.events.connection.execute( + """ + SELECT group_id, uuid, display_name, sent_at_ms, day, local_hour + FROM message_events + WHERE is_command = 1 AND local_hour >= 0 AND sent_at_ms < ? + ORDER BY sent_at_ms + """, + (floor,), + ).fetchall() + taken = _assign(rows, grouped) + for index, row in enumerate(rows): + chosen = taken.get(index) + if chosen is None: + # A message the journal cannot explain. Either it never held a + # line for it -- vacuumed, or older than the log -- or every line + # in its window went to a closer message, which is the honest + # reading of "two of these happened and I have one record". + # Dropped either way: a wrong attribution is worse than a gap, + # because it is indistinguishable from data. + if _in_window(row, grouped): + tally.ambiguous += 1 + else: + tally.unmatched += 1 + continue + line = chosen.line + outcome = OUTCOME_COOLDOWN if line.refused else OUTCOME_OK + day = str(row["day"]) + tally.events.append( + CommandEvent( + group_id=str(row["group_id"]), + user_uuid=str(row["uuid"]), + command=line.command, + outcome=outcome, + invoked_at_ms=int(row["sent_at_ms"]), + day=day, + local_hour=int(row["local_hour"]), + weekday=weekday_of(day), + source=SOURCE_JOURNAL, + ) + ) + if outcome == OUTCOME_OK: + # Only successes are in daily_uses, so only successes may be + # subtracted from it below. + tally.matched[(str(row["group_id"]), str(row["uuid"]), line.command, day)] += 1 + return tally diff --git a/signal_bot/analytics/content.py b/signal_bot/analytics/content.py new file mode 100644 index 0000000..93c956e --- /dev/null +++ b/signal_bot/analytics/content.py @@ -0,0 +1,106 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""Family F: which items get served, and how often the same one comes back. + +Reads ``item_id``, which holds public identifiers only -- a Wikidata QID, a +Commons page id, a Wolne Lektury slug, a content-addressed filename. Never a +title, a caption or a name. That constraint is what makes this family +possible at all: "the pope pool feels repetitive" is an impression, and +"106 draws produced 61 distinct popes" is a number. + +The repeat rate is the headline. For a pool of size N drawn uniformly, the +expected number of distinct items after k draws is N(1 - (1 - 1/N)^k), so a +distinct count well below that means the draw is not uniform -- which is +exactly the bug that put one identical photograph into a live group chat +when iNaturalist silently ignored ``order_by=random``. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from typing import TYPE_CHECKING, Final + +if TYPE_CHECKING: + import sqlite3 + + from signal_bot.analytics.scope import Scope + +ITEMS_SQL: Final = """ +SELECT command, + COUNT(*) AS draws, + COUNT(DISTINCT item_id) AS distinct_items +FROM command_events +WHERE (:group IS NULL OR group_id = :group) + AND item_id IS NOT NULL + AND outcome = 'ok' +GROUP BY command +ORDER BY draws DESC, command +""" + +TOP_ITEMS_SQL: Final = """ +SELECT command, item_id, COUNT(*) AS draws +FROM command_events +WHERE (:group IS NULL OR group_id = :group) + AND item_id IS NOT NULL + AND outcome = 'ok' +GROUP BY command, item_id +HAVING draws > 1 +ORDER BY draws DESC, command, item_id +""" + + +@dataclass(frozen=True, slots=True) +class Pool: + """One command's draws, and how varied they turned out to be.""" + + command: str + draws: int + distinct: int + + @property + def repeat_rate(self) -> float: + """Share of draws that produced something already seen. + + Zero means every draw was new. Approaching one means the pool is + exhausted -- or was never as large as it looked. + """ + if self.draws <= 0: + return 0.0 + return 1.0 - (self.distinct / self.draws) + + +def pools(conn: sqlite3.Connection, scope: Scope) -> list[Pool]: + """Every command that names what it served, most drawn first.""" + return [ + Pool( + command=str(row["command"]), + draws=int(row["draws"]), + distinct=int(row["distinct_items"]), + ) + for row in conn.execute(ITEMS_SQL, scope.params()) + ] + + +def repeats(conn: sqlite3.Connection, scope: Scope) -> list[tuple[str, str, int]]: + """(command, item, times) for every item served more than once. + + The identifiers are safe to show -- that is the whole reason the column + may exist -- but they are only meaningful next to the command, so they + are never returned on their own. + """ + return [ + (str(row["command"]), str(row["item_id"]), int(row["draws"])) + for row in conn.execute(TOP_ITEMS_SQL, scope.params()) + ] + + +def expected_distinct(pool_size: int, draws: int) -> float: + """How many distinct items a *uniform* draw would have produced. + + The comparison that turns a repeat rate into a verdict. A pool of 266 + traps drawn 19 times should yield about 18.4 distinct ones; yielding 9 + would mean the randomness is not what it claims, which has happened in + this repository before and reached a live group chat. + """ + if pool_size <= 0 or draws <= 0: + return 0.0 + return pool_size * (1.0 - ((pool_size - 1) / pool_size) ** draws) diff --git a/signal_bot/analytics/demand.py b/signal_bot/analytics/demand.py new file mode 100644 index 0000000..7fed9f4 --- /dev/null +++ b/signal_bot/analytics/demand.py @@ -0,0 +1,102 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""Family D: what the group asked for that does not exist. + +The management page's Ideas tab already lists ``unknown_commands`` with the +words and the messages they came in. This adds the part it does not have: +the same words **grouped by the trigger they nearly hit**, so that "17 +near-misses on !pułapka" reads as *this command wants another alias* rather +than as seventeen unrelated typos. + +The grouping is done here rather than in SQL because the distance function +is ``nearest``'s, and it is optimal string alignment rather than a ratio -- +a transposition costs one edit there and shreds the matching blocks a ratio +counts. Reimplementing that in SQL is not possible and approximating it +would put a different answer on the page from the one the bot gives in the +group. +""" + +from __future__ import annotations + +from collections import Counter +from dataclasses import dataclass +from typing import TYPE_CHECKING, Final + +from signal_bot.nearest import nearest + +if TYPE_CHECKING: + import sqlite3 + from collections.abc import Sequence + + from signal_bot.analytics.scope import Scope + +WORDS_SQL: Final = """ +SELECT word, COUNT(*) AS hits, MAX(sent_at_ms) AS last_at +FROM unknown_commands +WHERE (:group IS NULL OR group_id = :group) +GROUP BY word +ORDER BY hits DESC, word +""" + +FILED_SQL: Final = """ +SELECT COUNT(*) AS filed +FROM suggestions +WHERE (:group IS NULL OR group_id = :group) +""" + + +@dataclass(frozen=True, slots=True) +class NearMiss: + """A real trigger, and how often somebody nearly typed it.""" + + target: str + hits: int + variants: int + + +@dataclass(frozen=True, slots=True) +class Demand: + """What the group tried to type, and what it filed on purpose.""" + + near_misses: tuple[NearMiss, ...] = () + unclaimed: int = 0 + unclaimed_words: int = 0 + filed: int = 0 + + +def collect(conn: sqlite3.Connection, scope: Scope, triggers: Sequence[str]) -> Demand: + """Group every unrecognised word by the command it was probably meant to be. + + A word that resolves to nothing at all is counted but not attributed: + those are the genuinely new ideas, and pretending one of them is a typo + of an existing command is how a real request gets filed as noise. + + The words themselves never leave this function. A half-typed name is + still a name -- ``check_personal_data`` learns the names it refuses from + the live database, so it catches ``anna`` and could never catch + ``!annna`` -- and the Ideas tab is the one page that already carries + them, deliberately, and never writes them into the repository. + """ + hits: Counter[str] = Counter() + variants: Counter[str] = Counter() + unclaimed = 0 + unclaimed_words = 0 + for row in conn.execute(WORDS_SQL, scope.params()): + word = str(row["word"]) + count = int(row["hits"]) + target = nearest(word, triggers) + if target is None: + unclaimed += count + unclaimed_words += 1 + continue + hits[target] += count + variants[target] += 1 + filed_row = conn.execute(FILED_SQL, scope.params()).fetchone() + return Demand( + near_misses=tuple( + NearMiss(target=target, hits=count, variants=variants[target]) + for target, count in sorted(hits.items(), key=lambda pair: (-pair[1], pair[0])) + ), + unclaimed=unclaimed, + unclaimed_words=unclaimed_words, + filed=int(filed_row["filed"] or 0) if filed_row is not None else 0, + ) diff --git a/signal_bot/analytics/health.py b/signal_bot/analytics/health.py new file mode 100644 index 0000000..cc38980 --- /dev/null +++ b/signal_bot/analytics/health.py @@ -0,0 +1,194 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""Family C: which commands are broken, slow, or refusing people. + +This is the family that had *no* data at all before ``command_events``: +``store.record_use`` fires only on success, so a command that crashed every +time was indistinguishable from one nobody ran. + +Latency narrows to ``live``. A reconstructed row has no duration -- NULL, +never 0, because a zero would drag a p50 down and read as a fast command. +Every aggregate here therefore says ``handler_ms IS NOT NULL`` rather than +relying on the source column, so a future third source with real timings +would be included automatically and one without would not. +""" + +from __future__ import annotations + +import math +from dataclasses import dataclass +from typing import TYPE_CHECKING, Final + +if TYPE_CHECKING: + import sqlite3 + + from signal_bot.analytics.scope import Scope + +# Which percentiles the panel shows. The median says what it is usually +# like; the 95th says what it is like when somebody complains. +P50: Final = 0.50 +P95: Final = 0.95 + + +@dataclass(frozen=True, slots=True) +class Latency: + """How long one command takes, split the way DOCS-latency.md splits it.""" + + command: str + samples: int + handler_p50: int + handler_p95: int + send_p50: int + upstream_p50: int + + +@dataclass(frozen=True, slots=True) +class Upstream: + """One external service, and how well it has been behaving.""" + + host: str + calls: int + failures: int + p50_ms: int + + @property + def failure_rate(self) -> float: + """Share of calls that were not a 2xx.""" + return self.failures / self.calls if self.calls else 0.0 + + +GATES_SQL: Final = """ +SELECT command, outcome, COUNT(*) AS n +FROM command_events +WHERE (:group IS NULL OR group_id = :group) + AND outcome LIKE 'refused_%' +GROUP BY command, outcome +ORDER BY n DESC, command +""" + +ERRORS_SQL: Final = """ +SELECT command, error_class, COUNT(*) AS n +FROM command_events +WHERE (:group IS NULL OR group_id = :group) + AND outcome = 'error' +GROUP BY command, error_class +ORDER BY n DESC, command +""" + +TIMED_SQL: Final = """ +SELECT command, handler_ms, send_ms +FROM command_events +WHERE (:group IS NULL OR group_id = :group) + AND handler_ms IS NOT NULL +ORDER BY command, handler_ms +""" + +UPSTREAM_SQL: Final = """ +SELECT host, + COUNT(*) AS calls, + SUM(status < 200 OR status > 299) AS failures +FROM upstream_calls +WHERE (:group IS NULL OR group_id = :group) +GROUP BY host +ORDER BY calls DESC, host +""" + +UPSTREAM_TIMES_SQL: Final = """ +SELECT host, duration_ms +FROM upstream_calls +WHERE (:group IS NULL OR group_id = :group) +ORDER BY host, duration_ms +""" + +COMMAND_UPSTREAM_SQL: Final = """ +SELECT command, duration_ms +FROM upstream_calls +WHERE (:group IS NULL OR group_id = :group) +ORDER BY command, duration_ms +""" + + +def percentile(sorted_values: list[int], fraction: float) -> int: + """The value at a fraction of the way through an already-sorted list. + + Nearest-rank: ``ceil(p * n) - 1``, so the answer is always one of the + measurements rather than an interpolation between two of them. A p95 + that nobody actually experienced is a worse number to act on than a + real one that is slightly off the theoretical quantile. + + Spelled with ``math.ceil`` rather than ``round`` deliberately. Python + rounds halves to even, so ``round(0.5)`` is 0 and ``round(1.5)`` is 2 -- + which would make the median of a two-element sample depend on the + sample's length in a way no reader would predict. + + Computed here rather than in SQL: SQLite has no percentile function + without an extension, and adding one would mean a seventh dependency for + three lines of arithmetic. + """ + if not sorted_values: + return 0 + rank = math.ceil(fraction * len(sorted_values)) - 1 + return sorted_values[max(0, min(len(sorted_values) - 1, rank))] + + +def latencies(conn: sqlite3.Connection, scope: Scope) -> list[Latency]: + """Per-command timings, from the rows that actually carry them.""" + handler: dict[str, list[int]] = {} + send: dict[str, list[int]] = {} + for row in conn.execute(TIMED_SQL, scope.params()): + command = str(row["command"]) + handler.setdefault(command, []).append(int(row["handler_ms"])) + if row["send_ms"] is not None: + send.setdefault(command, []).append(int(row["send_ms"])) + upstream: dict[str, list[int]] = {} + for row in conn.execute(COMMAND_UPSTREAM_SQL, scope.params()): + upstream.setdefault(str(row["command"]), []).append(int(row["duration_ms"])) + for values in (*handler.values(), *send.values(), *upstream.values()): + values.sort() + return sorted( + ( + Latency( + command=command, + samples=len(values), + handler_p50=percentile(values, P50), + handler_p95=percentile(values, P95), + send_p50=percentile(send.get(command, []), P50), + upstream_p50=percentile(upstream.get(command, []), P50), + ) + for command, values in handler.items() + ), + key=lambda entry: (-entry.handler_p50, entry.command), + ) + + +def gates(conn: sqlite3.Connection, scope: Scope) -> list[tuple[str, str, int]]: + """(command, which gate, how often). A high count is mis-tuning.""" + return [ + (str(row["command"]), str(row["outcome"]), int(row["n"])) + for row in conn.execute(GATES_SQL, scope.params()) + ] + + +def errors(conn: sqlite3.Connection, scope: Scope) -> list[tuple[str, str, int]]: + """(command, exception class, how often). Never an exception message.""" + return [ + (str(row["command"]), str(row["error_class"]), int(row["n"])) + for row in conn.execute(ERRORS_SQL, scope.params()) + ] + + +def upstreams(conn: sqlite3.Connection, scope: Scope) -> list[Upstream]: + """Every external service the bot depends on, and how it is holding up.""" + times: dict[str, list[int]] = {} + for row in conn.execute(UPSTREAM_TIMES_SQL, scope.params()): + times.setdefault(str(row["host"]), []).append(int(row["duration_ms"])) + for values in times.values(): + values.sort() + return [ + Upstream( + host=str(row["host"]), + calls=int(row["calls"]), + failures=int(row["failures"] or 0), + p50_ms=percentile(times.get(str(row["host"]), []), P50), + ) + for row in conn.execute(UPSTREAM_SQL, scope.params()) + ] diff --git a/signal_bot/analytics/hours.py b/signal_bot/analytics/hours.py new file mode 100644 index 0000000..f0f79a5 --- /dev/null +++ b/signal_bot/analytics/hours.py @@ -0,0 +1,149 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""Family B, the time-of-day half: when the bot is actually used. + +Every query here filters ``local_hour >= 0``. That is the whole point of the +filter: ``daily_uses`` rows carry ``UNKNOWN_HOUR`` because their source knew +only a date, and letting them through would file a third of the history +under hour -1 -- or, worse, under midnight if anybody later "tidied" the +sentinel to 0. + +``journald`` rows *are* included. They were matched to a real message, so +their hour was observed even though their duration was not. Excluding them +would throw away most of the history for no gain; that distinction is why +``scope`` names two predicates rather than one. + +A Warsaw day is not always 24 hours -- 2026-03-29 has 23 and 2026-10-25 has +25 -- so callers must not assume the buckets sum to a day, and nothing here +divides by 24. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from typing import TYPE_CHECKING, Final + +if TYPE_CHECKING: + import sqlite3 + + from signal_bot.analytics.scope import Scope + +HOURS_IN_DAY: Final = 24 +DAYS_IN_WEEK: Final = 7 + + +@dataclass(frozen=True, slots=True) +class Peak: + """The busiest and quietest hours, when there is enough to say.""" + + busiest: int = -1 + busiest_uses: int = 0 + quietest: int = -1 + quietest_uses: int = 0 + + +BY_HOUR_SQL: Final = """ +SELECT local_hour AS hour, COUNT(*) AS uses +FROM command_events +WHERE (:group IS NULL OR group_id = :group) + AND local_hour >= 0 + AND outcome != 'unknown_word' +GROUP BY local_hour +ORDER BY local_hour +""" + +BY_WEEKDAY_HOUR_SQL: Final = """ +SELECT weekday, local_hour AS hour, COUNT(*) AS uses +FROM command_events +WHERE (:group IS NULL OR group_id = :group) + AND local_hour >= 0 + AND weekday >= 0 + AND outcome != 'unknown_word' +GROUP BY weekday, local_hour +ORDER BY weekday, local_hour +""" + +BY_COMMAND_HOUR_SQL: Final = """ +SELECT command, local_hour AS hour, COUNT(*) AS uses +FROM command_events +WHERE (:group IS NULL OR group_id = :group) + AND local_hour >= 0 + AND outcome != 'unknown_word' +GROUP BY command, local_hour +ORDER BY command, local_hour +""" + +BY_WEEKDAY_SQL: Final = """ +SELECT weekday, COUNT(*) AS uses, COUNT(DISTINCT day) AS days +FROM command_events +WHERE (:group IS NULL OR group_id = :group) + AND weekday >= 0 + AND outcome != 'unknown_word' +GROUP BY weekday +ORDER BY weekday +""" + + +def by_hour(conn: sqlite3.Connection, scope: Scope) -> list[int]: + """Uses per hour of the day, as 24 buckets with the empty ones present. + + A dense list rather than the rows sqlite returns: a bar chart with a + missing 04:00 draws the wrong shape, and "nobody uses it at four in the + morning" is one of the things the panel is for. + """ + # Indexed without a bounds check: local_hour is written by hour_of(), + # which cannot return anything outside 0-23, and the query already + # excludes the UNKNOWN_HOUR sentinel. A guard here would be dead code + # that hides a corrupt row instead of failing on one. + buckets = [0] * HOURS_IN_DAY + for row in conn.execute(BY_HOUR_SQL, scope.params()): + buckets[int(row["hour"])] = int(row["uses"]) + return buckets + + +def by_weekday_hour(conn: sqlite3.Connection, scope: Scope) -> list[list[int]]: + """A 7x24 grid, Monday first, matching date.weekday().""" + grid = [[0] * HOURS_IN_DAY for _ in range(DAYS_IN_WEEK)] + for row in conn.execute(BY_WEEKDAY_HOUR_SQL, scope.params()): + grid[int(row["weekday"])][int(row["hour"])] = int(row["uses"]) + return grid + + +def by_command_hour(conn: sqlite3.Connection, scope: Scope) -> dict[str, list[int]]: + """One 24-bucket row per command: which commands belong to which hours.""" + grid: dict[str, list[int]] = {} + for row in conn.execute(BY_COMMAND_HOUR_SQL, scope.params()): + buckets = grid.setdefault(str(row["command"]), [0] * HOURS_IN_DAY) + buckets[int(row["hour"])] = int(row["uses"]) + return grid + + +def by_weekday(conn: sqlite3.Connection, scope: Scope) -> list[tuple[int, int, int]]: + """(weekday, uses, days seen) so an average can be taken honestly. + + Dividing by the number of weeks would be wrong on a history this short: + a Tuesday that has happened once is not a weekly average. + """ + return [ + (int(row["weekday"]), int(row["uses"]), int(row["days"])) + for row in conn.execute(BY_WEEKDAY_SQL, scope.params()) + ] + + +def peak(buckets: list[int]) -> Peak: + """The busiest hour, and the quietest hour that anybody ever used. + + "Quietest" skips hours with nothing at all: at three days old most of + the night is empty, and "the deadest hour is 04:00 with zero" would be + true of about nine different hours and interesting about none of them. + """ + if not any(buckets): + return Peak() + used = [(hour, uses) for hour, uses in enumerate(buckets) if uses] + busiest = max(used, key=lambda pair: (pair[1], -pair[0])) + quietest = min(used, key=lambda pair: (pair[1], pair[0])) + return Peak( + busiest=busiest[0], + busiest_uses=busiest[1], + quietest=quietest[0], + quietest_uses=quietest[1], + ) diff --git a/signal_bot/analytics/journal.py b/signal_bot/analytics/journal.py new file mode 100644 index 0000000..b75e05a --- /dev/null +++ b/signal_bot/analytics/journal.py @@ -0,0 +1,90 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""Reading the bot's own journal back into the command history. + +Pure parsing, no I/O: the lines arrive as an iterable so that the joining +logic and its tests never shell out, and so that a captured log file works +exactly as well as a live ``journalctl``. + +The journal is the only record of what ran before ``command_events`` +existed, and it is a poor one. It carries no group id and no uuid -- only a +*display name*, because ``dispatcher.py``'s log line was written to be read +by a human. That is why the join in ``backfill`` runs the other way round, +over ``message_events``, which knows everything except which command it was. + +A display name read here is a lookup key and nothing else. It never reaches +a row, a log line or a file: ``group_people`` turns it into a uuid inside +the joiner and the string is dropped. ``DOCS-privacy.md`` names the journal +as the one place display names live, and ``--vacuum-time=`` as how that is +emptied -- so this parser must never be the thing that copies them out. +""" + +from __future__ import annotations + +import datetime as dt +import re +from dataclasses import dataclass +from typing import TYPE_CHECKING, Final + +if TYPE_CHECKING: + from collections.abc import Iterable, Iterator + +MS_PER_SECOND: Final = 1000 + +# `journalctl -o short-iso` prefixes each line with an offset-aware timestamp +# -- 2026-08-31T21:28:26+02:00 -- which is the only spelling in the file that +# survives a DST boundary. The bot's own asctime, further along the same +# line, is naive and would be an hour wrong for half the year. +_RUNNING: Final = re.compile(r"^(?P\S+)\s.*?running command (?P\S+) for (?P.+)$") +# The one gate that already logged its refusals. daily_limit and vote_gated +# logged nothing at all, so those simply cannot be recovered. +_REFUSED: Final = re.compile( + r"^(?P\S+)\s.*?cooldown refused (?P\S+) for (?P.+?): told them" +) + + +@dataclass(frozen=True, slots=True) +class JournalLine: + """One thing the bot said it did, with the name it said it for.""" + + at_ms: int + command: str + display_name: str + refused: bool + + +def _at_ms(stamp: str) -> int | None: + """Parse the journal's own timestamp, or None if this is not one.""" + try: + moment = dt.datetime.fromisoformat(stamp) + except ValueError: + return None + # An offset-naive stamp would silently be read as UTC and land the whole + # import two hours out, which is exactly the class of error the hour + # columns exist to avoid. Refuse it instead. + if moment.tzinfo is None: + return None + return int(moment.timestamp() * MS_PER_SECOND) + + +def parse(lines: Iterable[str]) -> Iterator[JournalLine]: + """Yield one record per line the bot wrote about running a command. + + Everything else -- startup banners, tracebacks, envelope shapes, other + units -- simply does not match and is skipped. The parser is deliberately + permissive about what sits between the timestamp and the phrase, because + the journal identifier is ``run.sh[]`` and the pid changes. + """ + for line in lines: + for pattern, refused in ((_RUNNING, False), (_REFUSED, True)): + found = pattern.match(line) + if found is None: + continue + at_ms = _at_ms(found["at"]) + if at_ms is not None: + yield JournalLine( + at_ms=at_ms, + command=found["command"], + display_name=found["who"].strip(), + refused=refused, + ) + break diff --git a/signal_bot/analytics/probe.py b/signal_bot/analytics/probe.py new file mode 100644 index 0000000..7cef26a --- /dev/null +++ b/signal_bot/analytics/probe.py @@ -0,0 +1,130 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""What one running command did, collected without changing any signature. + +Timing a command means knowing things the handler learns and the dispatcher +records: how long the upstream fetch took, how long the send took, which +item was served. Threading that back would mean a return value or an extra +argument on every handler, every fetcher method and every reply helper. + +A ``ContextVar`` avoids all of it. The dispatcher opens a probe around the +handler call; anything running underneath finds it and notes what it did. +``contextvars`` is stdlib, so the runtime stays at exactly six dependencies. + +``Bot.listen`` awaits ``dispatcher.handle`` directly rather than spawning a +task per frame, so set/reset in a ``try/finally`` cannot interleave. A +handler that does spawn a task gets a *copy* of the context, which is the +behaviour wanted: the copy's notes land in the probe, and the reset in the +parent cannot be undone by the child. + +Every ``note_*`` is a no-op when nothing is probing. That branch lives here, +once, so no caller needs to ask whether it is being measured -- which is +what keeps ``http_client`` and ``context`` free of analytics conditionals. +""" + +from __future__ import annotations + +import contextlib +import time +from contextvars import ContextVar +from dataclasses import dataclass, field +from typing import TYPE_CHECKING, Final + +if TYPE_CHECKING: + from collections.abc import AsyncIterator, Iterator + +# No default sentinel object: None is the whole vocabulary, and the helpers +# below are the only readers. +CURRENT: ContextVar[Probe | None] = ContextVar("command_probe", default=None) + +# A request that never got an answer at all -- DNS, connect, timeout. Kept +# distinct from a 5xx, which is upstream answering and answering badly; the +# two mean different things when a command starts failing. +NO_RESPONSE: Final = 0 + +MS_PER_SECOND: Final = 1000 + + +@dataclass(frozen=True, slots=True) +class UpstreamCall: + """One outbound HTTP request made while serving a command.""" + + host: str + status: int + ms: int + + +@dataclass(slots=True) +class Probe: + """The mutable notes one invocation accumulates. + + Upstream calls are *buffered* rather than written as they happen: they + belong to a ``command_events`` row that does not exist yet, because the + outcome and the duration are only known once the handler has returned. + """ + + send_ms: int = 0 + item_id: str | None = None + calls: list[UpstreamCall] = field(default_factory=list) + + +@contextlib.contextmanager +def probing() -> Iterator[Probe]: + """Open a probe for the duration of one command, and always close it.""" + active = Probe() + token = CURRENT.set(active) + try: + yield active + finally: + CURRENT.reset(token) + + +def note_send(ms: int) -> None: + """Add the cost of one outbound Signal send. + + Accumulated rather than assigned: a handler may send twice, and + ``!pułapka`` does -- a picture and then its description. + """ + active = CURRENT.get() + if active is not None: + active.send_ms += ms + + +@contextlib.asynccontextmanager +async def timed_send() -> AsyncIterator[None]: + """Time one outbound send and note it, whether or not it succeeds. + + ``DOCS-latency.md`` measured ``POST /v2/send`` at 2.65 s of a 3.7 s + reply -- 72% of the wall clock, none of it this repository's code -- by + reading three log lines by hand, once. This makes that measurement + continuous, which is what the document's own rule asks for: an upstream + latency claim gets measured before it gets designed around. + """ + started = time.perf_counter() + try: + yield + finally: + note_send(round((time.perf_counter() - started) * MS_PER_SECOND)) + + +def note_upstream(host: str, status: int, ms: int) -> None: + """Add one upstream request. Host only -- never the URL. + + A URL carries identifiers, and some of them are the ones ``item_id`` + exists to govern. A hostname cannot carry anything. + """ + active = CURRENT.get() + if active is not None: + active.calls.append(UpstreamCall(host=host, status=status, ms=ms)) + + +def note_item(item_id: str) -> None: + """Record which item was served, as a public identifier. + + A Wikidata QID, a Commons page id, a Wolne Lektury slug, a pseudonym + key, a content-addressed filename. Never a title, a caption or a name: + this is what makes "the pope pool feels repetitive" a number instead of + an impression, and an identifier cannot carry a person. + """ + active = CURRENT.get() + if active is not None: + active.item_id = item_id diff --git a/signal_bot/analytics/recorder.py b/signal_bot/analytics/recorder.py new file mode 100644 index 0000000..ed5c6ac --- /dev/null +++ b/signal_bot/analytics/recorder.py @@ -0,0 +1,194 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""Turning one attempted command into one row of history. + +This is where the dispatcher's ``try/except`` around a handler now lives, so +that the dispatcher gains a ``with`` rather than forty lines and stays under +the 250-line cap. It also means the six outcome branches can be tested +directly, without driving a whole websocket frame through to reach an +exception path. + +The invariant that must survive every future edit to this file: recording a +*use* and recording an *attempt* are two different writes with two different +rules, and they may never be merged. + +* ``store.record_use`` is success-only and feeds the cooldown and the daily + limit. ``CommandRefusedError``'s own docstring records what happened when + a refusal reached it: the cooldown window slid forward every time somebody + was told to wait, so waiting the stated time never helped. +* ``store.commands.record`` takes every attempt, including the refusals and + the crashes -- which is the entire point, since a command that fails every + time used to be indistinguishable from one nobody ran. + +The temptation this creates is to hoist one call above the other, or to route +both through a single store method. ``test_dispatcher_recording`` fires a +refused invocation and asserts both that the history grew *and* that +``last_used`` did not move, which is the cheapest way to notice. +""" + +from __future__ import annotations + +import contextlib +import logging +import time +from dataclasses import dataclass +from typing import TYPE_CHECKING, Final + +from signal_bot.analytics.uses import ( + ARGS_PRESENT, + OUTCOME_COOLDOWN, + OUTCOME_DAILY, + OUTCOME_ERROR, + OUTCOME_OK, + OUTCOME_VOTE, + CommandEvent, +) +from signal_bot.decorators import GATE_COOLDOWN, GATE_DAILY, GATE_VOTE +from signal_bot.registry import CommandRefusedError +from signal_bot.stats.periods import day_of, hour_of, weekday_of + +if TYPE_CHECKING: + from collections.abc import Iterator + + from signal_bot.analytics import probe + from signal_bot.analytics.uses import CommandEventStore + +logger = logging.getLogger(__name__) + +MS_PER_SECOND: Final = 1000 + +# A gate that did not name itself. Reachable only from a bare +# ``raise CommandRefusedError``, which nothing in this repository does any +# more -- but the constructor still allows it, so the mapping must too. +OUTCOME_REFUSED: Final = "refused_cooldown" + +_BY_GATE: Final = { + GATE_COOLDOWN: OUTCOME_COOLDOWN, + GATE_DAILY: OUTCOME_DAILY, + GATE_VOTE: OUTCOME_VOTE, +} + + +def refusal_outcome(gate: str) -> str: + """Name the outcome one gate's refusal should be filed under.""" + return _BY_GATE.get(gate, OUTCOME_REFUSED) + + +def args_token(args: tuple[str, ...]) -> str: + """Reduce a command's arguments to something that cannot hold a name. + + The registry has already replaced a photo album's arguments with their + pseudonym keys before the dispatcher sees them -- that is what + ``arg_resolver`` is for, and why it runs at resolution rather than in a + handler. Those keys are safe and worth keeping: "which person is asked + for most" is a real question. + + Anything else becomes a flag. A column of words people typed is exactly + the shape that put real names in this repository once, and the personal + data gate learns the names it refuses from the live database -- so it + would catch a name typed correctly and could never catch a typo. + """ + if not args: + return "" + # One resolved token is a key. Several words are words. + if len(args) == 1 and args[0].isalnum(): + return args[0] + return ARGS_PRESENT + + +@dataclass(slots=True) +class Attempt: + """What the dispatcher learned by trying to run one command.""" + + outcome: str = OUTCOME_OK + error_class: str = "" + handler_ms: int = 0 + + @property + def ok(self) -> bool: + """True when the handler actually ran to completion. + + The dispatcher asks this rather than comparing to a constant, so + "did it work" has one spelling and the *use* -- the write that feeds + the cooldown -- cannot accidentally start counting refusals. + """ + return self.outcome == OUTCOME_OK + + +@contextlib.contextmanager +def attempting(command: str) -> Iterator[Attempt]: + """Run one handler, and come out knowing how it went and what it cost. + + Swallows the handler's exception rather than re-raising: the dispatcher + has always treated a failed command as an answered frame, because a raise + here ends the receive loop and the bot goes silently deaf until the + watchdog notices. That behaviour is unchanged -- only now the failure + leaves a row behind. + """ + attempt = Attempt() + started = time.perf_counter() + try: + yield attempt + except CommandRefusedError as refusal: + attempt.outcome = refusal_outcome(refusal.gate) + logger.debug("command %s refused by a gate", command) + except Exception as error: + # The class name, never str(error): an exception's message carries + # whatever it was processing, and here that is somebody's message. + # "17 ImageFetchError" is the whole diagnostic value anyway. + attempt.outcome = OUTCOME_ERROR + attempt.error_class = type(error).__name__ + logger.exception("command %s failed", command) + attempt.handler_ms = round((time.perf_counter() - started) * MS_PER_SECOND) + + +@dataclass(frozen=True, slots=True) +class Invocation: + """Who ran what, where, and when the message that asked was sent. + + Grouped rather than passed as five arguments, because these five always + travel together and none of them means anything without the rest. + """ + + group_id: str + user_uuid: str + command: str + args: tuple[str, ...] + sent_at_ms: int + + +def record_attempt( + store: CommandEventStore, + invocation: Invocation, + attempt: Attempt, + notes: probe.Probe, +) -> None: + """Append one attempt, and never let doing so break the bot. + + The whole body is guarded for the reason ``_answer_unknown``'s is: this + runs inside the receive loop, and an exception escaping it stops the bot + answering anything at all. Analytics are worth a great deal less than + that, so a storage failure is logged and the frame carries on. + """ + sent_at_ms = invocation.sent_at_ms + day = day_of(sent_at_ms) + try: + store.record( + CommandEvent( + group_id=invocation.group_id, + user_uuid=invocation.user_uuid, + command=invocation.command, + args=args_token(invocation.args), + item_id=notes.item_id, + outcome=attempt.outcome, + error_class=attempt.error_class, + invoked_at_ms=sent_at_ms, + day=day, + local_hour=hour_of(sent_at_ms), + weekday=weekday_of(day), + handler_ms=attempt.handler_ms, + send_ms=notes.send_ms, + calls=tuple(notes.calls), + ) + ) + except Exception: + logger.exception("could not record the use of %s", invocation.command) diff --git a/signal_bot/analytics/render.py b/signal_bot/analytics/render.py new file mode 100644 index 0000000..33526a8 --- /dev/null +++ b/signal_bot/analytics/render.py @@ -0,0 +1,199 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""What !analityka says in the group. + +Twelve lines is a cap, not a quota: a line with nothing to say is dropped +rather than printed empty, the same way the nightly report drops its +highlights. No ``ljust`` anywhere -- Signal renders a proportional font, so +padding lines up in a terminal and on nobody's phone. + +Two rules about people, both inherited from ``DOCS-statistics.md``. + +Only somebody whose display name has actually been seen may be named; the +group listing knows everyone else as a raw uuid or a phone number, and +printing either into a chat is a leak. And a per-person breakdown of +everybody's habits stays on the management page: the group gets a top three +and the asker's own favourite, which is what was decided when this was +specified. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from typing import TYPE_CHECKING, Final + +from signal_bot.analytics import trend as trends +from signal_bot.stats import text +from signal_bot.stats.periods import pl_day_month + +if TYPE_CHECKING: + from collections.abc import Mapping, Sequence + + from signal_bot.analytics.demand import Demand + from signal_bot.analytics.hours import Peak + from signal_bot.analytics.scope import Window + from signal_bot.analytics.usage import CommandUse, PersonUse + +MAX_LINES: Final = 12 +TOP_COMMANDS: Final = 3 +TOP_PEOPLE: Final = 3 + +HEADING: Final = "Analityka komend" +NOTHING_YET: Final = "Nikt jeszcze nie użył żadnej komendy." + +DIRECTIONS: Final = { + trends.RISING: "użycie rośnie", + trends.FALLING: "użycie spada", + trends.FLAT: "użycie się trzyma", +} + + +def days_phrase(count: int) -> str: + """A day count with its noun agreeing: 3 dni.""" + return f"{count} {text.plural(count, 'dzień', 'dni', 'dni')}" + + +def times_phrase(count: int) -> str: + """A count with its noun agreeing: 5 razy.""" + return f"{count} {text.plural(count, 'raz', 'razy', 'razy')}" + + +def window_line(measured: Window) -> str: + """What the numbers actually cover. Always printed, always first. + + The bot is days old, not months. A reader who cannot see that is reading + a month panel drawn from three days as though it were a month. + """ + span = days_phrase(measured.days) + if measured.first_day == measured.last_day: + return f"Okno: {span} ({pl_day_month(measured.first_day)}), {times_phrase(measured.events)}" + return ( + f"Okno: {span} ({pl_day_month(measured.first_day)}" + f" — {pl_day_month(measured.last_day)}), {times_phrase(measured.events)}" + ) + + +def top_line(uses: Sequence[CommandUse]) -> str: + """The three most used commands, with their counts.""" + best = ", ".join(f"!{use.command} {use.uses}" for use in uses[:TOP_COMMANDS]) + return f"Najczęściej: {best}" + + +def rarest_line(uses: Sequence[CommandUse]) -> str: + """The least used command that anybody has used at all.""" + quietest = uses[-1] + return f"Najrzadziej: !{quietest.command} — {times_phrase(quietest.uses)}" + + +def hours_line(peak: Peak) -> str: + """When the group actually reaches for the bot.""" + return f"Najgłośniejsza godzina: {peak.busiest:02d}:00 ({times_phrase(peak.busiest_uses)})" + + +def trend_line(direction: str) -> str: + """The verdict, or nothing at all when it is too early to have one. + + Returning "" rather than "za mało danych" is deliberate: the window line + above already says how many days there are, and repeating it as an + apology wastes one of twelve lines. + """ + return DIRECTIONS.get(direction, "") + + +def health_line(uses: Sequence[CommandUse]) -> str: + """The command most in need of attention, if any is. + + Refusals count as much as errors here. A command refused half the time + is not popular, it is mis-tuned, and from the group's side both read as + "it does not work". + """ + troubled = [use for use in uses if use.errors or use.refused] + if not troubled: + return "" + worst = max(troubled, key=lambda use: (use.errors + use.refused, use.command)) + # "błędów: 3" rather than "3 błędy": a label and a number needs no + # agreement at all, and Polish agreement after a numeral has three forms + # whose rule reads the last two digits. Sidestepping it here is free. + parts = [] + if worst.errors: + parts.append(f"błędów: {worst.errors}") + if worst.refused: + parts.append(f"odmów: {worst.refused}") + return f"Do poprawki: !{worst.command} ({', '.join(parts)})" + + +def demand_line(demand: Demand) -> str: + """The trigger people keep almost typing, which wants another alias.""" + if not demand.near_misses: + return "" + miss = demand.near_misses[0] + return f"Blisko trafień: {miss.target} — {times_phrase(miss.hits)}" + + +def people_line(people: Sequence[PersonUse], names: Mapping[str, str]) -> str: + """The top three, and only those whose name the bot has actually seen.""" + named = [(names[person.uuid], person.uses) for person in people if person.uuid in names] + if not named: + return "" + best = ", ".join(f"{name} {uses}" for name, uses in named[:TOP_PEOPLE]) + return f"Kto najwięcej: {best}" + + +def mine_line(people: Sequence[PersonUse], uuid: str) -> str: + """The asker's own favourite. Self-scope, so it names nobody else.""" + for person in people: + if person.uuid == uuid and person.favourite: + return f"Twoja ulubiona: !{person.favourite} ({times_phrase(person.uses)})" + return "" + + +def share_line(commands: int, messages: int) -> str: + """How much of the group's talking is talking to the bot.""" + if not messages: + return "" + percent = round(100 * commands / messages) + return f"Komendy to {percent}% wiadomości w tej grupie" + + +@dataclass(frozen=True, slots=True) +class Summary: + """Everything the reply needs, gathered before any of it is worded. + + One object rather than ten arguments, because these always travel + together: the command builds it from the query layer and the renderer + turns it into Polish, and neither half needs to know the other. + """ + + measured: Window + uses: Sequence[CommandUse] + people: Sequence[PersonUse] + names: Mapping[str, str] + peak: Peak + direction: str + demand: Demand + asker: str + commands: int + messages: int + + +def render(summary: Summary) -> str: + """Build the reply, capped at twelve lines and never padded.""" + if not summary.uses: + return NOTHING_YET + lines = [ + HEADING, + window_line(summary.measured), + top_line(summary.uses), + rarest_line(summary.uses), + hours_line(summary.peak), + trend_line(summary.direction), + share_line(summary.commands, summary.messages), + health_line(summary.uses), + demand_line(summary.demand), + people_line(summary.people, summary.names), + mine_line(summary.people, summary.asker), + ] + # Dropped, then capped. A line with nothing to say is not printed empty, + # so the cap bites on content rather than on blanks -- and the cap is + # what keeps a reply readable on a phone. + kept = [line for line in lines if line][:MAX_LINES] + return "\n".join(kept) diff --git a/signal_bot/analytics/scope.py b/signal_bot/analytics/scope.py new file mode 100644 index 0000000..06b4a34 --- /dev/null +++ b/signal_bot/analytics/scope.py @@ -0,0 +1,119 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""What a question is being asked *about*, and which rows may answer it. + +Every query in this package is a complete literal string. None of them is +built by concatenation or by an f-string, which is the only arrangement ruff +S608 accepts and this repository takes no suppressions -- ``forget_group`` +spells its DELETEs out for the same reason. + +"One group, or all of them" is therefore not a clause that gets appended. It +is written into every query once, as:: + + WHERE (:group IS NULL OR group_id = :group) + +Binding ``:group`` to None makes the first half true and the filter vanish. +So the roll-up and the per-group answer run *the same SQL*, which is what +stops them drifting apart -- two queries written to agree eventually will +not. + +The other shared decision here is the source filter. Three classes of row +live in ``command_events``: + +* ``live`` -- observed as it happened; the only rows with real durations. +* ``journald`` -- reconstructed from the log, with a real hour taken from + the message it was matched to, but no duration and an outcome that was + inferred rather than seen. +* ``daily_uses`` -- a count with no hour at all, carrying ``UNKNOWN_HOUR``. + +Counting all three is right for "how often was this used" and wrong for "at +what hour" or "how slow is it". Those two predicates are named here, once, +so that the exclusion is a shared constant rather than something six modules +each have to remember. ``stats/backfill.py`` records what forgetting it +costs: a synthetic hour reached ``favourite_hour`` and the answer was +quietly wrong. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from typing import TYPE_CHECKING, Final + +if TYPE_CHECKING: + import sqlite3 + +# How long a person may go quiet before their next command starts a new +# session. Used only by burstiness and co-occurrence, the two questions that +# need "in one sitting" to mean anything. +# +# Ten minutes: comfortably longer than the pause between two commands +# somebody is idly firing off, far shorter than the gap between two separate +# times of day they picked their phone up. +SESSION_GAP_MS: Final = 10 * 60 * 1000 + +# Below this many days, a direction is not stated. The line is still drawn -- +# the reader can see it -- but no arrow and no "rośnie"/"spada" verdict, +# because three points of a new bot's novelty curve are not a trend. +TREND_MIN_DAYS: Final = 14 + + +@dataclass(frozen=True, slots=True) +class Scope: + """One group, or all of them.""" + + group_id: str | None = None + + def params(self, **extra: object) -> dict[str, object]: + """The bound parameters every query in this package expects.""" + return {"group": self.group_id, **extra} + + +@dataclass(frozen=True, slots=True) +class Window: + """Which days an answer actually covers, and where its rows came from. + + Carried on every response because this bot is days old, not months. A + month panel drawn from three days is not wrong, but a reader who cannot + see that it is three days will read it as though it were a month. + """ + + first_day: str = "" + last_day: str = "" + days: int = 0 + events: int = 0 + live: int = 0 + journald: int = 0 + daily: int = 0 + + @property + def enough_for_a_trend(self) -> bool: + """Whether a direction may be stated rather than merely plotted.""" + return self.days >= TREND_MIN_DAYS + + +WINDOW_SQL: Final = """ +SELECT MIN(day) AS first_day, + MAX(day) AS last_day, + COUNT(DISTINCT day) AS days, + COUNT(*) AS events, + SUM(source = 'live') AS live, + SUM(source = 'journald') AS journald, + SUM(source = 'daily_uses') AS daily +FROM command_events +WHERE (:group IS NULL OR group_id = :group) +""" + + +def window(conn: sqlite3.Connection, scope: Scope) -> Window: + """Measure what the data actually covers, before anything is drawn.""" + row = conn.execute(WINDOW_SQL, scope.params()).fetchone() + if row is None or not row["events"]: + return Window() + return Window( + first_day=str(row["first_day"]), + last_day=str(row["last_day"]), + days=int(row["days"]), + events=int(row["events"]), + live=int(row["live"] or 0), + journald=int(row["journald"] or 0), + daily=int(row["daily"] or 0), + ) diff --git a/signal_bot/analytics/sessions.py b/signal_bot/analytics/sessions.py new file mode 100644 index 0000000..710b644 --- /dev/null +++ b/signal_bot/analytics/sessions.py @@ -0,0 +1,113 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""Family E, the behavioural half: sittings, bursts and what goes together. + +These are the only questions that need a *trustworthy ordering in time*, so +they are the only ones restricted to ``live`` rows plus the journald ones +that carry a real hour. A ``daily_uses`` row is a count with a synthetic +timestamp built from midnight; treating those as a sequence would invent a +sitting in which somebody ran forty commands in forty milliseconds. + +A session is one person's run of commands with no gap longer than +``SESSION_GAP_MS``. It exists to make two things sayable: + +* **burstiness** -- how many commands a sitting contains. Three or four + means somebody is re-rolling for a better photo, which is a fact about + the pool rather than about them. +* **co-occurrence** -- which commands turn up in the same sitting, which is + what says whether two commands are alternatives or a habit. +""" + +from __future__ import annotations + +from collections import Counter +from dataclasses import dataclass +from itertools import combinations +from typing import TYPE_CHECKING, Final + +from signal_bot.analytics.scope import SESSION_GAP_MS + +if TYPE_CHECKING: + import sqlite3 + + from signal_bot.analytics.scope import Scope + +# Only rows whose timestamp is a real moment. daily_uses rows are excluded +# by local_hour, which is the one column that distinguishes them. +ORDERED_SQL: Final = """ +SELECT user_uuid, command, invoked_at_ms +FROM command_events +WHERE (:group IS NULL OR group_id = :group) + AND local_hour >= 0 + AND outcome != 'unknown_word' +ORDER BY user_uuid, invoked_at_ms +""" + + +@dataclass(frozen=True, slots=True) +class Sessions: + """How the group actually uses the bot, sitting by sitting.""" + + count: int = 0 + commands: int = 0 + longest: int = 0 + pairs: tuple[tuple[str, str, int], ...] = () + repeats: tuple[tuple[str, int], ...] = () + + @property + def average_length(self) -> float: + """Commands per sitting. One means nobody re-rolls.""" + return self.commands / self.count if self.count else 0.0 + + +def _split(rows: list[tuple[str, str, int]]) -> list[list[str]]: + """Cut one person's ordered commands into sittings on the idle gap.""" + sittings: list[list[str]] = [] + current: list[str] = [] + last_uuid = "" + last_at = 0 + for uuid, command, at_ms in rows: + broken = uuid != last_uuid or at_ms - last_at > SESSION_GAP_MS + if broken and current: + sittings.append(current) + current = [] + current.append(command) + last_uuid = uuid + last_at = at_ms + if current: + sittings.append(current) + return sittings + + +def collect(conn: sqlite3.Connection, scope: Scope) -> Sessions: + """Split the history into sittings and describe what they contain.""" + rows = [ + (str(row["user_uuid"]), str(row["command"]), int(row["invoked_at_ms"])) + for row in conn.execute(ORDERED_SQL, scope.params()) + ] + sittings = _split(rows) + if not sittings: + return Sessions() + + pairs: Counter[tuple[str, str]] = Counter() + repeats: Counter[str] = Counter() + for sitting in sittings: + # Sorted and de-duplicated, so (cat, dog) and (dog, cat) are one + # fact rather than two, and running !kot four times in a sitting + # does not make it co-occur with itself. + distinct = sorted(set(sitting)) + for left, right in combinations(distinct, 2): + pairs[(left, right)] += 1 + for command, times in Counter(sitting).items(): + if times > 1: + repeats[command] += times - 1 + + return Sessions( + count=len(sittings), + commands=sum(len(sitting) for sitting in sittings), + longest=max(len(sitting) for sitting in sittings), + pairs=tuple( + (left, right, count) + for (left, right), count in sorted(pairs.items(), key=lambda p: (-p[1], p[0])) + ), + repeats=tuple(sorted(repeats.items(), key=lambda pair: (-pair[1], pair[0]))), + ) diff --git a/signal_bot/analytics/trend.py b/signal_bot/analytics/trend.py new file mode 100644 index 0000000..935f668 --- /dev/null +++ b/signal_bot/analytics/trend.py @@ -0,0 +1,187 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""Family B, the calendar half: day series, months, and what is dying. + +Day-granular, so every source counts -- a ``daily_uses`` row has no hour but +it does know its day, which is exactly what these questions need. + +The direction of a trend is deliberately not always stated. ``Window`` +carries the day count and ``TREND_MIN_DAYS`` is the threshold; below it the +series is returned and drawn but ``direction`` stays ``UNKNOWN``. A bot that +is three days old has a novelty curve, not a trend, and an arrow drawn from +three points is a claim the data cannot support. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from typing import TYPE_CHECKING, Final + +from signal_bot.stats.periods import shift + +if TYPE_CHECKING: + import sqlite3 + from collections.abc import Sequence + + from signal_bot.analytics.scope import Scope, Window + +RISING: Final = "rising" +FALLING: Final = "falling" +FLAT: Final = "flat" +UNKNOWN: Final = "unknown" + +# How much the second half must differ from the first to be called a +# direction rather than noise. A tenth: below that, on counts this small, +# one busy evening decides the answer. +TREND_MARGIN: Final = 0.10 + +# A command not used for this many days is called dying rather than merely +# quiet. Deliberately short, because the whole history is days long; it is +# a relative statement and the panel says so. +DYING_AFTER_DAYS: Final = 3 + +# A direction needs a before and an after to compare. +MIN_POINTS: Final = 2 + +BY_DAY_SQL: Final = """ +SELECT day, COUNT(*) AS uses, COUNT(DISTINCT user_uuid) AS people +FROM command_events +WHERE (:group IS NULL OR group_id = :group) + AND outcome != 'unknown_word' +GROUP BY day +ORDER BY day +""" + +BY_COMMAND_DAY_SQL: Final = """ +SELECT command, day, COUNT(*) AS uses +FROM command_events +WHERE (:group IS NULL OR group_id = :group) + AND outcome != 'unknown_word' +GROUP BY command, day +ORDER BY command, day +""" + +BY_MONTH_SQL: Final = """ +SELECT substr(day, 1, 7) AS month, COUNT(*) AS uses, COUNT(DISTINCT day) AS days +FROM command_events +WHERE (:group IS NULL OR group_id = :group) + AND outcome != 'unknown_word' +GROUP BY month +ORDER BY month +""" + +LAST_USED_SQL: Final = """ +SELECT command, MAX(day) AS last_day, COUNT(*) AS uses +FROM command_events +WHERE (:group IS NULL OR group_id = :group) + AND outcome = 'ok' +GROUP BY command +ORDER BY last_day, command +""" + + +@dataclass(frozen=True, slots=True) +class Dying: + """A command that used to be used and has gone quiet.""" + + command: str + last_day: str + idle_days: int + uses: int + + +def by_day(conn: sqlite3.Connection, scope: Scope) -> list[tuple[str, int, int]]: + """(day, uses, people) for every day anything happened.""" + return [ + (str(row["day"]), int(row["uses"]), int(row["people"])) + for row in conn.execute(BY_DAY_SQL, scope.params()) + ] + + +def dense_days(series: Sequence[tuple[str, int, int]]) -> list[tuple[str, int]]: + """Fill in the days nothing happened on. + + A sparse series draws a line straight over a silent day, which reads as + "steady" when the truth is "nobody used it". Built by stepping the date + rather than by adding 86_400 seconds, because Warsaw days are not all + the same length. + """ + if not series: + return [] + filled: list[tuple[str, int]] = [] + day = series[0][0] + last = series[-1][0] + counts = {entry[0]: entry[1] for entry in series} + while day <= last: + filled.append((day, counts.get(day, 0))) + day = shift(day, 1) + return filled + + +def by_command_day(conn: sqlite3.Connection, scope: Scope) -> dict[str, dict[str, int]]: + """Per-command day series, for the sparklines.""" + series: dict[str, dict[str, int]] = {} + for row in conn.execute(BY_COMMAND_DAY_SQL, scope.params()): + series.setdefault(str(row["command"]), {})[str(row["day"])] = int(row["uses"]) + return series + + +def by_month(conn: sqlite3.Connection, scope: Scope) -> list[tuple[str, int, int]]: + """(month, uses, days seen). The days matter: a partial month is not low.""" + return [ + (str(row["month"]), int(row["uses"]), int(row["days"])) + for row in conn.execute(BY_MONTH_SQL, scope.params()) + ] + + +def direction(series: Sequence[tuple[str, int]], measured: Window) -> str: + """Whether use is rising, falling, flat -- or too early to say. + + Compares the mean of the second half against the first. Not a + least-squares fit: on a dozen points a fit is no more truthful and far + harder to explain in a chat message. + """ + if not measured.enough_for_a_trend or len(series) < MIN_POINTS: + return UNKNOWN + # MIN_POINTS above guarantees half >= 1, so neither slice is empty and + # neither division below is by zero. + half = len(series) // 2 + first = [count for _day, count in series[:half]] + second = [count for _day, count in series[half:]] + before = sum(first) / len(first) + after = sum(second) / len(second) + if before == 0: + return RISING if after > 0 else FLAT + change = (after - before) / before + if change > TREND_MARGIN: + return RISING + if change < -TREND_MARGIN: + return FALLING + return FLAT + + +def dying(conn: sqlite3.Connection, scope: Scope, today: str) -> list[Dying]: + """Commands that worked, were used, and then stopped being used.""" + found: list[Dying] = [] + for row in conn.execute(LAST_USED_SQL, scope.params()): + last_day = str(row["last_day"]) + idle = _days_between(last_day, today) + if idle >= DYING_AFTER_DAYS: + found.append( + Dying( + command=str(row["command"]), + last_day=last_day, + idle_days=idle, + uses=int(row["uses"]), + ) + ) + return sorted(found, key=lambda entry: (-entry.idle_days, entry.command)) + + +def _days_between(start: str, end: str) -> int: + """Whole calendar days from one day to another, never negative.""" + days = 0 + day = start + while day < end: + day = shift(day, 1) + days += 1 + return days diff --git a/signal_bot/analytics/usage.py b/signal_bot/analytics/usage.py new file mode 100644 index 0000000..70a182b --- /dev/null +++ b/signal_bot/analytics/usage.py @@ -0,0 +1,197 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""Family A: who uses which commands, how often, and which nobody uses. + +Counts include every source. "How many times was !papież used" is answered +by all three classes of row -- a reconstructed invocation still happened. +Only the questions that need an hour or a duration narrow to ``live``, and +those live in ``hours`` and ``health``. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from typing import TYPE_CHECKING, Final + +if TYPE_CHECKING: + import sqlite3 + from collections.abc import Sequence + + from signal_bot.analytics.scope import Scope + + +@dataclass(frozen=True, slots=True) +class CommandUse: + """One command, and everything countable about it.""" + + command: str + uses: int + people: int + ok: int + refused: int + errors: int + first_day: str + last_day: str + + @property + def failure_rate(self) -> float: + """Share of attempts that crashed. Refusals are not failures.""" + return self.errors / self.uses if self.uses else 0.0 + + @property + def refusal_rate(self) -> float: + """Share of attempts a gate turned away. + + High is not popular, it is mis-tuned: it means people keep asking + and keep being told no. + """ + return self.refused / self.uses if self.uses else 0.0 + + +@dataclass(frozen=True, slots=True) +class PersonUse: + """One person's total, by uuid. Names are attached by the caller.""" + + uuid: str + uses: int + commands: int + favourite: str + + +BY_COMMAND_SQL: Final = """ +SELECT command, + COUNT(*) AS uses, + COUNT(DISTINCT user_uuid) AS people, + SUM(outcome = 'ok') AS ok, + SUM(outcome LIKE 'refused_%') AS refused, + SUM(outcome = 'error') AS errors, + MIN(day) AS first_day, + MAX(day) AS last_day +FROM command_events +WHERE (:group IS NULL OR group_id = :group) + AND outcome != 'unknown_word' +GROUP BY command +ORDER BY uses DESC, command +""" + +BY_PERSON_SQL: Final = """ +SELECT user_uuid, + COUNT(*) AS uses, + COUNT(DISTINCT command) AS commands +FROM command_events +WHERE (:group IS NULL OR group_id = :group) + AND outcome = 'ok' +GROUP BY user_uuid +ORDER BY uses DESC, user_uuid +""" + +FAVOURITE_SQL: Final = """ +SELECT user_uuid, command, COUNT(*) AS uses +FROM command_events +WHERE (:group IS NULL OR group_id = :group) + AND outcome = 'ok' +GROUP BY user_uuid, command +ORDER BY user_uuid, uses DESC, command +""" + +MATRIX_SQL: Final = """ +SELECT user_uuid, command, COUNT(*) AS uses +FROM command_events +WHERE (:group IS NULL OR group_id = :group) + AND outcome = 'ok' +GROUP BY user_uuid, command +ORDER BY user_uuid, command +""" + +TOTALS_SQL: Final = """ +SELECT COUNT(*) AS attempts, + SUM(outcome = 'ok') AS ok, + COUNT(DISTINCT user_uuid) AS people, + COUNT(DISTINCT command) AS commands +FROM command_events +WHERE (:group IS NULL OR group_id = :group) + AND outcome != 'unknown_word' +""" + + +def by_command(conn: sqlite3.Connection, scope: Scope) -> list[CommandUse]: + """Every command that has ever been attempted, most used first.""" + return [ + CommandUse( + command=str(row["command"]), + uses=int(row["uses"]), + people=int(row["people"]), + ok=int(row["ok"] or 0), + refused=int(row["refused"] or 0), + errors=int(row["errors"] or 0), + first_day=str(row["first_day"]), + last_day=str(row["last_day"]), + ) + for row in conn.execute(BY_COMMAND_SQL, scope.params()) + ] + + +def _favourites(conn: sqlite3.Connection, scope: Scope) -> dict[str, str]: + """Each person's most-used command. + + The query orders by uses within each person, so the first row seen for + a uuid is the answer and the rest are skipped -- cheaper than a window + function and portable to the ancient sqlite a container might carry. + """ + best: dict[str, str] = {} + for row in conn.execute(FAVOURITE_SQL, scope.params()): + best.setdefault(str(row["user_uuid"]), str(row["command"])) + return best + + +def by_person(conn: sqlite3.Connection, scope: Scope) -> list[PersonUse]: + """Who uses the bot, by uuid and most active first. + + Deliberately returns uuids. Turning one into a name is the caller's job + and a different rule in each place: the group chat may name only people + whose display name has been seen, the management page may show + everyone, and neither may print a raw uuid or a phone number. + """ + favourites = _favourites(conn, scope) + return [ + PersonUse( + uuid=str(row["user_uuid"]), + uses=int(row["uses"]), + commands=int(row["commands"]), + favourite=favourites.get(str(row["user_uuid"]), ""), + ) + for row in conn.execute(BY_PERSON_SQL, scope.params()) + ] + + +def matrix(conn: sqlite3.Connection, scope: Scope) -> list[tuple[str, str, int]]: + """Every (person, command, count) triple, for the heatmap on the page. + + Never posted into a group -- a per-person breakdown of everybody's + habits is a management-page thing. + """ + return [ + (str(row["user_uuid"]), str(row["command"]), int(row["uses"])) + for row in conn.execute(MATRIX_SQL, scope.params()) + ] + + +def totals(conn: sqlite3.Connection, scope: Scope) -> dict[str, int]: + """Attempts, successes, distinct people and distinct commands.""" + row = conn.execute(TOTALS_SQL, scope.params()).fetchone() + return { + "attempts": int(row["attempts"] or 0), + "ok": int(row["ok"] or 0), + "people": int(row["people"] or 0), + "commands": int(row["commands"] or 0), + } + + +def never_used(known: Sequence[str], used: Sequence[CommandUse]) -> list[str]: + """Registered commands with no recorded attempt at all. + + Takes the registry's list rather than reading it, because which commands + exist depends on the group -- the album's are registered only where the + album is -- and this module must not know that. + """ + seen = {use.command for use in used} + return sorted(name for name in known if name not in seen) diff --git a/signal_bot/analytics/uses.py b/signal_bot/analytics/uses.py new file mode 100644 index 0000000..9d4314b --- /dev/null +++ b/signal_bot/analytics/uses.py @@ -0,0 +1,149 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""Writing the command history: one row per attempt, and what it cost. + +Shares the ``Store`` connection exactly as ``EventStore`` does, so a command +and the message that triggered it are committed by the same connection in +the same order they happened. + +The column set is pinned by ``test_command_events`` for the reason +``message_events``' is: this table sits beside a public repository that +leaked real names once, and a column added quietly is how the next one would +arrive. Nothing here holds free text. ``command`` is a registry name, +``args`` is a resolved token, ``item_id`` is a public identifier and +``error_class`` is an exception's class name. +""" + +from __future__ import annotations + +from dataclasses import dataclass, field +from typing import TYPE_CHECKING, Final + +if TYPE_CHECKING: + import sqlite3 + from collections.abc import Sequence + + from signal_bot.analytics.probe import UpstreamCall + +OUTCOME_OK: Final = "ok" +OUTCOME_COOLDOWN: Final = "refused_cooldown" +OUTCOME_DAILY: Final = "refused_daily" +OUTCOME_VOTE: Final = "refused_vote" +OUTCOME_ERROR: Final = "error" +OUTCOME_UNKNOWN: Final = "unknown_word" + +SOURCE_LIVE: Final = "live" +SOURCE_JOURNAL: Final = "journald" +SOURCE_DAILY: Final = "daily_uses" + +# What ``args`` holds when a command took arguments that resolve to nothing +# nameable. Never the words themselves: a user-typed argument column is the +# exact shape that put real names in this repository once. +ARGS_PRESENT: Final = "+args" + + +@dataclass(frozen=True, slots=True) +class CommandEvent: + """One attempt to run one command, whether or not it ran.""" + + group_id: str + user_uuid: str + command: str + outcome: str + invoked_at_ms: int + day: str + local_hour: int + weekday: int + args: str = "" + item_id: str | None = None + error_class: str = "" + # None rather than 0: a backfilled row has no duration, and zero + # milliseconds is a lie that reads as data and would drag a p50 down. + # The lesson is already written into backfill.UNKNOWN_HOUR's comment. + handler_ms: int | None = None + send_ms: int | None = None + source: str = SOURCE_LIVE + calls: Sequence[UpstreamCall] = field(default_factory=tuple) + + +class CommandEventStore: + """Reads and writes the command history. Shares the Store's connection.""" + + def __init__(self, conn: sqlite3.Connection) -> None: + """Wrap an open connection whose schema has already been applied.""" + self._conn = conn + + @property + def connection(self) -> sqlite3.Connection: + """The shared connection, for the query modules in this package.""" + return self._conn + + def record(self, event: CommandEvent) -> None: + """Append one attempt and the upstream calls it made. + + ``ON CONFLICT DO NOTHING`` against the natural key is what makes a + signal-cli redelivery harmless, the same property ``message_events`` + gets from its primary key -- ``invoked_at_ms`` is the envelope's own + timestamp, so a replayed frame collides with the row it already + wrote instead of adding a second one. + + It is also what makes the backfill re-runnable, which is why the + backfill needs no delete-then-rebuild pass and cannot lose rows to + a journal that has since been vacuumed. + """ + self._conn.execute( + """ + INSERT INTO command_events ( + group_id, user_uuid, command, args, item_id, outcome, + error_class, invoked_at_ms, day, local_hour, weekday, + handler_ms, send_ms, source + ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) + ON CONFLICT(group_id, user_uuid, command, invoked_at_ms, source) + DO NOTHING + """, + ( + event.group_id, + event.user_uuid, + event.command, + event.args, + event.item_id, + event.outcome, + event.error_class, + event.invoked_at_ms, + event.day, + event.local_hour, + event.weekday, + event.handler_ms, + event.send_ms, + event.source, + ), + ) + self._record_calls(event) + self._conn.commit() + + def _record_calls(self, event: CommandEvent) -> None: + """Append the upstream calls one invocation made. + + Keyed on (group, timestamp, seq) rather than on the parent's rowid. + ``lastrowid`` is unusable here: ``ON CONFLICT DO NOTHING`` leaves it + holding whatever the previous successful insert returned, so a + replayed frame would file its calls under an unrelated invocation. + """ + for seq, call in enumerate(event.calls): + self._conn.execute( + """ + INSERT INTO upstream_calls ( + group_id, invoked_at_ms, command, host, status, + duration_ms, seq + ) VALUES (?, ?, ?, ?, ?, ?, ?) + ON CONFLICT(group_id, invoked_at_ms, seq) DO NOTHING + """, + ( + event.group_id, + event.invoked_at_ms, + event.command, + call.host, + call.status, + call.ms, + seq, + ), + ) diff --git a/signal_bot/analytics_command.py b/signal_bot/analytics_command.py new file mode 100644 index 0000000..92e0904 --- /dev/null +++ b/signal_bot/analytics_command.py @@ -0,0 +1,96 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""!analityka -- what the group has actually been asking the bot for. + +Registered here rather than in ``stats/commands.py``, which is a few lines +under the 250 cap already, and because this reads a different table: +``stats`` is the *message* history, this is the *command* history. + +The reply is text only, in Polish, capped at twelve lines. The charts live +on the management page; a rendered image in the chat would mean a new +dependency and, per ``DOCS-latency.md``, would turn a 250 ms text reply into +a three-second one for a picture nobody can zoom into. + +Like ``!pomoc``, the registry is read when the command *runs* rather than +when it is registered -- at registration time the album's commands do not +exist yet, and those are exactly the triggers a near miss might be against. +""" + +from __future__ import annotations + +from typing import TYPE_CHECKING, Final + +from signal_bot.analytics import adoption, demand, hours, render, trend, usage +from signal_bot.analytics.scope import Scope, window +from signal_bot.animals import load_aliases + +if TYPE_CHECKING: + from collections.abc import Sequence + + from signal_bot.context import CommandContext + from signal_bot.decorators import Decorator + from signal_bot.registry import Command, CommandRegistry, Handler + +COMMAND_NAME: Final = "command_stats" +DESCRIPTION: Final = "kto ile używa bota i które komendy żyją" + + +def triggers_for(commands: Sequence[Command], group_id: str) -> tuple[str, ...]: + """Every trigger this group could plausibly have meant. + + Scoped to the group, and that scoping is the point: the album's triggers + are real people's names and exist only where the album does. Reporting a + near miss against one outside its group would announce that the album + exists, which is the same leak the dispatcher refuses at resolution. + """ + return tuple( + trigger + for command in commands + if command.allowed_in(group_id) + for trigger in command.triggers + ) + + +def summarise(ctx: CommandContext, commands: Sequence[Command]) -> render.Summary: + """Gather every number the reply needs, from the group that asked. + + Scoped to one group like every other statistic here: a test group's + numbers must never turn up in the production group's answer. + """ + conn = ctx.store.commands.connection + scope = Scope(ctx.group_id) + measured = window(conn, scope) + series = trend.dense_days(trend.by_day(conn, scope)) + share = adoption.share_of_chat(conn, scope) + return render.Summary( + measured=measured, + uses=usage.by_command(conn, scope), + people=usage.by_person(conn, scope), + # Only people the bot has actually seen a name for. Everyone else is + # counted and left unnamed: the roster knows them as a raw uuid or a + # phone number, and printing either into a chat is a leak. + names={ + person.uuid: person.display_name for person in ctx.store.events.people(ctx.group_id) + }, + peak=hours.peak(hours.by_hour(conn, scope)), + direction=trend.direction(series, measured), + demand=demand.collect(conn, scope, triggers_for(commands, ctx.group_id)), + asker=ctx.message.source_uuid, + commands=share.commands, + messages=share.messages, + ) + + +def register_analytics_command( + registry: CommandRegistry, + *, + decorate: Decorator | None = None, + aliases: dict[str, list[str]] | None = None, +) -> None: + """Register !analityka under every trigger aliases.json gives it.""" + table = load_aliases() if aliases is None else aliases + + async def handler(ctx: CommandContext) -> None: + await ctx.reply(render.render(summarise(ctx, registry.commands))) + + wrapped: Handler = decorate(handler) if decorate else handler + registry.register(COMMAND_NAME, table[COMMAND_NAME], wrapped, DESCRIPTION) diff --git a/signal_bot/art_command.py b/signal_bot/art_command.py index 8ecc09a..a338f05 100644 --- a/signal_bot/art_command.py +++ b/signal_bot/art_command.py @@ -28,6 +28,7 @@ from base64 import b64encode from pathlib import Path from typing import TYPE_CHECKING, Final +from signal_bot.analytics import probe from signal_bot.animals import load_aliases from signal_bot.commons import caption_for, imageinfo_params, page_of from signal_bot.endpoints import THUMB_WIDTH @@ -109,9 +110,14 @@ def art_handler( async def handler(ctx: CommandContext) -> None: # One request, because the caption and the picture have to describe # the same file; two would be a chance for them to disagree. + page_id = random_art(polish, world) + # A Commons page id: public, and an identifier cannot carry a + # name. It is what turns "the art pool feels repetitive" into a + # repeat rate. The caption is fetched and posted, never recorded. + probe.note_item(str(page_id)) document = await fetcher.fetch_json( api_url, - imageinfo_params(THUMB_WIDTH, page_id=random_art(polish, world), describe=True), + imageinfo_params(THUMB_WIDTH, page_id=page_id, describe=True), ) page = page_of(document) image = await fetcher.fetch_image(str(page["imageinfo"][0]["thumburl"])) diff --git a/signal_bot/bot.py b/signal_bot/bot.py index bba06a1..fc3d01f 100644 --- a/signal_bot/bot.py +++ b/signal_bot/bot.py @@ -14,6 +14,7 @@ from collections.abc import Callable from dataclasses import dataclass from typing import TYPE_CHECKING +from signal_bot.analytics_command import register_analytics_command from signal_bot.animals import load_aliases, register_animal_commands from signal_bot.announce import announce_all from signal_bot.art_command import register_art_command @@ -205,6 +206,10 @@ def build_bot( register_art_command(registry, fetcher, config.commons_api_url, decorate=rate_limit) register_poem_command(registry, fetcher, config.poem_text_url, decorate=rate_limit) register_stats_commands(registry, decorate=rate_limit) + # Reads the *command* history rather than the message history, and + # reads the registry when it runs -- so the album's commands, which + # are registered below this line, are still in the near-miss set. + register_analytics_command(registry, decorate=rate_limit) # Rate-limited like every other command, and unlike !add_photo, whose # "NO CAP, NO LIMITS" is a deliberate exception rather than a default. # The cost is real and accepted: somebody told "napisz !zaproponuj ..." diff --git a/signal_bot/context.py b/signal_bot/context.py index 08cca9d..9135717 100644 --- a/signal_bot/context.py +++ b/signal_bot/context.py @@ -12,6 +12,7 @@ from __future__ import annotations from dataclasses import dataclass from typing import TYPE_CHECKING +from signal_bot.analytics.probe import timed_send from signal_bot.quoting import quote_of if TYPE_CHECKING: @@ -45,13 +46,15 @@ class CommandContext: bot watches a test group and a production group at once -- and it quotes the asker, so a busy group can tell which "!cat" this is. """ - await self.transport.send_text(text, self.group_id, quote_of(self.message)) + async with timed_send(): + await self.transport.send_text(text, self.group_id, quote_of(self.message)) async def reply_image(self, image_base64: str, caption: str = "") -> None: """Answer the triggering message with an image, quoting it.""" - await self.transport.send_image( - image_base64, self.group_id, quote_of(self.message), caption - ) + async with timed_send(): + await self.transport.send_image( + image_base64, self.group_id, quote_of(self.message), caption + ) async def reply_media(self, payload_base64: str, media_type: str, filename: str) -> None: """Answer with an attachment that is not necessarily a photo. @@ -60,10 +63,11 @@ class CommandContext: attachment from what it is told it is, and "a jpeg" is a lie that makes a video arrive as a file nobody can play. """ - await self.transport.send_media( - payload_base64, - self.group_id, - quote_of(self.message), - media_type=media_type, - filename=filename, - ) + async with timed_send(): + await self.transport.send_media( + payload_base64, + self.group_id, + quote_of(self.message), + media_type=media_type, + filename=filename, + ) diff --git a/signal_bot/decorators.py b/signal_bot/decorators.py index ea1431d..63aedf0 100644 --- a/signal_bot/decorators.py +++ b/signal_bot/decorators.py @@ -17,7 +17,7 @@ import functools import logging import math from collections.abc import Callable -from typing import TYPE_CHECKING +from typing import TYPE_CHECKING, Final from signal_bot.registry import CommandRefusedError, Handler from signal_bot.stats.text import plural @@ -29,6 +29,15 @@ logger = logging.getLogger(__name__) Decorator = Callable[[Handler], Handler] +# Which gate said no. These are the analytics outcomes minus their prefix, so +# a refusal names the gate that caused it rather than arriving as one +# undifferentiated "refused" -- a command whose cooldown bites half the time +# is mis-tuned, and one whose vote never carries is unwanted, and the raw +# count cannot tell those apart. +GATE_COOLDOWN: Final = "cooldown" +GATE_DAILY: Final = "daily" +GATE_VOTE: Final = "vote" + # Padding on the *displayed* wait only, never on the gate itself. A number that # is a second short sends somebody back for another refusal, which reads as the # bot lying; a number that is a second long costs nothing. @@ -68,7 +77,7 @@ def cooldown(seconds: float, message: str) -> Decorator: unit, ) await ctx.reply(message.format(seconds=shown, unit=unit)) - raise CommandRefusedError + raise CommandRefusedError(GATE_COOLDOWN) await handler(ctx) return wrapper @@ -87,7 +96,7 @@ def daily_limit(max_uses: int, message: str) -> Decorator: ) if used >= max_uses: await ctx.reply(message.format(max_uses=max_uses, used=used)) - raise CommandRefusedError + raise CommandRefusedError(GATE_DAILY) await handler(ctx) return wrapper @@ -109,7 +118,7 @@ def vote_gated(required: int, window_seconds: float, message: str) -> Decorator: votes = ctx.store.count_votes(ctx.group_id, ctx.command_name, ctx.now - window_seconds) if votes < required: await ctx.reply(message.format(votes=votes, required=required)) - raise CommandRefusedError + raise CommandRefusedError(GATE_VOTE) ctx.store.clear_votes(ctx.group_id, ctx.command_name) await handler(ctx) diff --git a/signal_bot/dispatcher.py b/signal_bot/dispatcher.py index 4d07678..8e172db 100644 --- a/signal_bot/dispatcher.py +++ b/signal_bot/dispatcher.py @@ -9,18 +9,19 @@ from collections.abc import Callable from dataclasses import dataclass from typing import TYPE_CHECKING +from signal_bot.analytics import probe, recorder from signal_bot.context import CommandContext from signal_bot.counting import should_count, watched_group from signal_bot.envelope_shape import shape_of from signal_bot.models import parse_envelope -from signal_bot.registry import CommandRefusedError, CommandRegistry -from signal_bot.stats.recorder import record_message +from signal_bot.stats.recorder import record_message, sent_at_ms_of from signal_bot.unknown import answer_unknown if TYPE_CHECKING: from signal_bot.config import Config from signal_bot.discovery import GroupDirectory from signal_bot.models import IncomingMessage + from signal_bot.registry import CommandRegistry from signal_bot.store import Store from signal_bot.transport_base import Transport @@ -139,16 +140,33 @@ class Dispatcher: day=now.date().isoformat(), args=resolved.args if resolved is not None else (), ) - try: + # The envelope's own timestamp, not the clock, and the same value + # record_message keyed its row on -- that shared key is what lets + # the two tables be joined at all. + sent_at_ms = sent_at_ms_of(message, now) + # Two recordings, and they are deliberately not the same write. + # + # The *history* takes every attempt: a refusal and a crash are rows, + # because a command that fails every time used to be indistinguishable + # from one nobody ran. The *use* is success-only and feeds the + # cooldown -- recording a refusal there would push the wait N seconds + # further away every time somebody was told to wait N seconds. + # See analytics.recorder for the invariant and the test that holds it. + with probe.probing() as notes, recorder.attempting(command.name) as attempt: await command.handler(ctx) - except CommandRefusedError: - # A gate said no and already answered. Nothing ran, so nothing is - # recorded -- otherwise every "wait N seconds" reply would push the - # cooldown window N seconds further away and the wait never ended. - logger.debug("command %s refused by a gate", command.name) - return - except Exception: - logger.exception("command %s failed", command.name) + recorder.record_attempt( + self.store.commands, + recorder.Invocation( + group_id=group, + user_uuid=message.source_uuid, + command=command.name, + args=ctx.args, + sent_at_ms=sent_at_ms, + ), + attempt, + notes, + ) + if not attempt.ok: return # Recorded once, here, so stacked gates cannot double-count a use. self.store.record_use(group, message.source_uuid, command.name, ctx.now, ctx.day) diff --git a/signal_bot/help_command.py b/signal_bot/help_command.py index 3aa0247..0ec8789 100644 --- a/signal_bot/help_command.py +++ b/signal_bot/help_command.py @@ -54,6 +54,7 @@ GROUPS: Final[tuple[tuple[str, tuple[str, ...]], ...]] = ( "stats_crown", "stats_weekdays", "stats_me", + "command_stats", ), ), ) diff --git a/signal_bot/http_client.py b/signal_bot/http_client.py index 2c09ba0..0c184ab 100644 --- a/signal_bot/http_client.py +++ b/signal_bot/http_client.py @@ -8,12 +8,18 @@ and the offline environment can serve fixtures without the bot knowing. from __future__ import annotations import base64 +import time from typing import TYPE_CHECKING, Final +import httpx + +from signal_bot.analytics import probe + if TYPE_CHECKING: from collections.abc import Sequence - import httpx + +MS_PER_SECOND: Final = 1000 class ImageFetchError(RuntimeError): @@ -70,6 +76,35 @@ class ImageFetcher: image_url = image_url.replace(*rewrite) return await self.fetch_image(image_url) + async def _get(self, url: str, params: dict[str, str] | None = None) -> httpx.Response: + """Make one request, time it, and report it to whatever is probing. + + The single place this class touches the network, which is what makes + "how often does thecatapi fail, and how slow is Commons" answerable + without a change to any command. ``probe.note_upstream`` is a no-op + when nothing is measuring, so the offline runs and the tests are + unaffected. + + Only the hostname is reported. A URL carries identifiers -- the poem + one carries a Wolne Lektury slug -- and identifiers belong in + ``item_id``, where a deliberate decision governs them. + + A request that never got an answer is recorded as NO_RESPONSE and + re-raised unchanged: DNS, connect and timeout failures are the ones + that never reach a status code, and losing them would make an + upstream that is entirely down look merely unused. + """ + started = time.perf_counter() + host = httpx.URL(url).host + try: + response = await self._client.get(url, params=params) + except httpx.HTTPError: + probe.note_upstream(host, probe.NO_RESPONSE, _elapsed_ms(started)) + raise + probe.note_upstream(host, response.status_code, _elapsed_ms(started)) + response.raise_for_status() + return response + async def fetch_json(self, api_url: str, params: dict[str, str] | None = None) -> object: """Fetch one JSON document from an API. @@ -78,9 +113,7 @@ class ImageFetcher: asking twice would be both wasteful and a chance for the two halves to disagree. """ - response = await self._client.get(api_url, params=params) - response.raise_for_status() - return response.json() + return (await self._get(api_url, params)).json() async def fetch_text(self, url: str) -> str: """Fetch one plain-text document. @@ -90,9 +123,7 @@ class ImageFetcher: through this client like everything else, so the offline environment and the tests serve it the same way they serve a cat. """ - response = await self._client.get(url) - response.raise_for_status() - return response.text + return (await self._get(url)).text async def fetch_image(self, image_url: str) -> bytes: """Fetch the bytes at an already-resolved image URL. @@ -103,9 +134,12 @@ class ImageFetcher: traversal and unbounded. portraits.py does cache to disk, but under a name it derives from the pope's ordinal, never from upstream. """ - image = await self._client.get(image_url) - image.raise_for_status() - return image.content + return (await self._get(image_url)).content + + +def _elapsed_ms(started: float) -> int: + """Milliseconds since a perf_counter reading.""" + return round((time.perf_counter() - started) * MS_PER_SECOND) _MISSING: Final = object() diff --git a/signal_bot/manage/analytics_panels.py b/signal_bot/manage/analytics_panels.py new file mode 100644 index 0000000..1269354 --- /dev/null +++ b/signal_bot/manage/analytics_panels.py @@ -0,0 +1,246 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""One gatherer per panel on the Analytics tab. + +Split out of ``analytics_read`` so both stay under the 250-line cap. That +module owns the connection and the shapes; this one owns the six questions, +one function each, mirroring the six query modules exactly so that a panel +and its query are never more than one hop apart. + +Every response carries ``window``. The bot is days old, not months, and a +month panel drawn from three days is only honest if the reader can see how +many days it is. +""" + +from __future__ import annotations + +from typing import TYPE_CHECKING + +from signal_bot.analytics import ( + adoption, + content, + demand, + health, + hours, + sessions, + trend, + usage, +) +from signal_bot.analytics.scope import window +from signal_bot.manage.analytics_read import ( + Gathered, + empty, + measure, + names_in, + safely, +) +from signal_bot.stats.periods import today + +if TYPE_CHECKING: + import sqlite3 + from pathlib import Path + + from signal_bot.analytics.scope import Scope + +# How far the retention curve is drawn. Two weeks: long enough to show a +# drop-off, short enough that most of it is not empty on a young bot. +RETENTION_DAYS = 14 +# The window either side of a release that "what did this change" compares. +RELEASE_SPAN_DAYS = 3 + + +def overview(path: Path, scope: Scope) -> Gathered: + """Family A: what is used, by whom, and what has never been touched.""" + + def gather(conn: sqlite3.Connection) -> Gathered: + names = names_in(conn, scope) + uses = usage.by_command(conn, scope) + return { + "window": measure(conn, scope), + "totals": usage.totals(conn, scope), + "commands": [ + { + "command": use.command, + "uses": use.uses, + "people": use.people, + "ok": use.ok, + "refused": use.refused, + "errors": use.errors, + "first_day": use.first_day, + "last_day": use.last_day, + } + for use in uses + ], + "people": [ + { + "uuid": person.uuid, + "name": names.get(person.uuid, ""), + "uses": person.uses, + "commands": person.commands, + "favourite": person.favourite, + } + for person in usage.by_person(conn, scope) + ], + "matrix": [ + {"uuid": uuid, "name": names.get(uuid, ""), "command": command, "uses": count} + for uuid, command, count in usage.matrix(conn, scope) + ], + } + + return safely(path, gather) or {**empty("commands", "people", "matrix"), "totals": {}} + + +def times(path: Path, scope: Scope) -> Gathered: + """Family B: hours, weekdays, days, months and what has gone quiet.""" + + def gather(conn: sqlite3.Connection) -> Gathered: + series = trend.dense_days(trend.by_day(conn, scope)) + measured = window(conn, scope) + peak = hours.peak(hours.by_hour(conn, scope)) + return { + "window": measure(conn, scope), + "hours": hours.by_hour(conn, scope), + "weekday_hours": hours.by_weekday_hour(conn, scope), + "by_command_hour": hours.by_command_hour(conn, scope), + "days": [{"day": day, "uses": uses} for day, uses in series], + "months": [ + {"month": month, "uses": uses, "days": days} + for month, uses, days in trend.by_month(conn, scope) + ], + "peak": { + "busiest": peak.busiest, + "busiest_uses": peak.busiest_uses, + "quietest": peak.quietest, + "quietest_uses": peak.quietest_uses, + }, + "direction": trend.direction(series, measured), + "dying": [ + {"command": one.command, "last_day": one.last_day, "idle_days": one.idle_days} + for one in trend.dying(conn, scope, today()) + ], + } + + return safely(path, gather) or empty("hours", "days", "months", "dying") + + +def wellbeing(path: Path, scope: Scope) -> Gathered: + """Family C: what is slow, what refuses people, and what upstream does.""" + + def gather(conn: sqlite3.Connection) -> Gathered: + return { + "window": measure(conn, scope), + "latencies": [ + { + "command": one.command, + "samples": one.samples, + "handler_p50": one.handler_p50, + "handler_p95": one.handler_p95, + "send_p50": one.send_p50, + "upstream_p50": one.upstream_p50, + } + for one in health.latencies(conn, scope) + ], + "gates": [ + {"command": command, "gate": gate, "hits": hits} + for command, gate, hits in health.gates(conn, scope) + ], + "errors": [ + {"command": command, "error_class": name, "hits": hits} + for command, name, hits in health.errors(conn, scope) + ], + "upstream": [ + { + "host": one.host, + "calls": one.calls, + "failures": one.failures, + "p50_ms": one.p50_ms, + } + for one in health.upstreams(conn, scope) + ], + } + + return safely(path, gather) or empty("latencies", "gates", "errors", "upstream") + + +def wishes(path: Path, scope: Scope, triggers: tuple[str, ...]) -> Gathered: + """Family D: what people nearly typed, grouped by what they nearly hit.""" + + def gather(conn: sqlite3.Connection) -> Gathered: + found = demand.collect(conn, scope, triggers) + return { + "window": measure(conn, scope), + "near_misses": [ + {"target": miss.target, "hits": miss.hits, "variants": miss.variants} + for miss in found.near_misses + ], + "unclaimed": found.unclaimed, + "unclaimed_words": found.unclaimed_words, + "filed": found.filed, + } + + return safely(path, gather) or empty("near_misses") + + +def behaviour(path: Path, scope: Scope, releases: tuple[tuple[str, str], ...]) -> Gathered: + """Family E: sittings, retention, releases and share of chat.""" + + def gather(conn: sqlite3.Connection) -> Gathered: + series = trend.dense_days(trend.by_day(conn, scope)) + sitting = sessions.collect(conn, scope) + share = adoption.share_of_chat(conn, scope) + return { + "window": measure(conn, scope), + "sessions": { + "count": sitting.count, + "commands": sitting.commands, + "longest": sitting.longest, + "average_length": sitting.average_length, + }, + "pairs": [ + {"left": left, "right": right, "hits": hits} for left, right, hits in sitting.pairs + ], + "repeats": [{"command": command, "extra": extra} for command, extra in sitting.repeats], + "retention": [ + {"offset": offset, "active": active, "cohort": cohort} + for offset, active, cohort in adoption.retention(conn, scope, RETENTION_DAYS) + ], + "newcomers": [ + {"day": day, "people": people} for day, people in adoption.newcomers(conn, scope) + ], + "releases": [ + {"version": version, "day": day, "after": after, "before": before} + for version, day, after, before in adoption.around_releases( + series, releases, RELEASE_SPAN_DAYS + ) + ], + "share": { + "messages": share.messages, + "commands": share.commands, + "fraction": share.fraction, + }, + } + + return safely(path, gather) or empty("pairs", "repeats", "retention", "newcomers", "releases") + + +def items(path: Path, scope: Scope) -> Gathered: + """Family F: how varied each pool actually is.""" + + def gather(conn: sqlite3.Connection) -> Gathered: + return { + "window": measure(conn, scope), + "pools": [ + { + "command": pool.command, + "draws": pool.draws, + "distinct": pool.distinct, + "repeat_rate": pool.repeat_rate, + } + for pool in content.pools(conn, scope) + ], + "repeats": [ + {"command": command, "item_id": item, "draws": draws} + for command, item, draws in content.repeats(conn, scope) + ], + } + + return safely(path, gather) or empty("pools", "repeats") diff --git a/signal_bot/manage/analytics_read.py b/signal_bot/manage/analytics_read.py new file mode 100644 index 0000000..a0881fc --- /dev/null +++ b/signal_bot/manage/analytics_read.py @@ -0,0 +1,136 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""Reading the command history from the *other* process. + +The bot holds the database open and commits on every inbound message, so +this opens its own read-only connection and closes it explicitly -- the +pattern ``routes_suggestions.collect`` established, and for the same reason: +a reader left dangling is a lock waiting to happen, and ``Store`` on a plain +connection would apply its schema against a running bot. + +Every gatherer is wrapped the same way. A bot that has never run, a database +older than these tables and a half-written file are all the same answer, and +none of them is worth breaking the page over -- the page is most useful +exactly when something is wrong. +""" + +from __future__ import annotations + +import contextlib +import sqlite3 +from typing import TYPE_CHECKING, Any + +from signal_bot.analytics.scope import Scope, window + +if TYPE_CHECKING: + from collections.abc import Callable, Iterator + from pathlib import Path + +Gathered = dict[str, Any] + +GROUPS_SQL = """ +SELECT group_id, + COUNT(*) AS events, + MIN(day) AS first_day, + MAX(day) AS last_day +FROM command_events +GROUP BY group_id +ORDER BY events DESC +""" + +# The names shown beside a uuid on the page. Unlike the group chat, this is +# a loopback page the maintainer is looking at, so everybody who has a name +# gets one -- but a uuid still never appears without one beside it. +NAMES_SQL = """ +SELECT uuid, display_name +FROM group_people +WHERE (:group IS NULL OR group_id = :group) +""" + + +@contextlib.contextmanager +def reading(path: Path) -> Iterator[sqlite3.Connection | None]: + """A read-only connection, or None when there is nothing readable.""" + if not path.is_file(): + yield None + return + try: + with contextlib.closing(sqlite3.connect(f"file:{path}?mode=ro", uri=True)) as conn: + conn.row_factory = sqlite3.Row + yield conn + except sqlite3.Error: + yield None + + +def safely(path: Path, gather: Callable[[sqlite3.Connection], Gathered]) -> Gathered: + """Run one gatherer, answering with the empty shape if anything is wrong.""" + with reading(path) as conn: + if conn is None: + return {} + try: + return gather(conn) + except sqlite3.Error: + return {} + + +def measure(conn: sqlite3.Connection, scope: Scope) -> Gathered: + """The days covered and where the rows came from, on every response.""" + measured = window(conn, scope) + return { + "first_day": measured.first_day, + "last_day": measured.last_day, + "days": measured.days, + "events": measured.events, + "enough_for_a_trend": measured.enough_for_a_trend, + "sources": { + "live": measured.live, + "journald": measured.journald, + "daily_uses": measured.daily, + }, + } + + +def names_in(conn: sqlite3.Connection, scope: Scope) -> dict[str, str]: + """Map each uuid to a display name, so no bare identifier is shown.""" + return { + str(row["uuid"]): str(row["display_name"]) + for row in conn.execute(NAMES_SQL, scope.params()) + } + + +def groups(path: Path) -> Gathered: + """Every group with a history, for the picker.""" + + def gather(conn: sqlite3.Connection) -> Gathered: + return { + "groups": [ + { + "id": str(row["group_id"]), + "events": int(row["events"]), + "first_day": str(row["first_day"]), + "last_day": str(row["last_day"]), + } + for row in conn.execute(GROUPS_SQL) + ] + } + + return safely(path, gather) or {"groups": []} + + +def empty(*lists: str) -> Gathered: + """The shape every panel gets when there is nothing readable. + + Named keys with empty values rather than an error, so the page renders + its "nothing yet" state instead of a stack trace -- and so the frontend + needs no branch for a database that does not exist. + """ + return {"window": EMPTY_WINDOW, **{name: [] for name in lists}} + + +EMPTY_WINDOW: Gathered = { + "first_day": "", + "last_day": "", + "days": 0, + "events": 0, + "enough_for_a_trend": False, + "sources": {"live": 0, "journald": 0, "daily_uses": 0}, +} diff --git a/signal_bot/manage/app.py b/signal_bot/manage/app.py index cba9dba..69df394 100644 --- a/signal_bot/manage/app.py +++ b/signal_bot/manage/app.py @@ -23,6 +23,7 @@ from signal_bot.manage import ( guard, routes_actions, routes_album, + routes_analytics, routes_env, routes_status, routes_suggestions, @@ -51,5 +52,6 @@ def build_app(deps: ManageDeps, *, container: str = DEFAULT_CONTAINER) -> FastAP app.include_router(routes_actions.router(deps, container)) app.include_router(routes_album.router(deps)) app.include_router(routes_suggestions.router(deps)) + app.include_router(routes_analytics.router(deps)) assets.install(app, deps.dist_dir) return app diff --git a/signal_bot/manage/routes_analytics.py b/signal_bot/manage/routes_analytics.py new file mode 100644 index 0000000..66467a9 --- /dev/null +++ b/signal_bot/manage/routes_analytics.py @@ -0,0 +1,110 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""The Analytics tab's data, one route per panel. + +Read-only, every one of them. This page can already rewrite ``.env`` and +restart a container; the command history is the one thing on it that is +worth nothing to an attacker and everything to a maintainer, and there is no +reason for a write route to exist here at all. + +``?group=`` is optional everywhere. Absent means the all-groups roll-up, and +because ``Scope`` binds it as a nullable parameter the roll-up runs the same +SQL as the per-group answer rather than a second query that would drift. +""" + +from __future__ import annotations + +from typing import TYPE_CHECKING, Annotated + +from fastapi import APIRouter, Query + +from signal_bot.analytics.scope import Scope +from signal_bot.animals import load_aliases +from signal_bot.external import run +from signal_bot.manage import analytics_panels, analytics_read + +if TYPE_CHECKING: + from signal_bot.manage.deps import ManageDeps + +# Where the release tags live. Named so the shell-out reads as one thing. +TAGS = "refs/tags" + +# Routes are annotated ``-> analytics_read.Gathered`` rather than ``-> +# Gathered``. FastAPI resolves a return annotation through pydantic when the +# app is built, and ``from __future__ import annotations`` makes that a +# lookup in module globals -- where a name imported only under TYPE_CHECKING +# does not exist. Going through the module, which is imported at runtime +# anyway, keeps the annotation resolvable and the import honest. + + +# What the near-miss grouping is matched against. Read from the alias file +# rather than from a live registry: this is a second process with no bot in +# it, and the file is the same data the bot registers from. +def triggers() -> tuple[str, ...]: + """Every trigger any command claims, for grouping near misses.""" + return tuple(trigger for triggers in load_aliases().values() for trigger in triggers) + + +def releases() -> tuple[tuple[str, str], ...]: + """(version, day) for every release, taken from the git tags. + + Not from ``DOCS-changelog.md``, which was the obvious place to look and + turns out not to work: the changelog is the authority on *what* shipped + and carries no dates at all -- its headings are ``## — `` + and nothing more. The tags are the only record of *when*. + + Shelled out through ``signal_bot.external``, which never raises: a + checkout without tags, or a deployment that is not a git tree, comes back + as no releases and the timeline simply draws without annotations. + """ + result = run(["git", "for-each-ref", "--format=%(refname:short) %(creatordate:short)", TAGS]) + if not result.ok: + return () + found = [] + for line in result.stdout.splitlines(): + name, _, day = line.partition(" ") + if name.startswith("v") and day: + found.append((name.removeprefix("v"), day.strip())) + return tuple(sorted(found, key=lambda pair: pair[1])) + + +def router(deps: ManageDeps) -> APIRouter: + """Seven read-only routes, all under /api/analytics.""" + api = APIRouter(prefix="/api/analytics") + path = deps.database_path + + @api.get("/groups") + def groups() -> analytics_read.Gathered: + """Every group with a history, for the picker.""" + return analytics_read.groups(path) + + @api.get("/overview") + def overview(group: Annotated[str | None, Query()] = None) -> analytics_read.Gathered: + """Family A: what is used, by whom, and what is untouched.""" + return analytics_panels.overview(path, Scope(group)) + + @api.get("/time") + def times(group: Annotated[str | None, Query()] = None) -> analytics_read.Gathered: + """Family B: hours, weekdays, days, months, and what has gone quiet.""" + return analytics_panels.times(path, Scope(group)) + + @api.get("/health") + def wellbeing(group: Annotated[str | None, Query()] = None) -> analytics_read.Gathered: + """Family C: latency, refusals, errors and upstream behaviour.""" + return analytics_panels.wellbeing(path, Scope(group)) + + @api.get("/demand") + def wishes(group: Annotated[str | None, Query()] = None) -> analytics_read.Gathered: + """Family D: near misses, grouped by the trigger they nearly hit.""" + return analytics_panels.wishes(path, Scope(group), triggers()) + + @api.get("/behaviour") + def behaviour(group: Annotated[str | None, Query()] = None) -> analytics_read.Gathered: + """Family E: sittings, retention, releases and share of chat.""" + return analytics_panels.behaviour(path, Scope(group), releases()) + + @api.get("/content") + def items(group: Annotated[str | None, Query()] = None) -> analytics_read.Gathered: + """Family F: how varied each pool actually is.""" + return analytics_panels.items(path, Scope(group)) + + return api diff --git a/signal_bot/pack_command.py b/signal_bot/pack_command.py index 6a7795a..f237f62 100644 --- a/signal_bot/pack_command.py +++ b/signal_bot/pack_command.py @@ -20,6 +20,7 @@ import base64 import secrets from typing import TYPE_CHECKING, Final +from signal_bot.analytics import probe from signal_bot.pack import candidate_keysets, media_type if TYPE_CHECKING: @@ -85,6 +86,11 @@ def pack_handler(holder: PackHolder, entry: Entry) -> Handler: await ctx.reply(NO_PHOTO) return filename = pick(pack.matching(pick_keys(viable))) + # The stored name is content-addressed -- pack_store.content_name is + # the first 128 bits of the file's SHA-256 -- so it identifies the + # photograph without describing it or anyone in it. That is what + # makes "which photo keeps coming back" answerable at all. + probe.note_item(filename) payload = base64.b64encode(pack.media_path(filename).read_bytes()).decode("ascii") await ctx.reply_media(payload, media_type(filename), filename) diff --git a/signal_bot/pack_random.py b/signal_bot/pack_random.py index 0198163..f24a974 100644 --- a/signal_bot/pack_random.py +++ b/signal_bot/pack_random.py @@ -20,6 +20,7 @@ from __future__ import annotations import base64 from typing import TYPE_CHECKING, Final +from signal_bot.analytics import probe from signal_bot.pack import MAX_NAMES, media_type from signal_bot.pack_command import NAME_PREFIX, NO_PHOTO, UNKNOWN, pick @@ -67,6 +68,8 @@ def random_handler(holder: PackHolder) -> Handler: await ctx.reply(NO_PHOTO) return filename = pick(viable) + # Content-addressed, for the reason pack_command records one. + probe.note_item(filename) payload = base64.b64encode(pack.media_path(filename).read_bytes()).decode("ascii") await ctx.reply_media(payload, media_type(filename), filename) diff --git a/signal_bot/poem_command.py b/signal_bot/poem_command.py index 7346910..5c05d7a 100644 --- a/signal_bot/poem_command.py +++ b/signal_bot/poem_command.py @@ -41,6 +41,7 @@ import secrets from pathlib import Path from typing import TYPE_CHECKING, Final +from signal_bot.analytics import probe from signal_bot.animals import load_aliases from signal_bot.wolnelektury import CATALOGUE, parse @@ -133,6 +134,10 @@ def poem_handler( async def handler(ctx: CommandContext) -> None: slug = random_slug(pool) + # The Wolne Lektury slug, which is what the pool holds and all + # it holds -- the poem, its title and its author arrive over the + # wire and are never written down. See DOCS-privacy.md. + probe.note_item(slug) poem = parse(await fetcher.fetch_text(text_url.format(slug=slug))) if poem is None: msg = f"wolnelektury.pl no longer serves a usable poem at {slug!r}" diff --git a/signal_bot/popes_command.py b/signal_bot/popes_command.py index f4fe1a6..e02bcde 100644 --- a/signal_bot/popes_command.py +++ b/signal_bot/popes_command.py @@ -10,6 +10,7 @@ from __future__ import annotations import secrets from typing import TYPE_CHECKING, Final +from signal_bot.analytics import probe from signal_bot.animals import load_aliases from signal_bot.versioning import POPES, article_url, pope_line @@ -44,6 +45,9 @@ def pope_handler(cache: PortraitCache) -> Handler: async def handler(ctx: CommandContext) -> None: pope = random_pope() + # The ordinal, not the name: it is the pope's identifier here + # and in popes.json, and the caption is built at reply time. + probe.note_item(str(pope.ordinal)) image = await cache.get_base64(pope) await ctx.reply_image(image, caption_for(pope)) diff --git a/signal_bot/registry.py b/signal_bot/registry.py index d3462c5..88bb319 100644 --- a/signal_bot/registry.py +++ b/signal_bot/registry.py @@ -74,14 +74,29 @@ class DuplicateTriggerError(ValueError): class CommandRefusedError(Exception): - """A gate declined the command, so nothing happened and nothing is recorded. + """A gate declined the command, so nothing happened and no *use* is recorded. The dispatcher records one use per invocation, and it cannot see inside a handler. Without this signal a *refused* run looked exactly like a successful one, which slid the cooldown window forward every time somebody was told to wait -- so waiting the stated time never helped. + + The refusal is now recorded in the command *history*, which is a different + table and a different question: how often a gate actually bites is what + says whether it is tuned right. ``gate`` names which one refused, because + "this command is refused 40% of the time" and "this command is popular" + are the two readings of the same raw count. + + It defaults to empty so that raising it bare stays legal -- every existing + raise site said only "no", and a required argument would turn a behaviour + that works into a signature change across three decorators. """ + def __init__(self, gate: str = "") -> None: + """Record which gate refused, if the caller says.""" + super().__init__(gate) + self.gate = gate + @dataclass(frozen=True, slots=True) class Resolved: diff --git a/signal_bot/schema.py b/signal_bot/schema.py index 1b43ea4..e48c88a 100644 --- a/signal_bot/schema.py +++ b/signal_bot/schema.py @@ -100,28 +100,81 @@ CREATE TABLE IF NOT EXISTS suggestions ( ); """ -# Cleared when the bot leaves a group, and rebuilt on a schema-version bump. -# Losing any of these costs a cooldown or a vote in progress. -EPHEMERAL_TABLES: Final = ( +# These were one tuple, EPHEMERAL_TABLES, until command analytics needed the +# two jobs it was doing to come apart. "Dropped when the schema changes" and +# "cleared when the bot leaves a group" happened to name the same four tables, +# so one constant served both -- and then daily_uses became a source the +# analytics history is backfilled from, which must survive a schema bump but +# must still reset a rejoined group's daily limits. Two questions, two tuples. + +# Dropped and recreated when SCHEMA_VERSION rises. Only tables whose contents +# cost nothing to lose: a vote in progress, a per-group marker. +# +# The price of taking command_uses/daily_uses out of here is explicit: if +# their *shape* ever changes, CREATE TABLE IF NOT EXISTS is a no-op against +# the old one and every INSERT starts failing. Reshaping either of them now +# needs a written migration rather than a version bump. That is the trade +# that keeps the use history. +REBUILT_ON_BUMP: Final = ( + "votes", + "group_meta", +) +# Rows deleted for one group when the bot is removed from it, so that being +# added back starts with a clean cooldown rather than resurrecting whatever +# was true whenever it was last in there. +CLEARED_ON_LEAVE: Final = ( "command_uses", "daily_uses", "votes", "group_meta", ) -# Never dropped, by either path. These hold the whole statistical history, -# which cannot be recomputed from anything -- so a transient group listing -# that omits a member (the exact failure discovery._permitted guards against) -# must not be able to delete years of it, and neither must a future schema -# bump that only meant to reshape a cooldown table. +# Never dropped and never cleared, by any path. These hold the whole +# statistical history, which cannot be recomputed from anything -- so a +# transient group listing that omits a member (the exact failure +# discovery._permitted guards against) must not be able to delete years of +# it, and neither must a future schema bump that only meant to reshape a +# cooldown table. PERMANENT_TABLES: Final = ( "message_events", "group_people", # What the group wished existed. Neither can be recomputed from anything # -- an idea somebody typed once is gone the moment it is dropped -- and - # forget_group() clears the ephemeral set on the transient group-listing - # gap discovery._permitted guards against, which must not cost a year of + # forget_group() clears its own set on the transient group-listing gap + # discovery._permitted guards against, which must not cost a year of # collected suggestions. "unknown_commands", "suggestions", + # Every command attempt ever made, and every upstream call made serving + # one. In neither tuple above: a schema bump must not throw away the + # answer to "which command is dying", and leaving a group must not + # either -- message_events already survives both for the same reason. + "command_events", + "upstream_calls", ) -TABLES: Final = EPHEMERAL_TABLES + PERMANENT_TABLES +# Every table the schema declares. command_uses and daily_uses are in +# CLEARED_ON_LEAVE but not PERMANENT_TABLES, so the union is spelled out +# rather than concatenated -- test_schema_tables checks it against sqlite's +# own catalogue, which is what keeps this honest. +TABLES: Final = ( + "command_uses", + "daily_uses", + *REBUILT_ON_BUMP, + *PERMANENT_TABLES, +) + +# The two tables that carry history and are NOT rebuilt on a version bump. +# Because they survive, a bump can no longer fix them if their shape is +# wrong, and CREATE TABLE IF NOT EXISTS will not either -- which is the +# failure _apply_schema exists to prevent: a bot that boots clean and then +# fails on every INSERT. So their columns are declared here and checked, and +# a mismatch drops the table after all. Losing a cooldown is recoverable; +# a bot that answers nothing is not. +# +# This is the "written migration" the comment on REBUILT_ON_BUMP promises. +# A deliberate reshape means editing the tuple below in the same commit, and +# accepting that the old rows go. test_declared_columns_match_the_schema +# asserts these against what sqlite actually creates, so they cannot drift. +RETAINED_COLUMNS: Final = { + "command_uses": ("group_id", "user_uuid", "command", "used_at"), + "daily_uses": ("group_id", "user_uuid", "command", "day", "uses"), +} diff --git a/signal_bot/schema_commands.py b/signal_bot/schema_commands.py new file mode 100644 index 0000000..3058fff --- /dev/null +++ b/signal_bot/schema_commands.py @@ -0,0 +1,78 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""The command history's own tables, kept apart from the rest of the schema. + +Split out of ``schema.py`` so both stay under the 250-line cap, the same +reason ``schema.py`` was split out of ``store.py``. It is applied straight +after ``SCHEMA`` and by the same method, so this is a filing decision and +not a second schema: there is one database and one ``PRAGMA user_version``. + +Neither table is ever dropped -- see ``PERMANENT_TABLES``. The whole point +of them is answering questions about a past that nothing can recompute. +""" + +from __future__ import annotations + +from typing import Final + +COMMAND_SCHEMA: Final = """ +-- One row per command *attempt*, including the ones that never ran. The two +-- tables above it are aggregates a rate limiter reads; this is the history, +-- and nothing recomputes it. +-- +-- Keyed on the envelope's own timestamp for the reason unknown_commands is: +-- signal-cli replays undelivered frames on reconnect, and a clock reading is +-- fresh on every replay. invoked_at_ms is message.timestamp, so a replay +-- collides with the row it already wrote instead of adding a second one. +-- +-- source is part of the key because a backfilled row and a live row can +-- describe the same invocation from two sources that disagree about the +-- hour, and neither may overwrite the other. +CREATE TABLE IF NOT EXISTS command_events ( + group_id TEXT NOT NULL, + user_uuid TEXT NOT NULL, + command TEXT NOT NULL, + -- Never free text. A pack pseudonym key the registry already resolved + -- ('p1', 'add', 'random'), or '' / '+args'. See registry.resolve_with_args. + args TEXT NOT NULL DEFAULT '', + -- A public identifier for what was served -- a Wikidata QID, a Commons + -- page id, a Wolne Lektury slug, a content-addressed filename. Never a + -- title, a caption or a name: an identifier cannot carry one. + item_id TEXT, + outcome TEXT NOT NULL, + -- The exception's class name when outcome is 'error', '' otherwise. + -- A name, never a message: an exception's str() carries whatever was + -- being processed, which for this bot is somebody's message. + error_class TEXT NOT NULL DEFAULT '', + invoked_at_ms INTEGER NOT NULL, + day TEXT NOT NULL, + local_hour INTEGER NOT NULL, + weekday INTEGER NOT NULL, + handler_ms INTEGER, + send_ms INTEGER, + source TEXT NOT NULL DEFAULT 'live', + PRIMARY KEY (group_id, user_uuid, command, invoked_at_ms, source) +); +CREATE INDEX IF NOT EXISTS idx_command_events_group_day + ON command_events (group_id, day); +CREATE INDEX IF NOT EXISTS idx_command_events_group_command_day + ON command_events (group_id, command, day); +CREATE INDEX IF NOT EXISTS idx_command_events_group_user_day + ON command_events (group_id, user_uuid, day); +-- One row per outbound HTTP call made while a command was running, joined to +-- command_events by (group_id, invoked_at_ms). seq orders the calls within +-- one invocation and keeps two calls to the same host distinct. +CREATE TABLE IF NOT EXISTS upstream_calls ( + group_id TEXT NOT NULL, + invoked_at_ms INTEGER NOT NULL, + command TEXT NOT NULL, + host TEXT NOT NULL, + -- 0 means the request never got a reply at all: DNS, connect, timeout. + -- Distinguishable from a 5xx, which is upstream answering badly. + status INTEGER NOT NULL, + duration_ms INTEGER NOT NULL, + seq INTEGER NOT NULL, + PRIMARY KEY (group_id, invoked_at_ms, seq) +); +CREATE INDEX IF NOT EXISTS idx_upstream_calls_host + ON upstream_calls (group_id, host); +""" diff --git a/signal_bot/stats/recorder.py b/signal_bot/stats/recorder.py index 3bb7a56..5c5e048 100644 --- a/signal_bot/stats/recorder.py +++ b/signal_bot/stats/recorder.py @@ -16,6 +16,24 @@ if TYPE_CHECKING: from signal_bot.stats.events import EventStore +def sent_at_ms_of(message: IncomingMessage, now: dt.datetime) -> int: + """When a message was sent, in milliseconds, for use as a key. + + The timestamp falls back to the clock because ``models._timestamp`` + returns 0 when neither the payload nor the envelope carried one. Zero is + not merely inaccurate: it is part of the primary key, so every + zero-timestamp frame from one sender would collide and the whole run + would be recorded as a single message. + + One function rather than the same expression in two places, because + ``message_events`` and ``command_events`` are joined on exactly this + value -- to say which command a counted message was, and to attach an + hour to a backfilled row. Two spellings that drifted would not fail + anything; they would quietly stop matching. + """ + return message.timestamp or int(now.timestamp() * MS_PER_SECOND) + + def to_event( group_id: str, message: IncomingMessage, @@ -23,15 +41,8 @@ def to_event( *, is_command: bool, ) -> MessageEvent: - """Reduce a message to the metadata the statistics keep. - - The timestamp falls back to the clock because ``models._timestamp`` - returns 0 when neither the payload nor the envelope carried one. Zero is - not merely inaccurate: it is part of the primary key, so every - zero-timestamp frame from one sender would collide and the whole run - would be recorded as a single message. - """ - sent_at_ms = message.timestamp or int(now.timestamp() * MS_PER_SECOND) + """Reduce a message to the metadata the statistics keep.""" + sent_at_ms = sent_at_ms_of(message, now) return MessageEvent( group_id=group_id, uuid=message.source_uuid, diff --git a/signal_bot/store.py b/signal_bot/store.py index 35fbf8f..090cddc 100644 --- a/signal_bot/store.py +++ b/signal_bot/store.py @@ -14,10 +14,13 @@ recomputation can bring back once dropped. from __future__ import annotations import contextlib +import logging import sqlite3 from typing import TYPE_CHECKING, Self -from signal_bot.schema import EPHEMERAL_TABLES, SCHEMA, SCHEMA_VERSION +from signal_bot.analytics.uses import CommandEventStore +from signal_bot.schema import REBUILT_ON_BUMP, RETAINED_COLUMNS, SCHEMA, SCHEMA_VERSION +from signal_bot.schema_commands import COMMAND_SCHEMA from signal_bot.stats.events import EventStore from signal_bot.suggestions import SuggestionStore @@ -25,6 +28,8 @@ if TYPE_CHECKING: from collections.abc import Iterator from types import TracebackType +logger = logging.getLogger(__name__) + class Store: """Thin, synchronous SQLite wrapper. All state the bot keeps lives here.""" @@ -36,6 +41,7 @@ class Store: self._apply_schema() self.events = EventStore(self._conn) self.suggestions = SuggestionStore(self._conn) + self.commands = CommandEventStore(self._conn) def _apply_schema(self) -> None: """Create the tables, rebuilding them when the file predates them. @@ -53,15 +59,51 @@ class Store: """ version = int(self._conn.execute("PRAGMA user_version").fetchone()[0]) if version < SCHEMA_VERSION: - # EPHEMERAL_TABLES, not TABLES. Rebuilding a cooldown table is + # REBUILT_ON_BUMP, not TABLES. Rebuilding a vote in progress is # free; rebuilding message_events would throw away the entire - # statistical history, which nothing can recompute. - for table in EPHEMERAL_TABLES: + # statistical history, which nothing can recompute. command_uses + # and daily_uses used to be in here and are deliberately not any + # more -- see the comment on the tuple in schema.py. + for table in REBUILT_ON_BUMP: self._conn.execute(f"DROP TABLE IF EXISTS {table}") + self._drop_reshaped() self._conn.executescript(SCHEMA) + # A separate string, not a separate schema: one database, one + # user_version. Split only so both files stay under the cap. + self._conn.executescript(COMMAND_SCHEMA) self._conn.execute(f"PRAGMA user_version = {SCHEMA_VERSION}") self._conn.commit() + def _drop_reshaped(self) -> None: + """Drop a retained table whose columns no longer match the schema. + + command_uses and daily_uses survive a version bump so that the use + history survives with them, which means a bump can no longer repair + them. A database written before the group key still holds the old + two-column command_uses, and CREATE TABLE IF NOT EXISTS is a no-op + against it -- so without this the bot boots clean and then fails on + every INSERT, which is the exact failure the version check exists to + prevent. + + Keeping rows is worth a lot; keeping them at the price of a bot that + answers nothing is not. A mismatch is logged, because losing a + cooldown silently is fine and losing history silently is not. + """ + for table, expected in RETAINED_COLUMNS.items(): + # PRAGMA takes no parameters, and the name is a module constant + # walked here rather than anything a message could reach. + found = tuple( + str(row["name"]) for row in self._conn.execute(f"PRAGMA table_info({table})") + ) + if found and found != expected: + logger.warning( + "%s has columns %s, expected %s; dropping it to stay writable", + table, + found, + expected, + ) + self._conn.execute(f"DROP TABLE IF EXISTS {table}") + def __enter__(self) -> Self: """Enter a context manager that closes the connection on exit.""" return self @@ -80,7 +122,7 @@ class Store: self._conn.close() def forget_group(self, group_id: str) -> None: - """Drop one group's *ephemeral* rows: cooldowns, limits, votes, meta. + """Drop one group's live scaffolding: cooldowns, limits, votes, meta. Called when the bot is removed from a group, so that being added back starts with a clean cooldown rather than resurrecting whatever was @@ -92,10 +134,11 @@ class Store: years of statistics on the strength of one bad HTTP response. History is permanent; only the live scaffolding is disposable. """ - # Spelled out rather than looped over _TABLES: an f-string here is a - # SQL-injection shape even when the input is a module constant, and - # the repo does not take lint suppressions. test_forget_group_clears - # _every_table is what keeps this list in step with the schema. + # Spelled out rather than looped over CLEARED_ON_LEAVE: an f-string + # here is a SQL-injection shape even when the input is a module + # constant, and the repo does not take lint suppressions. + # test_forget_group_clears_every_cleared_table is what keeps this + # list in step with that tuple. self._conn.execute("DELETE FROM command_uses WHERE group_id = ?", (group_id,)) self._conn.execute("DELETE FROM daily_uses WHERE group_id = ?", (group_id,)) self._conn.execute("DELETE FROM votes WHERE group_id = ?", (group_id,)) diff --git a/signal_bot/traps_command.py b/signal_bot/traps_command.py index 96dafd1..74e07b3 100644 --- a/signal_bot/traps_command.py +++ b/signal_bot/traps_command.py @@ -22,6 +22,7 @@ from base64 import b64encode from pathlib import Path from typing import TYPE_CHECKING, Final +from signal_bot.analytics import probe from signal_bot.animals import load_aliases from signal_bot.commons import caption_for, imageinfo_params, page_of from signal_bot.endpoints import THUMB_WIDTH @@ -88,9 +89,12 @@ def trap_handler( async def handler(ctx: CommandContext) -> None: # One request, because the caption and the picture have to describe # the same file; two would be a chance for them to disagree. + page_id = random_trap(pool) + # A Commons page id, for the reason art_command records one. + probe.note_item(str(page_id)) document = await fetcher.fetch_json( api_url, - imageinfo_params(THUMB_WIDTH, page_id=random_trap(pool), describe=True), + imageinfo_params(THUMB_WIDTH, page_id=page_id, describe=True), ) page = page_of(document) image = await fetcher.fetch_image(str(page["imageinfo"][0]["thumburl"])) diff --git a/signal_bot/versioning.py b/signal_bot/versioning.py index f5778de..97bc232 100644 --- a/signal_bot/versioning.py +++ b/signal_bot/versioning.py @@ -30,7 +30,7 @@ PATCHES_PER_MINOR: Final = 8 # Which release this build is. Bump by one per release; everything else, # including the version string in pyproject.toml, follows from it. -RELEASE_ORDINAL: Final = 22 +RELEASE_ORDINAL: Final = 23 _POPES_FILE: Final = Path(__file__).with_name("popes.json") diff --git a/tests/analytics_seed.py b/tests/analytics_seed.py new file mode 100644 index 0000000..b162ee4 --- /dev/null +++ b/tests/analytics_seed.py @@ -0,0 +1,144 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""Building a command history to ask questions of. + +Everything is arithmetic -- no randomness, no clock -- so a failure names a +row rather than a seed. The people are invented (ala, basia, celina, darek) +and so are the group ids: ``check_personal_data`` learns real names from the +live database, so a real one here would fail only on the maintainer's +machine and pass in CI. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from typing import TYPE_CHECKING + +from signal_bot.analytics.backfill import midnight_ms +from signal_bot.analytics.probe import UpstreamCall +from signal_bot.analytics.uses import ( + OUTCOME_OK, + SOURCE_DAILY, + SOURCE_JOURNAL, + SOURCE_LIVE, + CommandEvent, +) +from signal_bot.stats.backfill import UNKNOWN_HOUR +from signal_bot.stats.periods import weekday_of + +if TYPE_CHECKING: + from collections.abc import Sequence + + from signal_bot.store import Store + +GROUP = "group-a==" +OTHER = "group-b==" + +ALA = "uuid-ala" +BASIA = "uuid-basia" +CELINA = "uuid-celina" + +HOUR_MS = 3_600_000 + + +def at(day: str, hour: int, minute: int = 0) -> int: + """A Warsaw wall-clock moment on one day, in epoch milliseconds.""" + return midnight_ms(day) + hour * HOUR_MS + minute * 60_000 + + +@dataclass(frozen=True, slots=True) +class Shape: + """Everything about one attempt except who ran what, and when. + + Grouped rather than passed as arguments, for the reason the repository + caps arguments at seven: these are the columns a given test usually + leaves entirely alone, and a call site that names three of them reads + better than one that names all eleven. + """ + + group_id: str = GROUP + outcome: str = OUTCOME_OK + source: str = SOURCE_LIVE + item_id: str | None = None + handler_ms: int | None = None + send_ms: int | None = None + error_class: str = "" + args: str = "" + calls: Sequence[UpstreamCall] = () + + +# A module-level singleton, because Shape is frozen and a default argument +# that is a call is evaluated once anyway -- naming it says so. +PLAIN = Shape() + + +def ran( + store: Store, + *, + day: str, + hour: int = 12, + minute: int = 0, + who: str = ALA, + command: str = "cat", + shape: Shape = PLAIN, +) -> int: + """Record one attempt and return the timestamp it was keyed on.""" + invoked_at_ms = at(day, hour, minute) + store.commands.record( + CommandEvent( + group_id=shape.group_id, + user_uuid=who, + command=command, + args=shape.args, + item_id=shape.item_id, + outcome=shape.outcome, + error_class=shape.error_class, + invoked_at_ms=invoked_at_ms, + day=day, + local_hour=UNKNOWN_HOUR if shape.source == SOURCE_DAILY else hour, + weekday=weekday_of(day), + handler_ms=shape.handler_ms, + send_ms=shape.send_ms, + source=shape.source, + calls=shape.calls, + ) + ) + return invoked_at_ms + + +def backfilled( + store: Store, *, day: str, command: str = "cat", who: str = ALA, group_id: str = GROUP +) -> None: + """One row from the source that knows a day and nothing finer.""" + ran( + store, + day=day, + command=command, + who=who, + hour=0, + shape=Shape(group_id=group_id, source=SOURCE_DAILY), + ) + + +def reconstructed( + store: Store, + *, + day: str, + hour: int, + command: str = "cat", + who: str = ALA, + group_id: str = GROUP, +) -> None: + """One row recovered from the journal: a real hour, but no durations.""" + ran( + store, + day=day, + hour=hour, + command=command, + who=who, + shape=Shape(group_id=group_id, source=SOURCE_JOURNAL), + ) + + +def a_call(host: str = "api.example.test", status: int = 200, ms: int = 40) -> UpstreamCall: + """One outbound request made while a command was running.""" + return UpstreamCall(host=host, status=status, ms=ms) diff --git a/tests/test_analytics_backfill.py b/tests/test_analytics_backfill.py new file mode 100644 index 0000000..2d65356 --- /dev/null +++ b/tests/test_analytics_backfill.py @@ -0,0 +1,250 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""Recovering the command history, without inventing any of it. + +Two properties matter more than the rest and each has its own test: the two +sources must not double-count the same invocations, and a second run must +insert nothing. +""" + +from __future__ import annotations + +from typing import TYPE_CHECKING + +from signal_bot.analytics import backfill +from signal_bot.analytics.backfill_join import WINDOW_MS +from signal_bot.analytics.journal import JournalLine +from signal_bot.analytics.uses import SOURCE_DAILY, SOURCE_JOURNAL +from signal_bot.stats.events import MessageEvent + +if TYPE_CHECKING: + from signal_bot.store import Store + +GROUP = "group-a==" +DAY = "2026-08-29" +# Midday on the day above, so a +/- window cannot cross a date boundary. +NOON_MS = backfill.midnight_ms(DAY) + 12 * 3600 * 1000 + + +def a_message(store: Store, at_ms: int, who: str = "ala", uuid: str = "u1") -> None: + """One counted command in the message history.""" + store.events.record( + MessageEvent( + group_id=GROUP, + uuid=uuid, + display_name=who, + sent_at_ms=at_ms, + day=DAY, + local_hour=12, + is_command=True, + ) + ) + + +def a_line(at_ms: int, command: str, who: str = "ala", *, refused: bool = False) -> JournalLine: + """One line the bot wrote about running a command.""" + return JournalLine(at_ms=at_ms, command=command, display_name=who, refused=refused) + + +def rows(store: Store) -> list[tuple[str, str, int, str]]: + """Every recorded attempt as (command, outcome, hour, source).""" + return [ + (str(r["command"]), str(r["outcome"]), int(r["local_hour"]), str(r["source"])) + for r in store.commands.connection.execute( + "SELECT command, outcome, local_hour, source FROM command_events ORDER BY invoked_at_ms" + ) + ] + + +def test_a_message_gets_its_name_from_the_journal(store: Store) -> None: + """The whole point: the database has the hour, the journal has the name.""" + a_message(store, NOON_MS) + report, events = backfill.plan(store, [a_line(NOON_MS + 200, "cat")]) + backfill.apply(store, events) + + assert report.journal_rows == 1 + assert rows(store) == [("cat", "ok", 12, SOURCE_JOURNAL)] + + +def test_a_journal_line_earlier_than_its_message_still_matches(store: Store) -> None: + """The window is symmetric, and this is why. + + sent_at_ms comes from the sender's phone and the journal stamp from this + host; nothing keeps two clocks in step. On the live database 194 of 299 + counted commands had a journal line *earlier* than their own timestamp. + An asymmetric window drops every one of them. + """ + a_message(store, NOON_MS) + _report, events = backfill.plan(store, [a_line(NOON_MS - 3_000, "cat")]) + backfill.apply(store, events) + assert rows(store) == [("cat", "ok", 12, SOURCE_JOURNAL)] + + +def test_a_line_outside_the_window_is_not_used(store: Store) -> None: + """Far enough away it is somebody else's invocation, or another day's.""" + a_message(store, NOON_MS) + report, events = backfill.plan(store, [a_line(NOON_MS + WINDOW_MS + 1, "cat")]) + backfill.apply(store, events) + assert report.journal_rows == 0 + assert report.unmatched == 1 + assert [row for row in rows(store) if row[3] == SOURCE_JOURNAL] == [] + + +def test_two_commands_seconds_apart_are_both_resolved(store: Store) -> None: + """Global closest-pair assignment, not best-for-each-row in turn. + + Both lines sit inside both messages' windows. Per-row greedy cannot tell + them apart and drops both; assignment pairs them by distance, which is + correct because the clock skew is the same for both. + """ + a_message(store, NOON_MS) + a_message(store, NOON_MS + 5_000) + report, events = backfill.plan( + store, + [a_line(NOON_MS + 200, "cat"), a_line(NOON_MS + 5_200, "dog")], + ) + backfill.apply(store, events) + + assert report.ambiguous == 0 + assert [(row[0], row[3]) for row in rows(store)] == [ + ("cat", SOURCE_JOURNAL), + ("dog", SOURCE_JOURNAL), + ] + + +def test_a_message_with_no_line_left_for_it_is_dropped(store: Store) -> None: + """Two messages, one line: the second is a gap, never a guess.""" + a_message(store, NOON_MS) + a_message(store, NOON_MS + 1_000) + report, events = backfill.plan(store, [a_line(NOON_MS + 100, "cat")]) + backfill.apply(store, events) + + assert report.journal_rows == 1 + assert report.ambiguous == 1 + + +def test_a_cooldown_refusal_is_recovered_as_a_refusal(store: Store) -> None: + """The one gate that already logged. daily_limit and vote_gated did not.""" + a_message(store, NOON_MS) + _report, events = backfill.plan(store, [a_line(NOON_MS, "cat", refused=True)]) + backfill.apply(store, events) + assert rows(store) == [("cat", "refused_cooldown", 12, SOURCE_JOURNAL)] + + +def test_the_two_sources_do_not_double_count(store: Store) -> None: + """The failure that would look exactly like data. + + daily_uses says three uses. The journal explains one of them exactly. + The residual must be two, not three -- otherwise every historical count + is inflated and nothing anywhere fails. + """ + a_message(store, NOON_MS) + for _ in range(3): + store.record_use(GROUP, "u1", "cat", 1.0, DAY) + + report, events = backfill.plan(store, [a_line(NOON_MS + 100, "cat")]) + backfill.apply(store, events) + + assert report.journal_rows == 1 + assert report.daily_rows == 2 + assert len(rows(store)) == 3 + + +def test_a_day_the_message_history_never_saw_arrives_whole(store: Store) -> None: + """The real shape of 2026-08-29: counts, and no message_events at all. + + Those rows carry UNKNOWN_HOUR, so every hour-of-day panel must exclude + them rather than file them under midnight. + """ + for _ in range(4): + store.record_use(GROUP, "u1", "pope", 1.0, DAY) + + report, events = backfill.plan(store, []) + backfill.apply(store, events) + + assert report.journal_rows == 0 + assert report.daily_rows == 4 + assert {row[2] for row in rows(store)} == {-1} + assert {row[3] for row in rows(store)} == {SOURCE_DAILY} + + +def test_running_it_twice_inserts_nothing_the_second_time(store: Store) -> None: + """Idempotency, and it must come from the key rather than from a wipe. + + journald has no retention configuration and will rotate. A + clear-then-reimport pass would delete rows it could no longer replace, + so a second run has to be harmless without one. + """ + a_message(store, NOON_MS) + store.record_use(GROUP, "u1", "cat", 1.0, DAY) + lines = [a_line(NOON_MS + 100, "cat")] + + _first, events = backfill.plan(store, lines) + backfill.apply(store, events) + before = rows(store) + + for candidate in lines: + assert candidate is not None + _second, again = backfill.plan(store, [a_line(NOON_MS + 100, "cat")]) + backfill.apply(store, again) + + assert rows(store) == before + + +def test_live_rows_are_never_re_imported(store: Store) -> None: + """The cutoff. A live row already describes that invocation exactly. + + Without it, a pass run after recording started would import the journal + line for an invocation that already exists -- under a different source, + so the natural key would not stop it. + """ + a_message(store, NOON_MS) + store.commands.connection.execute( + """ + INSERT INTO command_events ( + group_id, user_uuid, command, args, item_id, outcome, error_class, + invoked_at_ms, day, local_hour, weekday, handler_ms, send_ms, source + ) VALUES (?, ?, 'cat', '', NULL, 'ok', '', ?, ?, 12, 5, 10, 10, 'live') + """, + (GROUP, "u1", NOON_MS, DAY), + ) + store.commands.connection.commit() + + report, events = backfill.plan(store, [a_line(NOON_MS + 100, "cat")]) + backfill.apply(store, events) + + assert report.journal_rows == 0 + assert [row[3] for row in rows(store)] == ["live"] + + +def test_a_pass_marks_the_groups_it_touched(store: Store) -> None: + """The same marker stats/backfill.py writes, for the same reason.""" + a_message(store, NOON_MS) + _report, events = backfill.plan(store, [a_line(NOON_MS, "cat")]) + backfill.apply(store, events) + assert store.meta(GROUP, backfill.BACKFILL_KEY) == "done" + + +def test_the_report_counts_everything_and_names_nobody() -> None: + """It is printed to a terminal and pasted into issues. + + The join's only input that could carry a person is a display name, and + it dies inside the joiner -- so the summary has to be countable and + nothing else. + """ + report = backfill.Report(journal_rows=3, daily_rows=2, unmatched=1, ambiguous=4, groups=2) + text = "\n".join(report.lines()) + assert "3" in text + assert "ambiguous: 4" in text + assert text.count("\n") == 4 + + +def test_midnight_is_built_from_the_date_not_from_arithmetic() -> None: + """Warsaw days are not all 24 hours; subtracting 86_400_000 lands wrong. + + 2026-10-25 is the 25-hour day, so the gap from the 24th is not a day's + worth of milliseconds and any spelling that assumes it is drifts. + """ + short_day = backfill.midnight_ms("2026-10-25") - backfill.midnight_ms("2026-10-24") + assert short_day == 24 * 3600 * 1000 + long_day = backfill.midnight_ms("2026-10-26") - backfill.midnight_ms("2026-10-25") + assert long_day == 25 * 3600 * 1000 diff --git a/tests/test_analytics_behaviour.py b/tests/test_analytics_behaviour.py new file mode 100644 index 0000000..b13a311 --- /dev/null +++ b/tests/test_analytics_behaviour.py @@ -0,0 +1,234 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""Families D, E and F: demand, sittings, adoption and pool exhaustion.""" + +from __future__ import annotations + +from typing import TYPE_CHECKING + +import pytest + +from signal_bot.analytics import adoption, content, demand, sessions +from signal_bot.analytics.scope import SESSION_GAP_MS, Scope +from signal_bot.stats.events import MessageEvent +from tests.analytics_seed import ALA, BASIA, CELINA, GROUP, Shape, backfilled, ran + +if TYPE_CHECKING: + import sqlite3 + + from signal_bot.store import Store + +DAY = "2026-08-29" +NEXT = "2026-08-30" +TRIGGERS = ("!kot", "!pies", "!pulapka", "!papiez") + + +def conn_of(store: Store) -> sqlite3.Connection: + return store.commands.connection + + +# -- family D ---------------------------------------------------------------- + + +def test_near_misses_are_grouped_by_the_command_they_nearly_hit(store: Store) -> None: + """Seventeen typos of one trigger mean it wants another alias. + + Ungrouped they read as seventeen unrelated mistakes, which is exactly + the list the Ideas tab already shows and the reason this adds something. + """ + for at_ms, word in enumerate(("!pulapla", "!pulapak", "!pulakpa", "!kto")): + store.suggestions.record_unknown(GROUP, word, word, at_ms) + + found = demand.collect(conn_of(store), Scope(), TRIGGERS) + assert [(miss.target, miss.hits) for miss in found.near_misses] == [ + ("!pulapka", 3), + ("!kot", 1), + ] + + +def test_the_same_typo_twice_counts_twice(store: Store) -> None: + store.suggestions.record_unknown(GROUP, "!kto", "!kto", 1) + store.suggestions.record_unknown(GROUP, "!kto", "!kto raz jeszcze", 2) + found = demand.collect(conn_of(store), Scope(), TRIGGERS) + assert found.near_misses[0].hits == 2 + assert found.near_misses[0].variants == 1 + + +def test_a_word_close_to_nothing_is_a_new_idea_not_a_typo(store: Store) -> None: + """Filing a real request as noise is the failure worth avoiding.""" + store.suggestions.record_unknown(GROUP, "!pogodynka", "!pogodynka", 1) + found = demand.collect(conn_of(store), Scope(), TRIGGERS) + assert found.near_misses == () + assert (found.unclaimed, found.unclaimed_words) == (1, 1) + + +def test_deliberate_ideas_are_counted_beside_the_typos(store: Store) -> None: + store.suggestions.record_suggestion(GROUP, "!pogoda", "na jutro", 1) + assert demand.collect(conn_of(store), Scope(), TRIGGERS).filed == 1 + + +def test_an_empty_wish_list_reads_as_empty(store: Store) -> None: + assert demand.collect(conn_of(store), Scope(), TRIGGERS) == demand.Demand() + + +# -- family E ---------------------------------------------------------------- + + +def test_commands_close_together_are_one_sitting(store: Store) -> None: + ran(store, day=DAY, hour=12, minute=0, command="cat") + ran(store, day=DAY, hour=12, minute=1, command="dog") + found = sessions.collect(conn_of(store), Scope()) + assert (found.count, found.commands, found.longest) == (1, 2, 2) + assert found.average_length == 2.0 + + +def test_a_long_gap_starts_a_new_sitting(store: Store) -> None: + gap_minutes = SESSION_GAP_MS // 60_000 + 1 + ran(store, day=DAY, hour=12, minute=0, command="cat") + ran(store, day=DAY, hour=12, minute=gap_minutes, command="dog") + assert sessions.collect(conn_of(store), Scope()).count == 2 + + +def test_two_people_never_share_a_sitting(store: Store) -> None: + ran(store, day=DAY, hour=12, minute=0, who=ALA, command="cat") + ran(store, day=DAY, hour=12, minute=1, who=BASIA, command="dog") + found = sessions.collect(conn_of(store), Scope()) + assert found.count == 2 + assert found.pairs == () + + +def test_commands_used_together_are_paired_once(store: Store) -> None: + """(cat, dog) and (dog, cat) are one fact, not two.""" + ran(store, day=DAY, hour=12, minute=0, command="dog") + ran(store, day=DAY, hour=12, minute=1, command="cat") + assert sessions.collect(conn_of(store), Scope()).pairs == (("cat", "dog", 1),) + + +def test_re_rolling_the_same_command_is_counted_as_repeats(store: Store) -> None: + """Three !papież in a row is a fact about the pool, not about the person.""" + for minute in range(3): + ran(store, day=DAY, hour=12, minute=minute, command="pope") + found = sessions.collect(conn_of(store), Scope()) + assert found.repeats == (("pope", 2),) + assert found.pairs == () + + +def test_rows_with_no_real_time_never_form_a_sitting(store: Store) -> None: + """A daily_uses row's timestamp is built from midnight, not observed. + + Treating those as a sequence invents a sitting in which somebody ran + forty commands in forty milliseconds. + """ + for _ in range(3): + backfilled(store, day=DAY, command="cat") + assert sessions.collect(conn_of(store), Scope()) == sessions.Sessions() + + +def test_share_of_chat_is_commands_over_messages(store: Store) -> None: + for at_ms in range(4): + store.events.record( + MessageEvent( + group_id=GROUP, + uuid=ALA, + display_name="ala", + sent_at_ms=at_ms, + day=DAY, + local_hour=12, + is_command=at_ms < 1, + ) + ) + found = adoption.share_of_chat(conn_of(store), Scope()) + assert (found.messages, found.commands) == (4, 1) + assert found.fraction == 0.25 + + +def test_share_of_an_empty_group_is_zero(store: Store) -> None: + assert adoption.share_of_chat(conn_of(store), Scope()).fraction == 0.0 + assert adoption.Share().fraction == 0.0 + + +def test_newcomers_are_dated_by_their_first_use(store: Store) -> None: + ran(store, day=DAY, who=ALA) + ran(store, day=NEXT, who=ALA, minute=1) + ran(store, day=NEXT, who=BASIA, minute=2) + assert adoption.newcomers(conn_of(store), Scope()) == [(DAY, 1), (NEXT, 1)] + + +def test_retention_counts_who_came_back(store: Store) -> None: + """Of the people who started, how many were still there N days later.""" + ran(store, day=DAY, who=ALA) + ran(store, day=NEXT, who=ALA, minute=1) + ran(store, day=DAY, who=BASIA, minute=2) + + curve = adoption.retention(conn_of(store), Scope(), 2) + assert curve == [(0, 2, 2), (1, 1, 2), (2, 0, 2)] + + +def test_retention_of_nobody_is_empty(store: Store) -> None: + assert adoption.retention(conn_of(store), Scope(), 3) == [] + + +def test_a_release_is_compared_against_equal_windows(store: Store) -> None: + """A release on the first recorded day gets a before-window of zero. + + Unequal windows would read that as a spike when it is an edge. + """ + series = [ + ("2026-08-28", 2), + ("2026-08-29", 3), + ("2026-08-30", 10), + ("2026-08-31", 12), + ] + found = adoption.around_releases(series, [("0.2.5", "2026-08-30")], 1) + assert found == [("0.2.5", "2026-08-30", 22, 3)] + + +# -- family F ---------------------------------------------------------------- + + +def test_a_pool_that_never_repeats_reads_as_varied(store: Store) -> None: + for minute, item in enumerate(("Q1", "Q2", "Q3")): + ran(store, day=DAY, minute=minute, command="pope", shape=Shape(item_id=item)) + (pool,) = content.pools(conn_of(store), Scope()) + assert (pool.draws, pool.distinct) == (3, 3) + assert pool.repeat_rate == 0.0 + + +def test_a_repeating_pool_is_measurable(store: Store) -> None: + """The bug this exists to catch reached a live group chat once. + + iNaturalist accepted order_by=random and silently ignored it, so one + identical photograph came back forever. A repeat rate would have said so. + """ + for minute in range(4): + ran(store, day=DAY, minute=minute, command="pope", shape=Shape(item_id="Q1")) + (pool,) = content.pools(conn_of(store), Scope()) + assert (pool.draws, pool.distinct) == (4, 1) + assert pool.repeat_rate == 0.75 + assert content.repeats(conn_of(store), Scope()) == [("pope", "Q1", 4)] + + +def test_a_command_that_names_nothing_has_no_pool(store: Store) -> None: + ran(store, day=DAY, command="stats_today") + assert content.pools(conn_of(store), Scope()) == [] + + +def test_an_undrawn_pool_has_no_repeat_rate() -> None: + assert content.Pool("pope", 0, 0).repeat_rate == 0.0 + + +def test_expected_distinct_says_what_uniform_would_have_looked_like() -> None: + """The comparison that turns a repeat rate into a verdict.""" + assert content.expected_distinct(0, 10) == 0.0 + assert content.expected_distinct(266, 0) == 0.0 + # 266 traps drawn 19 times should give about 18.4 distinct ones. + assert 18.0 < content.expected_distinct(266, 19) < 18.9 + # Drawing once always gives exactly one -- approx because the identity + # holds in real arithmetic and not in binary floating point. + assert content.expected_distinct(10, 1) == pytest.approx(1.0) + + +def test_everything_is_scoped_to_one_group(store: Store) -> None: + ran(store, day=DAY, command="pope", who=CELINA, shape=Shape(group_id=GROUP, item_id="Q1")) + ran(store, day=DAY, command="pope", who=CELINA, shape=Shape(group_id="group-b==", item_id="Q2")) + assert content.pools(conn_of(store), Scope(GROUP))[0].draws == 1 + assert content.pools(conn_of(store), Scope())[0].draws == 2 diff --git a/tests/test_analytics_command.py b/tests/test_analytics_command.py new file mode 100644 index 0000000..67ebb35 --- /dev/null +++ b/tests/test_analytics_command.py @@ -0,0 +1,233 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""What !analityka says, and the two things it must never say.""" + +from __future__ import annotations + +import datetime as dt +from typing import TYPE_CHECKING + +from signal_bot.analytics import render +from signal_bot.analytics.demand import Demand, NearMiss +from signal_bot.analytics.hours import Peak +from signal_bot.analytics.render import Summary +from signal_bot.analytics.scope import Window +from signal_bot.analytics.usage import CommandUse, PersonUse +from signal_bot.analytics_command import register_analytics_command, triggers_for +from signal_bot.dispatcher import Dispatcher +from signal_bot.registry import CommandRegistry +from signal_bot.stats.events import MessageEvent +from tests.analytics_seed import ALA, BASIA, Shape, ran +from tests.conftest import WATCHED_GROUP, envelope + +if TYPE_CHECKING: + from signal_bot.context import CommandContext + from signal_bot.store import Store + from signal_bot.transport_fake import FakeTransport + +DAY = "2026-08-29" +NOW = dt.datetime(2026, 8, 29, 12, 0, tzinfo=dt.UTC) + + +def a_registry() -> CommandRegistry: + """A registry with the command under test on it.""" + registry = CommandRegistry() + register_analytics_command(registry, aliases={"command_stats": ["!analityka"]}) + return registry + + +def seed(store: Store, *, group_id: str = WATCHED_GROUP) -> None: + """A small history with a name attached to one of the two people.""" + for minute in range(3): + ran( + store, + day=DAY, + hour=19, + minute=minute, + who=ALA, + command="pope", + shape=Shape(group_id=group_id), + ) + ran(store, day=DAY, hour=9, who=BASIA, command="cat", shape=Shape(group_id=group_id)) + store.events.record( + MessageEvent( + group_id=group_id, + uuid=ALA, + display_name="ala", + sent_at_ms=1, + day=DAY, + local_hour=19, + is_command=True, + ) + ) + store.events.note_person(group_id, ALA, "ala", "", DAY) + + +async def reply_of(dispatcher: Dispatcher, transport: FakeTransport) -> str: + """Run !analityka and return what went into the group.""" + await dispatcher.handle(envelope(text="!analityka")) + return transport.sent[-1].text + + +def test_an_empty_history_says_so_in_one_line() -> None: + """A fresh install must not answer with a blank block.""" + summary = render.Summary( + measured=Window(), + uses=(), + people=(), + names={}, + peak=Peak(), + direction="unknown", + demand=Demand(), + asker=ALA, + commands=0, + messages=0, + ) + assert render.render(summary) == render.NOTHING_YET + + +def test_the_reply_is_capped_at_twelve_lines() -> None: + """The cap, on a summary that fills every optional line.""" + summary = Summary( + measured=Window(first_day=DAY, last_day="2026-09-30", days=30, events=99), + uses=( + CommandUse("pope", 40, 3, 30, 8, 2, DAY, DAY), + CommandUse("cat", 10, 2, 10, 0, 0, DAY, DAY), + CommandUse("dog", 1, 1, 1, 0, 0, DAY, DAY), + ), + people=(PersonUse(ALA, 30, 3, "pope"), PersonUse(BASIA, 5, 1, "cat")), + names={ALA: "ala", BASIA: "basia"}, + peak=Peak(busiest=19, busiest_uses=20, quietest=5, quietest_uses=1), + direction="rising", + demand=Demand(near_misses=(NearMiss("!pulapka", 17, 4),)), + asker=ALA, + commands=99, + messages=300, + ) + lines = render.render(summary).split("\n") + assert len(lines) <= render.MAX_LINES + assert lines[0] == render.HEADING + # Every line earned its place; none is blank padding. + assert all(line.strip() for line in lines) + + +def test_the_window_is_stated_before_anything_is_claimed(store: Store) -> None: + """The bot is days old, and a reader has to be able to see that.""" + line = render.window_line(Window(first_day=DAY, last_day=DAY, days=1, events=4)) + assert line.startswith("Okno: 1 dzień") + assert "4 razy" in line + + +def test_a_span_of_days_names_both_ends() -> None: + line = render.window_line( + Window(first_day="2026-08-29", last_day="2026-08-31", days=3, events=10) + ) + assert "3 dni" in line + assert "29 sierpnia" in line + assert "31 sierpnia" in line + + +def test_no_line_carries_padding() -> None: + """Signal renders a proportional font, so ljust lines up on nobody's phone.""" + line = render.window_line(Window(first_day=DAY, last_day=DAY, days=1, events=1)) + assert line == line.strip() + assert " " not in line + + +def test_a_direction_is_only_worded_when_there_is_one() -> None: + assert render.trend_line("unknown") == "" + assert render.trend_line("rising") == "użycie rośnie" + assert render.trend_line("falling") == "użycie spada" + assert render.trend_line("flat") == "użycie się trzyma" + + +def test_nobody_is_named_without_a_display_name() -> None: + """The roster knows the rest as a uuid or a phone number, and both leak.""" + people = (PersonUse(ALA, 5, 2, "pope"), PersonUse(BASIA, 3, 1, "cat")) + assert render.people_line(people, {}) == "" + named = render.people_line(people, {ALA: "ala"}) + assert named == "Kto najwięcej: ala 5" + assert BASIA not in named + + +def test_the_asker_gets_their_own_favourite_and_nobody_elses() -> None: + """Self-scope: a per-person breakdown of everybody stays off the chat.""" + people = (PersonUse(ALA, 5, 2, "pope"), PersonUse(BASIA, 3, 1, "cat")) + assert render.mine_line(people, ALA) == "Twoja ulubiona: !pope (5 razy)" + assert render.mine_line(people, "uuid-nobody") == "" + + +def test_a_healthy_bot_says_nothing_about_health() -> None: + """A line with nothing to say is dropped, never printed empty.""" + healthy = (CommandUse("cat", 3, 1, 3, 0, 0, DAY, DAY),) + assert render.health_line(healthy) == "" + + +def test_the_worst_command_is_named_with_what_is_wrong() -> None: + """Refusals count as much as errors: both read as "it does not work".""" + uses = ( + CommandUse("cat", 10, 2, 6, 3, 1, DAY, DAY), + CommandUse("dog", 4, 1, 4, 0, 0, DAY, DAY), + ) + line = render.health_line(uses) + assert line.startswith("Do poprawki: !cat") + assert "błędów: 1" in line + assert "odmów: 3" in line + + +def test_only_what_is_actually_wrong_is_listed() -> None: + """A command that only crashes says nothing about refusals, and vice versa.""" + crashing = (CommandUse("cat", 5, 1, 3, 0, 2, DAY, DAY),) + assert render.health_line(crashing) == "Do poprawki: !cat (błędów: 2)" + + refusing = (CommandUse("cat", 5, 1, 3, 2, 0, DAY, DAY),) + assert render.health_line(refusing) == "Do poprawki: !cat (odmów: 2)" + + +def test_share_of_chat_is_omitted_when_there_is_no_chat() -> None: + assert render.share_line(0, 0) == "" + assert render.share_line(1, 4) == "Komendy to 25% wiadomości w tej grupie" + + +def test_triggers_are_scoped_to_the_group_that_asked() -> None: + """The album's triggers are real people's names. + + Reporting a near miss against one outside its group would announce that + the album exists, which is the leak the dispatcher already refuses at + resolution time. + """ + + async def handler(_ctx: CommandContext) -> None: + pass + + registry = CommandRegistry() + registry.register("cat", ["!kot"], handler, "kot") + registry.register("pack:p1", ["!ala"], handler, "ala", groups=frozenset({"album=="})) + + here = triggers_for(registry.commands, "somewhere-else==") + there = triggers_for(registry.commands, "album==") + assert "!ala" not in here + assert "!ala" in there + + +async def test_the_command_answers_in_the_group_that_asked( + config: object, + directory: object, + store: Store, + transport: FakeTransport, +) -> None: + """The whole path: a real frame, the real dispatcher, a real reply.""" + seed(store) + dispatcher = Dispatcher( + config=config, + directory=directory, + registry=a_registry(), + store=store, + transport=transport, + clock=lambda: NOW, + ) + text = await reply_of(dispatcher, transport) + + assert text.startswith(render.HEADING) + assert "!pope" in text + assert "ala" in text + assert len(text.split("\n")) <= render.MAX_LINES diff --git a/tests/test_analytics_health.py b/tests/test_analytics_health.py new file mode 100644 index 0000000..2ae28f4 --- /dev/null +++ b/tests/test_analytics_health.py @@ -0,0 +1,201 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""Family C: the questions that had no data at all before command_events.""" + +from __future__ import annotations + +from typing import TYPE_CHECKING + +from signal_bot.analytics import health +from signal_bot.analytics.probe import NO_RESPONSE +from signal_bot.analytics.scope import Scope +from signal_bot.analytics.uses import ( + OUTCOME_COOLDOWN, + OUTCOME_DAILY, + OUTCOME_ERROR, +) +from tests.analytics_seed import GROUP, OTHER, Shape, a_call, backfilled, ran, reconstructed + +if TYPE_CHECKING: + import sqlite3 + + from signal_bot.store import Store + +DAY = "2026-08-29" + + +def conn_of(store: Store) -> sqlite3.Connection: + return store.commands.connection + + +def test_nothing_recorded_reads_as_nothing(store: Store) -> None: + conn = conn_of(store) + assert health.latencies(conn, Scope()) == [] + assert health.gates(conn, Scope()) == [] + assert health.errors(conn, Scope()) == [] + assert health.upstreams(conn, Scope()) == [] + + +def test_percentiles_come_from_the_rows_that_have_timings(store: Store) -> None: + for minute, ms in enumerate((100, 200, 300, 400, 500)): + ran(store, day=DAY, minute=minute, shape=Shape(handler_ms=ms, send_ms=ms // 2)) + (latency,) = health.latencies(conn_of(store), Scope()) + assert latency.samples == 5 + assert latency.handler_p50 == 300 + assert latency.handler_p95 == 500 + assert latency.send_p50 == 150 + + +def test_a_row_with_no_duration_never_reaches_a_percentile(store: Store) -> None: + """NULL, never 0. A zero would drag a p50 down and read as a fast command. + + This is the reason the duration columns are nullable rather than + defaulted -- and the reason the query says IS NOT NULL rather than + naming a source, so a future source with real timings joins in for free. + """ + ran(store, day=DAY, shape=Shape(handler_ms=400, send_ms=300)) + reconstructed(store, day=DAY, hour=10) + backfilled(store, day=DAY) + + (latency,) = health.latencies(conn_of(store), Scope()) + assert latency.samples == 1 + assert latency.handler_p50 == 400 + + +def test_a_command_that_never_replied_has_no_send_time(store: Store) -> None: + """Handler time without send time is a real shape, so it must not crash. + + The live recorder always writes both, but a source that timed the + handler and not the send would otherwise put a NULL through int(). + """ + ran(store, day=DAY, shape=Shape(handler_ms=400, send_ms=None, outcome=OUTCOME_ERROR)) + (latency,) = health.latencies(conn_of(store), Scope()) + assert latency.handler_p50 == 400 + assert latency.send_p50 == 0 + + +def test_an_empty_list_has_no_percentile() -> None: + assert health.percentile([], health.P50) == 0 + + +def test_a_single_sample_is_its_own_percentile() -> None: + assert health.percentile([7], health.P95) == 7 + + +def test_each_gate_is_counted_separately(store: Store) -> None: + """ "Refused 40% of the time" and "popular" are two readings of one count. + + Which gate bit is the difference between a cooldown that is too long and + a daily limit that is too low. + """ + ran(store, day=DAY, minute=1, shape=Shape(outcome=OUTCOME_COOLDOWN)) + ran(store, day=DAY, minute=2, shape=Shape(outcome=OUTCOME_COOLDOWN)) + ran(store, day=DAY, minute=3, shape=Shape(outcome=OUTCOME_DAILY)) + + assert health.gates(conn_of(store), Scope()) == [ + ("cat", "refused_cooldown", 2), + ("cat", "refused_daily", 1), + ] + + +def test_errors_are_grouped_by_class_never_by_message(store: Store) -> None: + """An exception's str() carries whatever it was processing. + + Here that is somebody's message, so only the class name is kept -- and + "17 ImageFetchError" is the whole diagnostic value anyway. + """ + ran(store, day=DAY, minute=1, shape=Shape(outcome=OUTCOME_ERROR, error_class="ImageFetchError")) + ran(store, day=DAY, minute=2, shape=Shape(outcome=OUTCOME_ERROR, error_class="ImageFetchError")) + ran(store, day=DAY, minute=3, shape=Shape(outcome=OUTCOME_ERROR, error_class="TimeoutError")) + + assert health.errors(conn_of(store), Scope()) == [ + ("cat", "ImageFetchError", 2), + ("cat", "TimeoutError", 1), + ] + + +def test_upstream_hosts_are_counted_and_timed(store: Store) -> None: + ran( + store, + day=DAY, + shape=Shape( + handler_ms=500, + calls=(a_call("api.example.test", 200, 40), a_call("cdn.example.test", 200, 90)), + ), + ) + ran( + store, + day=DAY, + minute=1, + shape=Shape(handler_ms=500, calls=(a_call("api.example.test", 200, 60),)), + ) + + hosts = {up.host: up for up in health.upstreams(conn_of(store), Scope())} + assert hosts["api.example.test"].calls == 2 + # Nearest-rank of [40, 60] is the lower: ceil(0.5 * 2) - 1 == 0. + assert hosts["api.example.test"].p50_ms == 40 + assert hosts["cdn.example.test"].calls == 1 + + +def test_a_request_that_never_answered_counts_as_a_failure(store: Store) -> None: + """Status 0 is DNS, connect or timeout: upstream never answered at all. + + Distinct from a 5xx, which is upstream answering badly. Losing it would + make a service that is entirely down look merely unused. + """ + ran( + store, + day=DAY, + shape=Shape(handler_ms=10, calls=(a_call("api.example.test", NO_RESPONSE, 5000),)), + ) + ran( + store, + day=DAY, + minute=1, + shape=Shape(handler_ms=10, calls=(a_call("api.example.test", 503, 20),)), + ) + ran( + store, + day=DAY, + minute=2, + shape=Shape(handler_ms=10, calls=(a_call("api.example.test", 200, 20),)), + ) + + (host,) = health.upstreams(conn_of(store), Scope()) + assert host.calls == 3 + assert host.failures == 2 + assert host.failure_rate == 2 / 3 + + +def test_a_host_never_called_has_no_failure_rate() -> None: + assert health.Upstream("api.example.test", 0, 0, 0).failure_rate == 0.0 + + +def test_upstream_time_is_attributed_to_the_command_that_spent_it(store: Store) -> None: + """Which half of a slow command is this repository's code.""" + ran( + store, + day=DAY, + command="pope", + shape=Shape(handler_ms=3000, send_ms=2400, calls=(a_call(ms=250),)), + ) + (latency,) = health.latencies(conn_of(store), Scope()) + assert latency.command == "pope" + assert latency.upstream_p50 == 250 + assert latency.send_p50 == 2400 + + +def test_everything_is_scoped_to_one_group(store: Store) -> None: + ran( + store, + day=DAY, + shape=Shape(group_id=GROUP, handler_ms=10, calls=(a_call("here.example.test"),)), + ) + ran( + store, + day=DAY, + shape=Shape(group_id=OTHER, handler_ms=10, calls=(a_call("there.example.test"),)), + ) + + here = health.upstreams(conn_of(store), Scope(GROUP)) + assert [host.host for host in here] == ["here.example.test"] + assert len(health.upstreams(conn_of(store), Scope())) == 2 diff --git a/tests/test_analytics_hours.py b/tests/test_analytics_hours.py new file mode 100644 index 0000000..3f5a70f --- /dev/null +++ b/tests/test_analytics_hours.py @@ -0,0 +1,177 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""Family B: hours, weekdays, day series, months and dying commands. + +Every hour test seeds a ``daily_uses`` row and asserts it is absent. That is +how the exclusion gets proved rather than remembered: those rows carry +UNKNOWN_HOUR because their source knew only a date, and letting one through +files it under hour -1 -- or under midnight, if a later edit "tidies" the +sentinel away. +""" + +from __future__ import annotations + +from typing import TYPE_CHECKING + +from signal_bot.analytics import hours, trend +from signal_bot.analytics.scope import Scope, Window, window +from tests.analytics_seed import ALA, BASIA, backfilled, ran, reconstructed + +if TYPE_CHECKING: + import sqlite3 + + from signal_bot.store import Store + +# A Saturday, so the weekday index is not accidentally right at 0. +DAY = "2026-08-29" +NEXT = "2026-08-30" +SATURDAY = 5 + + +def conn_of(store: Store) -> sqlite3.Connection: + return store.commands.connection + + +def test_the_hour_histogram_has_a_bucket_for_every_hour(store: Store) -> None: + """A missing 04:00 draws the wrong shape, and the empty hours are data.""" + ran(store, day=DAY, hour=9) + buckets = hours.by_hour(conn_of(store), Scope()) + assert len(buckets) == hours.HOURS_IN_DAY + assert buckets[9] == 1 + assert sum(buckets) == 1 + + +def test_a_row_with_no_hour_is_left_out(store: Store) -> None: + """The exclusion, asserted rather than trusted.""" + ran(store, day=DAY, hour=9) + backfilled(store, day=DAY) + assert sum(hours.by_hour(conn_of(store), Scope())) == 1 + + +def test_a_reconstructed_row_keeps_its_hour(store: Store) -> None: + """journald rows were matched to a real message, so their hour is real. + + Excluding them would throw away most of the history for no gain, which + is why scope names two predicates and not one. + """ + reconstructed(store, day=DAY, hour=21) + buckets = hours.by_hour(conn_of(store), Scope()) + assert buckets[21] == 1 + + +def test_the_weekday_grid_is_seven_by_twenty_four(store: Store) -> None: + ran(store, day=DAY, hour=13) + grid = hours.by_weekday_hour(conn_of(store), Scope()) + assert len(grid) == hours.DAYS_IN_WEEK + assert all(len(row) == hours.HOURS_IN_DAY for row in grid) + assert grid[SATURDAY][13] == 1 + + +def test_each_command_gets_its_own_hours(store: Store) -> None: + ran(store, day=DAY, hour=8, command="cat") + ran(store, day=DAY, hour=22, command="pope") + grid = hours.by_command_hour(conn_of(store), Scope()) + assert grid["cat"][8] == 1 + assert grid["pope"][22] == 1 + assert sum(grid["cat"]) == 1 + + +def test_weekdays_carry_the_days_they_were_seen_on(store: Store) -> None: + """An average needs a denominator, and one Tuesday is not a weekly rate.""" + ran(store, day=DAY, hour=9) + ran(store, day=DAY, hour=10, minute=5) + assert hours.by_weekday(conn_of(store), Scope()) == [(SATURDAY, 2, 1)] + + +def test_the_peak_ignores_hours_nobody_ever_used(store: Store) -> None: + """At three days old most of the night is empty. + + "The deadest hour is 04:00 with zero" would be true of nine hours and + interesting about none of them. + """ + ran(store, day=DAY, hour=19) + ran(store, day=DAY, hour=19, minute=1) + ran(store, day=DAY, hour=6) + found = hours.peak(hours.by_hour(conn_of(store), Scope())) + assert (found.busiest, found.busiest_uses) == (19, 2) + assert (found.quietest, found.quietest_uses) == (6, 1) + + +def test_a_silent_history_has_no_peak(store: Store) -> None: + assert hours.peak([0] * hours.HOURS_IN_DAY) == hours.Peak() + + +def test_a_silent_day_is_drawn_as_a_zero(store: Store) -> None: + """A sparse series draws a line straight over a day nobody used it.""" + ran(store, day="2026-08-29", hour=9) + ran(store, day="2026-08-31", hour=9) + dense = trend.dense_days(trend.by_day(conn_of(store), Scope())) + assert dense == [("2026-08-29", 1), ("2026-08-30", 0), ("2026-08-31", 1)] + + +def test_an_empty_series_densifies_to_nothing() -> None: + assert trend.dense_days([]) == [] + + +def test_months_carry_how_many_days_they_actually_saw(store: Store) -> None: + """A partial month is not a quiet one, and the panel must be able to say.""" + ran(store, day=DAY, hour=9) + ran(store, day=NEXT, hour=9) + assert trend.by_month(conn_of(store), Scope()) == [("2026-08", 2, 2)] + + +def test_per_command_day_series(store: Store) -> None: + ran(store, day=DAY, hour=9, command="cat") + ran(store, day=NEXT, hour=9, command="cat") + ran(store, day=NEXT, hour=10, command="pope") + series = trend.by_command_day(conn_of(store), Scope()) + assert series["cat"] == {DAY: 1, NEXT: 1} + assert series["pope"] == {NEXT: 1} + + +def test_no_direction_is_claimed_from_a_short_history(store: Store) -> None: + """Three points of a new bot's novelty curve are not a trend.""" + series = [("2026-08-29", 1), ("2026-08-30", 5), ("2026-08-31", 20)] + assert trend.direction(series, Window(days=3)) == trend.UNKNOWN + + +def test_a_direction_is_claimed_once_there_is_enough(store: Store) -> None: + rising = [(f"2026-08-{day:02d}", day) for day in range(1, 15)] + falling = [(f"2026-08-{day:02d}", 15 - day) for day in range(1, 15)] + flat = [(f"2026-08-{day:02d}", 10) for day in range(1, 15)] + measured = Window(days=14) + assert trend.direction(rising, measured) == trend.RISING + assert trend.direction(falling, measured) == trend.FALLING + assert trend.direction(flat, measured) == trend.FLAT + + +def test_a_direction_needs_two_points() -> None: + assert trend.direction([("2026-08-29", 3)], Window(days=20)) == trend.UNKNOWN + assert trend.direction([], Window(days=20)) == trend.UNKNOWN + + +def test_growth_from_nothing_is_rising_and_nothing_is_flat() -> None: + """A first half of zero cannot be divided by, and both cases are real.""" + measured = Window(days=20) + from_zero = [("a", 0), ("b", 0), ("c", 5), ("d", 5)] + stayed_zero = [("a", 0), ("b", 0), ("c", 0), ("d", 0)] + assert trend.direction(from_zero, measured) == trend.RISING + assert trend.direction(stayed_zero, measured) == trend.FLAT + + +def test_a_command_that_stopped_being_used_is_named(store: Store) -> None: + ran(store, day="2026-08-20", hour=9, command="gecko") + ran(store, day="2026-08-31", hour=9, command="cat") + dying = trend.dying(conn_of(store), Scope(), "2026-08-31") + assert [(entry.command, entry.idle_days) for entry in dying] == [("gecko", 11)] + + +def test_a_command_used_today_is_not_dying(store: Store) -> None: + ran(store, day="2026-08-31", hour=9, command="cat") + assert trend.dying(conn_of(store), Scope(), "2026-08-31") == [] + + +def test_two_people_on_one_day_are_counted_once_each(store: Store) -> None: + ran(store, day=DAY, hour=9, who=ALA) + ran(store, day=DAY, hour=10, who=BASIA) + assert trend.by_day(conn_of(store), Scope()) == [(DAY, 2, 2)] + assert window(conn_of(store), Scope()).days == 1 diff --git a/tests/test_analytics_journal.py b/tests/test_analytics_journal.py new file mode 100644 index 0000000..9f6e725 --- /dev/null +++ b/tests/test_analytics_journal.py @@ -0,0 +1,91 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""Reading the bot's own journal back, without copying names out of it.""" + +from __future__ import annotations + +from signal_bot.analytics.journal import parse + +PREFIX = "2026-08-31T21:28:26+02:00 archlinux run.sh[101862]:" +STAMP = "2026-08-31 21:28:26,217" + + +def running(command: str, who: str = "ala") -> str: + """One real journal line, in the shape journalctl -o short-iso emits.""" + return f"{PREFIX} {STAMP} INFO signal_bot.dispatcher: running command {command} for {who}" + + +def test_a_running_line_is_read() -> None: + (line,) = parse([running("cat")]) + assert (line.command, line.display_name, line.refused) == ("cat", "ala", False) + + +def test_the_timestamp_comes_from_the_journal_not_the_bot() -> None: + """The journal's stamp carries an offset; the bot's asctime does not. + + A naive stamp read as UTC lands the whole import two hours out, which is + exactly what the hour columns exist to get right. + """ + (line,) = parse([running("cat")]) + # 2026-08-31T21:28:26+02:00 is 19:28:26 UTC. + assert line.at_ms == 1_788_204_506_000 + + +def test_an_album_command_keeps_its_pseudonym_key() -> None: + """`pack:p1`, not `pack`. A \\w+ pattern silently truncates at the colon. + + That truncation would collapse thirteen different album commands into + one, which is the difference between "who is asked for most" and a + single meaningless total. + """ + (line,) = parse([running("pack:p1")]) + assert line.command == "pack:p1" + + +def test_a_cooldown_refusal_is_read_as_one() -> None: + text = ( + f"{PREFIX} {STAMP} INFO signal_bot.decorators: " + "cooldown refused cat for ala: told them 8 sekund" + ) + (line,) = parse([text]) + assert (line.command, line.refused) == ("cat", True) + + +def test_a_display_name_with_spaces_survives() -> None: + """The pattern runs to end of line, because a display name is free text.""" + (line,) = parse([running("cat", "ala z basia")]) + assert line.display_name == "ala z basia" + + +def test_lines_that_are_not_about_commands_are_skipped() -> None: + """Startup banners, tracebacks, envelope shapes, other units.""" + noise = [ + "2026-08-31T21:00:00+02:00 archlinux systemd[1474]: Started Signal group bot.", + f"{PREFIX} {STAMP} DEBUG signal_bot.dispatcher: envelope shape: {{'a': 'str'}}", + "not a journal line at all", + "", + ] + assert list(parse(noise)) == [] + + +def test_a_line_whose_stamp_is_not_a_timestamp_is_skipped() -> None: + """Never guess a time. A wrong hour is worse than a missing row.""" + assert list(parse(["wednesday archlinux run.sh[1]: running command cat for ala"])) == [] + + +def test_a_naive_stamp_is_refused() -> None: + """Without an offset there is no way to know which hour this was.""" + text = "2026-08-31T21:28:26 archlinux run.sh[1]: running command cat for ala" + assert list(parse([text])) == [] + + +def test_a_dst_boundary_is_read_from_the_offset() -> None: + """Warsaw is +02:00 in summer and +01:00 in winter, and both occur. + + Reading the wall clock and assuming one offset puts half the year's + rows an hour out, in a column whose whole purpose is the hour. + """ + summer = "2026-10-25T02:30:00+02:00 archlinux run.sh[1]: running command cat for ala" + winter = "2026-10-25T02:30:00+01:00 archlinux run.sh[1]: running command cat for ala" + (early,) = parse([summer]) + (late,) = parse([winter]) + assert late.at_ms - early.at_ms == 3_600_000 diff --git a/tests/test_analytics_polish.py b/tests/test_analytics_polish.py new file mode 100644 index 0000000..5e81c8b --- /dev/null +++ b/tests/test_analytics_polish.py @@ -0,0 +1,30 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""Polish agreement in the reply, split out of test_analytics_command. + +Three forms, not two, and the rule reads the last *two* digits -- so 12, 13 +and 14 take the many-form while 22, 23 and 24 do not. Getting this wrong is +the kind of thing the group notices immediately and mentions every time. +""" + +from __future__ import annotations + +import pytest + +from signal_bot.analytics import render + + +@pytest.mark.parametrize( + ("count", "expected"), + [(1, "1 raz"), (2, "2 razy"), (5, "5 razy"), (12, "12 razy"), (22, "22 razy")], +) +def test_polish_agreement_on_counts(count: int, expected: str) -> None: + """Twelve takes the many-form and twenty-two does not.""" + assert render.times_phrase(count) == expected + + +@pytest.mark.parametrize( + ("count", "expected"), + [(1, "1 dzień"), (2, "2 dni"), (5, "5 dni"), (12, "12 dni")], +) +def test_polish_agreement_on_days(count: int, expected: str) -> None: + assert render.days_phrase(count) == expected diff --git a/tests/test_analytics_probe.py b/tests/test_analytics_probe.py new file mode 100644 index 0000000..5334575 --- /dev/null +++ b/tests/test_analytics_probe.py @@ -0,0 +1,83 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""The ContextVar that lets a fetcher report a timing it was never asked for.""" + +from __future__ import annotations + +import pytest + +from signal_bot.analytics import probe + + +def test_nothing_is_probing_by_default() -> None: + """The var's default is what every no-op branch below tests against.""" + assert probe.CURRENT.get() is None + + +def test_notes_outside_a_probe_are_dropped() -> None: + """The whole point: a caller never asks whether it is being measured. + + ``http_client`` and ``context`` call these unconditionally, and the + offline end-to-end run drives both with no dispatcher above them. + """ + probe.note_send(5) + probe.note_upstream("example.test", 200, 5) + probe.note_item("Q1") + assert probe.CURRENT.get() is None + + +def test_a_probe_collects_what_ran_under_it() -> None: + """One invocation's notes, from three different layers.""" + with probe.probing() as active: + probe.note_send(120) + probe.note_send(80) + probe.note_upstream("api.example.test", 200, 40) + probe.note_upstream("cdn.example.test", 500, 15) + probe.note_item("Q42") + assert active.send_ms == 200 + assert active.item_id == "Q42" + assert [(call.host, call.status, call.ms) for call in active.calls] == [ + ("api.example.test", 200, 40), + ("cdn.example.test", 500, 15), + ] + + +def _fail() -> None: + """A handler that raises, in its own function so the raise is not inline.""" + raise RuntimeError + + +def test_the_probe_is_closed_even_when_the_handler_raises() -> None: + """A command that fails is still a command whose cost we wanted.""" + seen: list[probe.Probe] = [] + + def run() -> None: + with probe.probing() as active: + seen.append(active) + probe.note_upstream("api.example.test", probe.NO_RESPONSE, 30) + _fail() + + with pytest.raises(RuntimeError): + run() + + assert probe.CURRENT.get() is None + assert len(seen[0].calls) == 1 + + +def test_probes_do_not_leak_into_each_other() -> None: + """reset(token), not set(None) -- two invocations in a row must not merge.""" + with probe.probing() as first: + probe.note_send(10) + with probe.probing() as second: + probe.note_send(20) + assert first.send_ms == 10 + assert second.send_ms == 20 + + +def test_a_nested_probe_restores_the_outer_one() -> None: + """Nesting is not expected, but a token reset must survive it if it happens.""" + with probe.probing() as outer: + with probe.probing() as inner: + probe.note_send(3) + probe.note_send(7) + assert inner.send_ms == 3 + assert outer.send_ms == 7 diff --git a/tests/test_analytics_usage.py b/tests/test_analytics_usage.py new file mode 100644 index 0000000..e266449 --- /dev/null +++ b/tests/test_analytics_usage.py @@ -0,0 +1,159 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""Family A, and the scope every family shares.""" + +from __future__ import annotations + +from typing import TYPE_CHECKING + +from signal_bot.analytics import usage +from signal_bot.analytics.scope import Scope, window +from signal_bot.analytics.uses import OUTCOME_COOLDOWN, OUTCOME_ERROR, OUTCOME_UNKNOWN +from tests.analytics_seed import ALA, BASIA, GROUP, OTHER, Shape, backfilled, ran, reconstructed + +if TYPE_CHECKING: + import sqlite3 + + from signal_bot.store import Store + +DAY = "2026-08-29" +NEXT = "2026-08-30" + + +def conn_of(store: Store) -> sqlite3.Connection: + """The connection the query layer reads through.""" + return store.commands.connection + + +def test_an_empty_history_is_a_state_not_an_error(store: Store) -> None: + """Every panel renders on a fresh install, before anybody has typed.""" + conn = conn_of(store) + scope = Scope() + assert window(conn, scope) == window(conn, scope) + assert window(conn, scope).events == 0 + assert usage.by_command(conn, scope) == [] + assert usage.by_person(conn, scope) == [] + assert usage.totals(conn, scope)["attempts"] == 0 + + +def test_the_window_says_what_it_covers_and_where_it_came_from(store: Store) -> None: + """Three days is not a month, and the reader has to be able to tell.""" + ran(store, day=DAY, command="cat") + reconstructed(store, day=DAY, hour=9, command="dog") + backfilled(store, day=NEXT, command="pope") + + measured = window(conn_of(store), Scope()) + assert (measured.first_day, measured.last_day, measured.days) == (DAY, NEXT, 2) + assert (measured.live, measured.journald, measured.daily) == (1, 1, 1) + assert not measured.enough_for_a_trend + + +def test_counts_include_every_source(store: Store) -> None: + """ "How often was this used" is answered by all three classes of row. + + A reconstructed invocation still happened. Only the questions that need + an hour or a duration narrow further. + """ + ran(store, day=DAY, command="cat") + reconstructed(store, day=DAY, hour=9, command="cat") + backfilled(store, day=DAY, command="cat") + + (use,) = usage.by_command(conn_of(store), Scope()) + assert use.uses == 3 + + +def test_a_group_scope_and_the_roll_up_run_the_same_sql(store: Store) -> None: + """One query with a nullable parameter, so the two cannot drift.""" + ran(store, day=DAY, command="cat", shape=Shape(group_id=GROUP)) + ran(store, day=DAY, command="dog", shape=Shape(group_id=OTHER)) + + everywhere = usage.by_command(conn_of(store), Scope()) + here = usage.by_command(conn_of(store), Scope(GROUP)) + assert [use.command for use in everywhere] == ["cat", "dog"] + assert [use.command for use in here] == ["cat"] + + +def test_outcomes_are_counted_apart(store: Store) -> None: + """Refused is not failed, and neither is a success.""" + ran(store, day=DAY, command="cat", minute=1) + ran(store, day=DAY, command="cat", minute=2, shape=Shape(outcome=OUTCOME_COOLDOWN)) + ran( + store, + day=DAY, + command="cat", + minute=3, + shape=Shape(outcome=OUTCOME_ERROR, error_class="ValueError"), + ) + + (use,) = usage.by_command(conn_of(store), Scope()) + assert (use.uses, use.ok, use.refused, use.errors) == (3, 1, 1, 1) + assert use.failure_rate == 1 / 3 + assert use.refusal_rate == 1 / 3 + + +def test_a_command_nobody_ever_ran_has_no_rates(store: Store) -> None: + """Dividing by an empty history is a zero, never a crash.""" + empty = usage.CommandUse("cat", 0, 0, 0, 0, 0, "", "") + assert empty.failure_rate == 0.0 + assert empty.refusal_rate == 0.0 + + +def test_unknown_words_are_not_commands(store: Store) -> None: + """A word nothing claims belongs to family D, not to the usage counts.""" + ran(store, day=DAY, command="cat") + ran(store, day=DAY, command="pulapka", minute=5, shape=Shape(outcome=OUTCOME_UNKNOWN)) + + assert [use.command for use in usage.by_command(conn_of(store), Scope())] == ["cat"] + assert usage.totals(conn_of(store), Scope())["attempts"] == 1 + + +def test_people_come_back_with_their_favourite(store: Store) -> None: + """Most active first, each with the command they reach for.""" + for minute in range(3): + ran(store, day=DAY, minute=minute, who=ALA, command="pope") + ran(store, day=DAY, minute=9, who=ALA, command="cat") + ran(store, day=DAY, minute=10, who=BASIA, command="dog") + + people = usage.by_person(conn_of(store), Scope()) + assert [(person.uuid, person.uses, person.favourite) for person in people] == [ + (ALA, 4, "pope"), + (BASIA, 1, "dog"), + ] + assert people[0].commands == 2 + + +def test_the_matrix_is_every_person_against_every_command(store: Store) -> None: + """What the management page draws and the group chat never sees.""" + ran(store, day=DAY, who=ALA, command="cat") + ran(store, day=DAY, who=BASIA, command="cat", minute=1) + ran(store, day=DAY, who=BASIA, command="dog", minute=2) + + assert usage.matrix(conn_of(store), Scope()) == [ + (ALA, "cat", 1), + (BASIA, "cat", 1), + (BASIA, "dog", 1), + ] + + +def test_commands_nobody_has_tried_are_named(store: Store) -> None: + """Which registered commands are dead on arrival. + + Takes the registry's list rather than reading it, because which commands + exist depends on the group -- the album's exist only where it does. + """ + ran(store, day=DAY, command="cat") + used = usage.by_command(conn_of(store), Scope()) + assert usage.never_used(["cat", "dog", "pope"], used) == ["dog", "pope"] + assert usage.never_used([], used) == [] + + +def test_totals_count_distinct_people_and_commands(store: Store) -> None: + ran(store, day=DAY, who=ALA, command="cat") + ran(store, day=DAY, who=BASIA, command="cat", minute=1) + ran(store, day=DAY, who=BASIA, command="dog", minute=2) + + assert usage.totals(conn_of(store), Scope()) == { + "attempts": 3, + "ok": 3, + "people": 2, + "commands": 2, + } diff --git a/tests/test_command_events.py b/tests/test_command_events.py new file mode 100644 index 0000000..47799ae --- /dev/null +++ b/tests/test_command_events.py @@ -0,0 +1,168 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""The command history table: what it holds, and what a redelivery does to it.""" + +from __future__ import annotations + +from typing import TYPE_CHECKING + +from signal_bot.analytics.probe import NO_RESPONSE, UpstreamCall +from signal_bot.analytics.uses import ( + OUTCOME_COOLDOWN, + OUTCOME_OK, + SOURCE_DAILY, + SOURCE_LIVE, + CommandEvent, +) + +if TYPE_CHECKING: + from collections.abc import Sequence + + from signal_bot.store import Store + +GROUP = "group-a==" + +# The exact column set, in order. Pinned for the reason message_events' is: +# this table sits in a repository that leaked real names once, and a column +# that could hold free text must be a deliberate, test-visible addition. +EVENT_COLUMNS = ( + "group_id", + "user_uuid", + "command", + "args", + "item_id", + "outcome", + "error_class", + "invoked_at_ms", + "day", + "local_hour", + "weekday", + "handler_ms", + "send_ms", + "source", +) +CALL_COLUMNS = ( + "group_id", + "invoked_at_ms", + "command", + "host", + "status", + "duration_ms", + "seq", +) + + +def an_event( + *, + command: str = "cat", + outcome: str = OUTCOME_OK, + invoked_at_ms: int = 1, + local_hour: int = 12, + handler_ms: int | None = None, + source: str = SOURCE_LIVE, + calls: Sequence[UpstreamCall] = (), +) -> CommandEvent: + """One minimal attempt. ``invoked_at_ms`` also disambiguates the key.""" + return CommandEvent( + group_id=GROUP, + user_uuid="u1", + command=command, + outcome=outcome, + invoked_at_ms=invoked_at_ms, + day="2026-08-29", + local_hour=local_hour, + weekday=5, + handler_ms=handler_ms, + source=source, + calls=calls, + ) + + +def rows(store: Store, table: str) -> list[dict[str, object]]: + """Every row of one of the two tables, as plain dicts.""" + queries = { + "command_events": "SELECT * FROM command_events ORDER BY invoked_at_ms, source", + "upstream_calls": "SELECT * FROM upstream_calls ORDER BY invoked_at_ms, seq", + } + return [dict(row) for row in store.commands.connection.execute(queries[table])] + + +def test_the_column_set_is_exactly_this(store: Store) -> None: + """A new column here is a deliberate change, never a quiet one.""" + store.commands.record(an_event()) + assert tuple(rows(store, "command_events")[0]) == EVENT_COLUMNS + + +def test_the_call_column_set_is_exactly_this(store: Store) -> None: + """The same pin on the table that records where the time went.""" + store.commands.record(an_event(calls=(UpstreamCall("api.example.test", 200, 9),))) + assert tuple(rows(store, "upstream_calls")[0]) == CALL_COLUMNS + + +def test_a_replayed_frame_does_not_become_a_second_row(store: Store) -> None: + """signal-cli replays undelivered frames when the bot reconnects. + + Keyed on the envelope's own timestamp rather than the wall clock, so the + replay collides with the row it already wrote. + """ + for _ in range(3): + store.commands.record(an_event()) + assert len(rows(store, "command_events")) == 1 + + +def test_a_replay_does_not_duplicate_the_upstream_calls(store: Store) -> None: + """The calls are keyed independently, so they need their own guard. + + Without ON CONFLICT on this table too, a redelivered frame would triple + the recorded upstream traffic while the invocation count stayed at one. + """ + for _ in range(3): + store.commands.record(an_event(calls=(UpstreamCall("api.example.test", 200, 9),))) + assert len(rows(store, "upstream_calls")) == 1 + + +def test_the_same_invocation_from_two_sources_is_two_rows(store: Store) -> None: + """source is in the key: a backfilled row may not overwrite a live one. + + The two disagree about the hour by construction, and whichever wrote + last would win silently. + """ + store.commands.record(an_event()) + store.commands.record(an_event(source=SOURCE_DAILY, local_hour=-1)) + assert [row["source"] for row in rows(store, "command_events")] == ["daily_uses", "live"] + + +def test_a_refusal_is_recorded_like_any_other_attempt(store: Store) -> None: + """The rows that did not exist at all before this table did.""" + store.commands.record(an_event(outcome=OUTCOME_COOLDOWN, handler_ms=0)) + (row,) = rows(store, "command_events") + assert row["outcome"] == "refused_cooldown" + + +def test_an_invocation_with_no_upstream_calls_records_none(store: Store) -> None: + """Every stats_* command is this shape, and the loop must not mind.""" + store.commands.record(an_event(command="stats_today")) + assert rows(store, "upstream_calls") == [] + + +def test_several_calls_keep_their_order(store: Store) -> None: + """seq is the enumerate index, so two calls to one host stay distinct.""" + store.commands.record( + an_event( + calls=( + UpstreamCall("api.example.test", 200, 40), + UpstreamCall("api.example.test", NO_RESPONSE, 5000), + ) + ) + ) + assert [(row["seq"], row["status"]) for row in rows(store, "upstream_calls")] == [ + (0, 200), + (1, 0), + ] + + +def test_a_backfilled_row_has_no_duration(store: Store) -> None: + """NULL, never 0. Zero milliseconds reads as data and would drag a p50.""" + store.commands.record(an_event(source=SOURCE_DAILY, local_hour=-1)) + (row,) = rows(store, "command_events") + assert row["handler_ms"] is None + assert row["send_ms"] is None diff --git a/tests/test_dispatcher_outcomes.py b/tests/test_dispatcher_outcomes.py new file mode 100644 index 0000000..1f8374e --- /dev/null +++ b/tests/test_dispatcher_outcomes.py @@ -0,0 +1,171 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""The outcomes that had no data at all before command_events. + +Split out of ``test_dispatcher_recording`` so both stay under the 250-line +cap. This half is the one that matters most: a refusal and a crash are rows +now, and the *use* -- the write that feeds the cooldown -- must still not +move for either of them. +""" + +from __future__ import annotations + +import datetime as dt +from typing import TYPE_CHECKING + +import pytest + +from signal_bot.analytics import probe +from signal_bot.decorators import cooldown, daily_limit +from signal_bot.registry import CommandRegistry +from tests.conftest import WATCHED_GROUP, envelope +from tests.test_dispatcher_recording import SENDER, build, calls, history + +if TYPE_CHECKING: + from signal_bot.config import Config + from signal_bot.context import CommandContext + from signal_bot.discovery import GroupDirectory + from signal_bot.store import Store + from signal_bot.transport_fake import FakeTransport + +NOW = dt.datetime(2026, 8, 29, 12, 0, tzinfo=dt.UTC) + + +@pytest.fixture +def ok_registry() -> CommandRegistry: + """One command that succeeds and reports what it did on the way.""" + + async def handler(_ctx: CommandContext) -> None: + probe.note_upstream("api.example.test", 200, 40) + probe.note_send(120) + probe.note_item("Q42") + + reg = CommandRegistry() + reg.register("cat", ["!cat"], handler, "kot") + return reg + + +async def test_a_refusal_does_not_move_the_cooldown( + config: Config, + directory: GroupDirectory, + store: Store, + transport: FakeTransport, +) -> None: + """The invariant. A refused attempt is history, never a use. + + If these two writes are ever merged, this fails and the bug it names + comes back: every "wait N seconds" reply would push the wait N seconds + further away, and waiting would never end. + """ + + async def handler(_ctx: CommandContext) -> None: + pass + + reg = CommandRegistry() + reg.register( + "cat", + ["!cat"], + cooldown(3600.0, "poczekaj {seconds} {unit}")(handler), + "kot", + ) + dispatcher = build(config, directory, reg, store, transport) + + await dispatcher.handle(envelope(text="!cat", timestamp=1000)) + first_use = store.last_used(WATCHED_GROUP, SENDER, "cat") + await dispatcher.handle(envelope(text="!cat", timestamp=2000)) + + outcomes = [row["outcome"] for row in history(store)] + assert outcomes == ["ok", "refused_cooldown"] + # The whole point: the second attempt is a row, and it moved nothing. + assert store.last_used(WATCHED_GROUP, SENDER, "cat") == first_use + assert store.uses_today(WATCHED_GROUP, SENDER, "cat", "2026-08-29") == 1 + + +async def test_a_daily_refusal_names_its_own_gate( + config: Config, + directory: GroupDirectory, + store: Store, + transport: FakeTransport, +) -> None: + """Which gate bit is the difference between "popular" and "mis-tuned".""" + + async def handler(_ctx: CommandContext) -> None: + pass + + reg = CommandRegistry() + reg.register( + "cat", + ["!cat"], + daily_limit(1, "dosc na dzis")(handler), + "kot", + ) + dispatcher = build(config, directory, reg, store, transport) + + await dispatcher.handle(envelope(text="!cat", timestamp=1000)) + await dispatcher.handle(envelope(text="!cat", timestamp=2000)) + + assert [row["outcome"] for row in history(store)] == ["ok", "refused_daily"] + + +async def test_a_crash_is_recorded_and_the_bot_survives_it( + config: Config, + directory: GroupDirectory, + store: Store, + transport: FakeTransport, +) -> None: + """A command that fails every time used to look like one nobody ran.""" + + async def handler(_ctx: CommandContext) -> None: + raise ValueError + + reg = CommandRegistry() + reg.register("cat", ["!cat"], handler, "kot") + dispatcher = build(config, directory, reg, store, transport) + + await dispatcher.handle(envelope(text="!cat")) + + (row,) = history(store) + assert row["outcome"] == "error" + # The class name, never the message: an exception's str() carries + # whatever it was processing, and here that is somebody's message. + assert row["error_class"] == "ValueError" + assert store.last_used(WATCHED_GROUP, SENDER, "cat") is None + + +async def test_recording_can_fail_without_taking_the_bot_down( + config: Config, + directory: GroupDirectory, + ok_registry: CommandRegistry, + store: Store, + transport: FakeTransport, + monkeypatch: pytest.MonkeyPatch, +) -> None: + """A raise here would end the receive loop and the bot would go deaf. + + Analytics are worth a great deal less than answering at all, so the + storage failure is logged and the use is still recorded. + """ + + def explode(_event: object) -> None: + raise RuntimeError + + monkeypatch.setattr(store.commands, "record", explode) + dispatcher = build(config, directory, ok_registry, store, transport) + + await dispatcher.handle(envelope(text="!cat")) + + assert store.last_used(WATCHED_GROUP, SENDER, "cat") == NOW.timestamp() + + +async def test_a_replayed_frame_is_recorded_once( + config: Config, + directory: GroupDirectory, + ok_registry: CommandRegistry, + store: Store, + transport: FakeTransport, +) -> None: + """signal-cli replays undelivered frames when the bot reconnects.""" + dispatcher = build(config, directory, ok_registry, store, transport) + for _ in range(3): + await dispatcher.handle(envelope(text="!cat", timestamp=5000)) + assert len(history(store)) == 1 + assert len(calls(store)) == 1 diff --git a/tests/test_dispatcher_recording.py b/tests/test_dispatcher_recording.py new file mode 100644 index 0000000..b3d0946 --- /dev/null +++ b/tests/test_dispatcher_recording.py @@ -0,0 +1,133 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""Every attempt becomes a row, and a refusal still does not move the cooldown. + +The second half is the one that matters. ``CommandRefusedError``'s docstring +records the bug: recording a refused run slid the cooldown window forward +every time somebody was told to wait, so waiting the stated time never +helped. The command history now takes refusals *and* crashes, which makes +merging the two writes look like an obvious simplification. It is not, and +test_a_refusal_does_not_move_the_cooldown is what says so. +""" + +from __future__ import annotations + +import datetime as dt +from typing import TYPE_CHECKING, Any + +import pytest + +from signal_bot.analytics import probe +from signal_bot.dispatcher import Dispatcher +from signal_bot.registry import CommandRegistry +from tests.conftest import WATCHED_GROUP, envelope + +if TYPE_CHECKING: + from signal_bot.config import Config + from signal_bot.context import CommandContext + from signal_bot.discovery import GroupDirectory + from signal_bot.store import Store + from signal_bot.transport_fake import FakeTransport + +NOW = dt.datetime(2026, 8, 29, 12, 0, tzinfo=dt.UTC) +SENDER = "uuid-alice" + + +def history(store: Store) -> list[dict[str, Any]]: + """Every recorded attempt, oldest first.""" + return [ + dict(row) + for row in store.commands.connection.execute( + "SELECT * FROM command_events ORDER BY invoked_at_ms" + ) + ] + + +def calls(store: Store) -> list[dict[str, Any]]: + """Every recorded upstream call.""" + return [ + dict(row) + for row in store.commands.connection.execute("SELECT * FROM upstream_calls ORDER BY seq") + ] + + +def build( + config: Config, + directory: GroupDirectory, + registry: CommandRegistry, + store: Store, + transport: FakeTransport, +) -> Dispatcher: + """A dispatcher over one caller-supplied registry.""" + return Dispatcher( + config=config, + directory=directory, + registry=registry, + store=store, + transport=transport, + clock=lambda: NOW, + ) + + +@pytest.fixture +def ok_registry() -> CommandRegistry: + """One command that succeeds and reports what it did on the way.""" + + async def handler(_ctx: CommandContext) -> None: + probe.note_upstream("api.example.test", 200, 40) + probe.note_send(120) + probe.note_item("Q42") + + reg = CommandRegistry() + reg.register("cat", ["!cat"], handler, "kot") + return reg + + +async def test_a_success_is_recorded_with_what_it_cost( + config: Config, + directory: GroupDirectory, + ok_registry: CommandRegistry, + store: Store, + transport: FakeTransport, +) -> None: + """The row nothing wrote before: outcome, item, and where the time went.""" + dispatcher = build(config, directory, ok_registry, store, transport) + await dispatcher.handle(envelope(text="!cat")) + + (row,) = history(store) + assert row["command"] == "cat" + assert row["outcome"] == "ok" + assert row["item_id"] == "Q42" + assert row["send_ms"] == 120 + assert row["source"] == "live" + assert row["handler_ms"] >= 0 + assert calls(store) == [ + { + "group_id": WATCHED_GROUP, + "invoked_at_ms": row["invoked_at_ms"], + "command": "cat", + "host": "api.example.test", + "status": 200, + "duration_ms": 40, + "seq": 0, + } + ] + + +async def test_the_history_and_the_message_share_a_key( + config: Config, + directory: GroupDirectory, + ok_registry: CommandRegistry, + store: Store, + transport: FakeTransport, +) -> None: + """Both tables key on the envelope's own timestamp, or nothing joins.""" + dispatcher = build(config, directory, ok_registry, store, transport) + await dispatcher.handle(envelope(text="!cat")) + + (event,) = history(store) + row = store.events.connection.execute( + "SELECT sent_at_ms, local_hour, day FROM message_events WHERE is_command = 1" + ).fetchone() + assert event["invoked_at_ms"] == row["sent_at_ms"] + assert event["local_hour"] == row["local_hour"] + assert event["day"] == row["day"] diff --git a/tests/test_help_command.py b/tests/test_help_command.py index 01e4194..1beb1f2 100644 --- a/tests/test_help_command.py +++ b/tests/test_help_command.py @@ -8,6 +8,7 @@ from typing import TYPE_CHECKING import httpx import pytest +from signal_bot.analytics_command import register_analytics_command from signal_bot.animals import register_animal_commands from signal_bot.art_command import register_art_command from signal_bot.config import DEFAULT_COMMONS_API_URL, DEFAULT_POEM_TEXT_URL @@ -71,6 +72,7 @@ def full_registry() -> CommandRegistry: register_art_command(registry, fetcher, DEFAULT_COMMONS_API_URL, pools=((1,), (2,))) register_poem_command(registry, fetcher, DEFAULT_POEM_TEXT_URL, slugs=("impas",)) register_stats_commands(registry) + register_analytics_command(registry) register_suggest_command(registry) register_help_command(registry) return registry diff --git a/tests/test_routes_analytics.py b/tests/test_routes_analytics.py new file mode 100644 index 0000000..5b44c3e --- /dev/null +++ b/tests/test_routes_analytics.py @@ -0,0 +1,203 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""The Analytics tab's seven routes. + +The empty-database cases matter as much as the populated ones: this page is +most useful exactly when something is wrong, so every panel has to render +against a bot that has never run. +""" + +from __future__ import annotations + +from typing import TYPE_CHECKING + +from signal_bot.external import Result +from signal_bot.manage import routes_analytics +from signal_bot.store import Store +from tests.analytics_seed import ALA, BASIA, Shape, a_call, backfilled, ran, reconstructed +from tests.manage_fixture import FIXTURE_GROUP, client_for, deps_at + +if TYPE_CHECKING: + from pathlib import Path + + import pytest + + from signal_bot.manage.deps import ManageDeps + +DAY = "2026-08-29" +NEXT = "2026-08-30" + +PANELS = ("overview", "time", "health", "demand", "behaviour", "content") + + +def seeded(tmp_path: Path) -> ManageDeps: + """A management app pointed at a database with a small history in it.""" + deps = deps_at(tmp_path) + with Store(str(deps.database_path)) as store: + for minute in range(3): + ran( + store, + day=DAY, + hour=19, + minute=minute, + who=ALA, + command="pope", + shape=Shape( + group_id=FIXTURE_GROUP, + item_id=f"Q{minute}", + handler_ms=300 + minute, + send_ms=200, + calls=(a_call("commons.example.test", 200, 40),), + ), + ) + ran(store, day=NEXT, hour=9, who=BASIA, command="cat", shape=Shape(group_id=FIXTURE_GROUP)) + reconstructed(store, day=DAY, hour=8, command="dog", who=BASIA, group_id=FIXTURE_GROUP) + backfilled(store, day=DAY, command="gecko", who=ALA, group_id=FIXTURE_GROUP) + store.events.note_person(FIXTURE_GROUP, ALA, "ala", "", DAY) + store.suggestions.record_unknown(FIXTURE_GROUP, "!papiez!", "!papiez!", 5) + return deps + + +def test_every_panel_renders_against_a_bot_that_never_ran(tmp_path: Path) -> None: + """A fresh install must show empty panels, never a stack trace.""" + client = client_for(deps_at(tmp_path)) + for panel in PANELS: + response = client.get(f"/api/analytics/{panel}") + assert response.status_code == 200, panel + body = response.json() + assert body["window"]["events"] == 0 + assert client.get("/api/analytics/groups").json() == {"groups": []} + + +def test_every_panel_renders_against_a_file_of_nonsense(tmp_path: Path) -> None: + """Half a database is the same answer as none of one.""" + deps = deps_at(tmp_path) + deps.database_path.write_bytes(b"this is not a database") + client = client_for(deps) + for panel in PANELS: + assert client.get(f"/api/analytics/{panel}").status_code == 200 + + +def test_the_groups_route_lists_what_has_a_history(tmp_path: Path) -> None: + client = client_for(seeded(tmp_path)) + groups = client.get("/api/analytics/groups").json()["groups"] + assert [one["id"] for one in groups] == [FIXTURE_GROUP] + assert groups[0]["events"] == 6 + + +def test_the_window_names_all_three_sources(tmp_path: Path) -> None: + """The footnote's three numbers, so the page can say what is reconstructed.""" + client = client_for(seeded(tmp_path)) + window = client.get("/api/analytics/overview").json()["window"] + assert window["sources"] == {"live": 4, "journald": 1, "daily_uses": 1} + assert window["days"] == 2 + assert window["enough_for_a_trend"] is False + + +def test_overview_names_people_rather_than_uuids(tmp_path: Path) -> None: + """A uuid never appears on the page without a name beside it.""" + client = client_for(seeded(tmp_path)) + people = client.get("/api/analytics/overview").json()["people"] + named = {one["uuid"]: one["name"] for one in people} + assert named[ALA] == "ala" + assert named[BASIA] == "" + + +def test_time_returns_dense_buckets(tmp_path: Path) -> None: + """Twenty-four hours and seven weekdays, empty ones included.""" + body = client_for(seeded(tmp_path)).get("/api/analytics/time").json() + assert len(body["hours"]) == 24 + assert len(body["weekday_hours"]) == 7 + assert body["peak"]["busiest"] == 19 + assert body["direction"] == "unknown" + + +def test_health_reports_timings_and_upstream(tmp_path: Path) -> None: + body = client_for(seeded(tmp_path)).get("/api/analytics/health").json() + assert body["latencies"][0]["command"] == "pope" + assert body["upstream"][0]["host"] == "commons.example.test" + assert body["upstream"][0]["failures"] == 0 + + +def test_demand_groups_a_typo_by_what_it_nearly_hit(tmp_path: Path) -> None: + body = client_for(seeded(tmp_path)).get("/api/analytics/demand").json() + assert [one["target"] for one in body["near_misses"]] == ["!papiez"] + + +def test_behaviour_reports_sittings_and_share(tmp_path: Path) -> None: + body = client_for(seeded(tmp_path)).get("/api/analytics/behaviour").json() + assert body["sessions"]["count"] >= 1 + assert "fraction" in body["share"] + + +def test_content_reports_pool_variety(tmp_path: Path) -> None: + body = client_for(seeded(tmp_path)).get("/api/analytics/content").json() + assert body["pools"][0] == { + "command": "pope", + "draws": 3, + "distinct": 3, + "repeat_rate": 0.0, + } + + +def test_a_group_filter_narrows_the_answer(tmp_path: Path) -> None: + """Absent means the roll-up; present means one group, same SQL either way.""" + client = client_for(seeded(tmp_path)) + everywhere = client.get("/api/analytics/overview").json() + here = client.get("/api/analytics/overview", params={"group": FIXTURE_GROUP}).json() + nowhere = client.get("/api/analytics/overview", params={"group": "group.NOTHING"}).json() + assert everywhere["window"]["events"] == here["window"]["events"] + assert nowhere["window"]["events"] == 0 + + +def test_the_triggers_come_from_the_alias_file() -> None: + """A second process with no registry in it still knows every trigger.""" + triggers = routes_analytics.triggers() + assert "!analityka" in triggers + assert "!kot" in triggers + + +def test_release_dates_come_from_the_tags_and_degrade_to_nothing() -> None: + """The changelog says what shipped; only the tags say when. + + Never raises: a checkout with no tags, or a deployment that is not a git + tree at all, comes back empty and the timeline draws unannotated. + """ + for version, day in routes_analytics.releases(): + assert version[0].isdigit() + assert len(day) == len("2026-08-31") + + +def test_a_database_that_cannot_be_opened_is_the_empty_shape(tmp_path: Path) -> None: + """A file that exists and refuses to open is still just "nothing to show". + + Distinct from a corrupt one: sqlite opens a garbage file happily and + fails on the first query, so this covers the other half -- a permissions + problem, which raises at connect. + """ + deps = deps_at(tmp_path) + deps.database_path.write_bytes(b"") + deps.database_path.chmod(0o000) + try: + body = client_for(deps).get("/api/analytics/overview").json() + finally: + deps.database_path.chmod(0o600) + assert body["window"]["events"] == 0 + + +def test_release_parsing_ignores_anything_that_is_not_a_version_tag( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """Tags are not all releases, and a failed shell-out is not a crash.""" + monkeypatch.setattr( + routes_analytics, + "run", + lambda *_args, **_kwargs: Result( + ok=True, stdout="v0.2.5 2026-08-31\nnot-a-release 2026-08-30\nv0.1.0\n", stderr="" + ), + ) + assert routes_analytics.releases() == (("0.2.5", "2026-08-31"),) + + monkeypatch.setattr( + routes_analytics, "run", lambda *_a, **_k: Result(ok=False, stdout="", stderr="no git") + ) + assert routes_analytics.releases() == () diff --git a/tests/test_schema_tables.py b/tests/test_schema_tables.py new file mode 100644 index 0000000..8c8fd2f --- /dev/null +++ b/tests/test_schema_tables.py @@ -0,0 +1,185 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""Which tables survive what. + +EPHEMERAL_TABLES used to answer two different questions with one tuple, and +this file exists because splitting it is exactly the kind of change that looks +right and silently stops clearing something. Every assertion here is against +either sqlite's own catalogue or the code that does the dropping -- never +against a second copy of the list. +""" + +from __future__ import annotations + +import sqlite3 +from typing import TYPE_CHECKING + +from signal_bot.schema import ( + CLEARED_ON_LEAVE, + PERMANENT_TABLES, + REBUILT_ON_BUMP, + RETAINED_COLUMNS, + SCHEMA_VERSION, + TABLES, +) +from signal_bot.store import Store + +if TYPE_CHECKING: + from pathlib import Path + +GROUP = "group-a==" + + +def _declared(store: Store) -> set[str]: + """Every table sqlite actually holds, as opposed to every one we listed.""" + rows = store.events.connection.execute( + "SELECT name FROM sqlite_master WHERE type = 'table' AND name NOT LIKE 'sqlite_%'" + ).fetchall() + return {str(row["name"]) for row in rows} + + +def _seed(store: Store, group_id: str = GROUP) -> None: + """One row in every table a group can own.""" + store.record_use(group_id, "u1", "cat", 1.0, "2026-08-29") + store.add_vote(group_id, "cat", "u1", 1.0) + store.set_meta(group_id, "last_reported_day", "2026-08-29") + store.suggestions.record_suggestion(group_id, "!pogoda", "na jutro", 1) + store.suggestions.record_unknown(group_id, "!cta", "!cta", 2) + + +# Both maps are spelled out rather than interpolated, for the reason +# forget_group spells its DELETEs out: a table name in an f-string is a +# SQL-injection shape even when the input is a module constant, and this +# repository takes no lint suppressions. PRAGMA additionally takes no +# parameters at all. test_every_table_can_be_counted keeps COUNTS complete. +COLUMNS = { + "command_uses": "PRAGMA table_info(command_uses)", + "daily_uses": "PRAGMA table_info(daily_uses)", +} + +COUNTS = { + "command_uses": "SELECT count(*) AS n FROM command_uses WHERE group_id = ?", + "daily_uses": "SELECT count(*) AS n FROM daily_uses WHERE group_id = ?", + "votes": "SELECT count(*) AS n FROM votes WHERE group_id = ?", + "group_meta": "SELECT count(*) AS n FROM group_meta WHERE group_id = ?", + "message_events": "SELECT count(*) AS n FROM message_events WHERE group_id = ?", + "group_people": "SELECT count(*) AS n FROM group_people WHERE group_id = ?", + "unknown_commands": "SELECT count(*) AS n FROM unknown_commands WHERE group_id = ?", + "suggestions": "SELECT count(*) AS n FROM suggestions WHERE group_id = ?", + "command_events": "SELECT count(*) AS n FROM command_events WHERE group_id = ?", + "upstream_calls": "SELECT count(*) AS n FROM upstream_calls WHERE group_id = ?", +} + + +def _rows(store: Store, table: str, group_id: str = GROUP) -> int: + """How many rows one group owns in one table.""" + row = store.events.connection.execute(COUNTS[table], (group_id,)).fetchone() + return int(row["n"]) + + +def test_every_table_can_be_counted() -> None: + """A new table without a count query would skip every check below it.""" + assert set(COUNTS) == set(TABLES) + + +def test_declared_columns_match_the_schema(store: Store) -> None: + """RETAINED_COLUMNS is a second copy of the DDL, so it must be checked. + + _drop_reshaped compares against it to decide whether a retained table is + still writable. A stale entry here would drop a perfectly good table on + every bump -- silently, since dropping is the recovery path -- and take + the use history it was moved out of REBUILT_ON_BUMP to protect. + """ + for table, expected in RETAINED_COLUMNS.items(): + found = tuple(str(row["name"]) for row in store.events.connection.execute(COLUMNS[table])) + assert found == expected + + +def test_a_retained_table_with_the_wrong_shape_is_dropped(tmp_path: Path) -> None: + """The failure _apply_schema exists to prevent, on a table it no longer rebuilds. + + A database written before the group key holds a two-column command_uses. + CREATE TABLE IF NOT EXISTS will not touch it and the bump no longer drops + it, so without _drop_reshaped the bot boots clean and fails on every + INSERT. + """ + path = str(tmp_path / "legacy.sqlite3") + with sqlite3.connect(path) as raw: + raw.execute("CREATE TABLE daily_uses (uuid TEXT PRIMARY KEY, uses INTEGER NOT NULL)") + raw.execute("INSERT INTO daily_uses VALUES ('u1', 7)") + with Store(path) as store: + store.record_use(GROUP, "u1", "cat", 1.0, "2026-08-29") + assert store.uses_today(GROUP, "u1", "cat", "2026-08-29") == 1 + + +def test_the_list_matches_what_sqlite_holds(store: Store) -> None: + """TABLES is the schema's index, so it may not drift from the schema.""" + assert set(TABLES) == _declared(store) + + +def test_no_table_is_both_kept_and_thrown_away() -> None: + """A table in two of these tuples is a contradiction, not a configuration.""" + assert not set(PERMANENT_TABLES) & set(REBUILT_ON_BUMP) + assert not set(PERMANENT_TABLES) & set(CLEARED_ON_LEAVE) + + +def test_the_history_is_permanent() -> None: + """The four tables nothing can recompute, named where it is configured.""" + for table in ("message_events", "command_events", "upstream_calls", "group_people"): + assert table in PERMANENT_TABLES + + +def test_the_use_counters_reset_on_leave_but_survive_a_bump() -> None: + """The whole point of the split, asserted as one sentence. + + daily_uses feeds a rate limiter *and* is a source the analytics history is + backfilled from. Clearing it when the bot leaves a group is correct; + dropping it on a schema bump would throw away use history that predates + command_events. + """ + for table in ("command_uses", "daily_uses"): + assert table in CLEARED_ON_LEAVE + assert table not in REBUILT_ON_BUMP + + +def test_forget_group_clears_every_cleared_table(store: Store) -> None: + """The DELETEs in forget_group are spelled out, so something must count them. + + Without this, adding a table to CLEARED_ON_LEAVE and forgetting the + matching DELETE leaves a cooldown alive across a leave and a rejoin -- + invisible until somebody is told to wait for a command they never ran. + """ + _seed(store) + for table in CLEARED_ON_LEAVE: + assert _rows(store, table) > 0 + store.forget_group(GROUP) + for table in CLEARED_ON_LEAVE: + assert _rows(store, table) == 0 + + +def test_forget_group_leaves_the_permanent_tables_alone(store: Store) -> None: + """A transient group listing must not be able to delete years of history.""" + _seed(store) + store.forget_group(GROUP) + assert len(store.suggestions.filed()) == 1 + assert len(store.suggestions.unknown()) == 1 + + +def test_a_version_bump_keeps_the_use_history(tmp_path: Path) -> None: + """A bump drops REBUILT_ON_BUMP and nothing else. + + Written against a file rather than :memory: because the bump is only + observable across two openings of the same database. + """ + path = str(tmp_path / "bump.sqlite3") + with Store(path) as store: + _seed(store) + with sqlite3.connect(path) as raw: + raw.execute(f"PRAGMA user_version = {SCHEMA_VERSION - 1}") + with Store(path) as reopened: + # Thrown away, and cheap to lose. + assert _rows(reopened, "votes") == 0 + assert _rows(reopened, "group_meta") == 0 + # Kept, because nothing recomputes them. + assert _rows(reopened, "command_uses") == 1 + assert _rows(reopened, "daily_uses") == 1 + assert len(reopened.suggestions.filed()) == 1 diff --git a/tests/test_store.py b/tests/test_store.py index 02740f4..073c5cb 100644 --- a/tests/test_store.py +++ b/tests/test_store.py @@ -161,11 +161,17 @@ def test_a_pre_group_database_is_rebuilt(tmp_path: Path) -> None: def test_a_schema_rebuild_never_drops_the_history(tmp_path: Path) -> None: - """The second destructive path the EPHEMERAL/PERMANENT split governs. + """The second destructive path the table split governs. forget_group is the obvious one. This is the other: a future session bumping SCHEMA_VERSION for some cooldown table must not take years of message history with it. + + The use history now survives too, which it did not before command + analytics. daily_uses stopped being disposable the day it became the + only source covering the days that predate command_events, so it left + REBUILT_ON_BUMP. A bump can therefore no longer repair its shape -- + _drop_reshaped is what took that job over. """ db = str(tmp_path / "bump.sqlite3") with open_store(db) as first: @@ -179,7 +185,25 @@ def test_a_schema_rebuild_never_drops_the_history(tmp_path: Path) -> None: with open_store(db) as second: assert counted(second, GROUP) == 1 - assert second.last_used(GROUP, "u1", "cat") is None + assert second.last_used(GROUP, "u1", "cat") == 100.0 + assert second.uses_today(GROUP, "u1", "cat", "2026-08-29") == 1 + + +def test_a_bump_still_clears_a_vote_in_progress(tmp_path: Path) -> None: + """What REBUILT_ON_BUMP still holds is still rebuilt.""" + db = str(tmp_path / "votes.sqlite3") + with open_store(db) as first: + first.add_vote(GROUP, "cat", "u1", 100.0) + first.set_meta(GROUP, "last_reported_day", "2026-08-29") + + aged = sqlite3.connect(db) + aged.execute("PRAGMA user_version = 0") + aged.commit() + aged.close() + + with open_store(db) as second: + assert second.count_votes(GROUP, "cat", 0.0) == 0 + assert second.meta(GROUP, "last_reported_day") is None def test_a_current_database_is_left_alone(tmp_path: Path) -> None: diff --git a/tests/test_suggestions_store.py b/tests/test_suggestions_store.py index 9c62e00..9a34b4c 100644 --- a/tests/test_suggestions_store.py +++ b/tests/test_suggestions_store.py @@ -5,7 +5,7 @@ from __future__ import annotations from typing import TYPE_CHECKING -from signal_bot.schema import EPHEMERAL_TABLES, PERMANENT_TABLES +from signal_bot.schema import CLEARED_ON_LEAVE, PERMANENT_TABLES, REBUILT_ON_BUMP if TYPE_CHECKING: from signal_bot.store import Store @@ -69,7 +69,7 @@ def test_two_people_can_ask_for_the_same_command(store: Store) -> None: def test_leaving_a_group_does_not_discard_the_wish_list(store: Store) -> None: - """forget_group clears the ephemeral tables, and these are not among them. + """forget_group clears CLEARED_ON_LEAVE, and these are not among them. A transient group listing that omits a member is the exact failure discovery._permitted guards against, and it must not be able to delete @@ -84,7 +84,7 @@ def test_leaving_a_group_does_not_discard_the_wish_list(store: Store) -> None: def test_both_tables_are_declared_permanent() -> None: """The property above, asserted where it is actually configured.""" - assert "suggestions" in PERMANENT_TABLES - assert "unknown_commands" in PERMANENT_TABLES - assert "suggestions" not in EPHEMERAL_TABLES - assert "unknown_commands" not in EPHEMERAL_TABLES + for table in ("suggestions", "unknown_commands"): + assert table in PERMANENT_TABLES + assert table not in CLEARED_ON_LEAVE + assert table not in REBUILT_ON_BUMP diff --git a/tests/test_upstream_timing.py b/tests/test_upstream_timing.py new file mode 100644 index 0000000..46e3582 --- /dev/null +++ b/tests/test_upstream_timing.py @@ -0,0 +1,130 @@ +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""Where a command's seconds actually go, recorded as it goes. + +``DOCS-latency.md`` established the numbers that matter by reading three +journal lines by hand, once, and its own rule is that an upstream latency +claim gets measured before it gets designed around. These tests cover the +seam that makes the measurement continuous instead of anecdotal. +""" + +from __future__ import annotations + +from collections.abc import Callable + +import httpx +import pytest + +from signal_bot.analytics import probe +from signal_bot.http_client import ImageFetcher + +IMAGE = b"\x89PNG\r\n\x1a\nfake" + +MockHandler = Callable[[httpx.Request], httpx.Response] + + +def _fetcher(handler: MockHandler) -> ImageFetcher: + return ImageFetcher(httpx.AsyncClient(transport=httpx.MockTransport(handler))) + + +def _ok(_request: httpx.Request) -> httpx.Response: + return httpx.Response(200, content=IMAGE) + + +async def test_a_fetch_records_its_host_and_status() -> None: + """One row per outbound request, per invocation.""" + fetcher = _fetcher(_ok) + with probe.probing() as notes: + await fetcher.fetch_image("https://cdn.example.test/cat.png") + (call,) = notes.calls + assert call.host == "cdn.example.test" + assert call.status == 200 + assert call.ms >= 0 + + +async def test_only_the_host_is_recorded_never_the_path() -> None: + """A URL carries identifiers; item_id is where those are governed. + + The poem URL carries a Wolne Lektury slug, which is exactly the kind of + thing that must be a deliberate column rather than a side effect of + timing a request. + """ + fetcher = _fetcher(_ok) + with probe.probing() as notes: + await fetcher.fetch_image("https://wolnelektury.test/lektura/some-slug.txt") + (call,) = notes.calls + assert call.host == "wolnelektury.test" + assert "slug" not in repr(call) + + +async def test_a_json_fetch_and_its_image_are_two_calls() -> None: + """fetch_base64 is a lookup and a download, and both cost time.""" + + def handler(request: httpx.Request) -> httpx.Response: + if "search" in str(request.url): + return httpx.Response(200, json=[{"url": "https://cdn.example.test/cat.png"}]) + return httpx.Response(200, content=IMAGE) + + fetcher = _fetcher(handler) + with probe.probing() as notes: + await fetcher.fetch_base64("https://api.example.test/search", [0, "url"]) + assert [call.host for call in notes.calls] == [ + "api.example.test", + "cdn.example.test", + ] + + +async def test_a_bad_status_is_recorded_and_still_raises() -> None: + """Upstream answering badly is data; it must not become silence.""" + fetcher = _fetcher(lambda _request: httpx.Response(503)) + with probe.probing() as notes, pytest.raises(httpx.HTTPStatusError): + await fetcher.fetch_image("https://cdn.example.test/cat.png") + (call,) = notes.calls + assert call.status == 503 + + +async def test_a_request_that_never_answered_is_recorded_as_zero() -> None: + """DNS, connect and timeout failures never reach a status code. + + Losing them would make an upstream that is entirely down look merely + unused, which is the opposite of what the health panel is for. + """ + + refused = "connection refused" + + def refuse(request: httpx.Request) -> httpx.Response: + raise httpx.ConnectError(refused, request=request) + + fetcher = _fetcher(refuse) + with probe.probing() as notes, pytest.raises(httpx.ConnectError): + await fetcher.fetch_image("https://cdn.example.test/cat.png") + (call,) = notes.calls + assert call.status == probe.NO_RESPONSE + + +async def test_a_fetch_outside_a_probe_records_nothing() -> None: + """The offline runs and most tests drive the fetcher with no dispatcher.""" + fetcher = _fetcher(_ok) + await fetcher.fetch_image("https://cdn.example.test/cat.png") + assert probe.CURRENT.get() is None + + +async def test_a_send_is_timed_even_when_it_fails() -> None: + """A failed send still cost the seconds it took to fail.""" + + async def boom() -> None: + async with probe.timed_send(): + raise RuntimeError + + with probe.probing() as notes, pytest.raises(RuntimeError): + await boom() + assert notes.send_ms >= 0 + + +async def test_two_sends_accumulate() -> None: + """!pułapka sends a picture and then its description.""" + with probe.probing() as notes: + async with probe.timed_send(): + pass + async with probe.timed_send(): + pass + assert notes.send_ms >= 0 diff --git a/tool/backfill_analytics.py b/tool/backfill_analytics.py new file mode 100755 index 0000000..9d866b4 --- /dev/null +++ b/tool/backfill_analytics.py @@ -0,0 +1,89 @@ +#!/usr/bin/env python3 +# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence. +"""Import the command history that predates the table holding it. + + python tool/backfill_analytics.py --db signal_bot.sqlite3 --dry-run + python tool/backfill_analytics.py --db signal_bot.sqlite3 + +Two sources, joined so that neither double-counts the other -- see +``signal_bot.analytics.backfill`` for why, which is the part worth reading +before trusting the numbers this prints. + +The logic lives in ``signal_bot/``, where the 100% coverage gate can see it. +This file is the acquisition and the argparse, and lives in ``tool/``, which +is outside ``--cov`` and covered instead by ``tests/test_tool_imports.py``. + +``--db`` is required and has no default, for the reason ManageDeps gives: +this writes to a database that holds a real group's history, and a default +is how the wrong one gets written to. + +Run it against a copy first. It is insert-only and re-runnable, but "I ran +it on the live file to see what it would do" is not a thing to want back. +""" + +from __future__ import annotations + +import argparse +import sys +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) + +from signal_bot.analytics import backfill, journal +from signal_bot.external import run +from signal_bot.store import open_store + +UNIT = "signal-bot.service" + + +def read_journal(path: str | None) -> list[str]: + """The bot's log, from a file or from journald itself. + + Shells out through ``signal_bot.external``, the one module in this + repository allowed to start a process. ``tests/test_external.py`` keeps + that list closed by grepping every file for the import itself, so this + one routes through the sanctioned seam rather than joining the list -- + and note that naming the import in prose is enough to trip that grep. + """ + if path is not None: + return Path(path).read_text(encoding="utf-8").splitlines() + result = run( + ["journalctl", "--user", "-u", UNIT, "--no-pager", "-o", "short-iso"], + timeout=120.0, + ) + if not result.ok: + sys.stderr.write(f"could not read the journal: {result.stderr.strip()}\n") + return [] + return result.stdout.splitlines() + + +def main(argv: list[str] | None = None) -> int: + """Parse arguments, run one pass, and print what it did.""" + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--db", required=True, help="the database to write to") + parser.add_argument("--log", help="a captured journal, instead of reading journald") + parser.add_argument( + "--dry-run", + action="store_true", + help="report what would be imported, and write nothing", + ) + args = parser.parse_args(argv) + + lines = read_journal(args.log) + sys.stdout.write(f"journal lines read: {len(lines)}\n") + records = list(journal.parse(lines)) + sys.stdout.write(f"command lines found: {len(records)}\n") + + with open_store(args.db) as store: + report, events = backfill.plan(store, records) + if args.dry_run: + sys.stdout.write("-- dry run, nothing written --\n") + else: + backfill.apply(store, events) + for line in report.lines(): + sys.stdout.write(f"{line}\n") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/web/package.json b/web/package.json index 49aa5f7..2bc313a 100644 --- a/web/package.json +++ b/web/package.json @@ -3,7 +3,10 @@ "private": true, "type": "module", "packageManager": "pnpm@11.24.0", - "engines": { "node": ">=24", "pnpm": ">=10" }, + "engines": { + "node": ">=24", + "pnpm": ">=10" + }, "scripts": { "dev": "vite", "build": "tsc -b && vite build", @@ -14,6 +17,11 @@ }, "dependencies": { "@kuhyx/web-ui": "github:kuhyx/utils#web_ui-v0.3.1&path:/web_ui", + "@nivo/bar": "0.99.0", + "@nivo/calendar": "0.99.0", + "@nivo/core": "0.99.0", + "@nivo/heatmap": "0.99.0", + "@nivo/line": "0.99.0", "react": "19.2.8", "react-dom": "19.2.8" }, diff --git a/web/pnpm-lock.yaml b/web/pnpm-lock.yaml index cb61365..7197a2f 100644 --- a/web/pnpm-lock.yaml +++ b/web/pnpm-lock.yaml @@ -11,6 +11,21 @@ importers: '@kuhyx/web-ui': specifier: github:kuhyx/utils#web_ui-v0.3.1&path:/web_ui version: https://codeload.github.com/kuhyx/utils/tar.gz/6714d131983f05a0e67f55ac836e3e8d57045425#path:/web_ui(react@19.2.8) + '@nivo/bar': + specifier: 0.99.0 + version: 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/calendar': + specifier: 0.99.0 + version: 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/core': + specifier: 0.99.0 + version: 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/heatmap': + specifier: 0.99.0 + version: 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/line': + specifier: 0.99.0 + version: 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) react: specifier: 19.2.8 version: 19.2.8 @@ -299,9 +314,107 @@ packages: peerDependencies: react: '>=19' + '@nivo/annotations@0.99.0': + resolution: {integrity: sha512-jCuuXPbvpaqaz4xF7k5dv0OT2ubn5Nt0gWryuTe/8oVsC/9bzSuK8bM9vBty60m9tfO+X8vUYliuaCDwGksC2g==} + peerDependencies: + react: ^16.14 || ^17.0 || ^18.0 || ^19.0 + + '@nivo/axes@0.99.0': + resolution: {integrity: sha512-3KschnmEL0acRoa7INSSOSEFwJLm54aZwSev7/r8XxXlkgRBriu6ReZy/FG0wfN+ljZ4GMvx+XyIIf6kxzvrZg==} + peerDependencies: + react: ^16.14 || ^17.0 || ^18.0 || ^19.0 + + '@nivo/bar@0.99.0': + resolution: {integrity: sha512-9yfMn7H6UF/TqtCwVZ/vihVAXUff9wWvSaeF2Z1DCfgr5S07qs31Qb2p0LZA+YgCWpaU7zqkeb3VZ4WCpZbrDA==} + peerDependencies: + react: ^16.14 || ^17.0 || ^18.0 || ^19.0 + + '@nivo/calendar@0.99.0': + resolution: {integrity: sha512-8Y6rIjAMt4hMWpkIUmb7skr8boAdoE7Bb4liAFSkkyRgsK4iFnFbaf1x5wLk7YmyY4v8LZ4PjNyGOYPBJU2wtw==} + peerDependencies: + react: ^16.14 || ^17.0 || ^18.0 || ^19.0 + + '@nivo/canvas@0.99.0': + resolution: {integrity: sha512-UxA8zb+NPwqmNm81hoyUZSMAikgjU1ukLf4KybVNyV8ejcJM+BUFXsb8DxTcLdt4nmCFHqM56GaJQv2hnAHmzg==} + + '@nivo/colors@0.99.0': + resolution: {integrity: sha512-hyYt4lEFIfXOUmQ6k3HXm3KwhcgoJpocmoGzLUqzk7DzuhQYJo+4d5jIGGU0N/a70+9XbHIdpKNSblHAIASD3w==} + peerDependencies: + react: ^16.14 || ^17.0 || ^18.0 || ^19.0 + + '@nivo/core@0.99.0': + resolution: {integrity: sha512-olCItqhPG3xHL5ei+vg52aB6o+6S+xR2idpkd9RormTTUniZb8U2rOdcQojOojPY5i9kVeQyLFBpV4YfM7OZ9g==} + peerDependencies: + react: ^16.14 || ^17.0 || ^18.0 || ^19.0 + + '@nivo/heatmap@0.99.0': + resolution: {integrity: sha512-cnd5V6XYLvHtQ8GObUyYyEgmAUa5/ERYVmNXR1znA+yzmB+tGthJsOeGar+ntCb43pIL5HbhEaD3YpsYzBxOhQ==} + peerDependencies: + react: ^16.14 || ^17.0 || ^18.0 || ^19.0 + + '@nivo/legends@0.99.0': + resolution: {integrity: sha512-P16FjFqNceuTTZphINAh5p0RF0opu3cCKoWppe2aRD9IuVkvRm/wS5K1YwMCxDzKyKh5v0AuTlu9K6o3/hk8hA==} + peerDependencies: + react: ^16.14 || ^17.0 || ^18.0 || ^19.0 + + '@nivo/line@0.99.0': + resolution: {integrity: sha512-bAqTXSjpnpcGMs341qWFUi7hJTqQiNoSeJHsYPuPS3icuXPcp3WETQH+zRZACeEF79ZigeOWCW+dzODgne1y9w==} + peerDependencies: + react: ^16.14 || ^17.0 || ^18.0 || ^19.0 + + '@nivo/scales@0.99.0': + resolution: {integrity: sha512-g/2K4L6L8si6E2BWAHtFVGahtDKbUcO6xHJtlIZMwdzaJc7yB16EpWLK8AfI/A42KadLhJSJqBK3mty+c7YZ+w==} + + '@nivo/text@0.99.0': + resolution: {integrity: sha512-ho3oZpAZApsJNjsIL5WJSAdg/wjzTBcwo1KiHBlRGUmD+yUWO8qp7V+mnYRhJchwygtRVALlPgZ/rlcW2Xr/MQ==} + peerDependencies: + react: ^16.14 || ^17.0 || ^18.0 || ^19.0 + + '@nivo/theming@0.99.0': + resolution: {integrity: sha512-KvXlf0nqBzh/g2hAIV9bzscYvpq1uuO3TnFN3RDXGI72CrbbZFTGzprPju3sy/myVsauv+Bb+V4f5TZ0jkYKRg==} + peerDependencies: + react: ^16.14 || ^17.0 || ^18.0 || ^19.0 + + '@nivo/tooltip@0.99.0': + resolution: {integrity: sha512-weoEGR3xAetV4k2P6k96cdamGzKQ5F2Pq+uyDaHr1P3HYArM879Pl+x+TkU0aWjP6wgUZPx/GOBiV1Hb1JxIqg==} + peerDependencies: + react: ^16.14 || ^17.0 || ^18.0 || ^19.0 + + '@nivo/voronoi@0.99.0': + resolution: {integrity: sha512-KfmMdidbYzhiUCki1FG4X4nHEFT4loK8G5bMBnmCl9U+S78W+gvkfrgD2Aoqp/Q9yKQvr3Y8UcZKSFZnn3HgjQ==} + peerDependencies: + react: ^16.14 || ^17.0 || ^18.0 || ^19.0 + '@oxc-project/types@0.147.0': resolution: {integrity: sha512-IJ3s6ltHLp45S0bh7phkX+gJO7A1Wuz2EaqpAhb8WjqDwbzMiWKHhyyT42tskaWjEYXtHtVCPpnBJVT9+dcRLg==} + '@react-spring/animated@10.1.2': + resolution: {integrity: sha512-yAsQ/bbp6+vko7WNCI1M00c6KLE9XKTGCrgRhQqS4JcK3oF5qBV4rHYrQEprvEvYXxt+1H5FsLS2eVValEPfFw==} + peerDependencies: + react: ^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0 + + '@react-spring/core@10.1.2': + resolution: {integrity: sha512-lPGOAg0V+PV3ucopOajD+YCxoe8twALqAeG4c/+sqVjBsyUNNdx8qfz/DcNSPKA8PV3+AyQELlCxlDfap9cmBQ==} + peerDependencies: + react: ^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0 + + '@react-spring/rafz@10.1.2': + resolution: {integrity: sha512-KC6vSFZyPnRJ2rXqipV9QqR4SaYbYXjvKfdYKWipNK2mWuV79gr20WmpVUOsTiEHGRY3WSOdWCHN+P9Pdaqb7Q==} + + '@react-spring/shared@10.1.2': + resolution: {integrity: sha512-47/8bNQ/o0uEmxEnPBuERlC29VSqiTn5P9ln9tUMSVkYKYxBtouYE686F5kAGj+eHRPoNCw7drxWE9nv2d1LMw==} + peerDependencies: + react: ^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0 + + '@react-spring/types@10.1.2': + resolution: {integrity: sha512-G4CWowmVPz+rDG1y9QRVq/prZNxiNwQaHC0kgT/MgY6jAGgdRgTV4VThpvJDwWvUitk3xZB4soP4d36fjeQ09g==} + + '@react-spring/web@10.1.2': + resolution: {integrity: sha512-KxDB3zaDqy9qFsu7fdxjyraAxweHH4k5TW5WGT/OuMK6hKaxSDfhrQfRBzfFaSVvMBf+NghbJFanllV4ia6i7A==} + peerDependencies: + react: ^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0 + react-dom: ^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0 + '@rolldown/binding-android-arm-eabi@1.2.6': resolution: {integrity: sha512-b+jTcARdTiFLI6jB4a5XjTm0RWd6KcRfQj/I2356fxUZemiho9zQLxo0RtCuMDAyKcLo6cEltkgbQp6d1+sjjQ==} engines: {node: ^20.19.0 || >=22.12.0} @@ -445,6 +558,42 @@ packages: '@types/chai@5.2.3': resolution: {integrity: sha512-Mw558oeA9fFbv65/y4mHtXDs9bPnFMZAL/jxdPFUpOHHIXX91mcgEHbS5Lahr+pwZFR8A7GQleRWeI6cGFC2UA==} + '@types/d3-color@3.1.3': + resolution: {integrity: sha512-iO90scth9WAbmgv7ogoq57O9YpKmFBbmoEoCHDB2xMBY0+/KVrqAaCDyCE16dUspeOvIxFFRI+0sEtqDqy2b4A==} + + '@types/d3-delaunay@6.0.4': + resolution: {integrity: sha512-ZMaSKu4THYCU6sV64Lhg6qjf1orxBthaC161plr5KuPHo3CNm8DTHiLw/5Eq2b6TsNP0W0iJrUOFscY6Q450Hw==} + + '@types/d3-format@1.4.5': + resolution: {integrity: sha512-mLxrC1MSWupOSncXN/HOlWUAAIffAEBaI4+PKy2uMPsKe4FNZlk7qrbTjmzJXITQQqBHivaks4Td18azgqnotA==} + + '@types/d3-interpolate@3.0.4': + resolution: {integrity: sha512-mgLPETlrpVV1YRJIglr4Ez47g7Yxjl1lj7YKsiMCb27VJH9W8NVM6Bb9d8kkpG/uAQS5AmbA48q2IAolKKo1MA==} + + '@types/d3-path@3.1.1': + resolution: {integrity: sha512-VMZBYyQvbGmWyWVea0EHs/BwLgxc+MKi1zLDCONksozI4YJMcTt8ZEuIR4Sb1MMTE8MMW49v0IwI5+b7RmfWlg==} + + '@types/d3-scale-chromatic@3.1.0': + resolution: {integrity: sha512-iWMJgwkK7yTRmWqRB5plb1kadXyQ5Sj8V/zYlFGMUBbIPKQScw+Dku9cAAMgJG+z5GYDoMjWGLVOvjghDEFnKQ==} + + '@types/d3-scale@4.0.9': + resolution: {integrity: sha512-dLmtwB8zkAeO/juAMfnV+sItKjlsw2lKdZVVy6LRr0cBmegxSABiLEpGVmSJJ8O08i4+sGR6qQtb6WtuwJdvVw==} + + '@types/d3-shape@3.2.0': + resolution: {integrity: sha512-kVd74ta9eof3eJOvbNd1vGKS/XERRyQbT26Og63hIsvDO84cjD5gEOhsXf26w3FSoNlPVz84DOFcKv/oou+fMw==} + + '@types/d3-time-format@2.3.4': + resolution: {integrity: sha512-xdDXbpVO74EvadI3UDxjxTdR6QIxm1FKzEA/+F8tL4GWWUg/hgvBqf6chql64U5A9ZUGWo7pEu4eNlyLwbKdhg==} + + '@types/d3-time-format@3.0.4': + resolution: {integrity: sha512-or9DiDnYI1h38J9hxKEsw513+KVuFbEVhl7qdxcaudoiqWWepapUen+2vAriFGexr6W5+P4l9+HJrB39GG+oRg==} + + '@types/d3-time@1.1.4': + resolution: {integrity: sha512-JIvy2HjRInE+TXOmIGN5LCmeO0hkFZx5f9FZ7kiN+D+YTcc8pptsiLiuHsvwxwC7VVKmJ2ExHUgNlAiV7vQM9g==} + + '@types/d3-time@3.0.4': + resolution: {integrity: sha512-yuzZug1nkAAaBlBBikKZTgzCeA+k1uy4ZFwWANOfKw5z5LRhV0gNA7gNkKm7HoK+HRN0wX3EkxGk0fpbWhmB7g==} + '@types/deep-eql@4.0.2': resolution: {integrity: sha512-c9h9dVVMigMPc4bwTvC5dxqtqJZwQPePsWjPlpSOnojbor6pGqdk541lfA7AqFQr5pB1BRdq0juY9db81BwyFw==} @@ -688,6 +837,59 @@ packages: csstype@3.2.3: resolution: {integrity: sha512-z1HGKcYy2xA8AGQfwrn0PAy+PB7X/GSj3UVJW9qKyn43xWa+gl5nXmU4qqLMRzWVLFC8KusUX8T/0kCiOYpAIQ==} + d3-array@3.2.4: + resolution: {integrity: sha512-tdQAmyA18i4J7wprpYq8ClcxZy3SC31QMeByyCFyRt7BVHdREQZ5lpzoe5mFEYZUWe+oq8HBvk9JjpibyEV4Jg==} + engines: {node: '>=12'} + + d3-color@3.1.0: + resolution: {integrity: sha512-zg/chbXyeBtMQ1LbD/WSoW2DpC3I0mpmPdW+ynRTj/x2DAWYrIY7qeZIHidozwV24m4iavr15lNwIwLxRmOxhA==} + engines: {node: '>=12'} + + d3-delaunay@6.0.4: + resolution: {integrity: sha512-mdjtIZ1XLAM8bm/hx3WwjfHt6Sggek7qH043O8KEjDXN40xi3vx/6pYSVTwLjEgiXQTbvaouWKynLBiUZ6SK6A==} + engines: {node: '>=12'} + + d3-format@1.4.5: + resolution: {integrity: sha512-J0piedu6Z8iB6TbIGfZgDzfXxUFN3qQRMofy2oPdXzQibYGqPB/9iMcxr/TGalU+2RsyDO+U4f33id8tbnSRMQ==} + + d3-format@3.1.2: + resolution: {integrity: sha512-AJDdYOdnyRDV5b6ArilzCPPwc1ejkHcoyFarqlPqT7zRYjhavcT3uSrqcMvsgh2CgoPbK3RCwyHaVyxYcP2Arg==} + engines: {node: '>=12'} + + d3-interpolate@3.0.1: + resolution: {integrity: sha512-3bYs1rOD33uo8aqJfKP3JWPAibgw8Zm2+L9vBKEHJ2Rg+viTR7o5Mmv5mZcieN+FRYaAOWX5SJATX6k1PWz72g==} + engines: {node: '>=12'} + + d3-path@3.1.0: + resolution: {integrity: sha512-p3KP5HCf/bvjBSSKuXid6Zqijx7wIfNW+J/maPs+iwR35at5JCbLUT0LzF1cnjbCHWhqzQTIN2Jpe8pRebIEFQ==} + engines: {node: '>=12'} + + d3-scale-chromatic@3.1.0: + resolution: {integrity: sha512-A3s5PWiZ9YCXFye1o246KoscMWqf8BsD9eRiJ3He7C9OBaxKhAd5TFCdEx/7VbKtxxTsu//1mMJFrEt572cEyQ==} + engines: {node: '>=12'} + + d3-scale@4.0.2: + resolution: {integrity: sha512-GZW464g1SH7ag3Y7hXjf8RoUuAFIqklOAq3MRl4OaWabTFJY9PN/E1YklhXLh+OQ3fM9yS2nOkCoS+WLZ6kvxQ==} + engines: {node: '>=12'} + + d3-shape@3.2.0: + resolution: {integrity: sha512-SaLBuwGm3MOViRq2ABk3eLoxwZELpH6zhl3FbAoJ7Vm1gofKx6El1Ib5z23NUEhF9AsGl7y+dzLe5Cw2AArGTA==} + engines: {node: '>=12'} + + d3-time-format@3.0.0: + resolution: {integrity: sha512-UXJh6EKsHBTjopVqZBhFysQcoXSv/5yLONZvkQ5Kk3qbwiUYkdX17Xa1PT6U1ZWXGGfB1ey5L8dKMlFq2DO0Ag==} + + d3-time-format@4.1.0: + resolution: {integrity: sha512-dJxPBlzC7NugB2PDLwo9Q8JiTR3M3e4/XANkreKSUxF8vvXKqm1Yfq4Q5dl8budlunRVlUUaDUgFt7eA8D6NLg==} + engines: {node: '>=12'} + + d3-time@1.1.0: + resolution: {integrity: sha512-Xh0isrZ5rPYYdqhAVk8VLnMEidhz5aP7htAADH6MfzgmmicPkTo8LhkLxci61/lCB7n7UmE3bN0leRt+qvkLxA==} + + d3-time@3.1.0: + resolution: {integrity: sha512-VqKjzBLejbSMT4IgbmVgDjpkYrNWUYJnbCGo874u7MMKIWsILRX+OpX/gTk8MqjpT1A/c6HY2dCA77ZN0lkQ2Q==} + engines: {node: '>=12'} + data-urls@7.0.0: resolution: {integrity: sha512-23XHcCF+coGYevirZceTVD7NdJOqVn+49IHyxgszm+JIiHLoB2TkmPtsYkNWT1pvRSGkc35L6NHs0yHkN2SumA==} engines: {node: ^20.19.0 || ^22.12.0 || >=24.0.0} @@ -707,6 +909,9 @@ packages: deep-is@0.1.4: resolution: {integrity: sha512-oIPzksmTg4/MriiaYGO+okXDT7ztn/w3Eptv/+gSIdMdKsJo0u4CfYNFJPy+4SKMuCqGw2wxnA+URMg3t8a/bQ==} + delaunator@5.1.0: + resolution: {integrity: sha512-AGrQ4QSgssa1NGmWmLPqN5NY2KajF5MqxetNEO+o0n3ZwZZeTmt7bBnvzHWrmkZFxGgr4HdyFgelzgi06otLuQ==} + dequal@2.0.3: resolution: {integrity: sha512-0je+qPKHEMohvfRTCEo3CrPG6cAzAYgmzKyxRiYSSDkS6eGJdyVJm7WaYA5ECaAD9wLB2T4EEeymA5aFVcYXCA==} engines: {node: '>=6'} @@ -917,6 +1122,10 @@ packages: resolution: {integrity: sha512-m6FAo/spmsW2Ab2fU35JTYwtOKa2yAwXSwgjSv1TJzh4Mh7mC3lzAOVLBprb72XsTrgkEIsl7YrFNAiDiRhIGg==} engines: {node: '>=12'} + internmap@2.0.3: + resolution: {integrity: sha512-5Hh7Y1wQbvY5ooGgPbDaL5iYLAPzMTUrjMulskHLH6wnv/A+1q5rgEaiuqEjB+oxGXIVZs1FF+R/KPN3ZSQYYg==} + engines: {node: '>=12'} + is-builtin-module@5.0.0: resolution: {integrity: sha512-f4RqJKBUe5rQkJ2eJEJBXSticB3hGbN9j0yxxMQFqIW89Jp9WYFtzfTcRlstDKVUTRzSOTLKRfO9vIztenwtxA==} engines: {node: '>=18.20'} @@ -1077,6 +1286,9 @@ packages: lodash.merge@4.6.2: resolution: {integrity: sha512-0KpjqXRVvrYyCsX1swR/XTK0va6VQkQM6MNo7PqW77ByjAhoARA8EfrP1N4+KlKj8YS0ZUCtRT/YUuhyYDujIQ==} + lodash@4.18.1: + resolution: {integrity: sha512-dMInicTPVE8d1e5otfwmmjlxkZoUpiVLwyeTdUsi/Caj/gfzzblBcCE5sRHV/AsjuCmxWrte2TNGSYuCeCq+0Q==} + lru-cache@11.5.2: resolution: {integrity: sha512-4pfM1Ff0x50o0tQwb5ucw/RzNyD0/YJME6IVcStalZuMWxdt3sR3huStTtxz4PUmvZfRguvDejasvQ2kifR11g==} engines: {node: 20 || >=22} @@ -1212,6 +1424,12 @@ packages: react-is@17.0.2: resolution: {integrity: sha512-w2GsyukL62IJnlaff/nRegPQR94C/XXamvMWmSHRJ4y7Ts/4ocGRmTHvOs8PSE6pB3dWOrD/nueuU5sduBsQ4w==} + react-virtualized-auto-sizer@1.0.26: + resolution: {integrity: sha512-CblNyiNVw2o+hsa5/49NH2ogGxZ+t+3aweRvNSq7TVjDIlwk7ir4lencEg5HxHeSzwNarSkNkiu0qJSOXtxm5A==} + peerDependencies: + react: ^15.3.0 || ^16.0.0-alpha || ^17.0.0 || ^18.0.0 || ^19.0.0 + react-dom: ^15.3.0 || ^16.0.0-alpha || ^17.0.0 || ^18.0.0 || ^19.0.0 + react@19.2.8: resolution: {integrity: sha512-PWaYA1L/q9u2u7xYQi+Y3L3Yfnie7XyLeaJICV1MGD6LprsBxcAqGjYyr0eY3p+QdsA+x/Irkt4Qif8D63+Sbw==} engines: {node: '>=0.10.0'} @@ -1240,6 +1458,9 @@ packages: resolution: {integrity: sha512-yE7KUfFvaBFzGPs5H3Ops1RevfUEsDc5Iz65rOwWg4lE8HJSYtle77uul3+573457oHvBKuHYDl/xqUkKpEEdw==} engines: {node: '>=18'} + robust-predicates@3.0.3: + resolution: {integrity: sha512-NS3levdsRIUOmiJ8FZWCP7LG3QpJyrs/TE0Zpf1yvZu8cAJJ6QMW92H1c7kWpdIHo8RvmLxN/o2JXTKHp74lUA==} + rolldown@1.2.6: resolution: {integrity: sha512-vMM4q3aixf46GiF1Kok8jDPFsEpXgFWGjUHXNkNHNm+Y2adXAG2dbX91jkti3i0ZRsOlcmbuzAz1poObSHCmUA==} engines: {node: ^20.19.0 || >=22.12.0} @@ -1381,6 +1602,12 @@ packages: uri-js@4.4.1: resolution: {integrity: sha512-7rKUyy33Q1yc98pQ1DAmLtwX109F7TIfWlW1Ydo8Wl1ii1SeHieeh0HHfPeL2fMXK6z0s8ecKs9frCuLJvndBg==} + use-debounce@10.1.1: + resolution: {integrity: sha512-kvds8BHR2k28cFsxW8k3nc/tRga2rs1RHYCqmmGqb90MEeE++oALwzh2COiuBLO1/QXiOuShXoSN2ZpWnMmvuQ==} + engines: {node: '>= 16.0.0'} + peerDependencies: + react: '*' + vite@8.2.2: resolution: {integrity: sha512-cFKLV/PRgAUlIRm5WjMjJ86jrftzpqcgH+Us+DS8mI3CDNiH30Whrz8uHL3+MOLPAgqbMBAqWdAHAphOAM+z/Q==} engines: {node: ^20.19.0 || >=22.12.0} @@ -1761,8 +1988,241 @@ snapshots: dependencies: react: 19.2.8 + '@nivo/annotations@0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)': + dependencies: + '@nivo/colors': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/core': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/theming': 0.99.0(react@19.2.8) + '@react-spring/web': 10.1.2(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + lodash: 4.18.1 + react: 19.2.8 + transitivePeerDependencies: + - react-dom + + '@nivo/axes@0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)': + dependencies: + '@nivo/core': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/scales': 0.99.0 + '@nivo/text': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/theming': 0.99.0(react@19.2.8) + '@react-spring/web': 10.1.2(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@types/d3-format': 1.4.5 + '@types/d3-time-format': 2.3.4 + d3-format: 1.4.5 + d3-time-format: 3.0.0 + react: 19.2.8 + transitivePeerDependencies: + - react-dom + + '@nivo/bar@0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)': + dependencies: + '@nivo/annotations': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/axes': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/canvas': 0.99.0 + '@nivo/colors': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/core': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/legends': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/scales': 0.99.0 + '@nivo/text': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/theming': 0.99.0(react@19.2.8) + '@nivo/tooltip': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@react-spring/web': 10.1.2(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@types/d3-scale': 4.0.9 + '@types/d3-shape': 3.2.0 + d3-scale: 4.0.2 + d3-shape: 3.2.0 + lodash: 4.18.1 + react: 19.2.8 + transitivePeerDependencies: + - react-dom + + '@nivo/calendar@0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)': + dependencies: + '@nivo/core': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/legends': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/text': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/theming': 0.99.0(react@19.2.8) + '@nivo/tooltip': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@types/d3-scale': 4.0.9 + '@types/d3-time': 1.1.4 + '@types/d3-time-format': 3.0.4 + d3-scale: 4.0.2 + d3-time: 1.1.0 + d3-time-format: 3.0.0 + lodash: 4.18.1 + react: 19.2.8 + transitivePeerDependencies: + - react-dom + + '@nivo/canvas@0.99.0': {} + + '@nivo/colors@0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)': + dependencies: + '@nivo/core': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/theming': 0.99.0(react@19.2.8) + '@types/d3-color': 3.1.3 + '@types/d3-scale': 4.0.9 + '@types/d3-scale-chromatic': 3.1.0 + d3-color: 3.1.0 + d3-scale: 4.0.2 + d3-scale-chromatic: 3.1.0 + lodash: 4.18.1 + react: 19.2.8 + transitivePeerDependencies: + - react-dom + + '@nivo/core@0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)': + dependencies: + '@nivo/theming': 0.99.0(react@19.2.8) + '@nivo/tooltip': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@react-spring/web': 10.1.2(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@types/d3-shape': 3.2.0 + d3-color: 3.1.0 + d3-format: 1.4.5 + d3-interpolate: 3.0.1 + d3-scale: 4.0.2 + d3-scale-chromatic: 3.1.0 + d3-shape: 3.2.0 + d3-time-format: 3.0.0 + lodash: 4.18.1 + react: 19.2.8 + react-virtualized-auto-sizer: 1.0.26(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + use-debounce: 10.1.1(react@19.2.8) + transitivePeerDependencies: + - react-dom + + '@nivo/heatmap@0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)': + dependencies: + '@nivo/annotations': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/axes': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/colors': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/core': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/legends': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/scales': 0.99.0 + '@nivo/text': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/theming': 0.99.0(react@19.2.8) + '@nivo/tooltip': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@react-spring/core': 10.1.2(react@19.2.8) + '@react-spring/web': 10.1.2(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@types/d3-scale': 4.0.9 + d3-scale: 4.0.2 + react: 19.2.8 + transitivePeerDependencies: + - react-dom + + '@nivo/legends@0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)': + dependencies: + '@nivo/colors': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/core': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/text': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/theming': 0.99.0(react@19.2.8) + '@types/d3-scale': 4.0.9 + d3-scale: 4.0.2 + react: 19.2.8 + transitivePeerDependencies: + - react-dom + + '@nivo/line@0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)': + dependencies: + '@nivo/annotations': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/axes': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/colors': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/core': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/legends': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/scales': 0.99.0 + '@nivo/theming': 0.99.0(react@19.2.8) + '@nivo/tooltip': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/voronoi': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@react-spring/web': 10.1.2(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@types/d3-shape': 3.2.0 + d3-shape: 3.2.0 + react: 19.2.8 + transitivePeerDependencies: + - react-dom + + '@nivo/scales@0.99.0': + dependencies: + '@types/d3-interpolate': 3.0.4 + '@types/d3-scale': 4.0.9 + '@types/d3-time': 1.1.4 + '@types/d3-time-format': 3.0.4 + d3-interpolate: 3.0.1 + d3-scale: 4.0.2 + d3-time: 1.1.0 + d3-time-format: 3.0.0 + lodash: 4.18.1 + + '@nivo/text@0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)': + dependencies: + '@nivo/core': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/theming': 0.99.0(react@19.2.8) + '@react-spring/web': 10.1.2(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + react: 19.2.8 + transitivePeerDependencies: + - react-dom + + '@nivo/theming@0.99.0(react@19.2.8)': + dependencies: + lodash: 4.18.1 + react: 19.2.8 + + '@nivo/tooltip@0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)': + dependencies: + '@nivo/core': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/theming': 0.99.0(react@19.2.8) + '@react-spring/web': 10.1.2(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + react: 19.2.8 + transitivePeerDependencies: + - react-dom + + '@nivo/voronoi@0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8)': + dependencies: + '@nivo/core': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@nivo/theming': 0.99.0(react@19.2.8) + '@nivo/tooltip': 0.99.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + '@types/d3-delaunay': 6.0.4 + '@types/d3-scale': 4.0.9 + d3-delaunay: 6.0.4 + d3-scale: 4.0.2 + react: 19.2.8 + transitivePeerDependencies: + - react-dom + '@oxc-project/types@0.147.0': {} + '@react-spring/animated@10.1.2(react@19.2.8)': + dependencies: + '@react-spring/shared': 10.1.2(react@19.2.8) + '@react-spring/types': 10.1.2 + react: 19.2.8 + + '@react-spring/core@10.1.2(react@19.2.8)': + dependencies: + '@react-spring/animated': 10.1.2(react@19.2.8) + '@react-spring/shared': 10.1.2(react@19.2.8) + '@react-spring/types': 10.1.2 + react: 19.2.8 + + '@react-spring/rafz@10.1.2': {} + + '@react-spring/shared@10.1.2(react@19.2.8)': + dependencies: + '@react-spring/rafz': 10.1.2 + '@react-spring/types': 10.1.2 + react: 19.2.8 + + '@react-spring/types@10.1.2': {} + + '@react-spring/web@10.1.2(react-dom@19.2.8(react@19.2.8))(react@19.2.8)': + dependencies: + '@react-spring/animated': 10.1.2(react@19.2.8) + '@react-spring/core': 10.1.2(react@19.2.8) + '@react-spring/shared': 10.1.2(react@19.2.8) + '@react-spring/types': 10.1.2 + csstype: 3.2.3 + react: 19.2.8 + react-dom: 19.2.8(react@19.2.8) + '@rolldown/binding-android-arm-eabi@1.2.6': optional: true @@ -1856,6 +2316,36 @@ snapshots: '@types/deep-eql': 4.0.2 assertion-error: 2.0.1 + '@types/d3-color@3.1.3': {} + + '@types/d3-delaunay@6.0.4': {} + + '@types/d3-format@1.4.5': {} + + '@types/d3-interpolate@3.0.4': + dependencies: + '@types/d3-color': 3.1.3 + + '@types/d3-path@3.1.1': {} + + '@types/d3-scale-chromatic@3.1.0': {} + + '@types/d3-scale@4.0.9': + dependencies: + '@types/d3-time': 3.0.4 + + '@types/d3-shape@3.2.0': + dependencies: + '@types/d3-path': 3.1.1 + + '@types/d3-time-format@2.3.4': {} + + '@types/d3-time-format@3.0.4': {} + + '@types/d3-time@1.1.4': {} + + '@types/d3-time@3.0.4': {} + '@types/deep-eql@4.0.2': {} '@types/esrecurse@4.3.1': {} @@ -2115,6 +2605,57 @@ snapshots: csstype@3.2.3: {} + d3-array@3.2.4: + dependencies: + internmap: 2.0.3 + + d3-color@3.1.0: {} + + d3-delaunay@6.0.4: + dependencies: + delaunator: 5.1.0 + + d3-format@1.4.5: {} + + d3-format@3.1.2: {} + + d3-interpolate@3.0.1: + dependencies: + d3-color: 3.1.0 + + d3-path@3.1.0: {} + + d3-scale-chromatic@3.1.0: + dependencies: + d3-color: 3.1.0 + d3-interpolate: 3.0.1 + + d3-scale@4.0.2: + dependencies: + d3-array: 3.2.4 + d3-format: 3.1.2 + d3-interpolate: 3.0.1 + d3-time: 3.1.0 + d3-time-format: 4.1.0 + + d3-shape@3.2.0: + dependencies: + d3-path: 3.1.0 + + d3-time-format@3.0.0: + dependencies: + d3-time: 1.1.0 + + d3-time-format@4.1.0: + dependencies: + d3-time: 3.1.0 + + d3-time@1.1.0: {} + + d3-time@3.1.0: + dependencies: + d3-array: 3.2.4 + data-urls@7.0.0: dependencies: whatwg-mimetype: 5.0.0 @@ -2132,6 +2673,10 @@ snapshots: deep-is@0.1.4: {} + delaunator@5.1.0: + dependencies: + robust-predicates: 3.0.3 + dequal@2.0.3: {} detect-indent@7.0.2: {} @@ -2356,6 +2901,8 @@ snapshots: indent-string@5.0.0: {} + internmap@2.0.3: {} + is-builtin-module@5.0.0: dependencies: builtin-modules: 5.3.0 @@ -2494,6 +3041,8 @@ snapshots: lodash.merge@4.6.2: {} + lodash@4.18.1: {} + lru-cache@11.5.2: {} lru-cache@5.1.1: @@ -2608,6 +3157,11 @@ snapshots: react-is@17.0.2: {} + react-virtualized-auto-sizer@1.0.26(react-dom@19.2.8(react@19.2.8))(react@19.2.8): + dependencies: + react: 19.2.8 + react-dom: 19.2.8(react@19.2.8) + react@19.2.8: {} redent@3.0.0: @@ -2632,6 +3186,8 @@ snapshots: reserved-identifiers@1.2.0: {} + robust-predicates@3.0.3: {} + rolldown@1.2.6: dependencies: '@oxc-project/types': 0.147.0 @@ -2767,6 +3323,10 @@ snapshots: dependencies: punycode: 2.3.1 + use-debounce@10.1.1(react@19.2.8): + dependencies: + react: 19.2.8 + vite@8.2.2(@types/node@26.4.0)(yaml@2.9.0): dependencies: lightningcss: 1.33.0 diff --git a/web/src/api/analytics-types.ts b/web/src/api/analytics-types.ts new file mode 100644 index 0000000..0e0bfc1 --- /dev/null +++ b/web/src/api/analytics-types.ts @@ -0,0 +1,162 @@ +// The Analytics tab's shapes, split out of types.ts so both stay under the +// 250-line cap. Hand-written like everything in that file, and mirroring the +// dataclasses in signal_bot/analytics one for one -- so a field that moves on +// the server fails the build here rather than rendering as undefined. + +export interface Sources { + daily_uses: number; + journald: number; + live: number; +} + +export interface Coverage { + days: number; + enough_for_a_trend: boolean; + events: number; + first_day: string; + last_day: string; + sources: Sources; +} + +export interface AnalyticsGroup { + events: number; + first_day: string; + id: string; + last_day: string; +} + +export interface CommandUse { + command: string; + errors: number; + first_day: string; + last_day: string; + ok: number; + people: number; + refused: number; + uses: number; +} + +export interface PersonUse { + commands: number; + favourite: string; + // Empty when the bot has never seen a display name for this UUID. + name: string; + uses: number; + uuid: string; +} + +export interface MatrixCell { + command: string; + name: string; + uses: number; + uuid: string; +} + +export interface Overview { + commands: CommandUse[]; + matrix: MatrixCell[]; + people: PersonUse[]; + totals: { attempts: number; commands: number; ok: number; people: number }; + window: Coverage; +} + +export interface DayCount { + day: string; + uses: number; +} + +export interface MonthCount { + days: number; + month: string; + uses: number; +} + +export interface Dying { + command: string; + idle_days: number; + last_day: string; +} + +export interface Times { + by_command_hour: Record; + days: DayCount[]; + direction: string; + dying: Dying[]; + hours: number[]; + months: MonthCount[]; + peak: { + busiest: number; + busiest_uses: number; + quietest: number; + quietest_uses: number; + }; + weekday_hours: number[][]; + window: Coverage; +} + +export interface Latency { + command: string; + handler_p50: number; + handler_p95: number; + samples: number; + send_p50: number; + upstream_p50: number; +} + +export interface Upstream { + calls: number; + failures: number; + host: string; + p50_ms: number; +} + +export interface Health { + errors: { command: string; error_class: string; hits: number }[]; + gates: { command: string; gate: string; hits: number }[]; + latencies: Latency[]; + upstream: Upstream[]; + window: Coverage; +} + +export interface NearMiss { + hits: number; + target: string; + variants: number; +} + +export interface Demand { + filed: number; + near_misses: NearMiss[]; + unclaimed: number; + unclaimed_words: number; + window: Coverage; +} + +export interface Behaviour { + newcomers: { day: string; people: number }[]; + pairs: { hits: number; left: string; right: string }[]; + releases: { after: number; before: number; day: string; version: string }[]; + repeats: { command: string; extra: number }[]; + retention: { active: number; cohort: number; offset: number }[]; + sessions: { + average_length: number; + commands: number; + count: number; + longest: number; + }; + share: { commands: number; fraction: number; messages: number }; + window: Coverage; +} + +export interface Pool { + command: string; + distinct: number; + draws: number; + repeat_rate: number; +} + +export interface Content { + pools: Pool[]; + repeats: { command: string; draws: number; item_id: string }[]; + window: Coverage; +} diff --git a/web/src/api/client.ts b/web/src/api/client.ts index e06e34f..2c13b4e 100644 --- a/web/src/api/client.ts +++ b/web/src/api/client.ts @@ -1,3 +1,12 @@ +import type { + AnalyticsGroup, + Behaviour, + Content, + Demand, + Health, + Overview, + Times, +} from "./analytics-types.ts"; import type { Roster } from "./setup.ts"; import type { AlbumState, @@ -53,8 +62,28 @@ async function send(path: string, method: string, body: unknown): Promise ); } +// One optional group, everywhere. Absent means the all-groups roll-up, +// which the server answers with the same SQL rather than a second query. +function scoped(path: string, group: null | string): string { + return group === null ? path : `${path}?group=${encodeURIComponent(group)}`; +} + export const api = { album: async (): Promise => get("/api/album/state"), + analyticsBehaviour: async (group: null | string): Promise => + get(scoped("/api/analytics/behaviour", group)), + analyticsContent: async (group: null | string): Promise => + get(scoped("/api/analytics/content", group)), + analyticsDemand: async (group: null | string): Promise => + get(scoped("/api/analytics/demand", group)), + analyticsGroups: async (): Promise<{ groups: AnalyticsGroup[] }> => + get<{ groups: AnalyticsGroup[] }>("/api/analytics/groups"), + analyticsHealth: async (group: null | string): Promise => + get(scoped("/api/analytics/health", group)), + analyticsOverview: async (group: null | string): Promise => + get(scoped("/api/analytics/overview", group)), + analyticsTime: async (group: null | string): Promise => + get(scoped("/api/analytics/time", group)), env: async (): Promise => get("/api/env"), exportSuggestions: async (): Promise => send("/api/suggestions/export", "POST", {}), diff --git a/web/src/app.tsx b/web/src/app.tsx index d7997e8..642db76 100644 --- a/web/src/app.tsx +++ b/web/src/app.tsx @@ -1,11 +1,13 @@ import { useState } from "react"; import { Album } from "./views/album.tsx"; +import { Analytics } from "./views/analytics.tsx"; import { Health } from "./views/health.tsx"; import { Settings } from "./views/settings.tsx"; import { Suggestions } from "./views/suggestions.tsx"; const VIEWS = { + Analytics: Analytics, Health: Health, Ideas: Suggestions, Photos: Album, diff --git a/web/src/charts/nivo.test.tsx b/web/src/charts/nivo.test.tsx new file mode 100644 index 0000000..2c5b570 --- /dev/null +++ b/web/src/charts/nivo.test.tsx @@ -0,0 +1,99 @@ +import { render, screen } from "@testing-library/react"; +import { expect, it } from "vitest"; + +import { BarChart, CalendarChart, HeatChart, LineChart } from "./nivo.tsx"; + +// Assertions are on the wrapper's own testid, on the empty state, and on the +// axis labels Nivo emits as . Deliberately not on Nivo's internals: +// those are its business and would break on an upgrade that changed nothing +// a reader can see. + +it("draws bars, with the categories on the axis", () => { + render( + , + ); + expect(screen.getByTestId("bars")).toBeInTheDocument(); + expect(screen.getByText("pope")).toBeInTheDocument(); + expect(screen.getByText("command")).toBeInTheDocument(); +}); + +it("says so rather than drawing an empty axis", () => { + // The state a fresh install is in, so it has to read as a sentence. + render(); + expect(screen.getByTestId("bars-empty")).toHaveTextContent("Nothing recorded yet."); +}); + +it("draws a line for a series", () => { + render( + , + ); + expect(screen.getByTestId("line")).toBeInTheDocument(); +}); + +it("treats a series with no points as empty", () => { + // A series that exists but holds nothing is the shape the API returns on + // a group nobody has used, and it must not draw an axis over nothing. + render(); + expect(screen.getByTestId("line-empty")).toBeInTheDocument(); +}); + +it("treats no series at all as empty", () => { + render(); + expect(screen.getByTestId("line-empty")).toBeInTheDocument(); +}); + +it("draws a heat grid", () => { + render( + , + ); + expect(screen.getByTestId("heat")).toBeInTheDocument(); + expect(screen.getByText("Mon")).toBeInTheDocument(); +}); + +it("says so when there is no grid to draw", () => { + render(); + expect(screen.getByTestId("heat-empty")).toBeInTheDocument(); +}); + +it("draws a calendar between two days", () => { + render( + , + ); + expect(screen.getByTestId("calendar")).toBeInTheDocument(); +}); + +it("refuses to draw a calendar with no range", () => { + // An empty window is what the API returns before anything is recorded, + // and Nivo's Calendar throws on a blank from/to rather than drawing. + render(); + expect(screen.getByTestId("cal-empty")).toBeInTheDocument(); +}); + +it("refuses to draw a calendar with no days", () => { + render( + , + ); + expect(screen.getByTestId("cal-empty")).toBeInTheDocument(); +}); diff --git a/web/src/charts/nivo.tsx b/web/src/charts/nivo.tsx new file mode 100644 index 0000000..230e177 --- /dev/null +++ b/web/src/charts/nivo.tsx @@ -0,0 +1,190 @@ +import { Bar } from "@nivo/bar"; +import { Calendar } from "@nivo/calendar"; +import { HeatMap } from "@nivo/heatmap"; +import { Line } from "@nivo/line"; + +import { useBoxSize } from "./size.ts"; + +// Four thin wrappers over Nivo, and every one of them says the same three +// things. +// +// 1. Fixed size, never the Responsive* variants. Those measure their own +// container, and a jsdom container has no size -- so a suite full of +// charts would assert against empty divs. useBoxSize supplies a real +// measurement in a browser and a stated fallback everywhere else. +// 2. animate={false}. Nivo animates through @react-spring, which schedules +// frames; under jsdom that is a source of flake and in a dashboard +// nobody is watching a bar grow. +// 3. An empty state with its own testid. "No data" is the state this page +// is in on a fresh install, and it has to render as a sentence rather +// than as an axis with nothing under it. +// +// Colours come from @kuhyx/web-ui's tokens rather than from a Nivo scheme, +// so the charts match the rest of the page in both themes. +const CATEGORICAL = [ + "var(--cat-1)", + "var(--cat-2)", + "var(--cat-3)", + "var(--cat-4)", + "var(--cat-5)", + "var(--cat-6)", +]; + +const MARGIN = { bottom: 64, left: 64, right: 24, top: 16 }; + +const THEME = { + axis: { + legend: { text: { fill: "var(--text)" } }, + ticks: { text: { fill: "var(--text-muted)" } }, + }, + grid: { line: { stroke: "var(--border)" } }, + text: { fontFamily: "var(--font-sans)" }, +}; + +function Empty({ label, testid }: Readonly<{ label: string; testid: string }>): React.JSX.Element { + return ( +

+ {label} +

+ ); +} + +// Nivo's own BarDatum is an index signature, so a closed interface will not +// satisfy it. Named differently from Nivo's to keep the two apart at a +// glance, and widened with the index signature it demands. +export interface Bars { + [key: string]: number | string; + label: string; + value: number; +} + +export function BarChart({ + data, + legend, + testid, +}: Readonly<{ + data: Bars[]; + legend: string; + testid: string; +}>): React.JSX.Element { + const { ref, size } = useBoxSize(); + if (data.length === 0) { + return ; + } + return ( +
+ +
+ ); +} + +export interface Series { + data: { x: number | string; y: number }[]; + id: string; +} + +export function LineChart({ + data, + legend, + testid, +}: Readonly<{ + data: Series[]; + legend: string; + testid: string; +}>): React.JSX.Element { + const { ref, size } = useBoxSize(); + const isEmpty = data.every((series) => series.data.length === 0); + if (isEmpty || data.length === 0) { + return ; + } + return ( +
+ +
+ ); +} + +export interface HeatRow { + data: { x: string; y: number }[]; + id: string; +} + +export function HeatChart({ + data, + testid, +}: Readonly<{ + data: HeatRow[]; + testid: string; +}>): React.JSX.Element { + const { ref, size } = useBoxSize(); + if (data.length === 0) { + return ; + } + return ( +
+ +
+ ); +} + +export function CalendarChart({ + data, + from, + testid, + to, +}: Readonly<{ + data: { day: string; value: number }[]; + from: string; + testid: string; + to: string; +}>): React.JSX.Element { + const { ref, size } = useBoxSize(); + if (from === "" || to === "" || data.length === 0) { + return ; + } + return ( +
+ +
+ ); +} diff --git a/web/src/charts/size.test.ts b/web/src/charts/size.test.ts new file mode 100644 index 0000000..b0a2654 --- /dev/null +++ b/web/src/charts/size.test.ts @@ -0,0 +1,74 @@ +import { renderHook } from "@testing-library/react"; +import { afterEach, expect, it, vi } from "vitest"; + +import { FALLBACK, useBoxSize } from "./size.ts"; + +afterEach(() => { + vi.restoreAllMocks(); + vi.unstubAllGlobals(); +}); + +// One probe, used by every case that needs the ref to point at something. +// jsdom never attaches it for us, because nothing here renders the element +// the hook is meant to be handed. +function useProbe(): ReturnType> { + const box = useBoxSize(); + box.ref.current ??= document.createElement("div"); + return box; +} + +function measuring(width: number, height: number): void { + vi.spyOn(HTMLElement.prototype, "clientWidth", "get").mockReturnValue(width); + vi.spyOn(HTMLElement.prototype, "clientHeight", "get").mockReturnValue(height); +} + +it("falls back to a stated size when the container has none", () => { + // Every jsdom element reports 0x0, which is exactly the case the fallback + // exists for: Nivo's Responsive* components render nothing at that size, + // so a suite full of charts would assert against empty divs. + const { result } = renderHook(() => useBoxSize()); + expect(result.current.size).toEqual(FALLBACK); +}); + +it("uses a real measurement when there is one", () => { + measuring(800, 400); + const { rerender, result } = renderHook(useProbe); + rerender(); + expect(result.current.size).toEqual({ height: 400, width: 800 }); + expect(result.current.size).not.toEqual(FALLBACK); +}); + +it("ignores a container that reports one dimension and not the other", () => { + // Half a measurement is not a measurement: a container mid-layout reports + // a width and no height, and drawing at zero height draws nothing. + measuring(800, 0); + const { rerender, result } = renderHook(useProbe); + rerender(); + expect(result.current.size).toEqual(FALLBACK); +}); + +it("survives an environment with no ResizeObserver at all", () => { + // Guarded rather than assumed: a missing observer must degrade to + // "measured once", never throw on mount. + vi.stubGlobal("ResizeObserver", undefined); + const { result } = renderHook(useProbe); + expect(result.current.size).toEqual(FALLBACK); +}); + +it("observes its container and disconnects when the tab goes away", () => { + const disconnect = vi.fn(); + const observe = vi.fn(); + vi.stubGlobal( + "ResizeObserver", + class { + disconnect = disconnect; + observe = observe; + unobserve = vi.fn(); + }, + ); + + const { unmount } = renderHook(useProbe); + expect(observe).toHaveBeenCalledTimes(1); + unmount(); + expect(disconnect).toHaveBeenCalledTimes(1); +}); diff --git a/web/src/charts/size.ts b/web/src/charts/size.ts new file mode 100644 index 0000000..b1170b6 --- /dev/null +++ b/web/src/charts/size.ts @@ -0,0 +1,63 @@ +import { useCallback, useEffect, useRef, useState } from "react"; + +// Nivo ships ResponsiveBar and friends, which measure their container and +// render nothing when it has no size. A jsdom container always has no size, +// so every chart in the test suite would render an empty
and every +// assertion about a chart would pass against nothing. +// +// So the charts here take explicit width and height, and this hook is what +// supplies them: a real measurement where one exists, and a stated fallback +// where one does not. The fallback is exported so a test can assert against +// a number rather than against whatever jsdom happened to report. +export const FALLBACK = { height: 320, width: 640 } as const; + +export interface Box { + height: number; + width: number; +} + +// A box with no area is not a measurement, it is a container that has not +// been laid out yet -- which in jsdom is permanent and in a browser is the +// first frame. Both want the fallback. +function hasArea(box: Box): boolean { + return box.width > 0 && box.height > 0; +} + +/** + * Measure an element, falling back to a fixed size when it has none. + * + * Returns the ref to attach and the size to draw at. + */ +export function useBoxSize(): { + ref: React.RefObject; + size: Box; +} { + const reference = useRef(null); + const [size, setSize] = useState(FALLBACK); + + const measure = useCallback(() => { + const element = reference.current; + if (element === null) { + return; + } + const box = { height: element.clientHeight, width: element.clientWidth }; + setSize(hasArea(box) ? box : FALLBACK); + }, []); + + useEffect(() => { + measure(); + // ResizeObserver does not exist in jsdom and is not polyfilled here -- + // the test setup stubs it. Guarded rather than assumed so that a + // missing one degrades to "measured once" instead of throwing. + if (typeof ResizeObserver === "undefined" || reference.current === null) { + return; + } + const observer = new ResizeObserver(measure); + observer.observe(reference.current); + return () => { + observer.disconnect(); + }; + }, [measure]); + + return { ref: reference, size }; +} diff --git a/web/src/index.css b/web/src/index.css index e0b7e70..6c29c08 100644 --- a/web/src/index.css +++ b/web/src/index.css @@ -205,3 +205,97 @@ input[type="checkbox"] { [data-testid="note"] { color: var(--success); } + +/* --------------------------------------------------------------------------- + Analytics. + + Every value here is a token from @kuhyx/web-ui. The charts get their + colours the same way, so the page reads as one thing in both themes and a + palette change lands everywhere at once. + --------------------------------------------------------------------------- */ + +/* What the numbers are, before any of them are read. */ + +.footnote { + max-width: var(--measure); + font-size: var(--text-label); + line-height: 1.5; + color: var(--text-muted); +} + +/* The headline row. Auto-fit rather than a fixed column count, so it stays + readable in the narrow window this page is usually opened in. */ + +.tiles { + display: grid; + grid-template-columns: repeat(auto-fit, minmax(9rem, 1fr)); + gap: var(--space-2); + margin: var(--space-3) 0; +} + +.tile { + display: flex; + flex-direction: column; + gap: var(--space-1); + padding: var(--space-3); + background: var(--surface-1); + border: 1px solid var(--border); + border-radius: var(--radius-md); +} + +.tile-value { + font-size: var(--text-title); + font-variant-numeric: tabular-nums; + line-height: 1; + color: var(--text); +} + +.tile-label { + font-size: var(--text-label); + color: var(--text-muted); +} + +/* A chart is given a height here rather than in the component: useBoxSize + measures this box, so the CSS is what decides the drawing size in a real + browser and the component's fallback only applies where there is no + layout at all. */ + +.chart { + width: 100%; + height: 22rem; + margin: var(--space-2) 0 var(--space-4); + overflow: hidden; +} + +.chart-empty { + padding: var(--space-4); + font-size: var(--text-label); + color: var(--text-muted); + text-align: center; + background: var(--surface-1); + border: 1px dashed var(--border); + border-radius: var(--radius-md); +} + +/* Latency figures are compared down a column, so they line up. */ + +[data-testid="health-table"] { + width: 100%; + border-collapse: collapse; +} + +[data-testid="health-table"] th { + font-size: var(--text-label); + color: var(--text-muted); + text-align: left; +} + +[data-testid="health-table"] td { + font-variant-numeric: tabular-nums; +} + +[data-testid="health-table"] td, +[data-testid="health-table"] th { + padding: var(--space-1) var(--space-2); + border-bottom: 1px solid var(--border); +} diff --git a/web/src/test/analytics-fixtures.ts b/web/src/test/analytics-fixtures.ts new file mode 100644 index 0000000..6d759c7 --- /dev/null +++ b/web/src/test/analytics-fixtures.ts @@ -0,0 +1,172 @@ +// The Analytics tab's fixtures, split out of fixtures.ts so both stay under +// the 250-line cap. +// +// Three days of history, because that is the shape the real page is in: +// enough to draw, not enough to claim a trend. Invented people and an +// invented group, like every other value in these files. +import type { + Behaviour, + Content, + Coverage, + Demand, + Health, + Overview, + Times, +} from "../api/analytics-types.ts"; + +import { GROUP } from "./fixtures.ts"; + +const COVERAGE: Coverage = { + days: 3, + enough_for_a_trend: false, + events: 40, + first_day: "2026-08-29", + last_day: "2026-08-31", + sources: { daily_uses: 9, journald: 25, live: 6 }, +}; + +export const EMPTY_COVERAGE: Coverage = { + days: 0, + enough_for_a_trend: false, + events: 0, + first_day: "", + last_day: "", + sources: { daily_uses: 0, journald: 0, live: 0 }, +}; + +export const ANALYTICS_GROUPS = { + groups: [{ events: 40, first_day: "2026-08-29", id: GROUP.id, last_day: "2026-08-31" }], +}; + +export const OVERVIEW: Overview = { + commands: [ + { + command: "pope", + errors: 0, + first_day: "2026-08-29", + last_day: "2026-08-31", + ok: 24, + people: 3, + refused: 1, + uses: 25, + }, + { + command: "cat", + errors: 2, + first_day: "2026-08-29", + last_day: "2026-08-30", + ok: 13, + people: 2, + refused: 0, + uses: 15, + }, + ], + matrix: [{ command: "pope", name: "ala", uses: 20, uuid: "uuid-ala" }], + people: [ + { commands: 2, favourite: "pope", name: "ala", uses: 30, uuid: "uuid-ala" }, + { commands: 1, favourite: "cat", name: "", uses: 10, uuid: "uuid-basia" }, + ], + totals: { attempts: 40, commands: 2, ok: 37, people: 2 }, + window: COVERAGE, +}; + +export const TIMES: Times = { + by_command_hour: { pope: Array.from({ length: 24 }, (_, hour) => (hour === 19 ? 25 : 0)) }, + days: [ + { day: "2026-08-29", uses: 10 }, + { day: "2026-08-30", uses: 12 }, + { day: "2026-08-31", uses: 18 }, + ], + direction: "unknown", + dying: [{ command: "gecko", idle_days: 4, last_day: "2026-08-27" }], + hours: Array.from({ length: 24 }, (_, hour) => (hour === 19 ? 25 : 1)), + months: [{ days: 3, month: "2026-08", uses: 40 }], + peak: { busiest: 19, busiest_uses: 25, quietest: 4, quietest_uses: 1 }, + weekday_hours: Array.from({ length: 7 }, () => Array.from({ length: 24 }, () => 1)), + window: COVERAGE, +}; + +export const HEALTH: Health = { + errors: [{ command: "cat", error_class: "ImageFetchError", hits: 2 }], + gates: [{ command: "pope", gate: "refused_cooldown", hits: 1 }], + latencies: [ + { + command: "pope", + handler_p50: 2845, + handler_p95: 2996, + samples: 6, + send_p50: 2156, + upstream_p50: 259, + }, + ], + upstream: [{ calls: 12, failures: 1, host: "commons.example.test", p50_ms: 254 }], + window: COVERAGE, +}; + +export const DEMAND: Demand = { + filed: 2, + near_misses: [{ hits: 17, target: "!pulapka", variants: 4 }], + unclaimed: 3, + unclaimed_words: 2, + window: COVERAGE, +}; + +export const BEHAVIOUR: Behaviour = { + newcomers: [{ day: "2026-08-29", people: 2 }], + pairs: [{ hits: 4, left: "cat", right: "pope" }], + releases: [{ after: 30, before: 10, day: "2026-08-31", version: "0.2.5" }], + repeats: [{ command: "pope", extra: 12 }], + retention: [ + { active: 2, cohort: 2, offset: 0 }, + { active: 1, cohort: 2, offset: 1 }, + ], + sessions: { average_length: 3.9, commands: 40, count: 10, longest: 12 }, + share: { commands: 40, fraction: 0.35, messages: 114 }, + window: COVERAGE, +}; + +export const CONTENT: Content = { + pools: [{ command: "pope", distinct: 20, draws: 25, repeat_rate: 0.2 }], + repeats: [{ command: "pope", draws: 3, item_id: "Q42" }], + window: COVERAGE, +}; + +export const EMPTY_PANELS = { + behaviour: { + newcomers: [], + pairs: [], + releases: [], + repeats: [], + retention: [], + sessions: { average_length: 0, commands: 0, count: 0, longest: 0 }, + share: { commands: 0, fraction: 0, messages: 0 }, + window: EMPTY_COVERAGE, + }, + content: { pools: [], repeats: [], window: EMPTY_COVERAGE }, + demand: { + filed: 0, + near_misses: [], + unclaimed: 0, + unclaimed_words: 0, + window: EMPTY_COVERAGE, + }, + health: { errors: [], gates: [], latencies: [], upstream: [], window: EMPTY_COVERAGE }, + overview: { + commands: [], + matrix: [], + people: [], + totals: { attempts: 0, commands: 0, ok: 0, people: 0 }, + window: EMPTY_COVERAGE, + }, + times: { + by_command_hour: {}, + days: [], + direction: "unknown", + dying: [], + hours: [], + months: [], + peak: { busiest: -1, busiest_uses: 0, quietest: -1, quietest_uses: 0 }, + weekday_hours: [], + window: EMPTY_COVERAGE, + }, +}; diff --git a/web/src/test/fixtures.ts b/web/src/test/fixtures.ts index 1a6391d..c1ad3c3 100644 --- a/web/src/test/fixtures.ts +++ b/web/src/test/fixtures.ts @@ -80,3 +80,9 @@ export const SUGGESTIONS: SuggestionsView = { }; export const NO_SUGGESTIONS: SuggestionsView = { suggestions: [], unknown: [] }; + +// -- analytics --------------------------------------------------------------- +// +// Invented people and an invented group, like everything else here. Three +// days of history, because that is the shape the real page is in: enough to +// draw, not enough to claim a trend. diff --git a/web/src/test/setup.ts b/web/src/test/setup.ts index f149f27..4dd80d2 100644 --- a/web/src/test/setup.ts +++ b/web/src/test/setup.ts @@ -1 +1,28 @@ +import { beforeEach, vi } from "vitest"; import "@testing-library/jest-dom/vitest"; + +// jsdom has no ResizeObserver and the charts observe their container. +// +// Stubbed rather than polyfilled: nothing here needs it to fire, only to +// exist. useBoxSize's own test covers both the present and the absent case +// by stubbing it away again. +class StubResizeObserver implements ResizeObserver { + disconnect(): void { + // Nothing to disconnect: this observer never fires. + } + + observe(): void { + // A jsdom element has no size, so there is nothing to report. + } + + unobserve(): void { + // Symmetry with observe. + } +} + +// Re-stubbed per test rather than assigned once, because most suites here +// call vi.unstubAllGlobals() in afterEach to put fetch back -- which would +// take this with it and leave every later chart throwing on mount. +beforeEach(() => { + vi.stubGlobal("ResizeObserver", StubResizeObserver); +}); diff --git a/web/src/views/analytics-panels.tsx b/web/src/views/analytics-panels.tsx new file mode 100644 index 0000000..5968bad --- /dev/null +++ b/web/src/views/analytics-panels.tsx @@ -0,0 +1,207 @@ +import { useEffect, useState } from "react"; + +import type { + Behaviour, + Content, + Demand, + Health, + Overview, + Times, +} from "../api/analytics-types.ts"; + +import { api } from "../api/client.ts"; +import { BarChart, CalendarChart, HeatChart, LineChart } from "../charts/nivo.tsx"; +import { Sources } from "./analytics.tsx"; + +const WEEKDAYS = ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"]; +const TOP = 12; + +interface Panels { + behaviour: Behaviour; + content: Content; + demand: Demand; + health: Health; + overview: Overview; + times: Times; +} + +function percent(fraction: number): string { + return `${String(Math.round(fraction * 100))}%`; +} + +function Tile({ label, value }: Readonly<{ label: string; value: string }>): React.JSX.Element { + return ( +
+ {value} + {label} +
+ ); +} + +export function AnalyticsPanels({ group }: Readonly<{ group: null | string }>): React.JSX.Element { + const [panels, setPanels] = useState(null); + + useEffect(() => { + // One state object rather than six, so a panel never renders against a + // window that belongs to the previously selected group. + void Promise.all([ + api.analyticsOverview(group), + api.analyticsTime(group), + api.analyticsHealth(group), + api.analyticsDemand(group), + api.analyticsBehaviour(group), + api.analyticsContent(group), + ]).then(([overview, times, health, demand, behaviour, content]) => { + setPanels({ behaviour, content, demand, health, overview, times }); + }); + }, [group]); + + if (panels === null) { + return

Reading the database…

; + } + const { behaviour, content, demand, health, overview, times } = panels; + + return ( + <> + + +
+ + + + + + +
+ +

Which commands get used

+ ({ label: one.command, value: one.uses }))} + legend="command" + testid="chart-commands" + /> + +

When

+ ({ label: `${String(hour).padStart(2, "0")}:00`, value: uses }))} + legend="hour of day (Europe/Warsaw)" + testid="chart-hours" + /> + { + return { + data: row.map((uses, hour) => ({ x: String(hour).padStart(2, "0"), y: uses })), + id: WEEKDAYS[index] ?? String(index), + }; + })} + testid="chart-weekday-hours" + /> + ({ x: one.day, y: one.uses })), + id: "uses", + }, + ]} + legend="day" + testid="chart-days" + /> + ({ day: one.day, value: one.uses }))} + from={times.window.first_day} + testid="chart-calendar" + to={times.window.last_day} + /> + +

What needs work

+ + + + + + + + + + + + {health.latencies.map((one) => { + return ( + + + + + + + + ); + })} + +
commandp50p95send p50upstream p50
{one.command}{one.handler_p50} ms{one.handler_p95} ms{one.send_p50} ms{one.upstream_p50} ms
+ {health.upstream.length === 0 ? ( +

No upstream calls recorded yet.

+ ) : ( +
    + {health.upstream.map((one) => { + return ( +
  • + {one.host} — {one.calls} calls, {one.failures} failed, p50 {one.p50_ms} ms +
    • + ); + })} +
+ )} + +

What people wanted and could not get

+ {demand.near_misses.length === 0 ? ( +

No near misses recorded.

+ ) : ( +
    + {demand.near_misses.map((one) => { + return ( +
  • + {one.target} — {one.hits} near {one.hits === 1 ? "miss" : "misses"} across{" "} + {one.variants} {one.variants === 1 ? "spelling" : "spellings"} +
    • + ); + })} +
+ )} + +

How it gets used

+

+ {behaviour.sessions.count} sittings, {behaviour.sessions.average_length.toFixed(1)}{" "} + commands each, longest {behaviour.sessions.longest}. +

+ ({ x: one.offset, y: one.active })), + id: "still using it", + }, + ]} + legend="days after first use" + testid="chart-retention" + /> + +

Whether the pools repeat

+ {content.pools.length === 0 ? ( +

Nothing that names what it served yet.

+ ) : ( +
    + {content.pools.map((one) => { + return ( +
  • + {one.command} — {one.draws} draws, {one.distinct} distinct, repeat{" "} + {percent(one.repeat_rate)} +
    • + ); + })} +
+ )} + + ); +} diff --git a/web/src/views/analytics-routes.ts b/web/src/views/analytics-routes.ts new file mode 100644 index 0000000..c68c45e --- /dev/null +++ b/web/src/views/analytics-routes.ts @@ -0,0 +1,33 @@ +// The two servers the Analytics tests run against: one with a small history +// and one that has never seen a command. Shared so both test files describe +// the same server rather than two that drift. +import { + ANALYTICS_GROUPS, + BEHAVIOUR, + CONTENT, + DEMAND, + EMPTY_PANELS, + HEALTH, + OVERVIEW, + TIMES, +} from "../test/analytics-fixtures.ts"; + +export const ROUTES = { + "/api/analytics/behaviour": { body: BEHAVIOUR }, + "/api/analytics/content": { body: CONTENT }, + "/api/analytics/demand": { body: DEMAND }, + "/api/analytics/groups": { body: ANALYTICS_GROUPS }, + "/api/analytics/health": { body: HEALTH }, + "/api/analytics/overview": { body: OVERVIEW }, + "/api/analytics/time": { body: TIMES }, +}; + +export const EMPTY_ROUTES = { + "/api/analytics/behaviour": { body: EMPTY_PANELS.behaviour }, + "/api/analytics/content": { body: EMPTY_PANELS.content }, + "/api/analytics/demand": { body: EMPTY_PANELS.demand }, + "/api/analytics/groups": { body: { groups: [] } }, + "/api/analytics/health": { body: EMPTY_PANELS.health }, + "/api/analytics/overview": { body: EMPTY_PANELS.overview }, + "/api/analytics/time": { body: EMPTY_PANELS.times }, +}; diff --git a/web/src/views/analytics-states.test.tsx b/web/src/views/analytics-states.test.tsx new file mode 100644 index 0000000..678f48b --- /dev/null +++ b/web/src/views/analytics-states.test.tsx @@ -0,0 +1,157 @@ +import { render, screen, waitFor } from "@testing-library/react"; +import userEvent from "@testing-library/user-event"; +import { afterEach, expect, it, vi } from "vitest"; + +import { ANALYTICS_GROUPS, EMPTY_PANELS, TIMES } from "../test/analytics-fixtures.ts"; +import { serve } from "../test/http.ts"; +import { EMPTY_ROUTES, ROUTES } from "./analytics-routes.ts"; + +const [ONLY_GROUP] = ANALYTICS_GROUPS.groups; +import { Analytics } from "./analytics.tsx"; + +// The states that are not "a populated page": nothing recorded, exactly one +// of something, and a history long enough to have a direction. Split out of +// analytics.test.tsx so both stay under the 250-line cap. + +afterEach(() => { + vi.unstubAllGlobals(); +}); + +it("renders every panel against a bot that has never run", async () => { + // The state a fresh install is in. Every panel has to say so rather than + // draw an axis over nothing. + serve(EMPTY_ROUTES); + render(); + await waitFor(() => { + expect(screen.getByTestId("chart-commands-empty")).toBeInTheDocument(); + }); + expect(screen.getByTestId("no-upstream")).toBeInTheDocument(); + expect(screen.getByTestId("no-demand")).toBeInTheDocument(); + expect(screen.getByTestId("no-pools")).toBeInTheDocument(); + expect(screen.getByTestId("chart-calendar-empty")).toBeInTheDocument(); + expect(screen.getByTestId("tiles")).toHaveTextContent("—"); +}); + +it("asks the server for one group when one is picked", async () => { + // A group id is 44 characters of base64, so it is picked and never typed. + const fetcher = serve(ROUTES); + render(); + await waitFor(() => { + expect(screen.getByTestId("group-picker")).toBeInTheDocument(); + }); + await userEvent.selectOptions(screen.getByTestId("group-picker"), ONLY_GROUP?.id ?? ""); + await waitFor(() => { + const asked = fetcher.mock.calls.map((call) => String(call[0])); + expect(asked.some((url) => url.includes("group="))).toBe(true); + }); +}); + +it("agrees with itself about one of anything", async () => { + // Singular. The footnote reads as prose and "1 uses over 1 days" is the + // sort of thing that gets noticed every single time. + const one = { + days: 1, + enough_for_a_trend: false, + events: 1, + first_day: "2026-08-29", + last_day: "2026-08-29", + sources: { daily_uses: 0, journald: 0, live: 1 }, + }; + serve({ + ...EMPTY_ROUTES, + "/api/analytics/demand": { + body: { + ...EMPTY_PANELS.demand, + near_misses: [{ hits: 1, target: "!kot", variants: 1 }], + window: one, + }, + }, + "/api/analytics/overview": { body: { ...EMPTY_PANELS.overview, window: one } }, + }); + render(); + await waitFor(() => { + expect(screen.getByTestId("sources")).toHaveTextContent("1 recorded use over 1 day"); + }); + expect(screen.getByTestId("demand")).toHaveTextContent("1 near miss"); + expect(screen.getByTestId("demand")).toHaveTextContent("1 spelling"); +}); + +it("states a direction once there are enough days to have one", async () => { + const long = { + days: 40, + enough_for_a_trend: true, + events: 900, + first_day: "2026-07-01", + last_day: "2026-08-31", + sources: { daily_uses: 0, journald: 0, live: 900 }, + }; + serve({ + ...EMPTY_ROUTES, + "/api/analytics/overview": { body: { ...EMPTY_PANELS.overview, window: long } }, + }); + render(); + await waitFor(() => { + expect(screen.getByTestId("sources")).toHaveTextContent("900 recorded uses over 40 days"); + }); + expect(screen.getByTestId("sources")).not.toHaveTextContent("Too few days"); +}); + +it("says nothing about dates before there are any", async () => { + serve(EMPTY_ROUTES); + render(); + await waitFor(() => { + expect(screen.getByTestId("sources")).toBeInTheDocument(); + }); + // No parenthesised range at all rather than an empty one. + expect(screen.getByTestId("sources")).not.toHaveTextContent("()"); +}); + +it("lists an upstream host and a pool when there is one of each", async () => { + serve(ROUTES); + render(); + await waitFor(() => { + expect(screen.getByTestId("upstream")).toHaveTextContent("commons.example.test"); + }); + expect(screen.getByTestId("sessions")).toHaveTextContent("10 sittings"); +}); + +it("goes back to every group when the picker is cleared", async () => { + const fetcher = serve(ROUTES); + render(); + await waitFor(() => { + expect(screen.getByTestId("group-picker")).toBeInTheDocument(); + }); + const picker = screen.getByTestId("group-picker"); + await userEvent.selectOptions(picker, ONLY_GROUP?.id ?? ""); + await waitFor(() => { + expect(fetcher.mock.calls.some((call) => String(call[0]).includes("group="))).toBe(true); + }); + + fetcher.mockClear(); + await userEvent.selectOptions(picker, ""); + await waitFor(() => { + expect(fetcher.mock.calls.length).toBeGreaterThan(0); + }); + // The roll-up asks for no group at all rather than for an empty one. + expect(fetcher.mock.calls.every((call) => !String(call[0]).includes("group="))).toBe(true); +}); + +it("labels a weekday grid that is longer than a week", async () => { + // The server sends exactly seven rows. If it ever sent more, they get a + // number rather than an undefined label -- which is what the index + // fallback is for, and the only way to reach it. + serve({ + ...ROUTES, + "/api/analytics/time": { + body: { + ...TIMES, + weekday_hours: [...TIMES.weekday_hours, Array.from({ length: 24 }, () => 2)], + }, + }, + }); + render(); + await waitFor(() => { + expect(screen.getByTestId("chart-weekday-hours")).toBeInTheDocument(); + }); + expect(screen.getByText("7")).toBeInTheDocument(); +}); diff --git a/web/src/views/analytics.test.tsx b/web/src/views/analytics.test.tsx new file mode 100644 index 0000000..71e940b --- /dev/null +++ b/web/src/views/analytics.test.tsx @@ -0,0 +1,97 @@ +import { render, screen, waitFor } from "@testing-library/react"; +import { afterEach, expect, it, vi } from "vitest"; + +import { serve } from "../test/http.ts"; +import { ROUTES } from "./analytics-routes.ts"; +import { Analytics } from "./analytics.tsx"; + +afterEach(() => { + vi.unstubAllGlobals(); +}); + +it("says it is reading before the answer arrives", () => { + serve(ROUTES); + render(); + expect(screen.getByText("Reading the database…")).toBeInTheDocument(); +}); + +it("shows the headline numbers once they land", async () => { + serve(ROUTES); + render(); + await waitFor(() => { + expect(screen.getByTestId("tiles")).toHaveTextContent("40"); + }); + const tiles = screen.getByTestId("tiles"); + expect(tiles).toHaveTextContent("19:00"); + expect(tiles).toHaveTextContent("35%"); +}); + +it("says where every number came from", async () => { + // Two of the three classes of row were reconstructed rather than observed, + // and a page that does not say so overstates what is known. + serve(ROUTES); + render(); + await waitFor(() => { + expect(screen.getByTestId("sources")).toBeInTheDocument(); + }); + const sources = screen.getByTestId("sources"); + expect(sources).toHaveTextContent("6"); + expect(sources).toHaveTextContent("25"); + expect(sources).toHaveTextContent("9"); + expect(sources).toHaveTextContent("2026-08-29"); +}); + +it("refuses to claim a trend from three days", async () => { + serve(ROUTES); + render(); + await waitFor(() => { + expect(screen.getByTestId("sources")).toHaveTextContent( + "Too few days to state a trend", + ); + }); +}); + +it("draws every chart", async () => { + serve(ROUTES); + render(); + await waitFor(() => { + expect(screen.getByTestId("chart-commands")).toBeInTheDocument(); + }); + for (const id of [ + "chart-hours", + "chart-weekday-hours", + "chart-days", + "chart-calendar", + "chart-retention", + ]) { + expect(screen.getByTestId(id)).toBeInTheDocument(); + } +}); + +it("names the slowest command and where its time went", async () => { + serve(ROUTES); + render(); + await waitFor(() => { + expect(screen.getByTestId("health-table")).toHaveTextContent("pope"); + }); + const table = screen.getByTestId("health-table"); + expect(table).toHaveTextContent("2845 ms"); + expect(table).toHaveTextContent("2156 ms"); +}); + +it("groups near misses by the trigger they nearly hit", async () => { + serve(ROUTES); + render(); + await waitFor(() => { + expect(screen.getByTestId("demand")).toHaveTextContent("!pulapka"); + }); + expect(screen.getByTestId("demand")).toHaveTextContent("17"); +}); + +it("reports how repetitive each pool is", async () => { + serve(ROUTES); + render(); + await waitFor(() => { + expect(screen.getByTestId("pools")).toHaveTextContent("20%"); + }); +}); diff --git a/web/src/views/analytics.tsx b/web/src/views/analytics.tsx new file mode 100644 index 0000000..4c6c3f5 --- /dev/null +++ b/web/src/views/analytics.tsx @@ -0,0 +1,70 @@ +import { useCallback, useEffect, useState } from "react"; + +import type { AnalyticsGroup, Coverage } from "../api/analytics-types.ts"; + +import { api } from "../api/client.ts"; +import { AnalyticsPanels } from "./analytics-panels.tsx"; + +// The footnote every panel is read under. Two of the three classes of row +// were reconstructed rather than observed, and a chart that does not say so +// is a chart that quietly overstates what is known. +export function Sources({ coverage: measured }: Readonly<{ coverage: Coverage }>): React.JSX.Element { + return ( +

+ {measured.events} recorded {measured.events === 1 ? "use" : "uses"} over{" "} + {measured.days} {measured.days === 1 ? "day" : "days"} + {measured.first_day === "" ? "" : ` (${measured.first_day} – ${measured.last_day})`}.{" "} + {measured.sources.live} observed live,{" "} + {measured.sources.journald} reconstructed from the journal with a real hour,{" "} + {measured.sources.daily_uses} recovered as daily counts with no hour at + all — those are counted everywhere and excluded from every chart with an hour on + its axis, because they have none. + {measured.enough_for_a_trend + ? "" + : " Too few days to state a trend; the series is drawn, the direction is not claimed."} +

+ ); +} + +export function Analytics(): React.JSX.Element { + const [groups, setGroups] = useState(null); + const [group, setGroup] = useState(null); + + const refresh = useCallback(() => { + void api.analyticsGroups().then((body) => { + setGroups(body.groups); + }); + }, []); + + useEffect(refresh, [refresh]); + + const pick = useCallback((event: React.ChangeEvent) => { + setGroup(event.target.value === "" ? null : event.target.value); + }, []); + + if (groups === null) { + return

Reading the database…

; + } + + return ( +
+

Analytics

+ + +
+ ); +}