Two things the first live run found, neither of which any test could have.
**exiftool was invisible to the unit.** Arch's perl-image-exiftool installs to
/usr/bin/vendor_perl/exiftool. That is on an interactive shell's PATH, because
perl's profile.d puts it there, and it is *not* on a systemd user unit's, whose
default is "/usr/local/bin:/usr/bin". So shutil.which found it in every test
and in every shell, and the first real !add_photo under the unit answered
"exiftool is not installed; refusing to store .jpg" -- with exiftool plainly
installed. media_strip.locate() now tries PATH and then the known install
locations, and install_service.sh verifies the result through the bot's own
resolver under the unit's PATH rather than through `command -v`, which answers
for the wrong shell. Fixing the lookup rather than setting PATH in the unit:
the latter is a note somebody has to remember on the next machine.
The absent-exiftool test had to blind *both* lookups to keep covering its
branch -- on a machine that has the package the fallback finds it, which is
precisely how the gap survived.
**The self-test could not carry a photo.** Signal never delivers an account
its own sends: measured against the live container, a single-device account
gets no sync copy either, and a sent attachment is not added to the container's
attachment store (only inbound ones are). So !add_photo could not be exercised
the way every other command is. selftest.build_frame and POST /_selftest now
take attachment ids that the container's store already holds, so the handler
fetches them back over the real GET /v1/attachments/{id} -- the ingest path
under test is the live one, not a hand-fed buffer. Still fail-closed on
SELFTEST_GROUPS.
Verified live, against real Signal and the real container, in the testbot
group, on a throwaway album scoped to that group so the private one was
neither read nor written:
- a geotagged JPEG (GPS + Make) went in as "!dodaj alfa wakacjelive"
- the stored file reports zero non-structural tags, and neither the GPS
marker nor the Make string survives in its bytes
- GET /v1/attachments/<id> now 404s and the file is gone from disk, so the
second copy really is deleted
- the typed label appears zero times in the journal
- !losowe wakacjelive, !alfa and an unknown label all answered without error
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqqE22jp94B1ypwxppdr1o
Send a photo captioned "!add_photo alfa wakacje" in one of the album's own
groups and it lands in the pack tagged with both, retrievable through !alfa
and the new !losowe immediately -- no restart, and no tagger session on the PC.
A label matching nobody becomes a custom tag, and !losowe is the only door to
one, which is what keeps a live swap a *data* swap: it can never register a
trigger or widen a command's groups.
No typed label travels. Labels become pseudonym keys in a new registry
arg_resolver, at the dispatcher, before the line that logs the command name --
so no handler, journal entry, command_uses row or reply body ever holds a word
somebody typed. The reply is counts and pseudonyms: "dodano · p1: 4 · t1: 1".
Stripping is exiftool with a *positive* check on the output, never an exit
code. ffmpeg was tried first and fails silently on photos: given a JPEG with a
real APP1 Exif segment it returned a byte-identical file, GPS intact, exit 0.
The surviving-tag allowlist is per tag *name* rather than per group, because
measurement showed a group-level "must be empty" rule rejects every format
except JPEG -- a valid PNG always reports its IHDR, an MP4 some fifty QuickTime
tags that are the container itself. Codec configuration (CompressorName,
VendorID, the HEVC parameter set) is on the list by decision: CompressorName
lives in the avc1 sample description and exiftool cannot write it at all, so
the alternative was refusing every video and every iPhone HEIC. Location and
identity tags are still refused, and the e2e run re-reads the stored files to
prove a QuickTime geotag did not survive.
.webm and .svg stay on the format allowlist and are stored *unstripped*:
exiftool 13.55 can write neither, and its SVG reader does not even surface
<metadata>. That is a deliberate, documented hole, not an oversight.
Two things this had to defuse along the way:
- personal_data_sources now reads custom.json, so "!add_photo kot <photo>"
would have outlawed the word "kot" and refused every future commit, fixable
only by hand-editing an album the committer cannot see. Minting now refuses
a word that is already a command, and the gate subtracts the repo's own
alias vocabulary from custom labels only -- curated people.json names are
never excused.
- signal_bot/external.py is now the single subprocess site in the repository,
so ruff's S603 needs exactly one scoped per-file-ignore instead of one per
module that shells out. A test asserts nothing else imports subprocess.
A failed add changes nothing: media and the three manifests are written, the
album is reloaded, and only a load that succeeds is published; a load that
fails rolls all four back and keeps serving the pack that was already working,
without raising -- Restart=always would turn a ConfigError into a loop.
977 tests, 100% branch coverage, no omits and no suppressions. scripts/e2e.sh
gains 8 scenarios driven against the real exiftool on media generated at run
time, and asserts the invariants a reply cannot: that a refused upload is not
stored, that a message with no attachment leaves custom.json byte-identical,
and that signal-cli's own copy of every ingested attachment is gone.
Closes TODO-add-photo.md.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqqE22jp94B1ypwxppdr1o
The leak this repository had was not carelessness with a secret. It was
somebody reaching for a real name because it was the example in front of
them -- a fixture built from a live database, an envelope pasted out of a
running container, a help text written while looking at the actual group.
Three different people-shaped mistakes, none of which a rule about "be
careful" would have stopped.
So it is a gate. The hard part is that the thing being checked must not
contain the list of what to refuse, or the check *is* the leak. It reads the
names at run time from outside: the photo pack's manifest, the message
database, the local .env. It never prints the word it matched, because this
output reaches CI logs. On a machine with none of that private data it says
"shapes only" out loud rather than passing quietly, and CI enforces the half
that needs nothing -- phone numbers, uuids and Signal group ids, compared
against an allowlist of values invented for the fixtures.
It runs on staged files, on the commit message -- where two of the leaks
actually travelled -- and over the whole tree in CI.
It paid for itself twice before it was wired up. The live test group's id was
in tests/test_config.py in two spellings, and truncated in three more files,
after four rounds of grepping by hand had pronounced the tree clean -- nobody
thinks to grep for base64. Then it refused the commit that introduced it,
because DOCS-privacy.md had used a real name as the example of a name.
An allowlist line may be scoped to one file, which is how the given names
that are also popes stay legal in popes.json and refused in every fixture
that has no business carrying them. The threshold is three characters,
because one of this group's given names is three characters long.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017w1HZ3cZ7neL3rFdQ6VMBA
The scrub before this one caught the truncated spelling in the document and
missed the one in the test, which held the first half of a real Signal uuid
and the account name beside it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017w1HZ3cZ7neL3rFdQ6VMBA
This repository is public, and five things in it were not invented.
The worst was mine, today: the tagger's help text used the group's actual
first names and nicknames as its worked example, in a file that is on
GitHub. It uses ala/basia/celina now, like every other example here.
The rest predate this work. DOCS-signal-envelopes.md carried the operator's
own phone number and the bot's, captured from a live envelope; conftest.py
had the same number as its trusted inviter; and test_trust.py pinned a group
member's full Signal uuid as a fixture. Numbers keep their shape so the
documents still show what a real envelope looks like, but no digit of them
belongs to anybody.
Comments that reached for a real first name to illustrate the diacritic case
now use an invented one.
None of this removes them from the history, which is a separate decision.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017w1HZ3cZ7neL3rFdQ6VMBA
Tomorrow's run would have posted four times into the same group: the day
the bot owed from the catch-up, the week that closed inside it, today, and
August's summary. Four notifications in one evening is how a daily tally
becomes something people mute.
They are blocks in a single message now, separated by a rule because Signal
renders no markdown. That also makes the send atomic: the group gets all of
it or none of it, and nothing is marked either way, where before a day was
marked before its own weekly summary had even been attempted -- so a summary
that failed was simply lost.
The composition moved to reporting.py, both because bot.py hit the 250-line
cap and because it is worth having pure: "what would the group be told?" is
now answerable with no transport and no send.
The closing-time caveat reads the clock instead of REPORT_HOUR. The
scheduled run fires at the configured minute and still says exactly that; a
run started by hand at some other hour now tells the truth rather than
repeating a time nobody waited for.
The 0.1.6 entry gains a bullet rather than taking a new version: no group has
been told about 0.1.6 yet -- every one of them is still recorded at 0.1.5 --
so the release it describes has not happened from their side.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017w1HZ3cZ7neL3rFdQ6VMBA
The first group in the account's listing was checked by default, and the
first group in this account's listing is a test group. Somebody who typed
the names, hit save and never looked at the radios would have scoped a
private album to the wrong chat -- silently, because the commands then
simply do nothing in the chat they were meant for. Nothing is pre-selected
now, and saving without a group is refused.
Also replaces a test of mine that failed one run in sixteen: it drew a
two-way random choice five times and demanded both outcomes. The property
worth pinning is that a choice lands on a reading that *has* photos, which
is deterministic when only one of them does.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017w1HZ3cZ7neL3rFdQ6VMBA
Per-person photo commands, where the mechanism is public and the album is
not. Nothing identifying is in this repository: who the people are, what
fires their photos and which file shows whom all live in one directory
outside it, and the tests, the docs and the offline e2e run use three
invented people instead.
Several names in one message intersect -- "!alfa beta" is a photo with both
-- and where an entry offers a choice the bot picks among the readings that
actually have a photo, so a question the album could answer is not refused.
A combination nobody photographed gets one line rather than silence, which
from the group's side is indistinguishable from a broken bot.
Scoping is enforced in the dispatcher, not in the handler: a command the
group is not allowed resolves to nothing, so it is not answered, not counted
as a command, and never named in a log line outside its own groups. A
handler that quietly returned would leak all three. Commands are named by
pseudonym (pack:p1), so the journal and command_uses say somebody asked for
a photo and nothing about whose.
Video works: the attachment goes out as data:video/mp4;filename=...;base64,
which is the container's own spelling, read off its /swagger/doc.json.
Told it is a photo, Signal renders a clip as an unplayable file.
tool/tagger.py serves one page on loopback to fill the album in -- names are
typed into the page, never as arguments, and it prints how many photos each
command would have when it stops, because a command with zero is dead on
arrival.
Also renames the real people who were still sitting in the stats test
fixtures. They remain in the history; the point is that nothing new is
added to it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017w1HZ3cZ7neL3rFdQ6VMBA
"Wersja 0.1.7 «Callistus I»" told nobody whether Callistus was the
fifteenth pope or the hundredth, and the changelog was the one place the
ladder is actually read. pope_line() derives both facts from popes.json --
the ordinal, and the `reign` string exactly as the table spells it, so the
one reigning pope keeps his single year instead of being padded into a
range he never had.
It appears in all four places a version is named: the group announcement
(header and every block of a backlog), each section of DOCS-changelog.md,
/healthz, and the !papiez caption, which now describes the release's pope
in the same sentence it uses for every other one.
The changelog's copy is checked rather than trusted, like the heading name
already was: a test walks the raw file and compares each line against
popes.json, so a hand-edited ordinal fails here instead of in a group.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017w1HZ3cZ7neL3rFdQ6VMBA
The scheduler fired exactly on time tonight and posted nothing, because
publish_report covered the previous full day and the bot's first recorded
day was today: "no messages in group.WTI5dVptbG... on 2026-08-29". Correct by
code, a day late to everybody reading the group.
So the report covers the day it runs on. The count closes at the report
time and render_day says so, because a total that quietly omits three hours
is worse than one that admits to it. Everything that made a late day
recoverable stays: the per-group marker still prevents a double post, and
days_to_report still yields every day an outage swallowed.
closing_periods keeps taking a day rather than reading a clock, so a week
now closes on the Sunday evening it ends -- off a day that is itself still
running -- and a catch-up on a week-old Sunday still gets it right.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017w1HZ3cZ7neL3rFdQ6VMBA
Thirteen commands, and the only list of them was a README table nobody in a
Signal group can read — which had itself gone stale two releases earlier,
still advertising four commands after nine more had shipped. 0.1.1 gave each
command an output that says what the command is; that only helps once you
already know to type it.
!pomoc (also !pomocy, !help, !komendy, !commands) prints all fourteen,
grouped, one Polish line each.
The interesting part is that it cannot go stale. CommandRegistry.register now
takes the description as a REQUIRED argument, so a command that does not
explain itself does not compile; and render_help prints any command that
help_command.GROUPS forgot to file, under no heading, rather than dropping
it. Two independent ways to fail closed, because the README proved that a
hand-maintained second list is a list that drifts.
Command also gained `primary`, the first raw alias. The trigger set could not
supply it: those are folded skeletons in an unordered frozenset, so all that
survives of "!papież" there is "!papiez", and the styled Unicode aliases come
out as zalgo. aliases.json already lists the canonical Polish spelling first.
The full trigger table moves to DOCS-commands.md. It did not fit in README —
that file was already sitting exactly on the shared 250-line cap, and the
gate caught the overflow on the first commit attempt.
Exercised in testbot at the unbumped version before this bump: the live
dispatcher ran it and posted the list, journal clean. pytest 708 green at
100% branch coverage, e2e green including a new scenario that asserts !pomoc
mentions a command registered by a different module.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017w1HZ3cZ7neL3rFdQ6VMBA
Reported from the group: the same "Najczęściej piszesz między 12:00 a
13:00" line came back for Ala and for Jan Nowak, which is too much
of a coincidence to be a statistic.
It was the backfill. The 342 messages carried over from the pre-0.1.0
counter have no timestamps, so the seeded rows got a placeholder hour of
12 -- and favourite_hour was the one query that never excluded the
backfilled day. For every person those synthetic noons outnumbered every
real message they had sent since, so all seven got the same answer. The
genuine hours are 19-21: this group talks in the evening.
The comment above that placeholder said "the day is marked anyway, so
nothing that cares about hours will read these", which is exactly the
kind of reasoning that stops being true the moment somebody adds a
query. So the fix is not another `exclude` argument to remember: the
absent hour is now -1, which no real hour can be, and hour queries
filter `local_hour >= 0`. Forgetting the exclusion can no longer invent
an answer, and night_owl gets the same guard even though its 0..2 window
happened to miss the old placeholder.
Live database migrated with the service stopped and a backup taken: 342
rows moved from noon to unknown. Verified afterwards on the real data --
Ala 19:00, Jan Nowak 19:00, kuhy 20:00, and the two people whose
only messages are backfilled correctly get no hour line at all.
Not released. Sitting at 0.1.1 until kuhy says otherwise.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015d48t4qsQLBJnDwDeo2Nom
Two things the group hit within minutes of 0.1.0, both reported by real
users rather than found by a test.
Somebody typed !korona, got "Za mało danych — wróć za kilka dni." and
nothing else. That answer tells them neither what the command does nor
what it will eventually show, which makes it worse than useless on a
release day when almost every command is empty by design. Every command
now leads with what it is, and the empty answer says what it will show
once there is history and which date it is counting from.
And somebody asked whether "Najczęściej piszesz o 12:00" meant the whole
hour 12-13. It did, and the label did not say so. It now reads "między
12:00 a 13:00". Same fix in kind for the nightly summary line, which
said "średnio 47 znaków" without saying per what -- now "na wiadomość".
A one-line wording fix takes the next pope exactly like a feature does,
per DOCS-changelog.md. Exercised in testbot at the unbumped version
first: !korona, !ja, !dni and !rekordy all ran clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015d48t4qsQLBJnDwDeo2Nom
A Sunday closes a week, a month's last day closes the month, and
31 December closes the year. Keyed off the day being *reported* rather
than off today, which is the only reading that stays right when the bot
has been offline: catching up on a week-old Sunday now also posts that
week's summary, and there is a test asserting the five messages a
three-day outage produces.
One scheduler, a date predicate. Three background tasks firing at the
same wall-clock time into the same group is three independent ways for
an evening to go wrong, for no gain over an if-statement. A failed
period summary is deliberately best-effort and runs after the day has
been marked, so it cannot make the bot re-post the daily one tomorrow.
Also: the quote frame arrived while this was being written, and it
carries a trap. `quote` has its own nested `attachments` array holding
the *quoted* message's media -- so a text reply to somebody's photo has
attachments in the payload that belong to somebody else. Reading them
would credit the replier with a photo they did not post, in a group
where people reply to photos constantly, inflating one number without
ever looking wrong.
The parser already read the top level only and is correct. It is now
pinned by a test built from the exact captured frame, and written up in
DOCS-signal-envelopes.md, because "it happens to be right" and "it
cannot silently stop being right" are different things. This is the same
wrong-nesting-level mistake that once made the bot ignore its own
stickers.
test_bot.py went past the 250-line cap on the way, so the report and
catch-up tests move into files named for what they cover.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015d48t4qsQLBJnDwDeo2Nom
!staty, !wczoraj, !tydzień, !miesiąc, !rok, !rekordy, !korona, !dni and
!ja, under 29 triggers with both Polish spellings of each.
None takes an argument. registry.resolve matches the whole folded body
against a trigger, so "!ja ala" resolves to nothing; teaching it to
split on whitespace would silently make "!kot pls" fire where today it
does nothing. Each question gets its own trigger instead.
!wczoraj renders through the same code path as the nightly post, which
is not a convenience: the bot cannot trigger its own 21:37 report, so
without byte-identical output there is no way to check the headline
feature of this release short of waiting for the evening.
Two guards worth naming:
- test_every_trigger_is_unique builds the whole registry.
DuplicateTriggerError subclasses ValueError, and ValueError is in
startup_retry.RETRYABLE -- so a colliding alias would not crash the
bot, it would retry forever logging "signal-cli not reachable yet".
Checked at test time instead of never.
- No trigger may contain a digit: fold() maps 0134578 onto letters, so
"!7dni" would resolve as "!tdni". Also asserted.
Every command answers on an empty history rather than crashing, which is
the state the group will actually be in for the next few days: records,
the crown and the weekday table all say "za mało danych" because their
only day so far is the backfilled one, and they exclude it.
Verified against a copy of the live database -- all nine render
correctly on the real group's data.
One test bug caught while writing it: it seeded "today" and excluded
2026-08-30, which on the day it was written were the same date, so it
passed for the wrong reason. Fixed dates now.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015d48t4qsQLBJnDwDeo2Nom
Around twenty things could be said about a day. Saying all of them is a
wall nobody reads; saying whichever three are computed first is
arbitrary. So each candidate declares a priority and a magnitude and the
picker takes the top few, with two rules that make the result readable:
one line per family, so "set a record", "had 60% of the day" and "kept
the crown" do not become three sentences about the same person; and
fully deterministic ordering, so a day renders identically twice.
Nine candidates so far: personal records with what they beat, the
group's busiest day ever, a photo record, crown streaks (naming everyone
when it is shared), a silence broken, a monologue, one person dominating
the day, an unusual showing for that weekday, and a night owl.
Every candidate is (DayFacts) -> Highlight | None with the same
signature. On day one of this release almost all of them return None,
and a shape where "not yet" is the ordinary case is much easier to get
right than one where every guard is bespoke.
Two off-by-ones fixed by writing the sentence out and checking it:
- "odezwał się po 3 dniach ciszy" was counting four. The day somebody
last spoke is not silent, and neither is the day they came back on, so
the gap excludes both ends. The function is now named for that.
- A personal record on somebody's first ever day was being announced as
a record. It is, trivially, and saying so every time is noise.
Verified against the live database, read-only on a copy: the real group
renders correctly at 324 messages, claims no average length for the
backfilled day, and fires no highlights at all -- which is right, since
there is one day of history for them to stand against.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015d48t4qsQLBJnDwDeo2Nom
The queries the statistics are made of, all derived rather than cached --
two sources of truth for one number is how a leaderboard comes to
disagree with itself, and this history is small enough that recomputing
is free.
Periods take an inclusive span of stored Warsaw day strings, never a
range of timestamps, so a daylight-saving change cannot move a boundary.
Trend reports None rather than 0 when there is no previous period:
"nothing to compare against yet" and "no change" are different
statements and the group will read them differently.
A tie shares the crown and keeps every tied run alive, per kuhy's call.
Streaks count *recorded* days, so a day nobody spoke does not end a run
-- there was no crown to lose -- and neither does a day the bot was
offline for, which it could not have judged.
Every record excludes the backfilled day. It is a 22-hour window filed
as a day, and leaving it in would hand somebody a personal best they
never set.
Two bugs found by running the code rather than reading it:
- current_streaks reported every streak as zero. Ending somebody's run
dropped them from the tally along with the length they had already
accumulated, so the surviving dict was empty by the time it returned.
Now the run ends and the count it reached is kept.
- daily_winners depended on SQLite's grouping order to find a day's
leader. It happened to work because uuids sorted conveniently in the
fixtures. Ordering by messages DESC makes the first row per day the
leader by construction, which is both correct and obviously so.
No suppressions: the period queries compare an exclusion parameter
against '' instead of pasting half a WHERE clause into an f-string,
which is a SQL-injection shape even when every input is a constant.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015d48t4qsQLBJnDwDeo2Nom
DOCS-signal-envelopes.md exists because this wire format is not obvious
and guessing at it has cost real bugs -- the sticker read off the wrong
nesting level, and 0.0.3's !gekon shipping green on an assumption nobody
had checked against the live API. The recurring cost is that answering
"what does a frame with an attachment actually look like?" needed a code
change, a restart, and somebody sending one at the right moment.
So at DEBUG the dispatcher now logs each frame's *shape*: keys and value
types, never values. Two fields keep their values because they are wire
format rather than anything a person said -- a MIME type and a groupInfo
type. A message body, a name, a number, a uuid, quoted text and a
sticker id all come out as "str". An album collapses to one element and
a count instead of printing the same dict twenty times.
Verified against a live frame: the body, the quoted text, the uuid and
the phone number do not survive, and contentType does.
This also had to fix DEBUG being unusable. LOG_LEVEL=DEBUG set the root
logger, which turns on httpx, httpcore and asyncio; the bot polls a
group listing every 30 seconds, so the one interesting line arrived
buried and journald started dropping. Libraries are now held at WARNING
whatever the bot is set to -- at INFO httpx was already emitting an
"HTTP Request: GET ... 200 OK" line every thirty seconds, forever.
Measured on the running unit: 17 library lines per minute down to 2.
Two bugs in that setup found by its own tests rather than by review:
basicConfig is a documented no-op once the root logger has a handler, so
the level silently did not apply under pytest; and the floor belongs at
INFO too, not only below it.
A self-send does not produce a capturable attachment envelope, by the
way -- signal-cli emits no syncMessage for a send made over the same
connection. Two probe images returned HTTP 201 and no frame came back,
so the inbound shape still needs a human to post one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015d48t4qsQLBJnDwDeo2Nom
Two gaps kuhy pointed at, both about not losing messages the bot was not
awake for.
Missed *counts* already worked and now have a test saying so: an event
is keyed by Signal's own send timestamp, signal-cli keeps receiving in
json-rpc mode while the bot is down, and the replay on reconnect is
absorbed by the primary key. Messages sent during an outage come back
filed under the day they were actually sent.
Missed *reports* did not. A bot that was down at 21:37 never posted that
day and never went back for it. The last day reported is now written
down per group, so returning from an outage posts every day it owes,
oldest first, bounded at a week. A day nobody spoke on is marked settled
rather than reconsidered forever; a day whose send *failed* is
deliberately left unmarked so the next run retries it.
The pre-0.1.0 counter is carried over rather than discarded: 342
messages across four groups, 319 of them in the real group. What the
counter cannot say is when, so before assuming, four places were checked
for a real inbound timestamp -- journald (commands only), signal-cli's
account.db (19 tables, none inbound; message_send_log is the bot's own
outgoing resend buffer), Signal Desktop (unlinked, empty, and it does
not backfill on link), and signal-api/attachments (the one dated
artifact). The attachment mtimes put 12 files on 29 Aug against 79 on
30 Aug inside the counter's window, which is why the whole counter is
filed under 30 Aug rather than split.
That day is marked in group_meta, and a marked day claims no average
message length -- those messages have a count but no lengths, so
"średnio 0 znaków" would be a measurement nobody took. The mark is also
what will keep the day out of personal records: a merged 22-hour day
would otherwise hand somebody a record they never set.
render_day's options move into a DayView value object; the keyword list
had reached eight and this renderer is about to be shared by several
commands that each want a different subset.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015d48t4qsQLBJnDwDeo2Nom
The tally was one integer per person, deleted after each post. Nothing
dated survived it, so weeks, months, records, streaks and weekday
averages were all unanswerable. This replaces it with an append-only
per-message log holding metadata only -- never a message body -- and
rebuilds the nightly report on top.
What changes for the group: a photo posted without a caption now counts
(it was worth zero), stickers count, the bot's own messages no longer
do, people who said nothing appear with a 0, and the report covers the
previous full calendar day in Europe/Warsaw instead of "since the last
post" -- which used to merge two days whenever a group went quiet.
Notable decisions, each with the trap it avoids:
- Events and people are PERMANENT_TABLES: neither forget_group nor a
future SCHEMA_VERSION bump may drop them. Both paths used to cost a
day's counter and would now cost the entire history, and forget_group
fires on a listing that merely omits members.
- is_own comes from which envelope branch carried the payload, not from
comparing identities. The account is spelled as a uuid on some frames
and a +E.164 number on others, so any comparison misses one of them.
- The roster is stashed in refresh(), not _adopt(), which returns early
once the watched set stops changing -- i.e. always, in the steady
state.
- The roster matches on uuid OR number because /v1/groups spells kuhy as
+48123456789 while his messages carry an ACI uuid. Joining on uuid
alone would have listed him twice every night, once with 0.
- Members whose display name we have never seen are counted, not named:
their only spellings are a uuid or a phone number.
- (group, uuid, sent_at_ms) is the primary key, so a websocket
redelivery after a reconnect cannot inflate the counts.
- Days come from a stored Warsaw date string, never a timestamp range:
2026-03-29 is 23 hours long and 2026-10-25 is 25.
- Polish number agreement is a real rule with a 12-14 exception, so
"44 znaki" and "45 znaków" are both spelled correctly.
config.py and startup.py were two lines under the 250-line cap and both
had to grow, so config_env and startup_retry come out of them first, as
pure moves.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015d48t4qsQLBJnDwDeo2Nom
Adds !papież / !papiez / !pope: a random pope as a photo, captioned with his
Latin name, reign and pl.wikipedia link. popes.json already carried a Commons
portrait filename for all 267 and nothing had ever consumed it.
The Latin name comes from the Latin column of Poczet papieży, not from the
la.wikipedia sitelink. The sitelink resolved for all 267 and was still wrong:
la.wikipedia numbers several popes differently, so #154 came out as
"Stefan IX — Stephanus X" in a single caption, which reads as an off-by-one.
The table's own column says "Stephanus Nonus" and cannot disagree with the
name beside it. LATIN_OVERRIDES ships empty; it is a tripwire, not a backlog.
Portraits are cached on disk under CacheDirectory=signal-bot, named from the
ordinal so no upstream string ever reaches a path, and bounded at 267 entries
so nothing needs evicting. Verified that CacheDirectory wins against
ProtectHome=read-only before relying on it.
Also adds POST /_selftest and scripts/selftest.sh. Signal never delivers an
account its own sends, so no command has ever been exercised end-to-end
without asking a human to type it. This injects a real frame into the live
dispatcher; it is loopback-only and refuses any group not in SELFTEST_GROUPS,
which is empty by default.
Verified against the real endpoints, not just the fake: 267/267 portraits
resolve on Commons, regeneration is byte-identical on every previously
shipped field, and !papież was driven 7 times through the live dispatcher in
testbot, returning distinct popes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015d48t4qsQLBJnDwDeo2Nom
A "!cat" in a busy group produced a cat picture floating loose in the chat with
nothing tying it to whoever asked. Replies now quote the triggering message.
The container advertises this and always could: /v1/about reports
capabilities {"v2/send": ["quotes", "mentions"]}, and /v2/send takes
quote_timestamp, quote_author and quote_message. What was missing was on this
side -- parse_envelope discarded the envelope's timestamp, which is the id
Signal quotes a message by, so nothing downstream could have built a quote even
in principle. IncomingMessage now carries it, preferring the payload's own
value over the envelope's for the case where they ever disagree.
Quote is a value object rather than three more parameters threaded through
every send: the Transport protocol has five methods and two of them send.
CommandContext grows reply_image() so quoting lives in one place instead of
each handler reaching for ctx.transport itself, which is what animals.py did.
quote_of() returns None when the timestamp or author is missing. Answering the
room is a worse reply than a quoted one but a much better outcome than a
payload the container rejects.
The e2e gate learned to tell the two apart: Expectation.quotes demands a send
either does or does not point at a message, defaulting to None so the ten
existing scenarios stay indifferent. Without it the fake would have accepted an
unquoted reply as a pass.
Verified in the testbot group on a real Signal client before this bump, per
kuhy's rule that a bump is a publish rather than a checkpoint: he sent !cat and
confirmed the picture renders attached to his message as a reply.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015d48t4qsQLBJnDwDeo2Nom
Release n is pope n, so the version is a fact about the ordinal rather than a
judgement about the size of a change. The constraint that fixes the scheme is
Jan Pawel II at 2.13.7: eight patches per minor with majors opening at ordinals
33 and 153 gives (264 - 153) = 13 * 8 + 7 exactly, and both earlier majors
happen to divide evenly by eight too, so no bracket needs a fudge factor. The
anchors are not arbitrary -- 33 is Sylwester I, pope through the Edict of Milan
and Nicaea, and 153 is Wiktor II, the first pope elected after the 1054 schism.
popes.json is scraped by tool/gen_popes.py rather than recalled, because
antipopes, Benedykt IX's three separate reigns and the Stefan II question all
shift the count and being wrong by one moves Jan Pawel II off 2.13.7 silently.
The generator refuses to write a list that is not 267 contiguous entries with
Jan Pawel II at 264. A second, independent parse of the same page agreed on
every ordinal, which is the only reason to believe either.
On startup each watched group is told about the releases it has not seen. A
group with no record has seen none of them, so the first announcement starts at
0.0.0 "Swiety Piotr Apostol" rather than skipping to whatever is current --
watching the ladder start at Peter is the point of naming releases after popes.
A group added years from now would otherwise be handed the whole backlog, so
the message is capped at ten releases and says how many it left out. The
version is recorded before the message is sent: with Restart=always and
RestartSec=5, a crash between the two would re-announce on every restart, and a
missed announcement is far cheaper than a looping one.
DOCS-changelog.md is the single source -- the bot parses it, so there is no
second machine-readable copy to drift. It is DOCS-, not CHANGELOG.md, because
the md-naming gate allows four namespaces and that is not one of them.
Deliberately NOT bumping SCHEMA_VERSION for the new group_meta table:
_apply_schema drops every table on a bump, so bumping to add a table would have
thrown away the live tallies to gain nothing. Verified against the real
database -- group_meta appeared and all four counts survived the restart.
The schema moves to its own module so store.py stays under the 250-line cap,
and /healthz now names the build, because there was previously no way to ask a
running bot which version it is.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015d48t4qsQLBJnDwDeo2Nom
The vote and rate-limit gates were salvaged from an old command that is gone;
the docstring still named it, and test_store.py used its name as the arbitrary
string for the vote-dedup fixture. Neither reference means anything now, and
the pre-rewrite history that did contain it has been dropped, so these were the
only two left in the tree.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015d48t4qsQLBJnDwDeo2Nom
_permitted feeds both _join_pending and _adopt, and _adopt treats anything
missing from its result as a removal: it drops the group, purges its tally
with store.forget_group and logs "no longer a member". So one listing whose
membership no longer vouches -- the same group has been seen spelling a
member as +E.164 in one row and as a bare uuid in another, and a row can
come back with no members field at all -- would silently un-watch a live
group and delete its counts, all three claims false.
That is exactly the narrowing discovery.py's own docstring forbids ("a
failed refresh never narrows the set"), reintroduced by the gate one commit
earlier. Adoption is itself the vouch, so an already-watched id short-
circuits the check; only groups the bot has never adopted are tested.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KbjfWdAyHbCubac7wJKJZW
Today the bot was up, healthy and answering nothing for three hours. A host
firewall change (fixed in testsAndMisc 334c0cd9) cut every Docker container
off the internet; signal-cli lost Signal and reconnected in a loop from
15:32:50 onward. Nothing noticed:
- the container's own healthcheck answered 204
- /v1/groups answered 200 out of local state, so discovery saw no change
- the bot's WebSocket to the container stayed OPEN, so the transport's
reconnect logic never fired
- /healthz answered {"status": "ok"} the entire time
The only symptom was silence, which is also what a quiet evening looks like.
signal_bot/liveness.py asks the one question that cannot be answered from a
cache -- GET /v1/devices, which needs a live round trip -- but only after
SILENCE_SECONDS (120) with no frame of any kind, so a busy group costs
nothing and an idle one costs one request per window. Three consecutive
failures and /healthz answers 503 with the reason. Verified end to end in
scripts/e2e.sh against the real process: 503 after 84s with the fake's
connection cut, back to 200 on its own 30s after restoring it.
scripts/signal_api_watchdog.sh + a two-minute user timer do the recovery the
bot cannot do for itself, since a wedged bot cannot restart the container it
depends on. Two tiers, so the steady state costs no Signal round trip at
all: the bot's own /healthz first, the authenticated probe only when that is
not 200. It stops after 3 restarts in 6 hours and escalates instead -- a
deregistered account or a blocked network is not fixable by restarting, and
a restart loop would hide it. Its own bash suite stubs curl and docker and
drives every branch including the give-up cap.
signal_bot/trust.py answers the other half: TRUSTED_INVITERS decides who may
put this bot in a group. The gate sits in GroupDirectory._permitted, which
both _join_pending and _adopt already route through -- gating only invites
would gate the door nobody uses, since prime_profile.sh exists precisely so
that an add becomes a direct join. Evidence is either the group-update
envelope, which is the only frame naming an inviter, or a trusted identity
being a full member in the listing. A refusal is logged once, with that
group's members, so the operator can see which string to add.
verify_allowlist moved to membership.py to keep discovery.py under the
250-line gate; it was already the module that owns MembershipError.
Verified live, by stopping the container by hand and touching nothing else:
18:54:35 docker stop signal-api
18:56:36 liveness: failure 1 of 3 after 123s of silence
18:57:36 failure 3 of 3 -> /healthz answers 503 from 18:57:45
18:58:18 watchdog: probe failed (1/2)
19:00:28 watchdog: probe failed (2/2) -> docker restart signal-api
19:00:36 liveness: receive path recovered after 8 failed probe(s)
19:01:13 watchdog: signal-api is back
Six minutes, unattended, against three hours of silence this afternoon. All
four real groups are still adopted with TRUSTED_INVITERS=+48123456789 set,
and no group was refused.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KbjfWdAyHbCubac7wJKJZW
The wrap-up script stages dirty *tracked* files, so every newly created
file in the multi-group change -- the group-id module, the membership
verifier, their tests, the isolation scenario and prime_profile.sh --
was listed as untracked and then left behind. The tree compiled locally
because the files exist on disk; CI checked out a repo where config.py
imported signal_bot.groups and nothing provided it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013yaBT45KhHqxTKVCTA87iy
First live reply, as its own registered account. `!cat` and `!dog` from the
phone return images -- verified in the production shape (the bot *invited to
an already-existing group*) and under a systemd user unit.
Four failures, each of which left the bot looking perfectly healthy:
- Invited is not joined. Signal only adds an account outright if the adder
holds its profile key; with no profile the add becomes a pending invite,
and signal-cli cannot accept one addressed to a phone number ("Cannot find
service ID for self"). Group listed, websocket up, /healthz OK, zero
messages. register_bot.sh now sets the profile *before* the add, and
startup refuses on `member: false`.
- MODE=json-rpc-native is a GraalVM build with no AWT: text sends work,
every attachment 400s with "Can't load library: awt". Both mode guards now
demand exactly json-rpc, and start_container replaces a wrong-mode
container instead of reusing it.
- The API spells one group two ways -- /v1/groups returns `group.<base64>`,
envelopes carry what that decodes to -- so the scoping gate rejected the
bot's own group. e2e.sh now models both spellings; reverting the fix fails
4 scenarios, which is why the offline suite missed it before.
- BIND_PORT in .env was dead: __main__ read os.getenv before anything loaded
a .env. The bind address comes from Config now.
A taken port used to be uvicorn logging the collision and exiting 0, which
every supervisor reads as success. portcheck.py refuses to start, exits
non-zero, and names the holder by reading /proc directly -- no subprocess, no
PATH assumptions, testable against a fake proc tree. Default port is 8137:
not 8000, and below the ephemeral range so the kernel cannot take it.
The bot now runs as signal-bot.service (Restart=always, linger checked rather
than assumed, StartLimitBurst so a config error fails visibly instead of
looping). Verified by SIGKILL and by re-running a command under the unit.
100% branch coverage, 230 tests, all gates green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013yaBT45KhHqxTKVCTA87iy
Shelter cannot work on this phone, and it would not have helped anyway.
It cannot work: setting a Device Owner makes Android enable
no_add_managed_profile, no_add_private_profile and no_add_clone_profile on
every user, which kills Shelter, Island, Private Space and app cloning
alike. Verified on-device rather than inferred -- `pm create-user
--profileOf 0 --managed` returns "Cannot add user: no_add_managed_profile is
enabled". This is the Android 11 COMP deprecation, enforced in three places
in DevicePolicyManagerService and unchanged through Android 16.
It would not have helped: a second Signal *app* is not a second Signal
*account*. Every account needs its own phone number, so the second number
was always the real requirement -- and once you have one, signal-cli can
register it as a PRIMARY device with no phone app, no QR scan and no work
profile at all. That is strictly better than both the linked-device setup
and the Shelter plan: the bot cannot read personal DMs and never posts as
the account owner.
scripts/register_bot.sh drives that flow. It gates on signal-cli >= 0.14.7
rather than hoping: stable container tag 0.100 bundles 0.14.5, and on
2026-03-06 Signal made the post-quantum ratchet mandatory and de-registered
every account whose client lacked it -- a stale client is how a bot dies
silently months later. Verified that the recommended CI tag really does
bundle 0.14.7 instead of assuming it.
Rehearsed offline, not written blind. fake_signal grew /v1/accounts,
/v1/register/{number}, /v1/register/{number}/verify/{token} and
/v1/groups/{number}, so the whole runbook including every error path
(missing captcha, stray verify, wrong code, unregistered number, empty group
list, out-of-range and non-numeric picks) runs against the fake. The picker
was proven to resolve the group the user actually pointed at.
Prompts now read from $SIGNAL_TTY instead of /dev/tty directly. A hard-coded
/dev/tty made the interactive path untestable without a real terminal; the
seam is the same idea as the injected clock and transport on the Python side,
and is what let the picker be rehearsed at all.
live_smoke.sh and register_bot.sh share scripts/lib_signal.sh so the two
cannot drift, and shellcheck now runs with -x so the sourced lib is actually
linted from its callers.
201 tests, 100% branch coverage, all 23 gates green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013yaBT45KhHqxTKVCTA87iy
The bot did not work. `run.sh` started a FastAPI app whose
`@app.on_event("startup")` awaited `asyncio.gather()` on two infinite
tasks, so uvicorn never finished starting up and no HTTP was ever served;
the readme's `python main.py` was a no-op with no `__main__` block and no
`uvicorn.run()` call. All three CI workflows were red, one referencing an
`environment.yml` that does not exist.
Rewritten as a package behind an injectable transport seam, which is what
makes the offline environment possible.
Bugs fixed:
- startup deadlock: lifespan spawns the loops instead of awaiting them
- sticker filter read dataMessage.sticker even on syncMessage envelopes, so
the account's own stickers were never filtered (e038772, 53028cc, 07fa90d,
2416cef all fought this symptom)
- no websocket reconnect: a container restart killed the bot silently
- blocking `requests` inside async functions stalled the event loop
- every counted message DMed the whole tally to the bot itself
- images were written to CWD under a filename taken from the remote URL
- one global 10s cooldown shared across all users and commands
- dead code: RECEIVE_URL, get_attachments (no call site), USER_MESSAGE_COUNT
- .env.sample declared PRIMARY/SECONDARY_PHONE_NUMBER and MODE, none read
Scoping is now an enforced invariant. A linked device receives every DM and
every group on the account; GROUP_ID was the only thing narrowing that and
defaulted to "". Config refuses to start unscoped, direct messages are
dropped explicitly rather than by a falsy check, and a scenario asserts that
DMs and other groups produce zero outbound messages.
Offline environment: fake_signal/ serves the real protocol surface, so the
unmodified bot connects to it by pointing API_BASE_URL at it. scripts/e2e.sh
boots both, replays 7 scenarios and exits non-zero on mismatch. A local
Signal server is not achievable (signal-cli compiles its endpoints and trust
store in; Signal-Server needs FoundationDB), so faking this boundary is the
only way to test every feature without an account.
Transport pinned to bbernhard/signal-cli-rest-api:0.100 with
MODE=json-rpc-native. The readme pinned 0.167-dev; the dev tag series counts
separately, so that was older than 0.100, not newer. /v1/receive is only a
WebSocket in a json-rpc mode, so the mode is verified at startup.
Repo standards: ruff select=ALL, mypy --strict, 189 tests at 100% branch
coverage, 250-line cap, exact pins at newest stable, shared gates delegated
to ~/utils, one CI workflow replacing three red ones. No noqa, no
type: ignore, no coverage omits anywhere.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013yaBT45KhHqxTKVCTA87iy
