One row per command attempt, not two aggregates written only on success.
command_uses holds last-use only and daily_uses counts successes; both are
rate-limiter state and both are written by store.record_use, which the
dispatcher calls only when a handler returns cleanly. A gate refusal and a
crash returned before it, so a command that failed every time looked exactly
like one nobody ran, and nothing knew which command a counted message was.
command_events now takes every attempt with its outcome, its duration and
the item it served; upstream_calls takes every outbound request made while
serving one. The two writes stay separate on purpose -- recording a refusal
as a use is what slid the cooldown window forward every time somebody was
told to wait -- and test_dispatcher_outcomes asserts last_used does not move.
Backfilled from journald joined against message_events, with daily_uses
filling the residual so the two sources cannot double-count. Two things were
wrong on the first attempt and both were found by measuring rather than by
reasoning: the window has to be symmetric, because sent_at_ms comes from the
sender's phone and the journal stamp from this host (194 of 299 lines landed
*before* their own message, by up to 3.2 s); and matching has to be a global
closest-pair assignment, because per-row greedy cannot separate two commands
seconds apart. Together those took the dropped rows from 137 to zero, and
recovered 418 invocations back to 29 August.
Also: !analityka in the group, twelve Polish lines, naming at most three
people and only ones whose display name has been seen; an Analytics tab with
Nivo charts; and continuous latency measurement, which answers the open
question in DOCS-latency.md -- a text send is 235 ms and an image send about
2.2 s, so the upload really is where the time goes.
EPHEMERAL_TABLES was doing two unrelated jobs and is now two tuples: what a
schema bump rebuilds, and what leaving a group clears. daily_uses left the
first (it is a backfill source now) and stayed in the second, and
_drop_reshaped took over the repair a bump used to do for it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T4QTp3h5Q3MvfrwGV8xqPf
!sztuka posts a work of art, skewed 60% Polish; !wiersz posts one short
public-domain Polish poem, whole. Both follow the rule the rest of the bot
already keeps: the repository commits identifiers, and the words arrive over
the wire when somebody asks.
!wiersz was written the other way first, and the gate caught it. A pack of
short public-domain poems was harvested, name-filtered and committed, and
check_personal_data.sh refused it: three lines carried a real group member's
given name, and two still did after a filter that had discarded 31% of the
corpus. That is structural, not a tuning problem -- Polish verse capitalises
every line, so a line-initial capital carries no signal, and that is exactly
where a name sits. The pack now holds Wolne Lektury slugs; the poem, its
title and its author are fetched at post time and appear nowhere in git.
It is also the better version: with no name filter there are no false
positives, so the pool is larger and the attribution is a full name again.
The gate then refused the art category list too, over a painter whose given
name is somebody's. That is the popes.json case exactly, so it takes the
same answer: a file-scoped allowlist entry, still refused everywhere else.
The Commons harvest is now three modules by what they are, not by where the
line count fell: commons_api is the conversation with Commons, commons_harvest
turns categories into a pool of licensed page ids, commons_cache makes an
interrupted run resumable. Along the way it gained licences in the same
request that lists a category (the second pass was 1,500 requests at 2.3s
each), maxlag instead of a guessed sleep, 5xx retries because one 504 used to
destroy a whole run, and Wikidata for works by creators of Polish
citizenship: the Polish pool grew 15,221 -> 20,092.
Batching categories through CirrusSearch looked like a further 6x and was
measured out again: 2,404 files where the authoritative enumeration returns
2,429, a different ~1% missing on every run.
The offline run is 19s where it was four minutes, almost all of it the
deafness gate waiting out three thirty-second liveness probes to prove
something provable in three; the tick is configuration now, and e2e enforces
a 40s budget on itself so that never silently comes back.
!pułapka (and !trap, !sidła, !wnyk, !potrzask, 26 spellings in all) posts a
photograph of the literal device -- a bear trap, a mousetrap, a snare, a fish
weir -- captioned with what Commons says it is.
The pool is 2218 Commons page ids, harvested by tool/gen_traps.py from a
curated seed list and committed. Page ids and not filenames on purpose: a
filename carries photographers' and sitters' names, and one of those
colliding with a label somebody mints from the chat would wedge
check_personal_data.py permanently, over data nobody can edit by hand.
_repo_vocabulary() subtracts aliases.json and nothing else, so it would not
have caught it. An integer cannot carry a name.
Nothing is cached. !papież caches because 267 portraits are a closed set a
group works through; a uniform draw from 2218 warms a cache essentially
never, so this fetches per call the way !kot and !pies do.
What the harvest measured, and now enforces:
- five plausible seed categories do not exist (Snares, Man traps, Hunting
traps, Wasp traps, Animal trapping). A missing category is an empty answer,
not an error, so verify_seeds() asks Commons before harvesting and fails
loudly naming all of them.
- depth three from Category:Fish traps reaches Alfred Sisley's net paintings.
Depth one, from seeds picked by hand.
- "<thing> in art" categories are paintings: Mousetraps in art put a Robert
Campin altarpiece in the pool. Pruned by pattern, since Commons mints a new
one whenever somebody needs it. Booby traps, the Wolfsangel, camera traps
and animals dying in traps are pruned by name.
- Commons rate-limits hard, so a partial harvest must not be committed
silently: 429 backs off, and a short pool refuses to write at all.
Descriptions are HTML and sometimes say nothing, so the tags come out, the
Fisheries records' trailing catalogue list is cut, a description naming only
the country falls back to the filename, and the file's own page is always
linked.
!łapka is not an alias: it resolves to !papa at distance two, which
test_nearest.py fails on, and rightly -- the suggestion would be a coin toss.
signal_bot/commons.py is the imageinfo query both Commons commands now share;
portraits.py keeps its behaviour and its tests unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01416TegJ3sa8xL5qGGgrygP
An unrecognised "!word" was dropped in silence, which made a typo
indistinguishable from the bot being down and threw away the best available
list of commands worth writing. Three things now happen at the one place in
dispatcher.py that used to be a bare `return`.
A near miss is corrected -- "!cta" asks whether you meant "!kot". The
distance is optimal string alignment rather than difflib's ratio: a
transposition is the typo people actually make, it costs one edit here, and
it shreds the matching blocks a ratio counts ("!cta"/"!cat" scores 0.75 but
"!cta"/"!kot" scores 0.50, so no cutoff separates them on principle).
A word with no near miss points at !zaproponuj, which files an idea from
inside the group. Either way the word and the message are recorded, and the
management page grows an Ideas tab that lists them.
Three things stay silent, each for its own reason. "!kot ladny" is a real
trigger with words after it and always has been silent. "!!" is punctuation.
And a command claimed in some *other* group is neither answered nor recorded
-- the album's triggers are real people's names, and pack_scoping.yaml says
it out loud: a refusal would itself announce that the album exists. The
existence check is therefore global, which is what pack_add already asks.
Measured rather than guessed, against the shipped aliases.json:
- a minimum candidate length, because "!ja" is one edit from ~40 of the 676
two-letter words somebody could type; 69 of them got a suggestion before
the rule and 18 after.
- the two-edit budget starts at six characters, not five. At five it put
four pairs of *different* commands within reach of each other --
!pies/!popes, !days/!cats, !dzis/!dni, !mnie/!dni -- and costs nothing to
give up, because every real typo this was written for is one edit away.
test_no_two_commands_are_within_reach_of_each_other fails if a future
alias reopens the gap.
Both new tables are keyed on the envelope's own sent_at_ms, never the wall
clock: signal-cli replays undelivered frames on reconnect, and a clock-keyed
table collects a duplicate row per reconnect instead of ignoring one.
The branch is gated on should_count -- which excludes the bot's own envelope,
and that is the loop guard, since handle() checks is_own nowhere else -- and
wrapped in its own try/except, because it runs above the one that wraps a
handler and Bot.listen has no exception handling of its own.
Suggestions live in SQLite, not in a tracked file, and the export goes to the
state directory. check_personal_data learns the names it refuses from the
live database, so it catches a name somebody has typed and cannot catch a
misspelling of one -- and an unrecognised-command corpus is made of
misspellings. A group-writable file in a public tree is both a leak and a
wedge. The export therefore carries
deliberate !zaproponuj notes in full and unrecognised words as counts only;
bodies stay on the loopback-only page.
Verified on the live bot: restarted, both tables created with every existing
row count unchanged, !cta and !zaproponuj !pogoda delivered through
/_selftest, both rows read back out of the live database, and the journal
names neither the word nor the body.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01416TegJ3sa8xL5qGGgrygP
Two things the last commits could not have caught.
CI was red on every commit for as long as scripts/e2e.sh has existed, and
for two reasons, not one. ci.yml never installed ffmpeg or exiftool while
install_service.sh installs both for the host, so tool/e2e_fixtures.py could
not build its JPEG and MP4 fixtures. That step is added and verified by
running the binaries rather than by trusting apt's exit code.
Underneath it was a flake that ffmpeg alone would not have fixed. With the
tools present, e2e still failed two scenarios per run -- never the same two
twice, always one reporting "captured 0" and another "captured 2". The cause
is fake_signal/cli.py sleeping a fixed settle after each injection and then
reading the sends: a reply that fetches an image and shells out to exiftool
lands after that expires, so it arrives after the NEXT scenario's reset and
is counted against that one instead. It now waits for the count it expects,
with a deadline, and then for a beat of quiet -- returning the moment the
count arrives would make "expected 1, captured 2" unobservable, because the
surplus send is exactly the one that has not landed yet. A scenario expecting
nothing skips the poll and waits out the settle, which is the only way to
assert that nothing happens. --patience is exposed so the tests can drive the
give-up path without waiting it out; test_cli went from 45s to 0.4s. Two
consecutive full runs: 28 ok, 0 FAIL, exit 0.
And the page had no styling at all. Times New Roman on white, default
buttons, and the tagger's charcoal theme lost in the absorption -- the jsdom
suite asserts behaviour and cannot see any of that. @kuhyx/web-ui's tokens.css
is now imported and src/index.css lays the page out with those tokens only,
inventing no values.
The visual pass also found a real defect the tests could not: the active tab
was unreadable after being clicked. `button:hover` is more specific than a
bare attribute selector, so with the pointer still resting on the tab you
just clicked, the active rule lost and --on-fill dark text repainted onto
--surface-2 dark. Confirmed by reading the computed style, not by eye:
bg was rgb(56,49,46) where it should have been the accent. The active rule
now claims :hover too.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01416TegJ3sa8xL5qGGgrygP
./web.sh serves health, the configuration and the photo album on
127.0.0.1:8140 -- the port tool/tagger.py vacates. React 19, Vite 8,
TypeScript 6.0.3, ESLint 10 with strictTypeChecked + stylisticTypeChecked +
unicorn + sonarjs + perfectionist, Vitest 4 at 100% on all four metrics.
No coverage exclude list, including for main.tsx. This repository treats a
coverage omit as a suppression, and "the DOM bootstrap is awkward to assert"
is the same sentence as "this code is untested" -- so the bootstrap is a shim
thin enough to test in jsdom, and the album view holds its file list apart
from the album so the update path stopped carrying a "what if it is null"
branch that cannot happen. An untestable branch is an untested one.
TypeScript stays at 6.0.3: typescript-eslint 8.68.0 declares
typescript >=4.8.4 <6.1.0, and the fleet-wide allowlist entry says the
compiler waits for the linter. The freshness gate then caught two pins the
reference repo was carrying stale -- eslint-plugin-unicorn 73 -> 74 and a
floating node-version: "24" in CI -- both bumped rather than allowlisted.
Traps the gates found, all of them real:
- .gitignore has a bare lib/, which matches at any depth, so web/src/lib/
would have been silently untracked. The source lives in src/api/.
- CI's shellcheck glob was `run.sh scripts/*.sh`, so tag.sh was unlinted in
CI for its whole life and web.sh would have inherited that. Now ./*.sh.
- tsconfig.json is strict JSON, not JSONC: check-json is generic and
tsconfig being commentable is a convention, not a requirement.
- The personal-data gate refused an invented phone number in a test, which
is exactly what it is for.
The album's warning that names a command moved with the command, and every
path that pointed at tool/tagger.py now points at ./web.sh. The historical
mentions stay: they explain why the code is shaped as it is.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01416TegJ3sa8xL5qGGgrygP
A second process rather than more routes on the bot's own app. This one can
rewrite .env, and the bot's port already serves /_selftest, which posts for
real into a real group; keeping them apart means the surface that can
reconfigure the bot only listens while somebody has the page open.
The tagger's seven routes move here unchanged, out of tool/ and into the
coverage gate -- which is the reason an unconditional ImportError once sat
in tagger.py through two releases. The media route is still indexed by
integer, and that is not a style choice: a filename from a client is a path,
and the directory holds private photographs. FastAPI refuses a name with 422
before the handler runs, which the tests assert.
guard.py is the part that was not asked for. Binding to loopback stops the
network, not the browser already running on this machine, and two attacks
reach a loopback port from an ordinary web page: a cross-origin form, which
can only send urlencoded, multipart or plain text, so requiring
application/json refuses it with no token to keep; and DNS rebinding, whose
request still carries the attacker's hostname in Host. Forty lines instead
of authentication, which is the right trade for something started by hand
and stopped again a few minutes later.
assets.py branches on whether the page has been built, because StaticFiles
raises at construction on a missing directory -- mounting it unconditionally
would take the app down on a fresh clone, and in CI, where pytest never runs
pnpm build. Both branches are tested; "not built" answers with the command.
The container restart goes through signal_bot.external, wrapped in
asyncio.to_thread: it is synchronous with a 60s default, and blocking the
loop would freeze the page showing the restart.
100% branch coverage across all eleven modules, no omits, no pragmas.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01416TegJ3sa8xL5qGGgrygP
envfile.py edits the file in place, line by line: only the KEY= line of a
key that actually changed is replaced, so the paragraph explaining why
GROUP_IDS is empty survives an edit made in a browser. A regenerated file
would be correct and would still lose most of what that file is for.
Temp-then-replace with the mode reapplied before the swap, because .env
holds the account number and is edited from a tab that can be closed
mid-save.
envkeys.py is the policy, as data. Nothing here re-implements validation:
load_config(candidate, use_dotenv=False) is what startup runs, so an edit
this bot would refuse at boot is refused at the form instead, with the
journal's own message.
Read-only is wider than "the phone number". API_BASE_URL repoints every
outgoing message and attachment fetch at the next restart, which is worse
than changing the number; DATABASE_PATH orphans the history; PHOTO_PACK_DIR
cannot be validated at all, because _photo_pack_dirs() reads os.environ
directly rather than the mapping it is handed -- deliberately, since that is
what keeps the suite away from a real album.
GROUP_IDS and SELFTEST_GROUPS are editable but refused while no group
listing has been read. parse_group_ids only checks syntax; startup also
requires the group to be one the bot actually joined, so writing one blind
with the container down turns a scoping mistake into a unit that will not
start.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01416TegJ3sa8xL5qGGgrygP
The shell entry points got this in 99aa194; running the .py directly still
died with ModuleNotFoundError, which is the command the file's own docstring
had been telling people to use for months.
tool/venv_bootstrap.py creates .venv, installs the project and re-executes
the same command line through it. It is stdlib-only by necessity: it runs on
an interpreter where this package does not import yet, so it cannot route
through signal_bot.external, the one module allowed to start a process. That
is why it gets the second scoped per-file-ignore rather than the rule being
bent -- kuhy's call. test_external.py now asserts a closed list of two, so a
third file still fails the gate.
Two things that are less obvious than they look:
It runs on import rather than exposing a function to call. A call would have
to happen before the caller's own third-party imports, and code before
imports is E402 -- so a call-based design needs a lint suppression in every
file it protects. Importing the module is the whole interface.
And isort would sort "venv_bootstrap" after "fastapi", silently breaking the
ordering the module depends on. It gets its own isort section ahead of the
standard library, so the formatter enforces the order instead of undoing it.
Caught while verifying: ensure() used one check to decide both whether to
re-exec and whether to install. The interpreter you typed never grows a
fastapi, so every launch reinstalled the project first. Splitting them
(ready() probes the venv, missing() probes the caller) takes a warm run from
a full pip install to 0.47s. Pinned by a test.
Verified on a clone with no .venv: first run installs and prints one line,
second and third print nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBrE1b9HvFw3xvG9bJiDFt
tool/tagger.py did not run at all: 48957a1 renamed pack.default_dir to
default_dirs and missed the caller, so the tagger raised ImportError at
module scope through two releases. Nothing could have caught it -- tool/ is
outside --cov and mypy ran with [signal_bot, fake_signal]. Both are widened
here, and tests/test_tool_imports.py imports every script under tool/.
--import was a filename check and shutil.copy2. Pointed at a phone export
that overlaps the album it re-imported the same photo under whatever name
the exporter used that time. It now identifies files by magic bytes, strips
them through the same media_strip path !add_photo uses, and dedups on exact
bytes twice: the source hash, then -- only for what survives -- the hash
after stripping. sources.json records both, so a second run on the same
folder is a no-op. What the album refuses is moved to _skipped/<reason>/
beside the source, never deleted.
Two things that made "strip on import" impossible until now:
media_tags claims -all= zeroes the QuickTime date tags and allows them by
value. It does not; measured, it leaves CreateDate at its real value, and
every mp4 was being refused. -time:all= is what zeroes them.
And a tag allowlist cannot see the worst class at all. IMG_6924.mp4 carries
mebx timed-metadata holding per-frame face bounding boxes and a face id,
and those survive a complete tag strip because they live in mdat as opaque
samples. media_tracks.py drops the track with ffmpeg and verifies with
ffprobe that only video and audio survived. This is not the ffmpeg mistake
media_strip warns about: that was ffmpeg replacing exiftool for tags, this
is track removal with exiftool still doing every tag, and the gate is a
positive read-back rather than an exit code.
The roster was write-once. The setup form only appeared on an empty album,
Album.setup overwrote the file whole, and keys were positional (p{len+1}),
so inserting somebody renumbered everyone after them and repointed every
tags.json entry at the wrong person. Keys now come from the highest number
in use, the form merges, and a person can be added mid-session.
--untagged filters the page and never the server's list. /media/{index} and
/api/tag both index the full album, so a shorter list server-side would
show item i of the album while tagging item i of the filtered set.
Measured on the live album: 174 source files -> 1 not media, 82 already in
pack, 24 duplicate in batch, 67 imported, and re-running imports nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBrE1b9HvFw3xvG9bJiDFt
You asked for !add_photo in testbot without the real album being reachable
there. One album cannot do that: its people.json "groups" list *is* the
privacy guarantee, so making the commands exist in a second group necessarily
makes the real photos postable there. So the bot now loads several albums.
PHOTO_PACK_DIR takes a comma-separated list, the spelling GROUP_IDS and
TRUSTED_INVITERS already use. Each album carries its own group scope, its own
people, its own tags and its own custom labels; they share nothing but the
process they run in.
That makes trigger collisions between albums legal for the first time, and the
registry had to learn it. Two albums are two different sets of people and both
may reasonably call somebody "!alfa" -- no message can ever mean both, because
a command a group is not scoped to is not a command there at all. So a trigger
may be claimed more than once, and DuplicateTriggerError now fires only where
the group sets *overlap*. Overlapping albums are refused at startup rather than
resolved by registration order: a coin toss about which private album receives
a photo is the outcome this whole design exists to prevent.
Two consequences worth naming:
- registry.resolve() takes an optional group. Without one it still answers
"does anything claim this word", which is what pack_add's mint check wants;
with one it answers "what runs here", which is what the dispatcher wants.
resolve_with_args no longer resolves-then-checks, because with two claimants
the first one found may be the wrong group's.
- CommandContext moved to signal_bot/context.py to keep registry.py under the
250-line cap. A filing change: that file is "which word runs what", this one
is "what the thing that runs may do".
DOCS-add-photo.md is likewise a split, not new prose -- DOCS-photo-pack.md had
grown past the cap and the ingest path is its own subject now.
Verified live: the unit serves both albums, the real group resolves its 13
album commands and testbot its 7, and !losowe / !alfa / !dodaj all fire in
testbot through the live dispatcher. 997 tests, 100% branch coverage, 28 e2e
scenarios.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqqE22jp94B1ypwxppdr1o
Two things the first live run found, neither of which any test could have.
**exiftool was invisible to the unit.** Arch's perl-image-exiftool installs to
/usr/bin/vendor_perl/exiftool. That is on an interactive shell's PATH, because
perl's profile.d puts it there, and it is *not* on a systemd user unit's, whose
default is "/usr/local/bin:/usr/bin". So shutil.which found it in every test
and in every shell, and the first real !add_photo under the unit answered
"exiftool is not installed; refusing to store .jpg" -- with exiftool plainly
installed. media_strip.locate() now tries PATH and then the known install
locations, and install_service.sh verifies the result through the bot's own
resolver under the unit's PATH rather than through `command -v`, which answers
for the wrong shell. Fixing the lookup rather than setting PATH in the unit:
the latter is a note somebody has to remember on the next machine.
The absent-exiftool test had to blind *both* lookups to keep covering its
branch -- on a machine that has the package the fallback finds it, which is
precisely how the gap survived.
**The self-test could not carry a photo.** Signal never delivers an account
its own sends: measured against the live container, a single-device account
gets no sync copy either, and a sent attachment is not added to the container's
attachment store (only inbound ones are). So !add_photo could not be exercised
the way every other command is. selftest.build_frame and POST /_selftest now
take attachment ids that the container's store already holds, so the handler
fetches them back over the real GET /v1/attachments/{id} -- the ingest path
under test is the live one, not a hand-fed buffer. Still fail-closed on
SELFTEST_GROUPS.
Verified live, against real Signal and the real container, in the testbot
group, on a throwaway album scoped to that group so the private one was
neither read nor written:
- a geotagged JPEG (GPS + Make) went in as "!dodaj alfa wakacjelive"
- the stored file reports zero non-structural tags, and neither the GPS
marker nor the Make string survives in its bytes
- GET /v1/attachments/<id> now 404s and the file is gone from disk, so the
second copy really is deleted
- the typed label appears zero times in the journal
- !losowe wakacjelive, !alfa and an unknown label all answered without error
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqqE22jp94B1ypwxppdr1o
Send a photo captioned "!add_photo alfa wakacje" in one of the album's own
groups and it lands in the pack tagged with both, retrievable through !alfa
and the new !losowe immediately -- no restart, and no tagger session on the PC.
A label matching nobody becomes a custom tag, and !losowe is the only door to
one, which is what keeps a live swap a *data* swap: it can never register a
trigger or widen a command's groups.
No typed label travels. Labels become pseudonym keys in a new registry
arg_resolver, at the dispatcher, before the line that logs the command name --
so no handler, journal entry, command_uses row or reply body ever holds a word
somebody typed. The reply is counts and pseudonyms: "dodano · p1: 4 · t1: 1".
Stripping is exiftool with a *positive* check on the output, never an exit
code. ffmpeg was tried first and fails silently on photos: given a JPEG with a
real APP1 Exif segment it returned a byte-identical file, GPS intact, exit 0.
The surviving-tag allowlist is per tag *name* rather than per group, because
measurement showed a group-level "must be empty" rule rejects every format
except JPEG -- a valid PNG always reports its IHDR, an MP4 some fifty QuickTime
tags that are the container itself. Codec configuration (CompressorName,
VendorID, the HEVC parameter set) is on the list by decision: CompressorName
lives in the avc1 sample description and exiftool cannot write it at all, so
the alternative was refusing every video and every iPhone HEIC. Location and
identity tags are still refused, and the e2e run re-reads the stored files to
prove a QuickTime geotag did not survive.
.webm and .svg stay on the format allowlist and are stored *unstripped*:
exiftool 13.55 can write neither, and its SVG reader does not even surface
<metadata>. That is a deliberate, documented hole, not an oversight.
Two things this had to defuse along the way:
- personal_data_sources now reads custom.json, so "!add_photo kot <photo>"
would have outlawed the word "kot" and refused every future commit, fixable
only by hand-editing an album the committer cannot see. Minting now refuses
a word that is already a command, and the gate subtracts the repo's own
alias vocabulary from custom labels only -- curated people.json names are
never excused.
- signal_bot/external.py is now the single subprocess site in the repository,
so ruff's S603 needs exactly one scoped per-file-ignore instead of one per
module that shells out. A test asserts nothing else imports subprocess.
A failed add changes nothing: media and the three manifests are written, the
album is reloaded, and only a load that succeeds is published; a load that
fails rolls all four back and keeps serving the pack that was already working,
without raising -- Restart=always would turn a ConfigError into a loop.
977 tests, 100% branch coverage, no omits and no suppressions. scripts/e2e.sh
gains 8 scenarios driven against the real exiftool on media generated at run
time, and asserts the invariants a reply cannot: that a refused upload is not
stored, that a message with no attachment leaves custom.json byte-identical,
and that signal-cli's own copy of every ingested attachment is gone.
Closes TODO-add-photo.md.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqqE22jp94B1ypwxppdr1o
The leak this repository had was not carelessness with a secret. It was
somebody reaching for a real name because it was the example in front of
them -- a fixture built from a live database, an envelope pasted out of a
running container, a help text written while looking at the actual group.
Three different people-shaped mistakes, none of which a rule about "be
careful" would have stopped.
So it is a gate. The hard part is that the thing being checked must not
contain the list of what to refuse, or the check *is* the leak. It reads the
names at run time from outside: the photo pack's manifest, the message
database, the local .env. It never prints the word it matched, because this
output reaches CI logs. On a machine with none of that private data it says
"shapes only" out loud rather than passing quietly, and CI enforces the half
that needs nothing -- phone numbers, uuids and Signal group ids, compared
against an allowlist of values invented for the fixtures.
It runs on staged files, on the commit message -- where two of the leaks
actually travelled -- and over the whole tree in CI.
It paid for itself twice before it was wired up. The live test group's id was
in tests/test_config.py in two spellings, and truncated in three more files,
after four rounds of grepping by hand had pronounced the tree clean -- nobody
thinks to grep for base64. Then it refused the commit that introduced it,
because DOCS-privacy.md had used a real name as the example of a name.
An allowlist line may be scoped to one file, which is how the given names
that are also popes stay legal in popes.json and refused in every fixture
that has no business carrying them. The threshold is three characters,
because one of this group's given names is three characters long.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017w1HZ3cZ7neL3rFdQ6VMBA
The scrub before this one caught the truncated spelling in the document and
missed the one in the test, which held the first half of a real Signal uuid
and the account name beside it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017w1HZ3cZ7neL3rFdQ6VMBA
This repository is public, and five things in it were not invented.
The worst was mine, today: the tagger's help text used the group's actual
first names and nicknames as its worked example, in a file that is on
GitHub. It uses ala/basia/celina now, like every other example here.
The rest predate this work. DOCS-signal-envelopes.md carried the operator's
own phone number and the bot's, captured from a live envelope; conftest.py
had the same number as its trusted inviter; and test_trust.py pinned a group
member's full Signal uuid as a fixture. Numbers keep their shape so the
documents still show what a real envelope looks like, but no digit of them
belongs to anybody.
Comments that reached for a real first name to illustrate the diacritic case
now use an invented one.
None of this removes them from the history, which is a separate decision.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017w1HZ3cZ7neL3rFdQ6VMBA
Tomorrow's run would have posted four times into the same group: the day
the bot owed from the catch-up, the week that closed inside it, today, and
August's summary. Four notifications in one evening is how a daily tally
becomes something people mute.
They are blocks in a single message now, separated by a rule because Signal
renders no markdown. That also makes the send atomic: the group gets all of
it or none of it, and nothing is marked either way, where before a day was
marked before its own weekly summary had even been attempted -- so a summary
that failed was simply lost.
The composition moved to reporting.py, both because bot.py hit the 250-line
cap and because it is worth having pure: "what would the group be told?" is
now answerable with no transport and no send.
The closing-time caveat reads the clock instead of REPORT_HOUR. The
scheduled run fires at the configured minute and still says exactly that; a
run started by hand at some other hour now tells the truth rather than
repeating a time nobody waited for.
The 0.1.6 entry gains a bullet rather than taking a new version: no group has
been told about 0.1.6 yet -- every one of them is still recorded at 0.1.5 --
so the release it describes has not happened from their side.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017w1HZ3cZ7neL3rFdQ6VMBA
The first group in the account's listing was checked by default, and the
first group in this account's listing is a test group. Somebody who typed
the names, hit save and never looked at the radios would have scoped a
private album to the wrong chat -- silently, because the commands then
simply do nothing in the chat they were meant for. Nothing is pre-selected
now, and saving without a group is refused.
Also replaces a test of mine that failed one run in sixteen: it drew a
two-way random choice five times and demanded both outcomes. The property
worth pinning is that a choice lands on a reading that *has* photos, which
is deterministic when only one of them does.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017w1HZ3cZ7neL3rFdQ6VMBA
Per-person photo commands, where the mechanism is public and the album is
not. Nothing identifying is in this repository: who the people are, what
fires their photos and which file shows whom all live in one directory
outside it, and the tests, the docs and the offline e2e run use three
invented people instead.
Several names in one message intersect -- "!alfa beta" is a photo with both
-- and where an entry offers a choice the bot picks among the readings that
actually have a photo, so a question the album could answer is not refused.
A combination nobody photographed gets one line rather than silence, which
from the group's side is indistinguishable from a broken bot.
Scoping is enforced in the dispatcher, not in the handler: a command the
group is not allowed resolves to nothing, so it is not answered, not counted
as a command, and never named in a log line outside its own groups. A
handler that quietly returned would leak all three. Commands are named by
pseudonym (pack:p1), so the journal and command_uses say somebody asked for
a photo and nothing about whose.
Video works: the attachment goes out as data:video/mp4;filename=...;base64,
which is the container's own spelling, read off its /swagger/doc.json.
Told it is a photo, Signal renders a clip as an unplayable file.
tool/tagger.py serves one page on loopback to fill the album in -- names are
typed into the page, never as arguments, and it prints how many photos each
command would have when it stops, because a command with zero is dead on
arrival.
Also renames the real people who were still sitting in the stats test
fixtures. They remain in the history; the point is that nothing new is
added to it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017w1HZ3cZ7neL3rFdQ6VMBA
"Wersja 0.1.7 «Callistus I»" told nobody whether Callistus was the
fifteenth pope or the hundredth, and the changelog was the one place the
ladder is actually read. pope_line() derives both facts from popes.json --
the ordinal, and the `reign` string exactly as the table spells it, so the
one reigning pope keeps his single year instead of being padded into a
range he never had.
It appears in all four places a version is named: the group announcement
(header and every block of a backlog), each section of DOCS-changelog.md,
/healthz, and the !papiez caption, which now describes the release's pope
in the same sentence it uses for every other one.
The changelog's copy is checked rather than trusted, like the heading name
already was: a test walks the raw file and compares each line against
popes.json, so a hand-edited ordinal fails here instead of in a group.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017w1HZ3cZ7neL3rFdQ6VMBA
The scheduler fired exactly on time tonight and posted nothing, because
publish_report covered the previous full day and the bot's first recorded
day was today: "no messages in group.WTI5dVptbG... on 2026-08-29". Correct by
code, a day late to everybody reading the group.
So the report covers the day it runs on. The count closes at the report
time and render_day says so, because a total that quietly omits three hours
is worse than one that admits to it. Everything that made a late day
recoverable stays: the per-group marker still prevents a double post, and
days_to_report still yields every day an outage swallowed.
closing_periods keeps taking a day rather than reading a clock, so a week
now closes on the Sunday evening it ends -- off a day that is itself still
running -- and a catch-up on a week-old Sunday still gets it right.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017w1HZ3cZ7neL3rFdQ6VMBA
Thirteen commands, and the only list of them was a README table nobody in a
Signal group can read — which had itself gone stale two releases earlier,
still advertising four commands after nine more had shipped. 0.1.1 gave each
command an output that says what the command is; that only helps once you
already know to type it.
!pomoc (also !pomocy, !help, !komendy, !commands) prints all fourteen,
grouped, one Polish line each.
The interesting part is that it cannot go stale. CommandRegistry.register now
takes the description as a REQUIRED argument, so a command that does not
explain itself does not compile; and render_help prints any command that
help_command.GROUPS forgot to file, under no heading, rather than dropping
it. Two independent ways to fail closed, because the README proved that a
hand-maintained second list is a list that drifts.
Command also gained `primary`, the first raw alias. The trigger set could not
supply it: those are folded skeletons in an unordered frozenset, so all that
survives of "!papież" there is "!papiez", and the styled Unicode aliases come
out as zalgo. aliases.json already lists the canonical Polish spelling first.
The full trigger table moves to DOCS-commands.md. It did not fit in README —
that file was already sitting exactly on the shared 250-line cap, and the
gate caught the overflow on the first commit attempt.
Exercised in testbot at the unbumped version before this bump: the live
dispatcher ran it and posted the list, journal clean. pytest 708 green at
100% branch coverage, e2e green including a new scenario that asserts !pomoc
mentions a command registered by a different module.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017w1HZ3cZ7neL3rFdQ6VMBA
Reported from the group: the same "Najczęściej piszesz między 12:00 a
13:00" line came back for Ala and for Jan Nowak, which is too much
of a coincidence to be a statistic.
It was the backfill. The 342 messages carried over from the pre-0.1.0
counter have no timestamps, so the seeded rows got a placeholder hour of
12 -- and favourite_hour was the one query that never excluded the
backfilled day. For every person those synthetic noons outnumbered every
real message they had sent since, so all seven got the same answer. The
genuine hours are 19-21: this group talks in the evening.
The comment above that placeholder said "the day is marked anyway, so
nothing that cares about hours will read these", which is exactly the
kind of reasoning that stops being true the moment somebody adds a
query. So the fix is not another `exclude` argument to remember: the
absent hour is now -1, which no real hour can be, and hour queries
filter `local_hour >= 0`. Forgetting the exclusion can no longer invent
an answer, and night_owl gets the same guard even though its 0..2 window
happened to miss the old placeholder.
Live database migrated with the service stopped and a backup taken: 342
rows moved from noon to unknown. Verified afterwards on the real data --
Ala 19:00, Jan Nowak 19:00, kuhy 20:00, and the two people whose
only messages are backfilled correctly get no hour line at all.
Not released. Sitting at 0.1.1 until kuhy says otherwise.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015d48t4qsQLBJnDwDeo2Nom
Two things the group hit within minutes of 0.1.0, both reported by real
users rather than found by a test.
Somebody typed !korona, got "Za mało danych — wróć za kilka dni." and
nothing else. That answer tells them neither what the command does nor
what it will eventually show, which makes it worse than useless on a
release day when almost every command is empty by design. Every command
now leads with what it is, and the empty answer says what it will show
once there is history and which date it is counting from.
And somebody asked whether "Najczęściej piszesz o 12:00" meant the whole
hour 12-13. It did, and the label did not say so. It now reads "między
12:00 a 13:00". Same fix in kind for the nightly summary line, which
said "średnio 47 znaków" without saying per what -- now "na wiadomość".
A one-line wording fix takes the next pope exactly like a feature does,
per DOCS-changelog.md. Exercised in testbot at the unbumped version
first: !korona, !ja, !dni and !rekordy all ran clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015d48t4qsQLBJnDwDeo2Nom
A Sunday closes a week, a month's last day closes the month, and
31 December closes the year. Keyed off the day being *reported* rather
than off today, which is the only reading that stays right when the bot
has been offline: catching up on a week-old Sunday now also posts that
week's summary, and there is a test asserting the five messages a
three-day outage produces.
One scheduler, a date predicate. Three background tasks firing at the
same wall-clock time into the same group is three independent ways for
an evening to go wrong, for no gain over an if-statement. A failed
period summary is deliberately best-effort and runs after the day has
been marked, so it cannot make the bot re-post the daily one tomorrow.
Also: the quote frame arrived while this was being written, and it
carries a trap. `quote` has its own nested `attachments` array holding
the *quoted* message's media -- so a text reply to somebody's photo has
attachments in the payload that belong to somebody else. Reading them
would credit the replier with a photo they did not post, in a group
where people reply to photos constantly, inflating one number without
ever looking wrong.
The parser already read the top level only and is correct. It is now
pinned by a test built from the exact captured frame, and written up in
DOCS-signal-envelopes.md, because "it happens to be right" and "it
cannot silently stop being right" are different things. This is the same
wrong-nesting-level mistake that once made the bot ignore its own
stickers.
test_bot.py went past the 250-line cap on the way, so the report and
catch-up tests move into files named for what they cover.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015d48t4qsQLBJnDwDeo2Nom
!staty, !wczoraj, !tydzień, !miesiąc, !rok, !rekordy, !korona, !dni and
!ja, under 29 triggers with both Polish spellings of each.
None takes an argument. registry.resolve matches the whole folded body
against a trigger, so "!ja ala" resolves to nothing; teaching it to
split on whitespace would silently make "!kot pls" fire where today it
does nothing. Each question gets its own trigger instead.
!wczoraj renders through the same code path as the nightly post, which
is not a convenience: the bot cannot trigger its own 21:37 report, so
without byte-identical output there is no way to check the headline
feature of this release short of waiting for the evening.
Two guards worth naming:
- test_every_trigger_is_unique builds the whole registry.
DuplicateTriggerError subclasses ValueError, and ValueError is in
startup_retry.RETRYABLE -- so a colliding alias would not crash the
bot, it would retry forever logging "signal-cli not reachable yet".
Checked at test time instead of never.
- No trigger may contain a digit: fold() maps 0134578 onto letters, so
"!7dni" would resolve as "!tdni". Also asserted.
Every command answers on an empty history rather than crashing, which is
the state the group will actually be in for the next few days: records,
the crown and the weekday table all say "za mało danych" because their
only day so far is the backfilled one, and they exclude it.
Verified against a copy of the live database -- all nine render
correctly on the real group's data.
One test bug caught while writing it: it seeded "today" and excluded
2026-08-30, which on the day it was written were the same date, so it
passed for the wrong reason. Fixed dates now.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015d48t4qsQLBJnDwDeo2Nom
Around twenty things could be said about a day. Saying all of them is a
wall nobody reads; saying whichever three are computed first is
arbitrary. So each candidate declares a priority and a magnitude and the
picker takes the top few, with two rules that make the result readable:
one line per family, so "set a record", "had 60% of the day" and "kept
the crown" do not become three sentences about the same person; and
fully deterministic ordering, so a day renders identically twice.
Nine candidates so far: personal records with what they beat, the
group's busiest day ever, a photo record, crown streaks (naming everyone
when it is shared), a silence broken, a monologue, one person dominating
the day, an unusual showing for that weekday, and a night owl.
Every candidate is (DayFacts) -> Highlight | None with the same
signature. On day one of this release almost all of them return None,
and a shape where "not yet" is the ordinary case is much easier to get
right than one where every guard is bespoke.
Two off-by-ones fixed by writing the sentence out and checking it:
- "odezwał się po 3 dniach ciszy" was counting four. The day somebody
last spoke is not silent, and neither is the day they came back on, so
the gap excludes both ends. The function is now named for that.
- A personal record on somebody's first ever day was being announced as
a record. It is, trivially, and saying so every time is noise.
Verified against the live database, read-only on a copy: the real group
renders correctly at 324 messages, claims no average length for the
backfilled day, and fires no highlights at all -- which is right, since
there is one day of history for them to stand against.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015d48t4qsQLBJnDwDeo2Nom
The queries the statistics are made of, all derived rather than cached --
two sources of truth for one number is how a leaderboard comes to
disagree with itself, and this history is small enough that recomputing
is free.
Periods take an inclusive span of stored Warsaw day strings, never a
range of timestamps, so a daylight-saving change cannot move a boundary.
Trend reports None rather than 0 when there is no previous period:
"nothing to compare against yet" and "no change" are different
statements and the group will read them differently.
A tie shares the crown and keeps every tied run alive, per kuhy's call.
Streaks count *recorded* days, so a day nobody spoke does not end a run
-- there was no crown to lose -- and neither does a day the bot was
offline for, which it could not have judged.
Every record excludes the backfilled day. It is a 22-hour window filed
as a day, and leaving it in would hand somebody a personal best they
never set.
Two bugs found by running the code rather than reading it:
- current_streaks reported every streak as zero. Ending somebody's run
dropped them from the tally along with the length they had already
accumulated, so the surviving dict was empty by the time it returned.
Now the run ends and the count it reached is kept.
- daily_winners depended on SQLite's grouping order to find a day's
leader. It happened to work because uuids sorted conveniently in the
fixtures. Ordering by messages DESC makes the first row per day the
leader by construction, which is both correct and obviously so.
No suppressions: the period queries compare an exclusion parameter
against '' instead of pasting half a WHERE clause into an f-string,
which is a SQL-injection shape even when every input is a constant.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015d48t4qsQLBJnDwDeo2Nom
DOCS-signal-envelopes.md exists because this wire format is not obvious
and guessing at it has cost real bugs -- the sticker read off the wrong
nesting level, and 0.0.3's !gekon shipping green on an assumption nobody
had checked against the live API. The recurring cost is that answering
"what does a frame with an attachment actually look like?" needed a code
change, a restart, and somebody sending one at the right moment.
So at DEBUG the dispatcher now logs each frame's *shape*: keys and value
types, never values. Two fields keep their values because they are wire
format rather than anything a person said -- a MIME type and a groupInfo
type. A message body, a name, a number, a uuid, quoted text and a
sticker id all come out as "str". An album collapses to one element and
a count instead of printing the same dict twenty times.
Verified against a live frame: the body, the quoted text, the uuid and
the phone number do not survive, and contentType does.
This also had to fix DEBUG being unusable. LOG_LEVEL=DEBUG set the root
logger, which turns on httpx, httpcore and asyncio; the bot polls a
group listing every 30 seconds, so the one interesting line arrived
buried and journald started dropping. Libraries are now held at WARNING
whatever the bot is set to -- at INFO httpx was already emitting an
"HTTP Request: GET ... 200 OK" line every thirty seconds, forever.
Measured on the running unit: 17 library lines per minute down to 2.
Two bugs in that setup found by its own tests rather than by review:
basicConfig is a documented no-op once the root logger has a handler, so
the level silently did not apply under pytest; and the floor belongs at
INFO too, not only below it.
A self-send does not produce a capturable attachment envelope, by the
way -- signal-cli emits no syncMessage for a send made over the same
connection. Two probe images returned HTTP 201 and no frame came back,
so the inbound shape still needs a human to post one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015d48t4qsQLBJnDwDeo2Nom
Two gaps kuhy pointed at, both about not losing messages the bot was not
awake for.
Missed *counts* already worked and now have a test saying so: an event
is keyed by Signal's own send timestamp, signal-cli keeps receiving in
json-rpc mode while the bot is down, and the replay on reconnect is
absorbed by the primary key. Messages sent during an outage come back
filed under the day they were actually sent.
Missed *reports* did not. A bot that was down at 21:37 never posted that
day and never went back for it. The last day reported is now written
down per group, so returning from an outage posts every day it owes,
oldest first, bounded at a week. A day nobody spoke on is marked settled
rather than reconsidered forever; a day whose send *failed* is
deliberately left unmarked so the next run retries it.
The pre-0.1.0 counter is carried over rather than discarded: 342
messages across four groups, 319 of them in the real group. What the
counter cannot say is when, so before assuming, four places were checked
for a real inbound timestamp -- journald (commands only), signal-cli's
account.db (19 tables, none inbound; message_send_log is the bot's own
outgoing resend buffer), Signal Desktop (unlinked, empty, and it does
not backfill on link), and signal-api/attachments (the one dated
artifact). The attachment mtimes put 12 files on 29 Aug against 79 on
30 Aug inside the counter's window, which is why the whole counter is
filed under 30 Aug rather than split.
That day is marked in group_meta, and a marked day claims no average
message length -- those messages have a count but no lengths, so
"średnio 0 znaków" would be a measurement nobody took. The mark is also
what will keep the day out of personal records: a merged 22-hour day
would otherwise hand somebody a record they never set.
render_day's options move into a DayView value object; the keyword list
had reached eight and this renderer is about to be shared by several
commands that each want a different subset.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015d48t4qsQLBJnDwDeo2Nom
The tally was one integer per person, deleted after each post. Nothing
dated survived it, so weeks, months, records, streaks and weekday
averages were all unanswerable. This replaces it with an append-only
per-message log holding metadata only -- never a message body -- and
rebuilds the nightly report on top.
What changes for the group: a photo posted without a caption now counts
(it was worth zero), stickers count, the bot's own messages no longer
do, people who said nothing appear with a 0, and the report covers the
previous full calendar day in Europe/Warsaw instead of "since the last
post" -- which used to merge two days whenever a group went quiet.
Notable decisions, each with the trap it avoids:
- Events and people are PERMANENT_TABLES: neither forget_group nor a
future SCHEMA_VERSION bump may drop them. Both paths used to cost a
day's counter and would now cost the entire history, and forget_group
fires on a listing that merely omits members.
- is_own comes from which envelope branch carried the payload, not from
comparing identities. The account is spelled as a uuid on some frames
and a +E.164 number on others, so any comparison misses one of them.
- The roster is stashed in refresh(), not _adopt(), which returns early
once the watched set stops changing -- i.e. always, in the steady
state.
- The roster matches on uuid OR number because /v1/groups spells kuhy as
+48123456789 while his messages carry an ACI uuid. Joining on uuid
alone would have listed him twice every night, once with 0.
- Members whose display name we have never seen are counted, not named:
their only spellings are a uuid or a phone number.
- (group, uuid, sent_at_ms) is the primary key, so a websocket
redelivery after a reconnect cannot inflate the counts.
- Days come from a stored Warsaw date string, never a timestamp range:
2026-03-29 is 23 hours long and 2026-10-25 is 25.
- Polish number agreement is a real rule with a 12-14 exception, so
"44 znaki" and "45 znaków" are both spelled correctly.
config.py and startup.py were two lines under the 250-line cap and both
had to grow, so config_env and startup_retry come out of them first, as
pure moves.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015d48t4qsQLBJnDwDeo2Nom
Adds !papież / !papiez / !pope: a random pope as a photo, captioned with his
Latin name, reign and pl.wikipedia link. popes.json already carried a Commons
portrait filename for all 267 and nothing had ever consumed it.
The Latin name comes from the Latin column of Poczet papieży, not from the
la.wikipedia sitelink. The sitelink resolved for all 267 and was still wrong:
la.wikipedia numbers several popes differently, so #154 came out as
"Stefan IX — Stephanus X" in a single caption, which reads as an off-by-one.
The table's own column says "Stephanus Nonus" and cannot disagree with the
name beside it. LATIN_OVERRIDES ships empty; it is a tripwire, not a backlog.
Portraits are cached on disk under CacheDirectory=signal-bot, named from the
ordinal so no upstream string ever reaches a path, and bounded at 267 entries
so nothing needs evicting. Verified that CacheDirectory wins against
ProtectHome=read-only before relying on it.
Also adds POST /_selftest and scripts/selftest.sh. Signal never delivers an
account its own sends, so no command has ever been exercised end-to-end
without asking a human to type it. This injects a real frame into the live
dispatcher; it is loopback-only and refuses any group not in SELFTEST_GROUPS,
which is empty by default.
Verified against the real endpoints, not just the fake: 267/267 portraits
resolve on Commons, regeneration is byte-identical on every previously
shipped field, and !papież was driven 7 times through the live dispatcher in
testbot, returning distinct popes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015d48t4qsQLBJnDwDeo2Nom
A "!cat" in a busy group produced a cat picture floating loose in the chat with
nothing tying it to whoever asked. Replies now quote the triggering message.
The container advertises this and always could: /v1/about reports
capabilities {"v2/send": ["quotes", "mentions"]}, and /v2/send takes
quote_timestamp, quote_author and quote_message. What was missing was on this
side -- parse_envelope discarded the envelope's timestamp, which is the id
Signal quotes a message by, so nothing downstream could have built a quote even
in principle. IncomingMessage now carries it, preferring the payload's own
value over the envelope's for the case where they ever disagree.
Quote is a value object rather than three more parameters threaded through
every send: the Transport protocol has five methods and two of them send.
CommandContext grows reply_image() so quoting lives in one place instead of
each handler reaching for ctx.transport itself, which is what animals.py did.
quote_of() returns None when the timestamp or author is missing. Answering the
room is a worse reply than a quoted one but a much better outcome than a
payload the container rejects.
The e2e gate learned to tell the two apart: Expectation.quotes demands a send
either does or does not point at a message, defaulting to None so the ten
existing scenarios stay indifferent. Without it the fake would have accepted an
unquoted reply as a pass.
Verified in the testbot group on a real Signal client before this bump, per
kuhy's rule that a bump is a publish rather than a checkpoint: he sent !cat and
confirmed the picture renders attached to his message as a reply.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015d48t4qsQLBJnDwDeo2Nom
Release n is pope n, so the version is a fact about the ordinal rather than a
judgement about the size of a change. The constraint that fixes the scheme is
Jan Pawel II at 2.13.7: eight patches per minor with majors opening at ordinals
33 and 153 gives (264 - 153) = 13 * 8 + 7 exactly, and both earlier majors
happen to divide evenly by eight too, so no bracket needs a fudge factor. The
anchors are not arbitrary -- 33 is Sylwester I, pope through the Edict of Milan
and Nicaea, and 153 is Wiktor II, the first pope elected after the 1054 schism.
popes.json is scraped by tool/gen_popes.py rather than recalled, because
antipopes, Benedykt IX's three separate reigns and the Stefan II question all
shift the count and being wrong by one moves Jan Pawel II off 2.13.7 silently.
The generator refuses to write a list that is not 267 contiguous entries with
Jan Pawel II at 264. A second, independent parse of the same page agreed on
every ordinal, which is the only reason to believe either.
On startup each watched group is told about the releases it has not seen. A
group with no record has seen none of them, so the first announcement starts at
0.0.0 "Swiety Piotr Apostol" rather than skipping to whatever is current --
watching the ladder start at Peter is the point of naming releases after popes.
A group added years from now would otherwise be handed the whole backlog, so
the message is capped at ten releases and says how many it left out. The
version is recorded before the message is sent: with Restart=always and
RestartSec=5, a crash between the two would re-announce on every restart, and a
missed announcement is far cheaper than a looping one.
DOCS-changelog.md is the single source -- the bot parses it, so there is no
second machine-readable copy to drift. It is DOCS-, not CHANGELOG.md, because
the md-naming gate allows four namespaces and that is not one of them.
Deliberately NOT bumping SCHEMA_VERSION for the new group_meta table:
_apply_schema drops every table on a bump, so bumping to add a table would have
thrown away the live tallies to gain nothing. Verified against the real
database -- group_meta appeared and all four counts survived the restart.
The schema moves to its own module so store.py stays under the 250-line cap,
and /healthz now names the build, because there was previously no way to ask a
running bot which version it is.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015d48t4qsQLBJnDwDeo2Nom
The vote and rate-limit gates were salvaged from an old command that is gone;
the docstring still named it, and test_store.py used its name as the arbitrary
string for the vote-dedup fixture. Neither reference means anything now, and
the pre-rewrite history that did contain it has been dropped, so these were the
only two left in the tree.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015d48t4qsQLBJnDwDeo2Nom
_permitted feeds both _join_pending and _adopt, and _adopt treats anything
missing from its result as a removal: it drops the group, purges its tally
with store.forget_group and logs "no longer a member". So one listing whose
membership no longer vouches -- the same group has been seen spelling a
member as +E.164 in one row and as a bare uuid in another, and a row can
come back with no members field at all -- would silently un-watch a live
group and delete its counts, all three claims false.
That is exactly the narrowing discovery.py's own docstring forbids ("a
failed refresh never narrows the set"), reintroduced by the gate one commit
earlier. Adoption is itself the vouch, so an already-watched id short-
circuits the check; only groups the bot has never adopted are tested.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KbjfWdAyHbCubac7wJKJZW
Today the bot was up, healthy and answering nothing for three hours. A host
firewall change (fixed in testsAndMisc 334c0cd9) cut every Docker container
off the internet; signal-cli lost Signal and reconnected in a loop from
15:32:50 onward. Nothing noticed:
- the container's own healthcheck answered 204
- /v1/groups answered 200 out of local state, so discovery saw no change
- the bot's WebSocket to the container stayed OPEN, so the transport's
reconnect logic never fired
- /healthz answered {"status": "ok"} the entire time
The only symptom was silence, which is also what a quiet evening looks like.
signal_bot/liveness.py asks the one question that cannot be answered from a
cache -- GET /v1/devices, which needs a live round trip -- but only after
SILENCE_SECONDS (120) with no frame of any kind, so a busy group costs
nothing and an idle one costs one request per window. Three consecutive
failures and /healthz answers 503 with the reason. Verified end to end in
scripts/e2e.sh against the real process: 503 after 84s with the fake's
connection cut, back to 200 on its own 30s after restoring it.
scripts/signal_api_watchdog.sh + a two-minute user timer do the recovery the
bot cannot do for itself, since a wedged bot cannot restart the container it
depends on. Two tiers, so the steady state costs no Signal round trip at
all: the bot's own /healthz first, the authenticated probe only when that is
not 200. It stops after 3 restarts in 6 hours and escalates instead -- a
deregistered account or a blocked network is not fixable by restarting, and
a restart loop would hide it. Its own bash suite stubs curl and docker and
drives every branch including the give-up cap.
signal_bot/trust.py answers the other half: TRUSTED_INVITERS decides who may
put this bot in a group. The gate sits in GroupDirectory._permitted, which
both _join_pending and _adopt already route through -- gating only invites
would gate the door nobody uses, since prime_profile.sh exists precisely so
that an add becomes a direct join. Evidence is either the group-update
envelope, which is the only frame naming an inviter, or a trusted identity
being a full member in the listing. A refusal is logged once, with that
group's members, so the operator can see which string to add.
verify_allowlist moved to membership.py to keep discovery.py under the
250-line gate; it was already the module that owns MembershipError.
Verified live, by stopping the container by hand and touching nothing else:
18:54:35 docker stop signal-api
18:56:36 liveness: failure 1 of 3 after 123s of silence
18:57:36 failure 3 of 3 -> /healthz answers 503 from 18:57:45
18:58:18 watchdog: probe failed (1/2)
19:00:28 watchdog: probe failed (2/2) -> docker restart signal-api
19:00:36 liveness: receive path recovered after 8 failed probe(s)
19:01:13 watchdog: signal-api is back
Six minutes, unattended, against three hours of silence this afternoon. All
four real groups are still adopted with TRUSTED_INVITERS=+48123456789 set,
and no group was refused.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KbjfWdAyHbCubac7wJKJZW
The wrap-up script stages dirty *tracked* files, so every newly created
file in the multi-group change -- the group-id module, the membership
verifier, their tests, the isolation scenario and prime_profile.sh --
was listed as untracked and then left behind. The tree compiled locally
because the files exist on disk; CI checked out a repo where config.py
imported signal_bot.groups and nothing provided it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013yaBT45KhHqxTKVCTA87iy
First live reply, as its own registered account. `!cat` and `!dog` from the
phone return images -- verified in the production shape (the bot *invited to
an already-existing group*) and under a systemd user unit.
Four failures, each of which left the bot looking perfectly healthy:
- Invited is not joined. Signal only adds an account outright if the adder
holds its profile key; with no profile the add becomes a pending invite,
and signal-cli cannot accept one addressed to a phone number ("Cannot find
service ID for self"). Group listed, websocket up, /healthz OK, zero
messages. register_bot.sh now sets the profile *before* the add, and
startup refuses on `member: false`.
- MODE=json-rpc-native is a GraalVM build with no AWT: text sends work,
every attachment 400s with "Can't load library: awt". Both mode guards now
demand exactly json-rpc, and start_container replaces a wrong-mode
container instead of reusing it.
- The API spells one group two ways -- /v1/groups returns `group.<base64>`,
envelopes carry what that decodes to -- so the scoping gate rejected the
bot's own group. e2e.sh now models both spellings; reverting the fix fails
4 scenarios, which is why the offline suite missed it before.
- BIND_PORT in .env was dead: __main__ read os.getenv before anything loaded
a .env. The bind address comes from Config now.
A taken port used to be uvicorn logging the collision and exiting 0, which
every supervisor reads as success. portcheck.py refuses to start, exits
non-zero, and names the holder by reading /proc directly -- no subprocess, no
PATH assumptions, testable against a fake proc tree. Default port is 8137:
not 8000, and below the ephemeral range so the kernel cannot take it.
The bot now runs as signal-bot.service (Restart=always, linger checked rather
than assumed, StartLimitBurst so a config error fails visibly instead of
looping). Verified by SIGKILL and by re-running a command under the unit.
100% branch coverage, 230 tests, all gates green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013yaBT45KhHqxTKVCTA87iy
Shelter cannot work on this phone, and it would not have helped anyway.
It cannot work: setting a Device Owner makes Android enable
no_add_managed_profile, no_add_private_profile and no_add_clone_profile on
every user, which kills Shelter, Island, Private Space and app cloning
alike. Verified on-device rather than inferred -- `pm create-user
--profileOf 0 --managed` returns "Cannot add user: no_add_managed_profile is
enabled". This is the Android 11 COMP deprecation, enforced in three places
in DevicePolicyManagerService and unchanged through Android 16.
It would not have helped: a second Signal *app* is not a second Signal
*account*. Every account needs its own phone number, so the second number
was always the real requirement -- and once you have one, signal-cli can
register it as a PRIMARY device with no phone app, no QR scan and no work
profile at all. That is strictly better than both the linked-device setup
and the Shelter plan: the bot cannot read personal DMs and never posts as
the account owner.
scripts/register_bot.sh drives that flow. It gates on signal-cli >= 0.14.7
rather than hoping: stable container tag 0.100 bundles 0.14.5, and on
2026-03-06 Signal made the post-quantum ratchet mandatory and de-registered
every account whose client lacked it -- a stale client is how a bot dies
silently months later. Verified that the recommended CI tag really does
bundle 0.14.7 instead of assuming it.
Rehearsed offline, not written blind. fake_signal grew /v1/accounts,
/v1/register/{number}, /v1/register/{number}/verify/{token} and
/v1/groups/{number}, so the whole runbook including every error path
(missing captcha, stray verify, wrong code, unregistered number, empty group
list, out-of-range and non-numeric picks) runs against the fake. The picker
was proven to resolve the group the user actually pointed at.
Prompts now read from $SIGNAL_TTY instead of /dev/tty directly. A hard-coded
/dev/tty made the interactive path untestable without a real terminal; the
seam is the same idea as the injected clock and transport on the Python side,
and is what let the picker be rehearsed at all.
live_smoke.sh and register_bot.sh share scripts/lib_signal.sh so the two
cannot drift, and shellcheck now runs with -x so the sourced lib is actually
linted from its callers.
201 tests, 100% branch coverage, all 23 gates green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013yaBT45KhHqxTKVCTA87iy
