A photo sent to the chat no longer enters the album by itself. pack_add
strips it, checks it against the album and the queue, and writes it to
pending/ with a sidecar naming the people and labels the sender proposed;
the reply is "czeka na zatwierdzenie". The management page lists the queue
above the tagger, with the proposal pre-ticked, and Approve moves the file
in, mints the labels, records approved_at and asks the bot to rebuild its
table through /_reload -- the same publish path every other data change
takes, so PackHolder.reload is gone. A reload the bot refuses is reported
on the page, not rolled back: the photo is in the album either way.
The personal-data gate now reads pending/*.json too, so a label that is
still waiting for approval is forbidden in the tree already. The e2e run
boots the management page beside the bot with the same environment,
proves the queue is stripped and never reaches media/, approves it through
the real routes, and has !losowe answer with the photo afterwards without
a restart; exiftool is found through media_strip.locate() there rather
than PATH, which lacks vendor_perl on Arch.
!gadaj is dropped from !odcisz: it was within a typo of !dodaj, and the
nearest-command suggestion must never be a coin toss.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two commands, food_today and workout_today, each printing the current
Warsaw day out of a log another program owns: every meal with its time,
calories and source plus the day's total and macros; every workout with
every field it carries. Three workout fields are withheld (a reservation
phone number, a screenshot path, the sync record id), and the food budget
is never printed -- diet_guard keeps that number on the machine on purpose.
The spellings the group uses carry a surname, so they are custom aliases in
the state directory; aliases.json ships only plain words. The suite points
both logs at absent temp files so it can never read the operator's own,
and the e2e run writes one invented entry of each under today's date.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A custom alias is data the command table is built from, so it cannot be a
live switch like the ones already on the tab. Instead the management page
writes aliases_custom.json in the bot's state directory and then asks the
bot to reload, in one step: whenever the answer is anything but `reloaded`
-- a word another command has, a name no command has, a bot still
connecting -- the file is restored byte for byte and the bot's own reason is
shown next to the field. What is on disk is therefore always what the bot
serves, and a typo cannot sit in the file poisoning every later reload.
The registry appends the operator's words to the shipped list at
registration, which keeps `!pomoc` on the shipped spelling and routes the
new words through the same clash check as aliases.json. An alias for a name
no command has fails the whole build (ConfigError -> `kept`), because a
silently ignored entry is how a file drifts. The tagger treats custom
aliases as reserved words too, so a person cannot be given one.
registry.py crossed the 250-line cap and now re-exports the record and the
error types from command.py; the bot's state directory lookup, previously
duplicated in pack.py and the suggestions export, is state_dir.py.
Proven end to end: !e2ekot is unknown, the file is written and reload.sh
runs, !e2ekot posts a cat, a file naming a command that does not exist is
refused with 409 and the previous table still answers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
One row per live command on a new Commands tab of the management page,
each with an on/off switch, an optional "off until" moment and a cooldown
that overrides the bot's default for that command alone. Every switch is
live: the bot keeps them in its own SQLite (command_settings, a permanent
table) and reads the row on every attempt, so nothing goes through
/_reload and nothing restarts.
A command that is off answers "BEEP BOOP !kot WYŁĄCZONE." (with "DO HH:MM"
for a timed switch) rather than going quiet, so nobody reports it broken;
the refusal is filed as refused_off, which the analytics count with the
other refusals, and it neither starts the cooldown nor counts as a use.
The bot grows GET /_commands and PUT /_commands/{name}; the management app
proxies them under /api/commands and passes the bot's 404/503 through
unwrapped. !cisza and !odcisz carry no cooldown and the page says so
instead of accepting a number the bot would ignore (test_wiring_ungated
keeps that set honest against the real table).
Proven end to end: scripts/e2e.sh now brackets a PUT with switch_off.yaml
and switch_on.yaml, the same actor refused and then served, with no reload
between them. Also bumps vitest and @vitest/coverage-v8 to 5.0.0 and
@types/react-dom to 19.2.7, which the freshness gate asked for.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The registry used to be built once, inline in build_bot, so every change to
the data behind it -- a person tagged in the web UI, an alias, soon a command
switched off from the management page -- meant restarting the unit. That is
cheap and lossless, but it is a deploy step somebody has to remember, and a
21:37 report can land in the middle of it.
Now the registration sequence lives in wiring.build_registry, and
RegistryHolder runs it again on POST /_reload: the new table is built whole
and swapped in with one assignment, never patched in place, so a duplicate
trigger is found before anything the dispatcher reads has changed. Bad data
(ConfigError, DuplicateTriggerError) keeps the previous table and answers
409 with the reason; a programming error still surfaces as a 500.
scripts/reload.sh wraps it, with --code for the restart a Python change
needs, and an optional command to type in testbot so "reloaded" means
"answered". No Python hot-reload on purpose.
The loader change is what makes deploying this safe: two rows in the live
tags.json name files that are no longer in media/, and load_pack refused
them, so the next restart would have looped on ConfigError. A stale tag is
drift in a file a page edits mid-session, not a broken album -- it is now
skipped with a warning naming it. A tag that reaches outside the album is
still refused. pack_load.py split at the 250-line cap; pack_tags.py is the
half whose checks are not all fatal.
Proven offline in scripts/e2e.sh: !delta is unknown, a fifth person is
tagged into the synthetic pack, scripts/reload.sh runs, and the same
process posts delta's photo.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
!dodaj is unwired: register_add_command is no longer called, its aliases
are gone from aliases.json, and the word is an unknown one like any
other -- answered with "Nie znam !dodaj", storing nothing. The ingest
path itself (pack_add, pack_ingest, pack_store, the stripper) stays,
tested at 100%, because the approval queue that replaces the direct
write will reuse every piece of it: !dodaj lands in pending/, and the
tagger view gets approve / reject.
The eight e2e scenarios that proved the direct write, the media
generation step and prove_the_add_photo_invariants leave with it;
64852ae has them for the queue to bring back in their new shape. One
scenario takes their place and pins the disabled state.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Splitting e2e_proofs.sh out at the 250-line cap took the source line with
it. e2e.sh still called prove_the_add_photo_invariants, bash still reported
"command not found", and the script still exited 127 -- after every scenario
had already printed ok, which is why it read as a passing run with a stray
message at the end. CI has been red since 08599e6 for exactly this.
Six assertions were silently not running: that every stored file was
stripped, that a typed label never became a key in custom.json, that a
!dodaj carrying no attachment minted nothing, that refused uploads were not
stored, that signal-cli own copies were deleted, and that the adder was
recorded. All six pass now that they execute.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T4QTp3h5Q3MvfrwGV8xqPf
!sztuka posts a work of art, skewed 60% Polish; !wiersz posts one short
public-domain Polish poem, whole. Both follow the rule the rest of the bot
already keeps: the repository commits identifiers, and the words arrive over
the wire when somebody asks.
!wiersz was written the other way first, and the gate caught it. A pack of
short public-domain poems was harvested, name-filtered and committed, and
check_personal_data.sh refused it: three lines carried a real group member's
given name, and two still did after a filter that had discarded 31% of the
corpus. That is structural, not a tuning problem -- Polish verse capitalises
every line, so a line-initial capital carries no signal, and that is exactly
where a name sits. The pack now holds Wolne Lektury slugs; the poem, its
title and its author are fetched at post time and appear nowhere in git.
It is also the better version: with no name filter there are no false
positives, so the pool is larger and the attribution is a full name again.
The gate then refused the art category list too, over a painter whose given
name is somebody's. That is the popes.json case exactly, so it takes the
same answer: a file-scoped allowlist entry, still refused everywhere else.
The Commons harvest is now three modules by what they are, not by where the
line count fell: commons_api is the conversation with Commons, commons_harvest
turns categories into a pool of licensed page ids, commons_cache makes an
interrupted run resumable. Along the way it gained licences in the same
request that lists a category (the second pass was 1,500 requests at 2.3s
each), maxlag instead of a guessed sleep, 5xx retries because one 504 used to
destroy a whole run, and Wikidata for works by creators of Polish
citizenship: the Polish pool grew 15,221 -> 20,092.
Batching categories through CirrusSearch looked like a further 6x and was
measured out again: 2,404 files where the authoritative enumeration returns
2,429, a different ~1% missing on every run.
The offline run is 19s where it was four minutes, almost all of it the
deafness gate waiting out three thirty-second liveness probes to prove
something provable in three; the tick is configuration now, and e2e enforces
a 40s budget on itself so that never silently comes back.
Send a photo captioned "!add_photo alfa wakacje" in one of the album's own
groups and it lands in the pack tagged with both, retrievable through !alfa
and the new !losowe immediately -- no restart, and no tagger session on the PC.
A label matching nobody becomes a custom tag, and !losowe is the only door to
one, which is what keeps a live swap a *data* swap: it can never register a
trigger or widen a command's groups.
No typed label travels. Labels become pseudonym keys in a new registry
arg_resolver, at the dispatcher, before the line that logs the command name --
so no handler, journal entry, command_uses row or reply body ever holds a word
somebody typed. The reply is counts and pseudonyms: "dodano · p1: 4 · t1: 1".
Stripping is exiftool with a *positive* check on the output, never an exit
code. ffmpeg was tried first and fails silently on photos: given a JPEG with a
real APP1 Exif segment it returned a byte-identical file, GPS intact, exit 0.
The surviving-tag allowlist is per tag *name* rather than per group, because
measurement showed a group-level "must be empty" rule rejects every format
except JPEG -- a valid PNG always reports its IHDR, an MP4 some fifty QuickTime
tags that are the container itself. Codec configuration (CompressorName,
VendorID, the HEVC parameter set) is on the list by decision: CompressorName
lives in the avc1 sample description and exiftool cannot write it at all, so
the alternative was refusing every video and every iPhone HEIC. Location and
identity tags are still refused, and the e2e run re-reads the stored files to
prove a QuickTime geotag did not survive.
.webm and .svg stay on the format allowlist and are stored *unstripped*:
exiftool 13.55 can write neither, and its SVG reader does not even surface
<metadata>. That is a deliberate, documented hole, not an oversight.
Two things this had to defuse along the way:
- personal_data_sources now reads custom.json, so "!add_photo kot <photo>"
would have outlawed the word "kot" and refused every future commit, fixable
only by hand-editing an album the committer cannot see. Minting now refuses
a word that is already a command, and the gate subtracts the repo's own
alias vocabulary from custom labels only -- curated people.json names are
never excused.
- signal_bot/external.py is now the single subprocess site in the repository,
so ruff's S603 needs exactly one scoped per-file-ignore instead of one per
module that shells out. A test asserts nothing else imports subprocess.
A failed add changes nothing: media and the three manifests are written, the
album is reloaded, and only a load that succeeds is published; a load that
fails rolls all four back and keeps serving the pack that was already working,
without raising -- Restart=always would turn a ConfigError into a loop.
977 tests, 100% branch coverage, no omits and no suppressions. scripts/e2e.sh
gains 8 scenarios driven against the real exiftool on media generated at run
time, and asserts the invariants a reply cannot: that a refused upload is not
stored, that a message with no attachment leaves custom.json byte-identical,
and that signal-cli's own copy of every ingested attachment is gone.
Closes TODO-add-photo.md.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqqE22jp94B1ypwxppdr1o
Per-person photo commands, where the mechanism is public and the album is
not. Nothing identifying is in this repository: who the people are, what
fires their photos and which file shows whom all live in one directory
outside it, and the tests, the docs and the offline e2e run use three
invented people instead.
Several names in one message intersect -- "!alfa beta" is a photo with both
-- and where an entry offers a choice the bot picks among the readings that
actually have a photo, so a question the album could answer is not refused.
A combination nobody photographed gets one line rather than silence, which
from the group's side is indistinguishable from a broken bot.
Scoping is enforced in the dispatcher, not in the handler: a command the
group is not allowed resolves to nothing, so it is not answered, not counted
as a command, and never named in a log line outside its own groups. A
handler that quietly returned would leak all three. Commands are named by
pseudonym (pack:p1), so the journal and command_uses say somebody asked for
a photo and nothing about whose.
Video works: the attachment goes out as data:video/mp4;filename=...;base64,
which is the container's own spelling, read off its /swagger/doc.json.
Told it is a photo, Signal renders a clip as an unplayable file.
tool/tagger.py serves one page on loopback to fill the album in -- names are
typed into the page, never as arguments, and it prints how many photos each
command would have when it stops, because a command with zero is dead on
arrival.
Also renames the real people who were still sitting in the stats test
fixtures. They remain in the history; the point is that nothing new is
added to it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017w1HZ3cZ7neL3rFdQ6VMBA
Adds !papież / !papiez / !pope: a random pope as a photo, captioned with his
Latin name, reign and pl.wikipedia link. popes.json already carried a Commons
portrait filename for all 267 and nothing had ever consumed it.
The Latin name comes from the Latin column of Poczet papieży, not from the
la.wikipedia sitelink. The sitelink resolved for all 267 and was still wrong:
la.wikipedia numbers several popes differently, so #154 came out as
"Stefan IX — Stephanus X" in a single caption, which reads as an off-by-one.
The table's own column says "Stephanus Nonus" and cannot disagree with the
name beside it. LATIN_OVERRIDES ships empty; it is a tripwire, not a backlog.
Portraits are cached on disk under CacheDirectory=signal-bot, named from the
ordinal so no upstream string ever reaches a path, and bounded at 267 entries
so nothing needs evicting. Verified that CacheDirectory wins against
ProtectHome=read-only before relying on it.
Also adds POST /_selftest and scripts/selftest.sh. Signal never delivers an
account its own sends, so no command has ever been exercised end-to-end
without asking a human to type it. This injects a real frame into the live
dispatcher; it is loopback-only and refuses any group not in SELFTEST_GROUPS,
which is empty by default.
Verified against the real endpoints, not just the fake: 267/267 portraits
resolve on Commons, regeneration is byte-identical on every previously
shipped field, and !papież was driven 7 times through the live dispatcher in
testbot, returning distinct popes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015d48t4qsQLBJnDwDeo2Nom
Today the bot was up, healthy and answering nothing for three hours. A host
firewall change (fixed in testsAndMisc 334c0cd9) cut every Docker container
off the internet; signal-cli lost Signal and reconnected in a loop from
15:32:50 onward. Nothing noticed:
- the container's own healthcheck answered 204
- /v1/groups answered 200 out of local state, so discovery saw no change
- the bot's WebSocket to the container stayed OPEN, so the transport's
reconnect logic never fired
- /healthz answered {"status": "ok"} the entire time
The only symptom was silence, which is also what a quiet evening looks like.
signal_bot/liveness.py asks the one question that cannot be answered from a
cache -- GET /v1/devices, which needs a live round trip -- but only after
SILENCE_SECONDS (120) with no frame of any kind, so a busy group costs
nothing and an idle one costs one request per window. Three consecutive
failures and /healthz answers 503 with the reason. Verified end to end in
scripts/e2e.sh against the real process: 503 after 84s with the fake's
connection cut, back to 200 on its own 30s after restoring it.
scripts/signal_api_watchdog.sh + a two-minute user timer do the recovery the
bot cannot do for itself, since a wedged bot cannot restart the container it
depends on. Two tiers, so the steady state costs no Signal round trip at
all: the bot's own /healthz first, the authenticated probe only when that is
not 200. It stops after 3 restarts in 6 hours and escalates instead -- a
deregistered account or a blocked network is not fixable by restarting, and
a restart loop would hide it. Its own bash suite stubs curl and docker and
drives every branch including the give-up cap.
signal_bot/trust.py answers the other half: TRUSTED_INVITERS decides who may
put this bot in a group. The gate sits in GroupDirectory._permitted, which
both _join_pending and _adopt already route through -- gating only invites
would gate the door nobody uses, since prime_profile.sh exists precisely so
that an add becomes a direct join. Evidence is either the group-update
envelope, which is the only frame naming an inviter, or a trusted identity
being a full member in the listing. A refusal is logged once, with that
group's members, so the operator can see which string to add.
verify_allowlist moved to membership.py to keep discovery.py under the
250-line gate; it was already the module that owns MembershipError.
Verified live, by stopping the container by hand and touching nothing else:
18:54:35 docker stop signal-api
18:56:36 liveness: failure 1 of 3 after 123s of silence
18:57:36 failure 3 of 3 -> /healthz answers 503 from 18:57:45
18:58:18 watchdog: probe failed (1/2)
19:00:28 watchdog: probe failed (2/2) -> docker restart signal-api
19:00:36 liveness: receive path recovered after 8 failed probe(s)
19:01:13 watchdog: signal-api is back
Six minutes, unattended, against three hours of silence this afternoon. All
four real groups are still adopted with TRUSTED_INVITERS=+48123456789 set,
and no group was refused.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KbjfWdAyHbCubac7wJKJZW
First live reply, as its own registered account. `!cat` and `!dog` from the
phone return images -- verified in the production shape (the bot *invited to
an already-existing group*) and under a systemd user unit.
Four failures, each of which left the bot looking perfectly healthy:
- Invited is not joined. Signal only adds an account outright if the adder
holds its profile key; with no profile the add becomes a pending invite,
and signal-cli cannot accept one addressed to a phone number ("Cannot find
service ID for self"). Group listed, websocket up, /healthz OK, zero
messages. register_bot.sh now sets the profile *before* the add, and
startup refuses on `member: false`.
- MODE=json-rpc-native is a GraalVM build with no AWT: text sends work,
every attachment 400s with "Can't load library: awt". Both mode guards now
demand exactly json-rpc, and start_container replaces a wrong-mode
container instead of reusing it.
- The API spells one group two ways -- /v1/groups returns `group.<base64>`,
envelopes carry what that decodes to -- so the scoping gate rejected the
bot's own group. e2e.sh now models both spellings; reverting the fix fails
4 scenarios, which is why the offline suite missed it before.
- BIND_PORT in .env was dead: __main__ read os.getenv before anything loaded
a .env. The bind address comes from Config now.
A taken port used to be uvicorn logging the collision and exiting 0, which
every supervisor reads as success. portcheck.py refuses to start, exits
non-zero, and names the holder by reading /proc directly -- no subprocess, no
PATH assumptions, testable against a fake proc tree. Default port is 8137:
not 8000, and below the ephemeral range so the kernel cannot take it.
The bot now runs as signal-bot.service (Restart=always, linger checked rather
than assumed, StartLimitBurst so a config error fails visibly instead of
looping). Verified by SIGKILL and by re-running a command under the unit.
100% branch coverage, 230 tests, all gates green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013yaBT45KhHqxTKVCTA87iy
CI failed on shellcheck where the local hook passed: the runner's shellcheck
reports SC2015 (`A && B || C` is not if-then-else) and mine did not. Both now
run with --severity=style so a version skew cannot pass locally and fail in
CI. The cleanup traps are plain ifs, which is what the finding was about --
`kill` on an already-dead pid makes B fail and C run while A is true.
live_smoke.sh choose_group had two bugs that only surface when it is actually
run, neither reachable from the offline suite:
- it fetched /v1/groups twice, once to display and once to resolve the picked
index, so an unstable order would hand back a different group than the one
shown. Fetched once into a variable now.
- `read -r -p` wrote its prompt to stdout, which the caller captures with
$(...), so the prompt text would have been swallowed into the group id.
Prompt and listing both go to stderr; stdout carries only the id.
Also validates that the pick is a number and in range.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013yaBT45KhHqxTKVCTA87iy
The bot did not work. `run.sh` started a FastAPI app whose
`@app.on_event("startup")` awaited `asyncio.gather()` on two infinite
tasks, so uvicorn never finished starting up and no HTTP was ever served;
the readme's `python main.py` was a no-op with no `__main__` block and no
`uvicorn.run()` call. All three CI workflows were red, one referencing an
`environment.yml` that does not exist.
Rewritten as a package behind an injectable transport seam, which is what
makes the offline environment possible.
Bugs fixed:
- startup deadlock: lifespan spawns the loops instead of awaiting them
- sticker filter read dataMessage.sticker even on syncMessage envelopes, so
the account's own stickers were never filtered (e038772, 53028cc, 07fa90d,
2416cef all fought this symptom)
- no websocket reconnect: a container restart killed the bot silently
- blocking `requests` inside async functions stalled the event loop
- every counted message DMed the whole tally to the bot itself
- images were written to CWD under a filename taken from the remote URL
- one global 10s cooldown shared across all users and commands
- dead code: RECEIVE_URL, get_attachments (no call site), USER_MESSAGE_COUNT
- .env.sample declared PRIMARY/SECONDARY_PHONE_NUMBER and MODE, none read
Scoping is now an enforced invariant. A linked device receives every DM and
every group on the account; GROUP_ID was the only thing narrowing that and
defaulted to "". Config refuses to start unscoped, direct messages are
dropped explicitly rather than by a falsy check, and a scenario asserts that
DMs and other groups produce zero outbound messages.
Offline environment: fake_signal/ serves the real protocol surface, so the
unmodified bot connects to it by pointing API_BASE_URL at it. scripts/e2e.sh
boots both, replays 7 scenarios and exits non-zero on mismatch. A local
Signal server is not achievable (signal-cli compiles its endpoints and trust
store in; Signal-Server needs FoundationDB), so faking this boundary is the
only way to test every feature without an account.
Transport pinned to bbernhard/signal-cli-rest-api:0.100 with
MODE=json-rpc-native. The readme pinned 0.167-dev; the dev tag series counts
separately, so that was older than 0.100, not newer. /v1/receive is only a
WebSocket in a json-rpc mode, so the mode is verified at startup.
Repo standards: ruff select=ALL, mypy --strict, 189 tests at 100% branch
coverage, 250-line cap, exact pins at newest stable, shared gates delegated
to ~/utils, one CI workflow replacing three red ones. No noqa, no
type: ignore, no coverage omits anywhere.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013yaBT45KhHqxTKVCTA87iy
