signal-bot/DOCS-privacy.md
Krzysztof kuhy Rudnicki 6ec746fad2 feat: a word the bot does not know gets an answer, and gets kept
An unrecognised "!word" was dropped in silence, which made a typo
indistinguishable from the bot being down and threw away the best available
list of commands worth writing. Three things now happen at the one place in
dispatcher.py that used to be a bare `return`.

A near miss is corrected -- "!cta" asks whether you meant "!kot". The
distance is optimal string alignment rather than difflib's ratio: a
transposition is the typo people actually make, it costs one edit here, and
it shreds the matching blocks a ratio counts ("!cta"/"!cat" scores 0.75 but
"!cta"/"!kot" scores 0.50, so no cutoff separates them on principle).

A word with no near miss points at !zaproponuj, which files an idea from
inside the group. Either way the word and the message are recorded, and the
management page grows an Ideas tab that lists them.

Three things stay silent, each for its own reason. "!kot ladny" is a real
trigger with words after it and always has been silent. "!!" is punctuation.
And a command claimed in some *other* group is neither answered nor recorded
-- the album's triggers are real people's names, and pack_scoping.yaml says
it out loud: a refusal would itself announce that the album exists. The
existence check is therefore global, which is what pack_add already asks.

Measured rather than guessed, against the shipped aliases.json:

- a minimum candidate length, because "!ja" is one edit from ~40 of the 676
  two-letter words somebody could type; 69 of them got a suggestion before
  the rule and 18 after.
- the two-edit budget starts at six characters, not five. At five it put
  four pairs of *different* commands within reach of each other --
  !pies/!popes, !days/!cats, !dzis/!dni, !mnie/!dni -- and costs nothing to
  give up, because every real typo this was written for is one edit away.
  test_no_two_commands_are_within_reach_of_each_other fails if a future
  alias reopens the gap.

Both new tables are keyed on the envelope's own sent_at_ms, never the wall
clock: signal-cli replays undelivered frames on reconnect, and a clock-keyed
table collects a duplicate row per reconnect instead of ignoring one.

The branch is gated on should_count -- which excludes the bot's own envelope,
and that is the loop guard, since handle() checks is_own nowhere else -- and
wrapped in its own try/except, because it runs above the one that wraps a
handler and Bot.listen has no exception handling of its own.

Suggestions live in SQLite, not in a tracked file, and the export goes to the
state directory. check_personal_data learns the names it refuses from the
live database, so it catches a name somebody has typed and cannot catch a
misspelling of one -- and an unrecognised-command corpus is made of
misspellings. A group-writable file in a public tree is both a leak and a
wedge. The export therefore carries
deliberate !zaproponuj notes in full and unrecognised words as counts only;
bodies stay on the loopback-only page.

Verified on the live bot: restarted, both tables created with every existing
row count unchanged, !cta and !zaproponuj !pogoda delivered through
/_selftest, both rows read back out of the live database, and the journal
names neither the word nor the body.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01416TegJ3sa8xL5qGGgrygP
2026-08-31 16:48:26 +02:00

85 lines
4.3 KiB
Markdown

# What is private, and what stops it leaking
This repository is public. The bot it describes reads a private group chat,
posts from a private photo album, and keeps a database of who said how much.
None of that may be in here, and on 2026-08-30 some of it was.
## What leaked, and how
Real first names, one full name, two phone numbers and a group member's
Signal uuid were in the tree *and* in the public history. They arrived one
way every time: **somebody reached for a real name because it was the example
in front of them** — a test fixture built from a live database dump, an
envelope pasted verbatim out of a running container, a help text written
while looking at the actual group.
Fixing it needed the tree scrubbed, all 56 commits rewritten with
`git filter-repo`, and then the repository **deleted and recreated**, because
GitHub keeps orphaned commits fetchable by their SHA after a force-push and
only the repository going away takes them with it. The closed pull requests
and the star went too.
## What is private, and where it lives
| | Where | Never in git because |
|---|---|---|
| the photos and who is in them | `~/.local/share/signal-bot/pack/` | it is outside the repo, and `.gitignore` also names `people.json`, `tags.json`, `media/`, `pack/` and every media extension |
| the message database | `signal_bot.sqlite3` | `*.sqlite3` and its `-wal`/`-shm` siblings are ignored |
| the account's number, groups, inviters | `.env` | `.env` is ignored; `.env.sample` carries the shape and no values |
| everyone's names | the database and the pack manifest | nothing reads them into the repo; the bot logs `pack:p1`, never a name |
| what people typed after a `!` the bot did not know | `unknown_commands` in the database | the database is ignored; the log line names neither the word nor the message, and the export the management page writes goes to the state directory, never into this tree |
All of it is `700`/`600` on disk. The repository never needs any of it: the
tests, the documents and the offline end-to-end run use invented people —
alfa, beta, gamma, ala, basia, celina — and invented ids that base64-decode
twice into a sentence saying what they are.
## The gate
`scripts/check_personal_data.sh` refuses a commit that carries personal data.
It runs on staged files, on the commit message, and over the whole tree in
CI. Two halves, because they fail differently:
- **Names cannot be listed here** — a file of real names kept in the repo to
check against *is* the leak. So the checker learns them at run time from
outside: the photo pack's manifest, the message database, and `.env`. It
never prints the word it matched, because this output reaches CI logs.
- **Shapes can be listed** — a phone number, a uuid and a Signal group id
look like themselves anywhere, so those are matched by pattern and compared
against `.personal-data-allowlist`, which holds only invented values.
On a machine with none of that private data (CI, a fresh clone) the name half
has nothing to compare against and **says so** rather than passing quietly.
It found a real leak the first time it was run: the live test group's id, in
two spellings, in `tests/test_config.py` — which four rounds of grepping by
hand had missed, because nobody thinks to grep for base64.
## When it fires
```
tests/test_thing.py:12: a name from your private data
```
Use an invented name. `ala`, `basia`, `celina`, `darek` are already in use;
so are `+15550000000` and ids that decode to `e2e-watched-group-id-32-bytes!!!`.
```
DOCS-thing.md:40: phone number '+48123456789' is not in .personal-data-allowlist
```
Either it belongs to somebody — replace it — or it is invented, in which case
add it to `.personal-data-allowlist` **after checking that it is**. A line
there may be scoped to one file (`signal_bot/popes.json::hadrian`), which is how
a given name that is also a pope stays refused in every test fixture that has
no business carrying it.
## What the gate cannot do
- It does not scan history. Rewriting that is a deliberate, separate act.
- It knows only what is on *this* machine. A name nobody has ever typed into
the group is a name it cannot recognise.
- It does not stop the bot's own logs holding display names —
`journalctl --user -u signal-bot` does, and `--vacuum-time=` is how that is
emptied.