signal-bot/DOCS-photo-pack.md

# The photo pack

A pack is a private album the bot posts from: `!alfa` returns a random photo
of alfa, and `!alfa beta` returns one with both of them on it.

**Nothing about a real pack is in this repository, and nothing ever will be.**
Who the people are, what somebody types to get their photo and which file
shows whom live in one directory outside the repo. This document, the tests
and the offline end-to-end run all use the same invented people — alfa, beta,
gamma — so the mechanism can be public while the album stays private.

## Where it lives

```
~/.local/share/signal-bot/pack/
  people.json     # the commands, and the only place names exist
  tags.json       # which file shows whom
  custom.json     # labels approved from !add_photo (optional)
  adds.json       # who added what, read by nothing (optional)
  media/          # the photos and videos themselves
  pending/        # !add_photo sends waiting for approval (optional)
```

Found in this order: `PHOTO_PACK_DIR`, then `$STATE_DIRECTORY/pack` (what
systemd hands a **system** unit), then `$XDG_DATA_HOME/signal-bot/pack`.
`StateDirectory=` is deliberately **not** in the unit this repo installs: for
a *user* unit systemd resolves it below `$XDG_STATE_HOME`, which is not where
the album is. Discovery finds the same directory, and leaving `PHOTO_PACK_DIR`
unset keeps a not-yet-filled-in pack from turning `Restart=always` into a loop.

Two absences, deliberately different:

- a pack **nobody named** that is not there, or that has no `people.json`
  yet, is an installation without an album: the commands do not exist, and
  that is the normal case on every machine and in the whole test suite;
- a pack named in `PHOTO_PACK_DIR` that is missing, or whose manifest cannot
  be honoured, **fails startup**. `load_config` raising is an answer, and a
  bot that came up healthy and said "no such photo" forever would not be.

### More than one album

`PHOTO_PACK_DIR` takes a **comma-separated list**, the same spelling
`GROUP_IDS` and `TRUSTED_INVITERS` use:

```sh
PHOTO_PACK_DIR=~/.local/share/signal-bot/pack,~/.local/share/signal-bot/pack-testbot
```

Each album carries its own group scope in its own `people.json`, and the two
share nothing but the process: separate people, separate tags, separate
custom labels. That is what lets a test group have working `!add_photo` and
`!losowe` over invented people while the real album does not exist there at
all — which one album cannot do, because its `groups` list *is* the privacy
guarantee.

Trigger collisions between albums are therefore legal. Two albums are two
sets of people and both may call somebody `!alfa`; no message can ever mean
both, because a command a group is not scoped to is not a command there.
`!dodaj` and `!losowe` are registered once per album for exactly this reason.

**Two albums whose groups overlap are refused at startup** with
`DuplicateTriggerError`. Overlapping scopes would make "which album does
`!dodaj` write to?" a coin toss, and a coin toss about which private album
receives a photo is the outcome this whole design exists to prevent.

## people.json

```json
{
  "groups": ["group.WlRKbExYZGhkR05vWldRdA=="],
  "people": [
    {"key": "p1", "triggers": ["!alfa", "!alfka"]},
    {"key": "p2", "triggers": ["!beta"]}
  ],
  "combos": [
    {"key": "c1", "triggers": ["!oboje"], "all_of": ["p1", "p2"]},
    {"key": "c2", "triggers": ["!ktokolwiek"], "any_of": ["p1", "p2"]}
  ]
}
```

- **`groups` is the whole privacy guarantee.** In any other group these are
  not commands at all: not answered, not counted as a command, not named in a
  log line. A handler that quietly returned would still leak all three, which
  is why the *dispatcher* enforces it — `registry.resolve_with_args` takes the
  group and gives back nothing outside it.
- **`key` is a pseudonym**, and the only identifier that reaches the journal
  or the `command_uses` table. A command is `pack:p1`; reading either says
  somebody asked for a photo and nothing about whose.
- **`all_of` means one photo with all of them; `any_of` means either**, and
  where a choice exists the bot picks among the readings that actually have a
  photo, so a combination that could be answered is not refused.
- Triggers may not contain a digit — `folding.py` maps `0134578` onto letters,
  so `!alfa7` is not what you typed. Two spellings that fold the same are fine
  *within one entry* (`!celina`/`!celiną` is one person written twice) and
  refused across two.

## tags.json

```json
{"alfa.png": ["p1"], "both.png": ["p1", "p2"], "clip.mp4": ["p1"]}
```

A tag naming a file that is not in `media/` fails startup: a renamed file
would otherwise drop out of the album in silence. An empty list means "seen,
nobody in it", which is how the tagger records a photo you skipped.

Videos work. The attachment goes out as
`data:video/mp4;filename=<name>;base64,<data>` — the spelling is the
container's own, from its `/swagger/doc.json` — because told it is a photo,
Signal renders a clip as a file nobody can play.

## custom.json

```json
{"t1": ["wakacje2019"], "t2": ["gorybieszczady"]}
```

Labels `!add_photo` learned from the chat that match no person. The key is a
pseudonym in the same family as `p1` — it is what reaches `tags.json`, a log
line and a reply — and the label is the only place the typed word is kept.
Custom tags define **no commands**: `!random <label>` is the only door to one.
That is deliberate, and it is what makes a live album swap safe — a swap
replaces pack *data* only, and can never register a trigger or widen a
command's groups.

A label that folds onto a person's trigger, or onto another key's label, fails
startup rather than shadowing the command it collides with.

## adds.json

```json
{"a3ae52c7….jpg": {"by": "<uuid>", "at": 1756600000, "approved_at": 1756700000}}
```

Who added which file, and when it was approved. **Nothing reads it** —
`pack_load` ignores it — and it exists only so `!undo_photo` can be built
later without a gap in its history; the adder is recorded at add time or not
at all. It lives inside the album (`0700`, outside this repository), so it is
handled exactly like the photos.

## sources.json

```json
{
  "IMG_0234.jpg":  {"source": "9f2c…", "stored": "b71e…"},
  "a3ae52c7….jpg": {"source": null,    "stored": "a3ae…"}
}
```

The import ledger: every file the album has ever taken in, under **two**
SHA-256s, because stripping changes the bytes.

- `source` — the file as it *arrived*, before metadata was removed. This
  answers "have I imported this photo before?" when the same export folder is
  pointed at the album a second time.
- `stored` — the bytes on disk now. This answers "is the photo somebody just
  uploaded from the chat already here?", because `!add_photo` hashes *after*
  stripping.

Neither alone is enough. The same photo arriving through `--import` and
through Signal has different source bytes — the phone re-encodes — but strips
to the same output. `source` is `null` when the original bytes were never
kept, which is exactly the `!add_photo` files.

Like `adds.json`, **`pack_load` never touches it.** A corrupt ledger degrades
to "I have seen nothing", costing a duplicate photo; a corrupt ledger checked
at startup would be a restart loop under `Restart=always`.

## Adding from the chat: !add_photo

A member of one of the album's groups sends a photo captioned
`!add_photo alfa wakacje`. It is stripped and lands in **`pending/`** — under
its content hash, beside a `<file>.json` sidecar with the proposed `keys`,
`labels`, sender and group — until the operator approves it on `./web.sh`,
which is when it is tagged, moved into `media/` and retrievable. `pack_load`
never reads `pending/`. The path has its own document, because its metadata
handling and privacy invariants are longer than the album format itself:
**`DOCS-add-photo.md`**.

## Filling it in: ./web.sh

```sh
./web.sh --import ~/photos             # dedup, strip, import, then tag
./web.sh --import ~/photos --dry-run   # the report only
./web.sh --untagged                    # only items with no tags yet
./web.sh                               # carry on where you left off
```

`tag.sh` sets up `.venv` on first use, so this works on a fresh clone with
nothing installed. Same for `run.sh`, which used to print the two commands to
type and exit 1.

It serves one page on `127.0.0.1:8140`: the item, a labelled button per
person, `1`–`0` then `q`–`p` to toggle, `→`/`←` to move. Every toggle rewrites
`tags.json` atomically, so closing the tab loses nothing. Names are typed
**into the page**, never as arguments — an argument would land in the shell
history — and the group is picked from the live `/v1/groups` listing rather
than pasted.

**`--import` is not a copy.** Every file is identified by its magic bytes
(not its suffix), stripped through the same `media_strip` path `!add_photo`
uses, and checked against `sources.json` twice: the source hash first, then
the post-strip hash. What the album refuses is **moved, never deleted**, to
`_skipped/<reason>/` beside the source folder, and the printed report has to
account for every file seen:

```
   174  files seen
     1  not media              -> _skipped/not-media/
     0  could not be stripped  -> _skipped/unstrippable/
    82  already in pack        -> _skipped/already-in-pack/
    24  duplicate in batch     -> _skipped/exact-dup/
    67  imported
   ---
   174  accounted for        85 -> 152 in media/
```

Every bucket prints even at zero: a suppressed zero is a silently missing
category, and `0 could not be stripped` is how you see the video pipeline is
still working. Re-running on the same folder is a no-op — that is what the
ledger is for.

**`--untagged` filters the page, never the server's list.** `/media/{index}`
and `/api/tag` both index the full album, so a shorter list on the server
would show item *i* of the album while tagging item *i* of the filtered set —
every photo tagged with the wrong people, and no error anywhere.

**Adding a person works mid-session.** The roster used to be write-once: the
setup form only appeared on an empty album, `Album.setup` overwrote the file
whole, and keys were positional (`p{len+1}`), so inserting somebody renumbered
everyone after them and repointed every `tags.json` entry at the wrong person.
Now keys come from the highest number in use, the form merges, and there is an
"add person" box in the tagger itself — for the photo of somebody new that is
already on the screen.

When you stop it, it prints how many items were never opened and how many
photos each command would have. **A command showing `0` is dead on arrival**;
that is the number to check before restarting the bot, not after somebody in
the group tries it.

## Running the mechanism without an album

`scripts/e2e.sh` writes a synthetic pack with `tool/e2e_pack.py` and proves,
offline and with no Signal account, that one name posts a photo, two names
intersect, an impossible combination gets one line and no attachment, a video
leaves as a video, and the same command in another group produces **zero**
sends. It also drives `!dodaj` against the real exiftool on media generated at
run time (no photo enters this repository), approves the queue through the
management page it boots beside the bot, and has `!losowe` answer, no restart.