signal-bot/signal_bot/pack_store.py
Krzysztof kuhy Rudnicki aefffb2fd9
Some checks are pending
CI / check (push) Waiting to run
feat: add a photo to the album from the chat, tagged, stripped and verified
Send a photo captioned "!add_photo alfa wakacje" in one of the album's own
groups and it lands in the pack tagged with both, retrievable through !alfa
and the new !losowe immediately -- no restart, and no tagger session on the PC.
A label matching nobody becomes a custom tag, and !losowe is the only door to
one, which is what keeps a live swap a *data* swap: it can never register a
trigger or widen a command's groups.

No typed label travels. Labels become pseudonym keys in a new registry
arg_resolver, at the dispatcher, before the line that logs the command name --
so no handler, journal entry, command_uses row or reply body ever holds a word
somebody typed. The reply is counts and pseudonyms: "dodano · p1: 4 · t1: 1".

Stripping is exiftool with a *positive* check on the output, never an exit
code. ffmpeg was tried first and fails silently on photos: given a JPEG with a
real APP1 Exif segment it returned a byte-identical file, GPS intact, exit 0.
The surviving-tag allowlist is per tag *name* rather than per group, because
measurement showed a group-level "must be empty" rule rejects every format
except JPEG -- a valid PNG always reports its IHDR, an MP4 some fifty QuickTime
tags that are the container itself. Codec configuration (CompressorName,
VendorID, the HEVC parameter set) is on the list by decision: CompressorName
lives in the avc1 sample description and exiftool cannot write it at all, so
the alternative was refusing every video and every iPhone HEIC. Location and
identity tags are still refused, and the e2e run re-reads the stored files to
prove a QuickTime geotag did not survive.

.webm and .svg stay on the format allowlist and are stored *unstripped*:
exiftool 13.55 can write neither, and its SVG reader does not even surface
<metadata>. That is a deliberate, documented hole, not an oversight.

Two things this had to defuse along the way:
- personal_data_sources now reads custom.json, so "!add_photo kot <photo>"
  would have outlawed the word "kot" and refused every future commit, fixable
  only by hand-editing an album the committer cannot see. Minting now refuses
  a word that is already a command, and the gate subtracts the repo's own
  alias vocabulary from custom labels only -- curated people.json names are
  never excused.
- signal_bot/external.py is now the single subprocess site in the repository,
  so ruff's S603 needs exactly one scoped per-file-ignore instead of one per
  module that shells out. A test asserts nothing else imports subprocess.

A failed add changes nothing: media and the three manifests are written, the
album is reloaded, and only a load that succeeds is published; a load that
fails rolls all four back and keeps serving the pack that was already working,
without raising -- Restart=always would turn a ConfigError into a loop.

977 tests, 100% branch coverage, no omits and no suppressions. scripts/e2e.sh
gains 8 scenarios driven against the real exiftool on media generated at run
time, and asserts the invariants a reply cannot: that a refused upload is not
stored, that a message with no attachment leaves custom.json byte-identical,
and that signal-cli's own copy of every ingested attachment is gone.

Closes TODO-add-photo.md.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqqE22jp94B1ypwxppdr1o
2026-08-31 11:20:00 +02:00

147 lines
5.2 KiB
Python

# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Putting a file into the album, and taking it back out when the swap fails.
Every write here is temp-then-``Path.replace``, the pattern
``tool/tagger.py::Album.write`` already uses, because a manifest that is half
written is a manifest that fails the next load -- and with ``Restart=always``
a startup-time ConfigError is a loop, not an error message.
The stored name is a content hash. The original filename is discarded: it is
attacker-chosen, it is frequently somebody's name and a date, and hashing
makes "is this photo already here?" a lookup instead of a comparison.
``adds.json`` records who added what. Nothing reads it -- ``pack_load``
ignores the file entirely -- and it exists because ``!undo_photo`` cannot be
built later without it, and the id has to have been recorded at add time.
It stays inside the album directory, which is 0700 and outside the
repository, so it is subject to exactly the same handling as the photos.
"""
from __future__ import annotations
import hashlib
import json
from dataclasses import dataclass
from typing import TYPE_CHECKING, Any, Final
from signal_bot.pack import MEDIA_DIR, TAGS_FILE
from signal_bot.pack_custom import CUSTOM_FILE, FILE_MODE
if TYPE_CHECKING:
from collections.abc import Mapping
from pathlib import Path
ADDS_FILE: Final = "adds.json"
# Half a SHA-256 is 128 bits of name. Collisions are not a threat model here;
# a shorter name is simply easier to read in a directory listing.
NAME_LENGTH: Final = 32
def content_name(payload: bytes, suffix: str) -> str:
"""The filename one upload gets, derived from the bytes themselves."""
digest = hashlib.sha256(payload).hexdigest()[:NAME_LENGTH]
return f"{digest}{suffix}"
@dataclass(frozen=True, slots=True)
class Snapshot:
"""The album's manifests as they were before a write, for rolling back.
Bytes rather than parsed documents: restoring has to put back exactly
what was there, and a reparse-and-redump could quietly normalise a file
the tagger wrote by hand.
"""
tags: bytes | None
custom: bytes | None
adds: bytes | None
def snapshot(directory: Path) -> Snapshot:
"""Read the three manifests as they stand, before anything is written."""
return Snapshot(
tags=_read(directory / TAGS_FILE),
custom=_read(directory / CUSTOM_FILE),
adds=_read(directory / ADDS_FILE),
)
def restore(directory: Path, previous: Snapshot, written: tuple[str, ...]) -> None:
"""Undo a failed add: put the manifests back and delete the new files.
Called only when the album no longer loads after a write, which means the
bot is still serving the *old* pack from memory. Leaving the new files
behind would make the next start fail on a tag that no longer resolves.
"""
_restore(directory / TAGS_FILE, previous.tags)
_restore(directory / CUSTOM_FILE, previous.custom)
_restore(directory / ADDS_FILE, previous.adds)
for name in written:
(directory / MEDIA_DIR / name).unlink(missing_ok=True)
def write_media(directory: Path, name: str, payload: bytes) -> None:
"""Put one stripped file into the album, atomically."""
album = directory / MEDIA_DIR
album.mkdir(parents=True, exist_ok=True)
path = album / name
temporary = path.with_name(f"{name}.tmp")
temporary.write_bytes(payload)
temporary.chmod(FILE_MODE)
temporary.replace(path)
def write_tags(directory: Path, tags: Mapping[str, frozenset[str]]) -> None:
"""Replace tags.json with the merged table."""
document = {name: sorted(keys) for name, keys in sorted(tags.items())}
write_json(directory / TAGS_FILE, document)
def record_adds(directory: Path, entries: Mapping[str, dict[str, Any]]) -> None:
"""Append to adds.json, which nothing at run time ever reads back."""
document = _document(directory / ADDS_FILE)
document.update(entries)
write_json(directory / ADDS_FILE, document)
def write_json(path: Path, document: Mapping[str, object]) -> None:
"""Write one manifest atomically, 0600, so a crash cannot halve it."""
temporary = path.with_name(f"{path.name}.tmp")
temporary.write_text(json.dumps(document, indent=2), encoding="utf-8")
temporary.chmod(FILE_MODE)
temporary.replace(path)
def _document(path: Path) -> dict[str, Any]:
"""One JSON object, or an empty one for a file that is not there yet.
Deliberately forgiving, unlike pack_load: adds.json is write-only
bookkeeping, and refusing to store a photo because an unread audit file
got corrupted would be the tail wagging the dog.
"""
raw = _read(path)
if raw is None:
return {}
try:
document = json.loads(raw)
except json.JSONDecodeError:
return {}
return document if isinstance(document, dict) else {}
def _read(path: Path) -> bytes | None:
"""The file's bytes, or None when it does not exist."""
try:
return path.read_bytes()
except OSError:
return None
def _restore(path: Path, previous: bytes | None) -> None:
"""Put one manifest back, or remove it if there was none."""
if previous is None:
path.unlink(missing_ok=True)
return
path.write_bytes(previous)