signal-bot/tool/e2e_media.py
Krzysztof kuhy Rudnicki aefffb2fd9
Some checks are pending
CI / check (push) Waiting to run
feat: add a photo to the album from the chat, tagged, stripped and verified
Send a photo captioned "!add_photo alfa wakacje" in one of the album's own
groups and it lands in the pack tagged with both, retrievable through !alfa
and the new !losowe immediately -- no restart, and no tagger session on the PC.
A label matching nobody becomes a custom tag, and !losowe is the only door to
one, which is what keeps a live swap a *data* swap: it can never register a
trigger or widen a command's groups.

No typed label travels. Labels become pseudonym keys in a new registry
arg_resolver, at the dispatcher, before the line that logs the command name --
so no handler, journal entry, command_uses row or reply body ever holds a word
somebody typed. The reply is counts and pseudonyms: "dodano · p1: 4 · t1: 1".

Stripping is exiftool with a *positive* check on the output, never an exit
code. ffmpeg was tried first and fails silently on photos: given a JPEG with a
real APP1 Exif segment it returned a byte-identical file, GPS intact, exit 0.
The surviving-tag allowlist is per tag *name* rather than per group, because
measurement showed a group-level "must be empty" rule rejects every format
except JPEG -- a valid PNG always reports its IHDR, an MP4 some fifty QuickTime
tags that are the container itself. Codec configuration (CompressorName,
VendorID, the HEVC parameter set) is on the list by decision: CompressorName
lives in the avc1 sample description and exiftool cannot write it at all, so
the alternative was refusing every video and every iPhone HEIC. Location and
identity tags are still refused, and the e2e run re-reads the stored files to
prove a QuickTime geotag did not survive.

.webm and .svg stay on the format allowlist and are stored *unstripped*:
exiftool 13.55 can write neither, and its SVG reader does not even surface
<metadata>. That is a deliberate, documented hole, not an oversight.

Two things this had to defuse along the way:
- personal_data_sources now reads custom.json, so "!add_photo kot <photo>"
  would have outlawed the word "kot" and refused every future commit, fixable
  only by hand-editing an album the committer cannot see. Minting now refuses
  a word that is already a command, and the gate subtracts the repo's own
  alias vocabulary from custom labels only -- curated people.json names are
  never excused.
- signal_bot/external.py is now the single subprocess site in the repository,
  so ruff's S603 needs exactly one scoped per-file-ignore instead of one per
  module that shells out. A test asserts nothing else imports subprocess.

A failed add changes nothing: media and the three manifests are written, the
album is reloaded, and only a load that succeeds is published; a load that
fails rolls all four back and keeps serving the pack that was already working,
without raising -- Restart=always would turn a ConfigError into a loop.

977 tests, 100% branch coverage, no omits and no suppressions. scripts/e2e.sh
gains 8 scenarios driven against the real exiftool on media generated at run
time, and asserts the invariants a reply cannot: that a refused upload is not
stored, that a message with no attachment leaves custom.json byte-identical,
and that signal-cli's own copy of every ingested attachment is gone.

Closes TODO-add-photo.md.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqqE22jp94B1ypwxppdr1o
2026-08-31 11:20:00 +02:00

113 lines
4.2 KiB
Python
Executable File

#!/usr/bin/env python3
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Seed the fake's attachment store, and check what the album ended up storing.
The samples themselves are built by tool/e2e_fixtures.py; this half moves them
in and reads the result back out.
Two modes:
--upload <base-url> put the media into the fake's attachment store
--verify-pack <dir> assert every stored file came out clean
The second is the half that matters. It re-reads the album with exiftool and
fails on any surviving non-structural tag, which is the same check
``media_tags.is_allowed`` makes -- run here against files that went through
the whole live path rather than through a unit test's fake stripper.
"""
from __future__ import annotations
import argparse
import base64
import sys
from pathlib import Path
from typing import Final
import httpx
from e2e_fixtures import PROBE_CREATOR, PROBE_MAKE, build, run_tool
from signal_bot.media_tags import is_allowed
# The suffixes exiftool 13.55 cannot write, and which are therefore stored
# exactly as they arrived. Excluded from the verification sweep because they
# are *expected* to still carry everything.
UNSTRIPPED: Final = frozenset({".svg", ".webm"})
def upload(base_url: str) -> int:
"""Put every attachment into the fake container's store."""
for attachment_id, payload in build().items():
response = httpx.post(
f"{base_url}/_control/attachment",
json={"id": attachment_id, "base64": base64.b64encode(payload).decode("ascii")},
timeout=10,
)
response.raise_for_status()
sys.stdout.write(f" uploaded {attachment_id} ({len(payload)} bytes)\n")
return 0
def _survivors(path: Path) -> list[str]:
"""Every non-structural tag one stored file still reports."""
report = run_tool(["exiftool", "-G0", "-s", "-all", str(path)])
return [name for name, allowed in _tags(report) if not allowed]
def _tags(report: str) -> list[tuple[str, bool]]:
"""Each reported tag as "Group:Tag", and whether it may remain."""
return [_tag(line) for line in report.splitlines() if line.startswith("[")]
def _tag(line: str) -> tuple[str, bool]:
"""One "[Group] Tag : value" line as its name and whether it may remain."""
group, _, rest = line[1:].partition("]")
tag, _, value = rest.strip().partition(":")
return f"{group}:{tag.strip()}", is_allowed(group, tag.strip(), value.strip())
def verify_pack(directory: Path) -> int:
"""Fail unless every strippable file in the album came out clean."""
album = directory / "media"
failures: list[str] = []
checked = 0
for path in sorted(album.iterdir()):
if path.suffix.lower() in UNSTRIPPED:
sys.stdout.write(f" {path.name}: stored unstripped by decision, not checked\n")
continue
raw = path.read_bytes()
# The raw byte search is not redundant with the tag sweep: exiftool
# reports what it can parse, and a string left in a chunk it does not
# model would be invisible to the sweep and present in the file.
failures.extend(
f"{path.name} still contains {probe!r}"
for probe in (PROBE_MAKE.encode(), PROBE_CREATOR.encode())
if probe in raw
)
leaks = _survivors(path)
if leaks:
failures.append(f"{path.name} still reports {leaks}")
checked += 1
for line in failures:
sys.stderr.write(f"Error: {line}\n")
if failures:
return 1
sys.stdout.write(f" {checked} stored file(s) carry no metadata beyond the container\n")
return 0
def main() -> int:
"""Generate, upload or verify, depending on the mode asked for."""
parser = argparse.ArgumentParser(description=__doc__)
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument("--upload", metavar="BASE_URL", help="seed the fake's attachment store")
group.add_argument("--verify-pack", metavar="DIR", type=Path, help="check a written album")
arguments = parser.parse_args()
if arguments.upload:
return upload(arguments.upload)
return verify_pack(arguments.verify_pack)
if __name__ == "__main__":
raise SystemExit(main())