signal-bot/tool/e2e_fixtures.py
Krzysztof kuhy Rudnicki aefffb2fd9
Some checks are pending
CI / check (push) Waiting to run
feat: add a photo to the album from the chat, tagged, stripped and verified
Send a photo captioned "!add_photo alfa wakacje" in one of the album's own
groups and it lands in the pack tagged with both, retrievable through !alfa
and the new !losowe immediately -- no restart, and no tagger session on the PC.
A label matching nobody becomes a custom tag, and !losowe is the only door to
one, which is what keeps a live swap a *data* swap: it can never register a
trigger or widen a command's groups.

No typed label travels. Labels become pseudonym keys in a new registry
arg_resolver, at the dispatcher, before the line that logs the command name --
so no handler, journal entry, command_uses row or reply body ever holds a word
somebody typed. The reply is counts and pseudonyms: "dodano · p1: 4 · t1: 1".

Stripping is exiftool with a *positive* check on the output, never an exit
code. ffmpeg was tried first and fails silently on photos: given a JPEG with a
real APP1 Exif segment it returned a byte-identical file, GPS intact, exit 0.
The surviving-tag allowlist is per tag *name* rather than per group, because
measurement showed a group-level "must be empty" rule rejects every format
except JPEG -- a valid PNG always reports its IHDR, an MP4 some fifty QuickTime
tags that are the container itself. Codec configuration (CompressorName,
VendorID, the HEVC parameter set) is on the list by decision: CompressorName
lives in the avc1 sample description and exiftool cannot write it at all, so
the alternative was refusing every video and every iPhone HEIC. Location and
identity tags are still refused, and the e2e run re-reads the stored files to
prove a QuickTime geotag did not survive.

.webm and .svg stay on the format allowlist and are stored *unstripped*:
exiftool 13.55 can write neither, and its SVG reader does not even surface
<metadata>. That is a deliberate, documented hole, not an oversight.

Two things this had to defuse along the way:
- personal_data_sources now reads custom.json, so "!add_photo kot <photo>"
  would have outlawed the word "kot" and refused every future commit, fixable
  only by hand-editing an album the committer cannot see. Minting now refuses
  a word that is already a command, and the gate subtracts the repo's own
  alias vocabulary from custom labels only -- curated people.json names are
  never excused.
- signal_bot/external.py is now the single subprocess site in the repository,
  so ruff's S603 needs exactly one scoped per-file-ignore instead of one per
  module that shells out. A test asserts nothing else imports subprocess.

A failed add changes nothing: media and the three manifests are written, the
album is reloaded, and only a load that succeeds is published; a load that
fails rolls all four back and keeps serving the pack that was already working,
without raising -- Restart=always would turn a ConfigError into a loop.

977 tests, 100% branch coverage, no omits and no suppressions. scripts/e2e.sh
gains 8 scenarios driven against the real exiftool on media generated at run
time, and asserts the invariants a reply cannot: that a refused upload is not
stored, that a message with no attachment leaves custom.json byte-identical,
and that signal-cli's own copy of every ingested attachment is gone.

Closes TODO-add-photo.md.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqqE22jp94B1ypwxppdr1o
2026-08-31 11:20:00 +02:00

185 lines
5.4 KiB
Python
Executable File

#!/usr/bin/env python3
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Generating the media scripts/e2e.sh feeds to !add_photo.
Split out of e2e_media.py at the 250-line cap. Nothing here is committed as a
file: a fixture photo in this repository is exactly what the binary gate and
the personal-data gate exist to prevent, so every sample is built at run time
and carries *real* metadata -- a real APP1 Exif segment, a real QuickTime
geotag, a real HEVC parameter set. A file with nothing to strip proves nothing.
"""
from __future__ import annotations
import tempfile
from pathlib import Path
from typing import Final
from signal_bot.external import run
# Warsaw, in an actual GPS tag. If any of this survives, the run must fail.
LATITUDE: Final = "52.2297"
LONGITUDE: Final = "21.0122"
PROBE_MAKE: Final = "SecretPhoneModel"
PROBE_CREATOR: Final = "A Real Person Name"
# An SVG carrying both things exiftool cannot help with: a metadata block and
# a whole embedded raster. Stored as-is, per the decision recorded in
# DOCS-photo-pack.md -- this exists so the e2e run *demonstrates* that hole
# rather than leaving it to the prose.
SVG_DOCUMENT: Final = (
'<?xml version="1.0"?>'
'<svg xmlns="http://www.w3.org/2000/svg" width="8" height="8">'
f"<metadata>{PROBE_CREATOR}</metadata>"
'<rect width="8" height="8" fill="teal"/>'
"</svg>"
).encode()
# Not a media file in any sense. The sniffer must refuse it on the bytes.
JUNK: Final = b"this is not a photo, it is a text file pretending to be one\n"
class ToolError(RuntimeError):
"""An external tool this script needs did not do its job."""
def run_tool(command: list[str]) -> str:
"""Run a tool and return its stdout, failing loudly if it errors.
Goes through signal_bot.external like everything else that starts a
process, and turns its "ok=False" into an exception because this is a
script: there is nothing to degrade to.
"""
result = run(command)
if not result.ok:
msg = f"{command[0]} failed: {result.stderr.strip()}"
raise ToolError(msg)
return result.stdout
def _jpeg(workspace: Path, name: str, colour: str) -> bytes:
"""A JPEG with a real APP1 Exif segment: GPS, Make and an XMP creator."""
path = workspace / f"{name}.jpg"
run_tool(
[
"ffmpeg",
"-loglevel",
"error",
"-f",
"lavfi",
"-i",
f"color=c={colour}:s=48x48:d=1",
"-frames:v",
"1",
str(path),
"-y",
]
)
run_tool(
[
"exiftool",
"-q",
"-overwrite_original",
f"-GPSLatitude={LATITUDE}",
"-GPSLatitudeRef=N",
f"-GPSLongitude={LONGITUDE}",
"-GPSLongitudeRef=E",
f"-Make={PROBE_MAKE}",
f"-XMP:Creator={PROBE_CREATOR}",
str(path),
]
)
return path.read_bytes()
def _mp4(workspace: Path) -> bytes:
"""An MP4 geotagged the way a phone does it: a QuickTime atom, not GPS.
This is the case a naive "-Exif:all -GPS:all -XMP:all" gate passes
straight through, because exiftool files the atom under QuickTime.
"""
path = workspace / "clip.mp4"
run_tool(
[
"ffmpeg",
"-loglevel",
"error",
"-f",
"lavfi",
"-i",
"color=c=navy:s=48x48:d=1",
"-c:v",
"libx264",
"-pix_fmt",
"yuv420p",
str(path),
"-y",
]
)
run_tool(
[
"exiftool",
"-q",
"-overwrite_original",
f"-QuickTime:GPSCoordinates={LATITUDE} {LONGITUDE} 0",
f"-XMP:Creator={PROBE_CREATOR}",
str(path),
]
)
return path.read_bytes()
def _heic(workspace: Path) -> bytes:
"""A geotagged HEIC -- what an iPhone actually sends.
Its HEVC parameter set survives the strip and is on the allowlist; the
GPS and the Make must not survive, and verify_pack re-reads the stored
file to prove it.
"""
path = workspace / "shot.heic"
source = workspace / "heic-source.jpg"
run_tool(
[
"ffmpeg",
"-loglevel",
"error",
"-f",
"lavfi",
"-i",
"color=c=teal:s=48x48:d=1",
"-frames:v",
"1",
str(source),
"-y",
]
)
run_tool(["heif-enc", str(source), "-o", str(path)])
run_tool(
[
"exiftool",
"-q",
"-overwrite_original",
f"-GPSLatitude={LATITUDE}",
"-GPSLatitudeRef=N",
f"-Make={PROBE_MAKE}",
f"-XMP:Creator={PROBE_CREATOR}",
str(path),
]
)
return path.read_bytes()
def build() -> dict[str, bytes]:
"""Every attachment the scenarios reference, by the id they name."""
with tempfile.TemporaryDirectory(prefix="e2e-media-") as raw:
workspace = Path(raw)
return {
"att-person": _jpeg(workspace, "person", "orange"),
"att-custom": _jpeg(workspace, "custom", "purple"),
"att-second": _jpeg(workspace, "second", "green"),
"att-video": _mp4(workspace),
"att-heic": _heic(workspace),
"att-svg": SVG_DOCUMENT,
"att-junk": JUNK,
}