signal-bot/signal_bot/pack_people.py
Krzysztof kuhy Rudnicki 9d546fef9a feat: import a folder of photos without importing it twice
tool/tagger.py did not run at all: 48957a1 renamed pack.default_dir to
default_dirs and missed the caller, so the tagger raised ImportError at
module scope through two releases. Nothing could have caught it -- tool/ is
outside --cov and mypy ran with [signal_bot, fake_signal]. Both are widened
here, and tests/test_tool_imports.py imports every script under tool/.

--import was a filename check and shutil.copy2. Pointed at a phone export
that overlaps the album it re-imported the same photo under whatever name
the exporter used that time. It now identifies files by magic bytes, strips
them through the same media_strip path !add_photo uses, and dedups on exact
bytes twice: the source hash, then -- only for what survives -- the hash
after stripping. sources.json records both, so a second run on the same
folder is a no-op. What the album refuses is moved to _skipped/<reason>/
beside the source, never deleted.

Two things that made "strip on import" impossible until now:

media_tags claims -all= zeroes the QuickTime date tags and allows them by
value. It does not; measured, it leaves CreateDate at its real value, and
every mp4 was being refused. -time:all= is what zeroes them.

And a tag allowlist cannot see the worst class at all. IMG_6924.mp4 carries
mebx timed-metadata holding per-frame face bounding boxes and a face id,
and those survive a complete tag strip because they live in mdat as opaque
samples. media_tracks.py drops the track with ffmpeg and verifies with
ffprobe that only video and audio survived. This is not the ffmpeg mistake
media_strip warns about: that was ffmpeg replacing exiftool for tags, this
is track removal with exiftool still doing every tag, and the gate is a
positive read-back rather than an exit code.

The roster was write-once. The setup form only appeared on an empty album,
Album.setup overwrote the file whole, and keys were positional (p{len+1}),
so inserting somebody renumbered everyone after them and repointed every
tags.json entry at the wrong person. Keys now come from the highest number
in use, the form merges, and a person can be added mid-session.

--untagged filters the page and never the server's list. /media/{index} and
/api/tag both index the full album, so a shorter list server-side would
show item i of the album while tagging item i of the filtered set.

Measured on the live album: 174 source files -> 1 not media, 82 already in
pack, 24 duplicate in batch, 67 imported, and re-running imports nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBrE1b9HvFw3xvG9bJiDFt
2026-08-31 13:22:57 +02:00

184 lines
6.6 KiB
Python

# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Adding somebody to the roster of an album that already has one.
The tagger could only ever write ``people.json`` from empty: the setup form
appeared when there were no people, ``Album.setup`` overwrote the file whole,
and the keys were handed out by position -- ``p{len + 1}``. Together those
mean the obvious thing (edit the list to insert somebody) renumbers everyone
after them, and every ``tags.json`` entry then points at the wrong person,
silently, with no error at any layer. So in practice a roster was written once
and never touched, and a new face meant hand-editing JSON.
This module is the append-only path instead. Keys come from the highest number
in use, never the count -- the same rule ``pack_custom.next_key`` follows, and
for the same reason: deleting ``p3`` by hand must not let a later add reuse the
key and inherit its photos.
Two checks before anything is written, and they are different in kind:
* **Against the album**, by writing the file and then calling ``pack_load.load``
on it. That is the whole of pack_load's validation -- trigger shape, no
digits, uniqueness after folding, across people and combos and custom labels
-- rather than a second copy of it here that could drift. A failure rolls the
file back to the bytes it had.
* **Against the bot's own commands**, from ``aliases.json``. ``load`` cannot see
those: a person triggered by ``!losuj`` is a perfectly valid pack that
quietly shadows ``!random`` at run time. This is the same collision
``pack_add`` refuses a custom label for.
"""
from __future__ import annotations
import json
from pathlib import Path
from typing import TYPE_CHECKING, Any, Final
from signal_bot.folding import fold
from signal_bot.groups import ConfigError
from signal_bot.pack import PEOPLE_FILE
from signal_bot.pack_load import load
from signal_bot.pack_store import write_json
if TYPE_CHECKING:
from collections.abc import Sequence
ALIAS_FILE: Final = Path(__file__).with_name("aliases.json")
# The person-key shape, mirroring pack_custom.KEY_PATTERN for custom tags.
KEY_PREFIX: Final = "p"
def reserved() -> frozenset[str]:
"""Every folded word that is already one of the bot's own commands."""
document = json.loads(ALIAS_FILE.read_text(encoding="utf-8"))
return frozenset(
fold(trigger)
for name, triggers in document.items()
if not name.startswith("_")
for trigger in triggers
)
def next_key(people: Sequence[dict[str, Any]]) -> str:
"""The key a new person would take, without taking it.
One past the highest number in use rather than the count, so a roster
somebody has edited by hand cannot make an add reuse a retired key.
"""
used = [
int(key.removeprefix(KEY_PREFIX))
for person in people
if isinstance(key := person.get("key"), str)
and key.startswith(KEY_PREFIX)
and key.removeprefix(KEY_PREFIX).isdigit()
]
return f"{KEY_PREFIX}{max(used, default=0) + 1}"
def normalise(triggers: Sequence[str]) -> list[str]:
"""Trigger spellings as they are stored: "!"-prefixed, blanks dropped.
People type "alfa" as often as "!alfa" and the difference is not one
worth an error message.
"""
cleaned = []
for raw in triggers:
word = raw.strip()
if word:
cleaned.append(word if word.startswith("!") else f"!{word}")
return cleaned
def add_person(directory: Path, triggers: Sequence[str]) -> str:
"""Append one person to the roster and prove the album still loads.
Args:
directory: The album directory.
triggers: What somebody types to get this person's photos.
Returns:
The key the new person was given.
Raises:
ConfigError: The triggers are empty, collide with one of the bot's
own commands, or leave the album in a state that does not load.
The manifest is unchanged in every one of those cases.
"""
wanted = normalise(triggers)
if not wanted:
msg = "a person needs at least one trigger"
raise ConfigError(msg)
taken = reserved() & {fold(trigger) for trigger in wanted}
if taken:
msg = f"already a command: {sorted(taken)}"
raise ConfigError(msg)
path = directory / PEOPLE_FILE
document = _document(path)
people = list(document.get("people", []))
key = next_key(people)
document["people"] = [*people, {"key": key, "triggers": wanted}]
return _commit(directory, path, document, key)
def merge(document: dict[str, Any], people: Sequence[dict[str, Any]]) -> list[dict[str, Any]]:
"""The roster with ``people`` folded in, existing keys left where they are.
Used by the setup form, which used to replace the file wholesale. Anyone
already present keeps their key and gains any new triggers; anyone new is
appended with a fresh key.
"""
merged = [dict(person) for person in document.get("people", [])]
by_key = {person.get("key"): person for person in merged}
for person in people:
existing = by_key.get(person.get("key"))
if existing is None:
fresh = dict(person)
fresh["key"] = next_key(merged)
merged.append(fresh)
by_key[fresh["key"]] = fresh
continue
seen = set(existing.get("triggers", []))
existing["triggers"] = [
*existing.get("triggers", []),
*(t for t in person.get("triggers", []) if t not in seen),
]
return merged
def _commit(directory: Path, path: Path, document: dict[str, Any], key: str) -> str:
"""Write the roster, or put back exactly what was there and re-raise."""
previous = path.read_bytes() if path.exists() else None
write_json(path, document)
try:
load(directory)
except ConfigError:
if previous is None:
path.unlink(missing_ok=True)
else:
path.write_bytes(previous)
raise
return key
def _document(path: Path) -> dict[str, Any]:
"""The roster as it stands.
Strict, unlike the ledger's reader: this file *is* the album's manifest,
and appending to a document that failed to parse would write a roster
that silently dropped everyone already in it.
Raises:
ConfigError: The file is not a JSON object.
"""
if not path.exists():
return {}
try:
document = json.loads(path.read_text(encoding="utf-8"))
except json.JSONDecodeError as error:
msg = f"{path.name} is not valid JSON: {error}"
raise ConfigError(msg) from error
if not isinstance(document, dict):
msg = f"{path.name} is not an object"
raise ConfigError(msg)
return document