3 Commits

Author	SHA1	Message	Date
Krzysztof kuhy Rudnicki	9d546fef9a	feat: import a folder of photos without importing it twice ... tool/tagger.py did not run at all: 48957a1 renamed pack.default_dir to default_dirs and missed the caller, so the tagger raised ImportError at module scope through two releases. Nothing could have caught it -- tool/ is outside --cov and mypy ran with [signal_bot, fake_signal]. Both are widened here, and tests/test_tool_imports.py imports every script under tool/. --import was a filename check and shutil.copy2. Pointed at a phone export that overlaps the album it re-imported the same photo under whatever name the exporter used that time. It now identifies files by magic bytes, strips them through the same media_strip path !add_photo uses, and dedups on exact bytes twice: the source hash, then -- only for what survives -- the hash after stripping. sources.json records both, so a second run on the same folder is a no-op. What the album refuses is moved to _skipped/<reason>/ beside the source, never deleted. Two things that made "strip on import" impossible until now: media_tags claims -all= zeroes the QuickTime date tags and allows them by value. It does not; measured, it leaves CreateDate at its real value, and every mp4 was being refused. -time:all= is what zeroes them. And a tag allowlist cannot see the worst class at all. IMG_6924.mp4 carries mebx timed-metadata holding per-frame face bounding boxes and a face id, and those survive a complete tag strip because they live in mdat as opaque samples. media_tracks.py drops the track with ffmpeg and verifies with ffprobe that only video and audio survived. This is not the ffmpeg mistake media_strip warns about: that was ffmpeg replacing exiftool for tags, this is track removal with exiftool still doing every tag, and the gate is a positive read-back rather than an exit code. The roster was write-once. The setup form only appeared on an empty album, Album.setup overwrote the file whole, and keys were positional (p{len+1}), so inserting somebody renumbered everyone after them and repointed every tags.json entry at the wrong person. Keys now come from the highest number in use, the form merges, and a person can be added mid-session. --untagged filters the page and never the server's list. /media/{index} and /api/tag both index the full album, so a shorter list server-side would show item i of the album while tagging item i of the filtered set. Measured on the live album: 174 source files -> 1 not media, 82 already in pack, 24 duplicate in batch, 67 imported, and re-running imports nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBrE1b9HvFw3xvG9bJiDFt	2026-08-31 13:22:57 +02:00
Krzysztof kuhy Rudnicki	21ccf8d8eb	fix: find exiftool the way the service does, and self-test with a photo ... Two things the first live run found, neither of which any test could have. **exiftool was invisible to the unit.** Arch's perl-image-exiftool installs to /usr/bin/vendor_perl/exiftool. That is on an interactive shell's PATH, because perl's profile.d puts it there, and it is *not* on a systemd user unit's, whose default is "/usr/local/bin:/usr/bin". So shutil.which found it in every test and in every shell, and the first real !add_photo under the unit answered "exiftool is not installed; refusing to store .jpg" -- with exiftool plainly installed. media_strip.locate() now tries PATH and then the known install locations, and install_service.sh verifies the result through the bot's own resolver under the unit's PATH rather than through `command -v`, which answers for the wrong shell. Fixing the lookup rather than setting PATH in the unit: the latter is a note somebody has to remember on the next machine. The absent-exiftool test had to blind *both* lookups to keep covering its branch -- on a machine that has the package the fallback finds it, which is precisely how the gap survived. **The self-test could not carry a photo.** Signal never delivers an account its own sends: measured against the live container, a single-device account gets no sync copy either, and a sent attachment is not added to the container's attachment store (only inbound ones are). So !add_photo could not be exercised the way every other command is. selftest.build_frame and POST /_selftest now take attachment ids that the container's store already holds, so the handler fetches them back over the real GET /v1/attachments/{id} -- the ingest path under test is the live one, not a hand-fed buffer. Still fail-closed on SELFTEST_GROUPS. Verified live, against real Signal and the real container, in the testbot group, on a throwaway album scoped to that group so the private one was neither read nor written: - a geotagged JPEG (GPS + Make) went in as "!dodaj alfa wakacjelive" - the stored file reports zero non-structural tags, and neither the GPS marker nor the Make string survives in its bytes - GET /v1/attachments/<id> now 404s and the file is gone from disk, so the second copy really is deleted - the typed label appears zero times in the journal - !losowe wakacjelive, !alfa and an unknown label all answered without error Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JqqE22jp94B1ypwxppdr1o	2026-08-31 11:40:07 +02:00
Krzysztof kuhy Rudnicki	aefffb2fd9	feat: add a photo to the album from the chat, tagged, stripped and verified ... Send a photo captioned "!add_photo alfa wakacje" in one of the album's own groups and it lands in the pack tagged with both, retrievable through !alfa and the new !losowe immediately -- no restart, and no tagger session on the PC. A label matching nobody becomes a custom tag, and !losowe is the only door to one, which is what keeps a live swap a *data* swap: it can never register a trigger or widen a command's groups. No typed label travels. Labels become pseudonym keys in a new registry arg_resolver, at the dispatcher, before the line that logs the command name -- so no handler, journal entry, command_uses row or reply body ever holds a word somebody typed. The reply is counts and pseudonyms: "dodano · p1: 4 · t1: 1". Stripping is exiftool with a *positive* check on the output, never an exit code. ffmpeg was tried first and fails silently on photos: given a JPEG with a real APP1 Exif segment it returned a byte-identical file, GPS intact, exit 0. The surviving-tag allowlist is per tag *name* rather than per group, because measurement showed a group-level "must be empty" rule rejects every format except JPEG -- a valid PNG always reports its IHDR, an MP4 some fifty QuickTime tags that are the container itself. Codec configuration (CompressorName, VendorID, the HEVC parameter set) is on the list by decision: CompressorName lives in the avc1 sample description and exiftool cannot write it at all, so the alternative was refusing every video and every iPhone HEIC. Location and identity tags are still refused, and the e2e run re-reads the stored files to prove a QuickTime geotag did not survive. .webm and .svg stay on the format allowlist and are stored *unstripped*: exiftool 13.55 can write neither, and its SVG reader does not even surface <metadata>. That is a deliberate, documented hole, not an oversight. Two things this had to defuse along the way: - personal_data_sources now reads custom.json, so "!add_photo kot <photo>" would have outlawed the word "kot" and refused every future commit, fixable only by hand-editing an album the committer cannot see. Minting now refuses a word that is already a command, and the gate subtracts the repo's own alias vocabulary from custom labels only -- curated people.json names are never excused. - signal_bot/external.py is now the single subprocess site in the repository, so ruff's S603 needs exactly one scoped per-file-ignore instead of one per module that shells out. A test asserts nothing else imports subprocess. A failed add changes nothing: media and the three manifests are written, the album is reloaded, and only a load that succeeds is published; a load that fails rolls all four back and keeps serving the pack that was already working, without raising -- Restart=always would turn a ConfigError into a loop. 977 tests, 100% branch coverage, no omits and no suppressions. scripts/e2e.sh gains 8 scenarios driven against the real exiftool on media generated at run time, and asserts the invariants a reply cannot: that a refused upload is not stored, that a message with no attachment leaves custom.json byte-identical, and that signal-cli's own copy of every ingested attachment is gone. Closes TODO-add-photo.md. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JqqE22jp94B1ypwxppdr1o	2026-08-31 11:20:00 +02:00