feat: a word with two meanings gets the one made of steel

!pułapka (and !trap, !sidła, !wnyk, !potrzask, 26 spellings in all) posts a
photograph of the literal device -- a bear trap, a mousetrap, a snare, a fish
weir -- captioned with what Commons says it is.

The pool is 2218 Commons page ids, harvested by tool/gen_traps.py from a
curated seed list and committed. Page ids and not filenames on purpose: a
filename carries photographers' and sitters' names, and one of those
colliding with a label somebody mints from the chat would wedge
check_personal_data.py permanently, over data nobody can edit by hand.
_repo_vocabulary() subtracts aliases.json and nothing else, so it would not
have caught it. An integer cannot carry a name.

Nothing is cached. !papież caches because 267 portraits are a closed set a
group works through; a uniform draw from 2218 warms a cache essentially
never, so this fetches per call the way !kot and !pies do.

What the harvest measured, and now enforces:

- five plausible seed categories do not exist (Snares, Man traps, Hunting
  traps, Wasp traps, Animal trapping). A missing category is an empty answer,
  not an error, so verify_seeds() asks Commons before harvesting and fails
  loudly naming all of them.
- depth three from Category:Fish traps reaches Alfred Sisley's net paintings.
  Depth one, from seeds picked by hand.
- "<thing> in art" categories are paintings: Mousetraps in art put a Robert
  Campin altarpiece in the pool. Pruned by pattern, since Commons mints a new
  one whenever somebody needs it. Booby traps, the Wolfsangel, camera traps
  and animals dying in traps are pruned by name.
- Commons rate-limits hard, so a partial harvest must not be committed
  silently: 429 backs off, and a short pool refuses to write at all.

Descriptions are HTML and sometimes say nothing, so the tags come out, the
Fisheries records' trailing catalogue list is cut, a description naming only
the country falls back to the filename, and the file's own page is always
linked.

!łapka is not an alias: it resolves to !papa at distance two, which
test_nearest.py fails on, and rightly -- the suggestion would be a coin toss.

signal_bot/commons.py is the imageinfo query both Commons commands now share;
portraits.py keeps its behaviour and its tests unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01416TegJ3sa8xL5qGGgrygP
This commit is contained in:
Krzysztof Rudnicki 2026-08-31 18:01:27 +02:00
parent 262b874bd9
commit 14d422295f
18 changed files with 3130 additions and 37 deletions

View File

@ -15,6 +15,7 @@ This file is the trigger reference — every spelling that resolves, which
| cat | `!kot` `!koty` `!kots` `!cat` `!cats` `!meow` `!miau` + styled variants |
| dog | `!pies` `!psy` `!dog` `!dogs` `!woof` `!szczek` + styled variants |
| gecko | `!gekon` `!gekony` `!geko` `!gecko` `!geckon` `!geckos` |
| trap | `!pułapka` `!pulapka` `!pułapki` `!pulapki` `!pułapkę` `!pulapke` `!trap` `!traps` `!trapy` `!trapper` `!sidła` `!sidla` `!sidło` `!sidlo` `!wnyk` `!wnyki` `!potrzask` `!zasadzka` `!klatka` `!samołówka` `!samolowka` `!beartrap` `!bear_trap` `!snare` `!snares` `!mousetrap` |
| pope | `!papież` `!papiez` `!papieze` `!papieża` `!papa` `!pope` `!popes` `!pontifex` + styled variants |
| stats_today | `!staty` `!statystyki` `!stats` `!dziś` `!dzis` |
| stats_yesterday | `!wczoraj` `!yesterday` |

View File

@ -1,7 +1,7 @@
# signal-bot
A Signal group bot: `!cat` / `!dog` / `!gekon` image commands, `!papież`, and
group message statistics. The group's own data stays out: `DOCS-privacy.md`.
A Signal group bot: `!cat` / `!dog` / `!gekon` / `!pułapka` image commands,
`!papież`, and group statistics. The group's own data stays out: `DOCS-privacy.md`.
Runs on its **own registered Signal number** (or, if you prefer, as a linked
device on an existing account) via

View File

@ -0,0 +1,5 @@
name: "!pułapka posts a photograph of the device, captioned with what it is"
inject:
- { text: "!pułapka", uuid: uuid-trap, name: Trapper }
expect:
- { kind: image, to: reply }

View File

@ -30,6 +30,34 @@
"!geckon",
"!geckos"
],
"trap": [
"!pułapka",
"!pulapka",
"!pułapki",
"!pulapki",
"!pułapkę",
"!pulapke",
"!trap",
"!traps",
"!trapy",
"!trapper",
"!sidła",
"!sidla",
"!sidło",
"!sidlo",
"!wnyk",
"!wnyki",
"!potrzask",
"!zasadzka",
"!klatka",
"!samołówka",
"!samolowka",
"!beartrap",
"!bear_trap",
"!snare",
"!snares",
"!mousetrap"
],
"pope": [
"!papież",
"!papiez",

View File

@ -35,6 +35,7 @@ from signal_bot.stats import periods
from signal_bot.stats.commands import register_stats_commands
from signal_bot.stats.nightly import days_to_report, mark_reported
from signal_bot.suggest_command import register_suggest_command
from signal_bot.traps_command import register_trap_command
from signal_bot.versioning import POPE
if TYPE_CHECKING:
@ -198,6 +199,7 @@ def build_bot(
endpoints=config.image_api_urls,
)
register_pope_command(registry, portraits, decorate=rate_limit)
register_trap_command(registry, fetcher, config.commons_api_url, decorate=rate_limit)
register_stats_commands(registry, decorate=rate_limit)
# Rate-limited like every other command, and unlike !add_photo, whose
# "NO CAP, NO LIMITS" is a deliberate exception rather than a default.

126
signal_bot/commons.py Normal file
View File

@ -0,0 +1,126 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""What a Wikimedia Commons image fetch needs, shared by its two callers.
`!papież` names a file by title and caches the result; `!pułapka` names one by
page id and does not cache anything. Everything between those two ends -- the
query, the width, the place `thumburl` turns up in the answer -- is identical,
so it lives here rather than twice.
"""
from __future__ import annotations
import html
import re
from typing import Any, Final
from signal_bot.http_client import ImageFetchError
# Where imageinfo puts the thumbnail URL in its reply. The same path whether
# the file was named by title or by page id.
THUMB_PATH: Final = ("query", "pages", 0, "imageinfo", 0, "thumburl")
# Commons hosts these four for the 267 portraits: 245 jpg, 15 png, 6 gif,
# 1 jpeg. An unknown suffix is refused rather than guessed, because the
# extension is what tells Signal how to render the attachment.
SUFFIXES: Final = frozenset({".jpg", ".jpeg", ".png", ".gif"})
# Long enough to say what the thing is, short enough not to bury the photo.
CAPTION_LIMIT: Final = 220
_TAGS: Final = re.compile(r"<[^>]+>")
# The Fisheries records put their prose first and then an HTML list of
# catalogue fields ("Subject: Fishing nets", "Tag: Fisheries Techniques").
# Cutting at the list keeps the sentence and drops the filing cabinet.
_LIST: Final = "<ul"
# Commons templates that fill the description field with something that is
# not a description. Measured on a hand-woven Maltese fish trap: its whole
# description named only the country, where the filename said what the
# object was. Treated as no description at all.
_BOILERPLATE: Final = ("this photo has been taken in the country",)
def imageinfo_params(
width: int,
*,
title: str | None = None,
page_id: int | None = None,
describe: bool = False,
) -> dict[str, str]:
"""Build the imageinfo query for one Commons file.
Args:
width: Thumbnail width to ask for, in pixels.
title: Full page title, "File:" prefix included.
page_id: Commons page id. Preferred where the caller stores a list,
because an integer cannot carry somebody's name into the
repository the way a filename can.
describe: Also ask for the file's description, so one request
answers both what the picture is and where it lives.
Returns:
Query parameters, ready for ImageFetcher.fetch_bytes.
Raises:
ValueError: If both or neither of title and page_id were given.
Silently preferring one would make a caller's typo look like a
missing file, which is the same shape as a real 404.
"""
if (title is None) == (page_id is None):
msg = "name a Commons file by exactly one of title or page_id"
raise ValueError(msg)
selector = {"titles": title} if title is not None else {"pageids": str(page_id)}
described = (
{"iiprop": "url|extmetadata", "iiextmetadatafilter": "ImageDescription"}
if describe
else {"iiprop": "url"}
)
return {
"action": "query",
"format": "json",
"formatversion": "2",
"prop": "imageinfo",
"iiurlwidth": str(width),
**described,
**selector,
}
def page_of(document: object) -> dict[str, Any]:
"""Return the single page an imageinfo answer describes.
Raises:
ImageFetchError: If Commons answered without one -- a deleted or
renumbered page id comes back as `{"missing": true}` and has no
imageinfo at all.
"""
pages = document.get("query", {}).get("pages", []) if isinstance(document, dict) else []
if not pages or not pages[0].get("imageinfo"):
msg = f"Commons named no image: {document!r:.200}"
raise ImageFetchError(msg)
page: dict[str, Any] = pages[0]
return page
def caption_for(page: dict[str, Any]) -> str:
"""Return the line that rides along with the photograph.
Commons descriptions are HTML, sometimes several languages deep, and
sometimes only the filename repeated back. So the tags come out, the
catalogue list is cut, and a description that says nothing falls back to
the title -- but the file's own page is always linked, because that is
what answers "what am I even looking at" when the rest does not.
The text is fetched at run time and posted to the group; none of it is
ever written into this repository, which is why it may be free text
where traps.json may not.
"""
info = page["imageinfo"][0]
raw = str(info.get("extmetadata", {}).get("ImageDescription", {}).get("value", ""))
text = " ".join(html.unescape(_TAGS.sub(" ", raw.split(_LIST, 1)[0])).split())
if text.lower().startswith(_BOILERPLATE):
text = ""
if not text:
text = str(page.get("title", "")).removeprefix("File:").rsplit(".", 1)[0]
if len(text) > CAPTION_LIMIT:
text = text[:CAPTION_LIMIT].rstrip() + "…"
url = str(info.get("descriptionurl", ""))
return f"{text}\n{url}" if url else text

View File

@ -22,11 +22,12 @@ DEFAULT_GECKO_API_URL: Final = (
"?taxon_id=371057&photos=true&quality_grade=research"
"&photo_license=cc-by,cc-by-sa,cc0&per_page=1"
)
# Commons resolves a portrait *filename* to a real upload.wikimedia.org URL.
# Special:FilePath would be shorter but answers 302, and this repo's httpx
# client does not follow redirects; imageinfo hands back the final URL and a
# thumbnail in one hop, which is also what keeps a 4000px original off Signal.
# Commons resolves a file -- named by title for !papiez, by page id for
# !pulapka -- to a real upload.wikimedia.org URL. Special:FilePath would be
# shorter but answers 302, and this repo's httpx client does not follow
# redirects; imageinfo hands back the final URL and a thumbnail in one hop,
# which is also what keeps a 4000px original off Signal.
DEFAULT_COMMONS_API_URL: Final = "https://commons.wikimedia.org/w/api.php"
# Wide enough to look like a photograph rather than an avatar, small enough
# that the whole 267-portrait cache stays well under 100 MB.
PORTRAIT_WIDTH: Final = 640
THUMB_WIDTH: Final = 640

View File

@ -38,7 +38,7 @@ HEADING: Final = "Komendy"
# Anything missing from here still gets printed, in a block of its own -- see
# the module docstring.
GROUPS: Final[tuple[tuple[str, tuple[str, ...]], ...]] = (
("Obrazki", ("cat", "dog", "gecko")),
("Obrazki", ("cat", "dog", "gecko", "trap")),
("Papież", ("pope",)),
("Propozycje", ("suggest",)),
(

View File

@ -65,17 +65,32 @@ class ImageFetcher:
Raises:
ImageFetchError: The JSON did not contain a usable URL.
"""
response = await self._client.get(api_url, params=params)
response.raise_for_status()
image_url = _walk(response.json(), url_path)
image_url = _walk(await self.fetch_json(api_url, params), url_path)
if rewrite is not None:
image_url = image_url.replace(*rewrite)
return await self.fetch_image(image_url)
# These bytes are never written to a path built from the remote URL:
# the original bot named a local file after the URL's last path
# segment and wrote it into the process CWD, which is both path
# traversal and unbounded. portraits.py does cache to disk, but under
# a name it derives from the pope's ordinal, never from upstream.
async def fetch_json(self, api_url: str, params: dict[str, str] | None = None) -> object:
"""Fetch one JSON document from an API.
Separate from fetch_bytes because a Commons answer carries more than
the picture: !pułapka needs the description in the same reply, and
asking twice would be both wasteful and a chance for the two halves
to disagree.
"""
response = await self._client.get(api_url, params=params)
response.raise_for_status()
return response.json()
async def fetch_image(self, image_url: str) -> bytes:
"""Fetch the bytes at an already-resolved image URL.
These bytes are never written to a path built from the remote URL:
the original bot named a local file after the URL's last path
segment and wrote it into the process CWD, which is both path
traversal and unbounded. portraits.py does cache to disk, but under
a name it derives from the pope's ordinal, never from upstream.
"""
image = await self._client.get(image_url)
image.raise_for_status()
return image.content

View File

@ -42,7 +42,7 @@ MEDIA_DIR: Final = "media"
# What Signal is told an attachment is. Closed, and checked at load time: the
# suffix is the only thing that tells Signal how to render the file, so an
# unknown one is refused rather than guessed. Mirrors portraits._SUFFIXES.
# unknown one is refused rather than guessed. Mirrors commons.SUFFIXES.
MEDIA_TYPES: Final[dict[str, str]] = {
".jpg": "image/jpeg",
".jpeg": "image/jpeg",

View File

@ -12,22 +12,16 @@ from __future__ import annotations
import base64
import os
from pathlib import Path
from typing import TYPE_CHECKING, Final
from typing import TYPE_CHECKING
from signal_bot.endpoints import PORTRAIT_WIDTH
from signal_bot.commons import SUFFIXES, THUMB_PATH, imageinfo_params
from signal_bot.endpoints import THUMB_WIDTH
from signal_bot.http_client import ImageFetchError
if TYPE_CHECKING:
from signal_bot.http_client import ImageFetcher
from signal_bot.versioning import Pope
# Where imageinfo puts the thumbnail URL in its reply.
_THUMB_PATH: Final = ("query", "pages", 0, "imageinfo", 0, "thumburl")
# Commons hosts these four for the 267 portraits: 245 jpg, 15 png, 6 gif,
# 1 jpeg. An unknown suffix is refused rather than guessed, because the
# extension is what tells Signal how to render the attachment.
_SUFFIXES: Final = frozenset({".jpg", ".jpeg", ".png", ".gif"})
def default_cache_dir() -> Path:
"""Return the portrait directory, preferring the one systemd hands us.
@ -68,7 +62,7 @@ class PortraitCache:
data in a way an integer is not.
"""
suffix = Path(pope.portrait).suffix.lower()
if suffix not in _SUFFIXES:
if suffix not in SUFFIXES:
msg = f"portrait for #{pope.ordinal} has an unusable suffix: {pope.portrait!r}"
raise ImageFetchError(msg)
return self._dir / f"{pope.ordinal:03d}{suffix}"
@ -86,16 +80,8 @@ class PortraitCache:
image = await self._fetcher.fetch_bytes(
self._api_url,
_THUMB_PATH,
params={
"action": "query",
"format": "json",
"formatversion": "2",
"prop": "imageinfo",
"iiprop": "url",
"iiurlwidth": str(PORTRAIT_WIDTH),
"titles": f"File:{pope.portrait}",
},
THUMB_PATH,
params=imageinfo_params(THUMB_WIDTH, title=f"File:{pope.portrait}"),
)
_write(path, image)
return image

2223
signal_bot/traps.json Normal file

File diff suppressed because it is too large Load Diff

116
signal_bot/traps_command.py Normal file
View File

@ -0,0 +1,116 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""The !pułapka command: a photograph of the device, and what it is.
The joke is literalism -- whatever the word meant to whoever typed it, what
comes back is a bear trap, a mousetrap, a snare, a cage, a fish weir. The
caption is not an apology for the joke, it is the answer to the next
question: Commons has plenty of files whose subject is genuinely unguessable
("Molchreuse", "K Pañayacu 0811"), and a newt trap nobody can identify is
just a photograph of a basket.
Nothing is cached. !papież caches because 267 portraits are a closed set a
group works through; a uniform draw from a couple of thousand traps warms a
cache essentially never, so the disk would fill with images nobody asks for
twice. This works the way !kot and !pies do instead: fetch, post, forget.
"""
from __future__ import annotations
import json
import secrets
from base64 import b64encode
from pathlib import Path
from typing import TYPE_CHECKING, Final
from signal_bot.animals import load_aliases
from signal_bot.commons import caption_for, imageinfo_params, page_of
from signal_bot.endpoints import THUMB_WIDTH
if TYPE_CHECKING:
from collections.abc import Sequence
from signal_bot.context import CommandContext
from signal_bot.decorators import Decorator
from signal_bot.http_client import ImageFetcher
from signal_bot.registry import CommandRegistry, Handler
COMMAND_NAME: Final = "trap"
DESCRIPTION: Final = "losowe zdjęcie pułapki"
TRAPS_FILE: Final = Path(__file__).with_name("traps.json")
def load_traps(path: Path = TRAPS_FILE) -> tuple[int, ...]:
"""Read the trap pool.
The file holds Commons page ids -- bare integers -- and that is checked
here rather than assumed. Filenames would be free text carrying
photographers' and sitters' names, and a name committed here collides
sooner or later with a label somebody minted from the chat, at which
point scripts/check_personal_data.py refuses every commit for good over
data nobody can edit by hand. An integer cannot do that, and this is the
guard that keeps it an integer.
Args:
path: Where to read the pool from. Defaults to the shipped file.
Returns:
Every Commons page id in the pool.
Raises:
ValueError: If the pool is empty or holds anything but integers.
"""
document = json.loads(path.read_text(encoding="utf-8"))
traps = document["traps"]
if not traps or not all(isinstance(page, int) for page in traps):
msg = f"{path} must hold a non-empty list of Commons page ids"
raise ValueError(msg)
return tuple(traps)
def random_trap(traps: Sequence[int]) -> int:
"""Return one page id, uniformly at random.
secrets rather than random for the reason animals.paged_url gives: the
choice is not security-sensitive, but ruff's ALL rule set flags `random`
and this repository forbids the noqa that would silence it.
"""
return traps[secrets.randbelow(len(traps))]
def trap_handler(
fetcher: ImageFetcher,
api_url: str,
traps: Sequence[int] | None = None,
) -> Handler:
"""Build the handler that posts one random trap."""
pool = load_traps() if traps is None else tuple(traps)
async def handler(ctx: CommandContext) -> None:
# One request, because the caption and the picture have to describe
# the same file; two would be a chance for them to disagree.
document = await fetcher.fetch_json(
api_url,
imageinfo_params(THUMB_WIDTH, page_id=random_trap(pool), describe=True),
)
page = page_of(document)
image = await fetcher.fetch_image(str(page["imageinfo"][0]["thumburl"]))
await ctx.reply_image(b64encode(image).decode("ascii"), caption_for(page))
return handler
def register_trap_command(
registry: CommandRegistry,
fetcher: ImageFetcher,
api_url: str,
*,
decorate: Decorator | None = None,
aliases: dict[str, list[str]] | None = None,
traps: Sequence[int] | None = None,
) -> None:
"""Register !pułapka under every trigger aliases.json gives it."""
table = load_aliases() if aliases is None else aliases
handler = trap_handler(fetcher, api_url, traps)
if decorate is not None:
handler = decorate(handler)
registry.register(COMMAND_NAME, table[COMMAND_NAME], handler, DESCRIPTION)

123
tests/test_commons.py Normal file
View File

@ -0,0 +1,123 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""The imageinfo query both Commons commands send."""
from __future__ import annotations
import pytest
from signal_bot.commons import (
CAPTION_LIMIT,
SUFFIXES,
THUMB_PATH,
caption_for,
imageinfo_params,
page_of,
)
from signal_bot.http_client import ImageFetchError
def test_a_file_can_be_named_by_title() -> None:
params = imageinfo_params(640, title="File:1868 Mausefalle.jpg")
assert params["titles"] == "File:1868 Mausefalle.jpg"
assert "pageids" not in params
assert params["iiurlwidth"] == "640"
assert params["prop"] == "imageinfo"
def test_a_file_can_be_named_by_page_id() -> None:
# The page id arrives as an int and has to reach Commons as a string;
# sending the int would raise inside httpx rather than here.
params = imageinfo_params(480, page_id=111419226)
assert params["pageids"] == "111419226"
assert "titles" not in params
@pytest.mark.parametrize(
("title", "page_id"),
[(None, None), ("File:x.jpg", 1)],
)
def test_naming_a_file_neither_way_or_both_ways_is_refused(
title: str | None, page_id: int | None
) -> None:
# Preferring one silently would make a caller's mistake look like a 404,
# which is the one failure shape that already exists downstream.
with pytest.raises(ValueError, match="exactly one"):
imageinfo_params(640, title=title, page_id=page_id)
def test_the_thumbnail_path_is_where_commons_actually_puts_it() -> None:
# Pinned because both callers walk it and neither can see it fail until
# a real fetch returns something unwalkable.
assert THUMB_PATH == ("query", "pages", 0, "imageinfo", 0, "thumburl")
def test_the_suffix_allowlist_is_what_signal_can_render() -> None:
assert ".jpg" in SUFFIXES
assert ".svg" not in SUFFIXES
def _page(
description: str | None = None, *, title: str = "File:Molchreuse.JPG", url: str = ""
) -> dict[str, object]:
"""One imageinfo page, shaped the way Commons shapes it."""
info: dict[str, object] = {"thumburl": "https://cdn/x.jpg"}
if url:
info["descriptionurl"] = url
if description is not None:
info["extmetadata"] = {"ImageDescription": {"value": description}}
return {"title": title, "imageinfo": [info]}
def test_asking_for_a_description_widens_the_query() -> None:
plain = imageinfo_params(640, page_id=1)
described = imageinfo_params(640, page_id=1, describe=True)
assert plain["iiprop"] == "url"
assert "iiextmetadatafilter" not in plain
assert described["iiprop"] == "url|extmetadata"
assert described["iiextmetadatafilter"] == "ImageDescription"
def test_the_caption_strips_markup_and_links_the_page() -> None:
page = _page("Eine <b>Molchreuse</b> &amp; ein Korb", url="https://commons/File:M.JPG")
assert caption_for(page) == "Eine Molchreuse & ein Korb\nhttps://commons/File:M.JPG"
def test_the_catalogue_list_is_cut_and_the_sentence_kept() -> None:
# The Fisheries records append their filing fields as an HTML list.
page = _page("Round Fyke (Bollreuse). Prussia\n<ul><li>Subject: Fishing nets</li></ul>")
assert caption_for(page) == "Round Fyke (Bollreuse). Prussia"
@pytest.mark.parametrize("description", [None, "", "<ul><li>Subject: only filing</li></ul>"])
def test_a_description_that_says_nothing_falls_back_to_the_title(description: str | None) -> None:
assert caption_for(_page(description)) == "Molchreuse"
def test_a_long_description_is_cut_rather_than_burying_the_photo() -> None:
page = _page("word " * 200)
caption = caption_for(page)
assert caption.endswith("…")
# rstrip() before the ellipsis means the exact length depends on where
# the cut landed; the ceiling is what matters.
assert len(caption) <= CAPTION_LIMIT + 1
def test_a_page_commons_could_not_find_is_an_image_fetch_error() -> None:
# A deleted or renumbered page id comes back as {"missing": true}.
for document in ({"query": {"pages": [{"missing": True}]}}, {"query": {"pages": []}}, {}, []):
with pytest.raises(ImageFetchError, match="named no image"):
page_of(document)
def test_the_page_comes_back_when_commons_did_find_it() -> None:
assert page_of({"query": {"pages": [_page("x")]}})["title"] == "File:Molchreuse.JPG"
def test_a_template_that_is_not_a_description_falls_back_to_the_title() -> None:
# Measured on a Maltese fish trap: the filename said more than Commons'
# own description field did.
page = _page(
"This photo has been taken in the country: Malta",
title="File:Hand woven fish trap.jpg",
)
assert caption_for(page) == "Hand woven fish trap"

View File

@ -33,6 +33,7 @@ from signal_bot.portraits import PortraitCache
from signal_bot.registry import CommandRegistry
from signal_bot.stats.commands import register_stats_commands
from signal_bot.suggest_command import register_suggest_command
from signal_bot.traps_command import register_trap_command
from tests import pack_fixture
from tests.conftest import SECOND_GROUP, WATCHED_GROUP, envelope
@ -64,6 +65,7 @@ def full_registry() -> CommandRegistry:
fetcher = _offline_fetcher()
register_animal_commands(registry, fetcher)
register_pope_command(registry, PortraitCache(fetcher, DEFAULT_COMMONS_API_URL))
register_trap_command(registry, fetcher, DEFAULT_COMMONS_API_URL, traps=(1,))
register_stats_commands(registry)
register_suggest_command(registry)
register_help_command(registry)

191
tests/test_traps_command.py Normal file
View File

@ -0,0 +1,191 @@
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""The !pułapka command: a photograph of the device, with nothing attached."""
from __future__ import annotations
import base64
import json
from typing import TYPE_CHECKING
import httpx
import pytest
from signal_bot.context import CommandContext
from signal_bot.decorators import DISPLAY_MARGIN_SECONDS, cooldown
from signal_bot.http_client import ImageFetcher
from signal_bot.models import parse_envelope
from signal_bot.registry import CommandRefusedError, CommandRegistry
from signal_bot.traps_command import (
COMMAND_NAME,
TRAPS_FILE,
load_traps,
random_trap,
register_trap_command,
)
from tests.conftest import WATCHED_GROUP, envelope
if TYPE_CHECKING:
from pathlib import Path
from signal_bot.config import Config
from signal_bot.store import Store
from signal_bot.transport_fake import FakeTransport
IMAGE = b"trap-bytes"
API = "https://commons.example/w/api.php"
ALIASES = {COMMAND_NAME: ["!pułapka", "!pulapka", "!trap"]}
POOL = (111419226, 124481655, 151918804)
def _fetcher(seen: list[httpx.URL] | None = None) -> ImageFetcher:
def handler(request: httpx.Request) -> httpx.Response:
if "commons" in str(request.url):
if seen is not None:
seen.append(request.url)
page = {
"title": "File:Molchreuse.JPG",
"imageinfo": [
{
"thumburl": "https://cdn/trap.jpg",
"descriptionurl": "https://commons.wikimedia.org/wiki/File:Molchreuse.JPG",
"extmetadata": {
"ImageDescription": {"value": "Eine <b>Molchreuse</b>"},
},
}
],
}
return httpx.Response(200, json={"query": {"pages": [page]}})
return httpx.Response(200, content=IMAGE)
return ImageFetcher(httpx.AsyncClient(transport=httpx.MockTransport(handler)))
def _ctx(transport: FakeTransport, store: Store, config: Config, text: str) -> CommandContext:
message = parse_envelope(envelope(text=text))
assert message is not None
return CommandContext(
message=message,
group_id=WATCHED_GROUP,
transport=transport,
store=store,
config=config,
command_name=COMMAND_NAME,
now=100.0,
day="2026-08-31",
)
@pytest.mark.parametrize("trigger", ["!pułapka", "!pulapka", "!trap"])
def test_every_alias_resolves(trigger: str) -> None:
registry = CommandRegistry()
register_trap_command(registry, _fetcher(), API, aliases=ALIASES, traps=POOL)
command = registry.resolve(trigger)
assert command is not None
assert command.name == COMMAND_NAME
def test_the_real_alias_file_carries_the_command() -> None:
registry = CommandRegistry()
register_trap_command(registry, _fetcher(), API, traps=POOL)
# The Polish spelling is the primary one !pomoc prints; the English
# !trap is the word that prompted the command in the first place.
assert registry.resolve("!pułapka") is not None
assert registry.resolve("!trap") is not None
async def test_it_posts_an_image_captioned_with_what_the_thing_is(
transport: FakeTransport, store: Store, config: Config
) -> None:
registry = CommandRegistry()
register_trap_command(registry, _fetcher(), API, aliases=ALIASES, traps=POOL)
command = registry.resolve("!trap")
assert command is not None
await command.handler(_ctx(transport, store, config, "!trap"))
sent = transport.sent[0]
assert sent.recipient == WATCHED_GROUP
assert sent.image_base64 is not None
assert base64.b64decode(sent.image_base64) == IMAGE
assert len(transport.sent) == 1
# Commons is full of files whose subject is unguessable, so the caption
# says what it is and links the page that says more. Tags come out.
assert sent.caption == (
"Eine Molchreuse\nhttps://commons.wikimedia.org/wiki/File:Molchreuse.JPG"
)
async def test_it_names_the_file_by_page_id_and_never_by_title(
transport: FakeTransport, store: Store, config: Config
) -> None:
# A title would be free text carrying people's names into the repository;
# see load_traps. This is the assertion that keeps the query on page ids.
seen: list[httpx.URL] = []
registry = CommandRegistry()
register_trap_command(registry, _fetcher(seen), API, aliases=ALIASES, traps=POOL)
command = registry.resolve("!trap")
assert command is not None
await command.handler(_ctx(transport, store, config, "!trap"))
assert len(seen) == 1
assert seen[0].params["pageids"] in {str(page) for page in POOL}
assert "titles" not in seen[0].params
async def test_the_cooldown_decorator_applies(
transport: FakeTransport, store: Store, config: Config
) -> None:
registry = CommandRegistry()
register_trap_command(
registry,
_fetcher(),
API,
decorate=cooldown(10, "WAIT {seconds}"),
aliases=ALIASES,
traps=POOL,
)
store.record_use(WATCHED_GROUP, "uuid-alice", COMMAND_NAME, 100.0, "2026-08-31")
command = registry.resolve("!trap")
assert command is not None
with pytest.raises(CommandRefusedError):
await command.handler(_ctx(transport, store, config, "!trap"))
assert transport.texts == [f"WAIT {10 + DISPLAY_MARGIN_SECONDS}"]
def test_the_draw_actually_varies() -> None:
"""0.0.3 posted one identical gecko forever; that is what this catches."""
drawn = {random_trap(POOL) for _ in range(20)}
assert len(drawn) > 1
def test_the_shipped_pool_is_page_ids_and_nothing_else() -> None:
"""The personal-data invariant, as a unit test.
A filename here would carry photographers' and sitters' names into a
public repository, and one of them colliding with a label somebody minted
from the chat wedges check_personal_data.py permanently. Integers cannot.
"""
raw = json.loads(TRAPS_FILE.read_text(encoding="utf-8"))["traps"]
assert raw
assert all(isinstance(page, int) for page in raw)
assert len(set(raw)) == len(raw)
assert raw == sorted(raw)
def test_the_shipped_pool_is_large_enough_to_be_worth_drawing_from() -> None:
# kuhy asked for a set "as big as possible", and a regeneration that
# silently truncated would still pass every other test here.
assert len(load_traps()) >= 500
@pytest.mark.parametrize("traps", [[], ["111419226.jpg"], [None]])
def test_a_pool_that_is_not_page_ids_is_refused(tmp_path: Path, traps: list[object]) -> None:
path = tmp_path / "traps.json"
path.write_text(json.dumps({"traps": traps}), encoding="utf-8")
with pytest.raises(ValueError, match="page ids"):
load_traps(path)

216
tool/gen_traps.py Executable file
View File

@ -0,0 +1,216 @@
#!/usr/bin/env python3
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Regenerate ``signal_bot/traps.json`` from Wikimedia Commons.
Run this by hand; the output is committed, so the bot never searches Commons
at the moment somebody types ``!pułapka``.
**The file holds page ids, not filenames, and that is load-bearing.** Commons
filenames are free text carrying people's names -- the harvest turns up things
like ``"Fishing-Boat with Fishermen" - Vasileios Chatzis.jpg``. Committing two
thousand of those would eventually collide with a label somebody minted from
the chat with ``!add_photo``, and ``scripts/check_personal_data.py`` would then
refuse every commit for good, over data nobody can edit out by hand. An
integer cannot carry a name. ``prop=imageinfo`` accepts ``pageids=`` and puts
``thumburl`` at the same place ``titles=`` does, so nothing downstream cares.
Depth is one, and that is also measured: recursing three levels from
*Category:Fish traps* reaches Alfred Sisley's net paintings and Hawaiian
fisherman watercolours. Immediate subcategories only.
"""
from __future__ import annotations
import argparse
import json
import sys
import time
from pathlib import Path
from typing import Any, Final
import httpx
CATEGORIES_FILE: Final = Path(__file__).with_name("trap_categories.json")
API: Final = "https://commons.wikimedia.org/w/api.php"
# Wikimedia's policy requires a descriptive agent with a contact URL; the
# generic client default is rate-limited hard and may be blocked outright.
USER_AGENT: Final = "signal-bot traps generator (https://github.com/kuhyx/signal-bot)"
# The extension is what tells Signal how to render the attachment, so an
# unknown one is dropped rather than guessed. Same set as portraits.SUFFIXES.
SUFFIXES: Final = frozenset({".jpg", ".jpeg", ".png", ".gif"})
# Commons hosts non-free content too. !gekon filters licences at its query and
# endpoints.py calls that filter load-bearing; this is the same idea, on the
# only field Commons exposes for it. Matched case-folded, by prefix.
LICENCES: Final = ("public domain", "cc0", "cc by", "cc-by", "no restrictions")
# imageinfo takes fifty page ids per request, joined with "|". A comma is
# rejected outright ("badinteger"), which is worth writing down because the
# error names the parameter rather than the separator.
BATCH: Final = 50
# A harvest smaller than this is a truncated one, and committing it would
# quietly shrink the command's whole world.
MINIMUM: Final = 500
TOO_MANY_REQUESTS: Final = 429
RETRIES: Final = 6
PAUSE: Final = 0.4
def categories() -> tuple[tuple[str, ...], frozenset[str], tuple[str, ...]]:
"""Return the seeds, the pruned set and the pruned patterns."""
document = json.loads(CATEGORIES_FILE.read_text(encoding="utf-8"))
return (
tuple(document["seeds"]),
frozenset(document["excluded"]),
tuple(document["prune_patterns"]),
)
def _get(params: dict[str, str]) -> dict[str, Any]:
"""One API call, retried with a widening pause when Commons says slow down.
Commons answers 429 after a few hundred requests even at a steady 0.4s,
and a harvest that silently loses a category is worse than a slow one.
"""
query = {"action": "query", "format": "json", "formatversion": "2", **params}
for attempt in range(RETRIES):
time.sleep(PAUSE)
response = httpx.get(API, params=query, headers={"User-Agent": USER_AGENT}, timeout=60.0)
if response.status_code == TOO_MANY_REQUESTS:
time.sleep(2.0**attempt)
continue
response.raise_for_status()
document: dict[str, Any] = response.json()
return document
msg = f"Commons kept answering {TOO_MANY_REQUESTS} for {params}"
raise RuntimeError(msg)
def _members(category: str, kind: str) -> list[dict[str, Any]]:
"""Every member of one category, following continuations."""
found: list[dict[str, Any]] = []
carry_on: str | None = None
while True:
params = {
"list": "categorymembers",
"cmtitle": category,
"cmtype": kind,
"cmlimit": "500",
}
if carry_on is not None:
params["cmcontinue"] = carry_on
document = _get(params)
found += document.get("query", {}).get("categorymembers", [])
carry_on = document.get("continue", {}).get("cmcontinue")
if carry_on is None:
return found
def verify_seeds(seeds: tuple[str, ...]) -> None:
"""Fail before harvesting if any seed category does not exist.
A category that is not there is not an error to Commons -- it is an empty
answer -- so a plausible-looking typo removes a whole subject silently.
Five of the first draft's names were wrong this way.
Raises:
RuntimeError: If any seed is missing, naming all of them at once.
"""
missing: list[str] = []
for start in range(0, len(seeds), BATCH):
document = _get({"titles": "|".join(seeds[start : start + BATCH])})
missing += [
str(page["title"])
for page in document.get("query", {}).get("pages", [])
if page.get("missing")
]
if missing:
msg = f"these seed categories do not exist on Commons: {sorted(missing)}"
raise RuntimeError(msg)
def harvest() -> dict[int, str]:
"""Page id to title, for every usable file one step from the seeds."""
seeds, excluded, patterns = categories()
verify_seeds(seeds)
files: dict[int, str] = {}
seen: set[str] = set(excluded)
for number, seed in enumerate(seeds, start=1):
sys.stderr.write(f"[{number}/{len(seeds)}] {seed} ({len(files)} files so far)\n")
sys.stderr.flush()
branch = [seed, *(sub["title"] for sub in _members(seed, "subcat"))]
for category in branch:
if category in seen or any(pattern in category for pattern in patterns):
continue
seen.add(category)
for entry in _members(category, "file"):
title = str(entry["title"])
if Path(title).suffix.lower() in SUFFIXES:
files[int(entry["pageid"])] = title
return files
def licensed(files: dict[int, str]) -> dict[int, str]:
"""Drop everything whose licence does not permit reposting it."""
kept: dict[int, str] = {}
ids = sorted(files)
sys.stderr.write(f"checking licences on {len(ids)} files\n")
sys.stderr.flush()
for start in range(0, len(ids), BATCH):
chunk = ids[start : start + BATCH]
document = _get(
{
"prop": "imageinfo",
"iiprop": "extmetadata",
"iiextmetadatafilter": "LicenseShortName",
"pageids": "|".join(str(page) for page in chunk),
}
)
for page in document.get("query", {}).get("pages", []):
info = page.get("imageinfo") or [{}]
short = info[0].get("extmetadata", {}).get("LicenseShortName", {}).get("value", "")
name = str(short).casefold()
if name.startswith(LICENCES):
kept[int(page["pageid"])] = files[int(page["pageid"])]
return kept
def _rejection(traps: dict[int, str]) -> str:
"""Return why this harvest must not be written, or an empty string."""
if len(traps) < MINIMUM:
return f"got {len(traps)} traps, expected at least {MINIMUM}"
return ""
def main() -> int:
"""Write traps.json, refusing to write a harvest that failed its own checks."""
parser = argparse.ArgumentParser(description="Regenerate signal_bot/traps.json.")
parser.add_argument(
"--titles",
action="store_true",
help="print every kept title; the committed file is integers, so this is the review",
)
arguments = parser.parse_args()
traps = licensed(harvest())
rejection = _rejection(traps)
if rejection:
sys.stderr.write(f"refusing to write: {rejection}\n")
return 1
if arguments.titles:
for page in sorted(traps):
sys.stdout.write(f"{page}\t{traps[page]}\n")
out = Path(__file__).resolve().parent.parent / "signal_bot" / "traps.json"
document = {"source": str(CATEGORIES_FILE.name), "traps": sorted(traps)}
out.write_text(json.dumps(document, ensure_ascii=False, indent=1) + "\n", encoding="utf-8")
sys.stdout.write(f"wrote {len(traps)} traps to {out}\n")
return 0
if __name__ == "__main__":
raise SystemExit(main())

58
tool/trap_categories.json Normal file
View File

@ -0,0 +1,58 @@
{
"_comment": "Which Commons categories !pulapka draws from. Data, not source, for the reason aliases.json is: changing the policy should never mean touching Python. Every seed is checked against Commons before the harvest starts -- Snares, Man traps, Hunting traps, Wasp traps and Animal trapping all look like plausible names and none of them exists, so a typo would otherwise remove a whole subject in silence. by-function and by-animal are container categories whose immediate children hold the leg-hold, deadfall and per-species traps.",
"_excluded_comment": "Pruned as whole subtrees, never enqueued and never read. The command posts an attachment with no caption into a group chat, so this list is the only place the content of that attachment can be decided at all. Camera traps are photographs taken *by* a trap and drown the pool; booby traps reach wartime propaganda including a poster whose title carries a racial slur; the Wolfsangel is a heraldic charge that is also a neo-Nazi symbol; the mice categories are animals dying; the rest are geology, botany, paintings, architecture and bait rather than devices.",
"seeds": [
"Category:Traps",
"Category:Traps by function",
"Category:Traps by animal",
"Category:Mousetraps",
"Category:Rat traps",
"Category:Mole traps",
"Category:Bear traps",
"Category:Leghold traps",
"Category:Snare traps",
"Category:Cage traps",
"Category:Live capture traps",
"Category:Pitfall traps",
"Category:Trapping pits",
"Category:Deadfall traps",
"Category:Mantraps",
"Category:Fish traps",
"Category:Lobster traps",
"Category:Crab traps",
"Category:Eel traps",
"Category:Fyke nets",
"Category:Fly traps",
"Category:Flypaper",
"Category:Insect traps"
],
"excluded": [
"Category:Animals in traps",
"Category:Baits",
"Category:Booby traps",
"Category:Booby traps in the Vietnam War",
"Category:Camera traps",
"Category:Carnivorous plants with prey",
"Category:Fishing bait",
"Category:Fixed gillnets",
"Category:Glue traps",
"Category:Mammals in traps",
"Category:Mice in cage traps",
"Category:Mice in glue mousetraps",
"Category:Mice in spring-loaded bar traps",
"Category:Petroleum traps",
"Category:Rodent bait stations",
"Category:Tar baby",
"Category:Trapdoors",
"Category:Traps in art",
"Category:Traps in heraldry",
"Category:Traps of carnivorous plants",
"Category:Videos of mice in mousetraps",
"Category:Wolfsangel"
],
"prune_patterns": [
" in art",
" in heraldry"
],
"_prune_comment": "Any subcategory whose title contains one of these is pruned wholesale. Measured: Category:Mousetraps in art put a Robert Campin altarpiece panel and an 18th-century oil of a street vendor into the pool, and both arrive with no caption to explain why a painting is the answer to !pulapka. A pattern rather than a list because Commons mints a new '<thing> in art' category whenever somebody needs one."
}