signal-bot/signal_bot/commons.py
Krzysztof kuhy Rudnicki 14d422295f feat: a word with two meanings gets the one made of steel
!pułapka (and !trap, !sidła, !wnyk, !potrzask, 26 spellings in all) posts a
photograph of the literal device -- a bear trap, a mousetrap, a snare, a fish
weir -- captioned with what Commons says it is.

The pool is 2218 Commons page ids, harvested by tool/gen_traps.py from a
curated seed list and committed. Page ids and not filenames on purpose: a
filename carries photographers' and sitters' names, and one of those
colliding with a label somebody mints from the chat would wedge
check_personal_data.py permanently, over data nobody can edit by hand.
_repo_vocabulary() subtracts aliases.json and nothing else, so it would not
have caught it. An integer cannot carry a name.

Nothing is cached. !papież caches because 267 portraits are a closed set a
group works through; a uniform draw from 2218 warms a cache essentially
never, so this fetches per call the way !kot and !pies do.

What the harvest measured, and now enforces:

- five plausible seed categories do not exist (Snares, Man traps, Hunting
  traps, Wasp traps, Animal trapping). A missing category is an empty answer,
  not an error, so verify_seeds() asks Commons before harvesting and fails
  loudly naming all of them.
- depth three from Category:Fish traps reaches Alfred Sisley's net paintings.
  Depth one, from seeds picked by hand.
- "<thing> in art" categories are paintings: Mousetraps in art put a Robert
  Campin altarpiece in the pool. Pruned by pattern, since Commons mints a new
  one whenever somebody needs it. Booby traps, the Wolfsangel, camera traps
  and animals dying in traps are pruned by name.
- Commons rate-limits hard, so a partial harvest must not be committed
  silently: 429 backs off, and a short pool refuses to write at all.

Descriptions are HTML and sometimes say nothing, so the tags come out, the
Fisheries records' trailing catalogue list is cut, a description naming only
the country falls back to the filename, and the file's own page is always
linked.

!łapka is not an alias: it resolves to !papa at distance two, which
test_nearest.py fails on, and rightly -- the suggestion would be a coin toss.

signal_bot/commons.py is the imageinfo query both Commons commands now share;
portraits.py keeps its behaviour and its tests unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01416TegJ3sa8xL5qGGgrygP
2026-08-31 18:01:27 +02:00

127 lines
5.0 KiB
Python

# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""What a Wikimedia Commons image fetch needs, shared by its two callers.
`!papież` names a file by title and caches the result; `!pułapka` names one by
page id and does not cache anything. Everything between those two ends -- the
query, the width, the place `thumburl` turns up in the answer -- is identical,
so it lives here rather than twice.
"""
from __future__ import annotations
import html
import re
from typing import Any, Final
from signal_bot.http_client import ImageFetchError
# Where imageinfo puts the thumbnail URL in its reply. The same path whether
# the file was named by title or by page id.
THUMB_PATH: Final = ("query", "pages", 0, "imageinfo", 0, "thumburl")
# Commons hosts these four for the 267 portraits: 245 jpg, 15 png, 6 gif,
# 1 jpeg. An unknown suffix is refused rather than guessed, because the
# extension is what tells Signal how to render the attachment.
SUFFIXES: Final = frozenset({".jpg", ".jpeg", ".png", ".gif"})
# Long enough to say what the thing is, short enough not to bury the photo.
CAPTION_LIMIT: Final = 220
_TAGS: Final = re.compile(r"<[^>]+>")
# The Fisheries records put their prose first and then an HTML list of
# catalogue fields ("Subject: Fishing nets", "Tag: Fisheries Techniques").
# Cutting at the list keeps the sentence and drops the filing cabinet.
_LIST: Final = "<ul"
# Commons templates that fill the description field with something that is
# not a description. Measured on a hand-woven Maltese fish trap: its whole
# description named only the country, where the filename said what the
# object was. Treated as no description at all.
_BOILERPLATE: Final = ("this photo has been taken in the country",)
def imageinfo_params(
width: int,
*,
title: str | None = None,
page_id: int | None = None,
describe: bool = False,
) -> dict[str, str]:
"""Build the imageinfo query for one Commons file.
Args:
width: Thumbnail width to ask for, in pixels.
title: Full page title, "File:" prefix included.
page_id: Commons page id. Preferred where the caller stores a list,
because an integer cannot carry somebody's name into the
repository the way a filename can.
describe: Also ask for the file's description, so one request
answers both what the picture is and where it lives.
Returns:
Query parameters, ready for ImageFetcher.fetch_bytes.
Raises:
ValueError: If both or neither of title and page_id were given.
Silently preferring one would make a caller's typo look like a
missing file, which is the same shape as a real 404.
"""
if (title is None) == (page_id is None):
msg = "name a Commons file by exactly one of title or page_id"
raise ValueError(msg)
selector = {"titles": title} if title is not None else {"pageids": str(page_id)}
described = (
{"iiprop": "url|extmetadata", "iiextmetadatafilter": "ImageDescription"}
if describe
else {"iiprop": "url"}
)
return {
"action": "query",
"format": "json",
"formatversion": "2",
"prop": "imageinfo",
"iiurlwidth": str(width),
**described,
**selector,
}
def page_of(document: object) -> dict[str, Any]:
"""Return the single page an imageinfo answer describes.
Raises:
ImageFetchError: If Commons answered without one -- a deleted or
renumbered page id comes back as `{"missing": true}` and has no
imageinfo at all.
"""
pages = document.get("query", {}).get("pages", []) if isinstance(document, dict) else []
if not pages or not pages[0].get("imageinfo"):
msg = f"Commons named no image: {document!r:.200}"
raise ImageFetchError(msg)
page: dict[str, Any] = pages[0]
return page
def caption_for(page: dict[str, Any]) -> str:
"""Return the line that rides along with the photograph.
Commons descriptions are HTML, sometimes several languages deep, and
sometimes only the filename repeated back. So the tags come out, the
catalogue list is cut, and a description that says nothing falls back to
the title -- but the file's own page is always linked, because that is
what answers "what am I even looking at" when the rest does not.
The text is fetched at run time and posted to the group; none of it is
ever written into this repository, which is why it may be free text
where traps.json may not.
"""
info = page["imageinfo"][0]
raw = str(info.get("extmetadata", {}).get("ImageDescription", {}).get("value", ""))
text = " ".join(html.unescape(_TAGS.sub(" ", raw.split(_LIST, 1)[0])).split())
if text.lower().startswith(_BOILERPLATE):
text = ""
if not text:
text = str(page.get("title", "")).removeprefix("File:").rsplit(".", 1)[0]
if len(text) > CAPTION_LIMIT:
text = text[:CAPTION_LIMIT].rstrip() + "…"
url = str(info.get("descriptionurl", ""))
return f"{text}\n{url}" if url else text