mirror of
https://github.com/kuhyx/signal-bot.git
synced 2026-11-11 12:57:32 +01:00
Krzysztof kuhy Rudnicki
08599e6056
!sztuka posts a work of art, skewed 60% Polish; !wiersz posts one short public-domain Polish poem, whole. Both follow the rule the rest of the bot already keeps: the repository commits identifiers, and the words arrive over the wire when somebody asks. !wiersz was written the other way first, and the gate caught it. A pack of short public-domain poems was harvested, name-filtered and committed, and check_personal_data.sh refused it: three lines carried a real group member's given name, and two still did after a filter that had discarded 31% of the corpus. That is structural, not a tuning problem -- Polish verse capitalises every line, so a line-initial capital carries no signal, and that is exactly where a name sits. The pack now holds Wolne Lektury slugs; the poem, its title and its author are fetched at post time and appear nowhere in git. It is also the better version: with no name filter there are no false positives, so the pool is larger and the attribution is a full name again. The gate then refused the art category list too, over a painter whose given name is somebody's. That is the popes.json case exactly, so it takes the same answer: a file-scoped allowlist entry, still refused everywhere else. The Commons harvest is now three modules by what they are, not by where the line count fell: commons_api is the conversation with Commons, commons_harvest turns categories into a pool of licensed page ids, commons_cache makes an interrupted run resumable. Along the way it gained licences in the same request that lists a category (the second pass was 1,500 requests at 2.3s each), maxlag instead of a guessed sleep, 5xx retries because one 504 used to destroy a whole run, and Wikidata for works by creators of Polish citizenship: the Polish pool grew 15,221 -> 20,092. Batching categories through CirrusSearch looked like a further 6x and was measured out again: 2,404 files where the authoritative enumeration returns 2,429, a different ~1% missing on every run. The offline run is 19s where it was four minutes, almost all of it the deafness gate waiting out three thirty-second liveness probes to prove something provable in three; the tick is configuration now, and e2e enforces a 40s budget on itself so that never silently comes back.
121 lines
6.3 KiB
JSON
121 lines
6.3 KiB
JSON
{
|
|
"_comment": "Which Commons categories !sztuka draws from, in two pools. Data, not source, for the reason aliases.json is: changing the policy should never mean touching Python. Every seed below was checked against Commons before being written down, and gen_art.py checks them all again before each harvest -- fourteen names in the first draft (Polish paintings, Polish sculptures, Paintings by Polish painters, Polish drawings, Polish prints, Polish posters, Drawings by Polish artists, Sculptures by Polish sculptors, Polish decorative arts, Stained glass windows in Poland, Old master prints, Textile art, Icons (religion), Witkacy) all look entirely plausible and none of them exists, so a typo would otherwise remove a whole artist or medium in silence.",
|
|
"_pools_comment": "The pools are separate because the draw is weighted, not merged: art_command.POLISH_SHARE per cent of posts come from 'polish' whatever the two sizes are. They are also walked disjointly -- a category reached from a Polish seed is never walked again from a world seed, and any file that still turns up in both is subtracted from the world pool -- so no work can sit in both and be drawn at both odds. The consequence worth knowing: 'polish' means reachable from a Polish seed, not painted by a Pole. A Matejko filed only under Category:Portrait paintings and not under his own category counts as world art, so the real Polish share of the corpus is a little higher than POLISH_SHARE makes the output.",
|
|
"_polish_comment": "Museums, movements and media first, then artists whose own category is the granular unit Commons actually files paintings under. Bacciarelli and Bellotto were Italians who spent their careers painting in Warsaw, and belong to Polish art history for that reason rather than by birth.",
|
|
"polish": [
|
|
"Category:Art in Poland",
|
|
"Category:Art of Poland",
|
|
"Category:Paintings in Poland",
|
|
"Category:Paintings from Poland",
|
|
"Category:Paintings of Poland",
|
|
"Category:17th-century paintings from Poland",
|
|
"Category:18th-century paintings from Poland",
|
|
"Category:19th-century paintings from Poland",
|
|
"Category:20th-century paintings from Poland",
|
|
"Category:Paintings in the National Museum in Warsaw",
|
|
"Category:Paintings in the National Museum in Kraków",
|
|
"Category:Paintings in the National Museum in Gdańsk",
|
|
"Category:Paintings in the National Museum in Poznań",
|
|
"Category:Paintings in the National Museum in Wrocław",
|
|
"Category:Sculptures in Poland",
|
|
"Category:Sculptures from Poland",
|
|
"Category:Sculptures in Warsaw",
|
|
"Category:Sculptures in Kraków",
|
|
"Category:Drawings from Poland",
|
|
"Category:Prints from Poland",
|
|
"Category:Stained glass in Poland",
|
|
"Category:Folk art in Poland",
|
|
"Category:Baroque art in Poland",
|
|
"Category:Art Nouveau in Poland",
|
|
"Category:Young Poland",
|
|
"Category:Artists from Poland",
|
|
"Category:Painters from Poland",
|
|
"Category:Sculptors from Poland",
|
|
"Category:Jan Matejko",
|
|
"Category:Józef Chełmoński",
|
|
"Category:Jacek Malczewski",
|
|
"Category:Stanisław Wyspiański",
|
|
"Category:Aleksander Gierymski",
|
|
"Category:Maksymilian Gierymski",
|
|
"Category:Julian Fałat",
|
|
"Category:Leon Wyczółkowski",
|
|
"Category:Olga Boznańska",
|
|
"Category:Władysław Podkowiński",
|
|
"Category:Józef Brandt",
|
|
"Category:Henryk Siemiradzki",
|
|
"Category:Wojciech Kossak",
|
|
"Category:Juliusz Kossak",
|
|
"Category:Piotr Michałowski",
|
|
"Category:Tadeusz Makowski",
|
|
"Category:Zofia Stryjeńska",
|
|
"Category:Teodor Axentowicz",
|
|
"Category:Wlastimil Hofman",
|
|
"Category:Xawery Dunikowski",
|
|
"Category:Jerzy Nowosielski",
|
|
"Category:Zdzisław Beksiński",
|
|
"Category:Tamara de Lempicka",
|
|
"Category:Marcello Bacciarelli",
|
|
"Category:Bernardo Bellotto"
|
|
],
|
|
"_world_comment": "Media before movements before museums, so that the pool is not simply European oil painting: engraving, lithograph, woodcut, fresco, mosaic, tapestry, ceramic, icon, relief, illuminated manuscript, calligraphy and ukiyo-e are all in it.",
|
|
"world": [
|
|
"Category:Oil paintings",
|
|
"Category:Drawings",
|
|
"Category:Engravings",
|
|
"Category:Lithographs",
|
|
"Category:Etchings",
|
|
"Category:Woodcuts",
|
|
"Category:Watercolor paintings",
|
|
"Category:Pastels",
|
|
"Category:Frescos",
|
|
"Category:Mosaics",
|
|
"Category:Tapestries",
|
|
"Category:Ceramics",
|
|
"Category:Calligraphy",
|
|
"Category:Icons",
|
|
"Category:Illuminated manuscripts",
|
|
"Category:Ukiyo-e",
|
|
"Category:Art Nouveau posters",
|
|
"Category:Bronze sculptures",
|
|
"Category:Marble sculptures",
|
|
"Category:Terracotta sculptures",
|
|
"Category:Reliefs",
|
|
"Category:Statues",
|
|
"Category:Busts",
|
|
"Category:Altarpieces",
|
|
"Category:Triptychs",
|
|
"Category:Portrait paintings",
|
|
"Category:Self-portraits",
|
|
"Category:Landscape paintings",
|
|
"Category:Genre paintings",
|
|
"Category:History paintings",
|
|
"Category:Religious paintings",
|
|
"Category:Renaissance paintings",
|
|
"Category:Baroque paintings",
|
|
"Category:Impressionist paintings",
|
|
"Category:Post-Impressionist paintings",
|
|
"Category:Symbolist paintings",
|
|
"Category:Expressionist paintings",
|
|
"Category:Paintings in the Louvre",
|
|
"Category:Paintings in the Metropolitan Museum of Art",
|
|
"Category:Paintings in the National Gallery, London",
|
|
"Category:Paintings in the Uffizi Gallery"
|
|
],
|
|
"_excluded_comment": "Pruned as whole subtrees, never enqueued and never read. Book illustrations is not objectionable, it is simply eleven thousand five hundred files -- more than every other world seed put together -- and a uniform draw would turn !sztuka into a book-illustration command. Erotic art is excluded for the reason the trap generator excludes booby traps: the command posts an attachment into a group chat and this list is the only place its content can be decided at all.",
|
|
"excluded": [
|
|
"Category:Book illustrations",
|
|
"Category:Erotic art"
|
|
],
|
|
"prune_patterns": [
|
|
"Erotic",
|
|
"Nude",
|
|
"Pornograph",
|
|
"Photographs of",
|
|
"Graves of",
|
|
"Tombs of",
|
|
"Signatures of",
|
|
"Graffiti"
|
|
],
|
|
"_prune_comment": "Any subcategory whose title contains one of these is pruned wholesale. A pattern rather than a list because Commons mints a new '<thing>' category whenever somebody needs one. 'Photographs of' is here because an artist's own category holds photographs of the artist as well as their work, and a portrait photograph of a sculptor is not a sculpture; the same rule drops the graves, tombs and signatures those categories also collect."
|
|
}
|