mirror of
https://github.com/kuhyx/testsAndMisc.git
synced 2026-11-10 20:07:04 +01:00
Krzysztof kuhy Rudnicki
da8ca6d827
Adds the route-independent spine of the planned art-generation bake-off: a harness that decides whether generated art is well-formed, so "non-sloppy" becomes machine-checkable instead of a judgement call. Findings that shaped the design, all measured rather than assumed: - Integer-scale invariance separates true pixel art (impurity 0.0000) from interpolated fakes (1.0000) exactly -- but passes trivially at scale 1, so it must never be trusted alone on native-resolution output. - Gate ORDER is load-bearing. A soft-alpha image has no fully-opaque pixels, so an opaque-only colour count returns 0 and silently passes. ALPHA_NOT_BINARY now runs first and COLOR_COUNT fails closed on an empty opaque set. - Counting must be over opaque RGB only: transparent pixels retain arbitrary RGB and inflated a naive RGBA count, rejecting correct art at 18/16. - Declaring a gate was not enough to run it. hand_painted.toml shipped declaring SILHOUETTE_LOW_CONTRAST with no implementation, so it parsed, ran nothing, and reported a pass. load_spec now fails closed on unimplemented gates, result-only codes, and gates missing their config. Thresholds are calibrated against 250 genuine 32x32 CC0 icons, not guessed: p50=7 p90=13 p99=26 opaque colours; cap 32 rejects 0.0% of real art while raw generator output measured 34403 colours. An initial cap of 16 wrongly rejected a real DCSS icon -- re-derived from the corpus rather than nudged. Verified: 57 tests, 100% branch coverage, zero suppressions, all hooks pass. 58/60 (96.7%) of real CC0 icons pass the full stack; both failures confirmed true positives. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T9rQ12yZ7sJxiPQTCS5dUF
77 lines
6.9 KiB
JSON
77 lines
6.9 KiB
JSON
{
|
|
"intent": "Add `artgate`, a deterministic exit-code-adjudicated gate harness that decides whether generated game art is well-formed, so 'non-sloppy' becomes a machine-checkable property instead of a judgement call. This is the route-independent spine of a planned 4-route x 4-style generation bake-off.",
|
|
"scope": [
|
|
"python_pkg/artgate/{codes,spec,pixels,cli}.py - new package",
|
|
"python_pkg/artgate/styles/{pixel_icon,hand_painted}.toml - calibrated per-style specs",
|
|
"python_pkg/artgate/tests/ - 42 tests, 100% branch coverage",
|
|
".pre-commit-config.yaml - numpy added to pylint's isolated env (first numpy user in repo)",
|
|
"NON-GOAL: no art is generated here; no route drivers, no diffusion code",
|
|
"NON-GOAL: no binaries added - all generated output stays outside the repo by design"
|
|
],
|
|
"changes": [
|
|
"Gate ordering is load-bearing and enforced: ALPHA_NOT_BINARY runs before TOO_MANY_COLORS. Discovered empirically - a soft-alpha image has zero fully-opaque pixels, so an opaque-only colour count returns 0 and silently passes. Two individually-correct gates composed into a hole.",
|
|
"COLOR_COUNT fails closed (NO_OPAQUE_PIXELS) rather than reporting 0 when nothing is opaque.",
|
|
"Colour counting is over opaque RGB only, with transparent pixels canonicalised. A naive RGBA count is inflated by transparent pixels retaining arbitrary RGB, which rejected correct art at 18/16 during design.",
|
|
"SCALE_NOT_INVARIANT documents its precondition: it passes trivially at scale 1, so it is only meaningful at integer upscale >= 2. On native-resolution output the alpha and colour gates are the discriminating ones.",
|
|
"Three-valued exit code (0 pass / 1 gate failure / 2 harness error) so a malformed spec is never recorded as an art rejection.",
|
|
"Gate applicability is data (per-style TOML), not branching logic, so palette/colour/scale gates are simply absent for hand-painted work instead of manufacturing false failures.",
|
|
"FAIL-CLOSED spec validation: declaring a gate is no longer enough to make it run. load_spec now rejects (a) gates with no implementation, (b) result-only codes like NO_OPAQUE_PIXELS declared as gates, and (c) gates declared without their required config. All map to exit 2 (spec fault), never to a silent pass.",
|
|
"Code vocabulary split into DECLARABLE (a spec may request) and RESULT_ONLY (a gate may emit), so the two can never be confused again.",
|
|
"Implemented SILHOUETTE_LOW_CONTRAST (contrast.py) - alpha-weighted Rec.709 luma separation against BOTH a light and a dark background. It is one of only three gates valid for every style, and Phase 3's cells depend on it.",
|
|
"test_shipped_styles_load asserts every style TOML in the package parses, so a spec declaring a non-existent gate cannot ship again."
|
|
],
|
|
"verification": [
|
|
{
|
|
"command": "python -m pytest python_pkg/artgate/tests/ --cov=python_pkg.artgate --cov-branch",
|
|
"result": "pass",
|
|
"evidence": "42 passed. cli.py 100%, codes.py 100%, pixels.py 100%, spec.py 100% branch coverage. Zero lint suppressions."
|
|
},
|
|
{
|
|
"command": "python -m python_pkg.artgate.cli <real CC0 icon> --spec styles/pixel_icon.toml",
|
|
"result": "pass",
|
|
"evidence": "Real hand-made DCSS rambutan.png -> PASS, exit 0. Same image blurred to simulate un-finished generator output -> FAIL ALPHA_NOT_BINARY,TOO_MANY_COLORS, exit 1. Missing spec -> exit 2."
|
|
},
|
|
{
|
|
"command": "batch gate over 60 random genuine 32x32 CC0 icons",
|
|
"result": "pass",
|
|
"evidence": "59/60 PASS (98.3%). The single failure is a genuinely soft-alpha asset, a correct rejection under a spec demanding binary alpha - not a false positive."
|
|
},
|
|
{
|
|
"command": "calibration sweep over 250 genuine 32x32 CC0 icons",
|
|
"result": "pass",
|
|
"evidence": "Opaque colour distribution p50=7 p75=10 p90=13 p95=14 p99=26 max=30. Rejection of real art: cap16 -> 3.6%, cap24 -> 1.2%, cap32 -> 0.0%. max_colors set to 32 from this data after an initial guess of 16 wrongly rejected a real icon."
|
|
},
|
|
{
|
|
"command": "pre-commit run --files python_pkg/artgate/*.py python_pkg/artgate/tests/*.py",
|
|
"result": "pass",
|
|
"evidence": "ALL HOOKS PASS: ruff (select=ALL), ruff-format, mypy strict, pylint, bandit, codespell, no-noqa, check-docstring-first, pytest+coverage."
|
|
},
|
|
{
|
|
"command": "fail-closed spec validation, four paths",
|
|
"result": "pass",
|
|
"evidence": "Unimplemented gate -> \"gate(s) ['SEAM_ENERGY'] are declared but not implemented\", exit 2. Gate without config -> \"TOO_MANY_COLORS needs 'max_colors'\", exit 2. Result-only code declared -> \"['NO_OPAQUE_PIXELS'] are emitted by gates, not declarable\", exit 2. Near-white 32x32 asset -> FAIL SILHOUETTE_LOW_CONTRAST, exit 1."
|
|
},
|
|
{
|
|
"command": "re-validate 60 real CC0 icons against the FULL gate stack incl. silhouette",
|
|
"result": "pass",
|
|
"evidence": "58/60 (96.7%) PASS. Both failures verified as true positives: one genuinely soft-alpha asset, and jelly_ambrosia.png which is 98.9% empty with pure-black content (silhouette delta 0.0 vs dark - actually invisible on a dark background)."
|
|
},
|
|
{
|
|
"command": "silhouette calibration controls",
|
|
"result": "pass",
|
|
"evidence": "near-white: delta 250.0 vs dark but 5.0 vs light -> FAIL (proves both backgrounds are required). mid-grey: 128.0/127.0 -> PASS. Threshold 12.0 sits between them."
|
|
}
|
|
],
|
|
"risks": [
|
|
"A ruff autofix silently deleted every print() (T201) from cli.py, leaving orphaned expression statements so the CLI emitted nothing while still returning correct exit codes. Caught by running the program, not by the tests - exit-code-only assertions would have passed. Rewritten to sys.stdout.write per repo convention and re-verified by execution.",
|
|
"A gate could be DECLARED in a spec without being IMPLEMENTED, so it parsed, ran nothing, and reported a pass. hand_painted.toml shipped in exactly that state, and a test had codified the fail-open as correct behaviour. Now rejected at load time; the offending test was inverted to assert the raise. Lesson: the fail-open was in the layer that decides WHICH gates run, not in any gate itself.",
|
|
"Thresholds are calibrated against this specific CC0 corpus. Art from a different source or resolution may need a re-derived cap; the rule is to re-run the calibration sweep, never to nudge the number until one file passes.",
|
|
"SCALE_NOT_INVARIANT must not be trusted alone on native-resolution assets - documented in the function's docstring and the plan.",
|
|
"Only pixel-style gates exist so far. Hand-painted and 3D are weakly gated by construction and will lean on the human contact-sheet gate."
|
|
],
|
|
"rollback": [
|
|
"Delete python_pkg/artgate/ and revert the numpy line in .pre-commit-config.yaml; nothing else imports it.",
|
|
"After rollback, confirm `pre-commit run --all-files` still passes and that no other package regressed - artgate is additive and has no runtime consumers yet."
|
|
]
|
|
}
|