Commit Graph

13 Commits

Author SHA1 Message Date
f15f00f3fe Re-point paths for the ~ reorganisation (home-tidy)
Also bumps the 16 meta/requirements pins the freshness gate flagged
(anthropic 1.5.0, numpy 2.5.3, pytest-randomly 5, flake8-bugbear 26.9.9,
ruff 0.16.7, pyright 1.1.414, selenium 4.49.0, hypothesis 6.168.0, lxml
6.1.3, pipdeptree 4.2.5, the two types-* stubs) and Flutter 3.47.3 in
meta/.fvmrc. One test still expected the pre-move ~/mcp-servers path.

The pre-commit hook fed .zsh files to shfmt and shellcheck, neither of
which parses zsh (SC2148/SC1071), so no zsh file could be committed since
the hook was tightened on 2026-08-22; those now get `zsh -n` instead. The
hook's own shfmt pass also staged its reformat of 8 shell files.

CI_GREEN_SKIP because main is red on exactly the dependency-freshness
finding this commit fixes, which is the documented use of that flag.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AwtowrowDXx6ccmWVPkCaF
2026-09-11 20:50:19 +02:00
adf2e402df Grade against 3.14 too, not just run on it
`requires-python` and the CI pin already said 3.14, but ruff, mypy and pylint
were still grading `meta/` against 3.10 -- three tools judging an interpreter
nothing runs any more, and ruff withholding the pyupgrade fixes that only
become safe at the newer target. `pin_ci_python.py` rewrites these three knobs.

What the stale target was hiding: `datetime.timezone.utc` where `datetime.UTC`
is now correct (UP017), and ruff's stable formatter applying PEP 758 --
`except (A, B):` becomes `except A, B:`. That makes the source 3.14-only at the
syntax level, which is what `requires-python = ">=3.14"` already promised; kuhy
chose it deliberately over holding ruff a release back (2026-08-29).

Also drops a comment in `wsg_grabber/models.py` that the bump makes false: it
explained why the file used ``str, Enum`` instead of ``StrEnum`` under mypy at
3.10, sitting directly above a ``class Verdict(StrEnum)`` that had already
stopped obeying it.

ruff, ruff-format, mypy --strict and pylint all clean at the new target.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SfnJmxxsZ534AKk16rC2Gr
2026-08-29 11:50:39 +02:00
4b2cd0c398 Clear the wsg_grabber test modules: python_pkg is now under the cap
The last six over-cap files split into eleven new modules, in two passes
since the first cut left five still over. python_pkg has no violations
left at all, down from 50 when this effort started.

Each split carries the whole preamble including the module's pytest
fixtures - started, toolkit and session are taken as parameters and
resolve lexically, so a hand-written import block would have missed
them. ruff then pruned 82 unused imports.

Pass count is 277 before and after, checked after both passes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011w2Pj4Ek3ZNMQVYcFHgXDY
2026-08-16 18:19:26 +02:00
e03fbd17e9 Split wsg_grabber/tests/test_review.py under the 250-line cap
Split along the same seams the source took: test_review_render.py covers what
now lives in review._render, test_review_undo.py covers review._undo.

The two fixture builders they share moved into the existing conftest.py rather
than a _helpers.py -- the name-tests-test hook requires everything else under
tests/ to be named test_*.py, and syncyomi_guard/tests/conftest.py is the
precedent for plain shared builders living there. Sharing beats copying: jscpd
fails the commit above 2% duplication and this repo already carries four partN
test files.

277 passed, the same count as before, so nothing was dropped. Two intermediate
runs caught real NameErrors from imports the extracted module needed.

test_review.py 297 -> 112, test_review_render.py 102, test_review_undo.py 79.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011w2Pj4Ek3ZNMQVYcFHgXDY
2026-08-16 16:24:48 +02:00
d86965ff3a Split wsg_grabber/ui.py under the 250-line cap
DisplayMixin in _display.py takes the methods that reach widgets only through
self._widgets: apply, _play, _verdict, on_undo, on_quit, drain, refresh, tick
and attach_player.

Widget construction stays in ui.py on purpose. test_ui.py fakes the toolkit
with patch.object(ui, "tk", fake), so moving _build out makes 22 tests error
with "module ui does not have the attribute 'tk'". An earlier attempt did that
and was reverted rather than touching the tests.

Also dropped the four _key_* adapters; each only discarded Tk's event argument
before forwarding, so _bind_keys binds lambdas directly now.

One thing worth knowing for the next split: ruff's TC001 is NOT suppressed by
__all__ the way F401 is. Moving Callbacks to its own module and re-exporting it
left a TYPE_CHECKING-only import that lints clean and raises AttributeError at
runtime -- cli.py constructs ui.Callbacks. Reverted; it stays defined in ui.py.

ui.py 398 -> 242, _display.py 179. 277 tests pass, both files 100% covered.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011w2Pj4Ek3ZNMQVYcFHgXDY
2026-08-16 16:18:13 +02:00
16c626661d Split wsg_grabber/player.py under the 250-line cap
The stateless half of talking to mpv moves out:

  _mpv_ipc.py    read_reply, build_argv, encode, note_errors, and
                 probe_properties/query lifted out of MpvPlayer.probe/_query,
                 which only ever needed the socket path
  _player_api.py the Player Protocol that ui.py actually depends on

player.py keeps the class that owns the process and the socket, and re-exports
Player/build_argv/encode through __all__.

_note_errors stays as a bound delegate on MpvPlayer: test_player.py calls it on
the instance, and the tests are out of scope here. Caught by the suite failing,
not by guessing.

The probe constants moved across verbatim -- _PROBE_ID is 9001, and read_reply
matches replies on it, so a placeholder would have made every probe hang until
timeout while still returning a plausible error string.

player.py 373 -> 235, _mpv_ipc.py 158, _player_api.py 49. 277 tests pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011w2Pj4Ek3ZNMQVYcFHgXDY
2026-08-16 16:05:41 +02:00
324ec0c9b5 Split wsg_grabber/downloader.py under the 250-line cap
Worker did two separable jobs. Both become mixins, because the tests drive
their methods as bound attributes (worker._download_one(), worker._fetch_thread)
and free functions would have broken that:

  _scan.py    ScanMixin      -- threads.json, the archive, one changed thread
  _fetch.py   DownloadMixin  -- claim a file, fetch it, record the outcome

_worker_base.py holds WorkerBase(ABC) with the state both halves touch and the
three methods Worker supplies. Two earlier shapes were abandoned rather than
suppressed: NotImplementedError stubs are uncovered lines under the 100% gate,
and `if TYPE_CHECKING` stubs satisfy mypy but leave pylint reporting no-member.
The ABC satisfies both with no disable comment anywhere.

_downloaded needs a real class-level default: _fetch does `self._downloaded += 1`
and pylint only accepts an attribute as assignable when there is something to
assign to. Worker.__init__ still owns the instance value.

downloader.py 420 -> 238, _scan.py 134, _fetch.py 98, _worker_base.py 52.
277 tests pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011w2Pj4Ek3ZNMQVYcFHgXDY
2026-08-16 16:01:11 +02:00
f58caea2ab Split wsg_grabber/store.py under the 250-line cap
Some checks are pending
Pre-commit checks / pre-commit (push) Waiting to run
Python tests / test (push) Waiting to run
Shell tests / shell-tests (push) Waiting to run
Shell tests / arch-tests (push) Waiting to run
store.py mixed the writes that move a file through its lifecycle with the
read-only projections over the same table. The reads move to _store_queries.py:
claimable_values, ready_items, ready_item, counts, pending_downloads,
known_md5s, state_of, attempts_for.

claim_next goes with them despite being a write. It is the one write that has
to be a single atomic transaction against exactly those reads, so splitting it
from them would have left BEGIN IMMEDIATE on one side of a module boundary and
its SELECT on the other.

_scalar becomes scalar: record_files still counts rows before and after its
insert, so store imports it back. The dependency runs one way only.

store.py re-exports through __all__, so downloader.py, cli.py and test_store.py
keep reaching everything at store.* unchanged -- all 15 names verified to
resolve after ruff's autofix pass.

store.py 382 -> 202, _store_queries.py 232.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011w2Pj4Ek3ZNMQVYcFHgXDY
2026-08-16 15:52:08 +02:00
262ef38efe Split wsg_grabber/net.py under the 250-line cap
net.py spoke HTTP for two very different things. The media transfer moves out:

  _download.py  Transfer, download, _stream, _over_budget, _write_body
  _reject.py    classify() -- status line and headers to an Outcome

net.py keeps build_session and get_json and re-exports the transfer API
through __all__, so downloader.py and test_net.py keep using net.download /
net.Transfer / net.digest_of / net.size_of untouched.

digest_of had to move down into _download rather than stay put: _stream needs
it, and importing it back up from net was a real runtime cycle. net re-exports
it. _rejection became classify because it is now imported across a module
boundary.

Note for the next split: ruff's F401 autofix stripped 11 dead imports here in
one pass. __all__ is what stops it doing the same to a re-export.

net.py 335 -> 109, _download.py 205, _reject.py 96.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011w2Pj4Ek3ZNMQVYcFHgXDY
2026-08-16 15:45:39 +02:00
62ad1484a6 Split wsg_grabber/review.py under the 250-line cap
review.py held three separable concerns. Two move out:

  _render.py  emptiness, status_line, filename_line, title_line, render --
              pure state-to-text, no toolkit and no transitions
  _undo.py    can_undo, on_undo, forget_last -- correcting the tally and
              putting the on-screen video back at the front of the queue

review.py keeps the state transitions and re-exports all eight names through
__all__, so ui.py and test_review.py reach them at review.* exactly as before;
verified by resolving every name in __all__.

Both new modules reference SessionState under TYPE_CHECKING only, so there is
no runtime cycle. Also inlined _worker_message, a single-caller passthrough
that just returned event.message.

review.py 330 -> 196, _render.py 107, _undo.py 74.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011w2Pj4Ek3ZNMQVYcFHgXDY
2026-08-16 15:42:10 +02:00
8ce453690f Split wsg_grabber/cli.py under the 250-line cap
The `logs` subcommand's reporting -- show_logs() and summarise() -- is
read-only analysis of the JSONL session logs and shares nothing with the
scrape/review flow, so it moves to _logs_report.py.

cli.py re-exports both through __all__ so cli.show_logs / cli.summarise stay
valid for test_cli.py and the dispatch table. The PEP 484 `X as X` alias form
does NOT survive here: ruff flags it PLC0414, autofixes it away, and then
deletes the import as unused under F401 -- which would break cli.summarise
with lint still green. Recorded in the handoff so it is not re-derived.

cli.py 282 -> 247, new _logs_report.py 76. 277 wsg_grabber tests pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011w2Pj4Ek3ZNMQVYcFHgXDY
2026-08-16 15:39:10 +02:00
77c6d9f815 Split wsg_grabber/catalog.py under the 250-line cap
catalog.py was 251 lines. Its five defensive accessors for the loosely-typed
/wsg/ JSON -- _get, _as_list, _as_int, _as_md5 and the _MD5_SHAPE / int64
range constants they depend on -- form a cohesive unit distinct from the
parsing logic above them, so they move to a sibling _coerce.py and become
module-public there.

No caller or test referenced the private helpers (downloader.py uses only the
six public parse functions), so no re-export shim was needed and the public
API is byte-for-byte unchanged.

catalog.py 251 -> 175, new _coerce.py 87. Both modules stay at 100% branch
coverage; 277 wsg_grabber tests pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011w2Pj4Ek3ZNMQVYcFHgXDY
2026-08-16 15:36:02 +02:00
89ed9bc95d wsg_grabber: scrape /wsg/, download every new video, triage with keep/pass
Some checks failed
Pre-commit checks / pre-commit (push) Has been cancelled
Python tests / test (push) Has been cancelled
Shell tests / shell-tests (push) Has been cancelled
Shell tests / arch-tests (push) Has been cancelled
New package. Walks every thread on 4chan's /wsg/, downloads each video it has
not seen, and reviews them one at a time: mpv embedded in a Tk window via --wid
with Keep/Pass/Undo directly beneath the picture.

Dedupe is structural rather than a convention. Every post carrying a file also
carries md5 -- base64 MD5 of the whole file -- so it is the sqlite primary key.
An already-seen video costs zero bytes to skip and a byte-identical repost in
another thread collapses onto the existing row. Verified against a live file:
the API's Nz9OEKdMuMZEdYE6eLTKmA== matched base64(md5(bytes)) exactly.

Undo is unbounded and survives quitting: the verdict trail lives in the index,
not in memory, so yesterday's decisions can still be taken back.

No downloaded video is ever deleted. A pass moves the file to trash/ and leaves
it there; files.apply_move refuses to overwrite an existing destination, so the
guarantee holds by construction rather than per caller.

Three bugs only running it could find, each caught from evidence rather than
guesswork:
  - the video stayed black: a <Configure> handler set mpv's geometry, fighting
    the automatic sizing --wid already does
  - playback never advanced: refresh() ran every tick and re-sent loadfile ~7x
    a second, restarting the video continuously
  - it froze on one video around the 26th while filenames kept advancing: mpv
    pushes an event stream down the same socket and nothing drained it, so the
    buffer filled and mpv stopped reading commands. Isolated to command #35
    without a drain thread; all 60 honoured with one.

A five-lens review then found nine more, all reproduced before fixing: a failed
mpv start stranded the non-daemon worker forever; undo could clobber a file in
incoming/; a NUL-bearing md5 poisoned the index permanently; an oversized int
killed the worker with no event; gzip bombs on both hosts were accepted; the
board endpoints were fetched unconditionally forever; and Session.missing could
corrupt the undo trail.

mpv runs as a subprocess over its JSON IPC socket rather than via python-mpv,
which resolves libmpv at import time and would fail on CI while passing here.
Third-party surface is requests alone, already declared.

277 package tests, 100% branch coverage on all 22 modules, no suppressions.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 14:55:30 +02:00