testsAndMisc/docs/superpowers/evidence/syncyomi-library-recovery-2026-08-09.json
Krzysztof kuhy Rudnicki cef72fbab0 syncyomi_guard: refuse a collapsed payload as the new good copy
TachiyomiSY 1.13.2 can report a restore as SUCCESS while writing almost
nothing, then sync that near-empty library up and replace the only good
copy. On 2026-08-09 a 14 MB payload holding 2182 manga became a 267 KB
stub, and the real library survived only as stale pages in the SQLite
write-ahead log.

The upstream bug is Kotlin in an APK and cannot be fixed here, so this
targets the fixable half: decode the payload, count what is actually in
it, and refuse to treat a collapse as normal drift. A pure is-it-zero
check would have caught this incident but not the harder case upstream
describes, where a restore keeps some entries and drops the rest, so the
threshold is proportional. Categories get their own rule because all
2182 manga survived the round trip and all 18 categories did not.

Adds the setup script and a runbook covering the WAL-page salvage used
to recover the library on the day.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YQLwdquDCNmWtvgi3Lk72J
2026-08-10 15:33:18 +02:00

63 lines
4.9 KiB
JSON

{
"intent": "Recover a TachiyomiSY library destroyed by a silent partial restore, and add a guard so a collapsed sync payload can never again become the only copy.",
"scope": [
"python_pkg/syncyomi_guard/ — new package: protobuf payload decoder, collapse detection, snapshot store, CLI",
"linux_configuration/scripts/single_use/features/setup_syncyomi_guard.sh — user systemd timer installer",
"docs/syncyomi-restore-runbook.md — restore procedure and WAL recovery steps",
"Non-goal: fixing the upstream bug. It is Kotlin in a third-party APK (jobobby04/TachiyomiSY#1634, #1638, both open). The guard detects and preserves; it cannot prevent."
],
"changes": [
"Diagnosed the failure as phone-side, not server-side: android.database.SQLException code 5 in BackupRestoreJob via Room InvalidationTracker. Server logged zero errors and PRAGMA quick_check returned ok.",
"Recovered the pre-damage 14 MB payload by replaying all 3447 WAL frames over the base database (last writer wins) and extracting sync_data.data; wrote it out as a gzipped .tachibk.",
"Deliberately did NOT checkpoint the WAL when first asked: it held the only surviving copy of the library. Deferred until after recovery, then done via a graceful container restart (14201672 -> 0 bytes).",
"payload.py: dependency-free Backup protobuf reader counting manga (field 1), categories (field 2), chapters (field 16, nested) and sources (field 101); rejects trailing bytes so splice damage cannot decode as success.",
"verdict.py: fails on >10% drop or a fall to zero in manga, chapters or categories; a missing baseline reports FIRST_RUN rather than a pass.",
"store.py: reads the live database through an immutable URI (never locks it, never triggers the SQLITE_BUSY class of failure it detects) and keeps N restorable .tachibk snapshots.",
"Prune glob tightened to the timestamp shape so a hand-named recovery file cannot be deleted as a stale snapshot."
],
"verification": [
{
"command": "protoc --decode_raw < payload.bin | grep -cE '^1 \\{' (independent cross-check of the hand-written decoder)",
"result": "pass",
"evidence": "protoc and python_pkg.syncyomi_guard agree exactly on all three real payloads: recovered 2182 manga / 18 categories, stub 0 / 0, post-restore 2182 / 0."
},
{
"command": "python3 -m python_pkg.syncyomi_guard --db <db holding the real 267KB stub> --state-dir <real baseline>",
"result": "pass",
"evidence": "COLLAPSED: manga went from 2182 to 0; chapters went from 68418 to 0. EXIT=1, no snapshot written. Replays the actual 22:53 incident end to end through the binary."
},
{
"command": "python3 -m python_pkg.syncyomi_guard --db /home/kuhy/syncyomi/config/syncyomi.db",
"result": "pass",
"evidence": "ok: 2182 manga, 68418 chapters, 0 categories, 24 sources (12277887 bytes); snapshot written. Ran against the live database while the container was serving."
},
{
"command": "systemctl --user start syncyomi-guard.service",
"result": "pass",
"evidence": "status=0/SUCCESS in the real deploy environment; journal shows the ok line and the snapshot path."
},
{
"command": "pytest python_pkg/syncyomi_guard/tests/ --cov=python_pkg.syncyomi_guard",
"result": "pass",
"evidence": "59 passed; 100% branch coverage across all five modules."
},
{
"command": "distinct chapter URL comparison, recovered backup vs post-restore server payload",
"result": "pass",
"evidence": "Both sides 57389 distinct chapters. The 75376 -> 68418 fall was 6958 intra-manga duplicates being collapsed, not data loss. Prevented a false 'chapters lost' report and a wrongly tightened threshold."
}
],
"risks": [
"Categories are NOT recovered: 18 definitions and 1724 manga-to-category assignments were lost and only exist in syncyomi_recovered_2026-08-09.tachibk. Restoring them needs another full restore, which is another roll of the code-5 dice.",
"The current baseline records categories=0, and _drop_reason returns None when previous == 0, so category loss stays invisible until a healthy run raises the baseline above zero.",
"The guard reads the payload SyncYomi stores; it cannot see a phone-side library that has not yet synced.",
"The 10% threshold is calibrated to one incident. A slow drip below 10% per run would not trip it."
],
"rollback": [
"./linux_configuration/scripts/single_use/features/setup_syncyomi_guard.sh --uninstall (snapshots are kept)",
"Delete python_pkg/syncyomi_guard/ — nothing else imports it.",
"Library recovery is independent of the guard: ~/syncyomi/recovery/syncyomi_recovered_2026-08-09.tachibk (sha256 cf00bee4b449255716387efe0587045f7b0b92a6c54327dd9dd3cfc7746ed7c3) restores 2182 manga, 75376 chapter entries and 18 categories.",
"After rollback, verify the server payload still decodes to 2182 manga."
]
}