testsAndMisc/docs/DOCS-syncyomi-restore-runbook.md

# SyncYomi / TachiyomiSY restore runbook

Written after the 2026-08-09 incident, in which a restore that reported success
replaced a 2182-manga library with a 267 KB stub.

## The bug

TachiyomiSY **1.13.2** (current release as of 2026-08-09) fails a backup restore
with:

```
android.database.SQLException: Error code: 5, message: database is locked
```

The locked database is **the app's own `data.db` on the phone**, not the
SyncYomi server. The stack runs through Room's `InvalidationTracker` — UI
screens hold observer queries while the restore wants an exclusive write, and
the bundled SQLite returns `SQLITE_BUSY` instead of waiting.

Upstream, open and unfixed: [#1634] and [#1638].

[#1634]: https://github.com/jobobby04/TachiyomiSY/issues/1634
[#1638]: https://github.com/jobobby04/TachiyomiSY/issues/1638

## Fixed on this phone (2026-08-28)

The phone no longer runs the upstream APK. It runs
[kuhyx/TachiyomiSY](https://github.com/kuhyx/TachiyomiSY) (`~/vendor/tachiyomisy`,
commit "Serialize backup restore and refuse to sync a collapsed library"),
which carries three changes upstream does not have:

- `BackupRestorer` awaits the six restorers in order instead of `launch`ing
  them into one scope, so they stop contending for SQLite's single writer.
  That is the code-5 bug below.
- `SyncService.mergeSyncData` throws `SyncCollapseException` when more than
  10% of the server's entries are missing locally. A row that is simply gone
  is a damaged database, not a deletion -- deleting in the app keeps the row
  and clears `favorite`.
- `SyncYomiSyncService` additionally compares the outgoing payload against the
  entry count this device last pushed, which covers the 304 path where there
  is no remote data to merge against.

Both guards fail closed: the sync aborts and the message reaches the user
through `SyncDataJob`'s existing error notification. Verified on the phone on
2026-08-28 against a real 2163-entry payload -- guard A refused at a faked
baseline of 99999, guard B refused with 1300 of 2163 entries removed, and the
server row was byte-identical afterwards in both runs.

The build is `eu.kanade.tachiyomi.sy` version `1.13.2-kuhy`, i.e. it replaces
the official APK rather than sitting beside it, and is signed with
`~/.android/release/key.properties`. Rebuild and upgrade in place with:

```bash
JAVA_HOME=/usr/lib/jvm/java-21-openjdk \
  ~/vendor/tachiyomisy/gradlew -p ~/vendor/tachiyomisy assembleFoss -PsyReplaceUpstream
adb install -r ~/vendor/tachiyomisy/app/build/outputs/apk/foss/app-arm64-v8a-foss.apk
```

Never install an official TachiyomiSY APK over it: the signatures differ, so
the install fails, and taking the official build back means an uninstall and a
restore. The fork's `foss` build type has the in-app updater compiled out for
exactly that reason.

Rebasing on a new upstream release means replaying those commits; the fork is
not upstreamed.

## Why it is dangerous

The loud failure is survivable. The dangerous case is the quiet one:

1. A restore fails with code 5. The library is now empty or partial.
2. A retry reports `Worker result SUCCESS` — but writes almost nothing.
3. SyncYomi pushes that near-empty library to the server.
4. The good server copy is gone.

On 2026-08-09 the whole sequence took 51 seconds, and the real library survived
only as stale pages in the server's SQLite write-ahead log.

**A success toast is not evidence.** Verify by entry count, always.

## Restoring safely

1. **Turn sync OFF in SY first.** Settings → Data and storage → Sync.
   Skipping this lets a background sync overwrite the server mid-recovery.
2. **Force-stop the app** — `adb shell am force-stop eu.kanade.tachiyomi.sy`,
   or Settings → Apps → Force stop.
3. **Cold-start straight into the restore.** Do not open the library, do not
   let it sit on a tab loading covers. Every open screen is another observer
   holding a connection, which is what the restore loses the race to.
4. **Put the phone down while it runs.** Navigating during a restore adds
   exactly the contention that triggers the bug.
5. **Verify by count**, not by the toast. Compare against the guard's baseline:
   `cat ~/.local/share/syncyomi_guard/last_known_good.json`
6. **Only then re-enable sync**, so the verified library propagates up.

If it fails, force-stop and retry from step 3. Upstream reporters needed up to
five attempts. Retrying is expected, not a sign of a new problem.

If it will not restore at all, the upstream workaround is a downgrade:
install 1.12.0 → restore → upgrade back to 1.13.2. This wipes `data.db`, so do
it **only** with a verified-good `.tachibk` in hand.

## Known collateral: categories

A restore that recovers every manga can still drop **all categories and all
category memberships**. Observed 2026-08-09: 2182/2182 manga and 57 389/57 389
distinct chapters restored, while 18 categories and 1724 manga-to-category
assignments were lost. Matches the upstream report in [#1634].

Categories are not recoverable by re-syncing — the server copy is whatever the
phone last pushed. They come back only from a `.tachibk` that still has them.

## Chapter counts shrink legitimately

A restore **deduplicates** chapter entries. On 2026-08-09 the total fell from
75 376 to 68 418 (−9.2 %) while the distinct chapter count was unchanged at
57 389: 6958 duplicates were collapsed, and nothing was lost.

Do not read a falling chapter count as data loss without comparing _distinct_
chapter URLs. The guard's 10 % threshold is set above this specific event for
exactly that reason.

## The guard

`python_pkg/syncyomi_guard/` checks the server payload every 30 minutes, fails
loudly on a collapse, and keeps 14 restorable `.tachibk` snapshots.

```bash
# check now
python3 -m python_pkg.syncyomi_guard          # exit 0 healthy, 1 collapsed

# install / remove the timer
./linux_configuration/features/setup_syncyomi_guard.sh
./linux_configuration/features/setup_syncyomi_guard.sh --uninstall

systemctl --user list-timers syncyomi-guard.timer
journalctl --user -u syncyomi-guard.service -n 20
```

Snapshots live in `~/services/syncyomi/snapshots/` and are ordinary `.tachibk` files —
`adb push` one to the phone and restore it directly.

**Retention: 14 snapshots at 30-minute intervals is about 7 hours of history.**
That covers a fast incident — the 2026-08-09 window was 51 seconds — but not a
slow drift noticed days later. For that, the reference copy is
`~/services/syncyomi/recovery/`, which is never pruned.

After a **deliberate** library purge the guard will report a collapse; accept
the new state with `--accept` to re-baseline.

### Outcome of the 2026-08-09 incident

Manga and chapters were fully recovered. **The 18 categories were not, and the
loss was accepted deliberately** — the baseline now records 1 category, so the
guard runs green against reality.

Three attempts failed to get categories back onto the phone:

1. Restore the recovered `.tachibk` → app briefly showed categories, but the
   next sync pushed **1** category up.
2. `PUT` the full 18-category payload straight to the server → server held 18,
   phone synced and flattened it back to 1.
3. Repeat of 2 with the same result.

Round 2 is the informative one: the server had **18** and the phone had **1**,
and 1 won. The phone was not losing a merge — it was honestly reporting what it
had. The restore writes manga and chapters but silently drops the category
tables, which is the narrow form of the same 1.13.2 bug.

The categories still exist in `~/services/syncyomi/recovery/syncyomi_recovered_2026-08-09.tachibk`
(18 definitions, 1724 memberships). Getting them into the app needs the
downgrade path — 1.12.0 → restore → verify → upgrade — which was not attempted.
If a future SY release fixes #1634, retry the restore before the downgrade.

## Recovering from the server WAL (last resort)

If the server payload is already degraded and no snapshot predates it, the old
payload may still exist as stale pages in `syncyomi.db-wal`.

**Do not restart the container and do not checkpoint** — a clean close deletes
the WAL and the only remaining copy with it. Snapshot all three files first:

```bash
sudo cp -a ~/services/syncyomi/config/syncyomi.db{,-shm,-wal} /some/safe/dir/
```

Then rebuild by applying every WAL frame over the base database, last writer
wins, and extract `sync_data.data`. The 2026-08-09 reconstruction and its
verification are recorded in
`docs/superpowers/evidence/syncyomi-library-recovery-2026-08-09.json`.

**`~/services/syncyomi/recovery/recovered.db` is a forensic artifact, not a database.**
It was assembled by splicing pages from several checkpoint generations, so it is
not any single committed SQLite state — `quick_check` passes and the payload
blob decodes, but do **not** copy it over `config/syncyomi.db`. The deliverable
from that recovery is the `.tachibk` beside it.