diff --git a/DOCS-restructure-handoff.md b/DOCS-restructure-handoff.md index f54ac68e..8339a981 100644 --- a/DOCS-restructure-handoff.md +++ b/DOCS-restructure-handoff.md @@ -73,10 +73,6 @@ copies were the only ones on the machine. detached worktree at HEAD. Live-system state, not a regression. - **`workout_locker` reports error** in `check_and_enable_services.sh`. That is the separate `screen-locker` repo. -- **`syncyomi-guard` is red** since 00:00 on 2026-08-23, working as designed: - the library collapsed from 2185 manga to 8. See the - `syncyomi-restore-silent-partial` memory; do NOT checkpoint the WAL before - recovering. Newest snapshot: `~/services/syncyomi/snapshots/`, 2026-08-15. ## Traps worth keeping diff --git a/docs/DOCS-agent-harness-evaluation-2026-08-18.md b/docs/DOCS-agent-harness-evaluation-2026-08-18.md index 6072df49..707d02e7 100644 --- a/docs/DOCS-agent-harness-evaluation-2026-08-18.md +++ b/docs/DOCS-agent-harness-evaluation-2026-08-18.md @@ -45,7 +45,7 @@ which your own hook (`track_reads_pretool.sh`, since disabled) was already tryin ### Docker MCP Gateway (mcp, sandbox, cli, 1.5k★) You run 23+ Dockerfiles for self-hosted MCP server containers (`~/src/mcp-servers/servers/*`) per the repo survey, on top -of an already-Docker-heavy self-hosted stack (Gitea, SearXNG, Syncyomi, Joplin Server, all `docker-compose.yml`-based). +of an already-Docker-heavy self-hosted stack (Gitea, SearXNG, Joplin Server, all `docker-compose.yml`-based). A container-aware MCP gateway is a direct fit for consolidating discovery/auth across a fleet this size, rather than each server being wired in ad hoc. Speculative on the _specific_ tool, high-confidence on the _need_ — you already have the infrastructure pattern this tool assumes. @@ -159,7 +159,7 @@ with the browser tooling you already have that a swap would fix. ### OpenClaw, Khoj, nanobot, CowAgent (self-hosted "personal agent" runtimes) -Surface-level strong fit — you unambiguously self-host (Gitea, SearXNG, Syncyomi, Joplin Server + DuckDNS, Ollama on +Surface-level strong fit — you unambiguously self-host (Gitea, SearXNG, Joplin Server + DuckDNS, Ollama on an RTX 3090) and run 39 systemd user units as personal automation. But these products are chat/webhook-fronted always-on daemons (Telegram/Discord/WhatsApp bots, "second brain" Q&A) — a different shape of automation than what you actually build, which is narrow, single-purpose guard daemons with typed configs and 100%-coverage test suites diff --git a/docs/DOCS-restructure-live-state.md b/docs/DOCS-restructure-live-state.md index 1d6b8af4..63e593a2 100644 --- a/docs/DOCS-restructure-live-state.md +++ b/docs/DOCS-restructure-live-state.md @@ -27,9 +27,6 @@ its timer next fires. /home/kuhy/.config/systemd/user/morning-routine.service 13:Environment=PYTHONPATH=%h/src/testsAndMisc 16:WorkingDirectory=%h/src/testsAndMisc - /home/kuhy/.config/systemd/user/syncyomi-guard.service - 8:WorkingDirectory=/home/kuhy/src/testsAndMisc - 9:Environment=PYTHONPATH=/home/kuhy/src/testsAndMisc ## /etc references /etc/guard-lib/targets/nsswitch.conf diff --git a/docs/DOCS-syncyomi-restore-runbook.md b/docs/DOCS-syncyomi-restore-runbook.md deleted file mode 100644 index d54fb9f2..00000000 --- a/docs/DOCS-syncyomi-restore-runbook.md +++ /dev/null @@ -1,196 +0,0 @@ -# SyncYomi / TachiyomiSY restore runbook - -Written after the 2026-08-09 incident, in which a restore that reported success -replaced a 2182-manga library with a 267 KB stub. - -## The bug - -TachiyomiSY **1.13.2** (current release as of 2026-08-09) fails a backup restore -with: - -``` -android.database.SQLException: Error code: 5, message: database is locked -``` - -The locked database is **the app's own `data.db` on the phone**, not the -SyncYomi server. The stack runs through Room's `InvalidationTracker` — UI -screens hold observer queries while the restore wants an exclusive write, and -the bundled SQLite returns `SQLITE_BUSY` instead of waiting. - -Upstream, open and unfixed: [#1634] and [#1638]. - -[#1634]: https://github.com/jobobby04/TachiyomiSY/issues/1634 -[#1638]: https://github.com/jobobby04/TachiyomiSY/issues/1638 - -## Fixed on this phone (2026-08-28) - -The phone no longer runs the upstream APK. It runs -[kuhyx/TachiyomiSY](https://github.com/kuhyx/TachiyomiSY) (`~/vendor/tachiyomisy`, -commit "Serialize backup restore and refuse to sync a collapsed library"), -which carries three changes upstream does not have: - -- `BackupRestorer` awaits the six restorers in order instead of `launch`ing - them into one scope, so they stop contending for SQLite's single writer. - That is the code-5 bug below. -- `SyncService.mergeSyncData` throws `SyncCollapseException` when more than - 10% of the server's entries are missing locally. A row that is simply gone - is a damaged database, not a deletion -- deleting in the app keeps the row - and clears `favorite`. -- `SyncYomiSyncService` additionally compares the outgoing payload against the - entry count this device last pushed, which covers the 304 path where there - is no remote data to merge against. - -Both guards fail closed: the sync aborts and the message reaches the user -through `SyncDataJob`'s existing error notification. Verified on the phone on -2026-08-28 against a real 2163-entry payload -- guard A refused at a faked -baseline of 99999, guard B refused with 1300 of 2163 entries removed, and the -server row was byte-identical afterwards in both runs. - -The build is `eu.kanade.tachiyomi.sy` version `1.13.2-kuhy`, i.e. it replaces -the official APK rather than sitting beside it, and is signed with -`~/.android/release/key.properties`. Rebuild and upgrade in place with: - -```bash -JAVA_HOME=/usr/lib/jvm/java-21-openjdk \ - ~/vendor/tachiyomisy/gradlew -p ~/vendor/tachiyomisy assembleFoss -PsyReplaceUpstream -adb install -r ~/vendor/tachiyomisy/app/build/outputs/apk/foss/app-arm64-v8a-foss.apk -``` - -Never install an official TachiyomiSY APK over it: the signatures differ, so -the install fails, and taking the official build back means an uninstall and a -restore. The fork's `foss` build type has the in-app updater compiled out for -exactly that reason. - -Rebasing on a new upstream release means replaying those commits; the fork is -not upstreamed. - -## Why it is dangerous - -The loud failure is survivable. The dangerous case is the quiet one: - -1. A restore fails with code 5. The library is now empty or partial. -2. A retry reports `Worker result SUCCESS` — but writes almost nothing. -3. SyncYomi pushes that near-empty library to the server. -4. The good server copy is gone. - -On 2026-08-09 the whole sequence took 51 seconds, and the real library survived -only as stale pages in the server's SQLite write-ahead log. - -**A success toast is not evidence.** Verify by entry count, always. - -## Restoring safely - -1. **Turn sync OFF in SY first.** Settings → Data and storage → Sync. - Skipping this lets a background sync overwrite the server mid-recovery. -2. **Force-stop the app** — `adb shell am force-stop eu.kanade.tachiyomi.sy`, - or Settings → Apps → Force stop. -3. **Cold-start straight into the restore.** Do not open the library, do not - let it sit on a tab loading covers. Every open screen is another observer - holding a connection, which is what the restore loses the race to. -4. **Put the phone down while it runs.** Navigating during a restore adds - exactly the contention that triggers the bug. -5. **Verify by count**, not by the toast. Compare against the guard's baseline: - `cat ~/.local/share/syncyomi_guard/last_known_good.json` -6. **Only then re-enable sync**, so the verified library propagates up. - -If it fails, force-stop and retry from step 3. Upstream reporters needed up to -five attempts. Retrying is expected, not a sign of a new problem. - -If it will not restore at all, the upstream workaround is a downgrade: -install 1.12.0 → restore → upgrade back to 1.13.2. This wipes `data.db`, so do -it **only** with a verified-good `.tachibk` in hand. - -## Known collateral: categories - -A restore that recovers every manga can still drop **all categories and all -category memberships**. Observed 2026-08-09: 2182/2182 manga and 57 389/57 389 -distinct chapters restored, while 18 categories and 1724 manga-to-category -assignments were lost. Matches the upstream report in [#1634]. - -Categories are not recoverable by re-syncing — the server copy is whatever the -phone last pushed. They come back only from a `.tachibk` that still has them. - -## Chapter counts shrink legitimately - -A restore **deduplicates** chapter entries. On 2026-08-09 the total fell from -75 376 to 68 418 (−9.2 %) while the distinct chapter count was unchanged at -57 389: 6958 duplicates were collapsed, and nothing was lost. - -Do not read a falling chapter count as data loss without comparing _distinct_ -chapter URLs. The guard's 10 % threshold is set above this specific event for -exactly that reason. - -## The guard - -`python_pkg/syncyomi_guard/` checks the server payload every 30 minutes, fails -loudly on a collapse, and keeps 14 restorable `.tachibk` snapshots. - -```bash -# check now -python3 -m python_pkg.syncyomi_guard # exit 0 healthy, 1 collapsed - -# install / remove the timer -./linux_configuration/features/setup_syncyomi_guard.sh -./linux_configuration/features/setup_syncyomi_guard.sh --uninstall - -systemctl --user list-timers syncyomi-guard.timer -journalctl --user -u syncyomi-guard.service -n 20 -``` - -Snapshots live in `~/services/syncyomi/snapshots/` and are ordinary `.tachibk` files — -`adb push` one to the phone and restore it directly. - -**Retention: 14 snapshots at 30-minute intervals is about 7 hours of history.** -That covers a fast incident — the 2026-08-09 window was 51 seconds — but not a -slow drift noticed days later. For that, the reference copy is -`~/services/syncyomi/recovery/`, which is never pruned. - -After a **deliberate** library purge the guard will report a collapse; accept -the new state with `--accept` to re-baseline. - -### Outcome of the 2026-08-09 incident - -Manga and chapters were fully recovered. **The 18 categories were not, and the -loss was accepted deliberately** — the baseline now records 1 category, so the -guard runs green against reality. - -Three attempts failed to get categories back onto the phone: - -1. Restore the recovered `.tachibk` → app briefly showed categories, but the - next sync pushed **1** category up. -2. `PUT` the full 18-category payload straight to the server → server held 18, - phone synced and flattened it back to 1. -3. Repeat of 2 with the same result. - -Round 2 is the informative one: the server had **18** and the phone had **1**, -and 1 won. The phone was not losing a merge — it was honestly reporting what it -had. The restore writes manga and chapters but silently drops the category -tables, which is the narrow form of the same 1.13.2 bug. - -The categories still exist in `~/services/syncyomi/recovery/syncyomi_recovered_2026-08-09.tachibk` -(18 definitions, 1724 memberships). Getting them into the app needs the -downgrade path — 1.12.0 → restore → verify → upgrade — which was not attempted. -If a future SY release fixes #1634, retry the restore before the downgrade. - -## Recovering from the server WAL (last resort) - -If the server payload is already degraded and no snapshot predates it, the old -payload may still exist as stale pages in `syncyomi.db-wal`. - -**Do not restart the container and do not checkpoint** — a clean close deletes -the WAL and the only remaining copy with it. Snapshot all three files first: - -```bash -sudo cp -a ~/services/syncyomi/config/syncyomi.db{,-shm,-wal} /some/safe/dir/ -``` - -Then rebuild by applying every WAL frame over the base database, last writer -wins, and extract `sync_data.data`. The 2026-08-09 reconstruction and its -verification are recorded in -`docs/superpowers/evidence/syncyomi-library-recovery-2026-08-09.json`. - -**`~/services/syncyomi/recovery/recovered.db` is a forensic artifact, not a database.** -It was assembled by splicing pages from several checkpoint generations, so it is -not any single committed SQLite state — `quick_check` passes and the payload -blob decodes, but do **not** copy it over `config/syncyomi.db`. The deliverable -from that recovery is the `.tachibk` beside it. diff --git a/docs/explanations/2026-07-13-explanation-personal-website-edge.html b/docs/explanations/2026-07-13-explanation-personal-website-edge.html index 26fd1145..0acc250e 100644 --- a/docs/explanations/2026-07-13-explanation-personal-website-edge.html +++ b/docs/explanations/2026-07-13-explanation-personal-website-edge.html @@ -19,7 +19,7 @@
This home server hosts several things at once: a self-hosted
Gitea (a GitHub-like git host), a dufs
- file cloud, a SyncYomi manga-sync service — and now a
+ file cloud — and now a
personal website. They all share a single public IP and a single pair of
open ports (80/443).
127.0.0.1:3000127.0.0.1:5000127.0.0.1:8282kuhy.duckdns.org { reverse_proxy 127.0.0.1:3000 } # Gitea (the root!)
kuhy-cloud.duckdns.org{ reverse_proxy 127.0.0.1:5000 } # dufs
-kuhy-sync.duckdns.org { reverse_proxy 127.0.0.1:8282 } # SyncYomiTwo details matter for this change. First, the bare root @@ -71,7 +69,7 @@ kuhy-sync.duckdns.org { reverse_proxy 127.0.0.1:8282 } # SyncYomi
Finally, that Caddyfile isn't hand-written — it's generated by
- setup_gitea.sh. SyncYomi and dufs each appended their
+ setup_gitea.sh. dufs appended its
block to the same file from their own setup scripts. Hold that thought.
setup_gitea.sh is meant to be safe to re-run. But its way of
writing the Caddyfile was cat > Caddyfile — it
regenerated the whole file from scratch, containing only Gitea's
- block. dufs and SyncYomi had appended their blocks later.
- So every time you re-ran the Gitea setup, those blocks silently vanished and
+ block. dufs had appended its block later.
+ So every time you re-ran the Gitea setup, that block silently vanished and
two live services went dark. A latent trap, waiting for the next re-run.
@@ -108,7 +106,7 @@ kuhy-sync.duckdns.org { reverse_proxy 127.0.0.1:8282 } # SyncYomi
- All four services are up. Now re-run the Gitea setup…
+All three services are up. Now re-run the Gitea setup…
setup_personal_website.sh orchestrates the whole cutover. The
order is the safety property: existing blocks are migrated
into snippets before the root file is switched to the import line —
- otherwise dufs and SyncYomi would blink out during the migration itself.
+ otherwise dufs would blink out during the migration itself.
preflight # deps, docker up, gitea-caddy present -migrate_caddy_snippets # split TODAY's Caddyfile → sites/*.caddy (keep dufs, sync) +migrate_caddy_snippets # split TODAY's Caddyfile → sites/*.caddy (keep dufs) relocate_gitea # re-run setup_gitea.sh (now writes import + gitea.caddy) build_website # pnpm build → ~/src/personal-website/dist write_website_stack # loopback caddy compose + inner Caddyfile @@ -336,7 +334,6 @@ ensure_firewall # 80/443 already open — idempotent
How setup_syncyomi.sh stands up a manga-sync server that a phone can reach over the public internet — by slotting into infrastructure that already existed, rather than building a new stack.
SyncYomi is a small self-hosted server. The manga reader app TachiyomiSY on a phone talks to it to keep its library and reading progress in sync across devices — the phone uploads a blob of its library, the server stores it, and other devices download it. The phone authenticates with a single API key, and it needs to reach the server from anywhere, including mobile data.
- -The interesting part isn't SyncYomi itself — it's that this PC was already a public server for two other things, and the new script had to fit into that without disturbing them. Three existing facts shaped every decision:
- -A single Caddy container (named gitea-caddy) owns the public ports 80 and 443. It terminates TLS and forwards each request to a small backend that listens only on 127.0.0.1 (loopback — unreachable from the network). Two tenants already live behind it:
- kuhy.duckdns.org → 127.0.0.1:3000 (Gitea) - kuhy-cloud.duckdns.org → 127.0.0.1:5000 (dufs) -
-Caddy tells them apart by the hostname in the request. It also fetches and renews a free Let's Encrypt TLS certificate for each hostname automatically.
-The home connection has one changing public IP. A cron job (~/.joplin-server/duckdns-update.sh) tells DuckDNS "these labels all point at my current IP". Adding a service means adding its label to that one updater — not creating a second updater.
This host's nftables has a default-drop FORWARD chain. A normal Docker bridge network routes through FORWARD, so a bridged container's traffic gets dropped with no error. The existing services dodge this by using network_mode: host and binding to 127.0.0.1. The new service must do the same.
So the change is not "install a server." It's "add a third tenant to a shared, hostname-routed edge, reusing its TLS, its DNS updater, and its firewall posture."
- -Picture the whole public surface as a single reception desk. Everything from the outside world arrives at that one desk (port 443 on gitea-caddy). The desk reads the name on the envelope — the hostname — and walks it to the right internal room. The rooms themselves have no external doors at all.
Adding SyncYomi is therefore four small edits, not a new stack:
-127.0.0.1:8282.kuhy-sync.duckdns.org { … } block appended to the shared Caddyfile.0.0.0.0, it would have an external door of its own — bypassing the TLS desk entirely and serving your library in plaintext. Binding 127.0.0.1 forces every request to go through Caddy. SyncYomi actually defaults to 0.0.0.0, so the script rewrites that after first launch.
- Pick a hostname and watch Caddy decide where it goes. This mirrors the real site blocks in ~/services/gitea/Caddyfile.
The script is one file, setup_syncyomi.sh, with setup / status / help subcommands. Here are the parts that carry the design, grouped by what they achieve.
It refuses to invent infrastructure. It hard-requires the existing edge to be present and bails with a pointer if not — so it can never accidentally spin up a competing proxy.
-has_cmd docker || die "docker not found — run setup_gitea.sh first."
-docker ps --format '{{.Names}}' | grep -qx "${CADDY_CONTAINER}" ||
- die "Caddy container '${CADDY_CONTAINER}' not running — run setup_gitea.sh first."
-[[ -f ${CADDYFILE} ]] || die "Caddyfile ${CADDYFILE} missing — run setup_gitea.sh first."
-[[ -x ${DUCKDNS_UPDATER} ]] || die "DuckDNS updater missing — run install_joplin.sh first."
-
- network_mode: host sidesteps the FORWARD-chain trap; the loopback bind (enforced just below) keeps it private.
services:
- syncyomi:
- image: ghcr.io/syncyomi/syncyomi:v1.1.11 # pinned, not :latest
- restart: unless-stopped # ← survives reboot
- network_mode: host
- volumes: [ ./config:/config ]
- restart: unless-stopped plus the enabled Docker service is the entire "auto-start after reboot" story — no bespoke systemd unit needed.
SyncYomi v1.1.x has no public health route (/api/healthz is a 404). The script instead treats GET /api/auth/onboard as "up" — it answers 204 when no account exists yet, or 403 once one does. Either means the API is serving.
syncyomi_ready() {
- code="$(curl -sS -o /dev/null -w '%{http_code}' "${base}/api/auth/onboard" ...)"
- [[ ${code} == "204" || ${code} == "403" ]]
-}
- Once it's up, the script patches the generated config from the insecure default and restarts:
-sed -i -E 's|^\s*host\s*=.*|host = "127.0.0.1"|' "${CONFIG_TOML}" # was 0.0.0.0
-
- This is the part that lets the script hand you a key instead of making you click through a web UI. It onboards the first account (only if none exists), logs in for a session cookie, then reuses or creates a key. Two gotchas discovered live are baked in:
-tachiyomisy key so repeated runs don't pile up keys.scopes array — SyncYomi returns HTTP 500 on a null.key="$(curl -sS -b "${jar}" "${SYNCYOMI_LOCAL}/api/keys" \
- | jq -r '[.[] | select(.name=="tachiyomisy")][0].key // empty')"
-if [[ -z ${key} ]]; then
- key="$(curl -sS -b "${jar}" \
- -d '{"name":"tachiyomisy","scopes":["read","write"]}' \
- "${SYNCYOMI_LOCAL}/api/keys" | jq -r '.key // empty')"
-fi
-
- One site block appended to the shared Caddyfile, then an in-container reload. Caddy issues the TLS cert on its own once the name resolves.
-printf '\n%s {\n\treverse_proxy 127.0.0.1:%s\n}\n' \
- "${SYNCYOMI_SUBDOMAIN}" "${SYNCYOMI_PORT}" >>"${CADDYFILE}"
-docker exec "${CADDY_CONTAINER}" caddy reload --config "${CADDYFILE_IN_CONTAINER}"
- And one label spliced into the existing updater's domains= list — no second cron job:
sed -i -E "s|(domains=)([^&\"']*)|\1\2,${label}|" "${DUCKDNS_UPDATER}"
-
- setup_gitea.sh, which rewrites it from scratch when re-run. If that happens, SyncYomi's block vanishes — the fix is simply to re-run setup_syncyomi.sh setup, which idempotently re-appends it. This is a known, documented trade-off inherited from the sibling setup_dufs_cloud.sh.
- Five questions on the reasoning behind the change. Click an option to check it.
- -/explain-diff for commit 59430f9 ·
- linux_configuration/features/setup_syncyomi.sh
- This machine leads a double life. By day it is a desktop. Around the clock it is a small server: Gitea, a public Caddy TLS edge on ports 80/443, SyncYomi, a personal website, Open WebUI, Joplin, dufs, plus ollama, dnsmasq, wg-quick@wg0, nftables and sshd.
This machine leads a double life. By day it is a desktop. Around the clock it is a small server: Gitea, a public Caddy TLS edge on ports 80/443, a personal website, Open WebUI, Joplin, dufs, plus ollama, dnsmasq, wg-quick@wg0, nftables and sshd.
It also runs a digital-wellbeing curfew: a deliberately hard-to-defeat system that stopped you using the PC late at night. It is built as a fortress, and the distinction between its parts is the key to this whole change:
@@ -184,7 +184,7 @@ esacgetty@ masked)ollama, dnsmasq, wg-quick@wg0, nftables, sshdollama, dnsmasq, wg-quick@wg0, nftables, sshd