Commit Graph

2 Commits

Author SHA1 Message Date
6258b35f88 perf(sync): hoist the device listing and stop the curated-bank 404 storm
Some checks are pending
file-length / file length <= 250 lines (push) Waiting to run
pre-commit / pre-commit (push) Waiting to run
Tests / test (3.10) (push) Waiting to run
Tests / test (3.11) (push) Waiting to run
Tests / test (3.12) (push) Waiting to run
release-apk / release (APK) (push) Waiting to run
Two independent costs in the background tick, both pure waste.

`list_directory` was called four times per tick -- once by the log pull and
once by each of the three bank syncs. It is the single most expensive call
in a tick at ~445ms mirrored (the GitHub half alone is ~360ms), so that was
~1.3s spent asking the same question four times. It is now listed once in
`run_sync` and passed down.

The curated-bank loop was worse: 17s and 62 requests to fetch *zero* useful
bytes. Firebase returns None for every peer's `food_bank_manual.json`, so
`MirrorSyncClient.get_file_text` treated each miss as "this device has not
migrated yet" and fell through to GitHub, which answers `{}` after ~320ms.
`_primary_only_text` skips that per-peer fallthrough for this one artifact.

Checked before narrowing rather than assumed, because CLAUDE.md requires
both halves of the food bank to sync: reading every peer's curated bank
straight off the mirror found 23 absent, 2 empty, and zero non-empty. No
curated food exists on the mirror alone. The probe is written into the
docstring so it can be re-run before anyone widens this again.

Measured end to end: manual-bank loop 17.0s -> 3.4s, full tick ~27s -> 15.2s,
log/bank/peer state unchanged (441 entries, 117 foods, 22 peers).

`_pull_remote_logs` now returns the revisions it saw instead of mutating a
caller-supplied dict, which keeps it under the argument limit and makes the
data flow visible at the call site.

Deliberately NOT done: revision-skipping the derived food bank. `run_sync`
rebuilds that bank from the merged log immediately before syncing it, so
unlike the log the peer records are not durably held locally -- skipping a
peer would drop its counts from the pushed bank and oscillate under
max-count-wins. Verified empirically: the on-disk bank and a pure log-derived
rebuild are identical (117 foods, 0 differing counts), and the 2 foods peers
know that the local bank lacks are discarded by the existing `if name in
local` filter anyway. The loop is off every interactive path now, so the
remaining cost is background-only.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LJ5YnA3CERitQ57HjMu2Qz
2026-08-21 13:16:08 +02:00
e23f7ac3b4 Push under a persisted per-install uuid, not a fixed role id
Some checks are pending
pre-commit / pre-commit (push) Waiting to run
Tests / test (3.10) (push) Waiting to run
Tests / test (3.11) (push) Waiting to run
Tests / test (3.12) (push) Waiting to run
release-apk / release (APK) (push) Waiting to run
diet_guard pushed as "pc" (Python), "phone" (Android) and "desktop" (web).
A role constant collides the moment a second machine takes the same role,
and after a reinstall the new install inherits the previous one's CRDT
identity -- the same reasoning home_inventory already records for choosing a
uuid. Both halves now mint a uuid once and persist it: the PC in
~/.local/share/diet_guard/.device_id, the app under SharedPreferences
'crdt.nodeId' (the key todo and home_inventory already use).

The old constant survives as the legacy id and is passed to sync_log /
syncLog, so devices/<legacy>/ is skipped as this device's own. Without that
every tick would re-download and re-merge this device's own pre-migration
log as though a peer had written it -- idempotent under LWW, but hundreds of
KB, 96 times a day. It drops to None once that path is reclaimed; a test
covers both states, including a negative control proving the skip is the
legacy id's doing.

All four documents move together (food_log, food_bank, food_bank_manual,
budget), and the hist:<date> fields riding on the budget record are
untouched -- merge_record unions field names, so they relay through
unchanged.

The resolver lives in its own module (_device.py / sync_device_id.dart)
because _sync_merge needs it for HLC stamping too and importing _sync there
would be a cycle. Dart resolves it asynchronously at startup, before
LogStorageService.init(); currentSyncDeviceId falls back to the compile-time
role constant until then, so a too-early Hlc stamp gets a valid id rather
than crashing.

conftest redirects the new id file in the same change that introduces it --
without that a test mints a uuid into the real state dir and this machine's
live sync identity is decided by whichever test ran first. Verified by
checking ~/.local/share/diet_guard/ before and after: untouched.

Verified live, not just against mocks: the sync timer ran mid-change and
migrated this machine to f8f46da6. All 358 food_log records and all 100
food_bank records are present under the new path (0 missing vs devices/pc/),
and the new log's HLCs carry only its own id -- confirming it did not
re-merge the old path as a peer.

Python: 687 tests, 100% branch coverage. App: 518 tests, analyzer clean.
Pins bumped to crdt-sync-v0.6.0 / crdt_sync_dart-v0.7.0 (pyproject,
requirements.txt, .pre-commit-config.yaml, pubspec), and the new library is
installed into /usr/bin/python's user site-packages so the production timer
does not ImportError on its next tick.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZH89WEqAjuJT8WZdqc7AP
2026-08-09 17:18:42 +02:00