mirror of
https://github.com/kuhyx/diet-guard.git
synced 2026-11-11 12:07:42 +01:00
Some checks are pending
file-length / file length <= 250 lines (push) Waiting to run
pre-commit / pre-commit (push) Waiting to run
Tests / test (3.10) (push) Waiting to run
Tests / test (3.11) (push) Waiting to run
Tests / test (3.12) (push) Waiting to run
release-apk / release (APK) (push) Waiting to run
Two independent costs in the background tick, both pure waste.
`list_directory` was called four times per tick -- once by the log pull and
once by each of the three bank syncs. It is the single most expensive call
in a tick at ~445ms mirrored (the GitHub half alone is ~360ms), so that was
~1.3s spent asking the same question four times. It is now listed once in
`run_sync` and passed down.
The curated-bank loop was worse: 17s and 62 requests to fetch *zero* useful
bytes. Firebase returns None for every peer's `food_bank_manual.json`, so
`MirrorSyncClient.get_file_text` treated each miss as "this device has not
migrated yet" and fell through to GitHub, which answers `{}` after ~320ms.
`_primary_only_text` skips that per-peer fallthrough for this one artifact.
Checked before narrowing rather than assumed, because CLAUDE.md requires
both halves of the food bank to sync: reading every peer's curated bank
straight off the mirror found 23 absent, 2 empty, and zero non-empty. No
curated food exists on the mirror alone. The probe is written into the
docstring so it can be re-run before anyone widens this again.
Measured end to end: manual-bank loop 17.0s -> 3.4s, full tick ~27s -> 15.2s,
log/bank/peer state unchanged (441 entries, 117 foods, 22 peers).
`_pull_remote_logs` now returns the revisions it saw instead of mutating a
caller-supplied dict, which keeps it under the argument limit and makes the
data flow visible at the call site.
Deliberately NOT done: revision-skipping the derived food bank. `run_sync`
rebuilds that bank from the merged log immediately before syncing it, so
unlike the log the peer records are not durably held locally -- skipping a
peer would drop its counts from the pushed bank and oscillate under
max-count-wins. Verified empirically: the on-disk bank and a pure log-derived
rebuild are identical (117 foods, 0 differing counts), and the 2 foods peers
know that the local bank lacks are discarded by the existing `if name in
local` filter anyway. The loop is off every interactive path now, so the
remaining cost is background-only.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LJ5YnA3CERitQ57HjMu2Qz
237 lines
9.0 KiB
Python
237 lines
9.0 KiB
Python
"""Pull/merge/push for the food bank and the budget.
|
|
|
|
Split out of ``_sync.py`` for file size. These three run inside the same tick
|
|
as the food-log sync but are independent of it: each owns one remote document,
|
|
merges it with ``crdt_sync``'s LWW, writes the merged result back locally and
|
|
pushes it.
|
|
|
|
The derived bank merges by ``count`` (max-count-wins, idempotent -- the right
|
|
merge for a counter derived from the log), the curated bank by ``editedAt``,
|
|
and the budget by its ``t`` edit stamp. All three are per-field LWW over the
|
|
*union* of field names, so a device predating a field relays it untouched.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
import logging
|
|
from typing import TYPE_CHECKING
|
|
|
|
from crdt_sync import MirrorSyncClient, merge_logs
|
|
|
|
from diet_guard._budget import read_raw_record, write_raw_record
|
|
from diet_guard._budget_history import (
|
|
history_to_json,
|
|
load_entries,
|
|
write_raw_history,
|
|
)
|
|
from diet_guard._device import device_identity
|
|
from diet_guard._foodbank import read_food_bank, write_food_bank
|
|
from diet_guard._foodbank_manual import read_manual_bank, write_manual_bank
|
|
from diet_guard._meal_schedule_store import (
|
|
history_to_json as schedule_history_to_json,
|
|
)
|
|
from diet_guard._meal_schedule_store import (
|
|
load_entries as load_schedule_entries,
|
|
)
|
|
from diet_guard._meal_schedule_store import (
|
|
write_raw_history as write_raw_schedule,
|
|
)
|
|
from diet_guard._sync_paths import (
|
|
_device_budget_path,
|
|
_device_food_bank_path,
|
|
_device_manual_bank_path,
|
|
)
|
|
from diet_guard.sync_merge import (
|
|
budget_to_log,
|
|
food_bank_to_log,
|
|
log_to_budget,
|
|
log_to_food_bank,
|
|
log_to_history,
|
|
log_to_manual_bank,
|
|
log_to_schedule_history,
|
|
manual_bank_to_log,
|
|
parse_remote_budget,
|
|
parse_remote_food_bank,
|
|
parse_remote_manual_bank,
|
|
)
|
|
|
|
if TYPE_CHECKING:
|
|
from collections.abc import Sequence
|
|
|
|
from crdt_sync import GitHubSyncClient
|
|
|
|
_logger = logging.getLogger(__name__)
|
|
|
|
|
|
def _sync_food_bank(client: GitHubSyncClient, device_ids: Sequence[str]) -> None:
|
|
"""Pull, merge, persist and push the log-derived food bank.
|
|
|
|
Runs *after* the local rebuild in :func:`run_sync`, so this device's own
|
|
records already reflect the merged log; the merge then unions in whatever
|
|
another device knows and max-count wins per food (see
|
|
:func:`diet_guard.sync_merge.food_bank_to_log`).
|
|
|
|
Strictly speaking the bank is derivable from the already-synced log, so
|
|
both devices converge on their own eventually. Syncing it makes them
|
|
agree *now*, and publishes the bank so a fresh device has autocomplete
|
|
before it has replayed anything.
|
|
|
|
**The log stays authoritative for which foods exist.** A CRDT union
|
|
never shrinks, so without this the max-count merge would resurrect a food
|
|
whose entries were all undone -- a peer's stale copy would out-clock the
|
|
local absence and be written back and re-pushed forever, un-deletable.
|
|
Restricting the result to the foods the freshly-rebuilt local bank still
|
|
contains fixes that, and stays identical across devices because that
|
|
rebuild comes from the *merged* log both devices share.
|
|
"""
|
|
identity = device_identity()
|
|
local = read_food_bank()
|
|
merged = food_bank_to_log(local)
|
|
for device_id in device_ids:
|
|
if identity.is_own(device_id):
|
|
continue
|
|
text = client.get_file_text(_device_food_bank_path(device_id))
|
|
if text is None:
|
|
continue
|
|
try:
|
|
merged = merge_logs(merged, parse_remote_food_bank(text))
|
|
except (TypeError, KeyError, ValueError, json.JSONDecodeError):
|
|
_logger.warning(
|
|
"Unparsable food bank pushed by device %r, skipping",
|
|
device_id,
|
|
)
|
|
|
|
if not merged:
|
|
return
|
|
resolved = {
|
|
name: record
|
|
for name, record in log_to_food_bank(merged).items()
|
|
if name in local
|
|
}
|
|
write_food_bank(resolved)
|
|
merged = food_bank_to_log(resolved)
|
|
client.put_file_text(
|
|
_device_food_bank_path(identity.device_id),
|
|
json.dumps(
|
|
{record_id: record.to_dict() for record_id, record in merged.items()},
|
|
indent=2,
|
|
),
|
|
message="diet_guard sync",
|
|
)
|
|
|
|
|
|
def _primary_only_text(client: GitHubSyncClient, path: str) -> str | None:
|
|
"""Read ``path`` from the primary backend, skipping the mirror fallthrough.
|
|
|
|
``MirrorSyncClient.get_file_text`` treats a ``None`` from the primary as
|
|
"this device has not migrated yet" and consults the mirror. That is right
|
|
for the log and the derived bank, whose data really can live on either
|
|
side -- but for the *curated* bank it is 21 GitHub round trips per tick
|
|
(~406ms each, ~17s total) that return ``{}`` and nothing else.
|
|
|
|
Verified before narrowing, and worth re-checking before widening again:
|
|
reading every peer's curated bank straight off the mirror found 23 absent,
|
|
2 empty and **zero** non-empty. No curated food exists on the mirror alone,
|
|
so nothing is lost. A client with no ``primary`` (the GitHub-only case) is
|
|
read directly, which is the same backend either way.
|
|
"""
|
|
if isinstance(client, MirrorSyncClient):
|
|
return client.primary.get_file_text(path)
|
|
return client.get_file_text(path)
|
|
|
|
|
|
def _sync_manual_bank(client: GitHubSyncClient, device_ids: Sequence[str]) -> None:
|
|
"""Pull, merge, persist and push the hand-curated food bank.
|
|
|
|
Curated entries are the one part of the bank that is not derivable from
|
|
the food log (see :mod:`diet_guard._foodbank_manual`), so unlike
|
|
``food_bank.json`` they need a real merge: last-writer-wins per food name
|
|
by edit time, union across devices.
|
|
"""
|
|
identity = device_identity()
|
|
merged = manual_bank_to_log(read_manual_bank())
|
|
for device_id in device_ids:
|
|
if identity.is_own(device_id):
|
|
continue
|
|
text = _primary_only_text(client, _device_manual_bank_path(device_id))
|
|
if text is None:
|
|
continue
|
|
try:
|
|
merged = merge_logs(merged, parse_remote_manual_bank(text))
|
|
except (TypeError, KeyError, ValueError, json.JSONDecodeError):
|
|
_logger.warning(
|
|
"Unparsable curated food bank pushed by device %r, skipping",
|
|
device_id,
|
|
)
|
|
|
|
if not merged:
|
|
# No device has curated anything: nothing to persist, and pushing an
|
|
# empty object every tick would be pure churn.
|
|
return
|
|
write_manual_bank(log_to_manual_bank(merged))
|
|
client.put_file_text(
|
|
_device_manual_bank_path(identity.device_id),
|
|
json.dumps(
|
|
{record_id: record.to_dict() for record_id, record in merged.items()},
|
|
indent=2,
|
|
),
|
|
message="diet_guard sync",
|
|
)
|
|
|
|
|
|
def _sync_budget(client: GitHubSyncClient, device_ids: Sequence[str]) -> None:
|
|
"""Pull other devices' budgets, merge, write locally, push this device's.
|
|
|
|
Runs in the same tick as the food-log sync, reusing the already
|
|
authenticated ``client``. Merging is last-writer-wins by edit time (see
|
|
:mod:`diet_guard.sync_merge`'s budget adapters), not the food log's
|
|
union-of-immutable-entries -- a budget can be edited repeatedly. A
|
|
device that has never run ``init`` neither contributes a local record
|
|
to the merge nor overwrites a real budget pulled from elsewhere, and if
|
|
*no* device has ever set one, nothing is written or pushed.
|
|
"""
|
|
identity = device_identity()
|
|
merged = budget_to_log(read_raw_record(), load_entries(), load_schedule_entries())
|
|
for device_id in device_ids:
|
|
if identity.is_own(device_id):
|
|
continue
|
|
text = client.get_file_text(_device_budget_path(device_id))
|
|
if text is None:
|
|
continue
|
|
try:
|
|
merged = merge_logs(merged, parse_remote_budget(text))
|
|
except (TypeError, KeyError, ValueError, json.JSONDecodeError):
|
|
_logger.warning(
|
|
"Unparsable budget pushed by device %r, skipping",
|
|
device_id,
|
|
)
|
|
|
|
merged_record = log_to_budget(merged)
|
|
if merged_record is None:
|
|
return
|
|
write_raw_record(merged_record)
|
|
merged_history = log_to_history(merged)
|
|
# Only write back when the merge actually carried history. A pre-feature
|
|
# peer contributes none, and persisting an empty document would look like
|
|
# "history already exists" to any presence-based check and stop the local
|
|
# seed from ever running.
|
|
if merged_history:
|
|
write_raw_history(history_to_json(merged_history))
|
|
# Same guard for the meal schedule: a pre-feature peer contributes no
|
|
# `sched:` fields, and writing the empty document back would discard this
|
|
# device's own history.
|
|
merged_schedule = log_to_schedule_history(merged)
|
|
if merged_schedule:
|
|
write_raw_schedule(schedule_history_to_json(merged_schedule))
|
|
|
|
push_json = json.dumps(
|
|
{record_id: record.to_dict() for record_id, record in merged.items()},
|
|
indent=2,
|
|
)
|
|
client.put_file_text(
|
|
_device_budget_path(identity.device_id),
|
|
push_json,
|
|
message="diet_guard sync",
|
|
)
|