Serialise the relay's log writers

The narrow pull, the log write and the publish each read, merge and write
the food log. Run at once (a publish in flight while a second request
logs) they lose the append: the phone is told "logged" and the only copy
is gone. The new race test lost an entry before this change (1 of 2) and
keeps both after it; the shared temp-file name of the log writer also
collided between threads.

One asyncio lock now covers the pull, "resolve slot + log" and the
publish. A request arriving during a publish waits for it (up to one full
tick); the model call stays outside the lock.

Also: a test that the relay unlocks the header diet_guard's devices store
wrote (same key, a blob sealed by one opens on the other), the skill
returns grams 0 for nothing to estimate (parse fails closed, nothing is
logged), and the doc's session note is corrected: the pull and publish
refresh the gate's session file as the CLI's publish does.

Failure paths: a lost append is now impossible in-process (fail closed by
waiting); the pull failure, locked key, estimator and cleanup paths are
unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WzxdWZHWP4KZ6x5TcXDBFC
This commit is contained in:
Krzysztof Rudnicki 2026-11-17 21:21:05 +01:00
parent 642467dc1d
commit e6322f7f3e
6 changed files with 164 additions and 17 deletions

View File

@ -2,7 +2,7 @@ name: diet_guard_app
description: "Companion phone app for diet_guard: log meals on the go."
publish_to: 'none'
version: 1.0.168+1
version: 1.0.169+1
environment:
sdk: ^3.12.2

View File

@ -68,8 +68,11 @@ class MealHandler:
self._estimate = estimate
self._now = now
self._clock = clock
# Two requests must not take the same due slot.
self._log_lock = asyncio.Lock()
#: The one lock of everything that reads-merges-writes the food log
#: (the pull, the log write, the publish): two of them at once lose an
#: append, and the phone has already been told "logged". Also keeps
#: two requests from taking the same due slot. Shared with the server.
self.log_lock = asyncio.Lock()
async def handle(self, body: dict[str, Any]) -> dict[str, Any]:
"""Serve one request body (``rid`` and ``created`` injected by the server)."""
@ -87,7 +90,7 @@ class MealHandler:
estimate = await self._estimate(request.text, request.photo)
except EstimateError as err:
raise RequestError(err.code, str(err), retry=True) from err
async with self._log_lock:
async with self.log_lock:
return await asyncio.to_thread(
log_estimate, rid, estimate, request, self._now()
)
@ -110,13 +113,15 @@ class MealServer(RequestServer):
self.tree: dict[str, Any] = {}
self._retry: asyncio.Task[None] | None = None
self._collecting: set[str] = set()
self._publish_lock = asyncio.Lock()
self._log_lock = handler.log_lock
async def answer(self, rid: str, node: dict[str, Any]) -> dict[str, Any] | None:
"""Pull, serve, then clean up and publish; None when not ours to answer."""
if not valid_rid(rid) or "result" in node or "req" not in node:
return None
if (reason := await asyncio.to_thread(_claude_sync.pull_peers)) is not None:
async with self._log_lock:
reason = await asyncio.to_thread(_claude_sync.pull_peers)
if reason is not None:
_logger.warning("not serving %s, request stays pending: %s", rid, reason)
self._retry_later()
return None
@ -155,7 +160,7 @@ class MealServer(RequestServer):
async def _publish(self) -> None:
"""The full sync tick, after the answer; an outage is only logged."""
async with self._publish_lock:
async with self._log_lock:
reason = await asyncio.to_thread(_claude_sync.publish)
if reason is not None:
_logger.warning("post-answer sync skipped: %s", reason)

View File

@ -0,0 +1,68 @@
"""The relay opens the header diet_guard's own sync already wrote.
The contract rests on "same header, same key": the phone unlocks nothing new.
This builds the header the way production did (``open_encrypted_client`` over
the old devices prefix), hands its text to the relay's RTDB, and requires the
relay's sealer to read and write blobs the devices store's key opens.
"""
from __future__ import annotations
import asyncio
from crdt_sync import open_encrypted_client
from rtdb_relay.seal import Sealer
from rtdb_relay.tests._fake_rtdb import FakeRtdb
from diet_guard import _claude_connect, _sync_e2e
from diet_guard._claude_layout import LAYOUT
from diet_guard.tests._state_redirects import TEST_ARGON2, TEST_PHRASE
class _MemoryStore:
"""A dict-backed ``RemoteStore``: just enough for the key header."""
def __init__(self) -> None:
self.files: dict[str, str] = {}
def list_directory(self, path: str) -> list[str]:
del path
return []
def get_file_text(self, path: str) -> str | None:
return self.files.get(path)
def put_file_text(self, path: str, text: str, *, message: str) -> None:
del message
self.files[path] = text
def delete_file(self, path: str, *, message: str = "") -> None:
del message
self.files.pop(path, None)
def can_access_remote(self) -> bool:
return True
def test_relay_unlocks_the_header_the_devices_store_wrote() -> None:
"""One header, one key: a blob either side seals opens on the other."""
memory = _MemoryStore()
devices, layout = open_encrypted_client(
memory,
_sync_e2e.LEGACY_DEVICES_DIR,
passphrase=TEST_PHRASE,
params=TEST_ARGON2,
)
header = memory.files[layout.e2e_path]
fake = FakeRtdb(sealed=False)
fake.raw_set(layout.e2e_path, header)
rtdb = asyncio.run(_claude_connect.unlock_sealed(fake.rtdb()))
assert fake.raw_get(layout.e2e_path) == header # read, never rewritten
assert rtdb.sealer is not None
path = "requests/01J0000000000000000000000R/req"
sealed = rtdb.sealer.seal(path, {"kind": "meal", "created": 5})
devices_view = Sealer(devices.key.cipher(), LAYOUT)
devices_view.root = rtdb.sealer.root
assert devices_view.open(path, sealed) == {"kind": "meal", "created": 5}

View File

@ -0,0 +1,66 @@
"""Two close requests must never lose an entry to a pull or a publish.
The narrow pull and the full tick each read the food log, merge and write it
back; ``log_meal_as`` appends to it. Two of those at once lose the append: the
phone is told "logged" and the only copy is gone. conftest replaces pull and
publish with mocks, so this test gives them the read-pause-write shape of the
real ones.
"""
from __future__ import annotations
import asyncio
import time
from typing import TYPE_CHECKING, Any
from unittest.mock import AsyncMock
from diet_guard import _claude_sync
from diet_guard._state_sync import read_raw_log, write_raw_log
from diet_guard.tests._claude_worker_fakes import (
BODY,
EST,
RID,
RID2,
entries,
new_server,
put,
request_fake,
)
if TYPE_CHECKING:
from diet_guard._claude_schema import Estimate
def _merge_like_a_sync_tick(*_args: object) -> None:
"""Read the log, take a while merging, write it back: what a tick does."""
log = read_raw_log()
time.sleep(0.2)
write_raw_log(log)
async def _slow_for_b(text: str | None, photo: bytes | None) -> Estimate:
"""The second request's estimate finishes while the first one publishes."""
del photo
if text == "slow":
await asyncio.sleep(0.05)
return EST
def test_a_publish_in_flight_does_not_swallow_the_next_request() -> None:
"""Both entries survive when request B logs during request A's publish."""
_claude_sync.pull_peers.side_effect = _merge_like_a_sync_tick
_claude_sync.publish.side_effect = _merge_like_a_sync_tick
fake = request_fake({**BODY, "text": "fast"}, rid=RID)
put(fake, RID2, {**BODY, "text": "slow"})
async def go() -> None:
db = fake.rtdb()
server = new_server(db, AsyncMock(side_effect=_slow_for_b))
tree: dict[str, Any] = await server._raw.get("requests")
server.dispatch(tree)
while server.tasks:
await asyncio.gather(*list(server.tasks))
await db.aclose()
asyncio.run(go())
assert len(entries()) == 2

View File

@ -162,8 +162,13 @@ PC: claim (ETag lease) -> validate -> age check -> already logged? -> │
reaches it gets `expired` and nothing is logged; its `req` is deleted
like any other.
6. **Blocking work** (log file, sync, Claude) never runs on the event loop.
One lock serialises "resolve slot + `log_meal`" so two requests do not
take the same due slot.
**One lock** covers everything that reads, merges and writes the food log:
the narrow pull, "resolve slot + `log_meal_as`", and the publish. Two of
them at once lose an append (a publish writing back a log read before a
second request's entry) and the phone would already have been told
"logged". The price: a request that arrives during a publish waits for it,
up to one full tick (15 to 27 s) before its own pull. The model call
itself runs outside the lock.
## 6. The worker
@ -177,11 +182,13 @@ unit lives at the repo root with the other two, `install.sh` installs it).
node, so requests that arrived while the PC was off are served in order.
- Waits are only for faults: the e2e unlock and a failed pre-pull retry
every 60 s / 30 s while something is pending. No idle timer.
- Session: its own Firebase app name `diet_guard_claude`
(`~/.config/diet_guard_claude/firebase_auth.json`), never the gate's
`diet_guard` file (two processes refreshing one token file corrupt it).
Seed once, see "Deploy". The database URL and API key come from
`~/.config/crdt-sync/firebase.json`; the passphrase from
- Session: the RTDB stream uses its own Firebase app name `diet_guard_claude`
(`~/.config/diet_guard_claude/firebase_auth.json`), so the stream never
shares a refresh token file with another process. Seed once, see "Deploy".
The pull and publish go through the ordinary sync client, so the worker
also refreshes the gate's `~/.config/diet_guard/firebase_auth.json`,
exactly as the CLI's own publish does. The database URL and API key come
from `~/.config/crdt-sync/firebase.json`; the passphrase from
`~/.config/crdt-sync/password`.
- One worker per PC. Claim leases last 300 s and renew every 60 s.
- Dependencies (`rtdb_relay`, `httpx`, `claude_agent_sdk`) are imported only

View File

@ -31,9 +31,10 @@ The user message holds any of:
oil, sauces and dressings included: they are most of the hidden kcal).
4. Package or label visible: read the per-100 g values and use them.
5. Sum the items. Round grams to 5, kcal to 5, macros to 0.5.
6. Never refuse and never ask a question. A vague meal still gets the best
single estimate; a photo with no food in it gets a small guess from the
text, or `grams` 100 and `kcal` 100 only if there is no text at all.
6. Never ask a question. A vague meal still gets the best single estimate.
When there is nothing to estimate (a photo with no food in it and no
text), answer `grams` 0: the caller treats that as a failure and logs
nothing. Never invent a meal.
## Output