Serialise the relay's log writers

The narrow pull, the log write and the publish each read, merge and write
the food log. Run at once (a publish in flight while a second request
logs) they lose the append: the phone is told "logged" and the only copy
is gone. The new race test lost an entry before this change (1 of 2) and
keeps both after it; the shared temp-file name of the log writer also
collided between threads.

One asyncio lock now covers the pull, "resolve slot + log" and the
publish. A request arriving during a publish waits for it (up to one full
tick); the model call stays outside the lock.

Also: a test that the relay unlocks the header diet_guard's devices store
wrote (same key, a blob sealed by one opens on the other), the skill
returns grams 0 for nothing to estimate (parse fails closed, nothing is
logged), and the doc's session note is corrected: the pull and publish
refresh the gate's session file as the CLI's publish does.

Failure paths: a lost append is now impossible in-process (fail closed by
waiting); the pull failure, locked key, estimator and cleanup paths are
unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WzxdWZHWP4KZ6x5TcXDBFC
This commit is contained in:
Krzysztof Rudnicki 2026-11-17 21:21:05 +01:00
parent 642467dc1d
commit e6322f7f3e
6 changed files with 164 additions and 17 deletions

View File

@ -2,7 +2,7 @@ name: diet_guard_app
description: "Companion phone app for diet_guard: log meals on the go." description: "Companion phone app for diet_guard: log meals on the go."
publish_to: 'none' publish_to: 'none'
version: 1.0.168+1 version: 1.0.169+1
environment: environment:
sdk: ^3.12.2 sdk: ^3.12.2

View File

@ -68,8 +68,11 @@ class MealHandler:
self._estimate = estimate self._estimate = estimate
self._now = now self._now = now
self._clock = clock self._clock = clock
# Two requests must not take the same due slot. #: The one lock of everything that reads-merges-writes the food log
self._log_lock = asyncio.Lock() #: (the pull, the log write, the publish): two of them at once lose an
#: append, and the phone has already been told "logged". Also keeps
#: two requests from taking the same due slot. Shared with the server.
self.log_lock = asyncio.Lock()
async def handle(self, body: dict[str, Any]) -> dict[str, Any]: async def handle(self, body: dict[str, Any]) -> dict[str, Any]:
"""Serve one request body (``rid`` and ``created`` injected by the server).""" """Serve one request body (``rid`` and ``created`` injected by the server)."""
@ -87,7 +90,7 @@ class MealHandler:
estimate = await self._estimate(request.text, request.photo) estimate = await self._estimate(request.text, request.photo)
except EstimateError as err: except EstimateError as err:
raise RequestError(err.code, str(err), retry=True) from err raise RequestError(err.code, str(err), retry=True) from err
async with self._log_lock: async with self.log_lock:
return await asyncio.to_thread( return await asyncio.to_thread(
log_estimate, rid, estimate, request, self._now() log_estimate, rid, estimate, request, self._now()
) )
@ -110,13 +113,15 @@ class MealServer(RequestServer):
self.tree: dict[str, Any] = {} self.tree: dict[str, Any] = {}
self._retry: asyncio.Task[None] | None = None self._retry: asyncio.Task[None] | None = None
self._collecting: set[str] = set() self._collecting: set[str] = set()
self._publish_lock = asyncio.Lock() self._log_lock = handler.log_lock
async def answer(self, rid: str, node: dict[str, Any]) -> dict[str, Any] | None: async def answer(self, rid: str, node: dict[str, Any]) -> dict[str, Any] | None:
"""Pull, serve, then clean up and publish; None when not ours to answer.""" """Pull, serve, then clean up and publish; None when not ours to answer."""
if not valid_rid(rid) or "result" in node or "req" not in node: if not valid_rid(rid) or "result" in node or "req" not in node:
return None return None
if (reason := await asyncio.to_thread(_claude_sync.pull_peers)) is not None: async with self._log_lock:
reason = await asyncio.to_thread(_claude_sync.pull_peers)
if reason is not None:
_logger.warning("not serving %s, request stays pending: %s", rid, reason) _logger.warning("not serving %s, request stays pending: %s", rid, reason)
self._retry_later() self._retry_later()
return None return None
@ -155,7 +160,7 @@ class MealServer(RequestServer):
async def _publish(self) -> None: async def _publish(self) -> None:
"""The full sync tick, after the answer; an outage is only logged.""" """The full sync tick, after the answer; an outage is only logged."""
async with self._publish_lock: async with self._log_lock:
reason = await asyncio.to_thread(_claude_sync.publish) reason = await asyncio.to_thread(_claude_sync.publish)
if reason is not None: if reason is not None:
_logger.warning("post-answer sync skipped: %s", reason) _logger.warning("post-answer sync skipped: %s", reason)

View File

@ -0,0 +1,68 @@
"""The relay opens the header diet_guard's own sync already wrote.
The contract rests on "same header, same key": the phone unlocks nothing new.
This builds the header the way production did (``open_encrypted_client`` over
the old devices prefix), hands its text to the relay's RTDB, and requires the
relay's sealer to read and write blobs the devices store's key opens.
"""
from __future__ import annotations
import asyncio
from crdt_sync import open_encrypted_client
from rtdb_relay.seal import Sealer
from rtdb_relay.tests._fake_rtdb import FakeRtdb
from diet_guard import _claude_connect, _sync_e2e
from diet_guard._claude_layout import LAYOUT
from diet_guard.tests._state_redirects import TEST_ARGON2, TEST_PHRASE
class _MemoryStore:
"""A dict-backed ``RemoteStore``: just enough for the key header."""
def __init__(self) -> None:
self.files: dict[str, str] = {}
def list_directory(self, path: str) -> list[str]:
del path
return []
def get_file_text(self, path: str) -> str | None:
return self.files.get(path)
def put_file_text(self, path: str, text: str, *, message: str) -> None:
del message
self.files[path] = text
def delete_file(self, path: str, *, message: str = "") -> None:
del message
self.files.pop(path, None)
def can_access_remote(self) -> bool:
return True
def test_relay_unlocks_the_header_the_devices_store_wrote() -> None:
"""One header, one key: a blob either side seals opens on the other."""
memory = _MemoryStore()
devices, layout = open_encrypted_client(
memory,
_sync_e2e.LEGACY_DEVICES_DIR,
passphrase=TEST_PHRASE,
params=TEST_ARGON2,
)
header = memory.files[layout.e2e_path]
fake = FakeRtdb(sealed=False)
fake.raw_set(layout.e2e_path, header)
rtdb = asyncio.run(_claude_connect.unlock_sealed(fake.rtdb()))
assert fake.raw_get(layout.e2e_path) == header # read, never rewritten
assert rtdb.sealer is not None
path = "requests/01J0000000000000000000000R/req"
sealed = rtdb.sealer.seal(path, {"kind": "meal", "created": 5})
devices_view = Sealer(devices.key.cipher(), LAYOUT)
devices_view.root = rtdb.sealer.root
assert devices_view.open(path, sealed) == {"kind": "meal", "created": 5}

View File

@ -0,0 +1,66 @@
"""Two close requests must never lose an entry to a pull or a publish.
The narrow pull and the full tick each read the food log, merge and write it
back; ``log_meal_as`` appends to it. Two of those at once lose the append: the
phone is told "logged" and the only copy is gone. conftest replaces pull and
publish with mocks, so this test gives them the read-pause-write shape of the
real ones.
"""
from __future__ import annotations
import asyncio
import time
from typing import TYPE_CHECKING, Any
from unittest.mock import AsyncMock
from diet_guard import _claude_sync
from diet_guard._state_sync import read_raw_log, write_raw_log
from diet_guard.tests._claude_worker_fakes import (
BODY,
EST,
RID,
RID2,
entries,
new_server,
put,
request_fake,
)
if TYPE_CHECKING:
from diet_guard._claude_schema import Estimate
def _merge_like_a_sync_tick(*_args: object) -> None:
"""Read the log, take a while merging, write it back: what a tick does."""
log = read_raw_log()
time.sleep(0.2)
write_raw_log(log)
async def _slow_for_b(text: str | None, photo: bytes | None) -> Estimate:
"""The second request's estimate finishes while the first one publishes."""
del photo
if text == "slow":
await asyncio.sleep(0.05)
return EST
def test_a_publish_in_flight_does_not_swallow_the_next_request() -> None:
"""Both entries survive when request B logs during request A's publish."""
_claude_sync.pull_peers.side_effect = _merge_like_a_sync_tick
_claude_sync.publish.side_effect = _merge_like_a_sync_tick
fake = request_fake({**BODY, "text": "fast"}, rid=RID)
put(fake, RID2, {**BODY, "text": "slow"})
async def go() -> None:
db = fake.rtdb()
server = new_server(db, AsyncMock(side_effect=_slow_for_b))
tree: dict[str, Any] = await server._raw.get("requests")
server.dispatch(tree)
while server.tasks:
await asyncio.gather(*list(server.tasks))
await db.aclose()
asyncio.run(go())
assert len(entries()) == 2

View File

@ -162,8 +162,13 @@ PC: claim (ETag lease) -> validate -> age check -> already logged? -> │
reaches it gets `expired` and nothing is logged; its `req` is deleted reaches it gets `expired` and nothing is logged; its `req` is deleted
like any other. like any other.
6. **Blocking work** (log file, sync, Claude) never runs on the event loop. 6. **Blocking work** (log file, sync, Claude) never runs on the event loop.
One lock serialises "resolve slot + `log_meal`" so two requests do not **One lock** covers everything that reads, merges and writes the food log:
take the same due slot. the narrow pull, "resolve slot + `log_meal_as`", and the publish. Two of
them at once lose an append (a publish writing back a log read before a
second request's entry) and the phone would already have been told
"logged". The price: a request that arrives during a publish waits for it,
up to one full tick (15 to 27 s) before its own pull. The model call
itself runs outside the lock.
## 6. The worker ## 6. The worker
@ -177,11 +182,13 @@ unit lives at the repo root with the other two, `install.sh` installs it).
node, so requests that arrived while the PC was off are served in order. node, so requests that arrived while the PC was off are served in order.
- Waits are only for faults: the e2e unlock and a failed pre-pull retry - Waits are only for faults: the e2e unlock and a failed pre-pull retry
every 60 s / 30 s while something is pending. No idle timer. every 60 s / 30 s while something is pending. No idle timer.
- Session: its own Firebase app name `diet_guard_claude` - Session: the RTDB stream uses its own Firebase app name `diet_guard_claude`
(`~/.config/diet_guard_claude/firebase_auth.json`), never the gate's (`~/.config/diet_guard_claude/firebase_auth.json`), so the stream never
`diet_guard` file (two processes refreshing one token file corrupt it). shares a refresh token file with another process. Seed once, see "Deploy".
Seed once, see "Deploy". The database URL and API key come from The pull and publish go through the ordinary sync client, so the worker
`~/.config/crdt-sync/firebase.json`; the passphrase from also refreshes the gate's `~/.config/diet_guard/firebase_auth.json`,
exactly as the CLI's own publish does. The database URL and API key come
from `~/.config/crdt-sync/firebase.json`; the passphrase from
`~/.config/crdt-sync/password`. `~/.config/crdt-sync/password`.
- One worker per PC. Claim leases last 300 s and renew every 60 s. - One worker per PC. Claim leases last 300 s and renew every 60 s.
- Dependencies (`rtdb_relay`, `httpx`, `claude_agent_sdk`) are imported only - Dependencies (`rtdb_relay`, `httpx`, `claude_agent_sdk`) are imported only

View File

@ -31,9 +31,10 @@ The user message holds any of:
oil, sauces and dressings included: they are most of the hidden kcal). oil, sauces and dressings included: they are most of the hidden kcal).
4. Package or label visible: read the per-100 g values and use them. 4. Package or label visible: read the per-100 g values and use them.
5. Sum the items. Round grams to 5, kcal to 5, macros to 0.5. 5. Sum the items. Round grams to 5, kcal to 5, macros to 0.5.
6. Never refuse and never ask a question. A vague meal still gets the best 6. Never ask a question. A vague meal still gets the best single estimate.
single estimate; a photo with no food in it gets a small guess from the When there is nothing to estimate (a photo with no food in it and no
text, or `grams` 100 and `kcal` 100 only if there is no text at all. text), answer `grams` 0: the caller treats that as a failure and logs
nothing. Never invent a meal.
## Output ## Output