Commit Graph

11 Commits

Author SHA1 Message Date
f902bbe0aa Let a mirror read survive the primary being down
Some checks failed
crdt-sync pre-commit / pre-commit (push) Has been cancelled
crdt-sync tests / test (3.10) (push) Has been cancelled
crdt-sync tests / test (3.11) (push) Has been cancelled
crdt-sync tests / test (3.12) (push) Has been cancelled
MirrorSyncClient promised that reads consult both backends "so the
cutover order is irrelevant", but every read called self.primary
unguarded. A Firebase outage therefore produced no data at all and the
GitHub mirror was never consulted -- the union read silently degraded to
nothing, precisely when the fallback existed to help.

This is not hypothetical: screen-locker's workout sync hit it on
2026-08-10, where a DNS failure reading the primary took down the whole
locker rather than falling back.

Reads (list_directory, get_file_text, get_string_map) now degrade to the
mirror when the primary raises, and only fail when NEITHER backend can
answer -- returning an empty result would be indistinguishable from "no
data", which callers act on. Writes keep the old asymmetry on purpose: a
write the primary did not accept has not happened, so it must still fail
the tick.

A mirror that cannot bulk-read (GitHub) contributes nothing but is not a
failure, so it cannot by itself make a revision-map read raise. Both
language ports agree on that.

Same fix in the Dart port, which had all three read paths wrong too and
runs on the phone. Two older tests asserted the buggy read-fail-closed
behaviour and are renamed to describe what they actually exercise: the
both-sides-failed case.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JS5V44jkHZiH1wC7BH5h5n
2026-08-10 14:30:29 +02:00
6a080333c2 Let a desktop wrapper provision the sync account, no UI required
Desktop builds of todo/diet-guard/home_inventory are Flutter *web* in a
Chrome --app window, so FlutterSecureStorage resolves to the web backend:
AES-encrypted localStorage keyed inside the browser profile. That makes the
account unsettable from outside the page -- `secret-tool` writes the wrong
store (libsecret, which only the Linux build would read), and a Dart CLI
cannot even import flutter_secure_storage, since it is a Flutter plugin and
fails to resolve on the VM. Verified both, rather than assumed.

The one process that can read the filesystem *and* reach the page is the
wrapper already serving it -- the same seam diet-guard uses to keep the
GitHub token out of the browser. So this adds the shared half of a wrapper
route: the path constant, the opt-in env var, an opt-out marker key, and
accountFromWrapper(), which fetches and hands the body to the existing
FirebaseAccount.tryParse.

Additive only, so consumers adopt it per-app on their own schedule; nothing
forces a phone rebuild. Route handlers stay per-app because each wrapper owns
its own server.

Deliberately opt-in (CRDT_SYNC_SERVE_ACCOUNT): the route hands a credential
with database write access to anything that can reach the wrapper's port, so
it is set once to provision an install rather than left on. kSyncAccountOptOutKey
exists so "Disconnect" cannot be silently undone by the next launch
re-adopting the account.

Every failure path returns null rather than throwing -- an absent route is
the *normal* case -- so the caller degrades to "not configured", which the
settings screens already handle by asking.

254 tests, analyzer clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0177xwzcpfpWG2qca8Mr39q9
2026-08-09 21:33:22 +02:00
6eef99d661 Add DeviceIdentity and a legacy device id for the uuid migration
Some checks are pending
crdt-sync pre-commit / pre-commit (push) Waiting to run
crdt-sync tests / test (3.10) (push) Waiting to run
crdt-sync tests / test (3.11) (push) Waiting to run
crdt-sync tests / test (3.12) (push) Waiting to run
Groundwork for moving diet-guard, screen-locker and wake-alarm off fixed
role ids ('pc'/'phone'/'desktop') onto per-install uuids, the scheme todo
and home_inventory already use. A role constant collides the moment a
second machine takes the same role, and a reinstall silently inherits the
previous install's CRDT identity.

The migration cannot rewrite history: the device id is baked into every HLC
stamp already written, is the record path segment and the revision key. So
a device that has just switched ids must still recognise its former id as
its own -- otherwise skip-own-writes stops matching the old path, and the
device pulls back and re-merges its own pre-migration log as though a peer
had written it. Idempotent under LWW, but pure wasted transfer on every
tick.

Both halves therefore take an optional legacy id and test set membership
(own_ids / ownIds) rather than equality against a single id:

- crdt-sync (Python) 0.5.1 -> 0.6.0: new crdt_sync._device with
  DeviceIdentity + load_device_identity (generate-or-load, persisted); the
  mkdir is deliberately unguarded, since handing back an unpersisted id
  would mint a fresh uuid every restart and strand a device dir each time.
  sync_log gains legacy_device_id.
- crdt_sync_dart 0.6.0 -> 0.7.0: DeviceIdentity value type; syncLog gains
  legacyDeviceId. Persistence stays app-side, where the existing uuid apps
  already keep it in SharedPreferences under 'crdt.nodeId'.

Both changes are backward compatible: the parameter defaults to null, which
reproduces the previous single-id behaviour exactly.

Each half has a positive test and a negative control proving the old path
*is* pulled when no legacy id is declared -- so the skip is verified to be
the parameter's doing, not an artifact of the fixture.

Python: 239 tests, 100% branch coverage. Dart: 248 tests, analyzer clean,
device.dart 11/11 and sync.dart 57/57 lines covered.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZH89WEqAjuJT8WZdqc7AP
2026-08-09 16:54:33 +02:00
fc99eff005 crdt_sync_dart 0.6.0: split Firebase settings by what is safe to publish
Some checks are pending
crdt-sync pre-commit / pre-commit (push) Waiting to run
crdt-sync tests / test (3.10) (push) Waiting to run
crdt-sync tests / test (3.11) (push) Waiting to run
crdt-sync tests / test (3.12) (push) Waiting to run
Every repo in this fleet is public, so where each value lives is a disclosure
decision rather than a style choice.

FirebaseProject holds apiKey + databaseUrl: these identify the *project*, and
the Web API key is a public identifier by design -- it ships inside every
Android APK, and the security rules, not the key, are what protect the data.
Safe to bake into app source, which keeps a reinstall zero-setup.

FirebaseAccount holds the email and password: these identify *a person*, so
they are entered once per device and kept in the platform keystore, never in
SharedPreferences and never in a repo.

The uid is deliberately absent from both. Nothing client-side needs it -- the
rules compare auth.uid server-side, so a wrong account is refused by the
database. A local copy would add a way to be wrong without adding protection.

Keyed at `firebase.account`, distinct from `sync.token` (the GitHub PAT) and
`firebase.credentials` (the refresh token), so all three coexist during the
mirrored cutover.

13 tests, 100% line coverage; 241 in the suite.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SJ9sxEw3ymzXNjL7SHNpJc
2026-08-04 23:05:13 +02:00
6c9cc71ae1 crdt_sync_dart 0.5.1: a shared secure credential store
Every Flutter consumer already keeps its GitHub token in
flutter_secure_storage, which is where a Firebase refresh token belongs too:
it is the long-lived secret, while ID tokens expire in about an hour.

Takes read/write/delete closures rather than depending on
flutter_secure_storage directly -- this package is pure Dart, and a Flutter
plugin dependency would make it unusable from the command-line tools and from
plain `dart test`. The adapter is two lines at each call site.

Keyed at `firebase.credentials`, namespaced away from each app's `sync.token`
so both secrets coexist for the whole mirrored cutover.

Every load failure degrades to "not signed in" rather than throwing: an app
that threw here would be permanently unable to sync, with no way back short of
a reinstall.

10 tests, 100% line coverage; 228 in the suite.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SJ9sxEw3ymzXNjL7SHNpJc
2026-08-04 23:01:54 +02:00
dd6e6d5571 crdt_sync 0.5.0: the shared config and the cutover seam
Everything an app needs to move from GitHub to Firebase without each one
reinventing the wiring.

Python:
- _config.py: FirebaseConfig.load() + firebase_client_for(app) +
  mirror_client_for(app, github). One config file, read in one place.
  Credential caches stay per-app (~/.config/<app>/firebase_auth.json) --
  one file written by several processes is a corruption waiting to happen.
- tool/preflight_firebase.py: validates the config before any data moves.
  Catches a wrong-region databaseUrl (which answers 404 with a correctUrl
  body rather than an error, so it would otherwise pass every check and
  break mid-migration) and confirms the uid in the ID token matches the uid
  the rules pin -- sign_in() returns None and discards localId, so the uid
  is decoded from the JWT.
- tool/interop_seed.py: the missing Python half of the cross-language
  check. interop_check.dart asserts it can see a record written by Python,
  but nothing ever wrote one, so it could only ever fail.

Dart:
- firebase_config.dart: the same config shape, kept free of dart:io so it
  works on web and Android, where the file does not exist at all and
  credentials belong in flutter_secure_storage.
- PersistedSyncStateStore: the revision cache, persisted. Without it a
  phone forgets every peer revision on each cold start and re-downloads
  everything -- the exact traffic the revision scheme exists to avoid, and
  what the free-tier headroom depends on.

Verified live against the real database: preflight passes all five checks,
the Dart smoke test passes end to end, and Python <-> Dart interop is OK.

Python 226 tests at 100% branch coverage; Dart 218 tests, both new files
at 100% line coverage.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SJ9sxEw3ymzXNjL7SHNpJc
2026-08-04 22:59:48 +02:00
02e2e13790 crdt_sync: add a Firebase Realtime Database transport behind a RemoteStore seam
Extracts the three-method remote surface GitHub was reached through into a
`RemoteStore` interface, then implements it against Firebase Realtime
Database over REST, so apps can move backend with a constructor change.

Why REST and not the SDKs: official FlutterFire has no Linux desktop
support, and `wake_alarm`'s PC side is a dependency-free systemd job. The
RTDB REST endpoints are plain HTTPS and behave identically on Linux,
Android and headless. Firebase is a transport, never the datastore -- each
app keeps its local log as the source of truth, so offline-first survives.

Why RTDB and not Firestore: on the Spark plan RTDB bills only storage and
bandwidth with no per-operation quota, so a runaway loop cannot exhaust a
daily budget and silently stop syncing mid-day.

Two savings make the no-cost tier a guarantee rather than a hope. Measured
against the GitHub-backed history this replaces: 4894 of 5542 commits
(88.3%) were byte-identical no-ops, and every tick re-downloaded every
peer's whole log. Both are now driven by one sha256 revision -- an
unchanged log is not re-pushed, and an unchanged peer is not re-fetched.
That turns egress from "ticks x log size" (~700 MB/month, and growing with
the log) into "edits" (~40 MB/month against a 10 GiB cap).

Revisions are one node per device, written only by its owner, rather than
one shared map -- a whole-map write would erase every peer's entry, after
which those peers would look permanently unchanged and never be fetched
again. Per-device keys make that impossible rather than merely avoided.

`.` is illegal in an RTDB key and the REST `.json` is a format suffix, so
paths get a reversible `~XX` escape (`log.json` -> `log~2Ejson`). Lossy
extension-stripping would break `todo`, whose listings return filenames.

`MirrorStore` / `MirrorSyncClient` dual-writes during the cutover. Reads
consult *both* backends: with primary-only reads a migrated device would
never see an un-migrated peer's writes while still mirroring its own back
-- silent one-directional convergence with no error raised. Writes are
asymmetric on purpose: a primary failure fails the tick, a mirror failure
is logged and survived.

Auth is Firebase email/password over identitytoolkit REST, with mandatory
refresh-token rotation. No service-account key anywhere: it would bypass
the security rules and is trivially extractable from an APK.

Verified live against a real database, not only mocks: sign-in and refresh,
key escaping round-tripping byte-for-byte, an unchanged tick writing
nothing, two devices converging, a revoked session throwing rather than
returning null, and a log written by the Python client being read by the
Dart one.

No app is migrated by this commit; every app still runs on GitHub.

Dart: 194 tests. Python: 203 tests at 100% branch coverage. No lint
suppressions added.
2026-08-03 18:14:25 +02:00
5480e72335 GitHubClient: add canAccessRepo() + deleteFile() (v0.3.0)
Some checks failed
crdt-sync pre-commit / pre-commit (push) Has been cancelled
crdt-sync tests / test (3.10) (push) Has been cancelled
crdt-sync tests / test (3.11) (push) Has been cancelled
crdt-sync tests / test (3.12) (push) Has been cancelled
Make the shared client a superset of what todo's and diet_guard-app's local
GitHubClient copies provide, so every app can drop its copy and sync through
one client. canAccessRepo() is a lightweight connection test (bare repo GET,
never throws); deleteFile() resolves the file's own sha and no-ops when
absent. Mirrored on the Dart and Python sides. 90 Dart + 78 Python tests, 100%.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019KwUCPESvE5CjCtmUFz8xQ
2026-07-11 23:41:10 +02:00
5be99ab11d Add LogStore local-persistence layer to crdt_sync (v0.2.0)
Some checks are pending
crdt-sync pre-commit / pre-commit (push) Waiting to run
crdt-sync tests / test (3.10) (push) Waiting to run
crdt-sync tests / test (3.11) (push) Waiting to run
crdt-sync tests / test (3.12) (push) Waiting to run
crdt_sync_dart: new domain-agnostic LogStore -- an in-memory Log with a
reactive changes stream and upsert/delete(sticky tombstone)/replaceAll,
behind an injectable LogPersistence port -- plus logToJson/logFromJson.
FileLogPersistence (atomic temp-file-then-rename, generalizing diet_guard's
LogStorageService) lives behind a separate crdt_sync_io.dart entrypoint so
the core barrel stays pure Dart / web-safe.

crdt-sync (Python): mirror dump_log/load_log/read_log/write_log sharing the
same on-disk JSON format so a log written on one side parses on the other.
Load/save only -- the reactive store is a Dart/Flutter convenience with no
Python equivalent by design (Python consumers are headless sync ticks).

Querying stays app-side: a Record's fields are opaque to the library, so
filtering/sorting/search is the caller's job over LogStore.values.

Both packages bumped to 0.2.0. 83 Dart tests + 71 Python tests (100% branch
coverage) green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019KwUCPESvE5CjCtmUFz8xQ
2026-07-11 22:19:34 +02:00
6b14e6d36d Fix wire-format key mismatch with Python (deletedHlc -> deleted_hlc)
Record.toJson() was writing the delete-clock field as deletedHlc while
the Python package's Record.to_dict() writes deleted_hlc -- an
undetected cross-language wire divergence: a tombstoned record synced
Python<->Dart would silently lose its delete clock on the receiving
side. Also widen syncLog's decode error handling from FormatException
alone to also catch TypeError, matching the Python side's
(ValueError, KeyError, TypeError) so a valid-JSON-but-wrong-shape push
from an incompatible writer is skipped per-device instead of aborting
the whole sync. Adds a fixture test shared verbatim with the Python
package's test suite so this class of divergence fails loudly going
forward.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017McuA1nAi6jBT1EaS5BrVf
2026-07-05 20:55:22 +02:00
4b66703280 Initial implementation: unified CRDT merge scheme + GitHub sync transport
Pure-Dart mirror of the Python crdt-sync package, module-for-module and
test-for-test, so the merge algorithm canonically agrees on both sides of
a sync between a Dart client and a Python one. No Flutter SDK dependency
(package:http only) per Workstream D's scoping decision (2026-07-04).
59 tests, clean dart analyze.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017McuA1nAi6jBT1EaS5BrVf
2026-07-05 20:48:07 +02:00