mirror of
https://github.com/kuhyx/utils.git
synced 2026-11-10 22:07:15 +01:00
Krzysztof Rudnicki
02e2e13790
Extracts the three-method remote surface GitHub was reached through into a `RemoteStore` interface, then implements it against Firebase Realtime Database over REST, so apps can move backend with a constructor change. Why REST and not the SDKs: official FlutterFire has no Linux desktop support, and `wake_alarm`'s PC side is a dependency-free systemd job. The RTDB REST endpoints are plain HTTPS and behave identically on Linux, Android and headless. Firebase is a transport, never the datastore -- each app keeps its local log as the source of truth, so offline-first survives. Why RTDB and not Firestore: on the Spark plan RTDB bills only storage and bandwidth with no per-operation quota, so a runaway loop cannot exhaust a daily budget and silently stop syncing mid-day. Two savings make the no-cost tier a guarantee rather than a hope. Measured against the GitHub-backed history this replaces: 4894 of 5542 commits (88.3%) were byte-identical no-ops, and every tick re-downloaded every peer's whole log. Both are now driven by one sha256 revision -- an unchanged log is not re-pushed, and an unchanged peer is not re-fetched. That turns egress from "ticks x log size" (~700 MB/month, and growing with the log) into "edits" (~40 MB/month against a 10 GiB cap). Revisions are one node per device, written only by its owner, rather than one shared map -- a whole-map write would erase every peer's entry, after which those peers would look permanently unchanged and never be fetched again. Per-device keys make that impossible rather than merely avoided. `.` is illegal in an RTDB key and the REST `.json` is a format suffix, so paths get a reversible `~XX` escape (`log.json` -> `log~2Ejson`). Lossy extension-stripping would break `todo`, whose listings return filenames. `MirrorStore` / `MirrorSyncClient` dual-writes during the cutover. Reads consult *both* backends: with primary-only reads a migrated device would never see an un-migrated peer's writes while still mirroring its own back -- silent one-directional convergence with no error raised. Writes are asymmetric on purpose: a primary failure fails the tick, a mirror failure is logged and survived. Auth is Firebase email/password over identitytoolkit REST, with mandatory refresh-token rotation. No service-account key anywhere: it would bypass the security rules and is trivially extractable from an APK. Verified live against a real database, not only mocks: sign-in and refresh, key escaping round-tripping byte-for-byte, an unchanged tick writing nothing, two devices converging, a revoked session throwing rather than returning null, and a log written by the Python client being read by the Dart one. No app is migrated by this commit; every app still runs on GitHub. Dart: 194 tests. Python: 203 tests at 100% branch coverage. No lint suppressions added.
83 lines
3.2 KiB
Python
83 lines
3.2 KiB
Python
"""Backend-neutral remote-storage contract that sync talks through.
|
|
|
|
Deliberately tiny: :func:`crdt_sync.sync_log` and every app-level sync
|
|
service talk only to :class:`RemoteStore`, so swapping the storage backend
|
|
(GitHub Contents API, Firebase Realtime Database, a dual-writing mirror of
|
|
both) is a constructor change at the call site and nothing more.
|
|
|
|
Mirrors ``crdt_sync_dart``'s ``lib/src/remote_store.dart``; keep the two in
|
|
step.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from typing import Protocol, runtime_checkable
|
|
|
|
|
|
class RemoteSyncError(Exception):
|
|
"""Raised for a remote-storage failure the caller must not ignore.
|
|
|
|
The backend-neutral base of the error hierarchy: catch this to handle
|
|
"the sync transport failed" regardless of which backend is configured.
|
|
Backends narrow it -- :class:`crdt_sync.GitHubSyncError` and
|
|
:class:`crdt_sync.FirebaseSyncError` both derive from it -- so existing
|
|
``except GitHubSyncError`` handlers keep working while a caller that has
|
|
been migrated can catch the base type instead.
|
|
"""
|
|
|
|
|
|
class RemoteNotFoundError(RemoteSyncError):
|
|
"""Raised when the configured remote itself is unreachable.
|
|
|
|
Distinguished from a missing *path* (nothing pushed there yet, which is
|
|
benign -- it just means no other device has synced before) so the caller
|
|
can tell "the repo/database is wrong or the credential isn't scoped to
|
|
it" apart from "no other device has synced yet".
|
|
"""
|
|
|
|
|
|
@runtime_checkable
|
|
class RemoteStore(Protocol):
|
|
"""Dumb keyed storage of UTF-8 text blobs, with directory listing.
|
|
|
|
Implementations raise :class:`RemoteSyncError` (or a subclass) for any
|
|
failure the caller must not silently ignore, and treat a missing path as
|
|
a benign ``None`` / empty list rather than an error.
|
|
|
|
A ``Protocol`` rather than an ABC so the existing
|
|
:class:`crdt_sync.GitHubSyncClient` satisfies it structurally, with no
|
|
base class to add and no change to its own tests.
|
|
"""
|
|
|
|
def list_directory(self, path: str) -> list[str]:
|
|
"""Return the entry names directly under ``path``.
|
|
|
|
Returns an empty list when nothing has been written there yet.
|
|
Includes both files and subdirectories: the
|
|
``<path_prefix>/<device_id>/<filename>`` layout needs to discover
|
|
device *directories*, not just files.
|
|
"""
|
|
|
|
def get_file_text(self, path: str) -> str | None:
|
|
"""Return the text stored at ``path``, or ``None`` if absent."""
|
|
|
|
def put_file_text(self, path: str, text: str, *, message: str) -> None:
|
|
"""Create or replace the text at ``path``.
|
|
|
|
``message`` is a human-readable reason for the write. Backends that
|
|
record one (GitHub commits) use it; backends that do not (Firebase)
|
|
ignore it.
|
|
"""
|
|
|
|
def delete_file(self, path: str, *, message: str = ...) -> None:
|
|
"""Delete ``path``. A no-op when ``path`` does not exist."""
|
|
|
|
def can_access_remote(self) -> bool:
|
|
"""Return whether the configured credential can reach the remote.
|
|
|
|
A lightweight connection test for a settings "Test connection"
|
|
button: it probes the remote root, so it succeeds even before any
|
|
file has been pushed. Never raises -- a network failure or missing
|
|
remote returns ``False``.
|
|
"""
|