utils/crdt_sync_dart/lib/src/github_client.dart
Krzysztof Rudnicki 02e2e13790 crdt_sync: add a Firebase Realtime Database transport behind a RemoteStore seam
Extracts the three-method remote surface GitHub was reached through into a
`RemoteStore` interface, then implements it against Firebase Realtime
Database over REST, so apps can move backend with a constructor change.

Why REST and not the SDKs: official FlutterFire has no Linux desktop
support, and `wake_alarm`'s PC side is a dependency-free systemd job. The
RTDB REST endpoints are plain HTTPS and behave identically on Linux,
Android and headless. Firebase is a transport, never the datastore -- each
app keeps its local log as the source of truth, so offline-first survives.

Why RTDB and not Firestore: on the Spark plan RTDB bills only storage and
bandwidth with no per-operation quota, so a runaway loop cannot exhaust a
daily budget and silently stop syncing mid-day.

Two savings make the no-cost tier a guarantee rather than a hope. Measured
against the GitHub-backed history this replaces: 4894 of 5542 commits
(88.3%) were byte-identical no-ops, and every tick re-downloaded every
peer's whole log. Both are now driven by one sha256 revision -- an
unchanged log is not re-pushed, and an unchanged peer is not re-fetched.
That turns egress from "ticks x log size" (~700 MB/month, and growing with
the log) into "edits" (~40 MB/month against a 10 GiB cap).

Revisions are one node per device, written only by its owner, rather than
one shared map -- a whole-map write would erase every peer's entry, after
which those peers would look permanently unchanged and never be fetched
again. Per-device keys make that impossible rather than merely avoided.

`.` is illegal in an RTDB key and the REST `.json` is a format suffix, so
paths get a reversible `~XX` escape (`log.json` -> `log~2Ejson`). Lossy
extension-stripping would break `todo`, whose listings return filenames.

`MirrorStore` / `MirrorSyncClient` dual-writes during the cutover. Reads
consult *both* backends: with primary-only reads a migrated device would
never see an un-migrated peer's writes while still mirroring its own back
-- silent one-directional convergence with no error raised. Writes are
asymmetric on purpose: a primary failure fails the tick, a mirror failure
is logged and survived.

Auth is Firebase email/password over identitytoolkit REST, with mandatory
refresh-token rotation. No service-account key anywhere: it would bypass
the security rules and is trivially extractable from an APK.

Verified live against a real database, not only mocks: sign-in and refresh,
key escaping round-tripping byte-for-byte, an unchanged tick writing
nothing, two devices converging, a revoked session throwing rather than
returning null, and a log written by the Python client being read by the
Dart one.

No app is migrated by this commit; every app still runs on GitHub.

Dart: 194 tests. Python: 203 tests at 100% branch coverage. No lint
suppressions added.
2026-08-03 18:14:25 +02:00

215 lines
7.1 KiB
Dart

import 'dart:convert';
import 'package:http/http.dart' as http;
import 'remote_store.dart';
const _apiBase = 'https://api.github.com';
const _httpNotFound = 404;
/// Raised for a GitHub API failure the caller must not silently ignore.
class GitHubSyncError extends RemoteSyncError {
GitHubSyncError(super.message);
}
/// Raised when the configured repo itself is unreachable.
///
/// Distinguished from a path-404 (nothing pushed to that path yet, which is
/// benign -- it just means no other device has synced before) so the caller
/// can tell "the repo name is wrong or the token isn't scoped to it" apart
/// from "no other device has synced yet".
///
/// Extends [GitHubSyncError] so existing `on GitHubSyncError` catches still
/// cover it, and implements [RemoteNotFoundError] so backend-neutral callers
/// can catch "the remote itself is missing" without naming GitHub. Dart's
/// single inheritance is why one is `extends` and the other `implements`.
class RepoNotFoundError extends GitHubSyncError implements RemoteNotFoundError {
RepoNotFoundError(super.message);
}
/// Minimal GitHub Contents API client used as dumb file storage.
///
/// GitHub is used purely as file storage via the REST Contents API, not a
/// git clone -- there is no working tree and no git-level merge; the only
/// merge is the domain-level one in `mergeLogs`. Mirrors
/// `crdt_sync._github.GitHubSyncClient` on the Python side: unlike `todo`'s
/// original `GitHubClient`, [listDirectory] returns every entry name
/// regardless of type (not just files), because `crdt_sync`'s
/// `<pathPrefix>/<deviceId>/<filename>` layout needs to discover device
/// *subdirectories*; and [putFileText] resolves its own current SHA rather
/// than requiring the caller to track it.
class GitHubClient implements RemoteStore {
GitHubClient({
required this.owner,
required this.repo,
required String token,
http.Client? httpClient,
}) // Dart forbids private named params, so this can't be an initializing
// formal; assign it explicitly.
// ignore: prefer_initializing_formals
: _token = token,
_http = httpClient ?? http.Client();
final String owner;
final String repo;
final String _token;
final http.Client _http;
Map<String, String> get _headers => {
'Authorization': 'Bearer $_token',
'Accept': 'application/vnd.github+json',
};
Uri _contentsUri(String path) =>
Uri.parse('$_apiBase/repos/$owner/$repo/contents/$path');
Future<http.Response> _get(String path) async {
try {
return await _http.get(_contentsUri(path), headers: _headers);
} on http.ClientException catch (exc) {
throw GitHubSyncError('network error reading $path: $exc');
}
}
Future<bool> _repoExists() async {
try {
final res = await _http.get(
Uri.parse('$_apiBase/repos/$owner/$repo'),
headers: _headers,
);
return res.statusCode >= 200 && res.statusCode < 300;
} on http.ClientException {
return false;
}
}
Future<void> _raiseForMissingPath(String path) async {
if (!await _repoExists()) {
throw RepoNotFoundError(
'$owner/$repo not found, private without access, or the token '
'lacks contents permission (while reading $path)',
);
}
}
/// Returns the entry names directly under [path] (empty if unused).
///
/// Includes both files and subdirectories -- see the class doc for why
/// this differs from a plain "list files" helper.
@override
Future<List<String>> listDirectory(String path) async {
final res = await _get(path);
if (res.statusCode == _httpNotFound) {
await _raiseForMissingPath(path);
return [];
}
if (res.statusCode < 200 || res.statusCode >= 300) {
throw GitHubSyncError('GET $path (list) failed: ${res.statusCode}');
}
final decoded = jsonDecode(res.body);
if (decoded is! List) return [];
return decoded
.whereType<Map<String, dynamic>>()
.map((e) => e['name'])
.whereType<String>()
.toList();
}
/// Returns the decoded text content at [path], or null if unused.
@override
Future<String?> getFileText(String path) async {
final res = await _get(path);
if (res.statusCode == _httpNotFound) {
await _raiseForMissingPath(path);
return null;
}
if (res.statusCode < 200 || res.statusCode >= 300) {
throw GitHubSyncError('GET $path failed: ${res.statusCode}');
}
final decoded = jsonDecode(res.body) as Map<String, dynamic>;
final content = (decoded['content'] as String? ?? '').replaceAll('\n', '');
return utf8.decode(base64.decode(content));
}
Future<String?> _existingSha(String path) async {
final res = await _get(path);
if (res.statusCode == _httpNotFound) {
await _raiseForMissingPath(path);
return null;
}
if (res.statusCode < 200 || res.statusCode >= 300) {
throw GitHubSyncError('GET $path (for sha) failed: ${res.statusCode}');
}
final decoded = jsonDecode(res.body) as Map<String, dynamic>;
return decoded['sha'] as String?;
}
/// Creates or updates the file at [path] with [text].
@override
Future<void> putFileText(
String path,
String text, {
required String message,
}) async {
final sha = await _existingSha(path);
final body = <String, dynamic>{
'message': message,
'content': base64.encode(utf8.encode(text)),
'sha': ?sha,
};
late final http.Response res;
try {
res = await _http.put(
_contentsUri(path),
headers: _headers,
body: jsonEncode(body),
);
} on http.ClientException catch (exc) {
throw GitHubSyncError('network error pushing $path: $exc');
}
if (res.statusCode < 200 || res.statusCode >= 300) {
throw GitHubSyncError('PUT $path failed: ${res.statusCode}');
}
}
/// Returns whether the configured token can read this repo.
///
/// A lightweight connection test for a settings "Test connection" button: it
/// hits the bare repo endpoint, so it succeeds even before any file has been
/// pushed. Never throws -- a network failure or missing repo returns false.
///
/// The GitHub-era name for [canAccessRemote]; kept because app settings
/// screens call it. Both do the same probe.
Future<bool> canAccessRepo() => _repoExists();
@override
Future<bool> canAccessRemote() => _repoExists();
/// Deletes the file at [path], resolving its current sha itself so callers
/// don't have to track it. A no-op if [path] does not exist.
@override
Future<void> deleteFile(
String path, {
String message = 'crdt_sync: delete',
}) async {
final sha = await _existingSha(path);
if (sha == null) return;
late final http.Response res;
try {
res = await _http.delete(
_contentsUri(path),
headers: _headers,
body: jsonEncode({'message': message, 'sha': sha}),
);
} on http.ClientException catch (exc) {
throw GitHubSyncError('network error deleting $path: $exc');
}
if (res.statusCode < 200 || res.statusCode >= 300) {
throw GitHubSyncError('DELETE $path failed: ${res.statusCode}');
}
}
@override
void close() => _http.close();
}