Compare commits

..

47 Commits

Author SHA1 Message Date
3225af8365 crdt_sync_flutter 0.8.1, ssui 0.8.2: dart 0.14.1
Some checks are pending
crdt_sync_flutter tests / test (push) Waiting to run
dependency freshness / dependencies on newest stable (push) Waiting to run
file length / file-length (push) Waiting to run
md-naming / markdown naming convention (push) Waiting to run
palette drift / palette (push) Waiting to run
repo-contract / repo contract (push) Waiting to run
shared gate tests / gate unit tests (push) Waiting to run
sync_settings_ui tests / test (push) Waiting to run
Todo: fda71ed1

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 10:13:08 +01:00
6a40f7530b Python libs on crdt-sync 0.11.0
Some checks are pending
crdt_sync_dart tests / test (push) Waiting to run
crdt_sync_flutter tests / test (push) Waiting to run
crdt-sync pre-commit / pre-commit (push) Waiting to run
crdt-sync tests / test (push) Waiting to run
dependency freshness / dependencies on newest stable (push) Waiting to run
file length / file-length (push) Waiting to run
focus-project pre-commit / pre-commit (push) Waiting to run
focus-project tests / test (push) Waiting to run
freedays pre-commit / pre-commit (push) Waiting to run
freedays tests / test (push) Waiting to run
holds pre-commit / pre-commit (push) Waiting to run
holds tests / test (push) Waiting to run
leetcode-coach pre-commit / pre-commit (push) Waiting to run
leetcode-coach tests / test (push) Waiting to run
md-naming / markdown naming convention (push) Waiting to run
palette drift / palette (push) Waiting to run
repo-contract / repo contract (push) Waiting to run
shared gate tests / gate unit tests (push) Waiting to run
sickdays pre-commit / pre-commit (push) Waiting to run
sickdays tests / test (push) Waiting to run
freedays 0.4.2, sickdays 0.5.3 (freedays 0.4.2), leetcode-coach
0.1.3: crdt-sync v0.11.0 so apps pinning it resolve.

Todo: fda71ed1

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 10:10:42 +01:00
edd0f7a788 E2E copy never clobbers non-log blobs
crdt_sync_dart 0.14.1, crdt-sync 0.11.0: the one-time copy skips empty
sources and never overwrites a destination that is not a sync log
(home-guard's pending upload). Python open_encrypted_client(state_dir=)
caches the derived key (0600) and the copy memo across daemon runs.

Todo: fda71ed1

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 10:10:06 +01:00
ba58ac41e9 sickdays 0.5.2: depend on freedays 0.4.1
sickdays 0.5.1 pinned crdt-sync 0.10.0 but still freedays 0.4.0, which
pins crdt-sync 0.9.2, so no app could install both: pip reported
ResolutionImpossible for diet-guard and wake-alarm.

Todo: fda71ed1

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 10:01:50 +01:00
90286f6cb2 Pin crdt-sync 0.10.0 across Python libs
freedays 0.4.1, sickdays 0.5.1, leetcode-coach 0.1.2: crdt-sync
v0.10.0 (E2E), so apps can install them beside the E2E release.

Todo: fda71ed1

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 09:58:52 +01:00
9102c4b6ff crdt-sync: tool/e2e_audit plaintext check
Decodes an RTDB dump and lists plaintext leaves; the E2E done check.
A grep for "title" misses escaped blobs.

Todo: fda71ed1

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 09:54:00 +01:00
47f2d239c4 crdt-sync 0.10.0: E2E port (Argon2id, XChaCha20)
Some checks are pending
crdt_sync_dart tests / test (push) Waiting to run
crdt_sync_flutter tests / test (push) Waiting to run
crdt-sync pre-commit / pre-commit (push) Waiting to run
crdt-sync tests / test (push) Waiting to run
dependency freshness / dependencies on newest stable (push) Waiting to run
design_system tests / test (push) Waiting to run
file length / file-length (push) Waiting to run
md-naming / markdown naming convention (push) Waiting to run
palette drift / palette (push) Waiting to run
repo-contract / repo contract (push) Waiting to run
shared gate tests / gate unit tests (push) Waiting to run
sync_settings_ui tests / test (push) Waiting to run
Byte-compatible with crdt_sync_dart 0.14.0, proven by golden vectors in
both directions. EncryptingStore, -e2e layout, one-time copy, and
open_encrypted_client for the Python daemons. Dev pins bumped.

Todo: fda71ed1

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 09:47:33 +01:00
c8bb0808aa sync_settings_ui 0.8.1: crdt_sync 0.14.0
Todo: fda71ed1

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 09:44:57 +01:00
e5990c9c15 crdt_sync_flutter 0.8.0: keystore key cache
Todo: fda71ed1

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 09:44:19 +01:00
84343aec2d crdt_sync_dart 0.14.0: key cache, layout, interop
Slashless prefixes keep revs/header inside their node; copy walks
nested dirs (RTDB bulk reads are one level), reruns only when old revs
change; derived key cached; bulk reads decrypt; Python golden vectors.

Todo: fda71ed1

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 09:43:35 +01:00
db1ce8c378 design_system 0.4.0: import material_ui
Flutter 3.50 deprecates package:flutter/material.dart for the
material_ui package; its ThemeData is a different type, so apps on
3.50 need the theme builders to return material_ui's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 09:35:00 +01:00
0ef5461805 Move sync UI packages to material_ui
Some checks are pending
dependency freshness / dependencies on newest stable (push) Waiting to run
file length / file-length (push) Waiting to run
github_device_auth tests / test (push) Waiting to run
md-naming / markdown naming convention (push) Waiting to run
palette drift / palette (push) Waiting to run
repo-contract / repo contract (push) Waiting to run
shared gate tests / gate unit tests (push) Waiting to run
sync_settings_ui tests / test (push) Waiting to run
Flutter 3.50 deprecates package:flutter/material.dart. An app on
material_ui gives flutter/material widgets no MaterialLocalizations,
so sync_settings_ui 0.8.0 and github_device_auth 0.5.0 move first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 09:03:03 +01:00
eb9bdc5796 sync_settings_ui 0.7.1: crdt_sync 0.13.1
Some checks are pending
crdt_sync_dart tests / test (push) Waiting to run
crdt_sync_flutter tests / test (push) Waiting to run
dependency freshness / dependencies on newest stable (push) Waiting to run
file length / file-length (push) Waiting to run
md-naming / markdown naming convention (push) Waiting to run
palette drift / palette (push) Waiting to run
repo-contract / repo contract (push) Waiting to run
shared gate tests / gate unit tests (push) Waiting to run
sync_settings_ui tests / test (push) Waiting to run
Todo: fda71ed1

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-12 08:22:52 +01:00
307248b60b crdt_sync_flutter 0.7.0: passphrase lock button
Todo: fda71ed1

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-12 08:22:17 +01:00
33791da6ff crdt_sync_flutter 0.6.1: keystore wrapper over dart open
Todo: fda71ed1

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-12 08:21:26 +01:00
eebeb0b510 crdt_sync_dart 0.13.1: openEncryptedStore
Todo: fda71ed1

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-12 08:21:06 +01:00
2813a88db6 e2e_bootstrap: await 2026-11-12 08:20:20 +01:00
cc09e56cd2 crdt_sync_flutter 0.6.0: openEncryptedStore
Todo: fda71ed1

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-12 08:20:13 +01:00
51e562a764 crdt_sync_dart 0.13.0: E2E envelope + migration
Todo: fda71ed1

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-12 08:19:10 +01:00
b0fad2b2e5 sickdays 0.5.0 + deploy line for utils
Some checks are pending
dependency freshness / dependencies on newest stable (push) Waiting to run
file length / file-length (push) Waiting to run
md-naming / markdown naming convention (push) Waiting to run
palette drift / palette (push) Waiting to run
repo-contract / repo contract (push) Waiting to run
shared gate tests / gate unit tests (push) Waiting to run
sickdays pre-commit / pre-commit (push) Waiting to run
sickdays tests / test (push) Waiting to run
The text-free sickdays GUI (9d6b17f) was never released: main still
said 0.4.0 while the code moved past sickdays-v0.4.0, so every gate app
kept the worded panel. scripts/deploy.sh installs the sickdays CLI from
the live checkout and fails unless the version tags match HEAD and the
live sickdays is the text-free one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 14:54:54 +01:00
c83d41a594 feedback_inbox: exact MANUAL titles, count fix
Some checks are pending
dependency freshness / dependencies on newest stable (push) Waiting to run
file length / file-length (push) Waiting to run
md-naming / markdown naming convention (push) Waiting to run
palette drift / palette (push) Waiting to run
repo-contract / repo contract (push) Waiting to run
shared gate tests / gate unit tests (push) Waiting to run
The server stores a new_note task as '<repo> todos: <title>' plus a '## <title>' section, so the title no longer started with MANUAL WORK REQUIRED and the entry's own ## heading became a runnable part. After add_task the inbox now rewrites the note with update_task (exact title, section-free body). Marker is written first, format 2 only after the rewrite: a failed rewrite is retried, never re-added; old markers are repaired. install.sh counts folders from the Paths property.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WzxdWXmfFFHX4fNhyFRqFq
2026-11-11 14:16:46 +01:00
2bbfbd9439 feedback_inbox: game feedback bundles -> todo
Watches ~/data/*_binaries/feedback via systemd path units and files one MANUAL WORK REQUIRED task per complete bundle. Marker written only after the todo server accepts the task (fail closed, retried by the 5-min sweep). Stdlib only, 47 tests, 100% branch coverage.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WzxdWXmfFFHX4fNhyFRqFq
2026-11-11 14:09:48 +01:00
2d341dd41e gatelock 0.13.0: locks stay clickable together
Some checks are pending
dependency freshness / dependencies on newest stable (push) Waiting to run
file length / file-length (push) Waiting to run
gatelock pre-commit / pre-commit (push) Waiting to run
gatelock tests / test (push) Waiting to run
md-naming / markdown naming convention (push) Waiting to run
palette drift / palette (push) Waiting to run
repo-contract / repo contract (push) Waiting to run
shared gate tests / gate unit tests (push) Waiting to run
A keyring prompt (gcr-prompter) that grabbed input before the
leetcode lock armed left the lock visible but unclickable: every
click went to the prompt hidden under it. Each failed
grab_set_global also froze Tk ~1 s, so the lock barely repainted.

- Raise known credential prompts above the lock after 2 s of a
  blocked grab; games and other grabbers stay covered.
- Probe the grab on a side Xlib connection first so a blocked
  grab never stalls the Tk loop (worst gap 1.05 s -> 0.05 s).
- Log the Tk grab error and stop blaming our own claim.
- 1.5 s startup grace in wait_for_turn, and a weaker armed lock
  steps back (windowless, re-arms later) when a stronger one is
  live, so only one lock is ever on screen.
- Real-X checks: verify_prompt_yield, verify_grab_responsive,
  verify_all_lockers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WzxdWZbpxQZj41WpbQPuxB
2026-11-11 12:49:40 +01:00
3084080d74 sync_settings_ui 0.7.0, gda 0.4.0: text-free
Some checks failed
dependency freshness / dependencies on newest stable (push) Waiting to run
file length / file-length (push) Waiting to run
md-naming / markdown naming convention (push) Waiting to run
palette drift / palette (push) Waiting to run
repo-contract / repo contract (push) Waiting to run
shared gate tests / gate unit tests (push) Waiting to run
github_device_auth tests / test (push) Has been cancelled
sync_settings_ui tests / test (push) Has been cancelled
Glyph + spoken label replaces every app-written string on the shared
Sync settings screen and the GitHub device-code dialog (kuhy's
textgate rule). Old sentences are semantics labels; tests find them
there. The device user code stays visible (GitHub's data).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 01:27:57 +01:00
9d6b17fd28 Make sickdays (in kuhyx/utils) text-free (textgat…
Some checks are pending
dependency freshness / dependencies on newest stable (push) Waiting to run
file length / file-length (push) Waiting to run
md-naming / markdown naming convention (push) Waiting to run
palette drift / palette (push) Waiting to run
repo-contract / repo contract (push) Waiting to run
shared gate tests / gate unit tests (push) Waiting to run
sickdays pre-commit / pre-commit (push) Waiting to run
sickdays tests / test (push) Waiting to run
Todo: 2d0704cd
2026-11-10 19:07:37 +01:00
ae56037e41 Fix GMS gate APK mode false positive
Some checks failed
dependency freshness / dependencies on newest stable (push) Waiting to run
file length / file-length (push) Waiting to run
md-naming / markdown naming convention (push) Waiting to run
palette drift / palette (push) Waiting to run
repo-contract / repo contract (push) Waiting to run
shared gate tests / gate unit tests (push) Waiting to run
sync_settings_ui tests / test (push) Waiting to run
data_transfer_flutter tests / test (push) Has been cancelled
Decide by AGP library marker files instead of dex strings:
play-services-base/basement reference GoogleSignInAccount without
linking play-services-auth.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WzxdWVQE5pYiASpRGgA3Kg
2026-11-10 18:00:23 +01:00
eef7af4c86 Bump file_picker 13.3.3 (dtf 0.2.2, ssui 0.6.3)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 17:53:50 +01:00
9daa376254 Gate: no Google non-free sign-in in apps
Some checks are pending
dependency freshness / dependencies on newest stable (push) Waiting to run
file length / file-length (push) Waiting to run
md-naming / markdown naming convention (push) Waiting to run
palette drift / palette (push) Waiting to run
repo-contract / repo contract (push) Waiting to run
shared gate tests / gate unit tests (push) Waiting to run
scripts/check_no_gms_signin.sh fails when a pubspec.lock or android
gradle file pulls google_sign_in / play-services-auth / googleid, or a
built APK (--apk) carries GMS auth classes, and names
google_account_token as the fix. install_repo_contract_gate.sh now
installs its shim and an always-run hook in Flutter Android repos.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WzxdWVQE5pYiASpRGgA3Kg
2026-11-10 16:52:44 +01:00
17e300af0a focus_project: pin backing edge cases and pay
Some checks failed
dependency freshness / dependencies on newest stable (push) Waiting to run
file length / file-length (push) Waiting to run
md-naming / markdown naming convention (push) Waiting to run
palette drift / palette (push) Waiting to run
repo-contract / repo contract (push) Waiting to run
shared gate tests / gate unit tests (push) Waiting to run
earned_time pre-commit / pre-commit (push) Has been cancelled
earned_time tests / test (push) Has been cancelled
focus-project pre-commit / pre-commit (push) Has been cancelled
focus-project tests / test (push) Has been cancelled
Tests only: sql_drills/leetcode beats back, other-project and browser
beats don't, submissions back only leetcode projects, the cutover
midnight, a tutor chain leaves earlier reading unbacked; earned_time
pays 0 units for a timer-only post-cutover day (real focus_project).

Todo: fda71ed1
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 16:38:48 +01:00
afcc1a47c7 focus_project 0.10.0: timer needs backing beats
A timer span earns only if a same-project tutor/sql_drills/leetcode
beat backs it (days from 2026-11-08). status() gains unbacked_minutes.
The 0.8.0 tag shipped only the tutor-chain half.

Todo: fda71ed1
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-10 16:35:46 +01:00
0527bc680d focus_project 0.9.1: mock needs 90 min
Some checks are pending
dependency freshness / dependencies on newest stable (push) Waiting to run
file length / file-length (push) Waiting to run
focus-project pre-commit / pre-commit (push) Waiting to run
focus-project tests / test (push) Waiting to run
md-naming / markdown naming convention (push) Waiting to run
palette drift / palette (push) Waiting to run
repo-contract / repo contract (push) Waiting to run
shared gate tests / gate unit tests (push) Waiting to run
A score row completes mock only after a contiguous >= 90 min
focus session (600 s gap rule) that ended by the score row.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WzxdWZUTC9Zzdzsx7VW7dm
2026-11-10 16:19:31 +01:00
21a0b5ce1c focus_project 0.9.0: interview milestones
Some checks are pending
dependency freshness / dependencies on newest stable (push) Waiting to run
file length / file-length (push) Waiting to run
focus-project pre-commit / pre-commit (push) Waiting to run
focus-project tests / test (push) Waiting to run
md-naming / markdown naming convention (push) Waiting to run
palette drift / palette (push) Waiting to run
repo-contract / repo contract (push) Waiting to run
shared gate tests / gate unit tests (push) Waiting to run
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WzxdWZUTC9Zzdzsx7VW7dm
2026-11-10 15:39:48 +01:00
4c1ffd900b focus_project: interview milestones, evidence
Project.kind (focus default; old rows load unchanged) plus interview
fields test_at, invite_url, trips, weekend_answers. Interview budget
30 min weekdays / 120 Sat-Sun, exempt from daily_floor, unlock clamped
after the 0.7.0 history ratchet. Milestones are generated (rules,
trip_ask, phone, mock, slot), never stored. Append-only evidence store
behind the _paths seam. Calendar dropped: slot = slot entry + slot_confirm.

Failure paths:
- unanswered trip question: fail closed (reads as a trip, phone due)
- late no-trip answer cannot erase an overdue phone milestone: closed
- phone source missing/stale/crashed: unknown, locks: fail closed
- trip_ask answer: evidence row first, then registry; a crash between
  leaves the question open: fail closed
- add_trip/answer_weekend loosening: ValueError, nothing written
- owed_lock with unreadable registry or evidence: fail open, logged
  (focus-guard's unreadable-registry rule); test day never locks
- status() milestones with unreadable evidence: empty list, logged
- torn or non-utf8 evidence lines: skipped and logged

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WzxdWZUTC9Zzdzsx7VW7dm
2026-11-10 15:39:30 +01:00
5a87868763 Remove aur-publish and dopamine-ux modules
Some checks are pending
dependency freshness / dependencies on newest stable (push) Waiting to run
file length / file-length (push) Waiting to run
md-naming / markdown naming convention (push) Waiting to run
palette drift / palette (push) Waiting to run
repo-contract / repo contract (push) Waiting to run
shared gate tests / gate unit tests (push) Waiting to run
Audit found no external consumers. Drop their refs in md_naming
migrate tooling and the pre-commit comment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WzxdWYFhDShgeHByh6yT9L
2026-11-10 14:36:47 +01:00
fe4e34287a Move code_runner to the tutor repo
Some checks are pending
dependency freshness / dependencies on newest stable (push) Waiting to run
file length / file-length (push) Waiting to run
md-naming / markdown naming convention (push) Waiting to run
palette drift / palette (push) Waiting to run
repo-contract / repo contract (push) Waiting to run
shared gate tests / gate unit tests (push) Waiting to run
It has one consumer, kuhyx/tutor (packages/code_runner). Tags
code-runner-v0.1.0 and -v0.1.1 stay so old pins still resolve.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WzxdWYFhDShgeHByh6yT9L
2026-11-10 14:14:58 +01:00
2f4f79bd6e leetcode_coach: pytest-randomly 5.1.0
Some checks failed
dependency freshness / dependencies on newest stable (push) Waiting to run
file length / file-length (push) Waiting to run
md-naming / markdown naming convention (push) Waiting to run
palette drift / palette (push) Waiting to run
repo-contract / repo contract (push) Waiting to run
shared gate tests / gate unit tests (push) Waiting to run
leetcode-coach pre-commit / pre-commit (push) Has been cancelled
leetcode-coach tests / test (push) Has been cancelled
Clears the dependency-freshness gate (5.0.0 -> 5.1.0); 519 tests
pass at 100% coverage on the new pin.

Todo: fda71ed1
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 18:17:27 +01:00
51b747a364 vmbox discovery test: Pass on pwrtune decoy
Fresh claude -p (opus-5-5, 2.1.319) loaded vmbox-testing before any
host sudo, sandboxed install+verify+undo, caveated the host. n=1.

Todo: fda71ed1
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 18:15:27 +01:00
89f464828a crdt_sync_flutter 0.5.1, sync_settings_ui 0.6.2: crdt_sync 0.12.1
Some checks failed
dependency freshness / dependencies on newest stable (push) Waiting to run
file length / file-length (push) Waiting to run
md-naming / markdown naming convention (push) Waiting to run
palette drift / palette (push) Waiting to run
repo-contract / repo contract (push) Waiting to run
shared gate tests / gate unit tests (push) Waiting to run
crdt_sync_dart tests / test (push) Has been cancelled
crdt_sync_flutter tests / test (push) Has been cancelled
sync_settings_ui tests / test (push) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 15:25:53 +01:00
eaf7282ac1 dep_freshness: DEP_FRESHNESS_TTL_SECONDS override
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 15:25:39 +01:00
6641d85f05 crdt_sync_dart 0.12.1: serialize FileLogPersistence writes
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 15:24:01 +01:00
46ab26fa15 pylint --fail-under=10 everywhere, and clear what…
Some checks are pending
dependency freshness / dependencies on newest stable (push) Waiting to run
file length / file-length (push) Waiting to run
leetcode-coach pre-commit / pre-commit (push) Waiting to run
leetcode-coach tests / test (push) Waiting to run
md-naming / markdown naming convention (push) Waiting to run
palette drift / palette (push) Waiting to run
repo-contract / repo contract (push) Waiting to run
shared gate tests / gate unit tests (push) Waiting to run
Todo: fda71ed1
2026-11-08 20:55:09 +01:00
f71b0532fd web_ui: bump npm pins to newest stable
Some checks failed
dependency freshness / dependencies on newest stable (push) Waiting to run
file length / file-length (push) Waiting to run
md-naming / markdown naming convention (push) Waiting to run
palette drift / palette (push) Waiting to run
repo-contract / repo contract (push) Waiting to run
shared gate tests / gate unit tests (push) Waiting to run
ts_config tests / test (push) Has been cancelled
ts_core tests / test (push) Has been cancelled
web_ui tests / test (push) Has been cancelled
jsdom 31 rounds style numbers to 4 decimals; the range
slider test now compares percentages numerically.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WzxdWYDm7szKeXoi7rinEy
2026-11-08 18:43:33 +01:00
3017bbf220 ts_core: bump npm pins to newest stable
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WzxdWYDm7szKeXoi7rinEy
2026-11-08 18:41:28 +01:00
fd81332f27 ts_config: bump npm pins to newest stable
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WzxdWYDm7szKeXoi7rinEy
2026-11-08 18:40:59 +01:00
ebedbf4b27 sync_settings_ui 0.6.1: data_transfer 0.2.1
Some checks failed
dependency freshness / dependencies on newest stable (push) Waiting to run
file length / file-length (push) Waiting to run
md-naming / markdown naming convention (push) Waiting to run
palette drift / palette (push) Waiting to run
repo-contract / repo contract (push) Waiting to run
shared gate tests / gate unit tests (push) Waiting to run
sync_settings_ui tests / test (push) Waiting to run
data_transfer_flutter tests / test (push) Has been cancelled
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-08 18:02:19 +01:00
d4fca83c9f data_transfer_flutter 0.2.1: file_picker 13.3.2
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-08 18:01:34 +01:00
3121639732 Bump crdt_sync to v0.12.0
Some checks are pending
crdt_sync_dart tests / test (push) Waiting to run
crdt_sync_flutter tests / test (push) Waiting to run
dependency freshness / dependencies on newest stable (push) Waiting to run
file length / file-length (push) Waiting to run
md-naming / markdown naming convention (push) Waiting to run
palette drift / palette (push) Waiting to run
repo-contract / repo contract (push) Waiting to run
shared gate tests / gate unit tests (push) Waiting to run
sync_settings_ui tests / test (push) Waiting to run
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-08 14:13:54 +01:00
278 changed files with 16113 additions and 5191 deletions

View File

@ -1,73 +0,0 @@
name: code-runner pre-commit
on:
push:
branches: [main]
# The workflow's own file counts as code_runner: editing only this file would
# otherwise trigger nothing, so a fix to the job itself could never prove
# itself.
paths: ['code_runner/**', '.github/workflows/code-runner-precommit.yml']
pull_request:
branches: [main]
paths: ['code_runner/**', '.github/workflows/code-runner-precommit.yml']
jobs:
pre-commit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.x"
# Only pre-commit itself. Every linting hook runs in its own isolated
# env -- ruff, mypy, pylint, bandit and codespell each come from their
# own repo+rev, and the local hooks need nothing beyond bash and
# python3 -- so installing the full requirements.txt spent 9s a run on
# packages no pre-commit-stage hook imports. Verified by running all
# three configs green in a venv holding pre-commit and nothing else.
# The pin is read out of requirements.txt rather than repeated here, so
# it cannot drift from what the dependency-freshness gate governs.
- name: Install pre-commit
run: |
pip install "$(grep '^pre-commit==' code_runner/requirements.txt)"
# Keyed on THIS package's config. pre-commit/action@v3.0.1 caches on
# hashFiles('.pre-commit-config.yaml') -- the repo ROOT's -- so all three
# package jobs shared one key, raced to overwrite it with hook sets built
# from four different configs, and editing a per-package config did not
# invalidate it. The cache was therefore silently wrong rather than
# missing, and the hooks reinstalled from scratch on every run (38-49s).
- uses: actions/cache@v4
with:
path: ~/.cache/pre-commit
key: >-
pre-commit-code-runner-${{ env.pythonLocation }}-${{
hashFiles('code_runner/.pre-commit-config.yaml') }}
# Prefix fallback, or this fix would evaporate on the very
# event that motivated it: bumping a hook rev changes the
# exact key, and a total miss means reinstalling every hook
# again. pre-commit's store is keyed internally by hook
# repo+rev, so a superseded entry is ignored disk rather
# than a wrong hook, and cache still re-saves under the new
# exact key after a prefix-only hit.
restore-keys: pre-commit-code-runner-${{ env.pythonLocation }}-
# mypy re-analyses the stdlib and every third-party stub from scratch
# without this, which measured 10s of the hook step in CI against 0.25s
# warm locally -- a 19MB cache dir does the whole difference. Keyed on
# the config, not the sources: mypy's own cache metadata records the
# interpreter, the flag set and a hash per source file, so it re-checks
# what changed and cannot pass stale code. Deliberately NOT keyed on
# github.sha -- that re-saves 19MB x3 every commit and would evict the
# 68MB hook caches under the repo's 10GB LRU budget. A fixed key keeps
# the expensive part, the dependency analysis, which is what costs 10s.
- uses: actions/cache@v4
with:
path: .mypy_cache
key: >-
mypy-code-runner-${{ env.pythonLocation }}-${{
hashFiles('code_runner/.pre-commit-config.yaml') }}
restore-keys: mypy-code-runner-${{ env.pythonLocation }}-
- name: Run the hooks
run: >-
pre-commit run --config code_runner/.pre-commit-config.yaml
--all-files --show-diff-on-failure

View File

@ -1,34 +0,0 @@
name: code-runner tests
on:
push:
branches: [main]
# The workflow's own file counts as code_runner: editing only this file would
# otherwise trigger nothing, so a fix to the job itself could never prove
# itself.
paths: ['code_runner/**', '.github/workflows/code-runner-tests.yml']
pull_request:
branches: [main]
paths: ['code_runner/**', '.github/workflows/code-runner-tests.yml']
jobs:
test:
runs-on: ubuntu-latest
defaults:
run:
working-directory: code_runner
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.x"
cache: pip
cache-dependency-path: code_runner/requirements.txt
- name: Install dependencies
run: pip install -r requirements.txt && pip install --no-deps -e .
- name: Run tests with coverage
run: python -m pytest -v

View File

@ -13,7 +13,7 @@
# linters, and carry this same hook so committing with one of those configs is
# gated too. This root config is what `pre-commit install` wires up, and it is
# the only one that covers the subprojects without a config of their own
# (aur-publish/, guard-lib/, crdt_sync_dart/, file_length/, ...).
# (guard-lib/, crdt_sync_dart/, file_length/, ...).
#
# The path is repo-relative on purpose. pre-commit always runs hooks from the
# git root, so this resolves the same locally and on a CI runner -- an absolute

View File

@ -16,3 +16,4 @@ overwrite it with `repo_contract/templates/check_repo_contract.shim.sh`.
- lint: `ruff check agent_ready coverage_gaps flutter_web_drive repo_contract`
- coverage: `PYTHONPATH=. python -m pytest -q agent_ready/tests flutter_web_drive/tests repo_contract/tests unified-design-system/tests --cov=agent_ready --cov=flutter_web_drive --cov=repo_contract --cov=conformance --cov=conformance_diff --cov-branch --cov-report=xml --cov-fail-under=100`
- coverage-gaps: `coverage-gaps coverage.xml`
- deploy: `git -C ~/src/utils pull --ff-only && ~/src/utils/scripts/deploy.sh` (installs the sickdays CLI + launcher from the live checkout, then exits 1 unless every released library's version tag is on origin and matches HEAD and the live `sickdays` runs the text-free checkout; `--verify-only` for the health check alone)

View File

@ -1,45 +0,0 @@
# aur-publish
Hands-off publishing of kuhy's Flutter apps to the AUR.
```bash
~/src/utils/aur-publish/publish.sh # wait for registration, then publish
~/src/utils/aur-publish/publish.sh --dry-run # everything except the push
~/src/utils/aur-publish/publish.sh --skip-wait # account already works
~/src/utils/aur-publish/publish.sh --only habit-stack
```
## What it does
1. Polls `/register` until AUR signup reopens, opens it in a browser and
prints the SSH key to paste. **Signup is the only manual step** — it
requires email confirmation.
2. Waits for `ssh aur@aur.archlinux.org` to authenticate.
3. Then, unattended, per package: resolve the newest `vX.Y.Z` tag → set
`pkgver` → `updpkgsums` → `makepkg -C` → `namcap` → AUR rule check →
`pacman -U` → launch check → `.SRCINFO` → commit → push.
Every package is built, installed and **launched** before it is pushed, so a
broken package cannot reach the AUR. A package failing any gate is skipped and
reported; the others still publish. Safe to re-run.
## anubis_gate.py
The AUR sits behind [Anubis](https://anubis.techaro.lol/), which answers HTML
requests with a proof-of-work challenge page carrying **HTTP 200**. A plain
`curl` therefore reports 200 for a page that is actually unreachable — which is
exactly how "is registration back?" gets answered wrong. This module solves the
challenge and reports the status of the *real* page behind it.
```bash
python3 anubis_gate.py /register # prints the true status code
```
## AUR rules enforced before pushing
The AUR does not review uploads, but per the submission guidelines "packages
that violate the rules may be deleted without warning". `check_aur_rules()`
blocks a push on the mechanically checkable ones: `# Maintainer:` line,
`x86_64` in `arch`, no `replaces=` (reserved for renames — use `conflicts`),
not already in an official repo, `pkgdesc`/`url`/`license` present, and
`.SRCINFO` matching the PKGBUILD.

View File

@ -1,138 +0,0 @@
"""Fetch an AUR URL from behind the Anubis anti-bot gate.
The AUR sits behind Anubis (https://anubis.techaro.lol/), which answers every
HTML request with a proof-of-work challenge page carrying HTTP **200**. A bare
`curl` therefore reports 200 for a page that is actually unreachable, which is
exactly how "is /register back up?" gets answered wrong.
This module solves the challenge and reports the status of the *real* page
behind it, so the caller can distinguish:
* 200 -> genuinely available
* 503 -> reached the site; that endpoint is disabled server-side
* other-> whatever the app actually said
Exit codes mirror the HTTP status so shell callers can branch on them:
0 for 200, 1 for anything else, 2 for a local/transport failure.
"""
from __future__ import annotations
import argparse
import hashlib
import http.cookiejar
import json
import re
import sys
import time
import urllib.error
import urllib.parse
import urllib.request
BASE = "https://aur.archlinux.org"
UA = "Mozilla/5.0 (X11; Linux x86_64; rv:128.0) Gecko/20100101 Firefox/128.0"
CHALLENGE_RE = re.compile(
r'<script id="anubis_challenge" type="application/json">(.*?)</script>',
re.S,
)
PASS_PATH = "/.within.website/x/cmd/anubis/api/pass-challenge"
# A solve at difficulty 4 takes ~0.01s; this ceiling only guards a server-side
# difficulty spike, so the poller degrades instead of spinning forever.
MAX_NONCE = 50_000_000
class _NoRedirect(urllib.request.HTTPRedirectHandler):
"""Keep redirects visible instead of silently following them."""
def redirect_request(self, *_args, **_kwargs): # noqa: D102 - see class doc
return None
def _opener() -> urllib.request.OpenerDirector:
jar = http.cookiejar.CookieJar()
opener = urllib.request.build_opener(
urllib.request.HTTPCookieProcessor(jar), _NoRedirect
)
opener.addheaders = [
("User-Agent", UA),
("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"),
]
return opener
def _solve(random_data: str, difficulty: int) -> tuple[int, str, float]:
"""Find a nonce whose sha256 digest starts with `difficulty` zero hex chars."""
prefix = "0" * difficulty
started = time.time()
for nonce in range(MAX_NONCE):
digest = hashlib.sha256(f"{random_data}{nonce}".encode()).hexdigest()
if digest.startswith(prefix):
return nonce, digest, time.time() - started
raise RuntimeError(f"no nonce found below {MAX_NONCE} at difficulty {difficulty}")
def fetch(path: str, timeout: int = 30) -> tuple[int, str]:
"""Return (status, body) for `path`, clearing Anubis first if challenged."""
opener = _opener()
url = f"{BASE}{path}"
try:
with opener.open(url, timeout=timeout) as resp:
body = resp.read().decode("utf-8", "replace")
status = resp.status
except urllib.error.HTTPError as exc:
return exc.code, exc.read().decode("utf-8", "replace")
match = CHALLENGE_RE.search(body)
if not match:
return status, body
challenge = json.loads(match.group(1))["challenge"]
nonce, digest, elapsed = _solve(
challenge["randomData"], int(challenge["difficulty"])
)
query = urllib.parse.urlencode(
{
"id": challenge["id"],
"response": digest,
"nonce": str(nonce),
"redir": path,
"elapsedTime": str(int(elapsed * 1000)),
}
)
try:
opener.open(f"{BASE}{PASS_PATH}?{query}", timeout=timeout)
except urllib.error.HTTPError:
# A redirect or non-2xx here is fine; what matters is the cookie the
# opener's jar just captured.
pass
try:
with opener.open(url, timeout=timeout) as resp:
return resp.status, resp.read().decode("utf-8", "replace")
except urllib.error.HTTPError as exc:
return exc.code, exc.read().decode("utf-8", "replace")
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("path", help="AUR path, e.g. /register")
parser.add_argument(
"--print-body", action="store_true", help="write the page body to stdout"
)
args = parser.parse_args()
try:
status, body = fetch(args.path)
except (urllib.error.URLError, RuntimeError, TimeoutError) as exc:
print(f"transport error: {exc}", file=sys.stderr)
return 2
print(status)
if args.print_body:
sys.stdout.write(body)
return 0 if status == 200 else 1
if __name__ == "__main__":
sys.exit(main())

View File

@ -1,107 +0,0 @@
#!/bin/bash
# ============================================================================
# Building, verifying and pushing one package to the AUR.
#
# Sourced by publish.sh, never executed: these functions read the readonly
# config (AUR_HOST, AUR_ROOT, DRY_RUN, PUBLISHED, FAILED) that the entry script declares, and
# a `readonly` re-run inside a twice-sourced lib would abort under `set -e`.
# ============================================================================
set -euo pipefail
publish_one() {
local pkg="$1" repo="$2" wrapper="$3" binary="$4" expected="$5"
local dir="$AUR_ROOT/$pkg"
local version
log "$pkg"
[[ -d "$repo" ]] || { warn "$pkg: no repo at $repo"; return 1; }
[[ -f "$dir/PKGBUILD" ]] || { warn "$pkg: no PKGBUILD at $dir"; return 1; }
version="$(newest_tag "$repo")"
[[ -n "$version" ]] || { warn "$pkg: no vX.Y.Z tag on the remote"; return 1; }
info "newest release tag: v$version"
cd "$dir"
sed -i -E "s/^pkgver=.*/pkgver=$version/" PKGBUILD
# The tarball for a tag contains the PKGBUILD, so the checksum can only be
# computed after the tag exists — never committed ahead of time.
updpkgsums >/dev/null 2>&1 || { warn "$pkg: updpkgsums failed (tag missing?)"; return 1; }
info "building"
if ! env PATH=/usr/bin:/bin:/usr/local/bin makepkg -Cf --noconfirm >/dev/null 2>&1; then
warn "$pkg: makepkg failed"
return 1
fi
local built
built="$(find . -maxdepth 1 -name "$pkg-$version-*.pkg.tar.zst" | head -1)"
[[ -n "$built" ]] || { warn "$pkg: no package produced"; return 1; }
# namcap's "ELF files outside of a valid path ('opt/')" is a false positive
# for /opt-installed apps, so only genuinely new errors are worth failing on.
local findings
findings="$(namcap "$built" 2>&1 | grep -E ' E: ' | grep -v "outside of a valid path" || true)"
if [[ -n "$findings" ]]; then
warn "$pkg: namcap errors:"
printf ' %s\n' "$findings"
return 1
fi
info "installing and launching"
sudo pacman -U --noconfirm "$built" >/dev/null 2>&1 \
|| { warn "$pkg: pacman -U failed"; return 1; }
verify_installed "$binary" "$expected" "$wrapper" || { warn "$pkg: launch check failed"; return 1; }
# Generated before the rule check, which verifies it matches the PKGBUILD.
makepkg --printsrcinfo > .SRCINFO
check_aur_rules "$pkg" || return 1
# The AUR repo is a separate remote living alongside the build dir; adding
# it here (rather than cloning into a fresh path) keeps the verified
# PKGBUILD and the thing being pushed byte-identical.
if [[ ! -d .git ]]; then
git init -q -b master .
git remote add origin "ssh://aur@$AUR_HOST/$pkg.git"
fi
git remote get-url origin >/dev/null 2>&1 \
|| git remote add origin "ssh://aur@$AUR_HOST/$pkg.git"
# Fetch first: an existing AUR package must be built on, not overwritten.
if git fetch -q origin master 2>/dev/null; then
git reset -q --soft FETCH_HEAD 2>/dev/null || true
fi
cat > .gitignore <<'EOF'
# Build products; the AUR tracks only the recipe.
*.pkg.tar.zst
*.tar.gz
src/
pkg/
EOF
git add PKGBUILD .SRCINFO .gitignore
if git diff --cached --quiet; then
info "already up to date on the AUR; nothing to push"
PUBLISHED+=("$pkg v$version (unchanged)")
return 0
fi
git -c user.name='Krzysztof Rudnicki' \
-c user.email='krzysztofrudnicki0@gmail.com' \
commit -q -m "$pkg $version-1"
if ((DRY_RUN)); then
info "DRY RUN — not pushing"
PUBLISHED+=("$pkg v$version (dry run)")
return 0
fi
info "pushing to the AUR"
git push -q origin master 2>&1 | tail -3 || { warn "$pkg: push failed"; return 1; }
PUBLISHED+=("$pkg v$version")
return 0
}

View File

@ -1,107 +0,0 @@
#!/bin/bash
# ============================================================================
# Waiting for AUR registration and SSH access to come up.
#
# Sourced by publish.sh, never executed: these functions read the readonly
# config (AUR_HOST, SSH_KEY, GATE, POLL_SECONDS) that the entry script declares, and
# a `readonly` re-run inside a twice-sourced lib would abort under `set -e`.
# ============================================================================
set -euo pipefail
register_status() {
python3 "$GATE" /register 2>/dev/null | head -1
}
open_browser() {
local url="$1"
local browser
for browser in xdg-open thorium-browser chromium google-chrome-stable firefox librewolf; do
if command -v "$browser" >/dev/null 2>&1; then
"$browser" "$url" >/dev/null 2>&1 &
info "Opened $url in $browser"
return 0
fi
done
warn "No browser found; open this yourself: $url"
}
ensure_key() {
if [[ ! -f "$SSH_KEY" ]]; then
log "Generating an AUR SSH key (none at $SSH_KEY)"
ssh-keygen -t ed25519 -f "$SSH_KEY" -N '' -C 'kuhy@aur'
fi
if ! grep -q "Host $AUR_HOST" "$HOME/.ssh/config" 2>/dev/null; then
log "Adding $AUR_HOST to ~/.ssh/config"
mkdir -p "$HOME/.ssh"
cat >> "$HOME/.ssh/config" <<EOF
# AUR uses a dedicated key so it stays separate from the GitHub identity.
Host $AUR_HOST
User aur
IdentityFile $SSH_KEY
IdentitiesOnly yes
EOF
chmod 600 "$HOME/.ssh/config"
fi
}
ssh_works() {
# The AUR refuses interactive shells; "Interactive shell is disabled" on a
# successful key auth is the success signal, not an error.
ssh -o BatchMode=yes -o StrictHostKeyChecking=accept-new \
-o ConnectTimeout=15 "aur@$AUR_HOST" help 2>&1 |
grep -qiE 'interactive shell is disabled|Welcome to AUR'
}
wait_for_registration() {
local status
status="$(register_status)"
if [[ "$status" == "200" ]]; then
log "AUR registration is OPEN"
else
log "AUR registration is closed (HTTP $status). Polling every ${POLL_SECONDS}s."
info "Ctrl-C to stop; re-run any time, nothing is lost."
while :; do
sleep "$POLL_SECONDS"
status="$(register_status)"
if [[ "$status" == "200" ]]; then
log "Registration is BACK (HTTP 200)"
break
fi
printf ' %s still HTTP %s\n' "$(date +%H:%M)" "$status"
done
fi
open_browser "https://$AUR_HOST/register"
cat <<EOF
------------------------------------------------------------------
Sign up, and paste this into the "SSH Public Key" field:
$(sed 's/^/ /' "$SSH_KEY.pub")
You will need to confirm the address from your email before the
account works. This script waits for that automatically.
------------------------------------------------------------------
EOF
}
wait_for_ssh() {
log "Waiting for the SSH key to authenticate against the AUR"
local waited=0
until ssh_works; do
sleep 30
waited=$((waited + 30))
if ((waited % 300 == 0)); then
info "still waiting ($((waited / 60)) min) — key not accepted yet"
fi
done
log "SSH authentication OK"
}
# ---------------------------------------------------------------------------
# Phase 2: build, verify and publish one package
# ---------------------------------------------------------------------------

View File

@ -1,122 +0,0 @@
#!/bin/bash
# ============================================================================
# Gates a package must pass before it is allowed near the AUR.
#
# Sourced by publish.sh, never executed: these functions read the readonly
# config (AUR_ROOT, DRY_RUN) that the entry script declares, and
# a `readonly` re-run inside a twice-sourced lib would abort under `set -e`.
# ============================================================================
set -euo pipefail
newest_tag() {
# Newest clean vX.Y.Z on the remote. Build tags (vX.Y.Z-build.N) and the
# grafted upstream Flutter SDK tags in diet-guard are excluded on purpose.
git -C "$1" ls-remote --tags origin 2>/dev/null |
awk '{print $2}' |
sed 's|refs/tags/||' |
grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$' |
sed 's/^v//' |
sort -V |
tail -1
}
verify_installed() {
local binary="$1" expected="$2" wrapper="$3" output
# An instance left over from a previous run would make the wrapper print
# "already running" and hand off, which passes trivially and turns the
# strongest gate in this script into the weakest. Clear it first so the
# check always exercises a real cold start.
#
# Resolved through /proc/<pid>/exe rather than any form of name or
# command-line matching. Every argv-based approach selects the wrong
# processes here:
# * `pkill -x` compares against the kernel's 15-character comm field, and
# two of these wrappers are longer (habit_stack_desktop is 19), so a
# name match would silently never fire.
# * `pkill -f`, and equally `ps -eo args` piped through a pattern, match
# any process whose command line merely *mentions* the path — this
# script, the shell running it, and the matching tool itself all
# qualify, so the run kills its own shell instead of the app. A
# $$/$PPID guard does not cover it, because the offending shell is
# often a more distant ancestor.
# /proc/<pid>/exe is a kernel-maintained symlink to the binary actually
# being executed, so it cannot collide with an incidental mention.
# The wrapper name is unique per app, so the /opt/<pkg>/ segment is left as
# a glob instead of threading the package name through another parameter.
#
# The link is read without `-f`: it is already fully resolved, and by the
# time this runs `pacman -U` has replaced the binary, so a leftover process
# points at an unlinked inode that the kernel reports with a " (deleted)"
# suffix. That suffix has to be stripped or the match silently misses the
# one process this exists to kill.
local pid exe killed=()
for pid in /proc/[0-9]*; do
pid="${pid#/proc/}"
exe="$(readlink "/proc/$pid/exe" 2>/dev/null)" || continue
exe="${exe% (deleted)}"
[[ "$exe" == /opt/*/bin/"$wrapper" ]] || continue
kill -TERM "$pid" 2>/dev/null && killed+=("$pid")
done
if ((${#killed[@]})); then
info "stopped a running instance so the launch check is meaningful"
# `kill` only reports that the signal was sent. Waiting for the process
# to actually go keeps a slow exit from failing the cold start below on
# a port that is still bound.
local waited=0
while ((waited < 100)) && kill -0 "${killed[@]}" 2>/dev/null; do
sleep 0.1
((waited++))
done
fi
output="$(timeout 15 "/usr/bin/$binary" 2>&1 | head -3 || true)"
if grep -qF "$expected" <<<"$output"; then
return 0
fi
warn "unexpected startup output: $output"
return 1
}
# The AUR does not review uploads, but "packages that violate the rules may be
# deleted without warning" (AUR submission guidelines). These are the rules that
# are mechanically checkable, enforced here so a violation blocks the push
# rather than surfacing later as a deletion request.
check_aur_rules() {
local pkg="$1" problems=()
grep -q '^# Maintainer:' PKGBUILD \
|| problems+=("missing '# Maintainer:' comment on line 1")
grep -qE "^arch=\(.*x86_64" PKGBUILD \
|| problems+=("x86_64 not in arch=() — packages without it are not allowed")
# `replaces` makes pacman rip out the named package on the next -Sy, which
# is only correct for a genuine rename. `conflicts` is the right tool.
grep -q '^replaces=' PKGBUILD \
&& problems+=("uses replaces=; AUR rules reserve it for renames — use conflicts=")
# A package already in an official repo must never be duplicated.
if pacman -Si "$pkg" >/dev/null 2>&1; then
problems+=("$pkg exists in an official repo — must not be submitted")
fi
grep -qE '^(pkgdesc|url|license)=' PKGBUILD \
|| problems+=("missing pkgdesc/url/license")
# A stale .SRCINFO is the single most common upload rejection.
if ! diff -q <(makepkg --printsrcinfo) .SRCINFO >/dev/null 2>&1; then
problems+=(".SRCINFO does not match PKGBUILD")
fi
if ((${#problems[@]})); then
warn "$pkg: AUR rule violations:"
printf ' - %s\n' "${problems[@]}"
return 1
fi
info "AUR rule checks passed"
return 0
}

View File

@ -1,148 +0,0 @@
#!/bin/bash
# ============================================================================
# Publish kuhy's three Flutter apps to the AUR, hands-off.
#
# Run this once AUR registration is back. It:
# 1. Waits for https://aur.archlinux.org/register to stop returning 503,
# then opens it in a browser and prints the SSH key to paste.
# (Signup is the ONLY manual step: it needs an email confirmation, and
# creating accounts is not something this script will do for you.)
# 2. Waits for `ssh aur@aur.archlinux.org` to authenticate.
# 3. From there, fully unattended, for each package:
# newest release tag -> pkgver -> updpkgsums -> makepkg -C -> namcap
# -> pacman -U -> launch check -> .SRCINFO -> clone AUR repo
# -> commit -> push.
#
# Every package is BUILT, INSTALLED and LAUNCHED before it is pushed, so a
# broken package cannot reach the AUR. A package that fails any gate is
# skipped and reported; the others still publish.
#
# Idempotent and resumable: re-running skips work that is already done, and
# an already-published package is updated rather than duplicated.
# ============================================================================
set -euo pipefail
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
readonly SCRIPT_DIR
readonly GATE="$SCRIPT_DIR/anubis_gate.py"
readonly AUR_HOST="aur.archlinux.org"
readonly SSH_KEY="$HOME/.ssh/aur"
readonly AUR_ROOT="$HOME/sdk/aur"
# package:repo:wrapper-process:cli-name:expected-startup-string
# The wrapper process name is NOT derivable from the package name
# (diet-guard-app runs diet_guard_desktop), so it is listed explicitly.
readonly PACKAGES=(
"todo-flutter:$HOME/src/todo:todo_desktop:todo:serving on http://localhost:8730"
"habit-stack:$HOME/src/habit_stack:habit_stack_desktop:habit-stack:serving on http://localhost:8731"
"diet-guard-app:$HOME/src/diet-guard:diet_guard_desktop:diet-guard-app:serving on http://localhost:8732"
)
POLL_SECONDS=300
SKIP_WAIT=0
DRY_RUN=0
ONLY=""
FAILED=()
PUBLISHED=()
usage() {
cat <<EOF
Usage: $(basename "$0") [OPTIONS]
Options:
--skip-wait Assume the account + SSH key already work; go straight
to building and publishing.
--dry-run Do everything except the final 'git push' to the AUR.
--only NAME Publish just one package (todo-flutter, habit-stack,
diet-guard-app).
--poll SECONDS How often to re-check /register (default: $POLL_SECONDS).
-h, --help Show this help.
EOF
exit 0
}
log() { printf '\n\033[1;34m==>\033[0m %s\n' "$*"; }
info() { printf ' %s\n' "$*"; }
warn() { printf '\033[1;33m ! %s\033[0m\n' "$*" >&2; }
die() { printf '\033[1;31m==> ERROR: %s\033[0m\n' "$*" >&2; exit 1; }
require_tools() {
local missing=()
local tool
for tool in git makepkg updpkgsums namcap ssh python3 curl; do
command -v "$tool" >/dev/null 2>&1 || missing+=("$tool")
done
if ((${#missing[@]})); then
info "Installing missing tools: ${missing[*]}"
# namcap/updpkgsums come from namcap + pacman-contrib; the rest are base.
sudo pacman -S --needed --noconfirm namcap pacman-contrib git openssh python \
|| die "could not install: ${missing[*]}"
fi
}
# ---------------------------------------------------------------------------
# Phase 1: wait for registration, then for the key to work
# ---------------------------------------------------------------------------
# The rest lives in lib/, sourced (not executed) so every function sees the
# readonly config above and the mutable FAILED/PUBLISHED arrays main() reads.
# shellcheck source=lib/aur_registration.sh
source "$SCRIPT_DIR/lib/aur_registration.sh"
# shellcheck source=lib/aur_verify.sh
source "$SCRIPT_DIR/lib/aur_verify.sh"
# shellcheck source=lib/aur_publish_one.sh
source "$SCRIPT_DIR/lib/aur_publish_one.sh"
main() {
require_tools
[[ -f "$GATE" ]] || die "missing $GATE"
mkdir -p "$AUR_ROOT"
ensure_key
if ((SKIP_WAIT)); then
ssh_works || die "SSH key is not accepted yet; drop --skip-wait to wait for it"
else
wait_for_registration
wait_for_ssh
fi
local entry pkg repo wrapper binary expected
for entry in "${PACKAGES[@]}"; do
IFS=: read -r pkg repo wrapper binary expected <<<"$entry"
[[ -z "$ONLY" || "$ONLY" == "$pkg" ]] || continue
if ! publish_one "$pkg" "$repo" "$wrapper" "$binary" "$expected"; then
FAILED+=("$pkg")
fi
done
log "Summary"
if ((${#PUBLISHED[@]})); then
printf ' published: %s\n' "${PUBLISHED[@]}"
fi
if ((${#FAILED[@]})); then
printf '\033[1;31m FAILED: %s\033[0m\n' "${FAILED[@]}"
info "Nothing broken was pushed; fix and re-run (safe to repeat)."
return 1
fi
if ((${#PUBLISHED[@]} == 0)); then
warn "nothing to do"
return 1
fi
printf '\n Check: https://aur.archlinux.org/packages/todo-flutter\n'
return 0
}
while [[ $# -gt 0 ]]; do
case $1 in
--skip-wait) SKIP_WAIT=1; shift ;;
--dry-run) DRY_RUN=1; shift ;;
--only) ONLY="$2"; shift 2 ;;
--poll) POLL_SECONDS="$2"; shift 2 ;;
-h|--help) usage ;;
*) die "Unknown option: $1" ;;
esac
done
main "$@"

View File

@ -1,12 +0,0 @@
# code_runner
Runs a learner's solution (Python 3, JavaScript, TypeScript) against a problem statement's examples inside bubblewrap: no network, read-only `/usr`, scratch dir, CPU/memory/process/file/time limits.
```python
import code_runner as cr
statement = ... # markdown from ~/.claude/skills/leetcode/fetch_problem.py
results = cr.run("python3", code, cr.extract_starter(statement), cr.parse_examples(statement))
```
`run(lang, code, starter, examples) -> list[CaseResult]` with `n, input, expected, actual, ok, stdout, error, ms`. One sandbox process per example. TypeScript runs through node's type stripping (no type check, no enums). Needs `bwrap`, `prlimit`, `/usr/bin/python3`, node >= 22.18. Install: `./install.sh`.

View File

@ -1,20 +0,0 @@
"""Run a learner's solution against a problem's examples in a bubblewrap sandbox."""
from __future__ import annotations
from code_runner._examples import parse_examples
from code_runner._model import CaseResult, Example, Limits, RunnerError
from code_runner._run import run
from code_runner._starter import Entry, extract_starter, parse_starter
__all__ = [
"CaseResult",
"Entry",
"Example",
"Limits",
"RunnerError",
"extract_starter",
"parse_examples",
"parse_starter",
"run",
]

View File

@ -1,89 +0,0 @@
"""Output comparison: exact JSON equality, float tolerance, order rules."""
from __future__ import annotations
import json
import math
TOLERANCE = 1e-5 # LeetCode accepts floating answers within 1e-5
MAX_DEPTH = 64 # real answers nest a handful of levels; deeper is hostile
class TooDeepError(ValueError):
"""A compared value nests deeper than ``MAX_DEPTH``."""
def too_deep(value: object, limit: int = MAX_DEPTH) -> bool:
"""Whether ``value`` nests lists/dicts deeper than ``limit`` (iterative)."""
stack: list[tuple[object, int]] = [(value, 0)]
while stack:
node, depth = stack.pop()
if isinstance(node, list | dict):
if depth >= limit:
return True
children = node if isinstance(node, list) else node.values()
stack.extend((child, depth + 1) for child in children)
return False
def _as_number(value: object) -> float | None:
if isinstance(value, bool) or not isinstance(value, int | float):
return None
return value
def _key(value: object) -> str:
return json.dumps(value, sort_keys=True, ensure_ascii=False)
def _deep_sorted(value: object) -> object:
if isinstance(value, list):
return sorted((_deep_sorted(item) for item in value), key=_key)
return value
def _equal(expected: object, actual: object) -> bool:
want, got = _as_number(expected), _as_number(actual)
if want is not None and got is not None:
return math.isclose(want, got, rel_tol=TOLERANCE, abs_tol=TOLERANCE)
if isinstance(expected, list) and isinstance(actual, list):
return len(expected) == len(actual) and all(
_equal(e, a) for e, a in zip(expected, actual, strict=True)
)
if isinstance(expected, dict) and isinstance(actual, dict):
return expected.keys() == actual.keys() and all(
_equal(expected[k], actual[k]) for k in expected
)
return type(expected) is type(actual) and expected == actual
def _top_sorted(value: object) -> object:
if isinstance(value, list):
return sorted(value, key=_key)
return value
def matches(expected: object, actual: object, order: str) -> bool:
"""Whether ``actual`` is an accepted answer for ``expected``.
Args:
expected: the statement's answer (JSON value).
actual: what the solution returned (JSON value).
order: "exact", "any" (list order free) or "deep" (nested order free too).
Raises:
ValueError: ``order`` is none of those.
TooDeepError: a value nests deeper than ``MAX_DEPTH`` (the recursive
walks below would blow the stack).
"""
if too_deep(expected) or too_deep(actual):
msg = f"value nested deeper than {MAX_DEPTH} levels"
raise TooDeepError(msg)
if order == "exact":
return _equal(expected, actual)
if order == "any":
return _equal(_top_sorted(expected), _top_sorted(actual))
if order == "deep":
return _equal(_deep_sorted(expected), _deep_sorted(actual))
msg = f"unknown order {order!r}"
raise ValueError(msg)

View File

@ -1,75 +0,0 @@
from __future__ import annotations
import ast
import json
import logging
import re
from typing import Final
from code_runner._model import Example, Order
from code_runner._split import split_top_level
_logger: Final = logging.getLogger(__name__)
_LABEL: Final = re.compile(r"^(Input|Output)\s*:\s*(.*)$")
_PAIR: Final = re.compile(r"^([A-Za-z_]\w*)\s*=\s*(.+)$", re.DOTALL)
_ANY_ORDER: Final = re.compile(r"any order|order of the output", re.IGNORECASE)
_DEEP_ORDER: Final = re.compile(
r"order of the (?:triplets|quadruplets|subsets|groups|pairs|elements)",
re.IGNORECASE,
)
def _value(text: str) -> object:
try:
return json.loads(text)
except ValueError:
try:
return ast.literal_eval(text)
except (ValueError, SyntaxError) as exc:
raise ValueError(text[:40]) from exc
def _arguments(text: str) -> dict[str, object]:
args: dict[str, object] = {}
for part in split_top_level(text):
pair = _PAIR.match(part)
if pair is None:
msg = f"not a name = value pair: {part[:40]!r}"
raise ValueError(msg)
args[pair.group(1)] = _value(pair.group(2))
return args
def _labels(body: str) -> list[tuple[str, str]]:
cleaned = (
line.replace("**", "").replace("`", "").strip() for line in body.splitlines()
)
matched = (_LABEL.match(line) for line in cleaned)
return [(m.group(1), m.group(2).strip()) for m in matched if m is not None]
def _order(body: str) -> Order:
if _DEEP_ORDER.search(body):
return "deep"
return "any" if _ANY_ORDER.search(body) else "exact"
def parse_examples(statement: str) -> list[Example]:
body = re.split(r"^## ", statement, maxsplit=1, flags=re.MULTILINE)[0]
order = _order(body)
found: list[Example] = []
pending: str | None = None
number = 0
for label, text in _labels(body):
if label == "Input":
pending = text
continue
if pending is None:
continue
number += 1
try:
found.append(Example(number, _arguments(pending), _value(text), order))
except ValueError as exc:
_logger.warning("example %d skipped, not runnable: %s", number, exc)
pending = None
return found

View File

@ -1,119 +0,0 @@
from __future__ import annotations
from dataclasses import dataclass
from importlib import resources
import json
import os
from pathlib import Path
import shutil
from typing import TYPE_CHECKING
from code_runner._model import RunnerError
from code_runner._sandbox import WORK
if TYPE_CHECKING:
from code_runner._starter import Entry
MEGABYTE = 1024 * 1024
_NODE_FLAGS = (
"--permission",
f"--allow-fs-read={WORK}",
f"--allow-fs-write={WORK}",
"--max-old-space-size=128",
)
@dataclass(frozen=True)
class Language:
name: str
solution_file: str
harness_file: str
program: str
binary_env: str
binary_default: str
flags: tuple[str, ...]
memory_bytes: int
def resolve(self) -> Path:
for candidate in (
os.environ.get(self.binary_env),
self.binary_default,
shutil.which(self.program),
):
if candidate and Path(candidate).is_file():
return Path(candidate).resolve()
msg = f"{self.program} not found (set {self.binary_env})"
raise RunnerError(msg)
def command(self, binary: Path) -> list[str]:
return [str(binary), *self.flags, f"{WORK}/{self.harness_file}", WORK]
def ro_binds(self, binary: Path) -> tuple[str, ...]:
if binary.is_relative_to("/usr"):
return ()
return (str(binary.parent.parent),)
def prepare(
self, scratch: Path, code: str, entry: Entry, args: list[object]
) -> None:
harness = resources.files("code_runner").joinpath("harness", self.harness_file)
(scratch / self.harness_file).write_text(harness.read_text(encoding="utf-8"))
text = code
if self.name == "typescript":
text += f"\nmodule.exports = {{ __entry: {entry.cls or entry.name} }};\n"
(scratch / self.solution_file).write_text(text, encoding="utf-8")
spec = {
"cls": entry.cls,
"fn": entry.name,
"args": args,
"typescript": self.name == "typescript",
}
(scratch / "input.json").write_text(json.dumps(spec), encoding="utf-8")
PYTHON = Language(
"python3",
"solution.py",
"harness.py",
"python3",
"CODE_RUNNER_PYTHON",
"/usr/bin/python3",
("-I", "-B"),
512 * MEGABYTE,
)
JAVASCRIPT = Language(
"javascript",
"solution.js",
"harness.js",
"node",
"CODE_RUNNER_NODE",
"/usr/bin/node",
_NODE_FLAGS,
2048 * MEGABYTE,
)
TYPESCRIPT = Language(
"typescript",
"solution.cts",
"harness.js",
"node",
"CODE_RUNNER_NODE",
"/usr/bin/node",
_NODE_FLAGS,
2048 * MEGABYTE,
)
_BY_NAME = {
"python3": PYTHON,
"python": PYTHON,
"javascript": JAVASCRIPT,
"js": JAVASCRIPT,
"typescript": TYPESCRIPT,
"ts": TYPESCRIPT,
}
def get_language(name: str) -> Language:
try:
return _BY_NAME[name.lower()]
except KeyError:
msg = f"unsupported language {name!r}; use python3, javascript or typescript"
raise RunnerError(msg) from None

View File

@ -1,40 +0,0 @@
from __future__ import annotations
from dataclasses import dataclass
from typing import Literal
# "any": list order is free; "deep": the order inside its items is free too.
Order = Literal["exact", "any", "deep"]
MAX_CAPTURE = 64 * 1024
class RunnerError(Exception):
pass
@dataclass(frozen=True)
class Example:
n: int
args: dict[str, object]
expected: object
order: Order = "exact"
@dataclass(frozen=True)
class CaseResult:
n: int
input: dict[str, object]
expected: object
actual: object
ok: bool
stdout: str = ""
error: str | None = None
ms: float = 0.0
@dataclass(frozen=True)
class Limits:
timeout_s: float = 2.0
nproc: int = 64
fsize_bytes: int = 1024 * 1024
nofile: int = 256

View File

@ -1,134 +0,0 @@
from __future__ import annotations
from collections.abc import Callable, Sequence
from dataclasses import dataclass
import json
import os
from pathlib import Path
import stat
import tempfile
from code_runner._compare import TooDeepError, matches, too_deep
from code_runner._languages import Language, get_language
from code_runner._model import MAX_CAPTURE, CaseResult, Example, Limits
from code_runner._sandbox import BoxResult, execute
from code_runner._starter import Entry, parse_starter
Box = Callable[[list[str], Path, Limits, int, tuple[str, ...]], BoxResult]
_OUT_OF_MEMORY = ("out of memory", "allocation failed")
_MAX_ERROR = 200
# Everything hostile JSON can raise on the host: deep nesting, huge numbers.
_HOSTILE = (ValueError, TypeError, RecursionError, MemoryError, OverflowError)
def _positional(example: Example, entry: Entry) -> list[object]:
if set(example.args) == set(entry.params):
return [example.args[name] for name in entry.params]
if len(example.args) == len(entry.params):
return list(example.args.values())
given, wanted = len(example.args), len(entry.params)
msg = f"ArgumentError: example gives {given} arguments, {entry.name} takes {wanted}"
raise ValueError(msg)
def _read_result(scratch: Path) -> dict[str, object] | None:
# The scratch dir is the solution's: never follow a link it planted there.
flags = os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK
try:
fd = os.open(scratch / "result.json", flags)
except OSError:
return None
with os.fdopen(fd, "rb") as handle:
if not stat.S_ISREG(os.fstat(handle.fileno()).st_mode):
return None
raw = handle.read(MAX_CAPTURE * 16)
try:
payload = json.loads(raw)
except _HOSTILE: # ValueError, or RecursionError on "[" * 200000
return None
if not isinstance(payload, dict) or "ok" not in payload:
return None
return None if too_deep(payload) else payload
def _crash(box: BoxResult) -> str:
low = box.stderr.lower()
if any(marker in low for marker in _OUT_OF_MEMORY):
return "MemoryLimitExceeded"
return f"Crashed (exit {box.returncode})"
def _took(ms: object, fallback: float) -> float:
if isinstance(ms, bool) or not isinstance(ms, int | float):
return fallback
try:
return float(ms)
except OverflowError: # an int too big for a float
return fallback
def _outcome(scratch: Path, box: BoxResult) -> tuple[object, str | None, float]:
if box.timed_out:
return None, "TimeLimitExceeded", box.elapsed_ms
payload = _read_result(scratch)
if payload is None:
return None, _crash(box), box.elapsed_ms
took = _took(payload.get("ms"), box.elapsed_ms)
if payload["ok"] is True:
return payload.get("value"), None, took
return None, str(payload.get("error", "Error"))[:_MAX_ERROR], took
@dataclass(frozen=True)
class _Job:
language: Language
binary: Path
code: str
entry: Entry
limits: Limits
box: Box
def _case(job: _Job, example: Example) -> CaseResult:
try:
args = _positional(example, job.entry)
except ValueError as exc:
return CaseResult(
example.n, example.args, example.expected, None, ok=False, error=str(exc)
)
language = job.language
with tempfile.TemporaryDirectory(prefix="code-runner-") as tmp:
scratch = Path(tmp)
language.prepare(scratch, job.code, job.entry, args)
done = job.box(
language.command(job.binary),
scratch,
job.limits,
language.memory_bytes,
language.ro_binds(job.binary),
)
actual, error, ms = _outcome(scratch, done)
ok = False
if error is None:
try:
ok = matches(example.expected, actual, example.order)
except (TooDeepError, RecursionError, MemoryError) as exc:
actual, error = None, f"InvalidResult: {type(exc).__name__}"
return CaseResult(
example.n, example.args, example.expected, actual, ok, done.stdout, error, ms
)
def run(
lang: str,
code: str,
starter: str,
examples: Sequence[Example],
*,
limits: Limits | None = None,
box: Box = execute,
) -> list[CaseResult]:
language = get_language(lang)
entry = parse_starter(language.name, starter)
job = _Job(language, language.resolve(), code, entry, limits or Limits(), box)
return [_case(job, example) for example in examples]

View File

@ -1,147 +0,0 @@
from __future__ import annotations
from dataclasses import dataclass
import functools
import math
import os
from pathlib import Path
import signal
import subprocess
import tempfile
import time
from typing import IO
from code_runner._model import MAX_CAPTURE, Limits, RunnerError
WORK = "/work"
PRLIMIT = "/usr/bin/prlimit"
_ROOT_LINKS = ("bin", "sbin", "lib", "lib64")
@dataclass(frozen=True)
class BoxResult:
returncode: int
timed_out: bool
stdout: str
stderr: str
elapsed_ms: float
def _root_links() -> list[str]:
args: list[str] = []
for name in _ROOT_LINKS:
path = Path("/") / name
if path.is_symlink():
args += ["--symlink", str(path.readlink()), str(path)]
elif path.is_dir():
args += ["--ro-bind", str(path), str(path)]
return args
def _base_argv() -> list[str]:
return [
"bwrap",
"--unshare-all", # no --share-net: no network at all, loopback included
"--die-with-parent",
"--new-session",
"--clearenv",
"--ro-bind",
"/usr",
"/usr",
*_root_links(),
"--proc",
"/proc",
"--dev",
"/dev",
"--tmpfs",
"/tmp",
]
def bwrap_argv(
command: list[str],
scratch: Path,
limits: Limits,
memory_bytes: int,
ro_binds: tuple[str, ...] = (),
) -> list[str]:
argv = _base_argv()
for bind in ro_binds:
argv += ["--ro-bind", bind, bind]
argv += ["--bind", str(scratch), WORK, "--remount-ro", "/", "--chdir", WORK]
for name, value in (
("HOME", WORK),
("PATH", "/usr/bin"),
("LANG", "C.UTF-8"),
("PYTHONIOENCODING", "utf-8"),
):
argv += ["--setenv", name, value]
cpu = math.ceil(limits.timeout_s) + 1
return [
*argv,
"--",
PRLIMIT,
f"--as={memory_bytes}",
f"--nproc={limits.nproc}",
f"--fsize={limits.fsize_bytes}",
f"--nofile={limits.nofile}",
f"--cpu={cpu}",
"--core=0",
"--",
*command,
]
@functools.cache
def check_sandbox() -> None:
if not Path(PRLIMIT).is_file():
msg = f"{PRLIMIT} is missing (util-linux)"
raise RunnerError(msg)
try:
probe = subprocess.run(
[*_base_argv(), "/usr/bin/true"],
capture_output=True,
check=False,
timeout=20,
)
except OSError as exc:
msg = f"bubblewrap (bwrap) cannot start: {exc}"
raise RunnerError(msg) from exc
if probe.returncode != 0:
detail = probe.stderr.decode(errors="replace").strip()
msg = f"bubblewrap cannot create the sandbox: {detail}"
raise RunnerError(msg)
def _read(handle: IO[bytes]) -> str:
handle.seek(0)
return handle.read(MAX_CAPTURE).decode("utf-8", errors="replace")
def execute(
command: list[str],
scratch: Path,
limits: Limits,
memory_bytes: int,
ro_binds: tuple[str, ...] = (),
) -> BoxResult:
check_sandbox()
argv = bwrap_argv(command, scratch, limits, memory_bytes, ro_binds)
with tempfile.TemporaryFile() as out, tempfile.TemporaryFile() as err:
started = time.monotonic()
with subprocess.Popen(
argv,
stdin=subprocess.DEVNULL,
stdout=out,
stderr=err,
env={"PATH": "/usr/bin:/bin"},
start_new_session=True,
) as proc:
timed_out = False
try:
proc.wait(timeout=limits.timeout_s)
except subprocess.TimeoutExpired:
timed_out = True
os.killpg(proc.pid, signal.SIGKILL)
elapsed = (time.monotonic() - started) * 1000
return BoxResult(proc.returncode, timed_out, _read(out), _read(err), elapsed)

View File

@ -1,29 +0,0 @@
from __future__ import annotations
_OPEN = "([{<"
_CLOSE = ")]}>"
def split_top_level(text: str) -> list[str]:
parts: list[str] = []
depth = 0
quote = ""
escaped = False
start = 0
for i, ch in enumerate(text):
if escaped:
escaped = False
elif quote:
escaped = ch == "\\"
quote = "" if ch == quote else quote
elif ch in "\"'":
quote = ch
elif ch in _OPEN:
depth += 1
elif ch in _CLOSE and not (ch == ">" and text[i - 1] in "=-"):
depth = max(depth - 1, 0)
elif ch == "," and depth == 0:
parts.append(text[start:i])
start = i + 1
parts.append(text[start:])
return [part.strip() for part in parts]

View File

@ -1,108 +0,0 @@
from __future__ import annotations
from dataclasses import dataclass
import re
from code_runner._model import RunnerError
from code_runner._split import split_top_level
_IDENT = r"[A-Za-z_$][\w$]*"
_BLOCK_COMMENT = re.compile(r"/\*.*?\*/", re.DOTALL)
_LINE_COMMENT = re.compile(r"^\s*(?://|#).*$", re.MULTILINE)
_PY_CLASS = re.compile(r"^class\s+(\w+)", re.MULTILINE)
_PY_DEF = re.compile(r"\bdef\s+(?!__)(\w+)\s*\(")
_PY_FUNC = re.compile(r"^def\s+(\w+)\s*\(", re.MULTILINE)
_JS_CLASS = re.compile(rf"^\s*class\s+({_IDENT})", re.MULTILINE)
_JS_METHOD = re.compile(
rf"^\s+(?:async\s+)?(?!constructor\b)({_IDENT})\s*\(", re.MULTILINE
)
_JS_ASSIGNED = re.compile(
rf"\b(?:var|let|const)\s+({_IDENT})\s*(?::[^=]+)?=\s*(?:async\s+)?(?:function\b\s*\w*\s*)?\("
)
_JS_DECLARED = re.compile(rf"\bfunction\s+({_IDENT})\s*\(")
_NOT_METHODS = {"if", "for", "while", "switch", "catch", "return", "function"}
@dataclass(frozen=True)
class Entry:
cls: str | None
name: str
params: tuple[str, ...]
def _no_entry(lang: str) -> RunnerError:
return RunnerError(f"no entry point found in the {lang} starter code")
def _balanced(text: str, open_at: int) -> str:
depth = 0
for i in range(open_at, len(text)):
ch = text[i]
if ch in "([{" or (ch == "<" and depth > 0):
depth += 1
elif ch in ")]}" or (ch == ">" and text[i - 1] != "=" and depth > 1):
depth -= 1
if depth == 0:
return text[open_at + 1 : i]
return text[open_at + 1 :]
def _params(code: str, open_at: int) -> tuple[str, ...]:
names: list[str] = []
for part in split_top_level(_balanced(code, open_at)):
found = re.match(_IDENT, part.strip())
if found:
names.append(found.group())
return tuple(names)
def _python(text: str) -> Entry | None:
code = _LINE_COMMENT.sub("", text)
owner = _PY_CLASS.search(code)
if owner is None:
found = _PY_FUNC.search(code)
return found and Entry(None, found.group(1), _params(code, found.end() - 1))
found = _PY_DEF.search(code, owner.end())
if found is None:
return None
params = _params(code, found.end() - 1)
return Entry(
owner.group(1), found.group(1), tuple(p for p in params if p != "self")
)
def _javascript(text: str) -> Entry | None:
code = _LINE_COMMENT.sub("", _BLOCK_COMMENT.sub("", text))
owner = _JS_CLASS.search(code)
if owner is not None:
for method in _JS_METHOD.finditer(code, owner.end()):
if method.group(1) not in _NOT_METHODS:
return Entry(
owner.group(1), method.group(1), _params(code, method.end() - 1)
)
return None
found = _JS_ASSIGNED.search(code) or _JS_DECLARED.search(code)
return found and Entry(None, found.group(1), _params(code, found.end() - 1))
def parse_starter(lang: str, text: str) -> Entry:
entry = _python(text) if lang == "python3" else _javascript(text)
if entry is None:
raise _no_entry(lang)
return entry
def extract_starter(statement: str) -> str:
lines = statement.splitlines(keepends=True)
heads = [i for i, line in enumerate(lines) if line.startswith("## Starter code")]
if not heads:
return ""
body = lines[heads[0] + 1 :]
if body and body[0].startswith("```"):
body = body[1:]
end = next(
(i for i, line in enumerate(body) if line.startswith("```")), len(body)
)
return "".join(body[:end])
end = next((i for i, line in enumerate(body) if line.startswith("## ")), len(body))
return "".join(body[:end])

View File

@ -1,45 +0,0 @@
"use strict";
// Runs inside the sandbox: call the solution once, write result.json.
const fs = require("node:fs");
const path = require("node:path");
const root = process.argv[2];
const spec = JSON.parse(fs.readFileSync(path.join(root, "input.json"), "utf8"));
const IDENT = /^[A-Za-z_$][\w$]*$/;
function describe(e) {
const name = e && typeof e.name === "string" ? e.name : "Error";
const message = e && e.message !== undefined ? e.message : String(e);
return `${name}: ${message}`.slice(0, 300);
}
function writeResult(payload) {
// Unique name + rename: a symlink planted at result.json is replaced, never followed.
const unique = path.join(root, `.result-${process.pid}-${Math.random().toString(36).slice(2)}`);
fs.writeFileSync(unique, JSON.stringify(payload), { flag: "wx" });
fs.renameSync(unique, path.join(root, "result.json"));
}
function load(target) {
if (!IDENT.test(target)) throw new TypeError("bad entry name");
if (spec.typescript) return require(path.join(root, "solution.cts")).__entry;
const source = fs.readFileSync(path.join(root, "solution.js"), "utf8");
return new Function(`${source}\n;return ${target};`)();
}
function call() {
const entry = load(spec.cls || spec.fn);
if (spec.cls) return new entry()[spec.fn](...spec.args);
return entry(...spec.args);
}
const started = process.hrtime.bigint();
let payload;
try {
const value = call();
payload = { ok: true, value: value === undefined ? null : JSON.parse(JSON.stringify(value)) };
} catch (e) {
payload = { ok: false, error: describe(e) };
}
payload.ms = Number(process.hrtime.bigint() - started) / 1e6;
writeResult(payload);

View File

@ -1,69 +0,0 @@
"""Runs inside the sandbox: call the solution once, write result.json."""
from __future__ import annotations
import json
import os
from pathlib import Path
import sys
import tempfile
import time
from typing import Any
PRELUDE = (
"from typing import *\n"
"import bisect, collections, functools, heapq, itertools, math\n"
"import operator, re, string\n"
)
MAX_MESSAGE = 300
def describe(exc: BaseException) -> str:
return f"{type(exc).__name__}: {exc}"[:MAX_MESSAGE]
def write_result(root: Path, payload: dict[str, object]) -> None:
# Unique name + rename: a symlink the solution planted at result.json is
# replaced by this file, never followed.
handle, name = tempfile.mkstemp(dir=root, prefix=".result-")
with os.fdopen(handle, "w", encoding="utf-8") as out:
json.dump(payload, out)
Path(name).replace(root / "result.json")
def call(root: Path, spec: dict[str, Any]) -> object:
namespace: dict[str, Any] = {"__name__": "solution"}
exec(PRELUDE, namespace)
source = (root / "solution.py").read_text(encoding="utf-8")
exec(compile(source, "solution.py", "exec"), namespace)
owner, name = spec["cls"], str(spec["fn"])
if owner is None:
target = namespace.get(name)
else:
cls = namespace.get(str(owner))
target = getattr(cls() if cls else None, name, None)
if target is None:
message = f"name {owner or name!r} is not defined"
raise NameError(message)
return target(*spec["args"])
def main(directory: str) -> None:
root = Path(directory)
spec = json.loads((root / "input.json").read_text(encoding="utf-8"))
started = time.perf_counter()
try:
value = call(root, spec)
payload: dict[str, object] = {
"ok": True,
"value": json.loads(json.dumps(value)),
}
except BaseException as exc:
payload = {"ok": False, "error": describe(exc)}
payload["ms"] = (time.perf_counter() - started) * 1000
sys.stdout.flush()
write_result(root, payload)
if __name__ == "__main__":
main(sys.argv[1])

View File

@ -1,70 +0,0 @@
"""Shared fixtures: real-sandbox tests are skipped where bubblewrap cannot run."""
from __future__ import annotations
import shutil
import pytest
from code_runner._model import Example, RunnerError
from code_runner._sandbox import check_sandbox
def _bwrap_works() -> bool:
try:
check_sandbox()
except RunnerError:
return False
return True
needs_sandbox = pytest.mark.skipif(not _bwrap_works(), reason="bwrap/userns unusable")
needs_node = pytest.mark.skipif(
not _bwrap_works() or not shutil.which("node"), reason="node or bwrap missing"
)
TWO_SUM_STARTER = {
"python3": "class Solution:\n def twoSum(self, nums: list[int], target: int) -> list[int]:\n",
"javascript": "var twoSum = function(nums, target) {\n\n};",
"typescript": "function twoSum(nums: number[], target: number): number[] {\n};",
}
TWO_SUM_CODE = {
"python3": (
"class Solution:\n"
" def twoSum(self, nums, target):\n"
" seen = {}\n"
" for i, n in enumerate(nums):\n"
" if target - n in seen:\n"
" return [i, seen[target - n]]\n"
" seen[n] = i\n"
),
"javascript": (
"var twoSum = function(nums, target) {\n"
" const seen = new Map();\n"
" for (let i = 0; i < nums.length; i++) {\n"
" if (seen.has(target - nums[i])) return [i, seen.get(target - nums[i])];\n"
" seen.set(nums[i], i);\n"
" }\n"
"};"
),
"typescript": (
"function twoSum(nums: number[], target: number): number[] {\n"
" const seen = new Map<number, number>();\n"
" for (let i = 0; i < nums.length; i++) {\n"
" const j = seen.get(target - nums[i]);\n"
" if (j !== undefined) return [i, j];\n"
" seen.set(nums[i], i);\n"
" }\n"
" return [];\n"
"}"
),
}
@pytest.fixture
def two_sum_examples() -> list[Example]:
return [
Example(1, {"nums": [2, 7, 11, 15], "target": 9}, [0, 1], "any"),
Example(2, {"nums": [3, 2, 4], "target": 6}, [1, 2], "any"),
Example(3, {"nums": [3, 3], "target": 6}, [0, 1], "any"),
]

View File

@ -1,65 +0,0 @@
import pytest
from code_runner._compare import MAX_DEPTH, TooDeepError, matches, too_deep
@pytest.mark.parametrize(
("expected", "actual", "order", "want"),
[
([0, 1], [0, 1], "exact", True),
([0, 1], [1, 0], "exact", False),
([0, 1], [1, 0], "any", True),
([0, 1], [1, 0, 2], "any", False),
([[1, 2], [3]], [[3], [1, 2]], "any", True),
([[1, 2], [3]], [[3], [2, 1]], "any", False),
([[1, 2], [3]], [[3], [2, 1]], "deep", True),
([[1, 2], [3]], [[3], [2, 9]], "deep", False),
(3, 3.0, "exact", True),
(0.1 + 0.2, 0.3, "exact", True),
(2.0, 2.001, "exact", False),
(True, 1, "exact", False),
(1, True, "exact", False),
(True, True, "exact", True),
(None, None, "exact", True),
(None, 0, "exact", False),
("ab", "ab", "exact", True),
("ab", "ba", "exact", False),
("1", 1, "exact", False),
({"a": [1]}, {"a": [1]}, "exact", True),
({"a": 1}, {"b": 1}, "exact", False),
({"a": 1}, {"a": 2}, "exact", False),
(5, 5, "any", True),
([1], 1, "any", False),
([1.0, 2.0], [2.0, 1.0000001], "any", True),
],
)
def test_matches(expected: object, actual: object, order: str, *, want: bool) -> None:
assert matches(expected, actual, order) is want
def test_unknown_order_is_refused() -> None:
with pytest.raises(ValueError, match="order"):
matches(1, 1, "sideways")
def _nest(depth: int) -> object:
value: object = 1
for _ in range(depth):
value = [value]
return value
@pytest.mark.parametrize("order", ["exact", "any", "deep"])
def test_deep_nesting_is_refused_not_recursed(order: str) -> None:
deep = _nest(100_000) # would raise RecursionError in the recursive walks
with pytest.raises(TooDeepError):
matches(deep, deep, order)
with pytest.raises(TooDeepError):
matches(1, deep, order)
def test_nesting_up_to_the_limit_still_compares() -> None:
assert matches(_nest(MAX_DEPTH), _nest(MAX_DEPTH), "deep") is True
assert too_deep(_nest(MAX_DEPTH + 1)) is True
assert too_deep({"a": [{"b": _nest(MAX_DEPTH)}]}) is True
assert too_deep({"a": [{"b": 1}]}) is False

View File

@ -1,106 +0,0 @@
from code_runner._examples import parse_examples
from code_runner._model import Example
TWO_SUM = """# 1. Two Sum (Easy)
Tags: Array, Hash Table
Return indices.
You can return the answer in any order.
**Example 1:**
**Input:** `nums = [2,7,11,15], target = 9`
**Output:** `[0,1]`
**Explanation:** 2 + 7.
**Example 2:**
**Input:** `nums = [3,2,4], target = 6`
**Output:** `[1,2]`
## Starter code (python3)
```
class Solution:
def twoSum(self, nums, target):
```
"""
FENCED = """# 5. Pal
Output order does not matter.
**Example 1:**
```
Input: s = "a,b", flag = true
Output: "b"
Explanation: Output: not an example
```
**Example 2:**
```
Input: s = 'x', flag = false
Output: None
```
"""
THREE_SUM = """# 15. 3Sum
Notice that the order of the output and the order of the triplets does not matter.
**Example 1:**
**Input:** nums = [-1,0,1,2,-1,-4]
**Output:** [[-1,-1,2],[-1,0,1]]
"""
SKIPPED = """# 2. Add Two Numbers
**Example 1:**
**Input:** l1 = 1->2
**Output:** [1,2]
**Example 2:**
**Input:** a = 1, b = 2
**Output:** 3.00000
**Example 3:**
**Input:** a = 1
**Output:** 1->2
**Example 4:**
**Input:** a
**Output:** 1
"""
def test_two_sum_examples_are_typed_and_ordered_any() -> None:
found = parse_examples(TWO_SUM)
assert found == [
Example(1, {"nums": [2, 7, 11, 15], "target": 9}, [0, 1], "any"),
Example(2, {"nums": [3, 2, 4], "target": 6}, [1, 2], "any"),
]
def test_fenced_examples_and_python_literals() -> None:
found = parse_examples(FENCED)
assert [e.args for e in found] == [
{"s": "a,b", "flag": True},
{"s": "x", "flag": False},
]
assert [e.expected for e in found] == ["b", None]
assert {e.order for e in found} == {"exact"}
def test_triplets_sentence_means_deep_order() -> None:
assert parse_examples(THREE_SUM)[0].order == "deep"
def test_unparseable_examples_are_skipped_but_numbering_is_kept() -> None:
found = parse_examples(SKIPPED)
assert found == [Example(2, {"a": 1, "b": 2}, 3.0, "exact")]
def test_output_without_input_and_empty_statement() -> None:
assert parse_examples("**Output:** 1\n") == []
assert parse_examples("") == []
def test_starter_section_is_never_scanned() -> None:
text = "# t\n## Starter code (python3)\n```\nInput: a = 1\nOutput: 2\n```\n"
assert parse_examples(text) == []

View File

@ -1,235 +0,0 @@
"""Branches that a healthy sandbox never reaches: crashes, bad results, no bwrap."""
from __future__ import annotations
import dataclasses
import os
from pathlib import Path
import subprocess
from typing import TYPE_CHECKING
import pytest
from code_runner import Example, Limits, RunnerError, run
from code_runner._examples import parse_examples
from code_runner._languages import JAVASCRIPT, PYTHON, get_language
from code_runner._run import Box, _read_result
from code_runner._sandbox import BoxResult, bwrap_argv, check_sandbox
from code_runner._starter import parse_starter
if TYPE_CHECKING:
from collections.abc import Iterator
STARTER = "def f(a):\n"
EXAMPLE = Example(1, {"a": 1}, 1)
@pytest.fixture(autouse=True)
def _fresh_sandbox_check() -> Iterator[None]:
check_sandbox.cache_clear()
yield
check_sandbox.cache_clear()
def _fake(result: BoxResult, files: dict[str, str] | None = None) -> Box:
def box(
command: list[str],
scratch: Path,
limits: Limits,
memory: int,
binds: tuple[str, ...],
) -> BoxResult:
for name, text in (files or {}).items():
(scratch / name).write_text(text)
return result
return box
def _box(**kw: object) -> BoxResult:
fields = {
"returncode": 0,
"timed_out": False,
"stdout": "",
"stderr": "",
"elapsed_ms": 7.0,
}
return BoxResult(**(fields | kw))
def _run(box: Box) -> object:
return run("python3", "", STARTER, [EXAMPLE], box=box)[0]
def test_crash_without_result_names_the_exit_code() -> None:
result = _run(_fake(_box(returncode=139)))
assert (result.ok, result.actual, result.error, result.ms) == (
False,
None,
"Crashed (exit 139)",
7.0,
)
def test_out_of_memory_crash_is_named() -> None:
stderr = "FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory"
assert (
_run(_fake(_box(returncode=134, stderr=stderr))).error == "MemoryLimitExceeded"
)
def test_timeout_wins_over_a_result_file() -> None:
files = {"result.json": '{"ok": true, "value": 1}'}
result = _run(_fake(_box(timed_out=True, elapsed_ms=1000.0), files))
assert (result.error, result.ms) == ("TimeLimitExceeded", 1000.0)
def test_result_file_variants() -> None:
good = _run(
_fake(_box(stdout="hi"), {"result.json": '{"ok": true, "value": 1, "ms": 2.5}'})
)
assert (good.ok, good.ms, good.stdout) == (True, 2.5, "hi")
no_ms = _run(_fake(_box(), {"result.json": '{"ok": true, "value": 1}'}))
assert no_ms.ms == 7.0
failed = _run(_fake(_box(), {"result.json": '{"ok": false}'}))
assert failed.error == "Error"
named = _run(
_fake(_box(), {"result.json": '{"ok": false, "error": "KeyError: x"}'})
)
assert named.error == "KeyError: x"
def test_hostile_result_files_cost_one_example_not_the_run() -> None:
hostile = {
"bomb": "[" * 200_000,
"deep": '{"ok": true, "value": ' + "[" * 5000 + "]" * 5000 + "}",
"huge_ms": '{"ok": true, "value": 1, "ms": 1' + "0" * 400 + "}",
"bool_ms": '{"ok": true, "value": 1, "ms": true}',
"long_error": '{"ok": false, "error": "' + "x" * 5000 + '"}',
}
results = {
name: _run(_fake(_box(), {"result.json": text}))
for name, text in hostile.items()
}
assert results["bomb"].error == "Crashed (exit 0)"
assert results["deep"].error == "Crashed (exit 0)"
assert (results["huge_ms"].ok, results["huge_ms"].ms) == (True, 7.0)
assert results["bool_ms"].ms == 7.0
assert len(results["long_error"].error) == 200
def test_too_deep_expected_or_actual_is_that_examples_error() -> None:
deep: object = 1
for _ in range(100):
deep = [deep]
files = {"result.json": '{"ok": true, "value": 1}'}
cases = [Example(1, {"a": 1}, deep), EXAMPLE]
results = run("python3", "", STARTER, cases, box=_fake(_box(), files))
assert (results[0].ok, results[0].actual) == (False, None)
assert results[0].error == "InvalidResult: TooDeepError"
assert results[1].ok is True # the run went on
def test_recursion_in_compare_is_that_examples_error(
monkeypatch: pytest.MonkeyPatch,
) -> None:
def boom(*args: object) -> bool:
raise RecursionError
monkeypatch.setattr("code_runner._run.matches", boom)
files = {"result.json": '{"ok": true, "value": 1}'}
result = _run(_fake(_box(), files))
assert result.error == "InvalidResult: RecursionError"
def test_unreadable_result_files_are_a_crash(tmp_path: Path) -> None:
secret = tmp_path / "secret"
secret.write_text('{"ok": true, "value": "stolen"}')
(tmp_path / "result.json").symlink_to(secret)
assert _read_result(tmp_path) is None
(tmp_path / "result.json").unlink()
os.mkfifo(tmp_path / "result.json")
assert _read_result(tmp_path) is None
(tmp_path / "result.json").unlink()
for text in ("not json", "[1]", '{"no": "ok"}'):
(tmp_path / "result.json").write_text(text)
assert _read_result(tmp_path) is None
assert _read_result(tmp_path / "missing") is None
def test_binary_resolution(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
ghost = dataclasses.replace(
PYTHON, binary_default="/nope", program="no-such-program-xyz"
)
monkeypatch.delenv(ghost.binary_env, raising=False)
with pytest.raises(RunnerError, match="no-such-program-xyz not found"):
ghost.resolve()
tool = tmp_path / "bin" / "python3"
tool.parent.mkdir()
tool.write_text("#!/bin/sh\n")
monkeypatch.setenv(ghost.binary_env, str(tool))
assert ghost.resolve() == tool
assert ghost.ro_binds(tool) == (str(tool.parent.parent),)
assert PYTHON.ro_binds(Path("/usr/bin/python3")) == ()
assert get_language("TS").name == "typescript"
def _has(argv: list[str], seq: list[str]) -> bool:
return any(argv[i : i + len(seq)] == seq for i in range(len(argv)))
def test_argv_binds_links_and_limits(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
monkeypatch.setattr("code_runner._sandbox._ROOT_LINKS", ("usr", "no-such-root-dir"))
argv = bwrap_argv(["prog"], tmp_path, Limits(timeout_s=1.2), 123, ("/opt/x",))
assert _has(argv, ["--ro-bind", "/usr", "/usr"])
assert _has(argv, ["--ro-bind", "/opt/x", "/opt/x"])
assert _has(argv, ["--bind", str(tmp_path), "/work"])
assert "--share-net" not in argv
assert {"--unshare-all", "--die-with-parent", "--clearenv", "--remount-ro"} <= set(
argv
)
assert "--as=123" in argv
assert "--cpu=3" in argv
assert argv[-1] == "prog"
def test_sandbox_preflight_failures(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr("code_runner._sandbox.PRLIMIT", "/no/such/prlimit")
with pytest.raises(RunnerError, match="prlimit"):
check_sandbox()
monkeypatch.undo()
def missing(*args: object, **kwargs: object) -> None:
message = "bwrap"
raise FileNotFoundError(message)
monkeypatch.setattr(subprocess, "run", missing)
with pytest.raises(RunnerError, match="cannot start"):
check_sandbox()
def refused(*args: object, **kwargs: object) -> subprocess.CompletedProcess[bytes]:
return subprocess.CompletedProcess(args, 1, b"", b"bwrap: No permissions\n")
monkeypatch.setattr(subprocess, "run", refused)
with pytest.raises(RunnerError, match="No permissions"):
check_sandbox()
def test_statement_and_starter_edge_cases() -> None:
escaped = parse_examples('**Input:** s = "a\\"b,c", t = 1\n**Output:** "x"\n')
assert escaped[0].args == {"s": 'a"b,c', "t": 1}
assert parse_starter("python3", "def f(a, b").params == ("a", "b")
skipped = "class S {\n if (x) {\n solve(a) {\n }\n}\n"
assert parse_starter("javascript", skipped).name == "solve"
def test_typescript_class_entry_is_exported(tmp_path: Path) -> None:
entry = parse_starter(
"typescript", "class Solution {\n solve(a: number): number {\n }\n}\n"
)
get_language("typescript").prepare(tmp_path, "class Solution {}", entry, [1])
assert "__entry: Solution" in (tmp_path / "solution.cts").read_text()
JAVASCRIPT.prepare(tmp_path, "x", entry, [1])
assert (tmp_path / "solution.js").read_text() == "x"

View File

@ -1,206 +0,0 @@
"""Escape attempts: each one runs hostile code in the real sandbox and must fail."""
from __future__ import annotations
from pathlib import Path
import socket
import subprocess
import time
from typing import Any, cast
import pytest
from code_runner import CaseResult, Example, Limits, run
from code_runner.tests.conftest import needs_node, needs_sandbox
pytestmark = needs_sandbox
STARTER = "def f(a):\n"
FAST = Limits(timeout_s=1.0)
def _one(code: str, lang: str = "python3", limits: Limits = FAST) -> CaseResult:
starter = {"python3": STARTER, "javascript": "var f = function(a) {};"}[lang]
return run(lang, code, starter, [Example(1, {"a": 0}, 1)], limits=limits)[0]
def _pgrep(pattern: str) -> int:
done = subprocess.run(
["/usr/bin/pgrep", "-fc", pattern], capture_output=True, text=True, check=False
)
return int(done.stdout.strip() or 0)
def test_no_network_not_even_host_loopback() -> None:
with socket.socket() as listener:
listener.bind(("127.0.0.1", 0))
listener.listen()
listener.settimeout(0)
port = listener.getsockname()[1]
code = (
"import socket\n"
"def f(a):\n"
" out = []\n"
f" for host, port in [('127.0.0.1', {port}), ('1.1.1.1', 53), ('::1', {port})]:\n"
" try:\n"
" socket.create_connection((host, port), timeout=1).close()\n"
" out.append('connected')\n"
" except OSError:\n"
" out.append('blocked')\n"
" return out\n"
)
result = _one(code)
with pytest.raises(BlockingIOError):
listener.accept()
assert result.actual == ["blocked", "blocked", "blocked"]
@needs_node
def test_no_network_from_node() -> None:
with socket.socket() as listener:
listener.bind(("127.0.0.1", 0))
listener.listen()
port = listener.getsockname()[1]
code = (
"var f = function(a) {\n"
" const net = process.getBuiltinModule('node:net');\n"
" let seen = 'pending';\n"
f" const s = net.connect({port}, '127.0.0.1');\n"
" s.on('connect', () => { seen = 'connected'; });\n"
" s.on('error', () => {});\n"
" return seen;\n"
"};"
)
result = _one(code, "javascript")
listener.settimeout(0.2)
with pytest.raises(TimeoutError):
listener.accept()
assert result.actual == "pending"
def test_writes_outside_scratch_never_reach_the_host(tmp_path: Path) -> None:
marker = tmp_path / "marker"
code = (
"def f(a):\n"
" out = []\n"
f" for path in ['/usr/escape', '/etc/escape', '/escape', {str(marker)!r}]:\n"
" try:\n"
" open(path, 'w').write('x')\n"
" out.append('wrote')\n"
" except OSError:\n"
" out.append('denied')\n"
" return out\n"
)
result = _one(code)
assert result.actual == ["denied", "denied", "denied", "denied"]
assert not marker.exists()
assert not Path("/usr/escape").exists()
def test_host_files_and_environment_are_invisible(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
secret = tmp_path / "secret.txt"
secret.write_text("hunter2")
monkeypatch.setenv("CODE_RUNNER_SECRET", "hunter2")
code = (
"import os\n"
"def f(a):\n"
f" return [os.path.exists({str(secret)!r}), os.path.exists({str(Path.home())!r}),\n"
" 'CODE_RUNNER_SECRET' in os.environ, sorted(os.environ), os.getpid() < 20,\n"
" os.path.exists('/proc/1/environ') and 'hunter2' in open('/proc/1/environ').read()]\n"
)
values = cast("list[Any]", _one(code).actual)
exists, home, env_leak, names, own_pidns, proc_leak = values
assert (exists, home, env_leak, own_pidns, proc_leak) == (
False,
False,
False,
True,
False,
)
assert set(names) <= {"HOME", "LANG", "PATH", "PWD", "PYTHONIOENCODING"}
def test_symlink_in_scratch_cannot_exfiltrate_a_host_file(tmp_path: Path) -> None:
secret = tmp_path / "secret.txt"
secret.write_text("hunter2-secret-contents")
code = (
"import os\n"
"def f(a):\n"
f" os.symlink({str(secret)!r}, '/work/result.json')\n"
" return 7\n"
)
result = _one(code)
assert "hunter2" not in repr(result)
assert result.actual == 7 # the harness replaced the link, never followed it
def test_fork_bomb_is_contained_and_leaves_nothing_behind() -> None:
before = _pgrep("python3 -I /work/harness.py")
code = (
"import os, subprocess\n"
"def f(a):\n"
" subprocess.Popen(['sleep', '97'])\n"
" while True:\n"
" os.fork()\n"
)
started = time.monotonic()
result = _one(code)
assert time.monotonic() - started < 6
assert result.ok is False
assert result.error is not None
assert result.error.startswith(("TimeLimitExceeded", "BlockingIOError"))
time.sleep(0.3)
assert _pgrep("python3 -I /work/harness.py") == before
assert _pgrep("^sleep 97$") == 0
def test_memory_bomb_is_a_visible_error() -> None:
result = _one("def f(a):\n return len(bytearray(8 * 1024**3))\n")
assert result.ok is False
assert result.error is not None
assert result.error.startswith("MemoryError")
@needs_node
def test_node_heap_bomb_is_a_visible_error() -> None:
code = "var f = function(a) { const x = []; for (;;) x.push(new Array(1e6).fill(1)); };"
result = _one(code, "javascript", Limits(timeout_s=5.0))
assert result.ok is False
assert result.error in {"MemoryLimitExceeded", "RangeError: Invalid array length"}
def test_output_flood_is_capped() -> None:
code = "def f(a):\n while True:\n print('x' * 1000)\n"
started = time.monotonic()
result = _one(code)
assert time.monotonic() - started < 6
assert result.ok is False
assert len(result.stdout) <= 64 * 1024
def test_disk_fill_is_capped_by_the_file_size_limit() -> None:
code = (
"def f(a):\n"
" with open('/work/big', 'wb') as fh:\n"
" while True:\n"
" fh.write(b'x' * 65536)\n"
)
result = _one(code)
assert result.ok is False
assert result.error is not None
assert "OSError" in result.error
def test_timeout_kills_every_process_the_solution_started() -> None:
code = (
"import subprocess\n"
"def f(a):\n"
" subprocess.Popen(['sleep', '97'])\n"
" while True:\n"
" pass\n"
)
result = _one(code)
assert result.error == "TimeLimitExceeded"
time.sleep(0.3)
assert _pgrep("^sleep 97$") == 0

View File

@ -1,227 +0,0 @@
"""End to end through the real sandbox: every language, every kind of outcome."""
from __future__ import annotations
import pytest
from code_runner import CaseResult, Example, Limits, RunnerError, run
from code_runner.tests.conftest import (
TWO_SUM_CODE,
TWO_SUM_STARTER,
needs_node,
needs_sandbox,
)
LANGS = [
pytest.param("python3", marks=needs_sandbox),
pytest.param("javascript", marks=needs_node),
pytest.param("typescript", marks=needs_node),
]
FAST = Limits(timeout_s=1.0)
def _two_sum(lang: str, code: str | None, examples: list[Example]) -> list[CaseResult]:
return run(lang, code or TWO_SUM_CODE[lang], TWO_SUM_STARTER[lang], examples)
@pytest.mark.parametrize("lang", LANGS)
def test_correct_solution_passes_every_example(
lang: str, two_sum_examples: list[Example]
) -> None:
results = _two_sum(lang, None, two_sum_examples)
assert [r.ok for r in results] == [True, True, True]
assert [r.n for r in results] == [1, 2, 3]
first = results[0]
assert first.input == {"nums": [2, 7, 11, 15], "target": 9}
assert first.expected == [0, 1]
assert sorted(first.actual) == [0, 1] # any order: [1, 0] is fine
assert first.error is None
assert first.ms >= 0
WRONG = {
"python3": "class Solution:\n def twoSum(self, nums, target):\n return [0, 0]\n",
"javascript": "var twoSum = function(nums, target) { return [0, 0]; };",
"typescript": "function twoSum(nums: number[], target: number): number[] { return [0, 0]; }",
}
@pytest.mark.parametrize("lang", LANGS)
def test_wrong_answer_is_reported_with_actual_value(
lang: str, two_sum_examples: list[Example]
) -> None:
results = _two_sum(lang, WRONG[lang], two_sum_examples)
assert [r.ok for r in results] == [False, False, False]
assert results[0].actual == [0, 0]
assert results[0].error is None
THROWS = {
"python3": "class Solution:\n def twoSum(self, nums, target):\n return nums[99]\n",
"javascript": "var twoSum = function(nums, target) { throw new RangeError('boom'); };",
"typescript": "function twoSum(nums: number[], target: number): number[] { throw new RangeError('boom'); }",
}
@pytest.mark.parametrize("lang", LANGS)
def test_exception_becomes_an_error_per_case(
lang: str, two_sum_examples: list[Example]
) -> None:
results = _two_sum(lang, THROWS[lang], two_sum_examples[:1])
only = results[0]
assert only.ok is False
assert only.actual is None
assert only.error is not None
assert only.error.split(":")[0] in {"IndexError", "RangeError"}
BROKEN = {
"python3": "class Solution:\n def twoSum(self, nums, target) return 1\n",
"javascript": "var twoSum = function(nums, target) { return ; ;; }}",
"typescript": "function twoSum(nums: number[], target: number): number[] { return [ }",
}
@pytest.mark.parametrize("lang", LANGS)
def test_syntax_error_is_reported(lang: str, two_sum_examples: list[Example]) -> None:
results = _two_sum(lang, BROKEN[lang], two_sum_examples[:2])
assert [r.ok for r in results] == [False, False]
assert all(r.error and r.error.startswith("SyntaxError") for r in results)
PRINTS = {
"python3": "class Solution:\n def twoSum(self, nums, target):\n print('hi', target)\n return [0, 1]\n",
"javascript": "var twoSum = function(nums, target) { console.log('hi', target); return [0, 1]; };",
"typescript": "function twoSum(nums: number[], target: number): number[] { console.log('hi', target); return [0, 1]; }",
}
@pytest.mark.parametrize("lang", LANGS)
def test_prints_land_in_stdout_per_case(
lang: str, two_sum_examples: list[Example]
) -> None:
results = _two_sum(lang, PRINTS[lang], two_sum_examples[:2])
assert results[0].stdout.strip() == "hi 9"
assert results[1].stdout.strip() == "hi 6"
@pytest.mark.parametrize("lang", LANGS)
def test_missing_entry_point_in_code_is_a_case_error(
lang: str, two_sum_examples: list[Example]
) -> None:
code = {
"python3": "x = 1\n",
"javascript": "var other = 1;",
"typescript": "const other = 1;",
}
results = _two_sum(lang, code[lang], two_sum_examples[:1])
assert results[0].ok is False
assert results[0].error is not None
assert "twoSum" in results[0].error or "Solution" in results[0].error
def test_class_style_javascript_and_function_style_python() -> None:
js = run(
"javascript",
"class Solution { solve(a, b) { return a + b; } }",
"class Solution {\n solve(a, b) {\n }\n}",
[Example(1, {"a": 1, "b": 2}, 3)],
)
py = run(
"python3",
"def add(a, b):\n return a + b\n",
"def add(a: int, b: int) -> int:\n",
[Example(1, {"a": 1, "b": 2}, 3)],
)
assert js[0].ok is True
assert py[0].ok is True
def test_python_prelude_and_aliases() -> None:
code = "class Solution:\n def f(self, xs: List[int]) -> int:\n return collections.Counter(xs)[1]\n"
starter = "class Solution:\n def f(self, xs: List[int]) -> int:\n"
out = run("python", code, starter, [Example(1, {"xs": [1, 1, 2]}, 2)])
assert out[0].ok is True
def test_positional_fallback_when_statement_names_differ() -> None:
out = run(
"python3",
"def add(a, b):\n return a - b\n",
"def add(a, b):\n",
[Example(1, {"x": 5, "y": 2}, 3)],
)
assert out[0].ok is True
def test_argument_count_mismatch_is_a_case_error_without_running() -> None:
out = run(
"python3",
"def add(a, b):\n return a\n",
"def add(a, b):\n",
[Example(1, {"only": 5}, 3)],
)
assert out[0].ok is False
assert out[0].error is not None
assert out[0].error.startswith("ArgumentError")
def test_non_json_result_is_a_case_error() -> None:
out = run(
"python3",
"def f(a):\n return {1, 2}\n",
"def f(a):\n",
[Example(1, {"a": 1}, [1, 2])],
)
assert out[0].error is not None
assert out[0].error.startswith("TypeError")
def test_deep_order_and_float_tolerance() -> None:
out = run(
"python3",
"def f(a):\n return [[2, 1], [0, -1]]\n",
"def f(a):\n",
[Example(1, {"a": 0}, [[-1, 0], [1, 2]], "deep")],
)
assert out[0].ok is True
def test_timeout_is_per_case() -> None:
out = run(
"python3",
"def f(a):\n while a:\n pass\n return 1\n",
"def f(a):\n",
[Example(1, {"a": 1}, 1), Example(2, {"a": 0}, 1)],
limits=FAST,
)
assert out[0].error == "TimeLimitExceeded"
assert out[0].ok is False
assert out[0].ms >= 900
assert out[1].ok is True
def test_unsupported_language_and_empty_examples() -> None:
with pytest.raises(RunnerError, match="language"):
run("cobol", "", "", [])
assert run("python3", "def f(a): pass\n", "def f(a):\n", []) == []
@needs_sandbox
def test_json_bomb_result_costs_that_example_only() -> None:
code = (
"import os\n"
"def f(a):\n"
" if a:\n"
" open('/work/result.json', 'w').write('[' * 200000)\n"
" os._exit(0)\n"
" return 1\n"
)
out = run(
"python3",
code,
"def f(a):\n",
[Example(1, {"a": 1}, 1), Example(2, {"a": 0}, 1)],
)
assert (out[0].ok, out[0].error) == (False, "Crashed (exit 0)")
assert out[1].ok is True

View File

@ -1,78 +0,0 @@
import pytest
from code_runner._model import RunnerError
from code_runner._starter import Entry, extract_starter, parse_starter
PY_CLASS = """class Solution:
def twoSum(self, nums: list[int], target: int) -> list[int]:
"""
PY_COMMENTED = """# Definition for singly-linked list.
# class ListNode:
# def __init__(self, val=0, next=None):
class Solution:
def solve(self, grid: Dict[str, Tuple[int, int]], k: int = 3, *rest) -> int:
pass
"""
PY_FUNC = "def add(a: int, b: int = 2) -> int:\n pass\n"
JS_VAR = """/**
* @param {number[]} nums
*/
var twoSum = function(nums, target) {
};"""
JS_FUNC = "function isPalindrome(x, y = 3) {\n}\n"
JS_ARROW = "const f = (a, b) => {\n}\n"
JS_CLASS = "class Solution {\n constructor() {}\n solve(nums, ...rest) {\n }\n}\n"
TS_FUNC = "function twoSum(nums: number[], target: number): number[] {\n};"
TS_ARROW = "const f = (a: Map<string, number>, b?: number): number => {\n}\n"
@pytest.mark.parametrize(
("lang", "text", "want"),
[
("python3", PY_CLASS, Entry("Solution", "twoSum", ("nums", "target"))),
("python3", PY_COMMENTED, Entry("Solution", "solve", ("grid", "k"))),
("python3", PY_FUNC, Entry(None, "add", ("a", "b"))),
("javascript", JS_VAR, Entry(None, "twoSum", ("nums", "target"))),
("javascript", JS_FUNC, Entry(None, "isPalindrome", ("x", "y"))),
("javascript", JS_ARROW, Entry(None, "f", ("a", "b"))),
("javascript", JS_CLASS, Entry("Solution", "solve", ("nums",))),
("typescript", TS_FUNC, Entry(None, "twoSum", ("nums", "target"))),
("typescript", TS_ARROW, Entry(None, "f", ("a", "b"))),
],
)
def test_parse_starter(lang: str, text: str, want: Entry) -> None:
assert parse_starter(lang, text) == want
@pytest.mark.parametrize(
("lang", "text"),
[
("python3", "x = 1\n"),
("python3", "class Solution:\n pass\n"),
("javascript", "// nothing here\n"),
("javascript", "class Solution {\n constructor() {}\n}\n"),
("typescript", ""),
],
)
def test_starter_without_entry_is_refused(lang: str, text: str) -> None:
with pytest.raises(RunnerError, match="entry point"):
parse_starter(lang, text)
def test_extract_starter_takes_the_fenced_block() -> None:
statement = (
"# 1. Two Sum (Easy)\n\nbody\n\n## Starter code (python3)\n```\n"
+ PY_CLASS
+ "```\n\n## Site hints\nnope\n"
)
assert extract_starter(statement) == PY_CLASS
def test_extract_starter_without_section_is_empty() -> None:
assert extract_starter("# 1. Two Sum\n\nno starter\n") == ""
def test_extract_starter_unfenced_runs_to_next_section() -> None:
text = "intro\n## Starter code (javascript)\nvar f = function(a) {};\n## Hints\nx\n"
assert extract_starter(text) == "var f = function(a) {};\n"

View File

@ -1,12 +0,0 @@
#!/bin/bash
# Editable user install so /usr/bin/python3 (the tutor server, the CLI) can
# `import code_runner`. Stdlib only; idempotent.
set -euo pipefail
REPO_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
readonly REPO_DIR
readonly SYSTEM_PYTHON="/usr/bin/python3"
"$SYSTEM_PYTHON" -m pip install --user --no-deps --break-system-packages -e "$REPO_DIR"
cd /
"$SYSTEM_PYTHON" -c 'import code_runner; print(code_runner.__file__)'

View File

@ -65,7 +65,7 @@ repos:
types: [python]
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.17.2
rev: v0.17.4
hooks:
- id: ruff
args: [--fix, --unsafe-fixes, --exit-non-zero-on-fix, --show-fixes]

View File

@ -14,6 +14,29 @@ from crdt_sync._config import (
mirror_client_for,
)
from crdt_sync._device import DeviceIdentity, load_device_identity
from crdt_sync._e2e_cipher import (
E2eCipher,
E2eDecryptError,
is_e2e_envelope,
open_log_text,
seal_log_text,
)
from crdt_sync._e2e_key import (
Argon2Params,
E2eHeaderError,
E2eKey,
E2eWrongPassphraseError,
default_e2e_path,
unlock_e2e,
)
from crdt_sync._e2e_store import (
E2eLayout,
E2eLockedError,
EncryptingStore,
copy_to_encrypted,
open_encrypted_client,
reset_e2e_copy_memo,
)
from crdt_sync._firebase import (
DatabaseNotFoundError,
FirebaseSyncClient,
@ -51,10 +74,19 @@ __all__ = [
"CONFIG_DIR",
"CONFIG_FILE",
"PASSWORD_FILE",
"Argon2Params",
"ConfigError",
"CredentialStore",
"DatabaseNotFoundError",
"DeviceIdentity",
"E2eCipher",
"E2eDecryptError",
"E2eHeaderError",
"E2eKey",
"E2eLayout",
"E2eLockedError",
"E2eWrongPassphraseError",
"EncryptingStore",
"Field",
"FileCredentialStore",
"FileSyncStateStore",
@ -81,18 +113,26 @@ __all__ = [
"SyncState",
"SyncStateStore",
"SyncTarget",
"copy_to_encrypted",
"credential_store_for",
"default_e2e_path",
"default_revs_path",
"dump_log",
"firebase_client_for",
"is_e2e_envelope",
"load_device_identity",
"load_log",
"merge_field",
"merge_logs",
"merge_record",
"mirror_client_for",
"open_encrypted_client",
"open_log_text",
"read_log",
"reset_e2e_copy_memo",
"revision_of",
"seal_log_text",
"sync_log",
"unlock_e2e",
"write_log",
]

View File

@ -0,0 +1,57 @@
# Copyright (c) 2026 Krzysztof Rudnicki
"""On-disk E2E state for short-lived daemons (one process per tick).
The derived key is cached ``0600`` beside the app's other state, so Argon2id
runs once per machine rather than once a minute; it is no more sensitive
than ``~/.config/crdt-sync/password``, which it is derived from. The copy
memo records the old prefix's revisions last copied, so an unchanged old
prefix is not re-scanned every tick.
"""
from __future__ import annotations
import hashlib
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from pathlib import Path
_MASTER = 32
def _slug(old_prefix: str) -> str:
return hashlib.sha256(old_prefix.encode()).hexdigest()[:16]
def read_cached_key(state_dir: Path, old_prefix: str) -> bytes | None:
"""Return the cached 32-byte master key, or None."""
path = state_dir / f"e2e-{_slug(old_prefix)}.key"
if not path.exists():
return None
try:
master = bytes.fromhex(path.read_text(encoding="utf-8").strip())
except ValueError:
return None
return master if len(master) == _MASTER else None
def write_cached_key(state_dir: Path, old_prefix: str, master: bytes) -> None:
"""Store the master key owner-only."""
state_dir.mkdir(parents=True, exist_ok=True)
path = state_dir / f"e2e-{_slug(old_prefix)}.key"
path.touch(mode=0o600, exist_ok=True)
path.chmod(0o600)
path.write_text(master.hex(), encoding="utf-8")
def read_copy_memo(state_dir: Path, old_prefix: str) -> str | None:
"""Return the old-prefix fingerprint last copied, or None."""
path = state_dir / f"e2e-{_slug(old_prefix)}.copied"
return path.read_text(encoding="utf-8") if path.exists() else None
def write_copy_memo(state_dir: Path, old_prefix: str, fingerprint: str) -> None:
"""Record that the old prefix was copied at ``fingerprint``."""
state_dir.mkdir(parents=True, exist_ok=True)
path = state_dir / f"e2e-{_slug(old_prefix)}.copied"
path.write_text(fingerprint, encoding="utf-8")

View File

@ -0,0 +1,202 @@
# Copyright (c) 2026 Krzysztof Rudnicki
"""XChaCha20-Poly1305 blob cipher and per-record envelope.
Byte-compatible with ``crdt_sync_dart``'s ``E2eCipher`` and
``sealLogText``/``openLogText``: a blob is base64 ``[version][nonce][ct][tag]``
with ``[version] + utf8(path)`` as AAD; an envelope keeps record ids, the
tombstone and the newest HLC clear and seals each record's fields.
"""
from __future__ import annotations
import base64
import binascii
import json
import secrets
from typing import cast
from nacl.bindings import (
crypto_aead_xchacha20poly1305_ietf_decrypt,
crypto_aead_xchacha20poly1305_ietf_encrypt,
)
from nacl.exceptions import CryptoError
from crdt_sync._hlc import Hlc
E2E_FORMAT_VERSION = 1
_NONCE = 24
_TAG = 16
_KEY = 32
class E2eDecryptError(ValueError):
"""A blob that is tampered, under another key, or in an unknown format.
A ``ValueError`` so the pull loop skips the peer instead of aborting.
"""
def is_plaintext(stored: str) -> bool:
"""Return whether ``stored`` is legacy JSON rather than a sealed blob."""
return stored.lstrip().startswith(("{", "["))
def _aad(version: int, path: str) -> bytes:
return bytes([version]) + path.encode()
class E2eCipher:
"""Seals and opens blobs bound to their remote path."""
def __init__(self, key: bytes, *, accept_plaintext: bool = True) -> None:
"""Wrap a 32-byte ``key``; ``accept_plaintext`` reads old peers."""
if len(key) != _KEY:
msg = "key must be 32 bytes"
raise ValueError(msg)
self._key = bytes(key)
self.accept_plaintext = accept_plaintext
def seal(self, plaintext: str, *, path: str) -> str:
"""Return ``plaintext`` sealed for ``path``."""
nonce = secrets.token_bytes(_NONCE)
box = crypto_aead_xchacha20poly1305_ietf_encrypt(
plaintext.encode(), _aad(E2E_FORMAT_VERSION, path), nonce, self._key
)
return base64.b64encode(bytes([E2E_FORMAT_VERSION]) + nonce + box).decode()
def open(self, stored: str, *, path: str) -> str:
"""Return the plaintext of ``stored``; raise :class:`E2eDecryptError`."""
if is_plaintext(stored):
if self.accept_plaintext:
return stored
msg = f"plaintext blob at {path} after migration"
raise E2eDecryptError(msg)
try:
raw = base64.b64decode(stored, validate=True)
except binascii.Error as exc:
msg = f"blob at {path} is neither JSON nor base64"
raise E2eDecryptError(msg) from exc
if not raw:
msg = f"empty blob at {path}"
raise E2eDecryptError(msg)
if raw[0] != E2E_FORMAT_VERSION:
msg = f"blob at {path} uses format {raw[0]}; update the app"
raise E2eDecryptError(msg)
if len(raw) < 1 + _NONCE + _TAG:
msg = f"blob at {path} is truncated"
raise E2eDecryptError(msg)
try:
clear = crypto_aead_xchacha20poly1305_ietf_decrypt(
raw[1 + _NONCE :], _aad(raw[0], path), raw[1 : 1 + _NONCE], self._key
)
except CryptoError as exc:
msg = f"blob at {path} failed authentication"
raise E2eDecryptError(msg) from exc
return clear.decode()
def is_e2e_envelope(text: str) -> bool:
"""Return whether ``text`` is an envelope (integer ``"v"``)."""
try:
data = json.loads(text)
except ValueError:
return False
version = data.get("v") if isinstance(data, dict) else None
return isinstance(version, int) and not isinstance(version, bool)
def _record_aad(
path: str, rid: str, deleted: object, deleted_hlc: object, newest: object
) -> str:
flag = 1 if deleted is True else 0
return f"{path}#{rid}#{flag}#{deleted_hlc or ''}#{newest or ''}"
def _as_log(text: str) -> dict[str, dict[str, object]] | None:
try:
raw = json.loads(text)
except ValueError:
return None
if not isinstance(raw, dict):
return None
for rid, rec in raw.items():
if not isinstance(rec, dict) or rec.get("id") != rid:
return None
if not isinstance(rec.get("fields"), dict):
return None
return cast("dict[str, dict[str, object]]", raw)
def _newest(fields: dict[str, list[object]]) -> str | None:
clocks = [Hlc.from_str(str(pair[1])) for pair in fields.values()]
return max(clocks).to_str() if clocks else None
def seal_log_text(text: str, cipher: E2eCipher, *, path: str) -> str:
"""Seal a log (``dump_log`` shape) as an envelope; anything else whole."""
log = _as_log(text)
if log is None:
return json.dumps(
{"v": E2E_FORMAT_VERSION, "b": cipher.seal(text, path=path)},
separators=(",", ":"),
)
records: dict[str, object] = {}
for rid, rec in log.items():
fields = cast("dict[str, list[object]]", rec["fields"])
deleted = rec.get("deleted") is True
deleted_hlc = rec.get("deleted_hlc")
newest = _newest(fields)
records[rid] = {
"d": deleted,
"dh": deleted_hlc,
"h": newest,
"p": cipher.seal(
json.dumps(fields, separators=(",", ":"), ensure_ascii=False),
path=_record_aad(path, rid, deleted, deleted_hlc, newest),
),
}
return json.dumps(
{"v": E2E_FORMAT_VERSION, "r": records},
separators=(",", ":"),
ensure_ascii=False,
)
def open_log_text(text: str, cipher: E2eCipher, *, path: str) -> str:
"""Open an envelope back to the ``dump_log`` shape."""
if not is_e2e_envelope(text):
if cipher.accept_plaintext:
return text
msg = f"plaintext blob at {path} after migration"
raise E2eDecryptError(msg)
data = json.loads(text)
if data["v"] != E2E_FORMAT_VERSION:
msg = f"blob at {path} uses format {data['v']}; update the app"
raise E2eDecryptError(msg)
whole = data.get("b")
if isinstance(whole, str):
return cipher.open(whole, path=path)
records = data.get("r")
if not isinstance(records, dict):
msg = f"envelope at {path} is mis-shaped"
raise E2eDecryptError(msg)
out: dict[str, object] = {}
for rid, rec in records.items():
if (
not isinstance(rec, dict)
or not isinstance(rec.get("p"), str)
or not isinstance(rec.get("d"), bool)
):
msg = f"record {rid} at {path} is mis-shaped"
raise E2eDecryptError(msg)
clear = cipher.open(
rec["p"],
path=_record_aad(path, rid, rec["d"], rec.get("dh"), rec.get("h")),
)
out[rid] = {
"id": rid,
"fields": json.loads(clear),
"deleted": rec["d"],
"deleted_hlc": rec.get("dh"),
}
return json.dumps(out, separators=(",", ":"), ensure_ascii=False)

View File

@ -0,0 +1,191 @@
# Copyright (c) 2026 Krzysztof Rudnicki
"""Argon2id key from the shared crdt-sync password, plus HKDF subkeys.
Byte-compatible with ``crdt_sync_dart``'s ``E2eKey``/``unlockE2e``: the clear
header at ``e2e_path`` holds salt, Argon2id cost and a check MAC, never a key.
"""
from __future__ import annotations
import base64
from dataclasses import dataclass
import hashlib
import hmac
import json
import secrets
from typing import TYPE_CHECKING
from argon2.low_level import Type, hash_secret_raw
from crdt_sync._e2e_cipher import E2eCipher
if TYPE_CHECKING:
from crdt_sync._remote import RemoteStore
_SALT = 16
_MASTER = 32
_HEADER_VERSION = 1
@dataclass(frozen=True)
class Argon2Params:
"""Argon2id cost, stored in the header so it can be raised later."""
memory_kib: int = 19456
iterations: int = 2
parallelism: int = 1
class E2eHeaderError(ValueError):
"""The header exists but cannot be parsed; it is never overwritten."""
class E2eWrongPassphraseError(Exception):
"""The passphrase does not match the header's check."""
def default_e2e_path(path_prefix: str) -> str:
"""Return the header path: a sibling of the device dirs, like ``revs``."""
head, sep, _ = path_prefix.rpartition("/")
return f"{head}/e2e" if sep else f"{path_prefix}/e2e"
def _hkdf(master: bytes, label: str) -> bytes:
prk = hmac.new(b"", master, hashlib.sha256).digest()
info = f"crdt_sync e2e {label}".encode()
return hmac.new(prk, info + b"\x01", hashlib.sha256).digest()
@dataclass(frozen=True)
class E2eKey:
"""A 32-byte master key and its subkeys. Never written to the remote."""
master: bytes
enc: bytes
rev: bytes
check: bytes
@classmethod
def from_bytes(cls, master: bytes) -> E2eKey:
"""Expand a cached 32-byte master key."""
if len(master) != _MASTER:
msg = "master key must be 32 bytes"
raise ValueError(msg)
return cls(
master, _hkdf(master, "enc"), _hkdf(master, "rev"), _hkdf(master, "check")
)
def cipher(self, *, accept_plaintext: bool = True) -> E2eCipher:
"""Return the blob cipher for this key."""
return E2eCipher(self.enc, accept_plaintext=accept_plaintext)
def revision(self, text: str) -> str:
"""Return the keyed revision (hex HMAC-SHA256) of ``text``."""
return hmac.new(self.rev, text.encode(), hashlib.sha256).hexdigest()
def check_for(self, salt: bytes) -> str:
"""Return the header check value for ``salt``."""
mac = hmac.new(self.check, salt, hashlib.sha256).digest()
return base64.b64encode(mac).decode()
@dataclass(frozen=True)
class _Header:
salt: bytes
params: Argon2Params
check: str
@classmethod
def parse(cls, text: str) -> _Header:
try:
data = json.loads(text)
except ValueError as exc:
msg = "e2e header is not JSON"
raise E2eHeaderError(msg) from exc
if not isinstance(data, dict) or data.get("v") != _HEADER_VERSION:
msg = "e2e header is not a known version"
raise E2eHeaderError(msg)
if data.get("kdf") != "argon2id":
msg = "e2e header is malformed"
raise E2eHeaderError(msg)
try:
return cls(
base64.b64decode(data["salt"]),
Argon2Params(int(data["m"]), int(data["t"]), int(data["p"])),
str(data["check"]),
)
except (KeyError, TypeError, ValueError) as exc:
msg = "e2e header is malformed"
raise E2eHeaderError(msg) from exc
def encode(self) -> str:
return json.dumps(
{
"v": _HEADER_VERSION,
"kdf": "argon2id",
"salt": base64.b64encode(self.salt).decode(),
"m": self.params.memory_kib,
"t": self.params.iterations,
"p": self.params.parallelism,
"check": self.check,
},
separators=(",", ":"),
)
def derive_key(passphrase: str, salt: bytes, params: Argon2Params) -> E2eKey:
"""Run Argon2id (v1.3) and expand the subkeys."""
master = hash_secret_raw(
secret=passphrase.encode(),
salt=salt,
time_cost=params.iterations,
memory_cost=params.memory_kib,
parallelism=params.parallelism,
hash_len=32,
type=Type.ID,
)
return E2eKey.from_bytes(master)
def read_header(store: RemoteStore, path: str) -> _Header | None:
"""Return the parsed header at ``path``, or None when absent."""
text = store.get_file_text(path)
return None if text is None else _Header.parse(text)
def key_matches(key: E2eKey, store: RemoteStore, path: str) -> bool:
"""Return whether ``key`` matches the header at ``path``."""
header = read_header(store, path)
return header is not None and hmac.compare_digest(
key.check_for(header.salt), header.check
)
def unlock_e2e(
store: RemoteStore,
*,
path_prefix: str,
passphrase: str,
params: Argon2Params | None = None,
e2e_path: str | None = None,
) -> E2eKey:
"""Derive the key from ``passphrase``, creating the header on first use."""
if not passphrase:
msg = "passphrase must not be empty"
raise ValueError(msg)
path = e2e_path or default_e2e_path(path_prefix)
header = read_header(store, path)
if header is None:
salt = secrets.token_bytes(_SALT)
cost = params or Argon2Params()
key = derive_key(passphrase, salt, cost)
encoded = _Header(salt, cost, key.check_for(salt)).encode()
store.put_file_text(path, encoded, message=path)
stored = store.get_file_text(path) or encoded
if stored == encoded:
return key
header = _Header.parse(stored)
key = derive_key(passphrase, header.salt, header.params)
if not hmac.compare_digest(key.check_for(header.salt), header.check):
raise E2eWrongPassphraseError
return key

View File

@ -0,0 +1,267 @@
# Copyright (c) 2026 Krzysztof Rudnicki
"""Encrypted remote store, ``-e2e`` layout and the one-time copy.
Mirrors ``crdt_sync_dart``'s ``EncryptingStore``, ``E2eLayout``,
``copyToEncrypted`` and ``openEncryptedStore`` so Python daemons and Flutter
apps share one encrypted prefix per app.
"""
from __future__ import annotations
from dataclasses import dataclass
import logging
from typing import TYPE_CHECKING
from crdt_sync._e2e_cache import (
read_cached_key,
read_copy_memo,
write_cached_key,
write_copy_memo,
)
from crdt_sync._e2e_cipher import E2eDecryptError, open_log_text, seal_log_text
from crdt_sync._e2e_key import (
Argon2Params,
E2eHeaderError,
E2eKey,
E2eWrongPassphraseError,
key_matches,
unlock_e2e,
)
from crdt_sync._log import merge_logs
from crdt_sync._store import dump_log, load_log
from crdt_sync._sync import default_revs_path
if TYPE_CHECKING:
from pathlib import Path
from crdt_sync._remote import RemoteStore
_logger = logging.getLogger(__name__)
class E2eLockedError(Exception):
"""No usable key: the caller must not sync at all (fail closed)."""
@dataclass(frozen=True)
class E2eLayout:
"""Where an app's encrypted data lives, derived from its old prefix."""
prefix: str
revs_path: str
e2e_path: str
@classmethod
def of(cls, old_prefix: str) -> E2eLayout:
"""Return ``<old>-e2e`` with revs and header kept out of old paths."""
prefix = f"{old_prefix}-e2e"
parent, sep, _ = old_prefix.rpartition("/")
if not sep:
return cls(prefix, f"{prefix}/revs", f"{prefix}/e2e")
return cls(prefix, f"{parent}/revs-e2e", f"{parent}/e2e")
class EncryptingStore:
"""Seals log blobs under the layout's prefix and keys its revisions."""
def __init__(self, inner: RemoteStore, key: E2eKey, layout: E2eLayout) -> None:
"""Wrap ``inner``; other paths pass through unchanged."""
self._inner = inner
self.key = key
self._layout = layout
self._cipher = key.cipher()
def _is_rev(self, path: str) -> bool:
return path.startswith(f"{self._layout.revs_path}/")
def _is_log(self, path: str) -> bool:
return (
not self._is_rev(path)
and path != self._layout.e2e_path
and path.startswith(f"{self._layout.prefix}/")
)
def list_directory(self, path: str) -> list[str]:
"""Delegate."""
return self._inner.list_directory(path)
def get_file_text(self, path: str) -> str | None:
"""Read and open a sealed log; other paths pass through."""
text = self._inner.get_file_text(path)
if text is None or not self._is_log(path):
return text
return open_log_text(text, self._cipher, path=path)
def put_file_text(self, path: str, text: str, *, message: str) -> None:
"""Seal logs and key revisions before writing."""
if self._is_log(path):
text = seal_log_text(text, self._cipher, path=path)
elif self._is_rev(path):
text = self.key.revision(text)
self._inner.put_file_text(path, text, message=message)
def get_string_map(self, path: str) -> dict[str, str]:
"""Bulk read; sealed entries come back opened, bad ones dropped."""
reader = getattr(self._inner, "get_string_map", None)
if reader is None:
return {}
out: dict[str, str] = {}
for name, value in reader(path).items():
full = f"{path}/{name}"
if not self._is_log(full):
out[name] = value
continue
try:
out[name] = open_log_text(value, self._cipher, path=full)
except E2eDecryptError:
_logger.warning("Undecryptable entry %s dropped", full)
return out
def delete_file(self, path: str, *, message: str = "") -> None:
"""Delegate."""
self._inner.delete_file(path, message=message)
def can_access_remote(self) -> bool:
"""Delegate."""
return self._inner.can_access_remote()
def _blobs(store: RemoteStore, root: str) -> dict[str, str]:
"""Every text blob under ``root`` (flat leaves and ``<dir>/<file>``)."""
skip = {default_revs_path(root), f"{root}/e2e", f"{root}/revs"}
out: dict[str, str] = {}
for name in store.list_directory(root):
full = f"{root}/{name}"
if full in skip:
continue
children = store.list_directory(full)
if not children:
text = store.get_file_text(full)
if text is not None:
out[name] = text
continue
for child in children:
text = store.get_file_text(f"{full}/{child}")
if text is not None:
out[f"{name}/{child}"] = text
return out
def copy_to_encrypted(
store: RemoteStore, key: E2eKey, old_prefix: str, layout: E2eLayout
) -> int:
"""Merge every readable old log into its encrypted twin; return writes."""
dest = EncryptingStore(store, key, layout)
copied = 0
for rel, text in _blobs(store, old_prefix).items():
try:
source = load_log(text)
except ValueError, KeyError, TypeError:
_logger.warning("Unreadable old log %s/%s not copied", old_prefix, rel)
continue
if not source:
continue
path = f"{layout.prefix}/{rel}"
existing = dest.get_file_text(path)
try:
current = load_log(existing) if existing is not None else {}
except ValueError, KeyError, TypeError:
# Not a sync log (an app's own blob): never overwrite it.
_logger.warning("Non-log blob at %s left as is", path)
continue
merged = merge_logs(current, source)
if existing is not None and dump_log(merged) == dump_log(current):
continue
dest.put_file_text(path, dump_log(merged), message="e2e copy")
copied += 1
return copied
_copied: dict[str, str] = {}
def reset_e2e_copy_memo() -> None:
"""Forget which old prefixes were copied; for tests."""
_copied.clear()
def _fingerprint(store: RemoteStore, old_prefix: str) -> str:
reader = getattr(store, "get_string_map", None)
if reader is None:
return ""
revs = reader(default_revs_path(old_prefix))
return ";".join(f"{k}={revs[k]}" for k in sorted(revs))
@dataclass(frozen=True)
class _OpenRequest:
old_prefix: str
layout: E2eLayout
passphrase: str
params: Argon2Params | None
state_dir: Path | None
def _unlock(store: RemoteStore, req: _OpenRequest) -> E2eKey:
old_prefix, layout, state_dir = req.old_prefix, req.layout, req.state_dir
if state_dir is not None:
cached = read_cached_key(state_dir, old_prefix)
if cached is not None:
key = E2eKey.from_bytes(cached)
if key_matches(key, store, layout.e2e_path):
return key
if not req.passphrase:
msg = "no crdt-sync passphrase on this device"
raise E2eLockedError(msg)
try:
key = unlock_e2e(
store,
path_prefix=layout.prefix,
passphrase=req.passphrase,
params=req.params,
e2e_path=layout.e2e_path,
)
except (E2eWrongPassphraseError, E2eHeaderError) as exc:
raise E2eLockedError(str(exc) or type(exc).__name__) from exc
if state_dir is not None:
write_cached_key(state_dir, old_prefix, key.master)
return key
def open_encrypted_client(
store: RemoteStore,
old_prefix: str,
*,
passphrase: str,
params: Argon2Params | None = None,
state_dir: Path | None = None,
) -> tuple[EncryptingStore, E2eLayout]:
"""Unlock, copy the old prefix when it changed, and return the store.
Args:
store: The outermost remote (above any mirror).
old_prefix: The app's former plaintext prefix.
passphrase: The shared crdt-sync password.
params: Argon2id cost for a header created now; tests lower it.
state_dir: Where a short-lived daemon keeps its derived key (0600)
and copy memo between runs; None keeps both in memory only.
Raises:
E2eLockedError: No passphrase, a wrong one, or an unreadable header.
"""
layout = E2eLayout.of(old_prefix)
key = _unlock(
store, _OpenRequest(old_prefix, layout, passphrase, params, state_dir)
)
fingerprint = _fingerprint(store, old_prefix)
last = (
read_copy_memo(state_dir, old_prefix)
if state_dir is not None
else _copied.get(old_prefix)
)
if last != fingerprint:
copy_to_encrypted(store, key, old_prefix, layout)
_copied[old_prefix] = fingerprint
if state_dir is not None:
write_copy_memo(state_dir, old_prefix, fingerprint)
return EncryptingStore(store, key, layout), layout

View File

@ -13,6 +13,7 @@ from dataclasses import dataclass
import logging
from typing import TYPE_CHECKING
from crdt_sync._e2e_cipher import E2eDecryptError
from crdt_sync._revisions import revision_of
if TYPE_CHECKING:
@ -90,9 +91,16 @@ def _pull_remote_logs(
# skipped next tick.
seen_revs[other_device_id] = remote_rev
continue
text = ctx.client.get_file_text(
f"{ctx.path_prefix}/{other_device_id}/{ctx.filename}"
)
try:
text = ctx.client.get_file_text(
f"{ctx.path_prefix}/{other_device_id}/{ctx.filename}"
)
except E2eDecryptError:
_logger.warning(
"Undecryptable log from device %r (wrong key or newer app)",
other_device_id,
)
continue
if text is None:
continue
try:

View File

@ -0,0 +1,10 @@
{
"passphrase": "correct horse ✓",
"header": "{\"v\":1,\"kdf\":\"argon2id\",\"salt\":\"HGW+FZJe6hd71He22BrlUA==\",\"m\":64,\"t\":2,\"p\":1,\"check\":\"okKnATkNoZKvjSGfrgQGOjdRw5NORh2lC32zuBq7V6I=\"}",
"master_hex": "cad5afe8ddc6e21d2f39eb196de454db6b6d1f60629619ce8539f03a4bb0b086",
"revision_of_abc": "e7f845e1b0a7ea2b571fd3e3bdfb60641e650d63171bf73b6356c7909d2748c8",
"path": "vec/notes-e2e/phone/log.json",
"log": "{\"n1\":{\"id\":\"n1\",\"fields\":{\"title\":[\"buy milk\",\"1970-01-01T00:00:01.000Z-0000-node-a\"]},\"deleted\":false,\"deleted_hlc\":null},\"n2\":{\"id\":\"n2\",\"fields\":{\"title\":[\"zażółć\",\"1970-01-01T00:00:01.000Z-0000-node-a\"]},\"deleted\":false,\"deleted_hlc\":null}}",
"envelope": "{\"v\":1,\"r\":{\"n1\":{\"d\":false,\"dh\":null,\"h\":\"1970-01-01T00:00:01.000Z-0000-node-a\",\"p\":\"AR1zODx4TD67X3TGa0N7EV/V4FcZAaqc41jxZVtskFu3Pqb/pNlqm6db66rQUaMyNLtTPa4HMOJzZomBRyaSHjaPb4oNxgVPkMeIF5IbuI/GA30gR0sz/MSvYmfZ5wSnlBo1+fpH\"},\"n2\":{\"d\":false,\"dh\":null,\"h\":\"1970-01-01T00:00:01.000Z-0000-node-a\",\"p\":\"Aa0omlBGnmlsItzq7A21G0A1tqT8xMOJE0wgxcxJ1c3hVpRmNt4kq2IPNNGuW8v4A7fvs2FrnGX4A+NVtXPpu41ckKSShTqk8Ml6zTuxyrT85g50sgzbM7PKFczeEaS+ITWDj0N04u0=\"}}}",
"whole": "{\"v\":1,\"b\":\"AYdAwPehG84oHBRhRr5dP5VmleBGzQC1ryV5QHTCqBZE9ZsOZMMkJiP40+KzpnfIc2g=\"}"
}

View File

@ -0,0 +1,146 @@
# Copyright (c) 2026 Krzysztof Rudnicki
"""E2eCipher blob errors and the per-record envelope's edge shapes."""
from __future__ import annotations
import base64
import json
import pytest
from crdt_sync import (
E2eCipher,
E2eDecryptError,
is_e2e_envelope,
open_log_text,
seal_log_text,
)
PATH = "app/devices-e2e/pc/log.json"
CLOCK = "2026-01-01T00:00:00.000Z-0000-pc"
def _cipher(*, accept_plaintext: bool = True) -> E2eCipher:
return E2eCipher(bytes(range(32)), accept_plaintext=accept_plaintext)
def _blob(raw: bytes) -> str:
return base64.b64encode(raw).decode()
def _log(**records: object) -> str:
return json.dumps(records)
def _record(rid: str = "a", **extra: object) -> dict[str, object]:
return {
"id": rid,
"fields": {"title": ["x", CLOCK]},
"deleted": False,
"deleted_hlc": None,
**extra,
}
def test_key_must_be_32_bytes() -> None:
with pytest.raises(ValueError, match="32 bytes"):
E2eCipher(b"short")
def test_blob_round_trips() -> None:
cipher = _cipher()
assert cipher.open(cipher.seal("héllo", path=PATH), path=PATH) == "héllo"
def test_plaintext_is_read_only_while_accepted() -> None:
assert _cipher().open('{"a": 1}', path=PATH) == '{"a": 1}'
with pytest.raises(E2eDecryptError, match="after migration"):
_cipher(accept_plaintext=False).open("[1]", path=PATH)
@pytest.mark.parametrize(
("stored", "message"),
[
("!!not base64!!", "neither JSON nor base64"),
("", "empty blob"),
(_blob(b"\x02" + bytes(60)), "uses format 2"),
(_blob(b"\x01" + bytes(10)), "truncated"),
(_blob(b"\x01" + bytes(60)), "failed authentication"),
],
)
def test_bad_blobs_are_refused(stored: str, message: str) -> None:
with pytest.raises(E2eDecryptError, match=message):
_cipher().open(stored, path=PATH)
def test_blob_is_bound_to_its_path() -> None:
sealed = _cipher().seal("secret", path=PATH)
with pytest.raises(E2eDecryptError, match="failed authentication"):
_cipher().open(sealed, path="other")
@pytest.mark.parametrize("text", ["not json", "[1]", '{"v": true}', '{"v": "1"}', "{}"])
def test_non_envelopes_are_detected(text: str) -> None:
assert not is_e2e_envelope(text)
def test_envelope_is_detected() -> None:
assert is_e2e_envelope('{"v": 1}')
@pytest.mark.parametrize(
"text",
[
"not json",
"[1]",
_log(a=[1]),
_log(a=_record("b")),
_log(a={"id": "a", "fields": []}),
],
)
def test_non_log_text_is_sealed_whole(text: str) -> None:
sealed = seal_log_text(text, _cipher(), path=PATH)
assert "b" in json.loads(sealed)
assert open_log_text(sealed, _cipher(), path=PATH) == text
def test_record_without_fields_has_no_newest_clock() -> None:
empty = {"id": "a", "fields": {}, "deleted": True, "deleted_hlc": CLOCK}
sealed = seal_log_text(_log(a=empty), _cipher(), path=PATH)
assert json.loads(sealed)["r"]["a"]["h"] is None
opened = json.loads(open_log_text(sealed, _cipher(), path=PATH))
assert opened == {"a": empty}
def test_plaintext_log_passes_or_fails_by_policy() -> None:
text = _log(a=_record())
assert open_log_text(text, _cipher(), path=PATH) == text
with pytest.raises(E2eDecryptError, match="after migration"):
open_log_text(text, _cipher(accept_plaintext=False), path=PATH)
def test_unknown_envelope_version_is_refused() -> None:
with pytest.raises(E2eDecryptError, match="uses format 2"):
open_log_text('{"v": 2, "r": {}}', _cipher(), path=PATH)
def test_envelope_without_records_is_misshaped() -> None:
with pytest.raises(E2eDecryptError, match="mis-shaped"):
open_log_text('{"v": 1, "b": 5, "r": []}', _cipher(), path=PATH)
@pytest.mark.parametrize(
"record",
[[], {"p": "x", "d": 1}, {"d": False}, {"p": 3, "d": False}],
)
def test_misshaped_record_is_refused(record: object) -> None:
text = json.dumps({"v": 1, "r": {"a": record}})
with pytest.raises(E2eDecryptError, match="record a"):
open_log_text(text, _cipher(), path=PATH)
def test_tampered_record_metadata_fails_authentication() -> None:
sealed = json.loads(seal_log_text(_log(a=_record()), _cipher(), path=PATH))
sealed["r"]["a"]["d"] = True
with pytest.raises(E2eDecryptError, match="failed authentication"):
open_log_text(json.dumps(sealed), _cipher(), path=PATH)

View File

@ -0,0 +1,74 @@
# Copyright (c) 2026 Krzysztof Rudnicki
"""Cross-language golden vectors: Python opens what Dart sealed."""
from __future__ import annotations
import json
from pathlib import Path
import pytest
from crdt_sync import (
E2eDecryptError,
E2eKey,
E2eWrongPassphraseError,
open_log_text,
seal_log_text,
unlock_e2e,
)
from crdt_sync.tests.conftest import FakeStore as MemoryRemote
WRONG = "nope"
VECTORS = json.loads(
(Path(__file__).parent / "fixtures" / "e2e_dart_vectors.json").read_text(
encoding="utf-8"
)
)
def _key() -> E2eKey:
remote = MemoryRemote({"vec/e2e": VECTORS["header"]})
return unlock_e2e(
remote,
path_prefix="vec/notes-e2e",
passphrase=str(VECTORS["passphrase"]),
e2e_path="vec/e2e",
)
def test_argon2id_and_hkdf_match_dart() -> None:
assert _key().master.hex() == VECTORS["master_hex"]
def test_keyed_revision_matches_dart() -> None:
assert _key().revision("abc") == VECTORS["revision_of_abc"]
def test_opens_a_dart_envelope() -> None:
opened = open_log_text(VECTORS["envelope"], _key().cipher(), path=VECTORS["path"])
assert json.loads(opened) == json.loads(VECTORS["log"])
def test_opens_a_dart_whole_blob() -> None:
opened = open_log_text(VECTORS["whole"], _key().cipher(), path=VECTORS["path"])
assert opened == "not a log"
def test_wrong_passphrase_is_refused() -> None:
remote = MemoryRemote({"vec/e2e": VECTORS["header"]})
with pytest.raises(E2eWrongPassphraseError):
unlock_e2e(remote, path_prefix="x", passphrase=WRONG, e2e_path="vec/e2e")
def test_a_moved_envelope_is_refused() -> None:
with pytest.raises(E2eDecryptError):
open_log_text(VECTORS["envelope"], _key().cipher(), path="vec/other")
def test_python_round_trip() -> None:
cipher = _key().cipher()
sealed = seal_log_text(VECTORS["log"], cipher, path=VECTORS["path"])
assert "buy milk" not in sealed
assert json.loads(open_log_text(sealed, cipher, path=VECTORS["path"])) == (
json.loads(VECTORS["log"])
)

View File

@ -0,0 +1,123 @@
# Copyright (c) 2026 Krzysztof Rudnicki
"""Argon2id key, header parsing and unlock edge cases."""
from __future__ import annotations
import json
import pytest
from crdt_sync import (
Argon2Params,
E2eHeaderError,
E2eKey,
E2eWrongPassphraseError,
default_e2e_path,
unlock_e2e,
)
from crdt_sync._e2e_key import key_matches, read_header
from crdt_sync.tests.conftest import FakeStore
CHEAP = Argon2Params(8, 1, 1)
HEADER = {
"v": 1,
"kdf": "argon2id",
"salt": "AAAAAAAAAAAAAAAAAAAAAA==",
"m": 8,
"t": 1,
"p": 1,
"check": "c",
}
def _unlock(remote: FakeStore, phrase: str = "pw") -> E2eKey:
return unlock_e2e(
remote,
path_prefix="app/devices-e2e",
passphrase=phrase,
params=CHEAP,
e2e_path="app/e2e",
)
class RacedStore(FakeStore):
"""Another device's header wins the first write."""
def __init__(self, winner: str) -> None:
super().__init__()
self._winner = winner
def put_file_text(self, path: str, text: str, *, message: str) -> None:
super().put_file_text(path, self._winner, message=message)
def test_default_path_is_a_sibling_or_nested() -> None:
assert default_e2e_path("app/devices-e2e") == "app/e2e"
assert default_e2e_path("flat") == "flat/e2e"
def test_master_key_must_be_32_bytes() -> None:
with pytest.raises(ValueError, match="32 bytes"):
E2eKey.from_bytes(b"short")
def test_unlock_refuses_an_empty_passphrase() -> None:
with pytest.raises(ValueError, match="empty"):
_unlock(FakeStore(), "")
def test_unlock_creates_then_reuses_the_header() -> None:
remote = FakeStore()
first = _unlock(remote)
assert _unlock(remote).master == first.master
assert key_matches(first, remote, "app/e2e")
assert not key_matches(_unlock(FakeStore(), "other"), remote, "app/e2e")
def test_missing_header_matches_nothing() -> None:
key = E2eKey.from_bytes(bytes(32))
assert read_header(FakeStore(), "app/e2e") is None
assert not key_matches(key, FakeStore(), "app/e2e")
def test_wrong_passphrase_is_refused() -> None:
remote = FakeStore()
_unlock(remote)
with pytest.raises(E2eWrongPassphraseError):
_unlock(remote, "nope")
def test_losing_the_header_race_adopts_the_winner() -> None:
winner = FakeStore()
key = _unlock(winner)
raced = RacedStore(winner.files["app/e2e"])
assert _unlock(raced).master == key.master
with pytest.raises(E2eWrongPassphraseError):
_unlock(RacedStore(winner.files["app/e2e"]), "nope")
@pytest.mark.parametrize(
("text", "message"),
[
("not json", "not JSON"),
("[1]", "known version"),
(json.dumps({**HEADER, "v": 2}), "known version"),
(json.dumps({**HEADER, "kdf": "scrypt"}), "malformed"),
(json.dumps({k: v for k, v in HEADER.items() if k != "m"}), "malformed"),
(json.dumps({**HEADER, "m": "x"}), "malformed"),
(json.dumps({**HEADER, "salt": "A"}), "malformed"),
],
)
def test_bad_headers_are_refused(text: str, message: str) -> None:
remote = FakeStore({"app/e2e": text})
with pytest.raises(E2eHeaderError, match=message):
read_header(remote, "app/e2e")
with pytest.raises(E2eHeaderError, match=message):
_unlock(remote)
def test_a_good_header_parses() -> None:
header = read_header(FakeStore({"app/e2e": json.dumps(HEADER)}), "app/e2e")
assert header is not None
assert header.params == CHEAP
assert header.check == "c"

View File

@ -0,0 +1,101 @@
# Copyright (c) 2026 Krzysztof Rudnicki
"""A per-minute daemon must not pay Argon2id or a full copy every tick."""
from __future__ import annotations
import json
from typing import TYPE_CHECKING
import pytest
from crdt_sync import Argon2Params, E2eLockedError, open_encrypted_client
from crdt_sync import _e2e_key as key_mod
from crdt_sync import _e2e_store as store_mod
from crdt_sync.tests.conftest import FakeStore
if TYPE_CHECKING:
from pathlib import Path
from crdt_sync import E2eKey, RemoteStore
CHEAP = Argon2Params(8, 1, 1)
PHRASE = "pw"
WRONG = PHRASE + "x"
LOG = json.dumps(
{
"a": {
"id": "a",
"fields": {"t": ["x", "2026-01-01T00:00:00.000Z-0000-a"]},
"deleted": False,
"deleted_hlc": None,
}
}
)
def test_state_dir_caches_key_and_copy_across_processes(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
remote = FakeStore({"app/devices/pc/log.json": LOG, "app/revs/pc": "r1"})
derives: list[int] = []
real = key_mod.unlock_e2e
def counting(
remote: RemoteStore,
*,
path_prefix: str,
passphrase: str,
params: Argon2Params | None = None,
e2e_path: str | None = None,
) -> E2eKey:
derives.append(1)
return real(
remote,
path_prefix=path_prefix,
passphrase=passphrase,
params=params,
e2e_path=e2e_path,
)
monkeypatch.setattr(store_mod, "unlock_e2e", counting)
for _ in range(3):
store_mod.reset_e2e_copy_memo() # a fresh process each tick
remote.writes.clear()
open_encrypted_client(
remote, "app/devices", passphrase=PHRASE, params=CHEAP, state_dir=tmp_path
)
assert derives == [1]
assert remote.writes == []
key_files = list(tmp_path.glob("e2e-*.key"))
assert len(key_files) == 1
assert key_files[0].stat().st_mode & 0o777 == 0o600
def test_stale_cached_key_rederives(tmp_path: Path) -> None:
remote = FakeStore()
open_encrypted_client(
remote, "app/devices", passphrase=PHRASE, params=CHEAP, state_dir=tmp_path
)
for key_file in tmp_path.glob("e2e-*.key"):
key_file.write_text("00" * 32, encoding="utf-8")
store, _ = open_encrypted_client(
remote, "app/devices", passphrase=PHRASE, params=CHEAP, state_dir=tmp_path
)
assert next(tmp_path.glob("e2e-*.key")).read_text(encoding="utf-8") == (
store.key.master.hex()
)
def test_garbage_cached_key_is_ignored_and_wrong_passphrase_locks(
tmp_path: Path,
) -> None:
remote = FakeStore()
open_encrypted_client(
remote, "app/devices", passphrase=PHRASE, params=CHEAP, state_dir=tmp_path
)
for key_file in tmp_path.glob("e2e-*.key"):
key_file.write_text("not hex", encoding="utf-8")
with pytest.raises(E2eLockedError):
open_encrypted_client(
remote, "app/devices", passphrase=WRONG, params=CHEAP, state_dir=tmp_path
)

View File

@ -0,0 +1,118 @@
# Copyright (c) 2026 Krzysztof Rudnicki
"""EncryptingStore, layout, one-time copy and open, mirroring the Dart side."""
from __future__ import annotations
import json
from typing import TYPE_CHECKING
import pytest
from crdt_sync import (
Argon2Params,
E2eLayout,
E2eLockedError,
EncryptingStore,
LogCodec,
SyncTarget,
copy_to_encrypted,
dump_log,
load_log,
open_encrypted_client,
reset_e2e_copy_memo,
sync_log,
unlock_e2e,
)
from crdt_sync.tests.conftest import FakeStore
if TYPE_CHECKING:
from collections.abc import Callable
from crdt_sync import Hlc
PHRASE = "pw"
WRONG = "bad"
CHEAP = Argon2Params(8, 1, 1)
OLD = "app/devices"
def _log_text(make_hlc: Callable[..., Hlc], **titles: str) -> str:
return json.dumps(
{
rid: {
"id": rid,
"fields": {"title": [t, make_hlc(1000).to_str()]},
"deleted": False,
"deleted_hlc": None,
}
for rid, t in titles.items()
}
)
@pytest.fixture(autouse=True)
def _memo() -> None:
reset_e2e_copy_memo()
def test_layouts() -> None:
assert E2eLayout.of("app/devices") == E2eLayout(
"app/devices-e2e", "app/revs-e2e", "app/e2e"
)
assert E2eLayout.of("settings-sync") == E2eLayout(
"settings-sync-e2e", "settings-sync-e2e/revs", "settings-sync-e2e/e2e"
)
def test_sync_through_store_is_ciphertext_and_merges(
make_hlc: Callable[..., Hlc],
) -> None:
remote = FakeStore()
layout = E2eLayout.of(OLD)
key = unlock_e2e(
remote,
path_prefix=layout.prefix,
passphrase=PHRASE,
params=CHEAP,
e2e_path=layout.e2e_path,
)
store = EncryptingStore(remote, key, layout)
codec = LogCodec(encode=dump_log, decode=load_log, filename="log.json")
phone = load_log(_log_text(make_hlc, n1="buy milk"))
sync_log(SyncTarget(store, "phone", layout.prefix), phone, codec)
blob = remote.files[f"{layout.prefix}/phone/log.json"]
assert "buy milk" not in blob
assert '"title"' not in blob
merged = sync_log(SyncTarget(store, "pc", layout.prefix), {}, codec)
assert set(merged) == {"n1"}
def test_open_copies_old_prefix_once_and_locks_without_password(
make_hlc: Callable[..., Hlc],
) -> None:
remote = FakeStore({f"{OLD}/pc/log.json": _log_text(make_hlc, a="x")})
with pytest.raises(E2eLockedError):
open_encrypted_client(remote, OLD, passphrase="", params=CHEAP)
store, layout = open_encrypted_client(remote, OLD, passphrase=PHRASE, params=CHEAP)
text = store.get_file_text(f"{layout.prefix}/pc/log.json")
assert text is not None
assert set(load_log(text)) == {"a"}
assert "x" in remote.files[f"{OLD}/pc/log.json"]
with pytest.raises(E2eLockedError):
open_encrypted_client(remote, OLD, passphrase=WRONG, params=CHEAP)
def test_copy_is_idempotent(make_hlc: Callable[..., Hlc]) -> None:
remote = FakeStore({f"{OLD}/pc/log.json": _log_text(make_hlc, a="x")})
layout = E2eLayout.of(OLD)
key = unlock_e2e(
remote,
path_prefix=layout.prefix,
passphrase=PHRASE,
params=CHEAP,
e2e_path=layout.e2e_path,
)
assert copy_to_encrypted(remote, key, OLD, layout) == 1
remote.writes.clear()
assert copy_to_encrypted(remote, key, OLD, layout) == 0
assert remote.writes == []

View File

@ -0,0 +1,217 @@
# Copyright (c) 2026 Krzysztof Rudnicki
"""EncryptingStore passthroughs, bulk reads and copy/open edge cases."""
from __future__ import annotations
import json
import logging
from typing import TYPE_CHECKING
import pytest
from crdt_sync import (
Argon2Params,
E2eLayout,
E2eLockedError,
EncryptingStore,
copy_to_encrypted,
load_log,
open_encrypted_client,
reset_e2e_copy_memo,
unlock_e2e,
)
from crdt_sync._e2e_store import _blobs, _fingerprint
from crdt_sync.tests.conftest import FakeStore, FakeStoreWithoutBulkRead
if TYPE_CHECKING:
from collections.abc import Callable
from crdt_sync import E2eKey, Hlc
PHRASE = "pw"
CHEAP = Argon2Params(8, 1, 1)
OLD = "app/devices"
LAYOUT = E2eLayout.of(OLD)
CLOCK = "2026-01-01T00:00:00.000Z-0000-pc"
def _log(title: str = "x", rid: str = "a") -> str:
return json.dumps(
{
rid: {
"id": rid,
"fields": {"title": [title, CLOCK]},
"deleted": False,
"deleted_hlc": None,
}
}
)
def _key(remote: FakeStore) -> E2eKey:
return unlock_e2e(
remote,
path_prefix=LAYOUT.prefix,
passphrase=PHRASE,
params=CHEAP,
e2e_path=LAYOUT.e2e_path,
)
class HidingStore(FakeStore):
"""Lists every file but reads ``hidden`` paths as absent."""
def __init__(self, files: dict[str, str], hidden: set[str]) -> None:
super().__init__(files)
self._hidden = hidden
def get_file_text(self, path: str) -> str | None:
if path in self._hidden:
return None
return super().get_file_text(path)
@pytest.fixture(autouse=True)
def _memo() -> None:
reset_e2e_copy_memo()
def test_passthroughs_and_keyed_revisions() -> None:
remote = FakeStore({"other/file": "raw"})
store = EncryptingStore(remote, _key(remote), LAYOUT)
assert store.list_directory("other") == ["file"]
assert store.get_file_text("other/file") == "raw"
assert store.get_file_text(f"{LAYOUT.prefix}/pc/log.json") is None
assert store.can_access_remote()
store.put_file_text(f"{LAYOUT.revs_path}/pc", "text", message="m")
assert remote.files[f"{LAYOUT.revs_path}/pc"] == store.key.revision("text")
store.put_file_text("other/file", "plain", message="m")
assert remote.files["other/file"] == "plain"
store.delete_file("other/file")
assert "other/file" not in remote.files
def test_bulk_read_opens_sealed_and_drops_bad(
caplog: pytest.LogCaptureFixture,
) -> None:
remote = FakeStore()
store = EncryptingStore(remote, _key(remote), LAYOUT)
store.put_file_text(f"{LAYOUT.prefix}/pc", _log(), message="m")
remote.files[f"{LAYOUT.prefix}/bad"] = '{"v": 2, "r": {}}'
remote.files[f"{LAYOUT.revs_path}/pc"] = "rev"
with caplog.at_level(logging.WARNING):
out = store.get_string_map(LAYOUT.prefix)
assert set(out) == {"pc"}
assert set(load_log(out["pc"])) == {"a"}
assert "Undecryptable entry" in caplog.text
assert store.get_string_map(LAYOUT.revs_path) == {"pc": "rev"}
def test_bulk_read_is_empty_without_inner_support() -> None:
remote = FakeStoreWithoutBulkRead()
store = EncryptingStore(remote, _key(FakeStore()), LAYOUT)
assert store.get_string_map(LAYOUT.prefix) == {}
assert _fingerprint(remote, OLD) == ""
def test_blobs_cover_flat_nested_and_skipped_paths() -> None:
remote = HidingStore(
{
f"{OLD}/flat": "f",
f"{OLD}/gone": "g",
f"{OLD}/pc/log.json": "n",
f"{OLD}/pc/hidden.json": "h",
"app/revs/pc": "r",
f"{OLD}/e2e": "h",
f"{OLD}/revs": "r",
},
{f"{OLD}/gone", f"{OLD}/pc/hidden.json"},
)
assert _blobs(remote, OLD) == {"flat": "f", "pc/log.json": "n"}
def test_copy_skips_unreadable_and_flat_logs(
caplog: pytest.LogCaptureFixture,
) -> None:
remote = FakeStore({f"{OLD}/flat": _log(), f"{OLD}/pc/log.json": "not json"})
with caplog.at_level(logging.WARNING):
copied = copy_to_encrypted(remote, _key(remote), OLD, LAYOUT)
assert copied == 1
assert "Unreadable old log" in caplog.text
assert f"{LAYOUT.prefix}/flat" in remote.files
assert f"{LAYOUT.prefix}/pc/log.json" not in remote.files
def test_changed_old_prefix_is_copied_again() -> None:
remote = FakeStore({f"{OLD}/pc/log.json": _log("one")})
open_encrypted_client(remote, OLD, passphrase=PHRASE, params=CHEAP)
remote.files[f"{OLD}/pc/log.json"] = _log("two", rid="b")
remote.files["app/revs/pc"] = "newer"
store, layout = open_encrypted_client(remote, OLD, passphrase=PHRASE, params=CHEAP)
text = store.get_file_text(f"{layout.prefix}/pc/log.json")
assert text is not None
assert set(load_log(text)) == {"a", "b"}
def test_unchanged_prefix_is_not_copied_twice() -> None:
remote = FakeStore({f"{OLD}/pc/log.json": _log()})
open_encrypted_client(remote, OLD, passphrase=PHRASE, params=CHEAP)
remote.writes.clear()
open_encrypted_client(remote, OLD, passphrase=PHRASE, params=CHEAP)
assert remote.writes == []
def test_unreadable_header_locks_the_client() -> None:
remote = FakeStore({LAYOUT.e2e_path: "not json"})
with pytest.raises(E2eLockedError, match="not JSON"):
open_encrypted_client(remote, OLD, passphrase=PHRASE, params=CHEAP)
def test_copy_never_overwrites_non_log_and_skips_empty() -> None:
layout = E2eLayout.of("app/devices")
remote = FakeStore(
{
"app/devices/pc/current.json": "{}",
"app/devices-e2e/pc/current.json": '{"photos":["a"]}',
"app/devices/tab/current.json": "{}",
}
)
key = unlock_e2e(
remote,
path_prefix=layout.prefix,
passphrase="x" * 8,
params=Argon2Params(8, 1, 1),
e2e_path=layout.e2e_path,
)
assert copy_to_encrypted(remote, key, "app/devices", layout) == 0
assert remote.files["app/devices-e2e/pc/current.json"] == '{"photos":["a"]}'
assert "app/devices-e2e/tab/current.json" not in remote.files
def test_copy_leaves_a_non_log_destination_alone(
make_hlc: Callable[..., Hlc],
) -> None:
layout = E2eLayout.of("app/devices")
log = {
"n": {
"id": "n",
"fields": {"t": ["x", make_hlc(1).to_str()]},
"deleted": False,
"deleted_hlc": None,
}
}
remote = FakeStore(
{
"app/devices/pc/current.json": json.dumps(log),
"app/devices-e2e/pc/current.json": '{"photos":["a"]}',
}
)
key = unlock_e2e(
remote,
path_prefix=layout.prefix,
passphrase="x" * 8,
params=Argon2Params(8, 1, 1),
e2e_path=layout.e2e_path,
)
assert copy_to_encrypted(remote, key, "app/devices", layout) == 0
assert remote.files["app/devices-e2e/pc/current.json"] == '{"photos":["a"]}'

View File

@ -0,0 +1,32 @@
# Copyright (c) 2026 Krzysztof Rudnicki
"""A peer whose log cannot be decrypted is skipped, not fatal."""
from __future__ import annotations
import logging
from typing import TYPE_CHECKING
from unittest.mock import MagicMock
from crdt_sync import (
E2eDecryptError,
GitHubSyncClient,
LogCodec,
SyncTarget,
dump_log,
load_log,
sync_log,
)
if TYPE_CHECKING:
import pytest
def test_undecryptable_peer_is_skipped(caplog: pytest.LogCaptureFixture) -> None:
client = MagicMock(spec=GitHubSyncClient)
client.list_directory.return_value = ["phone"]
client.get_file_text.side_effect = E2eDecryptError("wrong key")
codec = LogCodec(encode=dump_log, decode=load_log, filename="log.json")
with caplog.at_level(logging.WARNING):
merged = sync_log(SyncTarget(client, "pc", "devices"), {}, codec)
assert merged == {}
assert "Undecryptable log from device 'phone'" in caplog.text

View File

@ -0,0 +1,62 @@
# Copyright (c) 2026 Krzysztof Rudnicki
"""The RTDB plaintext audit behind the E2E done check."""
from __future__ import annotations
import io
import json
from typing import TYPE_CHECKING
from crdt_sync import E2eCipher, seal_log_text
from tool import e2e_audit
if TYPE_CHECKING:
import pytest
LOG = json.dumps(
{
"n": {
"id": "n",
"fields": {"title": ["x", "2026-01-01T00:00:00.000Z-0000-a"]},
"deleted": False,
"deleted_hlc": None,
}
}
)
def _sealed() -> str:
return seal_log_text(LOG, E2eCipher(bytes(32)), path="a/devices-e2e/p/log.json")
def test_sealed_tree_is_clean() -> None:
tree = {
"devices-e2e": {"p": {"log~2Ejson": _sealed()}},
"revs-e2e": {"p": "ab" * 32},
"e2e": '{"v":1,"kdf":"argon2id"}',
}
assert e2e_audit.plaintext_paths(tree) == []
def test_plaintext_leaf_is_reported_with_decoded_key() -> None:
tree = {"devices": {"p": {"log~2Ejson": LOG}}, "settings": {"k": '{"a":1}'}}
assert e2e_audit.plaintext_paths(tree) == [
"devices/p/log.json",
"settings/k",
]
def test_raw_json_scalars_count_as_plaintext() -> None:
assert e2e_audit.plaintext_paths({"a": 5, "b": True, "c": None}) == ["a", "b"]
def test_main_reads_stdin_and_sets_exit_code(
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
monkeypatch.setattr("sys.stdin", io.StringIO(json.dumps({"x": LOG})))
assert e2e_audit.main(["todo-sync"]) == 1
assert "todo-sync/x" in capsys.readouterr().out
monkeypatch.setattr("sys.stdin", io.StringIO("null"))
assert e2e_audit.main(["todo-sync"]) == 0
monkeypatch.setattr("sys.stdin", io.StringIO(json.dumps({"x": _sealed()})))
assert e2e_audit.main([]) == 0

View File

@ -0,0 +1,41 @@
# Copyright (c) 2026 Krzysztof Rudnicki
"""tool/e2e_vectors writes vectors that Python itself can open."""
from __future__ import annotations
import json
from pathlib import Path
import runpy
import pytest
from crdt_sync import open_log_text, unlock_e2e
from crdt_sync.tests.conftest import FakeStore
from tool import e2e_vectors
def test_vectors_open_with_the_stated_passphrase() -> None:
vec = e2e_vectors.vectors()
key = unlock_e2e(
FakeStore({"vec/e2e": str(vec["header"])}),
path_prefix="vec/notes-e2e",
passphrase=str(vec["passphrase"]),
e2e_path="vec/e2e",
)
assert key.master.hex() == vec["master_hex"]
path = str(vec["path"])
opened = open_log_text(str(vec["envelope"]), key.cipher(), path=path)
assert json.loads(opened) == json.loads(str(vec["log"]))
assert open_log_text(str(vec["whole"]), key.cipher(), path=path) == "not a log"
def test_main_prints_json(capsys: pytest.CaptureFixture[str]) -> None:
assert e2e_vectors.main() == 0
assert "master_hex" in json.loads(capsys.readouterr().out)
def test_module_entry_point(capsys: pytest.CaptureFixture[str]) -> None:
with pytest.raises(SystemExit) as exc:
runpy.run_path(str(Path(e2e_vectors.__file__)), run_name="__main__")
assert exc.value.code == 0
assert "envelope" in capsys.readouterr().out

View File

@ -1,9 +1,13 @@
[project]
name = "crdt-sync"
version = "0.9.2"
version = "0.11.0"
description = "Shared CRDT merge scheme + GitHub-Contents-API sync transport for cross-device apps"
requires-python = ">=3.14"
dependencies = ["requests==2.34.2"]
dependencies = [
"argon2-cffi==25.1.0",
"pynacl==1.6.2",
"requests==2.34.2",
]
# Ship only the library. `tool/` is a package too (it has an __init__.py so
# its scripts import cleanly), and setuptools' flat-layout auto-discovery

View File

@ -1,12 +1,14 @@
# Exact pins, enforced by the shared dependency-freshness gate: a `>=`
# floor is satisfied by anything newer, so it tells a checker nothing
# about whether this repo is actually current.
argon2-cffi==25.1.0
bandit[toml]==1.9.4
mypy==2.4.0
mypy==2.4.1
pre-commit==4.7.0
pylint==4.1.3
pytest==9.1.1
pynacl==1.6.2
pytest==9.2.0
pytest-cov==7.1.0
requests==2.34.2
ruff==0.17.2
types-requests==2.33.0.20261006
ruff==0.17.4
types-requests==2.33.0.20261030

View File

@ -0,0 +1,63 @@
# Copyright (c) 2026 Krzysztof Rudnicki
"""Read-only E2E done check: list plaintext leaves in an RTDB JSON dump.
Feed it ``firebase database:get /<root>`` on stdin::
firebase database:get /todo-sync | python3 -m tool.e2e_audit todo-sync
Exit 0 when every leaf is sealed (or is the clear Argon2id header or a keyed
revision), 1 when any leaf is plaintext. A plain ``grep '"title"'`` misses
plaintext because RTDB serves each blob as an escaped JSON string; this
decodes the tree instead.
"""
from __future__ import annotations
import json
import re
import sys
from crdt_sync import is_e2e_envelope
_HEX64 = re.compile(r"[0-9a-f]{64}")
def _decode_key(key: str) -> str:
return re.sub(r"~([0-9A-F]{2})", lambda m: chr(int(m.group(1), 16)), key)
def _is_clear_leaf(value: str) -> bool:
# Envelopes and the Argon2id header both carry an integer "v"; keyed
# revisions are 64 hex chars; a bare E2eCipher blob is base64, not JSON.
if is_e2e_envelope(value) or _HEX64.fullmatch(value):
return True
return not value.lstrip().startswith(("{", "["))
def plaintext_paths(tree: object, prefix: str = "") -> list[str]:
"""Return the decoded paths of every plaintext leaf under ``tree``."""
found: list[str] = []
if isinstance(tree, dict):
for key, value in tree.items():
path = f"{prefix}/{_decode_key(key)}" if prefix else _decode_key(key)
found.extend(plaintext_paths(value, path))
elif isinstance(tree, str):
if not _is_clear_leaf(tree):
found.append(prefix)
elif tree is not None:
found.append(prefix)
return found
def main(argv: list[str]) -> int:
"""Audit the dump on stdin; ``argv[0]`` names the root for the report."""
root = argv[0] if argv else ""
paths = plaintext_paths(json.load(sys.stdin), root)
sys.stdout.write(f"plaintext leaves under /{root}: {len(paths)}\n")
for path in paths:
sys.stdout.write(f" {path}\n")
return 1 if paths else 0
if __name__ == "__main__":
raise SystemExit(main(sys.argv[1:]))

View File

@ -0,0 +1,64 @@
# Copyright (c) 2026 Krzysztof Rudnicki
"""Write golden vectors sealed by Python for the Dart side to open.
Run from ``crdt-sync``::
python -m tool.e2e_vectors > e2e_py_vectors.json
then move the file to ``crdt_sync_dart/test/fixtures/``.
"""
from __future__ import annotations
import json
import sys
from crdt_sync import Argon2Params, seal_log_text, unlock_e2e
from crdt_sync.tests.conftest import FakeStore
# A public test-vector passphrase, not a secret.
PHRASE = "correct horse \u2713"
def vectors() -> dict[str, object]:
"""Return a fresh vector set (random salt and nonces)."""
remote = FakeStore()
key = unlock_e2e(
remote,
path_prefix="vec/notes-e2e",
passphrase=PHRASE,
params=Argon2Params(64, 2, 1),
e2e_path="vec/e2e",
)
path = "vec/notes-e2e/pc/log.json"
log = json.dumps(
{
"n9": {
"id": "n9",
"fields": {"title": ["zażółć", "2026-01-01T00:00:00.000Z-0000-pc"]},
"deleted": False,
"deleted_hlc": None,
}
},
ensure_ascii=False,
)
cipher = key.cipher()
return {
"passphrase": PHRASE,
"header": remote.files["vec/e2e"],
"master_hex": key.master.hex(),
"path": path,
"log": log,
"envelope": seal_log_text(log, cipher, path=path),
"whole": seal_log_text("not a log", cipher, path=path),
}
def main() -> int:
"""Print the vectors as JSON."""
sys.stdout.write(json.dumps(vectors(), indent=2, ensure_ascii=False) + "\n")
return 0
if __name__ == "__main__":
raise SystemExit(main())

View File

@ -2,6 +2,13 @@
library;
export 'src/device.dart';
export 'src/e2e_audit.dart';
export 'src/e2e_cipher.dart';
export 'src/e2e_envelope.dart';
export 'src/e2e_key.dart';
export 'src/e2e_migration.dart';
export 'src/e2e_open.dart';
export 'src/encrypting_store.dart';
export 'src/firebase_account_provisioning.dart';
export 'src/firebase_auth_error.dart';
export 'src/firebase_auth_rest.dart';

View File

@ -0,0 +1,23 @@
import 'e2e_cipher.dart';
import 'e2e_envelope.dart';
import 'firebase_keys.dart';
/// Plaintext leaves in a decoded RTDB dump; empty once an app is migrated.
/// Decodes because RTDB serves blobs as escaped strings that defeat grep.
List<String> plaintextBlobPaths(Object? tree) {
final found = <String>[];
void walk(Object? node, String path) {
if (node is String) {
if (E2eCipher.isPlaintext(node) && !isE2eEnvelope(node)) found.add(path);
} else if (node is Map) {
for (final MapEntry(:key, :value) in node.entries) {
if (key == 'e2e' && value is String) continue;
final segment = decodeKey(key as String);
walk(value, path.isEmpty ? segment : '$path/$segment');
}
}
}
walk(tree, '');
return found;
}

View File

@ -0,0 +1,97 @@
import 'dart:convert';
import 'package:cryptography/cryptography.dart';
const int e2eFormatVersion = 1;
const _nonceLength = 24;
const _tagLength = 16;
/// A FormatException so `syncLog` skips the peer instead of aborting the tick.
class E2eDecryptError extends FormatException {
E2eDecryptError(super.message);
@override
String toString() => 'E2eDecryptError: $message';
}
/// XChaCha20-Poly1305 per blob: base64 `[version][nonce][ct][tag]`, with the
/// version byte and remote path as AAD so blobs cannot be swapped.
///
/// [acceptPlaintext] reads un-migrated peers; turn it off after migration.
class E2eCipher {
E2eCipher(List<int> key, {this.acceptPlaintext = true})
: _key = SecretKey(List<int>.unmodifiable(key)) {
if (key.length != 32) {
throw ArgumentError.value(key.length, 'key', 'must be 32 bytes');
}
}
final SecretKey _key;
final bool acceptPlaintext;
static final _aead = Xchacha20.poly1305Aead();
static bool isPlaintext(String stored) {
final trimmed = stored.trimLeft();
return trimmed.startsWith('{') || trimmed.startsWith('[');
}
Future<String> seal(String plaintext, {required String path}) async {
final box = await _aead.encrypt(
utf8.encode(plaintext),
secretKey: _key,
nonce: _aead.newNonce(),
aad: _aad(e2eFormatVersion, path),
);
return base64.encode([
e2eFormatVersion,
...box.nonce,
...box.cipherText,
...box.mac.bytes,
]);
}
Future<String> open(String stored, {required String path}) async {
if (isPlaintext(stored)) {
if (acceptPlaintext) return stored;
throw E2eDecryptError('plaintext blob at $path after migration');
}
final List<int> bytes;
try {
bytes = base64.decode(stored);
} on FormatException {
throw E2eDecryptError('blob at $path is neither JSON nor base64');
}
if (bytes.isEmpty) throw E2eDecryptError('empty blob at $path');
final version = bytes.first;
if (version != e2eFormatVersion) {
throw E2eDecryptError(
'blob at $path uses format $version; update the app to read it',
);
}
if (bytes.length < 1 + _nonceLength + _tagLength) {
throw E2eDecryptError('blob at $path is truncated');
}
final box = SecretBox(
bytes.sublist(1 + _nonceLength, bytes.length - _tagLength),
nonce: bytes.sublist(1, 1 + _nonceLength),
mac: Mac(bytes.sublist(bytes.length - _tagLength)),
);
try {
final clear = await _aead.decrypt(
box,
secretKey: _key,
aad: _aad(version, path),
);
return utf8.decode(clear);
} on SecretBoxAuthenticationError {
throw E2eDecryptError('blob at $path failed authentication');
}
}
static List<int> _aad(int version, String path) => [
version,
...utf8.encode(path),
];
}

View File

@ -0,0 +1,123 @@
/// The per-record wire envelope: record ids, the tombstone and the newest HLC
/// stay in clear so the relay can order and inspect; every field name and
/// value is sealed. Merge still runs on the decrypted [Log].
library;
import 'dart:convert';
import 'e2e_cipher.dart';
import 'hlc.dart';
const int _envelopeVersion = 1;
/// Whether [text] is an envelope (explicit `"v"` integer), not legacy plaintext.
///
/// A legacy log maps record ids to objects, so an id named `v` still maps to
/// a map, never an int.
bool isE2eEnvelope(String text) {
try {
final json = jsonDecode(text);
return json is Map && json['v'] is int;
} on FormatException {
return false;
}
}
String _aad(
String path,
String id,
bool deleted,
String? deletedHlc,
String? h,
) => '$path#$id#${deleted ? 1 : 0}#${deletedHlc ?? ''}#${h ?? ''}';
/// Seals a log (the `logToJson` shape) into an envelope. Text that is not
/// log-shaped is sealed whole, so nothing is ever pushed in clear.
Future<String> sealLogText(
String text,
E2eCipher cipher, {
required String path,
}) async {
final Map<String, dynamic> raw;
try {
raw = (jsonDecode(text) as Map).cast<String, dynamic>();
for (final entry in raw.entries) {
final rec = entry.value as Map;
if (rec['id'] != entry.key || rec['fields'] is! Map) {
throw const FormatException('not a log');
}
}
} on Object {
return jsonEncode({
'v': _envelopeVersion,
'b': await cipher.seal(text, path: path),
});
}
final records = <String, Object?>{};
for (final entry in raw.entries) {
final rec = (entry.value as Map).cast<String, dynamic>();
final fields = (rec['fields'] as Map).cast<String, dynamic>();
Hlc? newest;
for (final field in fields.values) {
final hlc = Hlc.fromStr((field as List)[1] as String);
if (newest == null || hlc > newest) newest = hlc;
}
final deleted = rec['deleted'] as bool? ?? false;
final deletedHlc = rec['deleted_hlc'] as String?;
final h = newest?.toStr();
records[entry.key] = {
'd': deleted,
'dh': deletedHlc,
'h': h,
'p': await cipher.seal(
jsonEncode(fields),
path: _aad(path, entry.key, deleted, deletedHlc, h),
),
};
}
return jsonEncode({'v': _envelopeVersion, 'r': records});
}
/// Opens an envelope back to the `logToJson` shape. Legacy plaintext is
/// returned as-is while the cipher accepts it.
Future<String> openLogText(
String text,
E2eCipher cipher, {
required String path,
}) async {
if (!isE2eEnvelope(text)) {
if (cipher.acceptPlaintext) return text;
throw E2eDecryptError('plaintext blob at $path after migration');
}
final json = (jsonDecode(text) as Map).cast<String, dynamic>();
if (json['v'] != _envelopeVersion) {
throw E2eDecryptError(
'blob at $path uses format ${json['v']}; update the app to read it',
);
}
final whole = json['b'];
if (whole is String) return cipher.open(whole, path: path);
final records = json['r'];
if (records is! Map) throw E2eDecryptError('envelope at $path is mis-shaped');
final out = <String, Object?>{};
for (final entry in records.entries) {
final id = entry.key as String;
final rec = entry.value;
if (rec is! Map || rec['p'] is! String || rec['d'] is! bool) {
throw E2eDecryptError('record $id at $path is mis-shaped');
}
final deletedHlc = rec['dh'] as String?;
final h = rec['h'] as String?;
final clear = await cipher.open(
rec['p'] as String,
path: _aad(path, id, rec['d'] as bool, deletedHlc, h),
);
out[id] = {
'id': id,
'fields': jsonDecode(clear),
'deleted': rec['d'],
'deleted_hlc': deletedHlc,
};
}
return jsonEncode(out);
}

View File

@ -0,0 +1,254 @@
import 'dart:convert';
import 'dart:math';
import 'package:cryptography/cryptography.dart';
import 'e2e_cipher.dart';
import 'encrypting_store.dart';
import 'remote_store.dart';
const _saltLength = 16;
const _headerVersion = 1;
/// Argon2id cost, stored in the header so it can be raised later.
class Argon2Params {
const Argon2Params({
required this.memoryKiB,
required this.iterations,
required this.parallelism,
});
/// OWASP's floor (19 MiB, t=2); higher stalls the Flutter-web desktop build.
static const recommended = Argon2Params(
memoryKiB: 19456,
iterations: 2,
parallelism: 1,
);
final int memoryKiB;
final int iterations;
final int parallelism;
}
class E2eHeaderError extends FormatException {
E2eHeaderError(super.message);
@override
String toString() => 'E2eHeaderError: $message';
}
class E2eWrongPassphraseError implements Exception {
@override
String toString() => 'E2eWrongPassphraseError: wrong sync passphrase';
}
/// The clear header's path: a sibling of the device dirs, like `revs`.
String defaultE2ePath(String pathPrefix) {
final cut = pathPrefix.lastIndexOf('/');
return cut < 0 ? '$pathPrefix/e2e' : '${pathPrefix.substring(0, cut)}/e2e';
}
class _Header {
_Header(this.salt, this.params, this.check);
factory _Header.parse(String text) {
final Object? json;
try {
json = jsonDecode(text);
} on FormatException {
throw E2eHeaderError('e2e header is not JSON');
}
if (json is! Map<String, dynamic>) {
throw E2eHeaderError('e2e header is not an object');
}
if (json['v'] != _headerVersion) {
throw E2eHeaderError('e2e header v${json['v']}; update the app');
}
try {
if (json['kdf'] != 'argon2id') throw const FormatException();
return _Header(
base64.decode(json['salt'] as String),
Argon2Params(
memoryKiB: json['m'] as int,
iterations: json['t'] as int,
parallelism: json['p'] as int,
),
json['check'] as String,
);
} on Object {
throw E2eHeaderError('e2e header is malformed');
}
}
final List<int> salt;
final Argon2Params params;
final String check;
String encode() => jsonEncode({
'v': _headerVersion,
'kdf': 'argon2id',
'salt': base64.encode(salt),
'm': params.memoryKiB,
't': params.iterations,
'p': params.parallelism,
'check': check,
});
}
/// A 32-byte master key plus its HKDF subkeys for sealing, revisions and the
/// passphrase check. Cache [bytes] in the OS keystore, never in RTDB.
class E2eKey {
E2eKey._(this.bytes, this._enc, this._rev, this._check);
static Future<E2eKey> fromBytes(List<int> bytes) {
if (bytes.length != 32) {
throw ArgumentError.value(bytes.length, 'bytes', 'must be 32 bytes');
}
return _expand(List<int>.unmodifiable(bytes));
}
static Future<E2eKey> _expand(List<int> master) async {
Future<List<int>> sub(String label) async {
final key = await Hkdf(hmac: Hmac.sha256(), outputLength: 32).deriveKey(
secretKey: SecretKey(master),
info: utf8.encode('crdt_sync e2e $label'),
);
return key.bytes;
}
return E2eKey._(
master,
await sub('enc'),
SecretKey(await sub('rev')),
SecretKey(await sub('check')),
);
}
final List<int> bytes;
final List<int> _enc;
final SecretKey _rev;
final SecretKey _check;
E2eCipher cipher({bool acceptPlaintext = true}) =>
E2eCipher(_enc, acceptPlaintext: acceptPlaintext);
/// Keyed `revisionOf`: a published revision cannot confirm a guessed log.
Future<String> revision(String text) async {
final mac = await Hmac.sha256().calculateMac(
utf8.encode(text),
secretKey: _rev,
);
return mac.bytes.map((b) => b.toRadixString(16).padLeft(2, '0')).join();
}
Future<String> _checkFor(List<int> salt) async {
final mac = await Hmac.sha256().calculateMac(salt, secretKey: _check);
return base64.encode(mac.bytes);
}
}
Future<E2eKey> _derive(
String passphrase,
List<int> salt,
Argon2Params params,
) async {
final key = await Argon2id(
parallelism: params.parallelism,
memory: params.memoryKiB,
iterations: params.iterations,
hashLength: 32,
).deriveKeyFromPassword(password: passphrase, nonce: salt);
return E2eKey._expand(await key.extractBytes());
}
Future<_Header?> _readHeader(RemoteStore store, String path) async {
final text = await store.getFileText(path);
return text == null ? null : _Header.parse(text);
}
/// Derives this app's key from [passphrase], creating the header on first use.
///
/// Throws [E2eWrongPassphraseError] on a failed check and [E2eHeaderError]
/// for an unreadable header, which is never overwritten.
Future<E2eKey> unlockE2e({
required RemoteStore store,
required String pathPrefix,
required String passphrase,
Argon2Params params = Argon2Params.recommended,
Random? random,
String? e2ePath,
}) async {
if (passphrase.isEmpty) {
throw ArgumentError.value('', 'passphrase', 'must not be empty');
}
final path = e2ePath ?? defaultE2ePath(pathPrefix);
var header = await _readHeader(store, path);
if (header == null) {
final rng = random ?? Random.secure();
final salt = List<int>.generate(_saltLength, (_) => rng.nextInt(256));
final key = await _derive(passphrase, salt, params);
final encoded = _Header(salt, params, await key._checkFor(salt)).encode();
await store.putFileText(path, encoded, message: path);
// Two devices setting up at once: the last write wins, so re-read.
final stored = await store.getFileText(path) ?? encoded;
if (stored == encoded) return key;
header = _Header.parse(stored);
}
final key = await _derive(passphrase, header.salt, header.params);
if (await key._checkFor(header.salt) != header.check) {
throw E2eWrongPassphraseError();
}
return key;
}
sealed class E2eState {}
/// No header and no key: this app has never been encrypted.
final class E2eOff extends E2eState {}
enum E2eLockReason { needsPassphrase, wrongKey, badHeader }
/// Encryption is on, but this device must not sync until it has the key.
final class E2eLocked extends E2eState {
E2eLocked(this.reason);
final E2eLockReason reason;
}
final class E2eReady extends E2eState {
E2eReady(this.store);
final EncryptingStore store;
}
Future<E2eState> openE2e({
required RemoteStore store,
required String pathPrefix,
E2eKey? key,
String? e2ePath,
String? revsPath,
bool acceptPlaintext = true,
}) async {
final _Header? header;
try {
header = await _readHeader(store, e2ePath ?? defaultE2ePath(pathPrefix));
} on E2eHeaderError {
return E2eLocked(E2eLockReason.badHeader);
}
if (key == null) {
return header == null ? E2eOff() : E2eLocked(E2eLockReason.needsPassphrase);
}
if (header == null || await key._checkFor(header.salt) != header.check) {
return E2eLocked(E2eLockReason.wrongKey);
}
return E2eReady(
EncryptingStore(
store,
key,
pathPrefix: pathPrefix,
revsPath: revsPath,
e2ePath: e2ePath,
acceptPlaintext: acceptPlaintext,
),
);
}

View File

@ -0,0 +1,217 @@
/// The one-time copy from a plaintext prefix to `<prefix>-e2e`.
///
/// Re-encrypting in place is rejected: an old build still writing the old
/// prefix would be broken mid-migration. Instead the old prefix stays
/// untouched until [deletePlaintextPrefix] verifies the copy.
library;
import 'dart:convert';
import 'e2e_key.dart';
import 'encrypting_store.dart';
import 'log.dart';
import 'record.dart';
import 'remote_store.dart';
import 'store.dart';
import 'sync.dart';
/// Where an app's encrypted data lives, derived from its old plaintext prefix.
class E2eLayout {
const E2eLayout._(this.prefix, this.revsPath, this.e2ePath);
/// `<oldPrefix>-e2e`, with revisions and header apart from the old ones so
/// old builds neither see nor clobber them.
///
/// A slashless prefix (`settings-sync`) keeps revisions and header inside
/// its own node, as `defaultRevsPath` does, never at the database root.
factory E2eLayout.of(String oldPrefix) {
final prefix = '$oldPrefix-e2e';
final cut = oldPrefix.lastIndexOf('/');
if (cut < 0) {
return E2eLayout._(prefix, '$prefix/revs', '$prefix/e2e');
}
final parent = oldPrefix.substring(0, cut);
return E2eLayout._(prefix, '$parent/revs-e2e', defaultE2ePath(prefix));
}
final String prefix;
final String revsPath;
final String e2ePath;
}
class MigrationReport {
const MigrationReport(this.copied, this.skipped);
/// Blobs written (new or changed) this run.
final int copied;
/// Source blobs unreadable as a log, relative path.
final List<String> skipped;
}
/// Every text blob under [root], keyed by path relative to it: flat
/// `<node>.json` leaves and nested `<device>/<file>` blobs. Real RTDB bulk
/// reads return direct string children only, so dirs are walked.
Future<Map<String, String>> _listBlobs(RemoteStore store, String root) async {
final out = <String, String>{};
if (store is BulkMapReader) {
out.addAll(await (store as BulkMapReader).getStringMap(root));
}
final revs = defaultRevsPath(root);
final header = defaultE2ePath(root);
for (final name in await store.listDirectory(root)) {
if (out.containsKey(name)) continue;
final full = '$root/$name';
if (full == header || full == revs) continue;
final children = await _safeList(store, full);
if (children.isEmpty) {
final text = await _safeRead(store, full);
if (text != null) out[name] = text;
continue;
}
for (final child in children) {
final text = await _safeRead(store, '$full/$child');
if (text != null) out['$name/$child'] = text;
}
}
out.removeWhere(
(rel, _) => '$root/$rel' == header || '$root/$rel'.startsWith('$revs/'),
);
return out;
}
Future<List<String>> _safeList(RemoteStore store, String path) async {
try {
return await store.listDirectory(path);
} on Object {
return const [];
}
}
Future<String?> _safeRead(RemoteStore store, String path) async {
try {
return await store.getFileText(path);
} on Object {
return null;
}
}
EncryptingStore _dest(RemoteStore store, E2eKey key, E2eLayout layout) =>
EncryptingStore(
store,
key,
pathPrefix: layout.prefix,
revsPath: layout.revsPath,
e2ePath: layout.e2ePath,
);
/// Order-independent signature of a log, to tell "same records" cheaply.
String _signature(Log log) {
final ids = log.keys.toList()..sort();
return jsonEncode([
for (final id in ids)
[
id,
log[id]!.deleted,
log[id]!.deletedHlc?.toStr(),
[
for (final name in (log[id]!.fields.keys.toList()..sort()))
[
name,
jsonEncode(log[id]!.fields[name]!.$1),
log[id]!.fields[name]!.$2.toStr(),
],
],
],
]);
}
Log? _parse(String text) {
try {
return logFromJson(text);
} on FormatException {
return null;
} on TypeError {
return null;
}
}
/// Copies every readable plaintext log under [fromPrefix] into the encrypted
/// prefix, merging into what is already there. Idempotent: an unchanged
/// destination is not rewritten. Never touches [fromPrefix].
Future<MigrationReport> copyToEncrypted({
required RemoteStore store,
required E2eKey key,
required String fromPrefix,
required E2eLayout layout,
}) async {
final dest = _dest(store, key, layout);
var copied = 0;
final skipped = <String>[];
final blobs = await _listBlobs(store, fromPrefix);
for (final MapEntry(key: rel, value: text) in blobs.entries) {
final source = _parse(text);
if (source == null) {
skipped.add(rel);
continue;
}
if (source.isEmpty) continue;
final path = '${layout.prefix}/$rel';
final existing = await dest.getFileText(path);
final current = existing == null ? null : _parse(existing);
// Not a sync log (an app's own blob): never overwrite it.
if (existing != null && current == null) continue;
final merged = mergeLogs(current ?? <String, Record>{}, source);
if (current != null && _signature(merged) == _signature(current)) continue;
await dest.putFileText(path, logToJson(merged), message: 'e2e copy');
copied++;
}
return MigrationReport(copied, skipped);
}
/// True when every readable plaintext blob is contained in the encrypted copy
/// and none is unreadable.
Future<bool> verifyCopy({
required RemoteStore store,
required E2eKey key,
required String fromPrefix,
required E2eLayout layout,
}) async {
final dest = _dest(store, key, layout);
for (final MapEntry(key: rel, value: text) in (await _listBlobs(
store,
fromPrefix,
)).entries) {
final source = _parse(text);
if (source == null) return false;
final existing = await dest.getFileText('${layout.prefix}/$rel');
final current = existing == null ? null : _parse(existing);
if (current == null) return false;
if (_signature(mergeLogs(current, source)) != _signature(current)) {
return false;
}
}
return true;
}
/// Deletes the plaintext prefix, only after [verifyCopy] passes. Returns
/// whether it deleted.
Future<bool> deletePlaintextPrefix({
required RemoteStore store,
required E2eKey key,
required String fromPrefix,
required E2eLayout layout,
}) async {
if (!await verifyCopy(
store: store,
key: key,
fromPrefix: fromPrefix,
layout: layout,
)) {
return false;
}
for (final rel in (await _listBlobs(store, fromPrefix)).keys) {
await store.deleteFile('$fromPrefix/$rel', message: 'e2e cleanup');
}
return true;
}

View File

@ -0,0 +1,142 @@
/// Opens the encrypted store for an app whose old plaintext prefix is
/// [oldPrefix]; data lives on `E2eLayout.of(oldPrefix)`.
///
/// [readPassphrase] supplies the shared crdt-sync passphrase from the OS
/// keystore, so the open is prompt-free. Fail closed: a missing or wrong
/// passphrase, or an unreadable header, returns [E2eLocked] and the caller
/// must not sync. On success the old plaintext prefix is copied over
/// (idempotent; picks up writes from builds not yet migrated) and is never
/// deleted here.
library;
import 'dart:developer';
import 'e2e_key.dart';
import 'e2e_migration.dart';
import 'remote_store.dart';
import 'sync.dart';
/// Keystore entry name apps store the shared passphrase under.
const String kE2ePassphraseStorageKey = 'crdt_sync_e2e_passphrase';
/// Where a device keeps its derived 32-byte key, so Argon2id runs once per
/// device rather than once per sync tick. Back it with the OS keystore.
class E2eKeyCache {
/// Creates a cache from [read] and [write] closures.
const E2eKeyCache({required this.read, required this.write});
/// Returns the cached key bytes, or null.
final Future<List<int>?> Function() read;
/// Stores freshly derived key bytes.
final Future<void> Function(List<int> bytes) write;
}
/// Old-prefix revision fingerprints already copied, per old prefix.
final Map<String, String> _copied = {};
/// Forgets which old prefixes were copied; for tests.
void resetE2eCopyMemo() => _copied.clear();
Future<String> _oldFingerprint(RemoteStore store, String oldPrefix) async {
if (store is! BulkMapReader) return '';
try {
final revs = await (store as BulkMapReader).getStringMap(
defaultRevsPath(oldPrefix),
);
final keys = revs.keys.toList()..sort();
return [for (final k in keys) '$k=${revs[k]}'].join(';');
} on Object {
return '';
}
}
Future<E2eReady?> _tryCached(
RemoteStore store,
E2eLayout layout,
E2eKeyCache? cache,
) async {
final bytes = await cache?.read();
if (bytes == null || bytes.length != 32) return null;
final state = await openE2e(
store: store,
pathPrefix: layout.prefix,
key: await E2eKey.fromBytes(bytes),
e2ePath: layout.e2ePath,
revsPath: layout.revsPath,
);
return state is E2eReady ? state : null;
}
/// Opens the encrypted store; see the library doc.
///
/// [keyCache] skips Argon2id when it holds the key for the current header.
/// The one-time copy reruns only when the old prefix's published revisions
/// change (or once per process where the backend cannot bulk-read them).
/// [onDerive] is called each time Argon2id actually runs; for tests.
Future<E2eState> openEncryptedStore({
required RemoteStore store,
required String oldPrefix,
required Future<String?> Function() readPassphrase,
Argon2Params params = Argon2Params.recommended,
E2eKeyCache? keyCache,
void Function()? onDerive,
}) async {
final layout = E2eLayout.of(oldPrefix);
final E2eReady ready;
final E2eReady? cached;
try {
cached = await _tryCached(store, layout, keyCache);
} on E2eHeaderError {
return E2eLocked(E2eLockReason.badHeader);
}
if (cached != null) {
ready = cached;
} else {
final passphrase = await readPassphrase();
if (passphrase == null || passphrase.isEmpty) {
return E2eLocked(E2eLockReason.needsPassphrase);
}
final E2eKey key;
try {
onDerive?.call();
key = await unlockE2e(
store: store,
pathPrefix: layout.prefix,
passphrase: passphrase,
params: params,
e2ePath: layout.e2ePath,
);
} on E2eWrongPassphraseError {
return E2eLocked(E2eLockReason.wrongKey);
} on E2eHeaderError {
return E2eLocked(E2eLockReason.badHeader);
}
await keyCache?.write(key.bytes);
final state = await openE2e(
store: store,
pathPrefix: layout.prefix,
key: key,
e2ePath: layout.e2ePath,
revsPath: layout.revsPath,
);
if (state is! E2eReady) return state;
ready = state;
}
try {
final print = await _oldFingerprint(store, oldPrefix);
if (_copied[oldPrefix] != print) {
await copyToEncrypted(
store: store,
key: ready.store.key,
fromPrefix: oldPrefix,
layout: layout,
);
_copied[oldPrefix] = print;
}
} on Object catch (error, stackTrace) {
// The copy is retried next open; sync itself must not stop for it.
log('e2e copy failed', level: 900, error: error, stackTrace: stackTrace);
}
return ready;
}

View File

@ -0,0 +1,98 @@
import 'e2e_cipher.dart';
import 'e2e_envelope.dart';
import 'e2e_key.dart';
import 'remote_store.dart';
import 'sync.dart';
/// A [RemoteStore] that seals every blob under `pathPrefix` (nested
/// `<device>/<file>` or flat `<node>.json`) except the header, as a
/// per-record envelope (ids and HLCs clear, names and values opaque), and
/// keys every revision under `revsPath`, so the remote is a blind relay.
///
/// Wrap the outermost store (above a `MirrorStore`) so every backend gets
/// ciphertext. Other paths pass through unchanged.
class EncryptingStore implements RemoteStore, BulkMapReader {
EncryptingStore(
this._inner,
this._key, {
required String pathPrefix,
String? revsPath,
String? e2ePath,
bool acceptPlaintext = true,
}) : _cipher = _key.cipher(acceptPlaintext: acceptPlaintext),
_prefix = pathPrefix,
_revs = revsPath ?? defaultRevsPath(pathPrefix),
_header = e2ePath ?? defaultE2ePath(pathPrefix);
final RemoteStore _inner;
final E2eKey _key;
/// The key this store seals with.
E2eKey get key => _key;
final E2eCipher _cipher;
final String _prefix;
final String _revs;
final String _header;
bool _isRev(String path) => path.startsWith('$_revs/');
bool _isLog(String path) =>
!_isRev(path) && path != _header && path.startsWith('$_prefix/');
@override
Future<String?> getFileText(String path) async {
final text = await _inner.getFileText(path);
if (text == null || !_isLog(path)) return text;
return openLogText(text, _cipher, path: path);
}
@override
Future<void> putFileText(
String path,
String text, {
required String message,
}) async {
final stored = _isLog(path)
? await sealLogText(text, _cipher, path: path)
: _isRev(path)
? await _key.revision(text)
: text;
await _inner.putFileText(path, stored, message: message);
}
@override
Future<Map<String, String>> getStringMap(String path) async {
if (_inner case final BulkMapReader reader) {
final map = await reader.getStringMap(path);
if (!_isLog('$path/x')) return map;
final out = <String, String>{};
for (final MapEntry(:key, :value) in map.entries) {
final full = '$path/$key';
if (!_isLog(full)) {
out[key] = value;
continue;
}
try {
out[key] = await openLogText(value, _cipher, path: full);
} on E2eDecryptError {
// Unreadable entries are dropped, as a corrupt value would be.
}
}
return out;
}
return const {};
}
@override
Future<List<String>> listDirectory(String path) => _inner.listDirectory(path);
@override
Future<void> deleteFile(String path, {String message = ''}) =>
_inner.deleteFile(path, message: message);
@override
Future<bool> canAccessRemote() => _inner.canAccessRemote();
@override
void close() => _inner.close();
}

View File

@ -7,8 +7,11 @@ import 'store.dart';
/// Writes atomically -- to a per-process temp file, then `rename`d over the
/// real path -- so a concurrent reader (e.g. a background sync isolate writing
/// while the foreground app also writes) never observes a half-written file.
/// The pid in the temp name keeps two writers from colliding. This generalizes
/// diet_guard's hand-rolled `LogStorageService` write scheme.
/// The pid in the temp name keeps two processes from colliding; within one
/// instance, writes are serialized in issue order (last issued wins), so
/// overlapping calls (e.g. per-keystroke saves) never share the temp file
/// mid-flight. This generalizes diet_guard's hand-rolled `LogStorageService`
/// write scheme.
///
/// Lives behind the `package:crdt_sync/crdt_sync_io.dart` entrypoint, not the
/// main barrel, because it imports `dart:io` and so is unavailable on web --
@ -20,6 +23,10 @@ class FileLogPersistence implements LogPersistence {
final File _file;
/// Completes when the most recently issued write has settled. Never
/// rejects, so one failed write cannot block the ones queued after it.
Future<void> _tail = Future<void>.value();
@override
Future<String?> read() async {
if (!_file.existsSync()) return null;
@ -31,7 +38,14 @@ class FileLogPersistence implements LogPersistence {
}
@override
Future<void> write(String text) async {
Future<void> write(String text) {
final result = _tail.then((_) => _writeNow(text));
// The caller still sees its own error via [result]; the chain ignores it.
_tail = result.then((_) {}, onError: (Object _) {});
return result;
}
Future<void> _writeNow(String text) async {
await _file.parent.create(recursive: true);
final tmp = File('${_file.path}.$pid.tmp');
await tmp.writeAsString(text);

View File

@ -2,6 +2,7 @@ import 'dart:convert';
import 'package:crypto/crypto.dart';
import 'e2e_cipher.dart';
import 'log.dart';
import 'record.dart';
import 'remote_store.dart';
@ -44,6 +45,9 @@ String revisionOf(String encodedLog) =>
/// treated as this device's own rather than as a peer's, so its
/// pre-migration log is not pulled back and re-merged every tick. Pass null
/// once that path has been reclaimed.
///
/// [onPeerSkipped] reports each peer skipped as unreadable, e.g. an
/// [E2eDecryptError] from a wrong key or an app that needs updating.
Future<Log> syncLog({
required RemoteStore client,
required String deviceId,
@ -56,6 +60,7 @@ Future<Log> syncLog({
SyncStateStore? stateStore,
String? revsPath,
String? legacyDeviceId,
void Function(String deviceId, Object error)? onPeerSkipped,
}) async {
final revs = revsPath ?? defaultRevsPath(pathPrefix);
final state = await stateStore?.load() ?? const SyncState();
@ -80,15 +85,21 @@ Future<Log> syncLog({
seenRevs[otherDeviceId] = remoteRev;
continue;
}
final text = await client.getFileText(
'$pathPrefix/$otherDeviceId/$filename',
);
final String? text;
try {
text = await client.getFileText('$pathPrefix/$otherDeviceId/$filename');
} on E2eDecryptError catch (error) {
onPeerSkipped?.call(otherDeviceId, error);
continue;
}
if (text == null) continue;
try {
mergedLog = mergeLogs(mergedLog, decode(text));
} on FormatException {
} on FormatException catch (error) {
onPeerSkipped?.call(otherDeviceId, error);
continue;
} on TypeError {
} on TypeError catch (error) {
onPeerSkipped?.call(otherDeviceId, error);
continue;
}
// Only remembered once the merge succeeded: a corrupt push must be

View File

@ -41,6 +41,14 @@ packages:
url: "https://pub.dev"
source: hosted
version: "2.1.2"
characters:
dependency: transitive
description:
name: characters
sha256: faf38497bda5ead2a8c7615f4f7939df04333478bf32e4173fcb06d428b5716b
url: "https://pub.dev"
source: hosted
version: "1.4.1"
cli_config:
dependency: transitive
description:
@ -81,6 +89,22 @@ packages:
url: "https://pub.dev"
source: hosted
version: "3.0.8"
cryptography:
dependency: "direct main"
description:
name: cryptography
sha256: "3eda3029d34ec9095a27a198ac9785630fe525c0eb6a49f3d575272f8e792ef0"
url: "https://pub.dev"
source: hosted
version: "2.9.0"
ffi:
dependency: transitive
description:
name: ffi
sha256: "6d7fd89431262d8f3125e81b50d3847a091d846eafcd4fdb88dd06f36d705a45"
url: "https://pub.dev"
source: hosted
version: "2.2.0"
file:
dependency: transitive
description:
@ -97,6 +121,14 @@ packages:
url: "https://pub.dev"
source: hosted
version: "4.0.0"
glados:
dependency: "direct dev"
description:
name: glados
sha256: c39446c0c4fe83a6ee6ee4fdee5a1afa27050619c7df390bf4c3b77b6f1acb33
url: "https://pub.dev"
source: hosted
version: "1.1.7"
glob:
dependency: transitive
description:

View File

@ -2,7 +2,7 @@ name: crdt_sync
description: >-
Shared CRDT merge scheme + GitHub-Contents-API sync transport for
cross-device apps. Pure Dart, no Flutter dependency.
version: 0.12.0
version: 0.14.1
publish_to: 'none'
environment:
@ -10,8 +10,10 @@ environment:
dependencies:
crypto: 3.0.8
cryptography: 2.9.0
http: 1.6.0
dev_dependencies:
glados: 1.1.7
lints: 6.1.0
test: 1.33.0

View File

@ -40,9 +40,7 @@ void main() {
// Nothing listens on this port, so the GET throws and the `on Exception`
// path returns null -- while the `finally` still closes the client the
// function created for itself.
final account = await accountFromWrapper(
Uri.parse('http://127.0.0.1:1'),
);
final account = await accountFromWrapper(Uri.parse('http://127.0.0.1:1'));
expect(account, isNull);
});
@ -50,8 +48,10 @@ void main() {
final mock = http_testing.MockClient(
(_) async => http.Response(jsonEncode({}), 404),
);
expect(await accountFromWrapper(Uri.parse('http://x'), client: mock),
isNull);
expect(
await accountFromWrapper(Uri.parse('http://x'), client: mock),
isNull,
);
// Still usable: the function must not close a client it did not create.
expect((await mock.get(Uri.parse('http://x'))).statusCode, 404);
});
@ -106,27 +106,65 @@ void main() {
expect(await store.read(), isNull);
});
test('read returns null when the file cannot be read', () async {
// A mode-000 file still passes existsSync, so the guard does not fire
// and readAsString throws PathAccessException (a FileSystemException).
// A directory would NOT work here: File.existsSync() is false for one,
// so read() would return early down the guard path instead.
final file = File('${dir.path}/unreadable.json')..writeAsStringSync('{}');
final chmod = Process.runSync('chmod', ['000', file.path]);
expect(chmod.exitCode, 0);
addTearDown(() => Process.runSync('chmod', ['644', file.path]));
test(
'read returns null when the file cannot be read',
() async {
// A mode-000 file still passes existsSync, so the guard does not fire
// and readAsString throws PathAccessException (a FileSystemException).
// A directory would NOT work here: File.existsSync() is false for one,
// so read() would return early down the guard path instead.
final file = File('${dir.path}/unreadable.json')
..writeAsStringSync('{}');
final chmod = Process.runSync('chmod', ['000', file.path]);
expect(chmod.exitCode, 0);
addTearDown(() => Process.runSync('chmod', ['644', file.path]));
expect(await FileLogPersistence(file).read(), isNull);
},
// root ignores the permission bits, so the exception never fires.
skip: Platform.environment['USER'] == 'root'
? 'permission bits do not apply to root'
: null);
expect(await FileLogPersistence(file).read(), isNull);
},
// root ignores the permission bits, so the exception never fires.
skip: Platform.environment['USER'] == 'root'
? 'permission bits do not apply to root'
: null,
);
test('round-trips what it wrote', () async {
final store = FileLogPersistence(File('${dir.path}/log.json'));
await store.write('{"hello":"world"}');
expect(await store.read(), '{"hello":"world"}');
});
test(
'50 overlapping writes all succeed and the last issued wins',
() async {
// Regression: every call shared one `<file>.<pid>.tmp`, so the first
// rename moved it away and a later rename threw PathNotFoundException.
final store = FileLogPersistence(File('${dir.path}/log.json'));
await Future.wait([
for (var i = 0; i < 50; i++) store.write('{"n":$i}'),
]);
expect(await store.read(), '{"n":49}');
expect(dir.listSync().map((e) => e.uri.pathSegments.last), [
'log.json',
]);
},
);
test(
'a failed write rejects its caller but does not block later writes',
() async {
final target = File('${dir.path}/log.json');
// A directory at the target path makes the rename fail.
Directory(target.path).createSync();
final store = FileLogPersistence(target);
await expectLater(
store.write('{"n":0}'),
throwsA(isA<FileSystemException>()),
);
Directory(target.path).deleteSync();
await store.write('{"n":1}');
expect(await store.read(), '{"n":1}');
},
);
});
}

View File

@ -0,0 +1,56 @@
/// The done check: find plaintext blobs in a raw RTDB dump.
///
/// RTDB returns each blob as a JSON string, so its quotes arrive escaped and
/// `grep -c '"title"'` prints 0 even on plaintext. The audit decodes instead.
library;
import 'dart:convert';
import 'package:crdt_sync/crdt_sync.dart';
import 'package:test/test.dart';
// Shape copied from a live `GET /<root>.json` (scratch data).
const _livePlaintextDump =
r'{"notes":{"pc":{"log~2Ejson":"{\"n2\":{\"id\":\"n2\",\"fields\":'
r'{\"title\":[\"call mum\",\"1970-01-01T00:00:01.001Z-0000-pc\"]},'
r'\"deleted\":false,\"deleted_hlc\":null}}"}},"revs":{"pc":"ab12"}}';
void main() {
test('the grep done check misses live plaintext; the audit does not', () {
expect('"title"'.allMatches(_livePlaintextDump), isEmpty);
expect(plaintextBlobPaths(jsonDecode(_livePlaintextDump)), [
'notes/pc/log.json',
]);
});
test('sealed blobs, revs and the clear e2e header pass', () async {
final sealed = await E2eCipher(
List<int>.filled(32, 1),
).seal('{"title":"x"}', path: 'notes/pc/log.json');
final dump = {
'e2e': '{"v":1,"kdf":"argon2id"}',
'notes': {
'pc': {'log~2Ejson': sealed},
},
'revs': {'pc': 'ab12'},
};
expect(plaintextBlobPaths(dump), isEmpty);
});
test('every plaintext leaf is reported, with keys unescaped', () {
final dump = {
'a': {
'x~2Ey': '[1]',
'b': {'c': ' {"k":1}'},
},
'n': 3,
'nil': null,
};
expect(plaintextBlobPaths(dump), unorderedEquals(['a/x.y', 'a/b/c']));
});
test('a root that is itself plaintext, or empty, is handled', () {
expect(plaintextBlobPaths('{"a":1}'), ['']);
expect(plaintextBlobPaths(null), isEmpty);
});
}

View File

@ -0,0 +1,130 @@
/// The record-blob AEAD: what makes the remote a blind relay.
library;
import 'dart:convert';
import 'package:crdt_sync/crdt_sync.dart';
import 'package:glados/glados.dart' hide expect, group, test, setUp;
import 'package:test/test.dart';
const _path = 'todo-sync/notes/phone/log.json';
E2eCipher _cipher([int fill = 7, bool acceptPlaintext = true]) =>
E2eCipher(List<int>.filled(32, fill), acceptPlaintext: acceptPlaintext);
void main() {
Glados(any.letterOrDigits, ExploreConfig(numRuns: 60)).test(
'open(seal(x)) == x for any text',
(text) async {
final cipher = _cipher();
final sealed = await cipher.seal('{"t":"$text"}', path: _path);
expect(await cipher.open(sealed, path: _path), '{"t":"$text"}');
},
);
test('sealed text contains no plaintext and is not JSON', () async {
final sealed = await _cipher().seal('{"title":"secret"}', path: _path);
expect(sealed, isNot(contains('title')));
expect(sealed, isNot(contains('secret')));
expect(() => jsonDecode(sealed), throwsFormatException);
expect(E2eCipher.isPlaintext(sealed), isFalse);
});
test('two seals of the same text differ (random nonce)', () async {
final cipher = _cipher();
expect(
await cipher.seal('{}', path: _path),
isNot(await cipher.seal('{}', path: _path)),
);
});
test('wire format starts with the version byte', () async {
final bytes = base64.decode(await _cipher().seal('{}', path: _path));
expect(bytes.first, e2eFormatVersion);
expect(bytes.length, 1 + 24 + 2 + 16);
});
test('plaintext JSON passes through during migration', () async {
expect(await _cipher().open(' {"a":1}', path: _path), ' {"a":1}');
expect(await _cipher().open('[1]', path: _path), '[1]');
expect(E2eCipher.isPlaintext('\n{'), isTrue);
});
test('plaintext is refused once migration is over', () async {
expect(
() => _cipher(7, false).open('{"a":1}', path: _path),
throwsA(isA<E2eDecryptError>()),
);
});
group('rejects, as a FormatException so syncLog skips the peer', () {
test('a wrong key', () async {
final sealed = await _cipher(1).seal('{}', path: _path);
expect(
() => _cipher(2).open(sealed, path: _path),
throwsA(isA<FormatException>()),
);
});
test('a blob moved to another path', () async {
final sealed = await _cipher().seal('{}', path: _path);
expect(
() => _cipher().open(sealed, path: 'todo-sync/notes/pc/log.json'),
throwsA(isA<E2eDecryptError>()),
);
});
test('a flipped ciphertext byte', () async {
final bytes = base64.decode(await _cipher().seal('{}', path: _path));
bytes[30] ^= 1;
expect(
() => _cipher().open(base64.encode(bytes), path: _path),
throwsA(isA<E2eDecryptError>()),
);
});
test('an unknown version byte', () async {
final bytes = base64.decode(await _cipher().seal('{}', path: _path));
bytes[0] = 9;
expect(
() => _cipher().open(base64.encode(bytes), path: _path),
throwsA(
isA<E2eDecryptError>().having(
(e) => e.message,
'message',
contains('update the app'),
),
),
);
});
test('text that is not base64', () {
expect(
() => _cipher().open('not base64 !!', path: _path),
throwsA(isA<E2eDecryptError>()),
);
});
test('a blob too short to hold nonce and tag', () {
expect(
() => _cipher().open(base64.encode([1, 2, 3]), path: _path),
throwsA(isA<E2eDecryptError>()),
);
});
test('an empty blob', () {
expect(
() => _cipher().open('', path: _path),
throwsA(isA<E2eDecryptError>()),
);
});
});
test('a key that is not 32 bytes is a programming error', () {
expect(() => E2eCipher(const [1, 2]), throwsArgumentError);
});
test('E2eDecryptError reads as itself', () {
expect(E2eDecryptError('x').toString(), 'E2eDecryptError: x');
});
}

View File

@ -0,0 +1,156 @@
/// Per-record envelope: ids and HLCs stay clear, field names and values are
/// opaque, and merge results are identical with and without encryption.
library;
import 'dart:convert';
import 'package:crdt_sync/crdt_sync.dart';
import 'package:glados/glados.dart' hide expect, group, test, setUp;
import 'package:test/test.dart';
import 'e2e_fakes.dart';
const _path = 'app/notes-e2e/phone/log.json';
E2eCipher _cipher([int fill = 7, bool acceptPlaintext = true]) =>
E2eCipher(List<int>.filled(32, fill), acceptPlaintext: acceptPlaintext);
Hlc _hlc(int ms, int counter, String node) =>
Hlc(wallTimeMs: ms, counter: counter, nodeId: node);
Log _logFrom(List<(int, int, String)> rows) {
final log = <String, Record>{};
for (final (id, field, value) in rows) {
final key = 'r$id';
final old = log[key];
log[key] = Record(
id: key,
fields: {
...?old?.fields,
'f$field': (value, _hlc(1000 + id * 7 + field, field, 'n$id')),
},
deleted: id % 4 == 0,
deletedHlc: id % 4 == 0 ? _hlc(5000, 0, 'n$id') : null,
);
}
return log;
}
final _rows = any.list(
any.combine3(
any.intInRange(0, 6),
any.intInRange(0, 4),
any.letterOrDigits,
(a, b, c) => (a, b, c),
),
);
void main() {
Glados(_rows, ExploreConfig(numRuns: 40)).test(
'decode(open(seal(encode(log)))) == log',
(rows) async {
final log = _logFrom(rows);
final cipher = _cipher();
final sealed = await sealLogText(encodeLog(log), cipher, path: _path);
final opened = await openLogText(sealed, cipher, path: _path);
expect(encodeLog(decodeLog(opened)), encodeLog(log));
},
);
Glados2(_rows, _rows, ExploreConfig(numRuns: 40)).test(
'merge through the envelope == merge without it',
(a, b) async {
final cipher = _cipher();
Future<Log> viaCipher(Log log) async => decodeLog(
await openLogText(
await sealLogText(encodeLog(log), cipher, path: _path),
cipher,
path: _path,
),
);
final la = _logFrom(a);
final lb = _logFrom(b);
final plain = mergeLogs(la, lb);
final sealed = mergeLogs(await viaCipher(la), await viaCipher(lb));
expect(encodeLog(sealed), encodeLog(plain));
},
);
test('ids and HLCs are clear, names and values are not', () async {
final log = logOf({'n1': 'buy milk'});
final sealed = await sealLogText(encodeLog(log), _cipher(), path: _path);
final json = jsonDecode(sealed) as Map<String, dynamic>;
expect(json['v'], 1);
final rec = (json['r'] as Map<String, dynamic>)['n1'] as Map;
expect(rec['h'], log['n1']!.fields['title']!.$2.toStr());
expect(rec['d'], false);
expect(sealed, isNot(contains('title')));
expect(sealed, isNot(contains('buy milk')));
expect(isE2eEnvelope(sealed), isTrue);
expect(isE2eEnvelope(encodeLog(log)), isFalse);
});
test('legacy plaintext passes only while acceptPlaintext', () async {
final text = encodeLog(logOf({'n1': 'x'}));
expect(await openLogText(text, _cipher(), path: _path), text);
expect(
() => openLogText(text, _cipher(7, false), path: _path),
throwsA(isA<E2eDecryptError>()),
);
});
test('a record id named v is still legacy plaintext', () async {
final text = encodeLog(logOf({'v': 'x'}));
expect(isE2eEnvelope(text), isFalse);
});
group('tamper and wrong key are E2eDecryptError', () {
Future<String> sealed() async =>
sealLogText(encodeLog(logOf({'n1': 'x'})), _cipher(), path: _path);
test('wrong key', () async {
final text = await sealed();
expect(
() => openLogText(text, _cipher(8), path: _path),
throwsA(isA<E2eDecryptError>()),
);
});
test('moved path', () async {
final text = await sealed();
expect(
() => openLogText(text, _cipher(), path: '$_path-x'),
throwsA(isA<E2eDecryptError>()),
);
});
test('flipped clear deleted flag', () async {
final json = jsonDecode(await sealed()) as Map<String, dynamic>;
((json['r'] as Map)['n1'] as Map)['d'] = true;
expect(
() => openLogText(jsonEncode(json), _cipher(), path: _path),
throwsA(isA<E2eDecryptError>()),
);
});
test('record swapped to another id', () async {
final json = jsonDecode(await sealed()) as Map<String, dynamic>;
final r = json['r'] as Map;
r['n2'] = r.remove('n1');
expect(
() => openLogText(jsonEncode(json), _cipher(), path: _path),
throwsA(isA<E2eDecryptError>()),
);
});
test('mis-shaped envelope', () async {
expect(
() => openLogText('{"v":1,"r":5}', _cipher(), path: _path),
throwsA(isA<E2eDecryptError>()),
);
});
});
test('text that is not a log is sealed whole and opens back', () async {
final cipher = _cipher();
final sealed = await sealLogText('{"n":{"id":1}}', cipher, path: _path);
expect(sealed, isNot(contains('"id"')));
expect(await openLogText(sealed, cipher, path: _path), '{"n":{"id":1}}');
});
}

View File

@ -0,0 +1,111 @@
import 'dart:convert';
import 'package:crdt_sync/crdt_sync.dart';
const cheapArgon = Argon2Params(memoryKiB: 8, iterations: 1, parallelism: 1);
Log logOf(Map<String, String> titles, {String nodeId = 'node-a'}) => {
for (final entry in titles.entries)
entry.key: Record(
id: entry.key,
fields: {
'title': (
entry.value,
Hlc(wallTimeMs: 1000, counter: 0, nodeId: nodeId),
),
},
),
};
String encodeLog(Log log) =>
jsonEncode(log.map((id, record) => MapEntry(id, record.toJson())));
Log decodeLog(String text) => (jsonDecode(text) as Map<String, dynamic>).map(
(id, data) => MapEntry(id, Record.fromJson(data as Map<String, dynamic>)),
);
/// In-memory remote; [bulk] toggles the [BulkMapReader] capability.
class MemoryRemote implements RemoteStore, BulkMapReader {
MemoryRemote([Map<String, String>? files]) : files = files ?? {};
final Map<String, String> files;
final List<String> writes = [];
final List<String> reads = [];
bool closed = false;
/// Overrides the next reads of a path, oldest first, to fake a race.
final Map<String, List<String?>> scriptedReads = {};
@override
Future<List<String>> listDirectory(String path) async => files.keys
.where((key) => key.startsWith('$path/'))
.map((key) => key.substring(path.length + 1).split('/').first)
.toSet()
.toList();
@override
Future<String?> getFileText(String path) async {
reads.add(path);
final script = scriptedReads[path];
if (script != null && script.isNotEmpty) return script.removeAt(0);
return files[path];
}
@override
Future<void> putFileText(
String path,
String text, {
required String message,
}) async {
writes.add(path);
files[path] = text;
}
@override
Future<Map<String, String>> getStringMap(String path) async => {
for (final entry in files.entries)
if (entry.key.startsWith('$path/') &&
!entry.key.substring(path.length + 1).contains('/'))
entry.key.substring(path.length + 1): entry.value,
};
@override
Future<void> deleteFile(String path, {String message = ''}) async =>
files.remove(path);
@override
Future<bool> canAccessRemote() async => true;
@override
void close() => closed = true;
}
/// Same storage without the bulk capability, like `GitHubClient`.
class PlainRemote implements RemoteStore {
PlainRemote(this.inner);
final MemoryRemote inner;
@override
Future<List<String>> listDirectory(String path) => inner.listDirectory(path);
@override
Future<String?> getFileText(String path) => inner.getFileText(path);
@override
Future<void> putFileText(
String path,
String text, {
required String message,
}) => inner.putFileText(path, text, message: message);
@override
Future<void> deleteFile(String path, {String message = ''}) =>
inner.deleteFile(path);
@override
Future<bool> canAccessRemote() => inner.canAccessRemote();
@override
void close() => inner.close();
}

View File

@ -0,0 +1,45 @@
/// Cross-language golden vectors: Dart opens what the Python port sealed.
library;
import 'dart:convert';
import 'dart:io';
import 'package:crdt_sync/crdt_sync.dart';
import 'package:test/test.dart';
import 'e2e_fakes.dart';
void main() {
final vec =
jsonDecode(File('test/fixtures/e2e_py_vectors.json').readAsStringSync())
as Map<String, dynamic>;
Future<E2eKey> key() => unlockE2e(
store: MemoryRemote({'vec/e2e': vec['header'] as String}),
pathPrefix: 'vec/notes-e2e',
passphrase: vec['passphrase'] as String,
e2ePath: 'vec/e2e',
);
test('Argon2id and HKDF match Python', () async {
final hex = (await key()).bytes
.map((b) => b.toRadixString(16).padLeft(2, '0'))
.join();
expect(hex, vec['master_hex']);
});
test('opens a Python envelope and whole blob', () async {
final cipher = (await key()).cipher();
final path = vec['path'] as String;
final opened = await openLogText(
vec['envelope'] as String,
cipher,
path: path,
);
expect(jsonDecode(opened), jsonDecode(vec['log'] as String));
expect(
await openLogText(vec['whole'] as String, cipher, path: path),
'not a log',
);
});
}

View File

@ -0,0 +1,181 @@
/// Passphrase -> key, the clear header beside the data, and the open gate.
library;
import 'dart:convert';
import 'dart:math';
import 'package:crdt_sync/crdt_sync.dart';
import 'package:test/test.dart';
import 'e2e_fakes.dart';
const _prefix = 'todo-sync/notes';
const _headerPath = 'todo-sync/e2e';
Future<E2eKey> _unlock(MemoryRemote remote, String passphrase) => unlockE2e(
store: remote,
pathPrefix: _prefix,
passphrase: passphrase,
params: cheapArgon,
random: Random(1),
);
void main() {
test('the header sits beside the device dirs, not among them', () {
expect(defaultE2ePath(_prefix), _headerPath);
expect(defaultE2ePath('wake-alarm'), 'wake-alarm/e2e');
});
test(
'first unlock writes a clear header with salt, params and check',
() async {
final remote = MemoryRemote();
await _unlock(remote, 'correct horse');
final header = jsonDecode(remote.files[_headerPath]!) as Map;
expect(header['v'], 1);
expect(header['kdf'], 'argon2id');
expect(base64.decode(header['salt'] as String), hasLength(16));
expect(header['m'], 8);
expect(header['t'], 1);
expect(header['p'], 1);
expect(header['check'], isA<String>());
expect(remote.files[_headerPath], isNot(contains('correct horse')));
},
);
test('the same passphrase on another device yields the same key', () async {
final remote = MemoryRemote();
final a = await _unlock(remote, 'pw');
final b = await unlockE2e(
store: remote,
pathPrefix: _prefix,
passphrase: 'pw',
random: Random(99),
);
expect(b.bytes, a.bytes);
expect(remote.writes, [_headerPath]);
});
test('a wrong passphrase is refused before anything is written', () async {
final remote = MemoryRemote();
await _unlock(remote, 'pw');
remote.writes.clear();
await expectLater(
_unlock(remote, 'not pw'),
throwsA(isA<E2eWrongPassphraseError>()),
);
expect(remote.writes, isEmpty);
});
test('a setup race derives from the header that won', () async {
final winner = MemoryRemote();
await unlockE2e(
store: winner,
pathPrefix: _prefix,
passphrase: 'pw',
params: cheapArgon,
random: Random(5),
);
final remote = MemoryRemote()
..scriptedReads[_headerPath] = [null, winner.files[_headerPath]];
final key = await _unlock(remote, 'pw');
final expected = await _unlock(winner, 'pw');
expect(key.bytes, expected.bytes);
});
test('an empty passphrase is refused', () {
expect(() => _unlock(MemoryRemote(), ''), throwsArgumentError);
});
test('a corrupt or newer header is refused, never overwritten', () async {
for (final bad in ['nope', '[]', '{"v":2}', '{"v":1,"kdf":"scrypt"}']) {
final remote = MemoryRemote({_headerPath: bad});
await expectLater(
_unlock(remote, 'pw'),
throwsA(isA<E2eHeaderError>()),
reason: bad,
);
expect(remote.files[_headerPath], bad);
}
});
test('E2eKey.fromBytes round-trips a cached key', () async {
final key = await _unlock(MemoryRemote(), 'pw');
final cached = await E2eKey.fromBytes(
base64.decode(base64.encode(key.bytes)),
);
expect(cached.bytes, key.bytes);
expect(await cached.revision('x'), await key.revision('x'));
expect(() => E2eKey.fromBytes(const [1]), throwsArgumentError);
});
test('revision is keyed: not sha256, and differs across keys', () async {
final a = await _unlock(MemoryRemote(), 'a');
final b = await _unlock(MemoryRemote(), 'b');
expect(await a.revision('{}'), isNot(revisionOf('{}')));
expect(await a.revision('{}'), isNot(await b.revision('{}')));
expect(await a.revision('{}'), await a.revision('{}'));
});
test('errors read as themselves', () {
expect(E2eWrongPassphraseError().toString(), contains('passphrase'));
expect(E2eHeaderError('x').toString(), 'E2eHeaderError: x');
});
group('openE2e', () {
test('no header and no key: off, plaintext sync allowed', () async {
final state = await openE2e(store: MemoryRemote(), pathPrefix: _prefix);
expect(state, isA<E2eOff>());
});
test('header but no key: locked, needs the passphrase', () async {
final remote = MemoryRemote();
await _unlock(remote, 'pw');
final state = await openE2e(store: remote, pathPrefix: _prefix);
expect(state, isA<E2eLocked>());
expect((state as E2eLocked).reason, E2eLockReason.needsPassphrase);
});
test('a cached key the header rejects: locked, wrong key', () async {
final remote = MemoryRemote();
await _unlock(remote, 'pw');
final stale = await _unlock(MemoryRemote(), 'old');
final state = await openE2e(
store: remote,
pathPrefix: _prefix,
key: stale,
);
expect((state as E2eLocked).reason, E2eLockReason.wrongKey);
});
test('a cached key with the header gone: locked, wrong key', () async {
final key = await _unlock(MemoryRemote(), 'pw');
final state = await openE2e(
store: MemoryRemote(),
pathPrefix: _prefix,
key: key,
);
expect((state as E2eLocked).reason, E2eLockReason.wrongKey);
});
test('a corrupt header: locked, unreadable header', () async {
final state = await openE2e(
store: MemoryRemote({_headerPath: 'garbage'}),
pathPrefix: _prefix,
);
expect((state as E2eLocked).reason, E2eLockReason.badHeader);
});
test('a matching key: ready with an encrypting store', () async {
final remote = MemoryRemote();
final key = await _unlock(remote, 'pw');
final state = await openE2e(store: remote, pathPrefix: _prefix, key: key);
expect((state as E2eReady).store, isA<EncryptingStore>());
});
});
test('recommended params meet the OWASP Argon2id floor', () {
expect(Argon2Params.recommended.memoryKiB, greaterThanOrEqualTo(19456));
expect(Argon2Params.recommended.iterations, greaterThanOrEqualTo(2));
});
}

View File

@ -0,0 +1,221 @@
/// One-time copy from the plaintext prefix to `<prefix>-e2e`.
library;
import 'package:crdt_sync/crdt_sync.dart';
import 'package:test/test.dart';
import 'e2e_fakes.dart';
const _old = 'app/notes';
Future<E2eKey> _key(MemoryRemote remote, E2eLayout layout) => unlockE2e(
store: remote,
pathPrefix: layout.prefix,
passphrase: 'pw',
params: cheapArgon,
e2ePath: layout.e2ePath,
);
MemoryRemote _plainRemote() => MemoryRemote({
'$_old/phone/log.json': encodeLog(logOf({'a': 'milk', 'b': 'eggs'})),
'$_old/pc/log.json': encodeLog(logOf({'c': 'tea'}, nodeId: 'node-b')),
});
void main() {
final layout = E2eLayout.of(_old);
test('layout puts ciphertext on a new prefix, revs apart from old', () {
expect(layout.prefix, 'app/notes-e2e');
expect(layout.revsPath, isNot(defaultRevsPath(_old)));
expect(layout.e2ePath, startsWith('app/'));
});
test('copy seals every device log under the new prefix', () async {
final remote = _plainRemote();
final key = await _key(remote, layout);
final report = await copyToEncrypted(
store: remote,
key: key,
fromPrefix: _old,
layout: layout,
);
expect(report.copied, 2);
final blob = remote.files['app/notes-e2e/phone/log.json']!;
expect(blob, isNot(contains('milk')));
expect(blob, isNot(contains('"title"')));
expect(remote.files['$_old/phone/log.json'], contains('milk'));
final enc = EncryptingStore(
remote,
key,
pathPrefix: layout.prefix,
revsPath: layout.revsPath,
e2ePath: layout.e2ePath,
);
final merged = await syncLog(
client: enc,
deviceId: 'tablet',
pathPrefix: layout.prefix,
localLog: {},
encode: encodeLog,
decode: decodeLog,
);
expect(merged.keys, unorderedEquals(['a', 'b', 'c']));
});
test('copying twice adds no duplicate and loses nothing', () async {
final remote = _plainRemote();
final key = await _key(remote, layout);
Future<MigrationReport> run() => copyToEncrypted(
store: remote,
key: key,
fromPrefix: _old,
layout: layout,
);
await run();
final first = Map.of(remote.files);
remote.writes.clear();
final second = await run();
expect(second.copied, 0);
expect(remote.writes, isEmpty);
expect(remote.files, first);
});
test('a later plaintext write by an old build is copied again', () async {
final remote = _plainRemote();
final key = await _key(remote, layout);
await copyToEncrypted(
store: remote,
key: key,
fromPrefix: _old,
layout: layout,
);
remote.files['$_old/pc/log.json'] = encodeLog(
logOf({'c': 'tea', 'd': 'jam'}, nodeId: 'node-b'),
);
final report = await copyToEncrypted(
store: remote,
key: key,
fromPrefix: _old,
layout: layout,
);
expect(report.copied, 1);
expect(
await verifyCopy(
store: remote,
key: key,
fromPrefix: _old,
layout: layout,
),
isTrue,
);
});
test('cleanup refuses before the copy is verified, then deletes', () async {
final remote = _plainRemote();
final key = await _key(remote, layout);
expect(
await deletePlaintextPrefix(
store: remote,
key: key,
fromPrefix: _old,
layout: layout,
),
isFalse,
);
expect(remote.files['$_old/phone/log.json'], isNotNull);
await copyToEncrypted(
store: remote,
key: key,
fromPrefix: _old,
layout: layout,
);
expect(
await deletePlaintextPrefix(
store: remote,
key: key,
fromPrefix: _old,
layout: layout,
),
isTrue,
);
expect(remote.files.keys.where((k) => k.startsWith('$_old/')), isEmpty);
expect(remote.files['app/notes-e2e/phone/log.json'], isNotNull);
});
test('a corrupt source blob is skipped and blocks cleanup', () async {
final remote = _plainRemote();
remote.files['$_old/bad/log.json'] = '{oops';
final key = await _key(remote, layout);
final report = await copyToEncrypted(
store: remote,
key: key,
fromPrefix: _old,
layout: layout,
);
expect(report.copied, 2);
expect(report.skipped, ['bad/log.json']);
expect(
await deletePlaintextPrefix(
store: remote,
key: key,
fromPrefix: _old,
layout: layout,
),
isFalse,
);
});
group('slashless prefix (settings-sync)', () {
final flat = E2eLayout.of('settings-sync');
test('revs and header stay inside the app node', () {
expect(flat.prefix, 'settings-sync-e2e');
expect(flat.revsPath, 'settings-sync-e2e/revs');
expect(flat.e2ePath, 'settings-sync-e2e/e2e');
});
test('old revs and header are not copied as logs', () async {
final remote = MemoryRemote({
'settings-sync/phone/log.json': encodeLog(logOf({'a': 'x'})),
'settings-sync/revs/phone': 'abc',
'settings-sync/e2e': '{"v":1}',
});
final key = await _key(remote, flat);
final report = await copyToEncrypted(
store: remote,
key: key,
fromPrefix: 'settings-sync',
layout: flat,
);
expect(report.copied, 1);
expect(report.skipped, isEmpty);
expect(
await verifyCopy(
store: remote,
key: key,
fromPrefix: 'settings-sync',
layout: flat,
),
isTrue,
);
});
});
test('never overwrites a non-log blob and skips empty sources', () async {
final remote = MemoryRemote({
'app/notes/pc/current.json': '{}',
'app/notes-e2e/pc/current.json': '{"photos":["a"]}',
'app/notes/tab/current.json': '{}',
});
final key = await _key(remote, layout);
final report = await copyToEncrypted(
store: remote,
key: key,
fromPrefix: _old,
layout: layout,
);
expect(report.copied, 0);
expect(remote.files['app/notes-e2e/pc/current.json'], '{"photos":["a"]}');
expect(remote.files.containsKey('app/notes-e2e/tab/current.json'), isFalse);
});
}

View File

@ -0,0 +1,98 @@
/// openEncryptedStore must not pay Argon2id or a full copy on every tick.
library;
import 'package:crdt_sync/crdt_sync.dart';
import 'package:test/test.dart';
import 'e2e_fakes.dart';
const _old = 'app/notes';
void main() {
setUp(resetE2eCopyMemo);
test('a cached key skips Argon2 and still unlocks', () async {
final remote = MemoryRemote();
List<int>? cached;
var derived = 0;
Future<E2eState> open() => openEncryptedStore(
store: remote,
oldPrefix: _old,
readPassphrase: () async => 'pw',
params: cheapArgon,
keyCache: E2eKeyCache(
read: () async => cached,
write: (bytes) async => cached = bytes,
),
onDerive: () => derived++,
);
expect(await open(), isA<E2eReady>());
expect(cached, hasLength(32));
expect(await open(), isA<E2eReady>());
expect(derived, 1);
});
test('a stale cached key falls back to the passphrase', () async {
final remote = MemoryRemote();
List<int>? cached = List<int>.filled(32, 9);
final state = await openEncryptedStore(
store: remote,
oldPrefix: _old,
readPassphrase: () async => 'pw',
params: cheapArgon,
keyCache: E2eKeyCache(
read: () async => cached,
write: (bytes) async => cached = bytes,
),
);
expect(state, isA<E2eReady>());
expect(cached, isNot(List<int>.filled(32, 9)));
});
test('the copy reruns only when old revisions change', () async {
final remote = MemoryRemote({
'$_old/pc/log.json': encodeLog(logOf({'a': 'x'})),
'app/revs/pc': 'r1',
});
Future<void> open() => openEncryptedStore(
store: remote,
oldPrefix: _old,
readPassphrase: () async => 'pw',
params: cheapArgon,
);
await open();
remote.reads.clear();
await open();
expect(remote.reads.where((p) => p.startsWith('app/notes-e2e/')), isEmpty);
remote.files['$_old/pc/log.json'] = encodeLog(logOf({'a': 'x', 'b': 'y'}));
remote.files['app/revs/pc'] = 'r2';
await open();
final state =
await openEncryptedStore(
store: remote,
oldPrefix: _old,
readPassphrase: () async => 'pw',
params: cheapArgon,
)
as E2eReady;
final text = await state.store.getFileText('app/notes-e2e/pc/log.json');
expect(decodeLog(text!).keys, unorderedEquals(['a', 'b']));
});
test('bulk reads of a data dir come back decrypted', () async {
final remote = MemoryRemote();
final state =
await openEncryptedStore(
store: remote,
oldPrefix: _old,
readPassphrase: () async => 'pw',
params: cheapArgon,
)
as E2eReady;
final log = encodeLog(logOf({'k': 'secret'}));
await state.store.putFileText('app/notes-e2e/k1', log, message: 'm');
expect(remote.files['app/notes-e2e/k1'], isNot(contains('secret')));
final map = await state.store.getStringMap('app/notes-e2e');
expect(map['k1'], log);
});
}

View File

@ -0,0 +1,63 @@
import 'package:crdt_sync/crdt_sync.dart';
import 'package:test/test.dart';
import 'e2e_fakes.dart';
const _old = 'app/notes';
Future<E2eState> _open(MemoryRemote r, String? pass) => openEncryptedStore(
store: r,
oldPrefix: _old,
readPassphrase: () async => pass,
params: cheapArgon,
);
void main() {
test('no passphrase is locked', () async {
for (final p in [null, '']) {
final s = await _open(MemoryRemote(), p);
expect((s as E2eLocked).reason, E2eLockReason.needsPassphrase);
}
});
test('right passphrase copies and is ready; wrong one is locked', () async {
final r = MemoryRemote({
'$_old/phone/log.json': encodeLog(logOf({'a': 'milk'})),
});
expect(await _open(r, 'pw'), isA<E2eReady>());
expect(r.files['app/notes-e2e/phone/log.json'], isNot(contains('milk')));
expect(await _open(r, 'pw'), isA<E2eReady>());
expect(
((await _open(r, 'bad')) as E2eLocked).reason,
E2eLockReason.wrongKey,
);
});
test('a corrupt header locks as badHeader', () async {
final r = MemoryRemote({'app/e2e': 'garbage'});
expect(
((await _open(r, 'pw')) as E2eLocked).reason,
E2eLockReason.badHeader,
);
});
test('a failing copy does not stop the open', () async {
final r = MemoryRemote();
r.files['$_old/phone/log.json'] = encodeLog(logOf({'a': 'x'}));
final failing = _FailingList(r);
final s = await openEncryptedStore(
store: failing,
oldPrefix: _old,
readPassphrase: () async => 'pw',
params: cheapArgon,
);
expect(s, isA<E2eReady>());
});
}
class _FailingList extends PlainRemote {
_FailingList(super.inner);
@override
Future<List<String>> listDirectory(String path) =>
throw StateError('list down');
}

View File

@ -0,0 +1,290 @@
/// syncLog over an EncryptingStore: the relay sees ciphertext, merge is
/// unchanged.
library;
import 'package:crdt_sync/crdt_sync.dart';
import 'package:glados/glados.dart' hide expect, group, test, setUp;
import 'package:test/test.dart';
import 'e2e_fakes.dart';
const _prefix = 'todo-sync/notes';
const _revs = 'todo-sync/revs';
Future<E2eKey> _key(MemoryRemote remote, [String passphrase = 'pw']) =>
unlockE2e(
store: remote,
pathPrefix: _prefix,
passphrase: passphrase,
params: cheapArgon,
random: Random(1),
);
Future<Log> _tick(
RemoteStore client,
String deviceId,
Log local, {
SyncStateStore? state,
List<String>? skipped,
}) => syncLog(
client: client,
deviceId: deviceId,
pathPrefix: _prefix,
localLog: local,
encode: encodeLog,
decode: decodeLog,
stateStore: state,
onPeerSkipped: (id, _) => skipped?.add(id),
);
void main() {
test('pushed blob is ciphertext and a peer merges it', () async {
final remote = MemoryRemote();
final key = await _key(remote);
final store = EncryptingStore(remote, key, pathPrefix: _prefix);
await _tick(store, 'phone', logOf({'n1': 'buy milk'}));
final blob = remote.files['$_prefix/phone/log.json']!;
expect(blob, isNot(contains('title')));
expect(blob, isNot(contains('buy milk')));
final merged = await _tick(store, 'pc', logOf({'n2': 'call mum'}));
expect(merged.keys, unorderedEquals(['n1', 'n2']));
expect(remote.files.values.where((v) => v.contains('"title"')), isEmpty);
});
test('published revisions are keyed, not sha256 of the plaintext', () async {
final remote = MemoryRemote();
final store = EncryptingStore(
remote,
await _key(remote),
pathPrefix: _prefix,
);
final local = logOf({'n1': 'x'});
await _tick(store, 'phone', local, state: InMemorySyncStateStore());
final rev = remote.files['$_revs/phone']!;
expect(rev, isNot(revisionOf(encodeLog(local))));
expect(rev, hasLength(64));
});
test('a peer that has not upgraded is still read', () async {
final remote = MemoryRemote();
await _tick(remote, 'pc', logOf({'old': 'plain'}));
final store = EncryptingStore(
remote,
await _key(remote),
pathPrefix: _prefix,
);
final merged = await _tick(store, 'phone', logOf({'n1': 'x'}));
expect(merged.keys, unorderedEquals(['old', 'n1']));
});
test('a tampered peer is skipped and the others still merge', () async {
final remote = MemoryRemote();
final store = EncryptingStore(
remote,
await _key(remote),
pathPrefix: _prefix,
);
await _tick(store, 'b', logOf({'b1': 'y'}));
await _tick(store, 'a', logOf({'a1': 'x'}));
final blob = remote.files['$_prefix/a/log.json']!;
remote.files['$_prefix/a/log.json'] = blob.replaceRange(
40,
41,
blob[40] == 'A' ? 'B' : 'A',
);
final skipped = <String>[];
final merged = await _tick(store, 'c', {}, skipped: skipped);
expect(merged.keys, ['b1']);
expect(skipped, ['a']);
});
test('unparsable and mis-shaped peers are reported as skipped', () async {
final remote = MemoryRemote({
'$_prefix/bad/log.json': '{oops',
'$_prefix/shape/log.json': '{"n":{"id":1}}',
});
final skipped = <String>[];
await _tick(remote, 'me', {}, skipped: skipped);
expect(skipped, unorderedEquals(['bad', 'shape']));
});
test('a peer under another passphrase is skipped, not fatal', () async {
final remote = MemoryRemote();
final other = MemoryRemote();
final mine = EncryptingStore(
remote,
await _key(remote),
pathPrefix: _prefix,
);
final theirs = EncryptingStore(
remote,
await _key(other, 'other'),
pathPrefix: _prefix,
);
await _tick(theirs, 'x', logOf({'x1': 'x'}));
expect(await _tick(mine, 'me', {}), isEmpty);
});
test('a prefix without a slash keeps revs out of the sealed set', () async {
final remote = MemoryRemote();
final key = await unlockE2e(
store: remote,
pathPrefix: 'wake-alarm',
passphrase: 'pw',
params: cheapArgon,
);
final store = EncryptingStore(remote, key, pathPrefix: 'wake-alarm');
await syncLog(
client: store,
deviceId: 'phone',
pathPrefix: 'wake-alarm',
localLog: logOf({'n': 'x'}),
encode: encodeLog,
decode: decodeLog,
stateStore: InMemorySyncStateStore(),
);
expect(remote.files['wake-alarm/revs/phone'], hasLength(64));
expect(remote.files['wake-alarm/phone/log.json'], isNot(contains('"x"')));
expect(remote.files['wake-alarm/e2e'], contains('argon2id'));
});
test('paths outside the prefix pass through untouched', () async {
final remote = MemoryRemote();
final store = EncryptingStore(
remote,
await _key(remote),
pathPrefix: _prefix,
);
await store.putFileText('other/x', '{"a":1}', message: 'm');
expect(remote.files['other/x'], '{"a":1}');
expect(await store.getFileText('other/x'), '{"a":1}');
expect(await store.getFileText('$_prefix/none/log.json'), isNull);
});
test('a flat <prefix>/<node>.json layout (todo) is sealed too', () async {
final remote = MemoryRemote();
final store = EncryptingStore(
remote,
await _key(remote),
pathPrefix: _prefix,
);
final log = encodeLog(logOf({'n1': 'buy milk'}));
await store.putFileText('$_prefix/node-a.json', log, message: 'm');
expect(remote.files['$_prefix/node-a.json'], isNot(contains('milk')));
expect(isE2eEnvelope(remote.files['$_prefix/node-a.json']!), isTrue);
expect(await store.getFileText('$_prefix/node-a.json'), log);
});
test('a slashless prefix keeps the header and revs out of sealing', () async {
final remote = MemoryRemote();
final key = await unlockE2e(
store: remote,
pathPrefix: 'wake-alarm',
passphrase: 'pw',
params: cheapArgon,
);
final store = EncryptingStore(remote, key, pathPrefix: 'wake-alarm');
final header = remote.files['wake-alarm/e2e']!;
expect(await store.getFileText('wake-alarm/e2e'), header);
await store.putFileText('wake-alarm/e2e', header, message: 'm');
expect(remote.files['wake-alarm/e2e'], header);
});
test('plaintext is refused when acceptPlaintext is off', () async {
final remote = MemoryRemote({
'$_prefix/pc/log.json': encodeLog(logOf({'a': 'b'})),
});
final store = EncryptingStore(
remote,
await _key(remote),
pathPrefix: _prefix,
acceptPlaintext: false,
);
expect(await _tick(store, 'phone', {}), isEmpty);
});
group('delegation', () {
test('bulk reads forward when the inner store has them', () async {
final remote = MemoryRemote({'$_revs/a': 'r'});
final store = EncryptingStore(
remote,
await _key(remote),
pathPrefix: _prefix,
);
expect(await store.getStringMap(_revs), {'a': 'r'});
});
test('bulk reads degrade to "fetch everything" without them', () async {
final remote = MemoryRemote({'$_revs/a': 'r'});
final store = EncryptingStore(
PlainRemote(remote),
await _key(remote),
pathPrefix: _prefix,
);
expect(await store.getStringMap(_revs), isEmpty);
});
test('list, delete, access and close reach the inner store', () async {
final remote = MemoryRemote({'$_prefix/a/log.json': '{}'});
final store = EncryptingStore(
remote,
await _key(remote),
pathPrefix: _prefix,
);
expect(await store.listDirectory(_prefix), ['a']);
await store.deleteFile('$_prefix/a/log.json', message: 'm');
expect(remote.files.containsKey('$_prefix/a/log.json'), isFalse);
expect(await store.canAccessRemote(), isTrue);
store.close();
expect(remote.closed, isTrue);
});
});
// Differential: the same sync rounds over a plain and an encrypting relay
// must converge to the same logs.
Glados(
any.list(
any.combine3(
any.intInRange(0, 3),
any.intInRange(0, 6),
any.letterOrDigits,
(int d, int n, String t) => (d, n, t),
),
),
ExploreConfig(numRuns: 25),
).test('merge is identical with and without encryption', (edits) async {
const devices = ['a', 'b', 'c'];
Log localFor(int d) => {
for (final (i, (dev, note, title)) in edits.indexed)
if (dev == d)
'n$note': Record(
id: 'n$note',
fields: {
'title': (
title,
Hlc(wallTimeMs: 1000 + i, counter: 0, nodeId: devices[d]),
),
},
),
};
final plain = MemoryRemote();
final sealedRemote = MemoryRemote();
final sealed = EncryptingStore(
sealedRemote,
await E2eKey.fromBytes(List<int>.filled(32, 3)),
pathPrefix: _prefix,
);
final plainLogs = [for (var d = 0; d < 3; d++) localFor(d)];
final sealedLogs = [for (var d = 0; d < 3; d++) localFor(d)];
for (var round = 0; round < 2; round++) {
for (var d = 0; d < 3; d++) {
plainLogs[d] = await _tick(plain, devices[d], plainLogs[d]);
sealedLogs[d] = await _tick(sealed, devices[d], sealedLogs[d]);
}
}
expect(sealedLogs, plainLogs);
expect(sealedLogs[0], sealedLogs[2]);
});
}

View File

@ -0,0 +1,10 @@
{
"passphrase": "correct horse ✓",
"header": "{\"v\":1,\"kdf\":\"argon2id\",\"salt\":\"HGW+FZJe6hd71He22BrlUA==\",\"m\":64,\"t\":2,\"p\":1,\"check\":\"okKnATkNoZKvjSGfrgQGOjdRw5NORh2lC32zuBq7V6I=\"}",
"master_hex": "cad5afe8ddc6e21d2f39eb196de454db6b6d1f60629619ce8539f03a4bb0b086",
"revision_of_abc": "e7f845e1b0a7ea2b571fd3e3bdfb60641e650d63171bf73b6356c7909d2748c8",
"path": "vec/notes-e2e/phone/log.json",
"log": "{\"n1\":{\"id\":\"n1\",\"fields\":{\"title\":[\"buy milk\",\"1970-01-01T00:00:01.000Z-0000-node-a\"]},\"deleted\":false,\"deleted_hlc\":null},\"n2\":{\"id\":\"n2\",\"fields\":{\"title\":[\"zażółć\",\"1970-01-01T00:00:01.000Z-0000-node-a\"]},\"deleted\":false,\"deleted_hlc\":null}}",
"envelope": "{\"v\":1,\"r\":{\"n1\":{\"d\":false,\"dh\":null,\"h\":\"1970-01-01T00:00:01.000Z-0000-node-a\",\"p\":\"AR1zODx4TD67X3TGa0N7EV/V4FcZAaqc41jxZVtskFu3Pqb/pNlqm6db66rQUaMyNLtTPa4HMOJzZomBRyaSHjaPb4oNxgVPkMeIF5IbuI/GA30gR0sz/MSvYmfZ5wSnlBo1+fpH\"},\"n2\":{\"d\":false,\"dh\":null,\"h\":\"1970-01-01T00:00:01.000Z-0000-node-a\",\"p\":\"Aa0omlBGnmlsItzq7A21G0A1tqT8xMOJE0wgxcxJ1c3hVpRmNt4kq2IPNNGuW8v4A7fvs2FrnGX4A+NVtXPpu41ckKSShTqk8Ml6zTuxyrT85g50sgzbM7PKFczeEaS+ITWDj0N04u0=\"}}}",
"whole": "{\"v\":1,\"b\":\"AYdAwPehG84oHBRhRr5dP5VmleBGzQC1ryV5QHTCqBZE9ZsOZMMkJiP40+KzpnfIc2g=\"}"
}

View File

@ -0,0 +1,9 @@
{
"passphrase": "correct horse ✓",
"header": "{\"v\":1,\"kdf\":\"argon2id\",\"salt\":\"fGy9SnWSTwxKbS/sAluVjA==\",\"m\":64,\"t\":2,\"p\":1,\"check\":\"ekpcuZxZl+Ei6Y5yQp+cwdi4yR6ItJiYR7S3cd214ds=\"}",
"master_hex": "834f54fef4eb086e4201b918c938be179f91cf72e157e28d56dfe72bc46cbefd",
"path": "vec/notes-e2e/pc/log.json",
"log": "{\"n9\": {\"id\": \"n9\", \"fields\": {\"title\": [\"zażółć\", \"2026-01-01T00:00:00.000Z-0000-pc\"]}, \"deleted\": false, \"deleted_hlc\": null}}",
"envelope": "{\"v\":1,\"r\":{\"n9\":{\"d\":false,\"dh\":null,\"h\":\"2026-01-01T00:00:00.000Z-0000-pc\",\"p\":\"ATAIR4jQmlBjDF+/4+AYp3ATcGgjZQ60tQ05uvF1howYly4W4h3lLI04BssCRASargNV/+hDhqTl7n2cz4hh7Z5lmZYgElso/4k85hwO1SRuDPOYMKsLCpYgTy8zYzEiiRlv+g==\"}}}",
"whole": "{\"v\":1,\"b\":\"ARvUAj5nIfmJdsxdaiAl5uYkgry9VG5VQLQBi6+BNbyJ0t//UGjc1Ji2hmbBJl3v9kw=\"}"
}

View File

@ -0,0 +1,47 @@
/// Read-only done check: lists plaintext blobs under an RTDB root.
///
/// Usage: `dart run tool/e2e_audit.dart <root> <firebase_auth.json>`
/// Exit 0 when every blob is sealed, 1 when any is plaintext, 2 on misuse.
library;
import 'dart:convert';
import 'dart:io';
import 'package:crdt_sync/crdt_sync.dart';
import 'package:http/http.dart' as http;
Future<void> main(List<String> args) async {
if (args.length != 2) {
stderr.writeln('usage: e2e_audit.dart <root> <firebase_auth.json>');
exit(2);
}
final [root, sessionFile] = args;
final home = Platform.environment['HOME'];
final cfg =
jsonDecode(
File('$home/.config/crdt-sync/firebase.json').readAsStringSync(),
)
as Map<String, dynamic>;
final session = FirebaseCredentials.fromJson(
jsonDecode(File(sessionFile).readAsStringSync()) as Map<String, dynamic>,
);
final auth = FirebaseTokenProvider(
apiKey: cfg['apiKey'] as String,
store: InMemoryCredentialStore(session),
);
final url = Uri.parse(
'${cfg['databaseUrl']}/${encodePath(root)}.json'
'?auth=${await auth.idToken()}',
);
final response = await http.get(url);
if (response.statusCode != 200) {
stderr.writeln('GET $root: HTTP ${response.statusCode}');
exit(2);
}
final plaintext = plaintextBlobPaths(jsonDecode(response.body));
stdout.writeln('plaintext blobs under $root: ${plaintext.length}');
for (final path in plaintext) {
stdout.writeln(' $root/$path');
}
exit(plaintext.isEmpty ? 0 : 1);
}

View File

@ -0,0 +1,41 @@
/// Writes cross-language golden vectors for the Python port to open.
///
/// Run: `dart run tool/e2e_vectors.dart > test/fixtures/e2e_dart_vectors.json`
library;
import 'dart:convert';
import 'dart:math';
import 'package:crdt_sync/crdt_sync.dart';
import '../test/e2e_fakes.dart';
Future<void> main() async {
const params = Argon2Params(memoryKiB: 64, iterations: 2, parallelism: 1);
final remote = MemoryRemote();
final key = await unlockE2e(
store: remote,
pathPrefix: 'vec/notes-e2e',
passphrase: 'correct horse ✓',
params: params,
random: Random(7),
e2ePath: 'vec/e2e',
);
final cipher = key.cipher();
const path = 'vec/notes-e2e/phone/log.json';
final log = encodeLog(logOf({'n1': 'buy milk', 'n2': 'zażółć'}));
print(
const JsonEncoder.withIndent(' ').convert({
'passphrase': 'correct horse ✓',
'header': remote.files['vec/e2e'],
'master_hex': key.bytes
.map((b) => b.toRadixString(16).padLeft(2, '0'))
.join(),
'revision_of_abc': await key.revision('abc'),
'path': path,
'log': log,
'envelope': await sealLogText(log, cipher, path: path),
'whole': await sealLogText('not a log', cipher, path: path),
}),
);
}

View File

@ -12,15 +12,15 @@ import 'package:crdt_sync/crdt_sync.dart';
/// mode 0600 -- never in source, because this repository is public.
final _configDir = '${Platform.environment['HOME']}/.config/crdt-sync';
Log _decode(String t) => (jsonDecode(t) as Map<String, dynamic>)
.map((k, v) => MapEntry(k, Record.fromJson(v as Map<String, dynamic>)));
String _encode(Log l) =>
jsonEncode(l.map((k, v) => MapEntry(k, v.toJson())));
Log _decode(String t) => (jsonDecode(t) as Map<String, dynamic>).map(
(k, v) => MapEntry(k, Record.fromJson(v as Map<String, dynamic>)),
);
String _encode(Log l) => jsonEncode(l.map((k, v) => MapEntry(k, v.toJson())));
Future<void> main(List<String> args) async {
final cfg = jsonDecode(
File('$_configDir/firebase.json').readAsStringSync(),
) as Map<String, dynamic>;
final cfg =
jsonDecode(File('$_configDir/firebase.json').readAsStringSync())
as Map<String, dynamic>;
final auth = FirebaseTokenProvider(
apiKey: cfg['apiKey'] as String,
store: InMemoryCredentialStore(),

View File

@ -23,7 +23,9 @@ const _prefix = '_smoketest/devices';
var _failures = 0;
void _check(String what, bool ok, [String detail = '']) {
stdout.writeln('${ok ? " ok " : " FAIL"} $what${detail.isEmpty ? '' : ' ($detail)'}');
stdout.writeln(
'${ok ? " ok " : " FAIL"} $what${detail.isEmpty ? '' : ' ($detail)'}',
);
if (!ok) _failures++;
}
@ -68,16 +70,24 @@ Future<void> main(List<String> args) async {
_check('canAccessRemote is false before sign-in', !await auth.hasSession());
await auth.signIn(email: email, password: password);
_check('signIn stored a session', await auth.hasSession());
_check('canAccessRemote is true after sign-in', await client.canAccessRemote());
_check(
'canAccessRemote is true after sign-in',
await client.canAccessRemote(),
);
stdout.writeln('\n== round-trip and key escaping ==');
const path = '$_prefix/pc/log.json';
await client.putFileText(path, '{"hello":"world"}', message: 'smoke');
final read = await client.getFileText(path);
_check('a .json filename round-trips byte-for-byte',
read == '{"hello":"world"}', read ?? 'null');
_check('missing path reads as null',
await client.getFileText('$_prefix/nope/log.json') == null);
_check(
'a .json filename round-trips byte-for-byte',
read == '{"hello":"world"}',
read ?? 'null',
);
_check(
'missing path reads as null',
await client.getFileText('$_prefix/nope/log.json') == null,
);
final listed = await client.listDirectory(_prefix);
_check('listDirectory sees the device', listed.contains('pc'), '$listed');
@ -96,8 +106,11 @@ Future<void> main(List<String> args) async {
final pushed = await client.getFileText(path);
_check('the log landed in the database', pushed != null);
final revs = await client.getStringMap('_smoketest/revs');
_check('a revision was published for this device',
revs['pc'] == revisionOf(_encode(pcLog)), '${revs.keys}');
_check(
'a revision was published for this device',
revs['pc'] == revisionOf(_encode(pcLog)),
'${revs.keys}',
);
stdout.writeln('\n== sync: an unchanged tick writes nothing ==');
final before = await client.getStringMap('_smoketest/revs');
@ -110,8 +123,10 @@ Future<void> main(List<String> args) async {
decode: _decode,
stateStore: storePc,
);
_check('revision unchanged, so nothing was re-uploaded',
(await client.getStringMap('_smoketest/revs'))['pc'] == before['pc']);
_check(
'revision unchanged, so nothing was re-uploaded',
(await client.getStringMap('_smoketest/revs'))['pc'] == before['pc'],
);
stdout.writeln('\n== sync: two devices converge ==');
final storePhone = InMemorySyncStateStore();
@ -135,8 +150,10 @@ Future<void> main(List<String> args) async {
stateStore: storePc,
);
_check('pc merged the phone record', pcAfter.containsKey('b'));
_check('each device wrote only its own revision key',
(await client.getStringMap('_smoketest/revs')).length == 2);
_check(
'each device wrote only its own revision key',
(await client.getStringMap('_smoketest/revs')).length == 2,
);
stdout.writeln('\n== fail-closed ==');
await auth.signOut();
@ -160,7 +177,9 @@ Future<void> main(List<String> args) async {
client.close();
auth.close();
stdout.writeln(
_failures == 0 ? '\nALL LIVE CHECKS PASSED' : '\n$_failures LIVE CHECK(S) FAILED',
_failures == 0
? '\nALL LIVE CHECKS PASSED'
: '\n$_failures LIVE CHECK(S) FAILED',
);
exit(_failures == 0 ? 0 : 1);
}

View File

@ -14,6 +14,8 @@ library;
export 'src/account_store.dart';
export 'src/bootstrap.dart';
export 'src/device_id.dart';
export 'src/e2e_bootstrap.dart';
export 'src/e2e_passphrase_button.dart';
export 'src/google_any_sign_in.dart';
export 'src/google_desktop_sign_in.dart';
export 'src/google_pkce.dart';

View File

@ -108,9 +108,7 @@ Future<FirebaseAccount?> _adoptSeededSession(
}) async {
final httpClient = client ?? http.Client();
try {
final response = await httpClient.get(
base.resolve(routes.credentialsPath),
);
final response = await httpClient.get(base.resolve(routes.credentialsPath));
if (response.statusCode != 200) return null;
final body = jsonDecode(utf8.decode(response.bodyBytes));
if (body is! Map<String, dynamic>) return null;

View File

@ -0,0 +1,77 @@
/// End-to-end encryption for an app's sync prefix, keyed by the shared
/// crdt-sync passphrase kept in the OS keystore (prompt-free after setup).
library;
import 'dart:convert';
import 'package:crdt_sync/crdt_sync.dart';
import 'package:crdt_sync_flutter/src/keystore.dart';
import 'package:flutter_secure_storage/flutter_secure_storage.dart';
/// Keystore key holding the shared crdt-sync passphrase.
const String kE2ePassphraseKey = kE2ePassphraseStorageKey;
/// Stores the passphrase this device derives its key from, dropping keys
/// cached from an earlier one so the new passphrase is what gets checked.
Future<void> saveE2ePassphrase(
String passphrase, {
FlutterSecureStorage storage = kSecureStorage,
}) async {
await _dropCachedKeys(storage);
await storage.write(key: kE2ePassphraseKey, value: passphrase);
}
Future<void> _dropCachedKeys(FlutterSecureStorage storage) async {
final all = await storage.readAll();
for (final key in all.keys) {
if (key.startsWith(kE2eKeyCachePrefix)) await storage.delete(key: key);
}
}
/// Keystore key prefix for derived keys, one per old prefix.
const String kE2eKeyCachePrefix = 'crdt_sync_e2e_key:';
/// Forgets the passphrase and every cached key; the next open is locked.
Future<void> clearE2ePassphrase({
FlutterSecureStorage storage = kSecureStorage,
}) async {
await storage.delete(key: kE2ePassphraseKey);
await _dropCachedKeys(storage);
}
/// The derived key for [oldPrefix], cached in the OS keystore so Argon2id
/// runs once per device instead of on every sync tick.
E2eKeyCache keystoreKeyCache(
String oldPrefix, {
FlutterSecureStorage storage = kSecureStorage,
}) {
final name = '$kE2eKeyCachePrefix$oldPrefix';
return E2eKeyCache(
read: () async {
final text = await storage.read(key: name);
if (text == null) return null;
try {
return base64.decode(text);
} on FormatException {
return null;
}
},
write: (bytes) => storage.write(key: name, value: base64.encode(bytes)),
);
}
/// [openEncryptedStore] reading the passphrase from the OS keystore.
Future<E2eState> openEncryptedStoreFromKeystore({
required RemoteStore store,
required String oldPrefix,
FlutterSecureStorage storage = kSecureStorage,
Argon2Params params = Argon2Params.recommended,
void Function()? onDerive,
}) => openEncryptedStore(
store: store,
oldPrefix: oldPrefix,
readPassphrase: () => storage.read(key: kE2ePassphraseKey),
params: params,
keyCache: keystoreKeyCache(oldPrefix, storage: storage),
onDerive: onDerive,
);

View File

@ -0,0 +1,77 @@
/// A text-free control to enter the shared crdt-sync passphrase: a lock icon
/// opens a hidden field; the check icon (or Enter) saves it to the keystore.
library;
import 'package:crdt_sync_flutter/src/e2e_bootstrap.dart';
import 'package:crdt_sync_flutter/src/keystore.dart';
import 'package:flutter/material.dart';
import 'package:flutter_secure_storage/flutter_secure_storage.dart';
/// Lock icon button; [locked] picks the closed or open lock glyph.
class E2ePassphraseButton extends StatelessWidget {
/// Creates the button. [onSaved] runs after the passphrase is stored.
const E2ePassphraseButton({
required this.locked,
this.onSaved,
this.storage = kSecureStorage,
super.key,
});
/// Whether sync is currently locked (no or wrong passphrase).
final bool locked;
/// Called after a passphrase is saved, e.g. to retry the sync.
final VoidCallback? onSaved;
/// The keystore; only tests pass another.
final FlutterSecureStorage storage;
Future<void> _ask(BuildContext context) async {
final entered = await showDialog<String>(
context: context,
builder: (_) => const _PassphraseDialog(),
);
if (entered == null || entered.isEmpty) return;
await saveE2ePassphrase(entered, storage: storage);
onSaved?.call();
}
@override
Widget build(BuildContext context) => IconButton(
icon: Icon(locked ? Icons.lock : Icons.lock_open),
onPressed: () => _ask(context),
);
}
class _PassphraseDialog extends StatefulWidget {
const _PassphraseDialog();
@override
State<_PassphraseDialog> createState() => _PassphraseDialogState();
}
class _PassphraseDialogState extends State<_PassphraseDialog> {
final _controller = TextEditingController();
@override
void dispose() {
_controller.dispose();
super.dispose();
}
@override
Widget build(BuildContext context) => AlertDialog(
content: TextField(
controller: _controller,
obscureText: true,
autofocus: true,
onSubmitted: (value) => Navigator.of(context).pop(value),
),
actions: [
IconButton(
icon: const Icon(Icons.check),
onPressed: () => Navigator.of(context).pop(_controller.text),
),
],
);
}

View File

@ -61,11 +61,11 @@ packages:
dependency: "direct main"
description:
path: crdt_sync_dart
ref: "crdt_sync_dart-v0.11.3"
resolved-ref: a2e7334dfec5131170a084510d70361d53dc40bc
ref: "crdt_sync_dart-v0.14.1"
resolved-ref: edd0f7a7882cb1d3904932ba94faa282d08165df
url: "https://github.com/kuhyx/utils"
source: git
version: "0.11.3"
version: "0.14.1"
crypto:
dependency: "direct main"
description:
@ -74,6 +74,14 @@ packages:
url: "https://pub.dev"
source: hosted
version: "3.0.8"
cryptography:
dependency: transitive
description:
name: cryptography
sha256: "3eda3029d34ec9095a27a198ac9785630fe525c0eb6a49f3d575272f8e792ef0"
url: "https://pub.dev"
source: hosted
version: "2.9.0"
fake_async:
dependency: transitive
description:

View File

@ -4,7 +4,7 @@ description: >-
persisted per-install device id, and a one-call bootstrap. Removes the
~150 lines of firebase_backend.dart every consuming app was copying.
publish_to: 'none'
version: 0.4.0
version: 0.8.1
environment:
sdk: ^3.12.2
@ -13,7 +13,7 @@ dependencies:
crdt_sync:
git:
url: https://github.com/kuhyx/utils
ref: crdt_sync_dart-v0.11.3
ref: crdt_sync_dart-v0.14.1
path: crdt_sync_dart
crypto: 3.0.8
flutter:

View File

@ -20,9 +20,7 @@ void main() {
group('loadAccount', () {
test('returns the stored account without touching the wrapper', () async {
const stored = FirebaseAccount(email: 'a@b.c', password: 'pw');
installFakeSecureStorage(
initial: {kAccountKey: stored.toJsonString()},
);
installFakeSecureStorage(initial: {kAccountKey: stored.toJsonString()});
var requests = 0;
final account = await loadAccount(
routes: routes,

View File

@ -0,0 +1,174 @@
import 'dart:convert';
import 'package:crdt_sync/crdt_sync.dart';
import 'package:crdt_sync_flutter/crdt_sync_flutter.dart';
import 'package:crdt_sync_flutter/testing/fake_secure_storage.dart';
import 'package:flutter_secure_storage/flutter_secure_storage.dart';
import 'package:flutter_test/flutter_test.dart';
const _old = 'todo-sync/notes';
const _cheap = Argon2Params(memoryKiB: 8, iterations: 1, parallelism: 1);
class _Remote implements RemoteStore {
final files = <String, String>{};
@override
Future<List<String>> listDirectory(String path) async => files.keys
.where((k) => k.startsWith('$path/'))
.map((k) => k.substring(path.length + 1).split('/').first)
.toSet()
.toList();
@override
Future<String?> getFileText(String path) async => files[path];
@override
Future<void> putFileText(
String path,
String text, {
required String message,
}) async => files[path] = text;
@override
Future<void> deleteFile(String path, {String message = ''}) async =>
files.remove(path);
@override
Future<bool> canAccessRemote() async => true;
@override
void close() {}
}
String _log(String title) => jsonEncode({
'n1': {
'id': 'n1',
'fields': {
'title': [title, '2026-01-01T00:00:00.000Z-0000-a'],
},
'deleted': false,
'deleted_hlc': null,
},
});
void main() {
TestWidgetsFlutterBinding.ensureInitialized();
const storage = FlutterSecureStorage();
setUp(installFakeSecureStorage);
test('no passphrase on the device is locked, never plaintext', () async {
final state = await openEncryptedStoreFromKeystore(
store: _Remote(),
oldPrefix: _old,
params: _cheap,
);
expect(state, isA<E2eLocked>());
expect((state as E2eLocked).reason, E2eLockReason.needsPassphrase);
});
test('right passphrase unlocks and copies old plaintext once', () async {
final remote = _Remote()..files['$_old/phone/log.json'] = _log('milk');
await saveE2ePassphrase('pw', storage: storage);
final state = await openEncryptedStoreFromKeystore(
store: remote,
oldPrefix: _old,
params: _cheap,
);
expect(state, isA<E2eReady>());
final layout = E2eLayout.of(_old);
expect(
remote.files['${layout.prefix}/phone/log.json'],
isNot(contains('milk')),
);
expect(remote.files['$_old/phone/log.json'], contains('milk'));
final again = await openEncryptedStoreFromKeystore(
store: remote,
oldPrefix: _old,
params: _cheap,
);
expect(again, isA<E2eReady>());
});
test('second device with the same passphrase reads the data', () async {
final remote = _Remote();
await saveE2ePassphrase('pw', storage: storage);
final first =
await openEncryptedStoreFromKeystore(
store: remote,
oldPrefix: _old,
params: _cheap,
)
as E2eReady;
final layout = E2eLayout.of(_old);
await first.store.putFileText(
'${layout.prefix}/phone/log.json',
_log('tea'),
message: 'm',
);
final second =
await openEncryptedStoreFromKeystore(
store: remote,
oldPrefix: _old,
params: _cheap,
)
as E2eReady;
final text = await second.store.getFileText(
'${layout.prefix}/phone/log.json',
);
expect(text, contains('tea'));
});
test('wrong passphrase is locked and merges nothing', () async {
final remote = _Remote();
await saveE2ePassphrase('pw', storage: storage);
await openEncryptedStoreFromKeystore(
store: remote,
oldPrefix: _old,
params: _cheap,
);
await saveE2ePassphrase('nope', storage: storage);
final state = await openEncryptedStoreFromKeystore(
store: remote,
oldPrefix: _old,
params: _cheap,
);
expect((state as E2eLocked).reason, E2eLockReason.wrongKey);
});
test('clearing the passphrase forgets it', () async {
await saveE2ePassphrase('pw', storage: storage);
await clearE2ePassphrase(storage: storage);
expect(await storage.read(key: kE2ePassphraseKey), isNull);
});
test('the derived key is cached: Argon2 runs once per device', () async {
final remote = _Remote();
await saveE2ePassphrase('pw', storage: storage);
var derived = 0;
for (var i = 0; i < 3; i++) {
resetE2eCopyMemo();
final state = await openEncryptedStoreFromKeystore(
store: remote,
oldPrefix: _old,
params: _cheap,
onDerive: () => derived++,
);
expect(state, isA<E2eReady>());
}
expect(derived, 1);
await clearE2ePassphrase(storage: storage);
final locked = await openEncryptedStoreFromKeystore(
store: remote,
oldPrefix: _old,
params: _cheap,
);
expect(locked, isA<E2eLocked>());
});
test('a corrupt cached key falls back to the passphrase', () async {
final remote = _Remote();
await saveE2ePassphrase('pw', storage: storage);
await storage.write(key: '${kE2eKeyCachePrefix}$_old', value: '%%%');
final state = await openEncryptedStoreFromKeystore(
store: remote,
oldPrefix: _old,
params: _cheap,
);
expect(state, isA<E2eReady>());
});
}

View File

@ -0,0 +1,65 @@
import 'package:crdt_sync_flutter/crdt_sync_flutter.dart';
import 'package:crdt_sync_flutter/testing/fake_secure_storage.dart';
import 'package:flutter/material.dart';
import 'package:flutter_secure_storage/flutter_secure_storage.dart';
import 'package:flutter_test/flutter_test.dart';
void main() {
TestWidgetsFlutterBinding.ensureInitialized();
setUp(installFakeSecureStorage);
Future<void> pump(WidgetTester t, {required bool locked, VoidCallback? cb}) =>
t.pumpWidget(
MaterialApp(
home: Scaffold(
body: E2ePassphraseButton(locked: locked, onSaved: cb),
),
),
);
testWidgets('icon follows the locked flag', (t) async {
await pump(t, locked: true);
expect(find.byIcon(Icons.lock), findsOneWidget);
await pump(t, locked: false);
expect(find.byIcon(Icons.lock_open), findsOneWidget);
});
testWidgets('entering a passphrase stores it and notifies', (t) async {
var saved = 0;
await pump(t, locked: true, cb: () => saved++);
await t.tap(find.byType(IconButton));
await t.pumpAndSettle();
await t.enterText(find.byType(TextField), 'pw');
await t.tap(find.byIcon(Icons.check));
await t.pumpAndSettle();
expect(
await const FlutterSecureStorage().read(key: kE2ePassphraseKey),
'pw',
);
expect(saved, 1);
});
testWidgets('submitting with Enter saves; empty input saves nothing', (
t,
) async {
await pump(t, locked: true);
await t.tap(find.byType(IconButton));
await t.pumpAndSettle();
await t.enterText(find.byType(TextField), 'x1');
await t.testTextInput.receiveAction(TextInputAction.done);
await t.pumpAndSettle();
expect(
await const FlutterSecureStorage().read(key: kE2ePassphraseKey),
'x1',
);
await clearE2ePassphrase();
await t.tap(find.byType(IconButton));
await t.pumpAndSettle();
await t.tap(find.byIcon(Icons.check));
await t.pumpAndSettle();
expect(
await const FlutterSecureStorage().read(key: kE2ePassphraseKey),
isNull,
);
});
}

View File

@ -11,7 +11,7 @@ dependencies:
data_transfer_flutter:
git:
url: https://github.com/kuhyx/utils
ref: data_transfer_flutter-v0.2.0
ref: data_transfer_flutter-v0.2.1
path: data_transfer_flutter
```

View File

@ -4,13 +4,13 @@ description: >-
Flutter apps: system Save-as on Android/iOS (never cloud-only), a save
dialog on desktop, a download on web, plus a versioned export envelope.
publish_to: 'none'
version: 0.2.0
version: 0.2.2
environment:
sdk: ^3.12.2
dependencies:
file_picker: 13.3.1
file_picker: 13.3.3
file_selector: 1.1.0
flutter:
sdk: flutter

View File

@ -91,6 +91,8 @@ NPM_ACCEPT: Final = "application/vnd.npm.install-v1+json"
# --- Cache ------------------------------------------------------------------
CACHE_PATH_ENV: Final = "DEP_FRESHNESS_CACHE"
# Seconds; overrides every ecosystem TTL (a repo that wants a once-a-day gate).
TTL_ENV: Final = "DEP_FRESHNESS_TTL_SECONDS"
DEFAULT_CACHE_DIR: Final = "~/.cache/dep-freshness"
CACHE_FILE: Final = "registry.json"
TTL_SECONDS: Final = 6 * 3600

View File

@ -21,6 +21,7 @@ from dep_freshness._tables import (
DEFAULT_CACHE_DIR,
GITCOMMIT,
GITTAG,
TTL_ENV,
TTL_GITTAG_SECONDS,
TTL_SECONDS,
)
@ -33,7 +34,14 @@ def cache_dir() -> Path:
def ttl_for(ecosystem: str) -> int:
"""Git refs move rarely and cost a subprocess, so they cache longer."""
"""Git refs move rarely and cost a subprocess, so they cache longer.
`DEP_FRESHNESS_TTL_SECONDS` overrides both; garbage is ignored, because a
typo in an env var must not break a commit hook.
"""
override = os.environ.get(TTL_ENV, "")
if override.isdigit() and int(override) > 0:
return int(override)
return TTL_GITTAG_SECONDS if ecosystem in (GITTAG, GITCOMMIT) else TTL_SECONDS

View File

@ -12,6 +12,7 @@ from pathlib import Path
import pytest
from dep_freshness._tables import TTL_SECONDS
from dep_freshness.cache import Cache, Entry, ttl_for
from dep_freshness.models import Dep
from dep_freshness.registries import http
@ -139,3 +140,16 @@ def _offline(*_args):
def _explode(*_args):
raise RuntimeError("boom")
def test_ttl_env_overrides_the_registry_ttl(monkeypatch) -> None:
"""A repo can stretch the refresh window (once-a-day pre-push gates)."""
monkeypatch.setenv("DEP_FRESHNESS_TTL_SECONDS", "86400")
assert ttl_for("npm") == 86400
assert ttl_for("gittag") == 86400
def test_ttl_env_garbage_falls_back_to_the_default(monkeypatch) -> None:
"""A malformed value must never break a commit hook."""
monkeypatch.setenv("DEP_FRESHNESS_TTL_SECONDS", "soon")
assert ttl_for("npm") == TTL_SECONDS

View File

@ -1,8 +1,8 @@
/// Confirmation for destructive actions.
library;
import 'package:flutter/material.dart';
import 'package:flutter/services.dart';
import 'package:material_ui/material_ui.dart';
/// Asks the user to confirm a destructive action, returning true if they do.
///

View File

@ -2,7 +2,7 @@
library;
import 'package:design_system/src/tokens.dart';
import 'package:flutter/material.dart';
import 'package:material_ui/material_ui.dart';
/// A centred icon, headline and explanation for an empty list.
///

View File

@ -8,7 +8,7 @@ library;
import 'package:design_system/src/status_colors.dart';
import 'package:design_system/src/tokens.dart';
import 'package:flutter/material.dart';
import 'package:material_ui/material_ui.dart';
/// How long a neutral confirmation stays up.
const Duration _toastDuration = Duration(seconds: 3);

View File

@ -1,7 +1,7 @@
import 'dart:math' as math;
import 'package:design_system/src/tokens.dart';
import 'package:flutter/material.dart';
import 'package:material_ui/material_ui.dart';
/// A grid of square cells that always fits its box, so it never scrolls
/// vertically.

Some files were not shown because too many files have changed in this diff Show More