mirror of
https://github.com/kuhyx/testsAndMisc.git
synced 2026-11-10 20:07:04 +01:00
Krzysztof kuhy Rudnicki
325d5a8baa
RunnerUp can now upload to Endurain directly as well as through the WebDAV inbox, and the two paths cannot see each other: the same run landed twice (activities 14 and 15, identical start time, distances differing only by tcx/gpx rounding). The ledger only knows what this importer handled, so it could not catch it. Add a third dedupe check that asks Endurain what it already holds, queried once per run. Endurain exposes no query-by-start-time endpoint, so a page of recent activities is fetched and compared client-side, with a +/-120s tolerance: the file for activity 15 embeds 21:51:04Z while the server stored 21:51:05Z, so exact equality misses the very duplicate that motivated this. A failed or unauthorised query returns None, never an empty list, so it can never be read as "nothing matches" and upload on the strength of it. Export filenames are parsed as LOCAL wall-clock time. RunnerUp_2026-08-14- 00-58-45_Running is 2026-08-13 22:58:45 UTC; reading it as UTC would move a late-evening run onto the wrong day and defeat the comparison. Also fix the adb fallback, which broke the moment a second device attached: phone_attached() returned true if ANY device was in the 'device' state, then every bare adb call failed with "more than one device/emulator" as a warning on a path nobody watches. The serial is now resolved once and threaded into every call as -s. Ambiguity is settled by configuration, never by guessing -- several devices and no ENDURAIN_ADB_SERIAL means the fallback is skipped. The remote check stays inert until the API key is granted activities:read; until then it falls back to the ledger, which is the previous behaviour. 85 tests, 100% branch coverage on all four modules, no suppressions. The oversized test modules are split by concern to stay under the 250-line cap. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GbYEnfGnf1988cdFdGebot
72 lines
7.1 KiB
JSON
72 lines
7.1 KiB
JSON
{
|
|
"intent": "Stop the endurain importer duplicating runs that RunnerUp uploaded to Endurain directly, and stop its adb fallback silently failing whenever more than one device is attached.",
|
|
"scope": [
|
|
"python_pkg/endurain_import/sources.py — adb device selection",
|
|
"python_pkg/endurain_import/ledger.py — start time parsed from the export filename",
|
|
"python_pkg/endurain_import/upload.py — recent-activity query and start-time comparison",
|
|
"python_pkg/endurain_import/__main__.py — orchestration wiring",
|
|
"Non-goal: deleting duplicates that already exist. This prevents new ones; it never deletes.",
|
|
"Non-goal: changing the WebDAV inbox or the phone-side files (screen-locker reads the same directory)."
|
|
],
|
|
"changes": [
|
|
"sources.py: replaced phone_attached() with attached_serials() + resolve_serial(). The old check returned true when ANY device was in the 'device' state, after which every bare adb call failed with 'more than one device/emulator' as a warning on a path nobody watches. The serial is now threaded into every _adb() call as -s. Ambiguity is resolved by configuration (ENDURAIN_ADB_SERIAL), never by guessing: several devices and no pin means the fallback is skipped outright.",
|
|
"ledger.py: added activity_start(), which parses the run's start time out of the RunnerUp export filename. RunnerUp names exports in LOCAL wall-clock time, so the value is read in the local zone and returned as UTC; reading it as UTC directly shifts a late-evening run onto the wrong day (RunnerUp_2026-08-14-00-58-45 is 2026-08-13 22:58 UTC).",
|
|
"upload.py: added EndurainClient.recent_start_times(), which fetches one page of recent activities and returns their start times, plus already_present() which compares with a +/-120s tolerance. Endurain exposes no query-by-start-time endpoint, so existence is decided client-side. The tolerance exists because the recorded value is not the one in the file: a run whose export embeds 21:51:04Z was stored by Endurain as 21:51:05Z, which exact equality would miss.",
|
|
"upload.py: recent_start_times() returns None (not an empty list) for a transport failure, an auth failure, or an unparseable body, so a failed lookup can never be read as 'no matching activity exists'.",
|
|
"__main__.py: _process() gained a third dedupe check against remote_starts, queried once per run rather than per file. A None result falls back to the existing digest/activity-key ledger checks, which is the previous behaviour. A run matched remotely is recorded in the ledger so the next run skips it without needing the query."
|
|
],
|
|
"verification": [
|
|
{
|
|
"command": "systemctl --user start endurain-import.service && journalctl --user -u endurain-import.service -n 5",
|
|
"result": "pass",
|
|
"evidence": "Ran against the live service with two devices genuinely attached (Pixel 6a 23181JEGR08034 over USB, BL9000 over 192.168.1.131:5555). Logged: '2 devices attached (23181JEGR08034, 192.168.1.131:5555); set ENDURAIN_ADB_SERIAL to choose one. Skipping the phone fallback rather than pulling from an arbitrary device.' Before this change the same situation produced 'could not list /sdcard/Documents/RunnerUp: adb: more than one device/emulator'."
|
|
},
|
|
{
|
|
"command": "python -c 'from python_pkg.endurain_import import sources; sources.resolve_serial()' under three real device configurations",
|
|
"result": "pass",
|
|
"evidence": "Two devices attached, no env: None + warning naming both serials. ENDURAIN_ADB_SERIAL=23181JEGR08034: returned '23181JEGR08034'. ENDURAIN_ADB_SERIAL=NOPE123: None + warning that the pinned device is not attached."
|
|
},
|
|
{
|
|
"command": "python -c 'from python_pkg.endurain_import.ledger import activity_start' on real export filenames",
|
|
"result": "pass",
|
|
"evidence": "RunnerUp_2026-08-22-23-51-04_Running.tcx -> 2026-08-22 21:51:04+00:00 (DB holds 21:51:05, inside tolerance). RunnerUp_Pixel_6a_2026-08-22-23-51-04_Running.gpx -> identical instant, confirming the device model is stripped. RunnerUp_2026-08-14-00-58-45_Running.tcx -> 2026-08-13 22:58:45+00:00, the day-boundary case. garbage.tcx and RunnerUp_2026-13-45-99-99-99_Running.tcx -> None."
|
|
},
|
|
{
|
|
"command": "python -c 'from python_pkg.endurain_import.upload import already_present' at the tolerance boundary",
|
|
"result": "pass",
|
|
"evidence": "1s skew (the real 14/15 collision) -> True. 120s -> True. 121s -> False. Unrelated run -> False. Empty known list -> False."
|
|
},
|
|
{
|
|
"command": "python -c 'EndurainClient(...).recent_start_times(1)' against the running server with both API keys",
|
|
"result": "pass",
|
|
"evidence": "Both keys returned None and logged 'could not list recent activities: 401 (the API key needs the activities:read scope for duplicate detection)'. Confirmed against the DB: both keys hold scopes [\"activities:upload\"] only. The fail-closed path is therefore exercised end to end; the positive path is covered by unit tests until the scope is granted."
|
|
},
|
|
{
|
|
"command": ".venv/bin/python -m pytest python_pkg/endurain_import/tests/ -q",
|
|
"result": "pass",
|
|
"evidence": "85 passed."
|
|
},
|
|
{
|
|
"command": ".venv/bin/python -m pytest python_pkg/endurain_import/tests/ --cov=python_pkg.endurain_import --cov-report=term-missing",
|
|
"result": "pass",
|
|
"evidence": "__init__.py, __main__.py (92 stmts, 20 branches), ledger.py (74/12), sources.py (77/30), upload.py (93/20) all at 100% with zero missed branches. No lint suppressions added."
|
|
},
|
|
{
|
|
"command": ".venv/bin/python -m ruff check python_pkg/endurain_import/",
|
|
"result": "pass",
|
|
"evidence": "All checks passed."
|
|
}
|
|
],
|
|
"risks": [
|
|
"The remote check is inert until the Endurain API key is granted the activities:read scope. Until then recent_start_times() returns None every run and dedupe falls back to the ledger, which is exactly the previous behaviour — no regression, but no new protection either.",
|
|
"A pre-existing duplicate (activities 14 and 15 share a start time) makes Endurain's own get_activity_by_start_time raise MultipleResultsFound, which 500s an upload of THAT run. Other runs are unaffected. This change does not delete duplicates; the collision has to be cleared in the UI.",
|
|
"The +/-120s tolerance would collapse two genuinely distinct runs starting within two minutes of each other into one. Judged not a real scenario for a running log; the alternative (exact match) demonstrably misses the real 1s recording skew.",
|
|
"resolve_serial() now refuses to pull when several devices are attached and no serial is pinned. That is a deliberate behaviour change: the fallback stops silently rather than pulling from an arbitrary device. Set ENDURAIN_ADB_SERIAL to restore pulling."
|
|
],
|
|
"rollback": [
|
|
"git revert the commit; the importer returns to phone_attached() and ledger-only dedupe.",
|
|
"After rollback confirm: systemctl --user start endurain-import.service exits 0, and the ledger at ~/.local/state/endurain-import/ledger.json is unchanged (this change never rewrites existing entries).",
|
|
"No data migration is involved — the ledger format is unchanged and no server state is written by the new code path (it only issues GETs)."
|
|
]
|
|
}
|