From ed9fb48c511afea4672f670ae1b25db8f7c924fd Mon Sep 17 00:00:00 2001 From: Krzysztof kuhy Rudnicki Date: Mon, 2 Nov 2026 11:29:12 +0100 Subject: [PATCH] Move TODOs to todo app Co-Authored-By: Claude Opus 5.5 (1M context) --- TODO-design-audit.md => DOCS-design-audit.md | 2 - .../plans/TODO-phone-focus-recovery.md | 26 ---- .../android_ui/DOCS-android-ui-rationale.md | 41 ++++++ .../android_ui/TODO-android-ui-mcp-wrapper.md | 126 ------------------ 4 files changed, 41 insertions(+), 154 deletions(-) rename TODO-design-audit.md => DOCS-design-audit.md (99%) delete mode 100644 docs/superpowers/plans/TODO-phone-focus-recovery.md create mode 100644 python_pkg/android_ui/DOCS-android-ui-rationale.md delete mode 100644 python_pkg/android_ui/TODO-android-ui-mcp-wrapper.md diff --git a/TODO-design-audit.md b/DOCS-design-audit.md similarity index 99% rename from TODO-design-audit.md rename to DOCS-design-audit.md index 775799a3..9bfe8853 100644 --- a/TODO-design-audit.md +++ b/DOCS-design-audit.md @@ -128,5 +128,3 @@ N/A below with spot-checks noted instead. - **Primary structural finding**: introduce a shared token source (CSS custom properties in a `src/theme.css` imported by `index.css`, or a TS `theme.ts`) before doing any rule-by-rule fix — otherwise each violation below has to be independently re-fixed per `*.module.css` file, and new files will keep reintroducing the same drift (as the reversed-gradient-stop-order bug on `PuzzleResult.module.css:25` already shows). - Once tokens exist, Rules 9 and 18 become checkable as whole-palette rules and should be re-audited. - The 4-gray inconsistency (Rule 2) and the duplicated-gradient literals (Rule 4) are the same underlying fix — collapsing to a `theme.css` resolves both simultaneously. - -REMOVE ME AFTER FINISH diff --git a/docs/superpowers/plans/TODO-phone-focus-recovery.md b/docs/superpowers/plans/TODO-phone-focus-recovery.md deleted file mode 100644 index 2b89ddd6..00000000 --- a/docs/superpowers/plans/TODO-phone-focus-recovery.md +++ /dev/null @@ -1,26 +0,0 @@ -# Live-verify phone recovery workflow (Chunk 8 of the original plan) - -## what -Chunks 1-7 of the plan are built and now live in `~/src/phone-focus-mode` -(`run_phone.sh`, `lib/{adb_common,monitor,backup,restore}.sh`, -`backup_manifest.sh`, `systemd/phone-auto-sync.*`, README). The visible -wrapper moved: the entry point is `~/src/phone-focus-mode/run_phone.sh`. What -is left is the one-off run against the real, rooted phone. - -## done -With the phone connected over ADB, from `~/src/phone-focus-mode`: -`./run_phone.sh --help`, `monitor`, `backup` and `auto` all exit 0, `backup` -writes under `~/phone_backups/`, and a temporary extra entry in -`FORMAT_INDICATORS` (`backup_manifest.sh`, reverted afterwards) makes `auto` -print the formatted-phone warning and stop without doing anything else. - -## verify -Needs kuhy: phone attached (`adb devices` lists it, none was attached on -2026-11-02). Library tests already pass: `lib/tests/test_adb_common.sh` -(10 passed) and `lib/tests/test_monitor.sh` (82 passed). - -## read first -- `~/src/phone-focus-mode/README.md` -- `~/src/phone-focus-mode/CLAUDE.md` - -REMOVE ME AFTER FINISH diff --git a/python_pkg/android_ui/DOCS-android-ui-rationale.md b/python_pkg/android_ui/DOCS-android-ui-rationale.md new file mode 100644 index 00000000..1789576d --- /dev/null +++ b/python_pkg/android_ui/DOCS-android-ui-rationale.md @@ -0,0 +1,41 @@ +# Android UI automation: rationale and Flutter pitfalls + +## why this is not a nicety + +Observed in one session (2026-08-10, workout_app Firebase verification): + +- **Silent mis-taps.** A tap at `(540, 1705)` intended for the password field + landed with the field unfocused. The typed password went nowhere. Nothing + errored — the next screenshot just showed an empty box. A no-op and a hit are + indistinguishable without dumping the tree and diffing. +- **Layout shift invalidates every coordinate.** Opening the soft keyboard moved + the email field from y=1572 to y=1319. Coordinates captured one step earlier + were already wrong. +- **Screenshot scaling is a manual multiply.** The harness renders 1080x2400 at + 900x2000, so every read coordinate needs `* 1.20`. One forgotten multiply is + an off-screen tap. +- **Screenshots are expensive.** A full-page PNG per step burns far more context + than the equivalent element list, and can't be grepped or asserted on. + +`uiautomator dump` already solves this and is on every device — no new +dependency: + +```bash +adb shell uiautomator dump /sdcard/ui.xml +adb pull /sdcard/ui.xml +# -> +``` + +This is how the same session recovered: dumping the tree found both `EditText` +nodes by class and gave exact centers, and the retry worked first time. + +- **Observed 2026-08-10, must be handled:** while a Flutter `TextField` holds + focus, the dump returns ONLY the `EditText` nodes — the rest of the tree + (including the button you are about to tap) is missing, and it stays missing + across retries, so a naive retry loop never converges. Worse, `KEYCODE_BACK` + to dismiss the keyboard is treated by Flutter as a route pop: it navigated + out of Settings and discarded the typed credentials, twice. The tool must + unfocus without popping the route (tap dead space, or `dumpsys input_method` + to check `mInputShown` first) and must distinguish "tree is partial" from + "element is genuinely absent" rather than reporting a false negative. diff --git a/python_pkg/android_ui/TODO-android-ui-mcp-wrapper.md b/python_pkg/android_ui/TODO-android-ui-mcp-wrapper.md deleted file mode 100644 index db9ab4f8..00000000 --- a/python_pkg/android_ui/TODO-android-ui-mcp-wrapper.md +++ /dev/null @@ -1,126 +0,0 @@ -# Task: replace blind coordinate tapping with element-targeted Android UI automation - -> **STATUS 2026-11-02: PARTIAL.** The CLI (`dump`/`find`/`tap`/`type`/`wait`/ -> `scroll-to`/`focus`) exists at `~/src/testsAndMisc/python_pkg/android_ui/` (see -> its README). Remaining: (1) the optional MCP wrapper over the same driver; -> (2) rolling it out to `~/src/todo` and `~/src/dufs-cloud/app` (other repos, -> needs their own sessions); (3) the `## done` run on the physical Pixel 6a: -> needs kuhy, phone not connected. Everything after this line is the original -> brief, kept because it records WHY each guarantee exists. - -## what - -Every Flutter/Android app in these repos is currently driven by an agent the -same bad way: screenshot -> read pixel positions by eye -> `adb shell input tap - `. Replace that with a reusable tool that finds elements by _identity_ -(text, content-desc, resource-id) and taps their real bounds, and that can -report what is on screen as structured text instead of an image. - -## why this is not a nicety - -Observed in one session (2026-08-10, workout_app Firebase verification): - -- **Silent mis-taps.** A tap at `(540, 1705)` intended for the password field - landed with the field unfocused. The typed password went nowhere. Nothing - errored — the next screenshot just showed an empty box. A no-op and a hit are - indistinguishable without dumping the tree and diffing. -- **Layout shift invalidates every coordinate.** Opening the soft keyboard moved - the email field from y=1572 to y=1319. Coordinates captured one step earlier - were already wrong. -- **Screenshot scaling is a manual multiply.** The harness renders 1080x2400 at - 900x2000, so every read coordinate needs `* 1.20`. One forgotten multiply is - an off-screen tap. -- **Screenshots are expensive.** A full-page PNG per step burns far more context - than the equivalent element list, and can't be grepped or asserted on. - -`uiautomator dump` already solves this and is on every device — no new -dependency: - -```bash -adb shell uiautomator dump /sdcard/ui.xml -adb pull /sdcard/ui.xml -# -> -``` - -This is how the same session recovered: dumping the tree found both `EditText` -nodes by class and gave exact centers, and the retry worked first time. - -## where - -New shared tool, not a per-repo copy. Two options, pick one: - -- `~/src/testsAndMisc/python_pkg/android_ui/` — a CLI (`android-ui find "Connect -Firebase"`, `android-ui tap "Connect Firebase"`, `android-ui dump`), matching - how `app_icons` is already shared across repos. -- An MCP server exposing the same as tools, which is what makes it available to - the agent without shelling out. Preferred if MCP is acceptable — the user - asked for "some tool ... (mcp?) which also translates to UI". - -Consumers (every repo with an Android surface): - -- `~/src/screen-locker/stronglift_replacement/workout_app` -- `~/src/todo` -- `~/src/dufs-cloud/app` -- any future Flutter app (this should be part of the app scaffold) - -## must - -- `find(query)` -> matching elements with class, text/content-desc, bounds, - center, and enabled/focused state. Match on text, content-desc, or - resource-id; substring and exact. -- `tap(query)` -> resolves the query, taps the center of the _current_ bounds - (re-dump before every tap; never reuse a stale coordinate), and **fails - loudly with a non-zero exit when the query matches zero or >1 elements**. - Ambiguity must be an error, not a silent first-match. -- `type(query, text)` -> focus the field, type, then **verify the field's text - actually changed** and fail if not. This is the specific bug that cost this - session a round trip. -- `dump()` -> flat, greppable text list of visible elements. This replaces the - screenshot for most steps. -- Waits: `wait_for(query, timeout)` that polls the tree, so no more blind - `sleep 14`. -- must not: require root, an emulator, a running Flutter debug connection, or - any change to app source. It must work against a plain release APK on a - physical device over adb. -- optional: `flutter drive` / integration_test as a second backend when a debug - build IS available — richer (widget keys, no dump latency), but it cannot - drive a release APK, so it can't be the only path. -- optional: screenshot fallback for genuinely visual checks (theming, icon - rendering) — where a picture is the point, not a workaround. - -## done - -A verification script that drives workout_app's "connect Firebase" flow -end-to-end with **zero hardcoded pixel coordinates** in it, and: - -- passes on the current layout; -- still passes after the SYNC and OFFLINE BACKUP sections are reordered in - `settings_screen.dart` (the test that coordinate-based automation fails); -- exits non-zero, naming the query, when an element is missing. - -## verify - -On the physical Pixel 6a over adb, against an installed release APK — not an -emulator and not a debug-only Flutter driver session. - -## read first - -- This session's working recovery: `uiautomator dump` + the bounds-parsing - python in the screen-locker scratchpad (`ui.xml`, `ui2.xml`). -- `~/src/screen-locker/CLAUDE.md` — "You are NOT done until you install the new - version on the phone itself", which is why agent-driven UI is on the critical - path for every task in this repo. -- Note `uiautomator dump` returns an empty/partial tree while an animation is - in flight; retry-with-backoff belongs in the tool, not in each caller. -- **Observed 2026-08-10, must be handled:** while a Flutter `TextField` holds - focus, the dump returns ONLY the `EditText` nodes — the rest of the tree - (including the button you are about to tap) is missing, and it stays missing - across retries, so a naive retry loop never converges. Worse, `KEYCODE_BACK` - to dismiss the keyboard is treated by Flutter as a route pop: it navigated - out of Settings and discarded the typed credentials, twice. The tool must - unfocus without popping the route (tap dead space, or `dumpsys input_method` - to check `mInputShown` first) and must distinguish "tree is partial" from - "element is genuinely absent" rather than reporting a false negative. - -REMOVE ME AFTER FINISH