{ "intent": "Replace blind coordinate tapping with element-targeted Android automation, shared across every repo with an Android surface, so an agent driving a phone cannot mistake a mis-tap or an unlanded keystroke for success.", "scope": [ "python_pkg/android_ui/ (driver, cli, __main__, tests, README)", "meta/requirements.txt + .pre-commit-config.yaml: defusedxml, needed for XML parsing under ruff S314 and on pylint's isolated path", "Non-goal: MCP wrapper, scroll_to(), and rollout to ~/todo and ~/dufs-cloud/app" ], "changes": [ "AndroidUi driver over adb + uiautomator dump: find/find_all/wait_for/tap/type_into/type_into_field/editable_fields/dismiss_keyboard/keyboard_is_up/current_focus, every action re-resolving the element immediately before acting", "Ambiguity is an error: a query matching 0 or >1 elements raises instead of acting on the first match", "Typing is verified (re-reads the field, raises if unchanged) and clears first, so a filled field is replaced rather than appended to", "keyboard_is_up() via dumpsys input_method, so 'element absent' and 'element behind the keyboard' are distinguishable; tap dismisses first; dismiss_keyboard uses ESCAPE then BACK and raises if the keyboard survives", "Partial dumps retried with backoff; empty EditText nodes kept despite having no label at all", "Fixed a real _escape bug found by its own test: backslash was inside the escape loop, so '&' became '\\\\&' and a password containing metacharacters would have been typed wrong" ], "verification": [ { "command": "python3 -m pytest python_pkg/android_ui/ --cov=python_pkg.android_ui", "result": "pass", "evidence": "57 passed; __init__ 100%, __main__ 100%, cli.py 100%, driver.py 100% (200 stmts, 46 branches, 0 missed)" }, { "command": "bash ~/.claude/scripts/finish.sh gate /home/kuhy/testsAndMisc python_pkg/android_ui meta/requirements.txt .pre-commit-config.yaml", "result": "pass", "evidence": "1436 passed in 21.14s; Required test coverage of 100.0% reached; FINISH_GATE phase:ok" }, { "command": "ruff check python_pkg/android_ui/", "result": "pass", "evidence": "All checks passed! Zero noqa/type:ignore suppressions; installed python-defusedxml rather than suppressing S314; a test type:ignore was removed by returning a Harness NamedTuple instead of bolting an attribute onto the driver" }, { "command": "PYTHONPATH=~/testsAndMisc python3 -m python_pkg.android_ui find 'Start Workout' # physical Pixel 6a", "result": "pass", "evidence": "exit 1 with 'no element matches ... On screen now: 192.0.0.4:8765, Done for today! ...' — correct, a workout was already logged. Also exercised on-device: dump, --exact tap 'Back' (refused ambiguous 'Back' vs 'OFFLINE BACKUP'), type_into_field on both sync credential fields, dismiss_keyboard without popping the route" } ], "risks": [ "A widget with no accessibility label cannot be targeted at all — workout_app's Settings gear is one. The tool surfaces the gap rather than hiding it; the fix is a Semantics(label:) in the app.", "Scrolling is not modelled, so a widget outside the viewport is absent from the tree and reads as 'not found'. Callers must scroll first until scroll_to() exists.", "The partial-tree heuristic (>4 labelled nodes) could in principle mis-classify a genuinely tiny screen as partial; it only costs retries, never a wrong action." ], "rollback": [ "git revert the commit; the package is additive and nothing else imports it yet", "Drop defusedxml from meta/requirements.txt and .pre-commit-config.yaml's pylint additional_dependencies if no other consumer appears", "After rollback, confirm `python3 -m pytest python_pkg/` still reports 100% coverage" ] }