mirror of
https://github.com/kuhyx/testsAndMisc.git
synced 2026-11-10 20:07:04 +01:00
Krzysztof kuhy Rudnicki
b73811c233
Shared across every repo with an Android surface (screen-locker's workout_app, todo, dufs-cloud/app). Works against a plain release APK on a physical device over adb — no root, no emulator, no Flutter debug connection, no app changes. Driving an app by tapping pixel positions read off a screenshot fails in ways that look like success. All observed in one session (2026-08-10) verifying workout_app's Firebase restore: * A tap aimed at a password field landed with the field unfocused. The typed password went nowhere and NOTHING errored — the next screenshot just showed an empty box. type_into/type_into_field now re-read the field and raise. * Opening the soft keyboard moved a field from y=1572 to y=1319, so a coordinate captured one step earlier was already wrong. Every action re-resolves the element immediately before acting. * A widget behind the keyboard reports its LAID-OUT position (1549) not its real one (1939) — a 390px error that taps a letter key and reports success. keyboard_is_up() distinguishes "absent" from "behind the keyboard", and tap closes the keyboard first. * KEYCODE_BACK is a route pop in Flutter, not a keyboard dismiss: it navigated out of Settings and discarded typed credentials, twice. dismiss_keyboard tries ESCAPE then BACK and verifies via dumpsys input_method. * `input text` inserts at the cursor, so typing over a filled field produced "canary@example.com321krzychu@gmail.com". Fields are cleared first. * uiautomator returns ONLY the focused EditText nodes while a Flutter text field has focus, and stays that way across retries — so a naive retry loop never converges. Partial trees are retried and the best result kept. * An EMPTY EditText has no text, content-desc or resource-id, so a "must be labelled" filter drops the very field you need to fill in. Editable nodes are kept regardless and addressable by position. Ambiguity is an error rather than a silent first-match: `tap "Back"` correctly refuses when "OFFLINE BACKUP" is also on screen. Its own test suite caught a real bug in _escape: backslash sat inside the escape loop, so "&" became "\\&" and a password containing shell metacharacters would have been typed wrong. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JupKvZCNKRmEeRQY3G1unG
21 lines
1.8 KiB
JSON
21 lines
1.8 KiB
JSON
{
|
|
"title": "android_ui: element-targeted Android automation shared across repos",
|
|
"objective": "Give agents and scripts a way to drive a real Android device by element identity (text, content-desc, resource-id) instead of pixel coordinates read off screenshots. Success means a flow can be driven with zero hardcoded coordinates, every failure exits non-zero naming the query, and a mis-tap or unlanded keystroke is impossible to mistake for success.",
|
|
"acceptance_criteria": [
|
|
"find/tap/wait resolve elements by identity and re-read the tree immediately before acting, never reusing a stale coordinate",
|
|
"A query matching zero or more than one element raises rather than acting on the first match",
|
|
"type_into/type_into_field verify the field's contents actually changed, and clear a filled field so typing replaces rather than appends",
|
|
"keyboard_is_up() distinguishes 'element absent' from 'element behind the soft keyboard'; tap closes the keyboard first",
|
|
"dismiss_keyboard closes the IME without popping the Flutter route, and raises if the keyboard stays up",
|
|
"Partial uiautomator dumps (only the focused EditText nodes) are retried, and empty EditText nodes remain addressable",
|
|
"100% line and branch coverage, no noqa/type:ignore suppressions anywhere"
|
|
],
|
|
"out_of_scope": [
|
|
"MCP server wrapper (the CLI and library land first)",
|
|
"scroll_to(): a widget outside the viewport is not in the tree yet",
|
|
"Rolling the tool out to ~/todo and ~/dufs-cloud/app",
|
|
"flutter drive / integration_test backend, which cannot drive a release APK"
|
|
],
|
|
"verifier": "pre-commit run --files <changed>; python3 -m pytest python_pkg/android_ui/ (57 tests, 100% coverage); plus manual exercise against the physical Pixel 6a driving workout_app's Settings/connect flow"
|
|
}
|