app_errors 0.1.0: shared error ingest + timer

RTDB and local spool reports merged by signature into the board's
app-error contract; resolve command; systemd --user timer via deploy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Krzysztof Rudnicki 2026-11-17 21:34:10 +01:00
parent da1d7c0692
commit 62e7ff801e
22 changed files with 1662 additions and 1 deletions

34
.github/workflows/app-errors-tests.yml vendored Normal file
View File

@ -0,0 +1,34 @@
name: app_errors tests
on:
push:
branches: [main]
# The workflow's own file counts as app_errors: editing only this file would
# otherwise trigger nothing, so a fix to the job itself could never prove
# itself.
paths: ['app_errors/**', '.github/workflows/app-errors-tests.yml']
pull_request:
branches: [main]
paths: ['app_errors/**', '.github/workflows/app-errors-tests.yml']
jobs:
test:
runs-on: ubuntu-latest
defaults:
run:
working-directory: app_errors
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.x"
cache: pip
cache-dependency-path: app_errors/requirements.txt
- name: Install dependencies
run: pip install -r requirements.txt && pip install --no-deps -e .
- name: Run tests with coverage
run: python -m pytest -v

52
app_errors/README.md Normal file
View File

@ -0,0 +1,52 @@
# app_errors
The PC side of kuhy's app error and slow-wait reporting. Every app reports
errors, and every wait the user sees over 1 s (`perf.slow`). This ingest merges
the reports by signature and writes `~/.local/state/app-errors/<app>.json` in
the todoImplementer `contracts/app-error.schema.json` `report` shape. The board
reads that folder and turns each open error into a self-fixing ticket
(todoImplementer `DOCS-errors.md`).
Generalised from kartka's `tools/errors`. Dart signatures are byte-identical to
kartka's. See `DOCS-app-errors.md` at the repo root for the whole pipeline.
## Where reports come from
| Reporter | Runs on | Writes |
|---|---|---|
| `crdt_sync_flutter` `installErrorReporting` | phone, desktop, web | RTDB `/<app>/errors/<device>/<id>` |
| `gatelock.error_report.install` | PC (Tk) | spool `~/.local/state/app-errors-spool/<app>.jsonl` |
| Godot `addons/app_errors` (`godot_dev_tools/`) | PC | the same spool |
The ingest skips `kartka` in RTDB, because kartka still runs its own ingest.
## Entry (one report, every language)
`at` (ISO UTC), `kind` (`uncaught.flutter`, `uncaught.python`, `uncaught.gdscript`,
`ui.snackbar`, `perf.slow`, ...), `error`, `stack` (list of frames), plus these
optional fields:
- `repo` (`owner/name`, default `kuhyx/<app>`)
- `mark` (the substring that marks the app's own frames: `package:todo/`,
`leetcode_guard/`, `res://`)
- `route`
- `operation`, `durationMs`, `waitId` and `still` (slow waits)
- `build` (versionCode, 0 when unknown) and `version`
## Commands
```bash
app-errors ingest # what the timer runs every 5 min
app-errors ingest --keep # leave RTDB entries in place
app-errors ingest --no-rtdb # local spool only
app-errors resolve <app> <signature> # mark it fixed; a newer build reopens it
```
`app_errors/install.sh` installs the venv
(`~/.local/share/app-errors/venv`), the `app-errors` link and
`app-errors-ingest.timer`. `scripts/deploy.sh` runs it.
RTDB access reuses an already seeded crdt-sync session (`--session`, default
`firebase_backup`), so the ingest needs no sign-in of its own.
Tests: `cd app_errors && python -m pytest` (100 % branch coverage).

View File

@ -0,0 +1,2 @@
# Copyright (c) 2026 Krzysztof Rudnicki
"""Shared app error and slow-wait ingest: RTDB and spool reports in, board tickets out."""

View File

@ -0,0 +1,66 @@
# Copyright (c) 2026 Krzysztof Rudnicki
"""``app-errors``: ``ingest`` (what the timer runs) and ``resolve <app> <signature>``."""
from __future__ import annotations
import argparse
import datetime as dt
import sys
from typing import TYPE_CHECKING
from app_errors import ingest, store
from app_errors.rtdb import DEFAULT_SESSION, RtdbError, open_rtdb
if TYPE_CHECKING:
from collections.abc import Sequence
def _parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(prog="app-errors", description=__doc__)
sub = parser.add_subparsers(dest="command", required=True)
run = sub.add_parser("ingest", help="read RTDB and the spool, write the board reports")
run.add_argument("--keep", action="store_true", help="leave RTDB entries in place")
run.add_argument("--no-rtdb", action="store_true", help="read the local spool only")
run.add_argument("--session", default=DEFAULT_SESSION, help="seeded crdt-sync credential store")
res = sub.add_parser("resolve", help="mark an error fixed (a newer build reopens it)")
res.add_argument("app")
res.add_argument("signature")
return parser
def _ingest(args: argparse.Namespace) -> int:
db, error = None, None
if not args.no_rtdb:
try:
db = open_rtdb(args.session)
except RtdbError as e:
error = str(e)
summary = ingest.run(ingest.default_paths(), db, keep=args.keep)
error = error or summary.rtdb_error
print(f"app-errors: {summary.merged} new") # noqa: T201 -- terminal-only CLI
if error:
print(f"app-errors: RTDB: {error}", file=sys.stderr) # noqa: T201
return 1
return 0
def _resolve(args: argparse.Namespace) -> int:
paths = ingest.default_paths()
state = store.load(paths.state)
now = dt.datetime.now(dt.UTC).strftime("%Y-%m-%dT%H:%M:%S.%fZ")
if not store.resolve(state, args.app, args.signature, now=now):
print(f"app-errors: unknown error {args.app} {args.signature}", file=sys.stderr) # noqa: T201
return 1
store.save(paths.state, state)
store.write_reports(paths.reports, state)
return 0
def main(argv: Sequence[str] | None = None) -> int:
"""Run the CLI; return the exit code."""
args = _parser().parse_args(argv)
return _ingest(args) if args.command == "ingest" else _resolve(args)
if __name__ == "__main__": # pragma: no cover
sys.exit(main())

View File

@ -0,0 +1,96 @@
# Copyright (c) 2026 Krzysztof Rudnicki
"""One ingest pass: RTDB and the local spool in, the board's reports out.
In: RTDB ``/<app>/errors/<device>/<id>`` and the spool. Out:
``<state>/app-errors/<app>.json``, the board's contract report.
RTDB entries are deleted once the state holding them is saved (``keep``
leaves them; ``seen`` stops a second count). kartka is skipped: it still runs
its own ingest (``~/src/kartka/tools/errors``), which the board reads through
its ``kartka`` adapter, and two readers of ``/kartka/errors`` would race.
"""
from __future__ import annotations
from dataclasses import dataclass
import os
from pathlib import Path
from typing import TYPE_CHECKING
from app_errors import spool, store
from app_errors.rtdb import RtdbError
if TYPE_CHECKING:
from app_errors.rtdb import Rtdb
#: Apps with their own ingest, never read here.
SKIP_APPS = frozenset({"kartka"})
@dataclass(frozen=True)
class Paths:
"""Where the ingest keeps its state, writes reports and reads the spool."""
state: Path
reports: Path
spool: Path
@dataclass(frozen=True)
class Summary:
"""What one pass did."""
merged: int
rtdb_error: str | None
def default_paths() -> Paths:
"""Return the paths under ``$XDG_STATE_HOME`` (default ``~/.local/state``)."""
base = Path(os.environ.get("XDG_STATE_HOME") or Path.home() / ".local" / "state")
return Paths(
base / "app-errors-ingest" / "state.json",
base / "app-errors",
base / "app-errors-spool",
)
def _rtdb_entries(
db: Rtdb,
) -> tuple[list[tuple[str, str, object]], dict[str, dict[str, object]]]:
"""Return (app, device/id, entry) for every RTDB report, and the deletes that consume them."""
found: list[tuple[str, str, object]] = []
deletes: dict[str, dict[str, object]] = {}
top = db.get("", shallow=True)
for app in sorted(top) if isinstance(top, dict) else []:
if app in SKIP_APPS or not store.APP_NAME.match(app):
continue
devices = db.get(f"{app}/errors")
for device, entries in devices.items() if isinstance(devices, dict) else []:
for entry_id, entry in entries.items() if isinstance(entries, dict) else []:
found.append((app, f"{device}/{entry_id}", entry))
deletes.setdefault(app, {})[f"{device}/{entry_id}"] = None
return found, deletes
def run(paths: Paths, db: Rtdb | None, *, keep: bool = False) -> Summary:
"""Ingest everything once; RTDB failures are returned, the spool still goes in."""
state = store.load(paths.state)
merged = 0
rtdb_error = None
deletes: dict[str, dict[str, object]] = {}
if db is not None:
try:
found, deletes = _rtdb_entries(db)
except RtdbError as e:
found, rtdb_error = [], str(e)
merged += sum(store.merge(state, app, key, entry) for app, key, entry in found)
merged += sum(store.merge(state, line.app, line.id, line.entry) for line in spool.drain(paths.spool))
store.save(paths.state, state)
store.write_reports(paths.reports, state)
if db is not None and not keep:
try:
for app, paths_gone in deletes.items():
db.patch(f"{app}/errors", paths_gone)
except RtdbError as e:
rtdb_error = str(e)
return Summary(merged, rtdb_error)

View File

View File

@ -0,0 +1,104 @@
# Copyright (c) 2026 Krzysztof Rudnicki
"""A minimal RTDB REST client on a seeded crdt-sync Google session.
Ported from kartka's ``tools/errors`` (``kartka_errors.rtdb``), plus
``shallow`` reads to list the apps. The sync account is Google-only, so the
session is an already seeded ``~/.config/<store>/firebase_auth.json`` (crdt-
sync's ``tool.seed_session`` writes one; it refreshes itself). The request URL
carries the ID token (``?auth=``), so no error raised here ever includes it.
"""
from __future__ import annotations
from collections.abc import Callable
import json
from typing import Protocol
import urllib.error
import urllib.request
#: The credential store read by default: the backup timer's, a PC-side reader
#: of the same database (any seeded store of the sync account works).
DEFAULT_SESSION = "firebase_backup"
#: Per-request timeout, seconds (token refresh has its own, below).
TIMEOUT = 15.0
TOKEN_TIMEOUT = 8.0
type Opener = Callable[..., object]
class RtdbError(RuntimeError):
"""An RTDB request failed, or there is no session (the message never names the URL)."""
class Rtdb(Protocol):
"""Read and multi-path write access to RTDB."""
def get(self, path: str, *, shallow: bool = False) -> object:
"""Return the decoded JSON at ``path`` (None when absent)."""
...
def patch(self, path: str, data: dict[str, object]) -> None:
"""Apply a multi-path update of ``path`` (a None value deletes)."""
...
class RestRtdb:
"""RTDB over REST with ``?auth=<id token>``."""
def __init__(
self,
database_url: str,
id_token: Callable[[], str],
opener: Opener = urllib.request.urlopen,
) -> None:
"""Talk to ``database_url`` with tokens from ``id_token``."""
self._url = database_url.rstrip("/")
self._id_token = id_token
self._open = opener
def _send(self, method: str, path: str, body: bytes | None, query: str = "") -> object:
try:
token = self._id_token()
except Exception as e: # noqa: BLE001 -- any refresh failure, typed only
msg = f"RTDB {method} {path}: token refresh failed ({type(e).__name__})"
raise RtdbError(msg) from None
req = urllib.request.Request( # noqa: S310 -- https only, built here
f"{self._url}/{path}.json?auth={token}{query}",
data=body,
method=method,
headers={"Content-Type": "application/json"},
)
try:
with self._open(req, timeout=TIMEOUT) as res: # type: ignore[attr-defined]
return json.loads(res.read() or b"null")
except urllib.error.HTTPError as e:
msg = f"RTDB {method} {path}: HTTP {e.code}"
raise RtdbError(msg) from None
except (urllib.error.URLError, OSError, ValueError) as e:
msg = f"RTDB {method} {path}: {type(e).__name__}"
raise RtdbError(msg) from None
def get(self, path: str, *, shallow: bool = False) -> object:
"""GET ``path``; raises :class:`RtdbError` on any failure."""
return self._send("GET", path, None, "&shallow=true" if shallow else "")
def patch(self, path: str, data: dict[str, object]) -> None:
"""PATCH ``data`` into ``path``; raises :class:`RtdbError`."""
self._send("PATCH", path, json.dumps(data).encode())
def open_rtdb(session: str = DEFAULT_SESSION) -> RestRtdb:
"""Return the client on the seeded ``session``; :class:`RtdbError` without one."""
import crdt_sync # noqa: PLC0415 -- only the RTDB path needs it; spool-only runs stay stdlib
try:
config = crdt_sync.FirebaseConfig.load()
except Exception as e: # noqa: BLE001 -- a missing or broken config, typed only
msg = f"no Firebase config ({type(e).__name__})"
raise RtdbError(msg) from None
store = crdt_sync.credential_store_for(session)
auth = crdt_sync.FirebaseTokenProvider(config.api_key, store, timeout_seconds=TOKEN_TIMEOUT)
if not auth.has_session():
msg = f"no seeded sign-in session '{session}'; seed one with crdt-sync's tool.seed_session --app {session}"
raise RtdbError(msg)
return RestRtdb(config.database_url, auth.id_token)

View File

@ -0,0 +1,128 @@
# Copyright (c) 2026 Krzysztof Rudnicki
"""Group error reports: one signature per bug, however often it fires.
Ported from kartka's ``tools/errors`` (``kartka_errors.signature``) and kept
byte-identical for Dart entries, so kartka's existing tickets keep their ids
when it moves onto this ingest. Generalised in one place only: which stack
frames belong to the app. An entry's ``mark`` names them (``package:todo/``
for Dart, ``leetcode_guard/`` for Python, ``res://`` for GDScript); without
one it is ``package:<app>/``.
A signature is the error type, the message with its volatile parts (numbers,
ids, URLs) masked, and where it happened: the first app frame without line
numbers (so an edit elsewhere in the file does not split it), else the route,
else the entry kind. A slow wait is its route + operation, never its time.
"""
from __future__ import annotations
from dataclasses import dataclass
import hashlib
import re
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from collections.abc import Mapping, Sequence
#: The longest normalised message kept in a signature.
MAX_MESSAGE = 160
#: The entry kind of a user-visible wait over a second.
SLOW_KIND = "perf.slow"
_TYPE = re.compile(r"^(?:Unhandled error: )?([A-Z][\w$]*(?:Exception|Error))\b")
#: Dart errors whose ``toString`` does not name their type.
_DART_PREFIXES = {
"Bad state": "StateError",
"Invalid argument": "ArgumentError",
"Unsupported operation": "UnsupportedError",
"Null check operator": "TypeError",
"Concurrent modification": "ConcurrentModificationError",
}
_URL = re.compile(r"(https?://\S+|<url>)")
_UUID = re.compile(r"\b[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}\b", re.IGNORECASE)
_HEX = re.compile(r"\b(?=[0-9a-f]*\d)[0-9a-f]{8,}\b", re.IGNORECASE)
_NUMBER = re.compile(r"\b\d+(\.\d+)?\b")
_SPACE = re.compile(r"\s+")
_FRAME = re.compile(r"^#\d+\s+")
_LINE_COL = re.compile(r"(\.(?:dart|py|gd))(:\d+)+")
#: Frames of the reporting plumbing itself, never the bug's location.
_PLUMBING = (
"package:kartka/app/error_hooks.dart",
"package:kartka/core/diagnostics/",
"package:crdt_sync_flutter/src/errors/",
"gatelock/error_report.py",
"res://addons/app_errors/",
)
@dataclass(frozen=True)
class Signature:
"""What groups reports of one bug."""
id: str
type: str
message: str
where: str
def error_type(error: str, kind: str) -> str:
"""Return the exception type named by ``error``, else ``kind``."""
if match := _TYPE.match(error):
return match[1]
for prefix, name in _DART_PREFIXES.items():
if error.startswith(prefix):
return name
return kind
def normalise(message: str) -> str:
"""Return ``message`` with URLs, uuids, long hex and numbers masked, trimmed."""
text = _URL.sub("<url>", message)
text = _UUID.sub("<uuid>", text)
text = _HEX.sub("<hex>", text)
text = _NUMBER.sub("<n>", text)
return _SPACE.sub(" ", text).strip()[:MAX_MESSAGE]
def top_frame(stack: Sequence[str], mark: str) -> str | None:
"""Return the first frame containing ``mark`` outside the reporting plumbing."""
for frame in stack:
if mark in frame and not any(p in frame for p in _PLUMBING):
return _LINE_COL.sub(r"\1", _FRAME.sub("", frame.strip()))
return None
def mark_of(entry: Mapping[str, object], app: str) -> str:
"""Return the substring that marks the app's own frames in ``entry``."""
mark = entry.get("mark")
return mark if isinstance(mark, str) and mark else f"package:{app}/"
def strings(value: object) -> list[str]:
"""Return ``value`` as a list of strings; empty when it is not a list."""
return [str(v) for v in value] if isinstance(value, list) else []
def _digest(text: str) -> str:
return hashlib.sha1(text.encode()).hexdigest()[:10] # noqa: S324 -- a grouping key, not security
def signature(entry: Mapping[str, object], *, app: str) -> Signature:
"""Return the signature of one report of ``app``."""
kind = str(entry.get("kind", "?"))
if kind == SLOW_KIND:
operation = str(entry.get("operation") or "?")
where = f"route {entry.get('route') or '?'}"
return Signature(
_digest(f"{SLOW_KIND}|{where}|{operation}"),
SLOW_KIND,
f"slow wait: {operation}",
where,
)
error = str(entry.get("error", ""))
frame = top_frame(strings(entry.get("stack")), mark_of(entry, app))
route = entry.get("route")
where = frame or (f"route {route}" if route else f"kind {kind}")
kind_type = error_type(error, kind)
message = normalise(error)
return Signature(_digest(f"{kind_type}|{message}|{where}"), kind_type, message, where)

View File

@ -0,0 +1,66 @@
# Copyright (c) 2026 Krzysztof Rudnicki
"""The local spool: how PC apps (Tk gate windows, Godot games) report.
They run on the PC the ingest runs on, so they need no RTDB: each report is
one JSON line appended to ``<spool>/<app>.jsonl`` (the writers are
``gatelock.error_report`` and the Godot ``app_errors`` addon; this module is
the reference writer and the reader). The ingest claims a file by renaming it
to ``.jsonl.ingesting`` before reading, so a line appended meanwhile lands in
a fresh file and is never lost. The spool sits beside, not inside, the
board's ``app-errors/`` folder, which reads every ``*.json`` there.
"""
from __future__ import annotations
from dataclasses import dataclass
import hashlib
import json
from typing import TYPE_CHECKING
from app_errors.store import APP_NAME
if TYPE_CHECKING:
from collections.abc import Iterator, Mapping
from pathlib import Path
SUFFIX = ".jsonl"
CLAIMED = ".jsonl.ingesting"
@dataclass(frozen=True)
class Line:
"""One spooled report: its app, a stable id and the decoded entry."""
app: str
id: str
entry: object
def append(folder: Path, app: str, entry: Mapping[str, object]) -> None:
"""Append ``entry`` as one line to ``app``'s spool file."""
folder.mkdir(parents=True, exist_ok=True)
with (folder / f"{app}{SUFFIX}").open("a", encoding="utf-8") as f:
f.write(json.dumps(entry, sort_keys=True) + "\n")
def _claim(folder: Path) -> list[Path]:
for path in folder.glob(f"*{SUFFIX}"):
path.replace(path.with_name(path.name.removesuffix(SUFFIX) + CLAIMED))
return sorted(folder.glob(f"*{CLAIMED}"))
def drain(folder: Path) -> Iterator[Line]:
"""Yield every spooled line, then delete the files it came from."""
if not folder.is_dir():
return
for path in _claim(folder):
app = path.name.removesuffix(CLAIMED)
if APP_NAME.match(app):
for text in path.read_text(encoding="utf-8").splitlines():
try:
entry = json.loads(text)
except ValueError:
continue
digest = hashlib.sha1(text.encode()).hexdigest()[:20] # noqa: S324 -- an id, not security
yield Line(app, f"spool/{digest}", entry)
path.unlink()

View File

@ -0,0 +1,207 @@
# Copyright (c) 2026 Krzysztof Rudnicki
"""The local record of every report ingested, grouped by app and signature.
It is the source of truth once RTDB entries and spool lines are consumed:
``seen`` makes a re-read count nothing twice, and each signature keeps its
count, first and last time, newest build and its resolution. :func:`report`
turns one app's records into the todoImplementer ``app-error`` contract
(``contracts/app-error.schema.json``, ``report``).
Resolve and reopen: ``resolve`` stores the time and the newest build seen. A
later report reopens the error only when it is a regression: a newer build
when both sides know the build, else a report made after the resolve.
"""
from __future__ import annotations
import json
import os
from pathlib import Path
import re
import tempfile
from typing import TYPE_CHECKING, NotRequired, TypedDict, cast
from app_errors.signature import SLOW_KIND, mark_of, signature, strings, top_frame
if TYPE_CHECKING:
from collections.abc import Mapping
#: Entry ids remembered for dedup (oldest dropped first).
MAX_SEEN = 5000
#: An app name the board accepts (the contract's ``app`` pattern).
APP_NAME = re.compile(r"^[a-z0-9][a-z0-9_-]*$")
_REPO = re.compile(r"^[^/\s]+/[^/\s]+$")
class Resolved(TypedDict):
"""When the error was marked fixed, and the newest build seen then."""
at: str
build: int
class Rec(TypedDict):
"""Everything known about one signature of one app."""
signature: str
repo: str
kind: str
message: str
top_frame: str | None
route: str | None
count: int
first_seen: str
last_seen: str
build: int
version_name: str | None
longest_ms: NotRequired[int]
resolved: Resolved | None
class State(TypedDict):
"""The whole state file."""
version: int
seen: dict[str, str]
apps: dict[str, dict[str, Rec]]
def empty() -> State:
"""Return a state that knows nothing."""
return {"version": 1, "seen": {}, "apps": {}}
def load(path: Path) -> State:
"""Return the state at ``path``; empty when missing or unreadable."""
try:
data = json.loads(path.read_text())
except OSError, ValueError:
return empty()
if not isinstance(data, dict) or data.get("version") != 1:
return empty()
return cast("State", data)
def _atomic_write(path: Path, text: str) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
fd, tmp = tempfile.mkstemp(dir=path.parent, prefix=f".{path.stem}-")
with os.fdopen(fd, "w") as f:
f.write(text)
Path(tmp).replace(path)
def save(path: Path, state: State) -> None:
"""Write ``state`` atomically (temp file + rename)."""
_atomic_write(path, json.dumps(state, indent=1, sort_keys=True))
def _int(value: object) -> int:
return value if isinstance(value, int) and not isinstance(value, bool) and value >= 0 else 0
def _str(value: object) -> str | None:
return value if isinstance(value, str) and value else None
def _reopens(resolved: Resolved, at: str, build: int) -> bool:
if build and resolved["build"]:
return build > resolved["build"]
return at > resolved["at"]
def merge(state: State, app: str, entry_id: str, raw: object) -> bool:
"""Count one report ``raw`` of ``app`` under its signature; False when seen or not a report."""
if not isinstance(raw, dict):
return False
entry = cast("Mapping[str, object]", raw)
at, kind = entry.get("at"), entry.get("kind")
if not isinstance(at, str) or not at or not isinstance(kind, str) or not kind:
return False
key = f"{app}/{entry_id}"
if key in state["seen"]:
return False
state["seen"][key] = at
while len(state["seen"]) > MAX_SEEN:
del state["seen"][next(iter(state["seen"]))]
sig = signature(entry, app=app)
build = _int(entry.get("build"))
recs = state["apps"].setdefault(app, {})
rec = recs.get(sig.id)
if rec is None:
repo = _str(entry.get("repo"))
rec = recs[sig.id] = {
"signature": sig.id,
"repo": repo if repo and _REPO.match(repo) else f"kuhyx/{app}",
"kind": kind,
"message": sig.message,
"top_frame": top_frame(strings(entry.get("stack")), mark_of(entry, app)),
"route": _str(entry.get("route")),
"count": 0,
"first_seen": at,
"last_seen": at,
"build": 0,
"version_name": None,
"resolved": None,
}
rec["count"] += 1
rec["first_seen"] = min(rec["first_seen"], at)
rec["last_seen"] = max(rec["last_seen"], at)
if build >= rec["build"]:
rec["build"] = build
rec["version_name"] = _str(entry.get("version")) or rec["version_name"]
if kind == SLOW_KIND:
rec["longest_ms"] = max(rec.get("longest_ms", 0), _int(entry.get("durationMs")))
resolved = rec["resolved"]
if resolved is not None and _reopens(resolved, at, build):
rec["resolved"] = None
return True
def resolve(state: State, app: str, sig: str, *, now: str) -> bool:
"""Mark ``app``'s ``sig`` fixed as of ``now``; False when there is no such error."""
rec = state["apps"].get(app, {}).get(sig)
if rec is None:
return False
rec["resolved"] = {"at": now, "build": rec["build"]}
return True
def _detail(rec: Rec) -> str | None:
route = f"route {rec['route']}" if rec["route"] else None
if rec["kind"] == SLOW_KIND:
longest = f"longest wait {rec.get('longest_ms', 0)} ms"
return f"{longest}, {route}" if route else longest
return route
def report(state: State, app: str) -> dict[str, object]:
"""Return ``app``'s records as a contract ``report``."""
errors = [
{
"app": app,
"repo": rec["repo"],
"signature": rec["signature"],
"kind": rec["kind"],
"message": rec["message"],
"top_frame": rec["top_frame"],
"count": rec["count"],
"first_seen": rec["first_seen"],
"last_seen": rec["last_seen"],
"build": rec["build"],
"version_name": rec["version_name"],
"detail": _detail(rec),
"open": rec["resolved"] is None,
}
for rec in sorted(state["apps"].get(app, {}).values(), key=lambda r: r["signature"])
]
return {"version": 1, "errors": errors}
def write_reports(folder: Path, state: State) -> list[Path]:
"""Write ``<folder>/<app>.json`` for every app with records; return the paths."""
written = []
for app in sorted(state["apps"]):
path = folder / f"{app}.json"
_atomic_write(path, json.dumps(report(state, app), indent=1))
written.append(path)
return written

View File

View File

@ -0,0 +1,74 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://todoimplementer.local/contracts/app-error.schema.json",
"title": "App errors",
"description": "What an app writes for the errors loop (DOCS-errors.md): `~/.local/state/app-errors/<app>.json` is a `report`; kartka's own state.json is mapped to the same `record` by an adapter. One record per identical error (signature).",
"$defs": {
"record": {
"type": "object",
"description": "One identical error or slow wait. The loop makes one ticket per `app:signature`.",
"properties": {
"app": { "type": "string", "pattern": "^[a-z0-9][a-z0-9_-]*$" },
"repo": { "type": "string", "pattern": "^[^/\\s]+/[^/\\s]+$", "description": "owner/name of the repo that holds the fix." },
"signature": { "type": "string", "minLength": 1, "pattern": "^\\S+$", "description": "Stable id of the error: the same error always gets the same one." },
"kind": { "type": "string", "minLength": 1, "description": "crash, ui.error, perf.slow, ..." },
"message": { "type": "string", "description": "Masked message (no secrets, no user data)." },
"top_frame": { "type": ["string", "null"] },
"count": { "type": "integer", "minimum": 1 },
"first_seen": { "type": "string", "minLength": 1 },
"last_seen": { "type": "string", "minLength": 1 },
"build": { "type": "integer", "minimum": 0, "description": "Monotonic build of the newest report (Android versionCode); 0 when the source does not know it." },
"version_name": { "type": ["string", "null"] },
"detail": { "type": ["string", "null"], "description": "Free text: durations, route." },
"open": { "type": "boolean", "description": "False once the source marks the error resolved; default true." }
},
"required": ["app", "repo", "signature", "kind", "message", "count", "first_seen", "last_seen", "build"],
"additionalProperties": false
},
"report": {
"type": "object",
"properties": {
"version": { "const": 1 },
"errors": { "type": "array", "items": { "$ref": "#/$defs/record" } }
},
"required": ["version", "errors"],
"additionalProperties": false
},
"source": {
"type": "object",
"description": "One entry of settings.json `error_sources`. `resolve` and `build` are argv lists; `{signature}`, `{repo_dir}` and `{commit}` are replaced.",
"properties": {
"app": { "type": "string", "pattern": "^[a-z0-9][a-z0-9_-]*$" },
"repo": { "type": "string", "pattern": "^[^/\\s]+/[^/\\s]+$" },
"adapter": { "enum": ["kartka", "contract"] },
"path": { "type": "string", "minLength": 1 },
"resolve": { "type": "array", "items": { "type": "string" } },
"build": { "type": "array", "items": { "type": "string" } }
},
"required": ["app", "repo", "adapter", "path"],
"additionalProperties": false
},
"ledger": {
"type": "object",
"description": "`<run root>/app-errors.json`: what the loop knows per `app:signature`.",
"additionalProperties": { "$ref": "#/$defs/entry" }
},
"entry": {
"type": "object",
"properties": {
"state": { "enum": ["ticket", "shipped", "fixed"] },
"repo": { "type": "string" },
"title": { "type": "string" },
"count": { "type": "integer", "minimum": 0 },
"starts": { "type": "integer", "minimum": 0 },
"commit": { "type": "string" },
"shipped_at": { "type": "number" },
"fixed_build": { "type": "integer", "minimum": 0 },
"installed_at": { "type": "number" },
"missing": { "type": "integer", "minimum": 0, "description": "Ticks the part was not on the board (a Ship may be closing it)." }
},
"required": ["state", "repo", "title", "count", "starts"],
"additionalProperties": false
}
}
}

View File

@ -0,0 +1,173 @@
"""The ingest: RTDB and local spools in, one board report per app out."""
from __future__ import annotations
import json
from pathlib import Path
import pytest
from app_errors import cli, ingest, spool
from app_errors.rtdb import RtdbError
class FakeRtdb:
"""An in-memory RTDB: ``get`` honours ``shallow``, ``patch`` deletes on None."""
def __init__(self, data: dict[str, object]) -> None:
self.data = data
self.patches: list[tuple[str, dict[str, object]]] = []
self.fail = False
def _at(self, path: str) -> object:
node: object = self.data
for part in [p for p in path.split("/") if p]:
if not isinstance(node, dict) or part not in node:
return None
node = node[part]
return node
def get(self, path: str, *, shallow: bool = False) -> object:
if self.fail:
raise RtdbError("down")
node = self._at(path)
if shallow and isinstance(node, dict):
return dict.fromkeys(node, True)
return node
def patch(self, path: str, data: dict[str, object]) -> None:
self.patches.append((path, data))
for key, value in data.items():
parts = [p for p in f"{path}/{key}".split("/") if p]
node = self.data
for part in parts[:-1]:
node = node.setdefault(part, {}) # type: ignore[assignment]
if value is None:
node.pop(parts[-1], None)
else:
node[parts[-1]] = value
def _crash(at: str) -> dict[str, object]:
return {
"repo": "kuhyx/todo",
"at": at,
"kind": "uncaught.flutter",
"error": "Bad state: x",
"build": 3,
}
@pytest.fixture
def dirs(tmp_path: Path) -> ingest.Paths:
return ingest.Paths(
state=tmp_path / "ingest" / "state.json",
reports=tmp_path / "app-errors",
spool=tmp_path / "app-errors-spool",
)
def test_rtdb_entries_become_reports_and_are_deleted(dirs: ingest.Paths) -> None:
db = FakeRtdb({
"todo": {"errors": {"dev1": {"e1": _crash("2026-11-17T10:00:00Z"), "e2": _crash("2026-11-17T10:00:01Z")}}},
"kartka": {"errors": {"dev": {"k": _crash("2026-11-17T10:00:00Z")}}},
"other": {"data": 1},
}) # fmt: skip
summary = ingest.run(dirs, db)
assert summary.merged == 2
report = json.loads((dirs.reports / "todo.json").read_text())
assert report["errors"][0]["count"] == 2
assert not (dirs.reports / "kartka.json").exists() # kartka keeps its own ingest
assert db.data["todo"] == {"errors": {"dev1": {}}} # ingested entries are removed from RTDB
assert "k" in db.data["kartka"]["errors"]["dev"] # type: ignore[index]
def test_keep_leaves_rtdb_and_still_counts_once(dirs: ingest.Paths) -> None:
db = FakeRtdb({"todo": {"errors": {"d": {"e": _crash("2026-11-17T10:00:00Z")}}}})
ingest.run(dirs, db, keep=True)
ingest.run(dirs, db, keep=True)
report = json.loads((dirs.reports / "todo.json").read_text())
assert report["errors"][0]["count"] == 1
assert db.patches == []
def test_rtdb_down_still_ingests_spool(dirs: ingest.Paths) -> None:
spool.append(dirs.spool, "tk-app", {"repo": "kuhyx/tk", "at": "2026-11-17T10:00:00Z",
"kind": "uncaught.python", "error": "ValueError: x"}) # fmt: skip
db = FakeRtdb({})
db.fail = True
summary = ingest.run(dirs, db)
assert summary.merged == 1
assert summary.rtdb_error == "down"
assert json.loads((dirs.reports / "tk-app.json").read_text())["errors"][0]["repo"] == "kuhyx/tk"
assert list(dirs.spool.glob("*")) == [] # the spool is consumed
def test_no_rtdb_at_all(dirs: ingest.Paths) -> None:
assert ingest.run(dirs, None).merged == 0
def test_spool_bad_lines_and_bad_names_are_skipped(dirs: ingest.Paths) -> None:
dirs.spool.mkdir(parents=True)
(dirs.spool / "ok.jsonl").write_text('not json\n{"at": "2026-11-17T10:00:00Z", "kind": "k", "error": "e"}\n[]\n')
(dirs.spool / "Bad Name.jsonl").write_text('{"at": "x", "kind": "k", "error": "e"}\n')
assert ingest.run(dirs, None).merged == 1
def test_leftover_claimed_spool_is_ingested(dirs: ingest.Paths) -> None:
dirs.spool.mkdir(parents=True)
(dirs.spool / "ok.jsonl.ingesting").write_text('{"at": "2026-11-17T10:00:00Z", "kind": "k", "error": "e"}\n')
assert ingest.run(dirs, None).merged == 1
assert list(dirs.spool.glob("*")) == []
def test_rtdb_odd_shapes_are_skipped(dirs: ingest.Paths) -> None:
db = FakeRtdb(
{
"todo": {"errors": {"d": "nope", "e": {"x": "nope"}}},
"bad key!": {"errors": {}},
}
)
assert ingest.run(dirs, db).merged == 0
def test_cli_ingest_and_resolve(dirs: ingest.Paths, monkeypatch: pytest.MonkeyPatch,
capsys: pytest.CaptureFixture[str]) -> None: # fmt: skip
monkeypatch.setattr(ingest, "default_paths", lambda: dirs)
spool.append(dirs.spool, "tk", {"at": "2026-11-17T10:00:00Z", "kind": "k", "error": "e"})
assert cli.main(["ingest", "--no-rtdb"]) == 0
sig = json.loads((dirs.reports / "tk.json").read_text())["errors"][0]["signature"]
assert cli.main(["resolve", "tk", sig]) == 0
assert json.loads((dirs.reports / "tk.json").read_text())["errors"][0]["open"] is False
assert cli.main(["resolve", "tk", "missing"]) == 1
assert "unknown" in capsys.readouterr().err
def test_cli_ingest_rtdb_session_missing(dirs: ingest.Paths, monkeypatch: pytest.MonkeyPatch,
capsys: pytest.CaptureFixture[str]) -> None: # fmt: skip
monkeypatch.setattr(ingest, "default_paths", lambda: dirs)
def no_session(_app: str) -> FakeRtdb:
raise RtdbError("no session")
monkeypatch.setattr(cli, "open_rtdb", no_session)
spool.append(dirs.spool, "tk", {"at": "2026-11-17T10:00:00Z", "kind": "k", "error": "e"})
assert cli.main(["ingest"]) == 1 # the spool still went in, but RTDB is reported broken
assert (dirs.reports / "tk.json").exists()
assert "no session" in capsys.readouterr().err
def test_cli_ingest_with_rtdb(dirs: ingest.Paths, monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr(ingest, "default_paths", lambda: dirs)
db = FakeRtdb({"todo": {"errors": {"d": {"e": _crash("2026-11-17T10:00:00Z")}}}})
monkeypatch.setattr(cli, "open_rtdb", lambda _app: db)
assert cli.main(["ingest", "--keep", "--session", "x"]) == 0
assert (dirs.reports / "todo.json").exists()
def test_default_paths_follow_xdg(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
monkeypatch.setenv("XDG_STATE_HOME", str(tmp_path))
paths = ingest.default_paths()
assert paths.reports == tmp_path / "app-errors"
assert paths.spool == tmp_path / "app-errors-spool"
assert paths.state == tmp_path / "app-errors-ingest" / "state.json"

View File

@ -0,0 +1,140 @@
"""The REST client: URLs, shallow reads, typed errors that never name the token."""
from __future__ import annotations
import email.message
import io
import json
from pathlib import Path
import sys
import types
from typing import cast
import urllib.error
import urllib.request
import pytest
from app_errors import ingest, rtdb, store
from app_errors.rtdb import RestRtdb, RtdbError
class _Res(io.BytesIO):
def __enter__(self) -> _Res:
return self
def __exit__(self, *_: object) -> None:
self.close()
def test_get_and_patch_build_urls() -> None:
seen: list[tuple[str, str, object]] = []
def opener(req: urllib.request.Request, timeout: float) -> _Res:
seen.append((req.get_method(), req.full_url, req.data))
return _Res(b'{"a": true}')
db = RestRtdb("https://db.example/", lambda: "TOK", opener)
assert db.get("", shallow=True) == {"a": True}
db.patch("todo/errors", {"d/e": None})
assert seen[0][:2] == ("GET", "https://db.example/.json?auth=TOK&shallow=true")
assert seen[1][0] == "PATCH"
assert json.loads(cast("bytes", seen[1][2])) == {"d/e": None}
def test_empty_body_is_none() -> None:
assert RestRtdb("https://x", lambda: "t", lambda req, timeout: _Res(b"")).get("p") is None
@pytest.mark.parametrize(
"exc",
[
urllib.error.HTTPError("u", 401, "no", email.message.Message(), None),
urllib.error.URLError("down"),
OSError("x"),
],
)
def test_failures_are_typed_and_tokenless(exc: Exception) -> None:
def opener(req: urllib.request.Request, timeout: float) -> _Res:
raise exc
with pytest.raises(RtdbError) as info:
RestRtdb("https://x", lambda: "SECRET", opener).get("p")
assert "SECRET" not in str(info.value)
def test_token_failure_is_typed() -> None:
def boom() -> str:
raise ValueError("SECRET")
with pytest.raises(RtdbError, match="token refresh failed"):
RestRtdb("https://x", boom).get("p")
def _fake_crdt(monkeypatch: pytest.MonkeyPatch, *, session: bool, config_ok: bool = True) -> None:
mod = types.ModuleType("crdt_sync")
class Config:
api_key, database_url = "k", "https://db"
@staticmethod
def load() -> Config:
if not config_ok:
raise FileNotFoundError("cfg")
return Config()
class Provider:
def __init__(self, *_: object, **__: object) -> None: ...
def has_session(self) -> bool:
return session
def id_token(self) -> str:
return "t"
mod.FirebaseConfig = Config # type: ignore[attr-defined]
mod.FirebaseTokenProvider = Provider # type: ignore[attr-defined]
mod.credential_store_for = lambda name: name # type: ignore[attr-defined]
monkeypatch.setitem(sys.modules, "crdt_sync", mod)
def test_open_rtdb(monkeypatch: pytest.MonkeyPatch) -> None:
_fake_crdt(monkeypatch, session=True)
assert isinstance(rtdb.open_rtdb("s"), RestRtdb)
_fake_crdt(monkeypatch, session=False)
with pytest.raises(RtdbError, match="no seeded"):
rtdb.open_rtdb("s")
_fake_crdt(monkeypatch, session=True, config_ok=False)
with pytest.raises(RtdbError, match="no Firebase config"):
rtdb.open_rtdb("s")
class _PatchFails:
def get(self, path: str, *, shallow: bool = False) -> object:
if shallow:
return {"todo": True}
return {"d": {"e": {"at": "2026-11-17T10:00:00Z", "kind": "k", "error": "x"}}}
def patch(self, path: str, data: dict[str, object]) -> None:
raise RtdbError("patch down")
def test_delete_failure_is_reported_after_save(tmp_path: Path) -> None:
paths = ingest.Paths(tmp_path / "s.json", tmp_path / "r", tmp_path / "sp")
summary = ingest.run(paths, _PatchFails())
assert summary == ingest.Summary(1, "patch down")
assert (tmp_path / "r" / "todo.json").exists()
def test_old_state_version_and_seen_trim(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
path = tmp_path / "s.json"
path.write_text('{"version": 0}')
assert store.load(path) == store.empty()
monkeypatch.setattr(store, "MAX_SEEN", 2)
state = store.empty()
for i in range(3):
store.merge(
state,
"a",
str(i),
{"at": f"2026-11-17T10:00:0{i}Z", "kind": "k", "error": "x"},
)
assert list(state["seen"]) == ["a/1", "a/2"]

View File

@ -0,0 +1,89 @@
"""Signatures: one per bug, kartka-identical for Dart, generalised for Python and GDScript."""
from __future__ import annotations
from app_errors.signature import normalise, signature, top_frame
def test_dart_crash_matches_kartka_vector() -> None:
# Vector from kartka_errors.signature (kartka 2026-11-17): moving kartka
# onto this ingest must not rename its open tickets.
entry = {
"kind": "uncaught.flutter",
"error": "Bad state: no element 42",
"stack": [
"#0 List.first (dart:core/list.dart:12:3)",
"#1 Reader.open (package:kartka/reader/reader.dart:88:7)",
],
}
sig = signature(entry, app="kartka")
assert sig.id == "ffcd864656"
assert sig.type == "StateError"
assert sig.where == "Reader.open (package:kartka/reader/reader.dart)"
def test_slow_wait_matches_kartka_vector() -> None:
entry = {
"kind": "perf.slow",
"operation": "reader.open",
"route": "/reader/:id",
"durationMs": 2100,
}
sig = signature(entry, app="kartka")
assert sig.id == "29a53e8824"
assert sig.message == "slow wait: reader.open"
def test_slow_wait_ignores_duration() -> None:
one = {"kind": "perf.slow", "operation": "load", "durationMs": 1100}
two = {"kind": "perf.slow", "operation": "load", "durationMs": 9000}
assert signature(one, app="a").id == signature(two, app="a").id
def test_mark_picks_app_frame_and_skips_plumbing() -> None:
stack = [
"#0 ErrorReporter.report (package:crdt_sync_flutter/src/errors/error_reporter.dart:5:1)",
"#1 Home.build (package:todo/home.dart:10:2)",
]
assert top_frame(stack, "package:todo/") == "Home.build (package:todo/home.dart)"
def test_python_frame_line_numbers_dropped() -> None:
entry = {
"kind": "uncaught.python",
"error": "ValueError: bad 12",
"stack": [
"gatelock/error_report.py:40 in hook",
"leetcode_guard/ui.py:123 in solve",
],
"mark": "leetcode_guard/",
}
sig = signature(entry, app="leetcode-guard")
assert sig.where == "leetcode_guard/ui.py in solve"
assert sig.type == "ValueError"
assert sig.message == "ValueError: bad <n>"
def test_godot_frame_and_kind_fallback() -> None:
entry = {
"kind": "uncaught.gdscript",
"error": "Invalid access to property 'x'",
"stack": [
"res://addons/app_errors/error_reporter.gd:9 in _log",
"res://scripts/car.gd:77 in _ready",
],
"mark": "res://",
}
sig = signature(entry, app="car-rental-sim")
assert sig.where == "res://scripts/car.gd in _ready"
assert sig.type == "uncaught.gdscript"
def test_no_frame_falls_back_to_route_then_kind() -> None:
assert signature({"kind": "ui.snackbar", "error": "x", "route": "/a"}, app="a").where == "route /a"
assert signature({"kind": "ui.snackbar", "error": "x"}, app="a").where == "kind ui.snackbar"
def test_normalise_masks_volatile_parts() -> None:
text = "GET https://x.y/z 3f2a9c1d00ff 123e4567-e89b-12d3-a456-426614174000 took 15 s"
assert normalise(text) == "GET <url> <hex> <uuid> took <n> s"

View File

@ -0,0 +1,144 @@
"""The merged state: counts, builds, resolve and reopen, and the board report."""
from __future__ import annotations
import json
from pathlib import Path
import jsonschema
from app_errors import store
SCHEMA = json.loads((Path(__file__).parent / "data" / "app-error.schema.json").read_text())
def _crash(at: str, build: int = 7, **extra: object) -> dict[str, object]:
return {
"app": "todo",
"repo": "kuhyx/todo",
"at": at,
"kind": "uncaught.flutter",
"error": "Bad state: boom",
"stack": ["#0 Home.build (package:todo/home.dart:3:1)"],
"build": build,
"version": "1.0.0",
**extra,
}
def _report(state: store.State, app: str) -> dict[str, object]:
report = store.report(state, app)
jsonschema.Draft202012Validator({**SCHEMA, "$ref": "#/$defs/report"}).validate(report)
return report
def test_ten_identical_errors_are_one_record_with_count_ten() -> None:
state = store.empty()
for i in range(10):
assert store.merge(state, "todo", f"dev/{i}", _crash(f"2026-11-17T10:00:{i:02d}Z"))
errors = _report(state, "todo")["errors"]
assert isinstance(errors, list)
assert len(errors) == 1
rec = errors[0]
assert rec["count"] == 10
assert rec["first_seen"] == "2026-11-17T10:00:00Z"
assert rec["last_seen"] == "2026-11-17T10:00:09Z"
assert rec["build"] == 7
assert rec["version_name"] == "1.0.0"
assert rec["open"] is True
assert rec["repo"] == "kuhyx/todo"
def test_same_entry_id_counts_once() -> None:
state = store.empty()
assert store.merge(state, "todo", "dev/a", _crash("2026-11-17T10:00:00Z"))
assert not store.merge(state, "todo", "dev/a", _crash("2026-11-17T10:00:00Z"))
assert state["apps"]["todo"][next(iter(state["apps"]["todo"]))]["count"] == 1
def test_newest_build_wins() -> None:
state = store.empty()
store.merge(state, "todo", "a", _crash("2026-11-17T10:00:00Z", build=9))
store.merge(state, "todo", "b", _crash("2026-11-17T11:00:00Z", build=4))
assert _report(state, "todo")["errors"][0]["build"] == 9 # type: ignore[index]
def test_slow_wait_detail_names_longest_wait() -> None:
state = store.empty()
for i, ms in enumerate((1500, 2100)):
store.merge(
state,
"todo",
f"w{i}",
{"app": "todo", "repo": "kuhyx/todo", "at": f"2026-11-17T10:00:0{i}Z", "kind": "perf.slow",
"error": f"list.load waited {ms} ms", "operation": "list.load", "route": "/", "durationMs": ms},
) # fmt: skip
rec = _report(state, "todo")["errors"][0] # type: ignore[index]
assert rec["kind"] == "perf.slow"
assert rec["count"] == 2
assert "2100 ms" in rec["detail"]
assert rec["build"] == 0
def test_resolve_closes_and_older_build_does_not_reopen() -> None:
state = store.empty()
store.merge(state, "todo", "a", _crash("2026-11-17T10:00:00Z", build=7))
sig = _report(state, "todo")["errors"][0]["signature"] # type: ignore[index]
assert store.resolve(state, "todo", sig, now="2026-11-17T12:00:00Z")
assert _report(state, "todo")["errors"][0]["open"] is False # type: ignore[index]
store.merge(state, "todo", "b", _crash("2026-11-17T13:00:00Z", build=7))
assert _report(state, "todo")["errors"][0]["open"] is False # type: ignore[index]
store.merge(state, "todo", "c", _crash("2026-11-17T14:00:00Z", build=8))
assert _report(state, "todo")["errors"][0]["open"] is True # type: ignore[index]
def test_resolve_without_build_reopens_on_later_report() -> None:
state = store.empty()
store.merge(state, "tk", "a", _crash("2026-11-17T10:00:00Z", build=0, app="tk"))
sig = _report(state, "tk")["errors"][0]["signature"] # type: ignore[index]
store.resolve(state, "tk", sig, now="2026-11-17T12:00:00Z")
store.merge(state, "tk", "b", _crash("2026-11-17T11:00:00Z", build=0, app="tk"))
assert _report(state, "tk")["errors"][0]["open"] is False # type: ignore[index]
store.merge(state, "tk", "c", _crash("2026-11-17T13:00:00Z", build=0, app="tk"))
assert _report(state, "tk")["errors"][0]["open"] is True # type: ignore[index]
def test_resolve_unknown_signature_is_false() -> None:
assert not store.resolve(store.empty(), "todo", "nope", now="2026-11-17T12:00:00Z")
def test_missing_repo_defaults_to_kuhyx_app() -> None:
state = store.empty()
entry = _crash("2026-11-17T10:00:00Z")
del entry["repo"]
store.merge(state, "todo", "a", entry)
assert _report(state, "todo")["errors"][0]["repo"] == "kuhyx/todo" # type: ignore[index]
def test_bad_entry_is_skipped() -> None:
state = store.empty()
assert not store.merge(state, "todo", "a", {"kind": 3})
assert not store.merge(state, "todo", "b", "nope")
assert store.report(state, "todo")["errors"] == []
def test_save_and_load_round_trip(tmp_path: Path) -> None:
state = store.empty()
store.merge(state, "todo", "a", _crash("2026-11-17T10:00:00Z"))
path = tmp_path / "s" / "state.json"
store.save(path, state)
assert store.load(path) == state
path.write_text("{broken")
assert store.load(path) == store.empty()
assert store.load(tmp_path / "missing.json") == store.empty()
def test_write_reports_writes_one_file_per_app(tmp_path: Path) -> None:
state = store.empty()
store.merge(state, "todo", "a", _crash("2026-11-17T10:00:00Z"))
store.merge(state, "tk", "a", _crash("2026-11-17T10:00:00Z", app="tk"))
written = store.write_reports(tmp_path, state)
assert sorted(p.name for p in written) == ["tk.json", "todo.json"]
data = json.loads((tmp_path / "todo.json").read_text())
assert data["version"] == 1
assert len(data["errors"]) == 1

54
app_errors/install.sh Executable file
View File

@ -0,0 +1,54 @@
#!/bin/bash
# ============================================================================
# Install the app-errors CLI and its ingest timer for the current user.
#
# Own venv at ~/.local/share/app-errors/venv, the package installed editable
# from this checkout (run it from the live ~/src/utils, never a worktree).
# crdt-sync comes from the sibling directory, so the venv runs what is on disk.
# Idempotent: rerun it after any change. Exits non-zero unless the timer is
# active and one ingest pass over the local spool succeeds.
# ============================================================================
set -euo pipefail
REPO_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
UTILS_DIR="$(dirname "$REPO_DIR")"
readonly REPO_DIR UTILS_DIR
readonly VENV_DIR="${HOME}/.local/share/app-errors/venv"
readonly BIN_DIR="${HOME}/.local/bin"
readonly UNIT_DIR="${HOME}/.config/systemd/user"
readonly TIMER="app-errors-ingest.timer"
log() { printf 'install: %s\n' "$1" >&2; }
fail() { printf 'install: FAILED -- %s\n' "$1" >&2; exit 1; }
build_venv() {
[[ -x "$VENV_DIR/bin/python" ]] || /usr/bin/python3 -m venv "$VENV_DIR" || fail "venv"
"$VENV_DIR/bin/pip" install --quiet "$UTILS_DIR/crdt-sync" || fail "crdt-sync"
"$VENV_DIR/bin/pip" install --quiet --no-deps -e "$REPO_DIR" || fail "app-errors"
mkdir -p "$BIN_DIR"
ln -sf "$VENV_DIR/bin/app-errors" "$BIN_DIR/app-errors"
}
install_timer() {
mkdir -p "$UNIT_DIR"
install -m 0644 "$REPO_DIR/systemd/app-errors-ingest.service" "$REPO_DIR/systemd/$TIMER" "$UNIT_DIR/"
systemctl --user daemon-reload
systemctl --user enable --now "$TIMER"
}
verify() {
systemctl --user is-active --quiet "$TIMER" || fail "$TIMER is not active"
"$VENV_DIR/bin/app-errors" ingest --no-rtdb >/dev/null || fail "a spool-only ingest pass failed"
log "ok: $TIMER active, ingest runs"
}
main() {
log "installing app-errors from $REPO_DIR"
build_venv
install_timer
verify
}
main "$@"

186
app_errors/pyproject.toml Normal file
View File

@ -0,0 +1,186 @@
[project]
name = "app-errors"
version = "0.1.0"
description = "Shared app error and slow-wait ingest: RTDB and local spool reports become todoImplementer error tickets"
requires-python = ">=3.14"
dependencies = [
"crdt-sync @ git+https://github.com/kuhyx/utils@crdt-sync-v0.11.1#subdirectory=crdt-sync",
]
[project.optional-dependencies]
test = ["jsonschema==4.26.0", "pytest==9.2.0", "pytest-cov==7.1.0"]
[project.scripts]
app-errors = "app_errors.cli:main"
[tool.setuptools]
packages = ["app_errors"]
# py.typed: consumers type-check against the package (PEP 561).
[tool.setuptools.package-data]
app_errors = ["py.typed"]
[tool.ruff]
target-version = "py314"
line-length = 120
include = ["*.py", "**/*.py"]
exclude = [".git", ".venv", "__pycache__", "build", "dist", ".eggs"]
[tool.ruff.lint]
select = ["ALL"]
ignore = [
"D203",
"D213",
"COM812",
"ISC001",
"S603",
]
fixable = ["ALL"]
unfixable = []
[tool.ruff.lint.per-file-ignores]
"**/tests/*.py" = ["ANN", "ARG", "CPY001", "D", "EM101", "PLC0415", "PLR2004", "PYI034", "S101", "SLF001", "TC002", "TC003", "TRY003"]
"**/_fake_*.py" = ["ANN401", "ARG002"]
"**/test_*.py" = ["ANN", "ARG", "CPY001", "D", "EM101", "PLC0415", "PLR2004", "PYI034", "S101", "SLF001", "TC002", "TC003", "TRY003"]
[tool.ruff.lint.pydocstyle]
convention = "google"
[tool.ruff.lint.isort]
force-single-line = false
force-sort-within-sections = true
known-first-party = ["app_errors"]
[tool.ruff.lint.flake8-quotes]
docstring-quotes = "double"
inline-quotes = "double"
[tool.ruff.lint.flake8-tidy-imports]
ban-relative-imports = "all"
[tool.ruff.format]
quote-style = "double"
indent-style = "space"
skip-magic-trailing-comma = false
line-ending = "auto"
docstring-code-format = true
[tool.mypy]
python_version = "3.14"
strict = true
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true
disallow_incomplete_defs = true
check_untyped_defs = true
disallow_untyped_decorators = true
no_implicit_optional = true
warn_redundant_casts = true
warn_unused_ignores = true
warn_no_return = true
warn_unreachable = true
# crdt_sync ships no py.typed: its names would all be "unfollowed".
disallow_any_unimported = false
disallow_any_explicit = false
disallow_any_generics = true
disallow_subclassing_any = true
strict_equality = true
extra_checks = true
ignore_missing_imports = true
show_error_codes = true
color_output = true
exclude = [".venv/"]
[tool.pylint.main]
# The repo root on the path, so pylint resolves the package's own imports the
# same way pytest does instead of reporting import-error for them.
init-hook = "import sys; sys.path.insert(0, 'app_errors')"
analyse-fallback-blocks = true
persistent = true
jobs = 0
py-version = "3.14"
ignore = [".venv", "__pycache__"]
# Test suites intentionally use patterns (protected-access, fixture-arg
# shadowing) that don't apply to source code; ruff's per-file-ignores carry
# the equivalent exemptions, so tests stay linted -- by ruff. This is the same
# source-only pylint scope wake-alarm / diet-guard / testsAndMisc use. It has
# to be `ignore-paths`, not `ignore = ["tests"]`: `ignore` only matches the
# basename of what pylint is *given*, and pre-commit hands it explicit file
# paths, so a directory name in `ignore` never fires under the hook.
ignore-paths = ["^(.*/)?tests/.*$"]
ignore-patterns = [".*\\.pyi$"]
[tool.pylint.basic]
# snake_case, plus the ``do_<METHOD>`` names http.server dispatches on: the
# stdlib looks handlers up as ``"do_" + command``, so ``do_GET`` is the only
# spelling that works and cannot be renamed to satisfy the style.
method-rgx = "([^\\W\\dA-Z][^\\WA-Z]{2,}|_[^\\WA-Z]*|__[^\\W\\dA-Z][^\\WA-Z]+__|do_[A-Z]+|of)$"
[tool.pylint.messages_control]
enable = "all"
disable = []
[tool.pylint.design]
min-public-methods = 0
max-module-lines = 1000
max-attributes = 10
[tool.pylint.typecheck]
generated-members = [
".*\\.assert_called_once_with",
".*\\.assert_called_once",
".*\\.assert_called",
".*\\.assert_not_called",
".*\\.assert_any_call",
".*\\.call_args",
".*\\.call_args_list",
".*\\.call_count",
]
[tool.bandit]
exclude_dirs = ["tests", ".venv"]
[tool.pytest.ini_options]
testpaths = ["app_errors/tests"]
python_files = ["test_*.py", "*_test.py"]
python_classes = ["Test*"]
python_functions = ["test_*"]
addopts = [
"-v",
"--strict-markers",
"--strict-config",
"-ra",
"--cov=app_errors",
"--cov-branch",
"--cov-report=term-missing",
"--cov-report=lcov",
]
filterwarnings = [
"error",
"ignore::DeprecationWarning",
"default::pytest.PytestUnraisableExceptionWarning",
]
[tool.coverage.run]
source = ["app_errors"]
branch = true
omit = [
"*/__pycache__/*",
"*/tests/*",
"*/.venv/*",
]
[tool.coverage.report]
fail_under = 100
show_missing = true
skip_covered = false
exclude_lines = [
"pragma: no cover",
"raise NotImplementedError",
"raise AssertionError",
"if TYPE_CHECKING:",
"^\\s*\\.\\.\\.$",
'if __name__ == "__main__":',
]
partial_branches = ["pragma: no branch"]

View File

@ -0,0 +1,5 @@
# Exact pins, enforced by the shared dependency-freshness gate.
crdt-sync @ git+https://github.com/kuhyx/utils@crdt-sync-v0.11.1#subdirectory=crdt-sync
jsonschema==4.26.0
pytest==9.2.0
pytest-cov==7.1.0

View File

@ -0,0 +1,10 @@
[Unit]
Description=Ingest app error and slow-wait reports for the todoImplementer board
After=network-online.target
[Service]
Type=oneshot
ExecStart=%h/.local/share/app-errors/venv/bin/app-errors ingest
Nice=19
IOSchedulingClass=idle
TimeoutStartSec=120

View File

@ -0,0 +1,10 @@
[Unit]
Description=Ingest app error reports every 5 minutes
[Timer]
OnBootSec=2min
OnUnitActiveSec=5min
Persistent=true
[Install]
WantedBy=timers.target

View File

@ -7,7 +7,9 @@
# 1. every released library's tag for its current version is on origin
# (sickdays, sync_settings_ui: the apps' pins resolve to the code on main);
# 2. the sickdays CLI + launcher run the checkout's code, via
# sickdays/install.sh (its venv installs sickdays editable from here).
# sickdays/install.sh (its venv installs sickdays editable from here);
# 3. the app-errors ingest timer is active and runs this checkout's code,
# via app_errors/install.sh.
#
# Run it from the live checkout (~/src/utils), never from a worktree: install.sh
# installs editable from its own directory, and a worktree path would point the
@ -80,12 +82,29 @@ check_sickdays() {
fi
}
# The ingest timer must be active and its venv must import this checkout.
check_app_errors() {
local module
if ! systemctl --user is-active --quiet app-errors-ingest.timer; then
bad "app-errors-ingest.timer is not active"
return
fi
module="$("$HOME/.local/share/app-errors/venv/bin/python" -I -c 'import app_errors; print(app_errors.__file__)' 2>/dev/null || true)"
if [[ "$module" != "$UTILS_DIR/app_errors/"* ]]; then
bad "the app-errors venv imports ${module:-nothing}, not $UTILS_DIR/app_errors"
else
log "ok: app-errors ingest runs $module"
fi
}
verify() {
check_tag sickdays sickdays
check_tag sync_settings_ui sync_settings_ui
check_tag game-recording game_recording
check_tag godot-dev-tools godot_dev_tools
check_tag app-errors app_errors
check_sickdays
check_app_errors
if [[ "$FAILED" -ne 0 ]]; then
log "health check FAILED"
exit 1
@ -97,6 +116,8 @@ main() {
if [[ "$VERIFY_ONLY" -eq 0 ]]; then
log "installing sickdays from $UTILS_DIR/sickdays"
"$UTILS_DIR/sickdays/install.sh"
log "installing the app-errors ingest from $UTILS_DIR/app_errors"
"$UTILS_DIR/app_errors/install.sh"
fi
verify
}