testsAndMisc/docs/superpowers/evidence/usage-report-parsing-split-2026-08-17.json

{
  "intent": "Split _usage_report_parsing.py (425 lines, over the 250 cap) by lifting the atop subprocess and native-helper machinery into its own module, verified by the 100%-coverage tests that landed in the preceding commit.",
  "scope": [
    "_usage_report_parsing.py 425 -> 243 (record parsing, epoch windows, per-day merging)",
    "New _usage_report_atop.py 206 (running atop, streaming its output, building/caching atop_agg, reading its TSV)",
    "Import sites updated in usage_report.py's caller chain and _usage_report_render.py",
    "Test files re-split to mirror the source and stay under the cap",
    "Non-goal: any behaviour change"
  ],
  "changes": [
    "Moved _run, _atop_read_cmd, _iter_atop_lines, _atop_agg_binary, _apply_native_name, _window_from_native and _aggregate_atop_native into _usage_report_atop.py, with the five constants they own.",
    "_usage_report_render.py imported _run from _usage_report_parsing; it now imports from _usage_report_atop. That is the same re-export-barrel trap caught in the previous split - fixing it in the same commit this time rather than a follow-up.",
    "usage_report.py needed no edit: it imports aggregate_atop and the three merge_* functions, all of which stayed in the parsing module.",
    "Re-homed test_atop_agg_binary_missing_source_falls_back from test_usage_report_modes.py into the new build-focused test file, so it sits with the other four _atop_agg_binary cases rather than referencing a constant that had moved out from under it.",
    "Split the two new test files, which were themselves over the cap at 373 and 272 lines, into five: atop_io (subprocess entry points), atop_build (binary build/cache), atop_pipeline (the atop | atop_agg TSV read), parse_records (record parsing) and aggregate (the pure-Python fallback)."
  ],
  "verification": [
    {
      "command": "AST comparison of every moved function against HEAD",
      "result": "pass",
      "evidence": "All 16 functions report IDENTICAL - _run, _atop_read_cmd, _iter_atop_lines, _parse_name, _parse_prc, _parse_prm, _window_from_epochs, _atop_agg_binary, _apply_native_name, _window_from_native, _aggregate_atop_native, aggregate_atop, _fold_pid_aggregates, merge_proc_aggs, merge_gpu_aggs, merge_windows. 'compared: 16, mismatches: 0'. The comparison normalises through ast.parse(ast.unparse(node)), so it is blind to formatting but catches any logic change."
    },
    {
      "command": "pytest linux_configuration/tests/ --cov of both modules --cov-branch",
      "result": "pass",
      "evidence": "_usage_report_atop.py 94 stmts / 28 branches 100%; _usage_report_parsing.py 130 stmts / 34 branches 100%. Zero missed statements and zero partial branches in both, matching the single-file 100% from the previous commit. 179 passed, the same count as before the split."
    },
    {
      "command": "pytest --collect-only per new test file, summed",
      "result": "pass",
      "evidence": "atop_io 8 + atop_build 6 + atop_pipeline 4 + parse_records 14 + aggregate 5 = 37. That is the 36 tests added last commit plus the one re-homed from test_usage_report_modes.py, and the directory total of 179 is unchanged - so nothing fell outside the runner's glob."
    },
    {
      "command": "python usage_report.py --date <today> --top 1 --no-clipboard --no-update-state --quiet",
      "result": "pass",
      "evidence": "Produced a full report from today's real logs (32 atop samples). This run went through the native atop_agg helper - exactly the code path this commit extracted - so the import rewiring is proven at runtime, not just under mocks."
    },
    {
      "command": "bash ~/utils/scripts/check_file_length.sh --all",
      "result": "pass",
      "evidence": "39 over-cap files before, 38 after. _usage_report_parsing.py is off the list and none of the two source or five test files exceeds 250."
    },
    {
      "command": "pre-commit run --files <all changed files>",
      "result": "pass",
      "evidence": "Every hook passes including ruff, ruff format, mypy, bandit and the noqa blockers. Two mypy findings introduced by the mechanical test split were fixed properly - a _parse_prc test that landed in the wrong file was moved to the record-parsing one, and a Self annotation ruff introduced got its import."
    }
  ],
  "risks": [
    "Five test files where there were two makes the mapping from test file to source module less obvious. Each is named after the concern it covers and the module docstrings say which source module they exercise.",
    "_usage_report_atop.py is 206 lines - comfortably under the cap, but it holds two distinct concerns (running atop, and building the C helper). If it grows, the build half is the natural next seam; its tests are already isolated in test_usage_report_atop_build.py.",
    "The tests fake subprocess at the module seam, so they do not prove atop's real output format still matches the field indices. The end-to-end program run covers that, and it exercised the native path this time rather than the Python fallback."
  ],
  "rollback": [
    "git revert the commit. _usage_report_parsing.py returns to 425 lines with all sixteen functions, _usage_report_atop.py disappears, and the five test files collapse back into two.",
    "After rollback, confirm pytest linux_configuration/tests still reports 179 passed and that usage_report.py still generates a report. Note that reverting this while keeping the previous commit's tests is safe - those tests were written against the pre-split module."
  ]
}