mirror of
https://github.com/kuhyx/testsAndMisc.git
synced 2026-11-11 18:34:04 +01:00
Krzysztof kuhy Rudnicki
11aea86920
Answers "what is billed on every turn regardless of use" by measurement instead of recall, which is the part earlier audits kept getting wrong. A surface is reported dead only when unused in BOTH windows. One earlier audit's seven-day dead list included six servers that had been used within thirty days, so a single window is not evidence. Shares are weighted with cache-read weight 0.1, so a cached surface is not billed as though it were re-read in full each turn. Verified by running it, not just by tests: the report renders over 44,311 turns, putting rules/ at 4,907 tokens (1.77%), MEMORY.md at 4,312 (1.55%), memories/ at 3,758 (1.35%), skill descriptions at 3,279 (1.18%) and CLAUDE.md at 2,057 (0.74%), with per-MCP 7d/30d call counts behind each verdict. The files were found untracked in the tree from an earlier session; every module is already under the 250-line cap, so nothing needed splitting, and the package was already at 100% branch coverage, so nothing needed new tests. This lands the instrument only -- no surface is unregistered on its say-so. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
17 lines
1.4 KiB
JSON
17 lines
1.4 KiB
JSON
{
|
|
"title": "Land the token_audit standing-cost checkup: surfaces, probe, unused, checkup",
|
|
"objective": "Answer 'what is billed on every turn regardless of use' with measurement instead of recall. Earlier audits answered this from memory and got it wrong -- notably a seven-day dead list that included six surfaces used within thirty days -- so the module pairs a two-window liveness test with a weighted share estimate.",
|
|
"acceptance_criteria": [
|
|
"python3 -m python_pkg.token_audit.checkup runs and prints the standing-cost report.",
|
|
"The whole token_audit package holds 100% branch coverage (116 tests).",
|
|
"Every file stays under the repo's 250-line cap.",
|
|
"A surface is only reported dead when unused in BOTH the short and long window, never on a single window.",
|
|
"pre-commit passes on all eight files with no suppression added."
|
|
],
|
|
"out_of_scope": [
|
|
"Acting on the report -- unregistering any MCP server, skill or rule file. This lands the instrument, not its conclusions.",
|
|
"The shell-coverage instrument work committed separately in 81191f40 and 29b8c5a7."
|
|
],
|
|
"verifier": "timeout 120 python3 -m python_pkg.token_audit.checkup (expects a rendered report), plus python3 -m coverage run --branch --source=python_pkg/token_audit -m pytest python_pkg/token_audit/tests (expects 116 passed and 100% with zero partial branches), plus pre-commit run --files on the eight files."
|
|
}
|