mirror of
https://github.com/kuhyx/testsAndMisc.git
synced 2026-11-13 13:50:48 +01:00
Krzysztof kuhy Rudnicki
35ddf4114f
Four evidence files committed today cite mutation runs -- 18, 13, 17 and
17 killed -- that lived in a scratch directory. Evidence pointing at a run
nobody can repeat is not evidence, so the lists move into the repo behind a
spec-driven runner.
Mutations are literal find/replace strings in JSON, not sed expressions.
The shell version this replaces escaped each pattern twice and used | as
its field separator, which collided with || and reported six mutations as
SURVIVED when they had never been applied at all -- a false negative in the
one tool whose entire job is catching false negatives.
Two deliberate choices follow from that:
- a no-op (find matched nothing) counts as a FAILURE, not a skip. A stale
spec entry otherwise stops testing its behaviour without saying so.
- a timeout counts as killed: a mutation that hangs the suite has still
been detected by it.
The conversion itself exercised the first rule -- it reported 8 no-ops from
mangled escapes rather than quietly scoring them as killed. Each was fixed
against the real source line.
All four specs now reproduce their committed figures exactly, and the
working tree is byte-clean after every run.
Not wired into pre-commit: it runs the whole suite once per mutation, which
is minutes, not seconds.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011w2Pj4Ek3ZNMQVYcFHgXDY
|
||
|---|---|---|
| .. | ||
| fixtures | ||
| lib | ||
| run_all | ||
| _schema_validation.py | ||
| _session_autopsy_mutations.py | ||
| check_agent_contract.sh | ||
| check_ai_evidence.sh | ||
| check_append_only_sessions.sh | ||
| check_file_length.sh | ||
| check_no_binaries.sh | ||
| check_no_dufs_conf.sh | ||
| check_no_inline_python.sh | ||
| check_no_secrets.sh | ||
| check_polling_antipatterns.sh | ||
| check_python_location.sh | ||
| check_requirements_sync.py | ||
| ci_mirror.sh | ||
| disk_cleanup_check.sh | ||
| extract_shell_functions.py | ||
| init_agent_artifacts.sh | ||
| install_hooks.sh | ||
| makepkg-manual-dlagent.sh | ||
| mutate_session_autopsy_hooks.py | ||
| mutate_shell.py | ||
| pytest_changed_packages.py | ||
| run_prettier_capped.sh | ||
| run_with_heavy_lock.sh | ||
| shell_coverage_report.py | ||
| shell_coverage.sh | ||
| trace_shell_split.sh | ||
| validate_contract.py | ||
| validate_evidence.py | ||
| verify_shell_split.sh | ||