testsAndMisc/docs/superpowers/evidence/install-plagiarism-tools-split-2026-08-18.json

{
  "intent": "Bring install_plagiarism_tools.sh under the repo's 250-line cap without changing its installed behavior, per NEXT_SESSION_PROMPT.md §3 (the first of three restructures, chosen first for being 'most mechanical').",
  "scope": [
    "linux_configuration/scripts/single_use/utils/install_plagiarism_tools.sh (entry script, kept: banner, color/log helpers, source lines, print_summary, main)",
    "linux_configuration/scripts/single_use/utils/lib/plagiarism_python.sh (new: stage 1, Python NLP venv + check_plagiarism.py/plagcheck install)",
    "linux_configuration/scripts/single_use/utils/lib/plagiarism_sherlock.sh (new: stage 2, Sherlock clone-or-fallback)",
    "linux_configuration/scripts/single_use/utils/lib/plagiarism_optional.sh (new: stages 3-4, Ferret/WCopyfind availability checks)",
    "linux_configuration/scripts/single_use/utils/check_plagiarism.py (new: extracted from a 'cat <<PYEOF' heredoc, now CLI-only after a second split)",
    "linux_configuration/scripts/single_use/utils/plagiarism_similarity.py (new: comparison engine split out of check_plagiarism.py to also fit under 250 lines)",
    "linux_configuration/scripts/single_use/utils/sherlock.py (new: extracted from the fallback path's 'cat <<SHERLOCKEOF' heredoc)",
    "linux_configuration/scripts/single_use/utils/lib/tests/plagiarism_harness.sh + test_plagiarism_python.sh + test_plagiarism_sherlock.sh + test_plagiarism_optional.sh + test_install_plagiarism_tools.sh (new: PATH-shim test suite)",
    "Not in scope: §3's other two restructures, §4, and any of the 14 still-over-cap files listed as out of scope in NEXT_SESSION_PROMPT.md §3."
  ],
  "changes": [
    "Wrapped the four top-level install stages (Python NLP tools, Sherlock, Ferret check, WCopyfind check) into named functions and moved each into its own lib/plagiarism_*.sh sibling, sourced by the entry script -- matching the existing install_exercism.sh + lib/exercism_config.sh convention in the same directory.",
    "Extracted the two embedded Python heredocs into real .py files (check_plagiarism.py, sherlock.py) installed via `install -m` instead of `cat <<EOF`, matching download_nltk_data.py's prior extraction in the same directory. check_plagiarism.py then needed a second split (into plagiarism_similarity.py) to also clear 250 lines, since the file-length gate covers .py.",
    "Made both nltk/sklearn (check_plagiarism.py/plagiarism_similarity.py) and PyPDF2/python-docx (conditionally, inside read_file) dynamic imports via importlib.import_module, so the modules stay importable and lintable (ruff/mypy/pylint clean) in this repo's environment, which deliberately does not have the plagiarism toolchain's runtime deps installed.",
    "Converted every print() call in check_plagiarism.py/sherlock.py to sys.stdout.write() to satisfy this repo's T201 ruff rule (select=ALL), after asking the user to confirm that approach over adding a per-file-ignore -- per code-quality.md, per-file-ignores need explicit sign-off every time.",
    "Fixed install_sherlock's use of a global $SCRIPT_DIR it was never passed: writing a harness test caught that the fallback-copy path silently failed to find sherlock.py when called with only $INSTALL_DIR in scope. Changed it to take script_dir as an explicit $1, matching install_python_nlp_tools' existing signature.",
    "Fixed lib/plagiarism_python.sh installing check_plagiarism.py without its new plagiarism_similarity.py dependency: caught by testing the installed output, not by inspection. Now both files are installed together.",
    "Wrote a PATH-shim test harness (fake python3/pip/git/java/wine) so every lib function and the entry script's main()/print_summary() run for real against recorded fakes rather than being mocked at the call-site level, per the user's explicit preference for this test seam.",
    "Committed the pre-existing unrelated CLAUDE.md -> AGENTS.md symlink rename and .gitignore update (cocoindex_code) as a separate first commit, per explicit user instruction to land tracked dirty files but leave untracked ones alone."
  ],
  "verification": [
    {
      "command": "bash meta/scripts/check_file_length.sh --all",
      "result": "pass",
      "evidence": "14 files over cap (was 15); install_plagiarism_tools.sh no longer listed."
    },
    {
      "command": "bash meta/scripts/shell_coverage.sh <test> plagiarism_python.sh / plagiarism_sherlock.sh / plagiarism_optional.sh / install_plagiarism_tools.sh",
      "result": "pass",
      "evidence": "plagiarism_python.sh 30/30=100.00%, plagiarism_sherlock.sh 17/17=100.00%, plagiarism_optional.sh 25/25=100.00%, install_plagiarism_tools.sh 57/57=100.00%."
    },
    {
      "command": "bash linux_configuration/scripts/single_use/utils/lib/tests/test_plagiarism_python.sh (and the 3 sibling test files)",
      "result": "pass",
      "evidence": "test_plagiarism_python.sh 15/15, test_plagiarism_sherlock.sh 11/11, test_plagiarism_optional.sh 9/9, test_install_plagiarism_tools.sh 12/12 -- 47 assertions total, 0 failed."
    },
    {
      "command": "python3 -m venv + pip install scikit-learn nltk PyPDF2 python-docx; then check_plagiarism.py doc1.txt doc2.txt doc3.txt --detailed and sherlock.py doc1.txt doc2.txt doc3.txt, doc1/doc2 identical, doc3 unrelated",
      "result": "pass",
      "evidence": "check_plagiarism.py reported doc1<->doc2 100.0% (flagged, detailed passages printed correctly) and both <->doc3 pairs 0.0%; sherlock.py reported the same 100.0%/0.0%/0.0% pattern with the SUSPICIOUS flag on the matching pair."
    },
    {
      "command": "shellcheck on every touched .sh (entry, 3 libs, harness, 4 test files)",
      "result": "pass",
      "evidence": "Zero warnings after fixing SC2097/SC2098 (a HOME=\"$HOME\" self-referential assignment in a test file, since removed as redundant)."
    },
    {
      "command": "ruff check / mypy --config-file meta/pyproject.toml / pylint --rcfile meta/pyproject.toml / bandit -c meta/pyproject.toml on check_plagiarism.py, plagiarism_similarity.py, sherlock.py",
      "result": "pass",
      "evidence": "ruff: All checks passed (all 3). mypy: Success, no issues (all 3). pylint: 10.00/10 (all 3). bandit: 0 issues, all severities/confidences (all 3, 315 lines scanned in check_plagiarism.py alone)."
    },
    {
      "command": "pre-commit run --files <full staged set>",
      "result": "pass",
      "evidence": "All hooks passed after two fixes caught by this run specifically: the git-index executable bit missing on 3 lib files + the harness (git add --chmod=+x), and this evidence/contract pair itself."
    }
  ],
  "risks": [
    "check_plagiarism.py/plagiarism_similarity.py/sherlock.py have no pytest coverage gate in this repo (only linux_configuration/tests/ and python_pkg/ are gated) -- correctness rests on the manual real-venv functional run recorded above, not an automated regression test. A future edit to these files could silently break plagcheck without pre-commit catching it.",
    "This installer was not run against the real system (no venv creation, no pip install, no git clone against the actual network) -- only against the PATH-shim harness and a separate throwaway venv for the Python-level functional check. A real end-to-end `bash install_plagiarism_tools.sh` run was not performed."
  ],
  "rollback": [
    "git revert the commit; the entry script and its libs/py files return to the single 534-line monolith with embedded heredocs.",
    "After rollback, confirm bash meta/scripts/check_file_length.sh --all reports install_plagiarism_tools.sh over cap again (534 lines), and that plagcheck/sherlock still work by re-running the same real-venv functional check documented above."
  ]
}