Commit Graph

10 Commits

Author SHA1 Message Date
8798105aa1 Grade against 3.14 too, not just run on it
The pre-commit workflow already installs `python-version: "3.x"`, but ruff was
still grading the archive against py310 -- so it kept withholding the pyupgrade
and type-checking fixes that only become safe at the newer target.
`pin_ci_python.py` rewrites the knob.

The 17 findings that appeared are what the stale target was hiding:
`datetime.timezone.utc` where `datetime.UTC` is now correct (UP017), and 14
imports used only in annotations that belong under `TYPE_CHECKING` (TC002,
TC003) -- safe here precisely because 3.14's PEP 649 defers annotation
evaluation, which is the same reason ruff would not offer them at py310.

`ruff check` and `ruff format --check` are clean again, so the zero-findings
state this repo reached earlier today holds at the new target.

Still contradictory and deliberately untouched: the four
`Programming/ERSMS-project/*/Dockerfile` files say `FROM python:3.10-slim`.
They build an archived coursework service against its own era's pinned
dependencies, and nothing in this repo runs them; bumping the base image would
be a rewrite of that project, not a lint-target change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SfnJmxxsZ534AKk16rC2Gr
2026-08-29 11:55:43 +02:00
42007d56e4 PORR runs in 28s, and three checks so the next pass cannot repeat my mistakes
PORR was the last course whose `run` did not work. Four measurements, not
guesses, explain why -- and none of them is the matrix size everyone assumes:

  ProcessingType.PROCESSES   20.9s at n=2, 32.8s at n=5, 78.2s at n=10. That
                             is process-pool setup PER TEST, and it grows.
  matrix_type poli3          densifies a sparse .mat into a 2.3 GB npz cache
                             (nemeth12: 723 MB) -- over the 2G ceiling before
                             any solving happens.
  n=10000                    800 MB matrix, and np.linalg.solve wants a copy.
  n=5000                     spd_5000.npz was a Git LFS pointer whose object
                             404s on GitHub, so it regenerates from scratch.

Remove those and what is left still exercises the whole Richardson
implementation at every size that fits: 18 tests, 28 seconds through the
harness, exit 0. The submitted parameter set is one documented environment
variable away. `main.py` now propagates pytest's status (verified: exit 5 when
nothing is collected) -- it discarded it before, so `run` could not tell a
passing suite from a failing one.

The rest of this commit is scripts/check_refactor_safety.py, which exists
because three regressions got past both ruff and a smoke run during this
campaign, and I would rather they could not get past the next one:

  --signatures      diffs two trees function by function, parameter by
                    parameter, default by default. This is what found the 22
                    EARIN functions whose defaults an annotation pass silently
                    dropped.
  --logging         a module with a __main__ guard that logs at INFO must call
                    basicConfig, or it prints nothing at all. Found two more
                    in PORR just now.
  --print-keywords  refuses to let an automated rewrite near print(x, end="")
                    or print(x, file=sys.stderr); losing those turns a
                    rendered board into one glyph per line, or silently moves
                    error output onto stdout.

All three pass on the whole archive. Also in here: 12 latent TypeErrors mypy
found in EARIN that ruff could not -- call sites still passing positionally to
parameters this campaign had made keyword-only, plus one caller of a function
whose argument I had removed. Every one would have raised at runtime.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
2026-08-28 20:50:46 +02:00
cc06bc0cd8 EARIN: zero findings, verified against byte-identical program output
749 findings across six labs and two projects. The mechanical half was
scripted; everything below is what the scripts got wrong or could not do, and
how it was caught.

Three regressions I introduced and then found by RUNNING the code, not by
reading it:

- Converting print() to logger.info() made the programs SILENT. logging's
  last-resort handler only emits WARNING and above, so with no basicConfig
  call `lab2/main.py -t 1 1` printed nothing at all. Fifteen files across the
  repo now configure logging where they log.
- The same conversion dropped `end=""`. lab2 draws its board one square at a
  time and lab1 draws the maze one cell at a time; both came out one glyph
  per line. Those thirteen call sites are sys.stdout.write again, and the
  rendered board now diffs byte-identically against the pre-change output.
- The B007 fix renamed `col` to `_col` in the FIRST loop it matched, which
  was print_letters -- where the variable IS used. That would have been a
  NameError on the first call.

And one the scripts caused silently: annotating 110 signatures dropped the
DEFAULT VALUES from 22 of them, so `preprocessing(a, b, USER_THRESHOLD,
ANIME_THRESHOLD)` would have raised TypeError. Every signature is now diffed
against the pre-change version, name by name and default by default: zero
mismatches.

Real defects the lint surfaced:

- `alpha_beta` ended with an unreachable `return None` that violated its own
  return type. Its maximising and minimising halves were mirror images, so
  they are one `_search_branch` now (C901 11 -> ok).
- `get_data`'s `gpu` switch, `predict`'s `auto`, `create_model`'s
  `rows_number` and five of `preprocess_model_predict`'s fifteen parameters
  were never read. Gone, with their call sites.
- np.random.* -> Generator throughout (NPY002), `.values` -> `.to_numpy()`,
  `datetime.now()` with a tzinfo, `inplace=True` removed, `except:` given a
  type, `random.*` -> `SystemRandom` (S311).
- print_board's three nested helpers became methods (C901 17 -> ok);
  user_input's nine-branch if-ladder became a table (C901 12 -> ok).

Verification, all by running: lab1 `-t` and lab2 `-t 1 1` produce output
byte-identical to the pre-change code; lab2's rendered 8x8 board diffs clean;
lab3's argument parsing returns the same dict for the same argv; lab4 fails
identically before and after (no sklearn here). Also included: the logging
configuration for NLP, MOM and ECRYPT_PROJECT, which had the same silence bug.

Repo-wide: 2,775 findings -> 1,883.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
2026-08-28 20:39:53 +02:00
bd24dd57ea Hold the coursework to the same lint bar as the tooling
kuhy overruled the "do not fix coursework to be better than it was submitted"
rule: git history is the archive, so the working tree can be clean. This lands
the config and the mechanical half of the cleanup -- 9,281 ruff findings under
the fleet config down to 3,686, with no per-file-ignores anywhere.

What is in here:

- pyproject.toml, the first one in this repo. select = ["ALL"], mypy strict,
  and the vendored list mirrored into ruff exclude and coverage omit. Five
  rules are off and each says why; CPY001 is the only judgement call, because
  stamping a copyright header onto 137 university assignments would assert a
  licence that was never claimed. No hook is wired up yet -- a gate lands only
  once its subject is clean.

- ruff format + the safe fixes, then a second pass over the docstring rules
  ruff marks unsafe (they only edit docstring text). 108 files.

- TRAK/sightpy is vendored, not coursework: it is rafael-fuente/Python-Raytracer
  (BSD 3-Clause). Verified against a fresh upstream clone with line endings
  normalised -- 30 of 32 files byte-identical, ray.py and scene.py carrying a
  12- and 13-line local patch that counts rays for the report. 35 files that
  should never have been linted, now excluded and restored.

- WDWR, SPD, MOM and ECRYPT are fully clean, by hand and verified by running
  them: WDWR's three plot scripts print the same statistics as before, MOM's
  PuLP model solves to the same cost. WDWR's graph.py had a 342-character CSV
  header embedded as a string literal, which cannot be wrapped -- the data
  moved to data/pareto_front.csv. Two 0-byte .py stubs are deleted; an empty
  file is not a deliverable and git still has it.

Three harness bugs found by actually running things:

- `run` now forces MPLBACKEND=Agg. TRAK finished its ray trace in 37 seconds
  and then sat for 45 minutes: plt.show() had opened an image viewer and was
  waiting for someone to close the window.
- TRAK's main.py only pops that viewer with --show now, and its runner writes
  to a gitignored build/ via the new --entry-args. The default output path is
  outputs/output.png, which is TRACKED -- a plain run overwrote a submitted
  deliverable.
- course_lib.sh was 357 lines, over this repo's own cap. Split into
  course_probe.sh and course_run.sh.

Verified: ECOTE 26 tests pass, ECRYPT_PROJECT 99 pass, TRAK renders
cornell_box in 37s and exits 0, every module compiles, and every import ruff
removed was checked to be genuinely unused.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
2026-08-28 19:53:04 +02:00
ffa5dad02a Split 19 over-cap LaTeX reports, proving each PDF is unchanged
The 250-line cap applies to prose too, and the reports were the worst
offenders -- ENUME's projectA.tex alone was 1,876 lines. Each is now a
preamble plus one \input{} per section, which is how a long LaTeX document is
normally organised; not one byte of prose was rewritten.

"The PDF is unchanged" is checked by a script rather than by eye, because
hashing the PDF cannot work: pdflatex stamps a /CreationDate and an /ID into
every build, so two runs of the SAME source already differ. tex_fingerprint.sh
compares what a reader sees instead -- page count plus a hash of the extracted
text -- and it is stable across rebuilds of untouched sources, which is the
property that makes it usable as a gate.

It earned its keep immediately. Both ECOTE reports quote an example LaTeX
document inside an lstlisting block, so \end{document} appears in the middle
of the file; splitting on the FIRST one stranded everything after it. Same
page count, different text, caught automatically. EOPSY lab3 exposed a second
bug: a \section{} title that runs onto the next line.

Result: 19 .tex over the cap -> 1. The survivor,
ELAC/.../03-building-nfa-...tex, is 418 lines of consecutive tikz figures
with no subsection to split on and needs a hand-placed boundary.

Verified: 29 splits across two passes (section, then subsection for the
pieces still over), every one reporting `unchanged` from
scripts/tex_fingerprint.sh --check.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
2026-08-28 19:47:12 +02:00
a8d873b4fd Run all 33 safe courses; fix what that broke
Some checks are pending
md-naming / markdown naming convention (push) Waiting to run
pre-commit / pre-commit (push) Waiting to run
runnability / runnability (push) Waiting to run
Running one directory (EADS) had already produced four bugs, so all of them
were run. 11 failed. Every one was real -- either in the coursework or in the
harness I had just written -- and the split was roughly even.

Coursework bugs:
  - 15 .tex files use Polish ogonek characters with no fontenc, so pdflatex
    dies with "Command \k unavailable in encoding OT1". Fixed where it was
    actually needed, not blanket-applied.
  - BSS had \documentclass{article}[12pt] -- options after the class name,
    where they are not options at all.
  - SDM's beamer deck had a stray } on an \item, which took out the frame.
  - EOOP/Various/currency included "currency2.hpp"; the file is currency.hpp.
  - ECRYPT_PROJECT had the SAME stdlib-shadowing bug as ECOTE: a package named
    code/. Renamed to sieve/, and its 99 tests now run. With ECOTE's 26 that
    is 125 tests that could not execute at all before today.

Harness bugs, which the run is what exposed:
  - COURSE_ENTRY pointed at directories for PORR and ECRYPT_PROJECT, so python
    looked for __main__.py and reported the wrong problem.
  - main.py was auto-picked via find's first result. EARIN has six labs and
    six main.py files, so that ran an arbitrary lab and called it the course.
    Now only auto-picked when exactly one exists.
  - Only the first requirements.txt was installed, which is why EARIN still
    died on gymnasium after a venv was created for it. All of them now.
  - The notebook probe accepted jupyter being on PATH. This machine's jupyter
    is a pipx shim whose interpreter is gone, so "runnable" was a lie; it now
    invokes jupyter --version and PBAD/twm_4 report blocked, honestly.
  - Nothing ran a course's tests. Where a suite exists it is now the check,
    which matters most for ECRYPT_PROJECT, whose program prompts on stdin and
    cannot be run headlessly at all.
  - Which directory pytest runs from is course-specific and unguessable, so
    stubs pass --test-dir: ECOTE needs program/ for imports, ECRYPT_PROJECT
    needs sieve/ because its fixtures open a CSV by relative path.

gates/known-incomplete.txt records the one file that genuinely cannot build:
EOOP/Various/set/set.cpp includes set.hpp, which `git log --all` shows has
never existed here. The course supplied it and only the .cpp was submitted.
That is a factual record with a stated check, not a lint allowlist -- and
fabricating the header would be inventing coursework.

Status is now 36 runnable / 5 no-code / 3 blocked.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
2026-08-28 18:52:31 +02:00
f7763bb1cd Add a run.sh to every course, and fix the code that would not build
All 44 directories now probe: 38 runnable, 5 no-code, 1 blocked (ENUME needs
MATLAB or Octave, neither of which is on this machine).

The runners are stubs over one shared scripts/course_lib.sh rather than 44
copies of the logic, for the same reason the gates live in ~/utils. A stub
passes its kind as ARGUMENTS -- course_main --kind python --entry ... -- "$@"
-- rather than setting globals, because globals a linter cannot see being read
mean SC2034 on every one of them, and the only cure would be 44 disables.

Fixed along the way:
  - Generating the stubs overwrote Programming/PSD/run.sh, the original
    coursework runner. Restored, and given a --probe early exit instead so it
    is otherwise untouched: it installs packages with sudo and starts Kafka,
    which is exactly what a probe must not trigger.
  - The probe guard only recognised runners containing the literal --probe, so
    the new stubs (which delegate to the library) all read unprobed. It now
    also accepts delegation to course_lib.sh.
  - Blocked probes reported "unknown" because _require_any prints the tool it
    found, so probes redirected its stdout -- swallowing the failure reason
    too. Split into _probe_any, and ENUME now says exactly what it needs.
  - shellcheck runs with -x so it follows the sourced library; without it every
    stub reports SC1091 and the only alternative is a per-file disable. The
    library itself carries `# shellcheck shell=bash` because it has no shebang
    on purpose -- it is sourced, never executed.

Then EADS was actually run, and three of its six C++ files did not compile:
  - lab3.cpp: copyDictionary was declared taking Dictionary* but its body used
    Node members, and the copy constructor called copyTree(), which does not
    exist. The copy also dropped key and balanceFactor. Now compiles AND runs,
    printing its AVL tests.
  - MidTerm/task2.cpp: a const mismatch between declaration and definition, a
    stray semicolon before a function body, findNode calling a Tree method on
    a treeNode*, and a while loop that never advanced. sumNodeSub also had no
    base case -- unreachable before, because the const mismatch meant its body
    was never compiled.
These compiled years ago because older GCC did not diagnose uninstantiated
template bodies. Fragments with no main() are now type-checked with -c rather
than linked, since two of these files are exam answers, not programs.

Binaries go to a gitignored build/ -- an extensionless executable dropped next
to its source is not something .gitignore catches by name.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
2026-08-28 18:37:55 +02:00
50263f075a Add pre-commit and fix everything it found
`pre-commit run --all-files` is green. The config only carries hooks whose
subject is already clean, so it stays green at every commit -- file-length,
ruff/mypy and dependency-freshness are listed as blocked, each on its own
cleanup, and the file-REWRITING hooks (trailing-whitespace, end-of-file-fixer)
are held back for their own reviewed commit.

Turning it on found four real things:

1. A committed private key: ERSMS-project/nginx/ssl/nginx.key. It is a
   self-signed CN=localhost cert that had already expired (notAfter
   2025-06-16), so the risk is nil -- but it is regenerable in one openssl
   call, so it is replaced by generate-cert.sh rather than excluded from the
   hook. The script says plainly that removing it from HEAD does NOT
   unpublish it: history is deliberately not rewritten, which is acceptable
   only because nothing ever trusted this key.

2. Four files with unresolved merge conflict markers committed, left over
   from the subtree merges. EARIN/.gitignore and .vscode/extensions.json are
   resolved as the union of both sides. The two Unity folder .meta files each
   had two candidate GUIDs; grepping showed neither is referenced anywhere
   (folder GUIDs rarely are), so either is safe and the HEAD side is kept --
   an arbitrary choice, but a checked one rather than a guess.

3. 62 shellcheck findings across 11 scripts, all fixed, no disables added.
   The interesting pattern was `cd X; cmd &; PID=$!; cd ..`, which trips both
   SC2164 and SC2103 and cannot use a plain subshell without losing the PID.
   `( cd X && exec cmd ) &` keeps the directory, the PID and the semantics.
   EOPSY lab1's submitted solution was verified by RUNNING it afterwards:
   a->A, B->b, foldera->FOLDERA, FOLDERB->folderb, exactly as before.

4. check-json failing on 11 files, which was the wrong parser rather than a
   defect -- tsconfig*.json and .vscode/*.json are JSONC, where TypeScript
   and VS Code both allow comments. Those two path classes are excluded and
   every other .json is still checked. Excluding them is what surfaced the
   merge conflict in (2) as the one genuine failure.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
2026-08-28 18:21:22 +02:00
d407527863 Untrack 189 regenerable artifacts and close the silent-ignore trap
Untracked (--cached, so they stay on disk): 76 LaTeX artifacts, 7 .xdv XeTeX
intermediates, 18 -eps-converted-to.pdf from epstopdf, 32 files of .NET
obj/Debug/net6.0 output including three copies of lab2.dll, 16 .class/.o, the
9 MB Nx cache with its native .node, and an entire committed Python
virtualenv at ECOTE/program/ecote (python 3.10.10, 18 files).

The existing 407-line LaTeX .gitignore was NOT unused as I first assumed --
it already covered *.aux and *.class, and the artifacts simply predate it. So
this appends the categories it missed (*.o, obj/, .venv/, .nx/, target/)
rather than rewriting something that was working.

The second half is the more interesting bug. Auditing with

    git ls-files | git check-ignore --stdin --no-index

found 130 files that were tracked AND ignored. That combination fails
silently: `git add` on such a file is not an error, the edit just never
lands. Causes were dist/ swallowing the vendored jQuery/Bootstrap bundle,
.vscode/ and .idea/, and -- the good one -- the LaTeX template's man-page
patterns *.[1-9] and *.[1-9][0-9] matching EPFU's date-named report
directories, so doc/21.11/ read as a man page section and took report.pdf
and seven screenshots with it.

32 of those 130 turned out to be more build output and were untracked too;
the rest got targeted ! overrides. The first attempt used a blanket
!NotProgramming/EMISY/** which re-enabled that course's own LaTeX artifacts,
so it is narrowed to the *.1 schematic exports that actually needed it.
TRAK's nested .gitignore excluded the .vscode DIRECTORY, which stops git
descending, so no root-level ! can reach inside -- that one had to be fixed
at its source.

Both invariants now verified: 0 tracked files ignored, and 0 of the 189
untracked artifacts leaked back into view.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
2026-08-28 17:35:38 +02:00
be962d0c35 Add 'Programming/EARIN/' from commit 'a09c96dd651633b20dc0703840308e2e0d8376df'
git-subtree-dir: Programming/EARIN
git-subtree-mainline: 635e287095
git-subtree-split: a09c96dd65
2026-02-06 22:15:23 +01:00