PORR was the last course whose `run` did not work. Four measurements, not
guesses, explain why -- and none of them is the matrix size everyone assumes:
ProcessingType.PROCESSES 20.9s at n=2, 32.8s at n=5, 78.2s at n=10. That
is process-pool setup PER TEST, and it grows.
matrix_type poli3 densifies a sparse .mat into a 2.3 GB npz cache
(nemeth12: 723 MB) -- over the 2G ceiling before
any solving happens.
n=10000 800 MB matrix, and np.linalg.solve wants a copy.
n=5000 spd_5000.npz was a Git LFS pointer whose object
404s on GitHub, so it regenerates from scratch.
Remove those and what is left still exercises the whole Richardson
implementation at every size that fits: 18 tests, 28 seconds through the
harness, exit 0. The submitted parameter set is one documented environment
variable away. `main.py` now propagates pytest's status (verified: exit 5 when
nothing is collected) -- it discarded it before, so `run` could not tell a
passing suite from a failing one.
The rest of this commit is scripts/check_refactor_safety.py, which exists
because three regressions got past both ruff and a smoke run during this
campaign, and I would rather they could not get past the next one:
--signatures diffs two trees function by function, parameter by
parameter, default by default. This is what found the 22
EARIN functions whose defaults an annotation pass silently
dropped.
--logging a module with a __main__ guard that logs at INFO must call
basicConfig, or it prints nothing at all. Found two more
in PORR just now.
--print-keywords refuses to let an automated rewrite near print(x, end="")
or print(x, file=sys.stderr); losing those turns a
rendered board into one glyph per line, or silently moves
error output onto stdout.
All three pass on the whole archive. Also in here: 12 latent TypeErrors mypy
found in EARIN that ruff could not -- call sites still passing positionally to
parameters this campaign had made keyword-only, plus one caller of a function
whose argument I had removed. Every one would have raised at runtime.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
749 findings across six labs and two projects. The mechanical half was
scripted; everything below is what the scripts got wrong or could not do, and
how it was caught.
Three regressions I introduced and then found by RUNNING the code, not by
reading it:
- Converting print() to logger.info() made the programs SILENT. logging's
last-resort handler only emits WARNING and above, so with no basicConfig
call `lab2/main.py -t 1 1` printed nothing at all. Fifteen files across the
repo now configure logging where they log.
- The same conversion dropped `end=""`. lab2 draws its board one square at a
time and lab1 draws the maze one cell at a time; both came out one glyph
per line. Those thirteen call sites are sys.stdout.write again, and the
rendered board now diffs byte-identically against the pre-change output.
- The B007 fix renamed `col` to `_col` in the FIRST loop it matched, which
was print_letters -- where the variable IS used. That would have been a
NameError on the first call.
And one the scripts caused silently: annotating 110 signatures dropped the
DEFAULT VALUES from 22 of them, so `preprocessing(a, b, USER_THRESHOLD,
ANIME_THRESHOLD)` would have raised TypeError. Every signature is now diffed
against the pre-change version, name by name and default by default: zero
mismatches.
Real defects the lint surfaced:
- `alpha_beta` ended with an unreachable `return None` that violated its own
return type. Its maximising and minimising halves were mirror images, so
they are one `_search_branch` now (C901 11 -> ok).
- `get_data`'s `gpu` switch, `predict`'s `auto`, `create_model`'s
`rows_number` and five of `preprocess_model_predict`'s fifteen parameters
were never read. Gone, with their call sites.
- np.random.* -> Generator throughout (NPY002), `.values` -> `.to_numpy()`,
`datetime.now()` with a tzinfo, `inplace=True` removed, `except:` given a
type, `random.*` -> `SystemRandom` (S311).
- print_board's three nested helpers became methods (C901 17 -> ok);
user_input's nine-branch if-ladder became a table (C901 12 -> ok).
Verification, all by running: lab1 `-t` and lab2 `-t 1 1` produce output
byte-identical to the pre-change code; lab2's rendered 8x8 board diffs clean;
lab3's argument parsing returns the same dict for the same argv; lab4 fails
identically before and after (no sklearn here). Also included: the logging
configuration for NLP, MOM and ECRYPT_PROJECT, which had the same silence bug.
Repo-wide: 2,775 findings -> 1,883.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
384 findings to none across ten scripts. Most of it is annotation and
docstring work, but the lint surfaced two defects that had nothing to do with
style:
- `gpt_chunks.py` stripped its GPT preamble with
`input_string.strip("iSTS chunks:")`. str.strip takes a SET OF CHARACTERS,
not a prefix, so it also ate any leading i/S/T/s/c/h/u/n/k/space/colon:
"iSTS chunks: sentence one" came out as "entence one", silently losing the
first letter of every chunked sentence. Now removeprefix. ruff found it as
B005 + PLE1310.
- `is_numbered_line_format` had `for string in string` in its list branch,
which raises NameError the moment a list is passed (F821).
- `processing.test_XML()` read a module-level `alignments_data` that does not
exist at module scope; it only ever ran when pasted into a live session. It
takes the data as an argument now and is renamed `log_first_alignment` so
pytest stops collecting a non-test as a test.
Also real rather than cosmetic:
- The OpenAI key was a placeholder string literal in three files, one of them
carrying a "Replace me with os.environ['API_KEY']" comment. All three read
OPENAI_API_KEY now and say so when it is missing -- as committed, none of
these scripts could actually run.
- `rating.rate_similarity` was submitted unfinished: it built a client, never
touched its two arguments and returned None despite `-> str`. It now makes
the same call the rest of the package makes.
- The two 2 kB alignment prompts and one 344-character chunking prompt moved
to prompts/*.txt. Wrapping them in source would have put newlines into the
middle of the prompt.
- `format_alignments` parsed GPT's replies with `eval()`; the file holds one
repr()'d string per line, so ast.literal_eval is both sufficient and safe.
- NLP/__init__.py deleted -- it made the course directory a package named
"NLP", which is an invalid module name (N999) and which nothing imports;
the scripts use plain `import processing`.
INP001 is settled the same way throughout: a coursework file run as
`python x.py` gets a shebang and the executable bit, which is what it is,
rather than an __init__.py that would make a course code into a module name.
Verified: every module compiles and `./run.sh run Programming/NLP` is clean.
The scripts themselves need the SemEval corpus and a paid API key, so they
are not executed here.
Repo-wide: 3,159 findings -> 2,775.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
kuhy chose the strict branch: no per-file-ignores at all, not even the S101
block that ruff's own docs and the rest of this fleet carry for test code. So
125 test assertions were rewritten to raise instead of assert.
Rather than 125 copies of `if x != y: raise AssertionError(...)`, each course
grows a small expect module -- expect_equal / expect_in / expect_true /
expect_false. That is also strictly better output than a bare assert here:
pytest's assertion rewriting does not reach into a helper, so the helper has
to print both values, and it does. The rewrite itself went through ast, not a
regex, because seven of ECOTE's asserts were wrapped over several lines.
Everything else in the two courses is gone as well:
- 66 argument and 37 return annotations on the translator, worked out from
what each function actually returns rather than stamped on as Any --
`list[str] | str` where a function returns "Error!" or the parsed columns.
- 16 prints became logger.error with %-style arguments. The messages still
reach stderr with no basicConfig, via logging's last-resort handler, so the
translator behaves exactly as before.
- Two `open()` calls became Path.open / Path.write_text, six RET504
assign-then-return pairs collapsed, one 380-character expected-HTML literal
split into implicitly concatenated pieces (it cannot be wrapped any other
way, which is what ISC001 being off is for), and one block of commented-out
code deleted.
- ECRYPT_PROJECT/__init__.py removed. It made the COURSE directory a package,
which nothing imports, and it was the sole cause of `N999 Invalid module
name: 'ECRYPT_PROJECT'`; with it gone, sieve is the top-level package and
its name is fine.
Verified by running both, not by reading: ECOTE 26 tests pass and
`python -m translator.main --filename examples/complicated/table1.tex` still
writes the same HTML with the same three column-count warnings on stderr;
ECRYPT_PROJECT 99 tests pass.
Repo-wide: 3,686 findings -> 3,159.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
kuhy overruled the "do not fix coursework to be better than it was submitted"
rule: git history is the archive, so the working tree can be clean. This lands
the config and the mechanical half of the cleanup -- 9,281 ruff findings under
the fleet config down to 3,686, with no per-file-ignores anywhere.
What is in here:
- pyproject.toml, the first one in this repo. select = ["ALL"], mypy strict,
and the vendored list mirrored into ruff exclude and coverage omit. Five
rules are off and each says why; CPY001 is the only judgement call, because
stamping a copyright header onto 137 university assignments would assert a
licence that was never claimed. No hook is wired up yet -- a gate lands only
once its subject is clean.
- ruff format + the safe fixes, then a second pass over the docstring rules
ruff marks unsafe (they only edit docstring text). 108 files.
- TRAK/sightpy is vendored, not coursework: it is rafael-fuente/Python-Raytracer
(BSD 3-Clause). Verified against a fresh upstream clone with line endings
normalised -- 30 of 32 files byte-identical, ray.py and scene.py carrying a
12- and 13-line local patch that counts rays for the report. 35 files that
should never have been linted, now excluded and restored.
- WDWR, SPD, MOM and ECRYPT are fully clean, by hand and verified by running
them: WDWR's three plot scripts print the same statistics as before, MOM's
PuLP model solves to the same cost. WDWR's graph.py had a 342-character CSV
header embedded as a string literal, which cannot be wrapped -- the data
moved to data/pareto_front.csv. Two 0-byte .py stubs are deleted; an empty
file is not a deliverable and git still has it.
Three harness bugs found by actually running things:
- `run` now forces MPLBACKEND=Agg. TRAK finished its ray trace in 37 seconds
and then sat for 45 minutes: plt.show() had opened an image viewer and was
waiting for someone to close the window.
- TRAK's main.py only pops that viewer with --show now, and its runner writes
to a gitignored build/ via the new --entry-args. The default output path is
outputs/output.png, which is TRACKED -- a plain run overwrote a submitted
deliverable.
- course_lib.sh was 357 lines, over this repo's own cap. Split into
course_probe.sh and course_run.sh.
Verified: ECOTE 26 tests pass, ECRYPT_PROJECT 99 pass, TRAK renders
cornell_box in 37s and exits 0, every module compiles, and every import ruff
removed was checked to be genuinely unused.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
The 250-line cap applies to prose too, and the reports were the worst
offenders -- ENUME's projectA.tex alone was 1,876 lines. Each is now a
preamble plus one \input{} per section, which is how a long LaTeX document is
normally organised; not one byte of prose was rewritten.
"The PDF is unchanged" is checked by a script rather than by eye, because
hashing the PDF cannot work: pdflatex stamps a /CreationDate and an /ID into
every build, so two runs of the SAME source already differ. tex_fingerprint.sh
compares what a reader sees instead -- page count plus a hash of the extracted
text -- and it is stable across rebuilds of untouched sources, which is the
property that makes it usable as a gate.
It earned its keep immediately. Both ECOTE reports quote an example LaTeX
document inside an lstlisting block, so \end{document} appears in the middle
of the file; splitting on the FIRST one stranded everything after it. Same
page count, different text, caught automatically. EOPSY lab3 exposed a second
bug: a \section{} title that runs onto the next line.
Result: 19 .tex over the cap -> 1. The survivor,
ELAC/.../03-building-nfa-...tex, is 418 lines of consecutive tikz figures
with no subsection to split on and needs a hand-placed boundary.
Verified: 29 splits across two passes (section, then subsection for the
pieces still over), every one reporting `unchanged` from
scripts/tex_fingerprint.sh --check.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
`./run.sh run Programming/ZPOB` exited 12. A clean-room rebuild (the previous
failure was reading a cached .aux) shows the real error is bibtex, not LaTeX:
I found no \citation commands---while reading file esej.aux
I found no \bibstyle command---while reading file esej.aux
esej.tex ends with \bibliography{references} but never cites anything and
never sets a \bibliographystyle. latexmk therefore runs bibtex, bibtex exits
2, and latexmk reports 12 -- after pdflatex has already written a correct
esej.pdf.
Adding \bibliographystyle would make a References section appear that is not
in the deliverable: the submitted esej.pdf (LuaTeX, 2023-12-12, 2 pages) has
no bibliography either, so bibtex never succeeded for the author. The fix
belongs in the harness, so course_lib grows --no-bibtex and ZPOB's stub says
in full why it passes it.
Verified: `rm -rf esej/build && ./run.sh run Programming/ZPOB` exits 0 and
writes esej/build/esej.pdf.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
Running the courses made this machine stutter during a game. That was me:
PORR's suite solves systems up to 10000x10000 and takes every core it is
given, and the pip installs are not cheap either.
A promise to be careful is not a fix, so `run` now executes inside a transient
systemd scope -- MemoryMax=2G, MemorySwapMax=0 (OOM-kill rather than thrash
swap), CPUQuota=200%, nice -n 19, ionice -c 3. Same mechanism the pre-push
hook in testsAndMisc uses. Verified from inside the scope: memory.max reads
2147483648 and cpu.max reads "200000 100000", so the kernel is enforcing it
rather than the script hoping.
RUN_MEMORY_MAX and RUN_CPU_QUOTA override it for a single deliberate run.
Recorded the two courses that still fail: PORR, which will now be OOM-killed
under the cap and needs a smaller default parameter set, and ZPOB, where
latexmk exits 12 from bibtex despite producing the PDF.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
Running one directory (EADS) had already produced four bugs, so all of them
were run. 11 failed. Every one was real -- either in the coursework or in the
harness I had just written -- and the split was roughly even.
Coursework bugs:
- 15 .tex files use Polish ogonek characters with no fontenc, so pdflatex
dies with "Command \k unavailable in encoding OT1". Fixed where it was
actually needed, not blanket-applied.
- BSS had \documentclass{article}[12pt] -- options after the class name,
where they are not options at all.
- SDM's beamer deck had a stray } on an \item, which took out the frame.
- EOOP/Various/currency included "currency2.hpp"; the file is currency.hpp.
- ECRYPT_PROJECT had the SAME stdlib-shadowing bug as ECOTE: a package named
code/. Renamed to sieve/, and its 99 tests now run. With ECOTE's 26 that
is 125 tests that could not execute at all before today.
Harness bugs, which the run is what exposed:
- COURSE_ENTRY pointed at directories for PORR and ECRYPT_PROJECT, so python
looked for __main__.py and reported the wrong problem.
- main.py was auto-picked via find's first result. EARIN has six labs and
six main.py files, so that ran an arbitrary lab and called it the course.
Now only auto-picked when exactly one exists.
- Only the first requirements.txt was installed, which is why EARIN still
died on gymnasium after a venv was created for it. All of them now.
- The notebook probe accepted jupyter being on PATH. This machine's jupyter
is a pipx shim whose interpreter is gone, so "runnable" was a lie; it now
invokes jupyter --version and PBAD/twm_4 report blocked, honestly.
- Nothing ran a course's tests. Where a suite exists it is now the check,
which matters most for ECRYPT_PROJECT, whose program prompts on stdin and
cannot be run headlessly at all.
- Which directory pytest runs from is course-specific and unguessable, so
stubs pass --test-dir: ECOTE needs program/ for imports, ECRYPT_PROJECT
needs sieve/ because its fixtures open a CSV by relative path.
gates/known-incomplete.txt records the one file that genuinely cannot build:
EOOP/Various/set/set.cpp includes set.hpp, which `git log --all` shows has
never existed here. The course supplied it and only the .cpp was submitted.
That is a factual record with a stated check, not a lint allowlist -- and
fabricating the header would be inventing coursework.
Status is now 36 runnable / 5 no-code / 3 blocked.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
Two invariants had been established by hand and left ungated, which is the
"a remedy that only works if someone remembers" pattern:
- scripts/check_ignore_consistency.sh fails the commit if any TRACKED file
is also gitignored. .binary-allowlist used to *instruct* the reader to
re-run that check; now it is enforced. Verified both ways -- passes on the
tree, and fails when an ignored file is force-added.
DOCS-runnability.md now really has two columns. Each environment writes only
its own gates/runnability-<env>.tsv and the doc renders from both, so a CI run
cannot clobber the local results. The ci tsv is gitignored: committing a copy
generated here would label this machine's capabilities as the runner's.
Rendering it surfaced a classic bug -- awk's NR==FNR idiom silently swallows
the SECOND file when the first is empty, so the table came out with a header
and no rows. It compares FILENAME instead.
runnability.yml deliberately does NOT diff against a committed baseline. A
status is a property of the machine, so `runnable` locally and `blocked` on a
runner is correct rather than drift, and diffing would make the workflow red
by construction. It fails on `error` instead -- a runner that crashes or
reports an unknown kind.
Then five more courses were actually run, because EADS alone had produced four
real bugs and one sample proves nothing:
- ECOAR and EPFU build (nasm + 32-bit gcc, and gcc respectively).
- ECOTE's 17 test modules could not run AT ALL. Its package is named `code/`,
which shadows the standard library: pytest's debugging plugin imports pdb,
pdb imports code, and it got this directory instead, so the suite died
with an INTERNALERROR before collecting a single test. Renamed to
translator/ with all 17 imports updated -- 26 tests now pass.
- A LaTeX report builds. That one exposed a hazard rather than a bug:
latexmk wrote its PDF beside the source, and in a course where the built
name matches the submitted one it would have silently overwritten a
tracked deliverable. Both the LaTeX and C++ runners now build into a
gitignored build/, and rebuilding ECOGR leaves `git status` empty.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
The remaining work is much larger than what has landed, and vague about it
would be worse than useless. Numbers are measured, not guessed:
- 9,281 ruff findings under the fleet config (select = ALL). ~3,000 are
auto-fixable and ~6,000 are hand work, dominated by 1,671 missing type
annotations and ~600 missing docstrings.
- 19 tests for ~28k lines of authored code, across 12 languages.
- 73 files still over the 250-line cap, 27 of which are blocked on the
decision parked in TODO-file-length-250.md.
- 22 dependency manifests, none pinned.
Also records the ordering rules that are easy to lose: the dependency upgrade
and its gate must land in the same commit; source files must not be split
before their tests exist; and the file-rewriting hooks are held back so that
touching hundreds of submitted coursework files is its own reviewed change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
All 44 directories now probe: 38 runnable, 5 no-code, 1 blocked (ENUME needs
MATLAB or Octave, neither of which is on this machine).
The runners are stubs over one shared scripts/course_lib.sh rather than 44
copies of the logic, for the same reason the gates live in ~/utils. A stub
passes its kind as ARGUMENTS -- course_main --kind python --entry ... -- "$@"
-- rather than setting globals, because globals a linter cannot see being read
mean SC2034 on every one of them, and the only cure would be 44 disables.
Fixed along the way:
- Generating the stubs overwrote Programming/PSD/run.sh, the original
coursework runner. Restored, and given a --probe early exit instead so it
is otherwise untouched: it installs packages with sudo and starts Kafka,
which is exactly what a probe must not trigger.
- The probe guard only recognised runners containing the literal --probe, so
the new stubs (which delegate to the library) all read unprobed. It now
also accepts delegation to course_lib.sh.
- Blocked probes reported "unknown" because _require_any prints the tool it
found, so probes redirected its stdout -- swallowing the failure reason
too. Split into _probe_any, and ENUME now says exactly what it needs.
- shellcheck runs with -x so it follows the sourced library; without it every
stub reports SC1091 and the only alternative is a per-file disable. The
library itself carries `# shellcheck shell=bash` because it has no shebang
on purpose -- it is sourced, never executed.
Then EADS was actually run, and three of its six C++ files did not compile:
- lab3.cpp: copyDictionary was declared taking Dictionary* but its body used
Node members, and the copy constructor called copyTree(), which does not
exist. The copy also dropped key and balanceFactor. Now compiles AND runs,
printing its AVL tests.
- MidTerm/task2.cpp: a const mismatch between declaration and definition, a
stray semicolon before a function body, findNode calling a Tree method on
a treeNode*, and a while loop that never advanced. sumNodeSub also had no
base case -- unreachable before, because the const mismatch meant its body
was never compiled.
These compiled years ago because older GCC did not diagnose uninstantiated
template bodies. Fragments with no main() are now type-checked with -c rather
than linked, since two of these files are exam answers, not programs.
Binaries go to a gitignored build/ -- an extensionless executable dropped next
to its source is not something .gitignore catches by name.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
Two things kept CI from being green, and neither was the code.
First, GitHub Actions was DISABLED at the repository level. The workflows
registered fine and simply never ran -- `gh api .../actions/permissions`
returned {"enabled": false}. Now enabled.
Second, once it ran, shellcheck failed on a file that is clean locally:
SC2236 in PSD/zin3/python/run_temperature_system.sh. `language: system` hooks
use whatever is on PATH, and Ubuntu's packaged shellcheck is not the 0.11.0
this machine has, so the build went red for a difference in tool version
rather than in the code. That is the failure mode testsAndMisc pins tools to
avoid, so the workflow now installs shellcheck v0.11.0 and verifies its
SHA256 -- checked here first: the digest matches and the archive really does
unpack to shellcheck-v0.11.0/shellcheck.
The `! -z` -> `-n` fix is applied too. Pinning stops the next version drift;
fixing the code means this instance is right under either version.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
Two workflows, one per gate, each checking out kuhyx/utils into .utils and
calling the shared script -- the pattern every other repo here uses.
pre-commit.yml sets UTILS_ROOT to that checkout, without which the shims fail
with "shared gate not found", and both use fetch-depth 1 because the history
is ~494 MB and no gate reads it.
install.sh ends by verifying and exits non-zero if the gate cannot run; a
warning there would just recreate the bug it exists to prevent, since this
repo had no hooks at all until now. Confirmed both ways: --check exited 1
while the hooks were missing, then 0 once installed.
run.sh separates `status` (probe, seconds, safe in CI) from `run <dir>`
(actually executes one project). Conflating them is not hypothetical -- the
first version of the probe did exactly that and it cost 120 seconds and a
scare. Some course dirs already ship a run.sh from years ago, and
Programming/PSD/run.sh installs packages with sudo and starts Kafka; calling
it with an unknown --probe flag does not probe anything, it runs the project.
It blocked on a sudo prompt rather than installing (verified against
/var/log/pacman.log and the absence of a yay clone), but only by luck.
So probe_one now invokes a runner ONLY if it declares --probe, wraps it in
`timeout 20s` as defence in depth, and reports anything else as unprobed
without touching it. All 44 directories now probe instantly.
DOCS-runnability.md is generated with a per-environment column. Recording one
global status would make the CI check red by construction, since `runnable`
locally and `blocked` on ubuntu-latest is the normal case for the MATLAB,
Unity and .NET directories rather than a drift signal.
Per-directory run.sh files are the next step; all 44 currently read unprobed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
`pre-commit run --all-files` is green. The config only carries hooks whose
subject is already clean, so it stays green at every commit -- file-length,
ruff/mypy and dependency-freshness are listed as blocked, each on its own
cleanup, and the file-REWRITING hooks (trailing-whitespace, end-of-file-fixer)
are held back for their own reviewed commit.
Turning it on found four real things:
1. A committed private key: ERSMS-project/nginx/ssl/nginx.key. It is a
self-signed CN=localhost cert that had already expired (notAfter
2025-06-16), so the risk is nil -- but it is regenerable in one openssl
call, so it is replaced by generate-cert.sh rather than excluded from the
hook. The script says plainly that removing it from HEAD does NOT
unpublish it: history is deliberately not rewritten, which is acceptable
only because nothing ever trusted this key.
2. Four files with unresolved merge conflict markers committed, left over
from the subtree merges. EARIN/.gitignore and .vscode/extensions.json are
resolved as the union of both sides. The two Unity folder .meta files each
had two candidate GUIDs; grepping showed neither is referenced anywhere
(folder GUIDs rarely are), so either is safe and the HEAD side is kept --
an arbitrary choice, but a checked one rather than a guess.
3. 62 shellcheck findings across 11 scripts, all fixed, no disables added.
The interesting pattern was `cd X; cmd &; PID=$!; cd ..`, which trips both
SC2164 and SC2103 and cannot use a plain subshell without losing the PID.
`( cd X && exec cmd ) &` keeps the directory, the PID and the semantics.
EOPSY lab1's submitted solution was verified by RUNNING it afterwards:
a->A, B->b, foldera->FOLDERA, FOLDERB->folderb, exactly as before.
4. check-json failing on 11 files, which was the wrong parser rather than a
defect -- tsconfig*.json and .vscode/*.json are JSONC, where TypeScript
and VS Code both allow comments. Those two path classes are excluded and
every other .json is still checked. Excluding them is what surfaced the
merge conflict in (2) as the one genuine failure.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
Five files were outside README / CLAUDE* / DOCS* / TODO*, and two were over
the 250-line cap. Both gates now pass repo-wide.
Renames: the two lowercase readme.md files, and three documents into DOCS-*.
The AIS requirements doc split at its own boundary into functional (101) and
non-functional (174). The WDWR report split into six numbered sections behind
an index that keeps the original filename, 38-235 lines each; 753 lines of
body plus the 18-line header account for all 771 original lines, so nothing
was dropped.
That WDWR file needed checking before splitting: it sits next to main.tex and
main.pdf and looked like a conversion of them. It is not -- main.tex is
project 25406 and the markdown is project 18402, a different document. So it
is authored content and the split is real work, not churn on a build artifact.
Filled the three empty READMEs and replaced ECOTE's, which was a stale copy of
the root README still claiming the group projects were excluded -- they were
merged in by subtree some time ago.
EOPSY/lab1 turned out not to be the code-free directory the survey suggested:
it has modify.sh and modify_examples.sh. The oddly-named empty files beside
them are the exercise's OUTPUT, committed so the before/after state is
visible, and its README now says so rather than leaving them looking like junk.
CLAUDE.md follows the testsAndMisc structure. Writing it caught a nice one:
the md-naming marker is a biconditional, so quoting the literal removal string
in a non-TODO file fails the gate. The gate was right; the text is rephrased.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
621 binaries are tracked here and they are supposed to be: 155 PDFs, 275
PNGs, 68 JPGs, the submitted reports and figures. Decision 3 keeps them and
skips the history rewrite, so a blanket block would be wrong. What the gate
is for is stopping NEW junk.
So the allowlist is scoped by path rather than by extension alone. Documents,
figures and datasets are allowed under Programming/ and NotProgramming/;
compiled output (.o .so .a .exe .dll .dylib .class .pyc) is allowed nowhere
and has no entry at all; and the repo root, gates/, scripts/ and .github/
have no entry either, so a binary dropped in the tooling directories is
blocked. The .tgz entries are load-bearing -- EOPSY ships its simulators as
ftp/{sched,memory}.tgz and `make setup` extracts them.
Verified three ways, because a gate that passes everything is vacuous:
- all 2749 tracked files -> exit 0
- probe.exe at root, probe.png in gates/ -> BLOCKED, exit 1
- a real submitted PDF -> exit 0
plus shellcheck clean.
The script is a copy of testsAndMisc's, which is where this gate lives today
-- it is not in the shared ~/utils set. The header says so and says to move
it to ~/utils rather than let the two drift, since a hand-copied gate is what
produced four forks of the 250-line cap.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
Untracked (--cached, so they stay on disk): 76 LaTeX artifacts, 7 .xdv XeTeX
intermediates, 18 -eps-converted-to.pdf from epstopdf, 32 files of .NET
obj/Debug/net6.0 output including three copies of lab2.dll, 16 .class/.o, the
9 MB Nx cache with its native .node, and an entire committed Python
virtualenv at ECOTE/program/ecote (python 3.10.10, 18 files).
The existing 407-line LaTeX .gitignore was NOT unused as I first assumed --
it already covered *.aux and *.class, and the artifacts simply predate it. So
this appends the categories it missed (*.o, obj/, .venv/, .nx/, target/)
rather than rewriting something that was working.
The second half is the more interesting bug. Auditing with
git ls-files | git check-ignore --stdin --no-index
found 130 files that were tracked AND ignored. That combination fails
silently: `git add` on such a file is not an error, the edit just never
lands. Causes were dist/ swallowing the vendored jQuery/Bootstrap bundle,
.vscode/ and .idea/, and -- the good one -- the LaTeX template's man-page
patterns *.[1-9] and *.[1-9][0-9] matching EPFU's date-named report
directories, so doc/21.11/ read as a man page section and took report.pdf
and seven screenshots with it.
32 of those 130 turned out to be more build output and were untracked too;
the rest got targeted ! overrides. The first attempt used a blanket
!NotProgramming/EMISY/** which re-enabled that course's own LaTeX artifacts,
so it is narrowed to the *.1 schematic exports that actually needed it.
TRAK's nested .gitignore excluded the .vscode DIRECTORY, which stops git
descending, so no root-level ! can reach inside -- that one had to be fixed
at its source.
Both invariants now verified: 0 tracked files ignored, and 0 of the 189
untracked artifacts leaked back into view.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
First step of bringing this archive up to the fleet standard: nothing can be
linted until the gates know what is not kuhy's code.
gates/vendored.txt is the source of truth, one path per line with the reason
AND how it was verified, because an unverified entry is a suppression wearing
a costume. Seven subtrees: the ASP.NET scaffolder's jQuery/Bootstrap bundle
(~98% of all JS and CSS here), stock Unity package and template assets,
manufacturer-supplied Altium/PADS component libraries for the OLED and
HMC5883L parts, and the two EOPSY simulator trees.
The EOPSY entries were the one judgement call, and they turned out not to be:
work/ is the extraction target of `make setup`, and extracting the committed
ftp/*.tgz and diffing shows 14/14 java files byte-identical to the department
originals. ControlPanel.java (1852 lines) and Kernel.java (511) are therefore
not student work and are not split.
Deliberately NOT vendored: EMISY's README, declaration and report are kuhy's
own writing; E-PSYOPS Assets/Scripts (12 .cs) is authored; and the Unity Input
System binding is already exempt via its auto-generated banner, so it needs no
entry. Narrow entries beat one blanket directory.
TODO-file-length-250.md parks the one thing this mechanism cannot express: 27
data .txt files whose lines are long enough that the cap reads them as prose.
They resist directory-prefix exemption because the SemEval corpus dirs mix the
official distribution with kuhy's own GPT outputs, and WDWR/projekt/R holds
both scenarios.txt and the authored model.R. That blocks only the installation
of the file-length gate, which is several milestones away.
Mirrored into the shared gate as kuhyx/utils@a4a3e0a.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
