Four follow-ups to the previous commit, three of them cases where the new
runner still did the thing it was written to stop doing.
A build that was already up to date ran nothing. `make` says "up to date" on
the second invocation, no executable changes mtime, the before/after snapshot
diff is empty -- and the runner printed a line and stopped, which is exactly
the complaint that started this. It now falls back to running the executables
that are there. Proved with a scratch course whose makefile has a real file
target, because both make courses in this repo rebuild unconditionally and
neither can reach the branch.
One broken target aborted the rest. `run <dir> all` stopped at the first
failure under set -e, so ELAC's second document was never attempted. Failures
are collected per target and reported together at the end instead.
An index was ignored when a course had one target: `run ECOAR 5` silently ran
target 1. Selections are validated before the single-target shortcut now.
And "ECOAR has 1 targets" read like a bug in the tool.
That last fix then falsified a claim in TODO-refactor-remaining.md. It said
ELAC's projectA and AIS's report/final/ver1 "fail with latexmk exit 12" and
were broken documents. With build/ deleted first, both courses build every
document and exit 0. The exit 12 reproduced once against pre-existing build
state and never from clean; the note now says that, and says which part is
inference rather than re-verified, because the original failing state is gone.
All 44 probes still produce a byte-identical runnability-local.tsv.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
Reported by kuhy: "I run run.sh and I have no idea where the actual program is
and how to run it". In EPFU that was one `gcc -Wall -Wextra -pedantic main.c`
line and nothing else -- the makefile has no -o, so the binary is an a.out that
nothing mentions, and it was never executed.
Two separate failures, both fixed.
Building is not running. make, cxx, maven and dotnet built and stopped; make,
maven and dotnet did not even print where the output went, while cxx and tex
did. They all execute what they produced now. `make` is the awkward one,
because a makefile need not say what it writes: the runner snapshots the
executables either side of the build and runs whatever is new.
A course is not always one program. Twelve of them are not -- EPFU a game and
a lab, EOOP six binaries, EARIN eight labs, MOM three reports. With no target
named, an interactive run prints a numbered menu and asks; `run <dir> 2`, a
path substring, or `all` skips the prompt. Enumeration deliberately scans the
course root rather than --entry, because EPFU pinning --entry to penguins/src
is precisely why labs/krudnic3_lab1.c had never been compiled. --entry is the
default now, not the only.
Interactive by default: EPFU's penguins and ECOAR's puzzle both read stdin,
and you typed `run`, so you are at the keyboard. --batch (and any
non-terminal stdin) means no prompt, stdin from /dev/null and a 60s cap per
program, so an unattended run cannot wedge. --probe never reaches any of this
and does not even enumerate targets: all 44 probes produce a byte-identical
gates/runnability-local.tsv, and `status` is still seconds long.
Running things immediately found two bugs that could not have surfaced before,
both recorded in TODO-refactor-remaining.md rather than fixed here, since both
mean touching a submitted .c:
- EPFU/labs/krudnic3_lab1.c dies with SIGFPE, integer division by zero.
- ECOAR's puzzle.out writes dest.bmp beside its source, so the tree guard
fails the run. That guard is doing exactly its job.
Verified: EPFU both targets, EOOP under a pty answering the prompt, EADS,
ECOAR, MOM. Full gate green, shellcheck clean on all 44 runners.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
The note in TODO-refactor-remaining.md said ERSMS's four were structural and
that a fix "would rewrite both models". Both halves were wrong.
flask-sqlalchemy 3.1.1 does ship py.typed; the real blocker was that
`class User(db.Model)` is attribute access, which mypy cannot resolve in a
base-class position.
backend/models.py now declares a named DeclarativeBase with SQLAlchemy 2.0
Mapped[] columns. The Model mixin goes away with the base, so the six legacy
Model.query call sites became db.session selects and the two table names are
spelled out rather than derived.
Verified by driving the real routes through Flask's test client with Firebase
stubbed out -- 16 scenarios covering every one of the six call sites, plus
both CREATE TABLE statements, byte-identical to the submitted version. That
harness caught the one thing review would not have: mapped_column(ForeignKey(
"user.uid")) silently infers VARCHAR(80) from the referenced column, which
had changed the submitted `user_id INTEGER`. An explicit Integer puts it back.
app.py went over the 250-line cap on the way, hence the split; models.py takes
the shebang + exec bit that NLP's nine import-only modules already use, since
__init__.py would make `from models import` unresolvable to mypy while it
still works at runtime. backend/Dockerfile copied only app.py, so it needed
the new file too.
PSD's two are pyflink's MapFunction and KeyedProcessFunction, which ship no
types and are not installed in that venv at all -- a hand-written .pyi would
assert an API nothing here can check. A [[tool.mypy.overrides]] scoped to that
one module turns off disallow_subclassing_any and nothing else.
ERSMS 4 -> 0, PSD 2 -> 0, under --strict. ruff and ruff format clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
`_works jupyter || _blocked "needs jupyter"` collapsed two different problems
into one message. "needs jupyter" and "jupyter is on PATH but does not run"
send you to different fixes -- install it, versus work out why the shim's
interpreter is gone -- and a status table is only worth keeping if it says
which. `_require_working` checks presence and then execution, and reports
them separately.
The wording is now what DOCS-runnability.md already said for twm_4 and PBAD,
except the probe produces it rather than someone having typed it. Regenerating
the table changed those two rows and nothing else, which is the useful part:
the run guard, the two new .venv directories under PSD and ERSMS-project, and
the tex entry-point change left all 44 courses' status exactly as it was.
One thing worth knowing for the next regeneration: the answer depends on PATH.
jupyter lives in ~/.local/bin, so a shell without it reports "needs jupyter"
and one with it reports the dead shim. Regenerate from a login shell.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The archive reached zero ruff findings this morning, so the standing rule in
.pre-commit-config.yaml -- a hook lands only once the thing it gates is clean
-- is finally satisfied for ruff. `ruff check` and `ruff format --check` are
hooks now, pinned at v0.16.5 per rules/dependency-freshness, plus
`no-suppressions`, which rejects `# noqa` and `# type: ignore`. CLAUDE.md has
always claimed a hook did that; until now the only thing stopping them was
whoever was reading ruff's output, and I typed `# noqa-free` three times in
this session by accident. mypy stays out: six of its errors are third-party
typing limits rather than our code.
The first attempt used `language: system`, which passed `pre-commit run
--all-files` and then failed the actual commit with "Executable `ruff` not
found" -- git hooks do not get ~/.local/bin on PATH. Taking the hooks from
astral-sh/ruff-pre-commit means pre-commit installs ruff itself, which is also
what makes the pin mean anything.
`ruff format --check` failed the moment it was wired up, on four TRAK files I
wrote today and never formatted. Formatted, and the ray_tracing0 render is
still 02484bdb6b6c1ddeff317d7e5566580f.
Then the run guard from f780d023 needed proving on more than TRAK, since it
now applies to every course kind. make, cpp and python courses pass it; node
and target and obj are all properly gitignored, so npm, maven and dotnet
builds cannot trip it either. But NotProgramming/MOM failed, and not because
of the guard:
`_first_file '*.tex'` used `find -print -quit`, which returns whatever readdir
hands back. After the 19 reports were split into sections, that was
`report_three/sections/02-model-dwukryterialny.tex`, and latexmk faithfully
tried to typeset a bare `\section` as a document. DOCS-runnability.md has
recorded MOM as runnable throughout. The tex branch now picks a file that
actually contains `\documentclass`.
Deliberately not sorted, and that is the interesting part. Sorting is the
obvious fix and it made things worse: it swapped ELAC's pick to projectA and
AIS's to report/final/ver1, both of which fail with latexmk exit 12. Those two
documents are broken independently, and sorting merely surfaced them; turning
two green courses red to fix a third is a bad trade. Recorded in
TODO-refactor-remaining.md instead. MOM, ELAC, AIS, SPD, ZPOB, SDM, DPZ and
ESOEN all exit 0 now.
`_works` also grew a `timeout 5`, because `unityhub --version` never returns
on this machine and a probe without one would hang the status sweep.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
156 ruff findings to 0. That takes the whole archive to 0, from 9,281.
Six of those findings were SQL injection, and they were the real ones. The
connector interpolated a username and an OAuth id straight out of the URL into
f-string SQL. With psycopg2 installed, the difference is not theoretical:
username = "x'; DROP TABLE users; --"
old: select * from users where username='x'; DROP TABLE users; --';
new: select * from users where username='x''; DROP TABLE users; --';
The old form is two statements and the second one runs. The new one is a
parameter, so the whole thing stays a string literal. Every query in that file
is parameterised now.
`get_recommendations` also called
`cursor.execute("... oauth_ID='{}'", oauth_ID)`, which is wrong twice over:
psycopg2 wants %s rather than {}, and its second argument must be a sequence,
not a bare string. That route could never have worked.
Ten blind `except Exception` handlers turned every programming error into a
500 with the message hidden. They are now the exceptions each block can
actually raise -- verified against the installed libraries, where
InvalidIdTokenError and ExpiredIdTokenError really are FirebaseError
subclasses -- and the handlers log a traceback instead of printing the
exception's str().
Three services bound `host="localhost"`. Each of them runs in a container with
a published port, on a compose bridge network, so binding loopback makes them
unreachable from both the host and their siblings. All four now read
ERSMS_HOST, which docker-compose sets to 0.0.0.0, and default to loopback when
run outside Docker. Flask's debug console executes arbitrary code, so that is
opt-in via FLASK_DEBUG too.
`movie_recommender.py` fitted the model at import: reading two CSVs and
building a 4803x4803 similarity matrix as a side effect of `import`. It failed
outright here, since the datasets are not committed. It is in `main()` now, and
the module imports cleanly. `error_decorator` was dead code that swallowed its
wrapped function's return value; it is gone.
Verified: all four modules import (the backend still needs its Firebase
credential JSON, which is not committed, exactly as before), the recommender's
helpers still pick the director out of a crew list, trim cast to three, fold
"Tom Hanks" to "tomhanks" and join a soup in the same order, and every route
parameter rename was matched by the URL rule it comes from.
Three of the four services are clean under mypy --strict. The backend keeps
four errors: flask-sqlalchemy builds `db.Model` dynamically, so `--strict`
refuses to subclass it. Migrating to a typed DeclarativeBase would rewrite both
models in an app that cannot be run here, so those four are reported in
TODO-refactor-remaining.md rather than hidden.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
152 ruff findings to 0 across twelve files, and mypy --strict is clean on all
of them but one -- see the end.
The course is marked unrunnable because it wants Kafka and Flink, so the
verification is what can be run without them. Ten of the twelve modules now
import cleanly against a venv with confluent-kafka, kafka-python and flask; the
two pyflink ones are skipped. That check did not exist before, because four of
these files WERE their own side effect:
- `kafka/generate_and_send.py` and `kafka/kafka_consumer.py` ran their
`while True:` loop at module level. Importing either never returns; I gave up
after twelve seconds of rdkafka connection failures.
- `zin1/producent.py` built a KafkaProducer at module level and raised
NoBrokersAvailable on import.
- `flink/anomaly_detector.py` called `env.execute()` at module level.
Each is a `main()` behind a `__main__` guard now, so importing one is free and
running it is unchanged.
`webinterface.py` called `json.loads` and never imported json, so the first
message on the topic would have raised NameError. mypy then found three more
of the same shape: confluent-kafka's `msg.error()` and `msg.value()` both
return Optional, and the code dereferenced them straight away. They are
narrowed to locals now, which also stops `msg.error()` being called twice per
message. A malformed alarm with no timestamp is skipped with a warning instead
of dividing None by 1000.
Two SLF001 findings were reads of `KafkaError._PARTITION_EOF`. That leading
underscore is confluent-kafka's own naming and there is no public alias, but
`KafkaError.name()` is public and returns exactly that string -- checked
against the installed library, where code() is -191 and name() is
"_PARTITION_EOF". Flask's `debug=True` is now opt-in via FLASK_DEBUG, because
the debug console executes arbitrary code.
The ten `random` calls are simulation, not crypto, so S311 is not a real
vulnerability -- but `secrets.SystemRandom()` is a drop-in with the same
methods and no rule to argue with, so that is what they use.
Behaviour is checked, not assumed: the sensor still returns a location in
1..10 and a temperature in -20..40, the thresholds still classify -15 as too
cold, 40 as too hot and 20 as normal, transactions still carry a four-digit
card id, and TemperatureReading survives a to_dict/from_dict round trip.
**Two mypy errors remain and are deliberate.** `--strict` implies
`disallow_subclassing_any`, and the Flink job must subclass pyflink's
MapFunction and KeyedProcessFunction, which ship no types. Fixing it means
weakening strict globally or writing an override for our own module. Neither is
worth it, so the two are reported in TODO-refactor-remaining.md rather than
hidden.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
115 ruff findings to 0. All three notebooks' non-source content hashes the
same before and after -- c7660e3e5e3b8d4ffc008d7f, b918553ae0cf91cd56fa4091
and 3f09c77e3edae1727e3781e2 -- so every stored output, execution count and
piece of metadata is exactly what was submitted.
Most of the work was 35 over-long lines, and they are over-long because they
are full Polish GUS column descriptions used as dict keys in .rename(). A key
like "udziały w podatkach stanowiących dochody budżetu państwa podatek
dochodowy od osób fizycznych" cannot be shortened, only wrapped, so a script
re-wrapped all of them with implicit concatenation inside parentheses -- the
one form both ruff's linter and its formatter accept.
A script doing 134 mechanical edits to strings that are database keys is
exactly the kind of change that silently drops a space. So it is checked
mechanically: an AST pass collects every string dict key in every code cell of
both versions and compares the multisets. 120 keys in data.ipynb, 9 in
main.ipynb, 5 in pbad.ipynb, all identical.
The other 27 findings were commented-out code (ERA001), and deleting it would
have thrown away what it recorded: which other operational programmes were
run, which axis limits the plots were read at, that MinMaxScaler was tried and
made no difference, that an earlier pass dropped a hand-maintained column list.
Each is now a sentence of prose saying that, and git history still has the
code. Two cells were nothing but commented-out scratch and are now just the
note.
Also real: five `inplace=True` calls became assignments, `zip()` in three
feature-importance loops got `strict=True`, `max(max(a), max(b))` became
`max(*a, *b)`, and 1.5 and 7 became DENSE_GMINA_THRESHOLD and
TERYT_GMINA_DIGITS -- the latter being "a full TERYT gmina code is seven
digits", which is worth saying once rather than leaving as a bare literal.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
228 ruff findings to 0 across the notebook, its partial export and twm.py, and
mypy --strict is clean on both .py files.
A notebook is a deliverable that happens to contain code, so the check that
mattered was not "does it still run" -- it needs a GPU, MNIST, CIFAR-10 and a
slice of Quick, Draw! to run at all -- but "is anything except the code still
exactly as submitted". Hashing every key of the .ipynb EXCEPT cell `source`
gives 60242e116d12549d11c93f7f before the first edit and after the last one,
across four passes and a `ruff format`. The file also round-trips byte-for-byte
through json.dumps(indent=1, ensure_ascii=False), which is what made editing
cells programmatically safe rather than a reformat of all 141 of them; the
diff touches 38.
print() became sys.stdout.write() rather than logging. In a notebook the print
IS the output, and a logger with no handler configured would have silently
emptied every cell -- the exact regression scripts/check_refactor_safety.py
was written for. matprint's `print(x, end=" ")` and bare `print()` are
preserved as writes with and without a trailing newline.
Both `show_samples` and `show_samples_rgb` took (indices, preds, images,
labels, count, names), which is six arguments and a mutable default. They now
take a frozen `Samples` holding the last four, which also removed the same
four-value tuple from seven call sites. Reviewing that diff is how I caught
that I had transcribed `real_score = preds[sample][real_id]` as
`samples.labels[sample]` -- a bug I introduced and then fixed before commit.
Other real things, not just rule satisfaction: `np.in1d` is removed in NumPy 2
(NPY201) and is now `np.isin(...).ravel()`; `test_model` computed
correct_indices and incorrect_indices and used neither; two cells grew `img`
inside a loop so it was only bound by the `else` branch; and the Quick, Draw!
downloader now builds a URL whose https scheme is a literal, which is what
S310 was asking for.
The README claimed the .py was "an nbconvert export of that notebook". It is
an export of the CIFAR-10 section only -- 28 of 81 code cells -- and has been
edited since. It says that now, and says the two are separate artifacts rather
than one being regenerable from the other.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Executing coursework has silently overwritten a submitted deliverable twice in
this campaign -- TRAK's outputs/output.png and SPD's results_*.png -- and both
times it was found by a later `git status`, not by the run. `course_main` now
snapshots what git sees under the course root before and after `_run`, and
fails with a diff naming the files if anything moved. Gitignored build output
does not show up there, which is why every runner already writes into build/.
Seen failing before being trusted: a stub whose `_run` appends a newline to
TRAK's config.ini exits 1 and names the file; the same stub without the append
exits 0; and a real `./run.sh run Programming/TRAK` still exits 0 after
rendering, because it writes into build/.
The notebook branch was the specific landmine. `nbconvert --execute --inplace`
rewrites the notebook it just ran, and all four notebooks here are tracked with
their outputs. It now executes to a temp --output-dir. That alone is not
enough, though: nbconvert runs the kernel with its cwd set to the notebook's
own directory, so a cell calling plt.savefig('x.png') still writes beside the
notebook. The tree check above is what actually catches that, which is why it
lives in course_main and applies to every course kind rather than to notebooks.
`_have jupyter` was also the wrong probe. jupyter on this machine is a pipx
shim whose interpreter is gone: `command -v` finds it, and running it exits
127. Both the probe and the runner now use `_works`, which runs `--version`.
`./Programming/twm_4/run.sh --probe` reports "needs jupyter" and exit 78
instead of failing confusingly mid-run.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
494 ruff findings to 0, and mypy --strict is clean too -- the first course in
this campaign where that is true of both.
Two real bugs turned up on the way, neither of which ruff can see:
- `main.py` passed `args.environment_map` to `ray_trace`, and argparse only
ever defined `--environment`. Every `--algorithm ray_tracing0` run died with
AttributeError before rendering a pixel, so `rendering.py` -- the largest
file in the course -- had no way to be exercised at all.
- `mouse_button` in the environment-map viewer assigned `mouse_last_x/y`
without declaring them `global`, so the drag origin went to locals and was
thrown away. Measured against HEAD: a 10x5 px drag turned the camera by
11.0/20.5 degrees because it measured from (0, 0); it is 1.0/0.5 now, which
is what the 0.1 degrees-per-pixel constant says it should be.
Fixing the first one is what made the rest safe. `ray_tracing0` has no RNG
anywhere in it, so it renders byte-identically every time, and the whole
refactor of `rendering.py` is verified by one md5 comparison:
02484bdb6b6c1ddeff317d7e5566580f, 248619 rays, before and after. The
cornell_box path cannot do that -- its ray count drifts by ~20 between
identical runs.
Three files were over the 250-line cap and are now split, geometry away from
the algorithm that uses it: rendering.py 462 -> 137 + raytracer.py,
photon_mapping.py 261 -> 160 + photon_geometry.py, and
code/photonmapping/python/main.py 257 -> 203 + geometry.py. No TRAK source
file is over the cap now.
The two photon mappers do use numpy's legacy global RNG, so NPY002 could not
be bit-identical. Each was done in two steps: restructure first, verified
identical under an external seed (digests 7220370f60206a62 and
21b4771504ab2351, and an identical 828 photon traces), then the swap to
`default_rng` on its own, checked against the before/after distributions.
Verified by running, not by reading:
- cornell_box through `./run.sh run Programming/TRAK`: 24.27s, 13294 rays
- all five scenes build after `from sightpy import *` became explicit imports
- the standalone photon mapper end to end: 81.4s, 38742 photon traces
- the EXR loader byte for byte: a0c3890350ce6101247c6948, 6291456 bytes
- the GLUT viewer under xvfb-run, which drew a real 1024x512 frame
`pyproject.toml` gains a mypy override for Imath, OpenEXR, OpenGL and
matplotlib. Those four ship neither stubs nor py.typed, and without it mypy
cannot import them at all and so reports nothing else about the file.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1,183 ruff findings left, in five courses. Nine are at zero. The numbers in
here were four hours stale.
Two things worth reading before the next pass: the notebook branch of
scripts/course_run.sh will overwrite tracked notebook outputs the moment it is
used (the same bug already found in TRAK and SPD), and eleven .py files are
now over the 250-line cap -- several because THIS work added the docstrings
and annotations that pushed them over.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
598 findings to none, and the suite ran through `./run.sh run` after each step
rather than at the end -- 18 passed, 27s, every time.
The shape of this one was different from the other courses. PORR is four
implementations of the same linear algebra (sequential, threads, processes,
dask), so nearly every finding came in groups of four:
- 229 missing annotations. Rather than `object`, which mypy strict rejects on
every attribute access, each module declares what these actually are:
`Matrix = np.ndarray | list[list[float]]` and the vector equivalent. The
backends really are called with both -- comparing them is the point of the
course.
- 76 names: `A` -> matrix, `Ab` -> augmented, `Ax` -> product, `I` ->
identity, `dA` -> dask_matrix, and the two CamelCase functions. Renamed
through `tokenize`, not regex, so no string or comment was touched.
- 121 missing docstrings, one per method name since the names repeat across
all four backends.
- Seven functions over the argument limit, all taking the same seven or nine
loose values. They are two frozen dataclasses now, in a new
richardson_problem.py: `Problem(matrix, rhs, lambda_min, lambda_max)` and
`Settings(max_iterations, x0, tol)`. `RichardsonMethod.__init__` also took a
`size` argument it never read.
- threads_indep_old's iteration did "split into chunks, submit, wait" four
times by hand (C901 11, PLR0915 54). One `_run_chunked` helper.
- tests.py's two bare asserts became explicit raises, per the no-ignores rule.
`assert False` was also B011: `python -O` deletes it, so the non-convergence
branch would have silently passed under optimisation.
Real fixes rather than style: np.random -> Generator, `os.path.exists` ->
Path, an `except Exception` around dask narrowed to the errors it actually
raises, and two runnable modules that logged at INFO without ever calling
basicConfig -- caught by scripts/check_refactor_safety.py, which is now doing
the job it was written for.
Repo-wide: 1,781 findings -> 1,183, all of them in the five courses left.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
PORR was the last course whose `run` did not work. Four measurements, not
guesses, explain why -- and none of them is the matrix size everyone assumes:
ProcessingType.PROCESSES 20.9s at n=2, 32.8s at n=5, 78.2s at n=10. That
is process-pool setup PER TEST, and it grows.
matrix_type poli3 densifies a sparse .mat into a 2.3 GB npz cache
(nemeth12: 723 MB) -- over the 2G ceiling before
any solving happens.
n=10000 800 MB matrix, and np.linalg.solve wants a copy.
n=5000 spd_5000.npz was a Git LFS pointer whose object
404s on GitHub, so it regenerates from scratch.
Remove those and what is left still exercises the whole Richardson
implementation at every size that fits: 18 tests, 28 seconds through the
harness, exit 0. The submitted parameter set is one documented environment
variable away. `main.py` now propagates pytest's status (verified: exit 5 when
nothing is collected) -- it discarded it before, so `run` could not tell a
passing suite from a failing one.
The rest of this commit is scripts/check_refactor_safety.py, which exists
because three regressions got past both ruff and a smoke run during this
campaign, and I would rather they could not get past the next one:
--signatures diffs two trees function by function, parameter by
parameter, default by default. This is what found the 22
EARIN functions whose defaults an annotation pass silently
dropped.
--logging a module with a __main__ guard that logs at INFO must call
basicConfig, or it prints nothing at all. Found two more
in PORR just now.
--print-keywords refuses to let an automated rewrite near print(x, end="")
or print(x, file=sys.stderr); losing those turns a
rendered board into one glyph per line, or silently moves
error output onto stdout.
All three pass on the whole archive. Also in here: 12 latent TypeErrors mypy
found in EARIN that ruff could not -- call sites still passing positionally to
parameters this campaign had made keyword-only, plus one caller of a function
whose argument I had removed. Every one would have raised at runtime.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
749 findings across six labs and two projects. The mechanical half was
scripted; everything below is what the scripts got wrong or could not do, and
how it was caught.
Three regressions I introduced and then found by RUNNING the code, not by
reading it:
- Converting print() to logger.info() made the programs SILENT. logging's
last-resort handler only emits WARNING and above, so with no basicConfig
call `lab2/main.py -t 1 1` printed nothing at all. Fifteen files across the
repo now configure logging where they log.
- The same conversion dropped `end=""`. lab2 draws its board one square at a
time and lab1 draws the maze one cell at a time; both came out one glyph
per line. Those thirteen call sites are sys.stdout.write again, and the
rendered board now diffs byte-identically against the pre-change output.
- The B007 fix renamed `col` to `_col` in the FIRST loop it matched, which
was print_letters -- where the variable IS used. That would have been a
NameError on the first call.
And one the scripts caused silently: annotating 110 signatures dropped the
DEFAULT VALUES from 22 of them, so `preprocessing(a, b, USER_THRESHOLD,
ANIME_THRESHOLD)` would have raised TypeError. Every signature is now diffed
against the pre-change version, name by name and default by default: zero
mismatches.
Real defects the lint surfaced:
- `alpha_beta` ended with an unreachable `return None` that violated its own
return type. Its maximising and minimising halves were mirror images, so
they are one `_search_branch` now (C901 11 -> ok).
- `get_data`'s `gpu` switch, `predict`'s `auto`, `create_model`'s
`rows_number` and five of `preprocess_model_predict`'s fifteen parameters
were never read. Gone, with their call sites.
- np.random.* -> Generator throughout (NPY002), `.values` -> `.to_numpy()`,
`datetime.now()` with a tzinfo, `inplace=True` removed, `except:` given a
type, `random.*` -> `SystemRandom` (S311).
- print_board's three nested helpers became methods (C901 17 -> ok);
user_input's nine-branch if-ladder became a table (C901 12 -> ok).
Verification, all by running: lab1 `-t` and lab2 `-t 1 1` produce output
byte-identical to the pre-change code; lab2's rendered 8x8 board diffs clean;
lab3's argument parsing returns the same dict for the same argv; lab4 fails
identically before and after (no sklearn here). Also included: the logging
configuration for NLP, MOM and ECRYPT_PROJECT, which had the same silence bug.
Repo-wide: 2,775 findings -> 1,883.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
384 findings to none across ten scripts. Most of it is annotation and
docstring work, but the lint surfaced two defects that had nothing to do with
style:
- `gpt_chunks.py` stripped its GPT preamble with
`input_string.strip("iSTS chunks:")`. str.strip takes a SET OF CHARACTERS,
not a prefix, so it also ate any leading i/S/T/s/c/h/u/n/k/space/colon:
"iSTS chunks: sentence one" came out as "entence one", silently losing the
first letter of every chunked sentence. Now removeprefix. ruff found it as
B005 + PLE1310.
- `is_numbered_line_format` had `for string in string` in its list branch,
which raises NameError the moment a list is passed (F821).
- `processing.test_XML()` read a module-level `alignments_data` that does not
exist at module scope; it only ever ran when pasted into a live session. It
takes the data as an argument now and is renamed `log_first_alignment` so
pytest stops collecting a non-test as a test.
Also real rather than cosmetic:
- The OpenAI key was a placeholder string literal in three files, one of them
carrying a "Replace me with os.environ['API_KEY']" comment. All three read
OPENAI_API_KEY now and say so when it is missing -- as committed, none of
these scripts could actually run.
- `rating.rate_similarity` was submitted unfinished: it built a client, never
touched its two arguments and returned None despite `-> str`. It now makes
the same call the rest of the package makes.
- The two 2 kB alignment prompts and one 344-character chunking prompt moved
to prompts/*.txt. Wrapping them in source would have put newlines into the
middle of the prompt.
- `format_alignments` parsed GPT's replies with `eval()`; the file holds one
repr()'d string per line, so ast.literal_eval is both sufficient and safe.
- NLP/__init__.py deleted -- it made the course directory a package named
"NLP", which is an invalid module name (N999) and which nothing imports;
the scripts use plain `import processing`.
INP001 is settled the same way throughout: a coursework file run as
`python x.py` gets a shebang and the executable bit, which is what it is,
rather than an __init__.py that would make a course code into a module name.
Verified: every module compiles and `./run.sh run Programming/NLP` is clean.
The scripts themselves need the SemEval corpus and a paid API key, so they
are not executed here.
Repo-wide: 3,159 findings -> 2,775.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
kuhy chose the strict branch: no per-file-ignores at all, not even the S101
block that ruff's own docs and the rest of this fleet carry for test code. So
125 test assertions were rewritten to raise instead of assert.
Rather than 125 copies of `if x != y: raise AssertionError(...)`, each course
grows a small expect module -- expect_equal / expect_in / expect_true /
expect_false. That is also strictly better output than a bare assert here:
pytest's assertion rewriting does not reach into a helper, so the helper has
to print both values, and it does. The rewrite itself went through ast, not a
regex, because seven of ECOTE's asserts were wrapped over several lines.
Everything else in the two courses is gone as well:
- 66 argument and 37 return annotations on the translator, worked out from
what each function actually returns rather than stamped on as Any --
`list[str] | str` where a function returns "Error!" or the parsed columns.
- 16 prints became logger.error with %-style arguments. The messages still
reach stderr with no basicConfig, via logging's last-resort handler, so the
translator behaves exactly as before.
- Two `open()` calls became Path.open / Path.write_text, six RET504
assign-then-return pairs collapsed, one 380-character expected-HTML literal
split into implicitly concatenated pieces (it cannot be wrapped any other
way, which is what ISC001 being off is for), and one block of commented-out
code deleted.
- ECRYPT_PROJECT/__init__.py removed. It made the COURSE directory a package,
which nothing imports, and it was the sole cause of `N999 Invalid module
name: 'ECRYPT_PROJECT'`; with it gone, sieve is the top-level package and
its name is fine.
Verified by running both, not by reading: ECOTE 26 tests pass and
`python -m translator.main --filename examples/complicated/table1.tex` still
writes the same HTML with the same three column-count warnings on stderr;
ECRYPT_PROJECT 99 tests pass.
Repo-wide: 3,686 findings -> 3,159.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
kuhy overruled the "do not fix coursework to be better than it was submitted"
rule: git history is the archive, so the working tree can be clean. This lands
the config and the mechanical half of the cleanup -- 9,281 ruff findings under
the fleet config down to 3,686, with no per-file-ignores anywhere.
What is in here:
- pyproject.toml, the first one in this repo. select = ["ALL"], mypy strict,
and the vendored list mirrored into ruff exclude and coverage omit. Five
rules are off and each says why; CPY001 is the only judgement call, because
stamping a copyright header onto 137 university assignments would assert a
licence that was never claimed. No hook is wired up yet -- a gate lands only
once its subject is clean.
- ruff format + the safe fixes, then a second pass over the docstring rules
ruff marks unsafe (they only edit docstring text). 108 files.
- TRAK/sightpy is vendored, not coursework: it is rafael-fuente/Python-Raytracer
(BSD 3-Clause). Verified against a fresh upstream clone with line endings
normalised -- 30 of 32 files byte-identical, ray.py and scene.py carrying a
12- and 13-line local patch that counts rays for the report. 35 files that
should never have been linted, now excluded and restored.
- WDWR, SPD, MOM and ECRYPT are fully clean, by hand and verified by running
them: WDWR's three plot scripts print the same statistics as before, MOM's
PuLP model solves to the same cost. WDWR's graph.py had a 342-character CSV
header embedded as a string literal, which cannot be wrapped -- the data
moved to data/pareto_front.csv. Two 0-byte .py stubs are deleted; an empty
file is not a deliverable and git still has it.
Three harness bugs found by actually running things:
- `run` now forces MPLBACKEND=Agg. TRAK finished its ray trace in 37 seconds
and then sat for 45 minutes: plt.show() had opened an image viewer and was
waiting for someone to close the window.
- TRAK's main.py only pops that viewer with --show now, and its runner writes
to a gitignored build/ via the new --entry-args. The default output path is
outputs/output.png, which is TRACKED -- a plain run overwrote a submitted
deliverable.
- course_lib.sh was 357 lines, over this repo's own cap. Split into
course_probe.sh and course_run.sh.
Verified: ECOTE 26 tests pass, ECRYPT_PROJECT 99 pass, TRAK renders
cornell_box in 37s and exits 0, every module compiles, and every import ruff
removed was checked to be genuinely unused.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
The 250-line cap applies to prose too, and the reports were the worst
offenders -- ENUME's projectA.tex alone was 1,876 lines. Each is now a
preamble plus one \input{} per section, which is how a long LaTeX document is
normally organised; not one byte of prose was rewritten.
"The PDF is unchanged" is checked by a script rather than by eye, because
hashing the PDF cannot work: pdflatex stamps a /CreationDate and an /ID into
every build, so two runs of the SAME source already differ. tex_fingerprint.sh
compares what a reader sees instead -- page count plus a hash of the extracted
text -- and it is stable across rebuilds of untouched sources, which is the
property that makes it usable as a gate.
It earned its keep immediately. Both ECOTE reports quote an example LaTeX
document inside an lstlisting block, so \end{document} appears in the middle
of the file; splitting on the FIRST one stranded everything after it. Same
page count, different text, caught automatically. EOPSY lab3 exposed a second
bug: a \section{} title that runs onto the next line.
Result: 19 .tex over the cap -> 1. The survivor,
ELAC/.../03-building-nfa-...tex, is 418 lines of consecutive tikz figures
with no subsection to split on and needs a hand-placed boundary.
Verified: 29 splits across two passes (section, then subsection for the
pieces still over), every one reporting `unchanged` from
scripts/tex_fingerprint.sh --check.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
`./run.sh run Programming/ZPOB` exited 12. A clean-room rebuild (the previous
failure was reading a cached .aux) shows the real error is bibtex, not LaTeX:
I found no \citation commands---while reading file esej.aux
I found no \bibstyle command---while reading file esej.aux
esej.tex ends with \bibliography{references} but never cites anything and
never sets a \bibliographystyle. latexmk therefore runs bibtex, bibtex exits
2, and latexmk reports 12 -- after pdflatex has already written a correct
esej.pdf.
Adding \bibliographystyle would make a References section appear that is not
in the deliverable: the submitted esej.pdf (LuaTeX, 2023-12-12, 2 pages) has
no bibliography either, so bibtex never succeeded for the author. The fix
belongs in the harness, so course_lib grows --no-bibtex and ZPOB's stub says
in full why it passes it.
Verified: `rm -rf esej/build && ./run.sh run Programming/ZPOB` exits 0 and
writes esej/build/esej.pdf.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
Running the courses made this machine stutter during a game. That was me:
PORR's suite solves systems up to 10000x10000 and takes every core it is
given, and the pip installs are not cheap either.
A promise to be careful is not a fix, so `run` now executes inside a transient
systemd scope -- MemoryMax=2G, MemorySwapMax=0 (OOM-kill rather than thrash
swap), CPUQuota=200%, nice -n 19, ionice -c 3. Same mechanism the pre-push
hook in testsAndMisc uses. Verified from inside the scope: memory.max reads
2147483648 and cpu.max reads "200000 100000", so the kernel is enforcing it
rather than the script hoping.
RUN_MEMORY_MAX and RUN_CPU_QUOTA override it for a single deliberate run.
Recorded the two courses that still fail: PORR, which will now be OOM-killed
under the cap and needs a smaller default parameter set, and ZPOB, where
latexmk exits 12 from bibtex despite producing the PDF.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
Running one directory (EADS) had already produced four bugs, so all of them
were run. 11 failed. Every one was real -- either in the coursework or in the
harness I had just written -- and the split was roughly even.
Coursework bugs:
- 15 .tex files use Polish ogonek characters with no fontenc, so pdflatex
dies with "Command \k unavailable in encoding OT1". Fixed where it was
actually needed, not blanket-applied.
- BSS had \documentclass{article}[12pt] -- options after the class name,
where they are not options at all.
- SDM's beamer deck had a stray } on an \item, which took out the frame.
- EOOP/Various/currency included "currency2.hpp"; the file is currency.hpp.
- ECRYPT_PROJECT had the SAME stdlib-shadowing bug as ECOTE: a package named
code/. Renamed to sieve/, and its 99 tests now run. With ECOTE's 26 that
is 125 tests that could not execute at all before today.
Harness bugs, which the run is what exposed:
- COURSE_ENTRY pointed at directories for PORR and ECRYPT_PROJECT, so python
looked for __main__.py and reported the wrong problem.
- main.py was auto-picked via find's first result. EARIN has six labs and
six main.py files, so that ran an arbitrary lab and called it the course.
Now only auto-picked when exactly one exists.
- Only the first requirements.txt was installed, which is why EARIN still
died on gymnasium after a venv was created for it. All of them now.
- The notebook probe accepted jupyter being on PATH. This machine's jupyter
is a pipx shim whose interpreter is gone, so "runnable" was a lie; it now
invokes jupyter --version and PBAD/twm_4 report blocked, honestly.
- Nothing ran a course's tests. Where a suite exists it is now the check,
which matters most for ECRYPT_PROJECT, whose program prompts on stdin and
cannot be run headlessly at all.
- Which directory pytest runs from is course-specific and unguessable, so
stubs pass --test-dir: ECOTE needs program/ for imports, ECRYPT_PROJECT
needs sieve/ because its fixtures open a CSV by relative path.
gates/known-incomplete.txt records the one file that genuinely cannot build:
EOOP/Various/set/set.cpp includes set.hpp, which `git log --all` shows has
never existed here. The course supplied it and only the .cpp was submitted.
That is a factual record with a stated check, not a lint allowlist -- and
fabricating the header would be inventing coursework.
Status is now 36 runnable / 5 no-code / 3 blocked.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
Two invariants had been established by hand and left ungated, which is the
"a remedy that only works if someone remembers" pattern:
- scripts/check_ignore_consistency.sh fails the commit if any TRACKED file
is also gitignored. .binary-allowlist used to *instruct* the reader to
re-run that check; now it is enforced. Verified both ways -- passes on the
tree, and fails when an ignored file is force-added.
DOCS-runnability.md now really has two columns. Each environment writes only
its own gates/runnability-<env>.tsv and the doc renders from both, so a CI run
cannot clobber the local results. The ci tsv is gitignored: committing a copy
generated here would label this machine's capabilities as the runner's.
Rendering it surfaced a classic bug -- awk's NR==FNR idiom silently swallows
the SECOND file when the first is empty, so the table came out with a header
and no rows. It compares FILENAME instead.
runnability.yml deliberately does NOT diff against a committed baseline. A
status is a property of the machine, so `runnable` locally and `blocked` on a
runner is correct rather than drift, and diffing would make the workflow red
by construction. It fails on `error` instead -- a runner that crashes or
reports an unknown kind.
Then five more courses were actually run, because EADS alone had produced four
real bugs and one sample proves nothing:
- ECOAR and EPFU build (nasm + 32-bit gcc, and gcc respectively).
- ECOTE's 17 test modules could not run AT ALL. Its package is named `code/`,
which shadows the standard library: pytest's debugging plugin imports pdb,
pdb imports code, and it got this directory instead, so the suite died
with an INTERNALERROR before collecting a single test. Renamed to
translator/ with all 17 imports updated -- 26 tests now pass.
- A LaTeX report builds. That one exposed a hazard rather than a bug:
latexmk wrote its PDF beside the source, and in a course where the built
name matches the submitted one it would have silently overwritten a
tracked deliverable. Both the LaTeX and C++ runners now build into a
gitignored build/, and rebuilding ECOGR leaves `git status` empty.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
The remaining work is much larger than what has landed, and vague about it
would be worse than useless. Numbers are measured, not guessed:
- 9,281 ruff findings under the fleet config (select = ALL). ~3,000 are
auto-fixable and ~6,000 are hand work, dominated by 1,671 missing type
annotations and ~600 missing docstrings.
- 19 tests for ~28k lines of authored code, across 12 languages.
- 73 files still over the 250-line cap, 27 of which are blocked on the
decision parked in TODO-file-length-250.md.
- 22 dependency manifests, none pinned.
Also records the ordering rules that are easy to lose: the dependency upgrade
and its gate must land in the same commit; source files must not be split
before their tests exist; and the file-rewriting hooks are held back so that
touching hundreds of submitted coursework files is its own reviewed change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
All 44 directories now probe: 38 runnable, 5 no-code, 1 blocked (ENUME needs
MATLAB or Octave, neither of which is on this machine).
The runners are stubs over one shared scripts/course_lib.sh rather than 44
copies of the logic, for the same reason the gates live in ~/utils. A stub
passes its kind as ARGUMENTS -- course_main --kind python --entry ... -- "$@"
-- rather than setting globals, because globals a linter cannot see being read
mean SC2034 on every one of them, and the only cure would be 44 disables.
Fixed along the way:
- Generating the stubs overwrote Programming/PSD/run.sh, the original
coursework runner. Restored, and given a --probe early exit instead so it
is otherwise untouched: it installs packages with sudo and starts Kafka,
which is exactly what a probe must not trigger.
- The probe guard only recognised runners containing the literal --probe, so
the new stubs (which delegate to the library) all read unprobed. It now
also accepts delegation to course_lib.sh.
- Blocked probes reported "unknown" because _require_any prints the tool it
found, so probes redirected its stdout -- swallowing the failure reason
too. Split into _probe_any, and ENUME now says exactly what it needs.
- shellcheck runs with -x so it follows the sourced library; without it every
stub reports SC1091 and the only alternative is a per-file disable. The
library itself carries `# shellcheck shell=bash` because it has no shebang
on purpose -- it is sourced, never executed.
Then EADS was actually run, and three of its six C++ files did not compile:
- lab3.cpp: copyDictionary was declared taking Dictionary* but its body used
Node members, and the copy constructor called copyTree(), which does not
exist. The copy also dropped key and balanceFactor. Now compiles AND runs,
printing its AVL tests.
- MidTerm/task2.cpp: a const mismatch between declaration and definition, a
stray semicolon before a function body, findNode calling a Tree method on
a treeNode*, and a while loop that never advanced. sumNodeSub also had no
base case -- unreachable before, because the const mismatch meant its body
was never compiled.
These compiled years ago because older GCC did not diagnose uninstantiated
template bodies. Fragments with no main() are now type-checked with -c rather
than linked, since two of these files are exam answers, not programs.
Binaries go to a gitignored build/ -- an extensionless executable dropped next
to its source is not something .gitignore catches by name.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
Two things kept CI from being green, and neither was the code.
First, GitHub Actions was DISABLED at the repository level. The workflows
registered fine and simply never ran -- `gh api .../actions/permissions`
returned {"enabled": false}. Now enabled.
Second, once it ran, shellcheck failed on a file that is clean locally:
SC2236 in PSD/zin3/python/run_temperature_system.sh. `language: system` hooks
use whatever is on PATH, and Ubuntu's packaged shellcheck is not the 0.11.0
this machine has, so the build went red for a difference in tool version
rather than in the code. That is the failure mode testsAndMisc pins tools to
avoid, so the workflow now installs shellcheck v0.11.0 and verifies its
SHA256 -- checked here first: the digest matches and the archive really does
unpack to shellcheck-v0.11.0/shellcheck.
The `! -z` -> `-n` fix is applied too. Pinning stops the next version drift;
fixing the code means this instance is right under either version.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
Two workflows, one per gate, each checking out kuhyx/utils into .utils and
calling the shared script -- the pattern every other repo here uses.
pre-commit.yml sets UTILS_ROOT to that checkout, without which the shims fail
with "shared gate not found", and both use fetch-depth 1 because the history
is ~494 MB and no gate reads it.
install.sh ends by verifying and exits non-zero if the gate cannot run; a
warning there would just recreate the bug it exists to prevent, since this
repo had no hooks at all until now. Confirmed both ways: --check exited 1
while the hooks were missing, then 0 once installed.
run.sh separates `status` (probe, seconds, safe in CI) from `run <dir>`
(actually executes one project). Conflating them is not hypothetical -- the
first version of the probe did exactly that and it cost 120 seconds and a
scare. Some course dirs already ship a run.sh from years ago, and
Programming/PSD/run.sh installs packages with sudo and starts Kafka; calling
it with an unknown --probe flag does not probe anything, it runs the project.
It blocked on a sudo prompt rather than installing (verified against
/var/log/pacman.log and the absence of a yay clone), but only by luck.
So probe_one now invokes a runner ONLY if it declares --probe, wraps it in
`timeout 20s` as defence in depth, and reports anything else as unprobed
without touching it. All 44 directories now probe instantly.
DOCS-runnability.md is generated with a per-environment column. Recording one
global status would make the CI check red by construction, since `runnable`
locally and `blocked` on ubuntu-latest is the normal case for the MATLAB,
Unity and .NET directories rather than a drift signal.
Per-directory run.sh files are the next step; all 44 currently read unprobed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
`pre-commit run --all-files` is green. The config only carries hooks whose
subject is already clean, so it stays green at every commit -- file-length,
ruff/mypy and dependency-freshness are listed as blocked, each on its own
cleanup, and the file-REWRITING hooks (trailing-whitespace, end-of-file-fixer)
are held back for their own reviewed commit.
Turning it on found four real things:
1. A committed private key: ERSMS-project/nginx/ssl/nginx.key. It is a
self-signed CN=localhost cert that had already expired (notAfter
2025-06-16), so the risk is nil -- but it is regenerable in one openssl
call, so it is replaced by generate-cert.sh rather than excluded from the
hook. The script says plainly that removing it from HEAD does NOT
unpublish it: history is deliberately not rewritten, which is acceptable
only because nothing ever trusted this key.
2. Four files with unresolved merge conflict markers committed, left over
from the subtree merges. EARIN/.gitignore and .vscode/extensions.json are
resolved as the union of both sides. The two Unity folder .meta files each
had two candidate GUIDs; grepping showed neither is referenced anywhere
(folder GUIDs rarely are), so either is safe and the HEAD side is kept --
an arbitrary choice, but a checked one rather than a guess.
3. 62 shellcheck findings across 11 scripts, all fixed, no disables added.
The interesting pattern was `cd X; cmd &; PID=$!; cd ..`, which trips both
SC2164 and SC2103 and cannot use a plain subshell without losing the PID.
`( cd X && exec cmd ) &` keeps the directory, the PID and the semantics.
EOPSY lab1's submitted solution was verified by RUNNING it afterwards:
a->A, B->b, foldera->FOLDERA, FOLDERB->folderb, exactly as before.
4. check-json failing on 11 files, which was the wrong parser rather than a
defect -- tsconfig*.json and .vscode/*.json are JSONC, where TypeScript
and VS Code both allow comments. Those two path classes are excluded and
every other .json is still checked. Excluding them is what surfaced the
merge conflict in (2) as the one genuine failure.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
Five files were outside README / CLAUDE* / DOCS* / TODO*, and two were over
the 250-line cap. Both gates now pass repo-wide.
Renames: the two lowercase readme.md files, and three documents into DOCS-*.
The AIS requirements doc split at its own boundary into functional (101) and
non-functional (174). The WDWR report split into six numbered sections behind
an index that keeps the original filename, 38-235 lines each; 753 lines of
body plus the 18-line header account for all 771 original lines, so nothing
was dropped.
That WDWR file needed checking before splitting: it sits next to main.tex and
main.pdf and looked like a conversion of them. It is not -- main.tex is
project 25406 and the markdown is project 18402, a different document. So it
is authored content and the split is real work, not churn on a build artifact.
Filled the three empty READMEs and replaced ECOTE's, which was a stale copy of
the root README still claiming the group projects were excluded -- they were
merged in by subtree some time ago.
EOPSY/lab1 turned out not to be the code-free directory the survey suggested:
it has modify.sh and modify_examples.sh. The oddly-named empty files beside
them are the exercise's OUTPUT, committed so the before/after state is
visible, and its README now says so rather than leaving them looking like junk.
CLAUDE.md follows the testsAndMisc structure. Writing it caught a nice one:
the md-naming marker is a biconditional, so quoting the literal removal string
in a non-TODO file fails the gate. The gate was right; the text is rephrased.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
621 binaries are tracked here and they are supposed to be: 155 PDFs, 275
PNGs, 68 JPGs, the submitted reports and figures. Decision 3 keeps them and
skips the history rewrite, so a blanket block would be wrong. What the gate
is for is stopping NEW junk.
So the allowlist is scoped by path rather than by extension alone. Documents,
figures and datasets are allowed under Programming/ and NotProgramming/;
compiled output (.o .so .a .exe .dll .dylib .class .pyc) is allowed nowhere
and has no entry at all; and the repo root, gates/, scripts/ and .github/
have no entry either, so a binary dropped in the tooling directories is
blocked. The .tgz entries are load-bearing -- EOPSY ships its simulators as
ftp/{sched,memory}.tgz and `make setup` extracts them.
Verified three ways, because a gate that passes everything is vacuous:
- all 2749 tracked files -> exit 0
- probe.exe at root, probe.png in gates/ -> BLOCKED, exit 1
- a real submitted PDF -> exit 0
plus shellcheck clean.
The script is a copy of testsAndMisc's, which is where this gate lives today
-- it is not in the shared ~/utils set. The header says so and says to move
it to ~/utils rather than let the two drift, since a hand-copied gate is what
produced four forks of the 250-line cap.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
Untracked (--cached, so they stay on disk): 76 LaTeX artifacts, 7 .xdv XeTeX
intermediates, 18 -eps-converted-to.pdf from epstopdf, 32 files of .NET
obj/Debug/net6.0 output including three copies of lab2.dll, 16 .class/.o, the
9 MB Nx cache with its native .node, and an entire committed Python
virtualenv at ECOTE/program/ecote (python 3.10.10, 18 files).
The existing 407-line LaTeX .gitignore was NOT unused as I first assumed --
it already covered *.aux and *.class, and the artifacts simply predate it. So
this appends the categories it missed (*.o, obj/, .venv/, .nx/, target/)
rather than rewriting something that was working.
The second half is the more interesting bug. Auditing with
git ls-files | git check-ignore --stdin --no-index
found 130 files that were tracked AND ignored. That combination fails
silently: `git add` on such a file is not an error, the edit just never
lands. Causes were dist/ swallowing the vendored jQuery/Bootstrap bundle,
.vscode/ and .idea/, and -- the good one -- the LaTeX template's man-page
patterns *.[1-9] and *.[1-9][0-9] matching EPFU's date-named report
directories, so doc/21.11/ read as a man page section and took report.pdf
and seven screenshots with it.
32 of those 130 turned out to be more build output and were untracked too;
the rest got targeted ! overrides. The first attempt used a blanket
!NotProgramming/EMISY/** which re-enabled that course's own LaTeX artifacts,
so it is narrowed to the *.1 schematic exports that actually needed it.
TRAK's nested .gitignore excluded the .vscode DIRECTORY, which stops git
descending, so no root-level ! can reach inside -- that one had to be fixed
at its source.
Both invariants now verified: 0 tracked files ignored, and 0 of the 189
untracked artifacts leaked back into view.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
First step of bringing this archive up to the fleet standard: nothing can be
linted until the gates know what is not kuhy's code.
gates/vendored.txt is the source of truth, one path per line with the reason
AND how it was verified, because an unverified entry is a suppression wearing
a costume. Seven subtrees: the ASP.NET scaffolder's jQuery/Bootstrap bundle
(~98% of all JS and CSS here), stock Unity package and template assets,
manufacturer-supplied Altium/PADS component libraries for the OLED and
HMC5883L parts, and the two EOPSY simulator trees.
The EOPSY entries were the one judgement call, and they turned out not to be:
work/ is the extraction target of `make setup`, and extracting the committed
ftp/*.tgz and diffing shows 14/14 java files byte-identical to the department
originals. ControlPanel.java (1852 lines) and Kernel.java (511) are therefore
not student work and are not split.
Deliberately NOT vendored: EMISY's README, declaration and report are kuhy's
own writing; E-PSYOPS Assets/Scripts (12 .cs) is authored; and the Unity Input
System binding is already exempt via its auto-generated banner, so it needs no
entry. Narrow entries beat one blanket directory.
TODO-file-length-250.md parks the one thing this mechanism cannot express: 27
data .txt files whose lines are long enough that the cap reads them as prose.
They resist directory-prefix exemption because the SemEval corpus dirs mix the
official distribution with kuhy's own GPT outputs, and WDWR/projekt/R holds
both scenarios.txt and the authored model.R. That blocks only the installation
of the file-length gate, which is several milestones away.
Mirrored into the shared gate as kuhyx/utils@a4a3e0a.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
