152 ruff findings to 0 across twelve files, and mypy --strict is clean on all
of them but one -- see the end.
The course is marked unrunnable because it wants Kafka and Flink, so the
verification is what can be run without them. Ten of the twelve modules now
import cleanly against a venv with confluent-kafka, kafka-python and flask; the
two pyflink ones are skipped. That check did not exist before, because four of
these files WERE their own side effect:
- `kafka/generate_and_send.py` and `kafka/kafka_consumer.py` ran their
`while True:` loop at module level. Importing either never returns; I gave up
after twelve seconds of rdkafka connection failures.
- `zin1/producent.py` built a KafkaProducer at module level and raised
NoBrokersAvailable on import.
- `flink/anomaly_detector.py` called `env.execute()` at module level.
Each is a `main()` behind a `__main__` guard now, so importing one is free and
running it is unchanged.
`webinterface.py` called `json.loads` and never imported json, so the first
message on the topic would have raised NameError. mypy then found three more
of the same shape: confluent-kafka's `msg.error()` and `msg.value()` both
return Optional, and the code dereferenced them straight away. They are
narrowed to locals now, which also stops `msg.error()` being called twice per
message. A malformed alarm with no timestamp is skipped with a warning instead
of dividing None by 1000.
Two SLF001 findings were reads of `KafkaError._PARTITION_EOF`. That leading
underscore is confluent-kafka's own naming and there is no public alias, but
`KafkaError.name()` is public and returns exactly that string -- checked
against the installed library, where code() is -191 and name() is
"_PARTITION_EOF". Flask's `debug=True` is now opt-in via FLASK_DEBUG, because
the debug console executes arbitrary code.
The ten `random` calls are simulation, not crypto, so S311 is not a real
vulnerability -- but `secrets.SystemRandom()` is a drop-in with the same
methods and no rule to argue with, so that is what they use.
Behaviour is checked, not assumed: the sensor still returns a location in
1..10 and a temperature in -20..40, the thresholds still classify -15 as too
cold, 40 as too hot and 20 as normal, transactions still carry a four-digit
card id, and TemperatureReading survives a to_dict/from_dict round trip.
**Two mypy errors remain and are deliberate.** `--strict` implies
`disallow_subclassing_any`, and the Flink job must subclass pyflink's
MapFunction and KeyedProcessFunction, which ship no types. Fixing it means
weakening strict globally or writing an override for our own module. Neither is
worth it, so the two are reported in TODO-refactor-remaining.md rather than
hidden.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
115 ruff findings to 0. All three notebooks' non-source content hashes the
same before and after -- c7660e3e5e3b8d4ffc008d7f, b918553ae0cf91cd56fa4091
and 3f09c77e3edae1727e3781e2 -- so every stored output, execution count and
piece of metadata is exactly what was submitted.
Most of the work was 35 over-long lines, and they are over-long because they
are full Polish GUS column descriptions used as dict keys in .rename(). A key
like "udziały w podatkach stanowiących dochody budżetu państwa podatek
dochodowy od osób fizycznych" cannot be shortened, only wrapped, so a script
re-wrapped all of them with implicit concatenation inside parentheses -- the
one form both ruff's linter and its formatter accept.
A script doing 134 mechanical edits to strings that are database keys is
exactly the kind of change that silently drops a space. So it is checked
mechanically: an AST pass collects every string dict key in every code cell of
both versions and compares the multisets. 120 keys in data.ipynb, 9 in
main.ipynb, 5 in pbad.ipynb, all identical.
The other 27 findings were commented-out code (ERA001), and deleting it would
have thrown away what it recorded: which other operational programmes were
run, which axis limits the plots were read at, that MinMaxScaler was tried and
made no difference, that an earlier pass dropped a hand-maintained column list.
Each is now a sentence of prose saying that, and git history still has the
code. Two cells were nothing but commented-out scratch and are now just the
note.
Also real: five `inplace=True` calls became assignments, `zip()` in three
feature-importance loops got `strict=True`, `max(max(a), max(b))` became
`max(*a, *b)`, and 1.5 and 7 became DENSE_GMINA_THRESHOLD and
TERYT_GMINA_DIGITS -- the latter being "a full TERYT gmina code is seven
digits", which is worth saying once rather than leaving as a bare literal.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
228 ruff findings to 0 across the notebook, its partial export and twm.py, and
mypy --strict is clean on both .py files.
A notebook is a deliverable that happens to contain code, so the check that
mattered was not "does it still run" -- it needs a GPU, MNIST, CIFAR-10 and a
slice of Quick, Draw! to run at all -- but "is anything except the code still
exactly as submitted". Hashing every key of the .ipynb EXCEPT cell `source`
gives 60242e116d12549d11c93f7f before the first edit and after the last one,
across four passes and a `ruff format`. The file also round-trips byte-for-byte
through json.dumps(indent=1, ensure_ascii=False), which is what made editing
cells programmatically safe rather than a reformat of all 141 of them; the
diff touches 38.
print() became sys.stdout.write() rather than logging. In a notebook the print
IS the output, and a logger with no handler configured would have silently
emptied every cell -- the exact regression scripts/check_refactor_safety.py
was written for. matprint's `print(x, end=" ")` and bare `print()` are
preserved as writes with and without a trailing newline.
Both `show_samples` and `show_samples_rgb` took (indices, preds, images,
labels, count, names), which is six arguments and a mutable default. They now
take a frozen `Samples` holding the last four, which also removed the same
four-value tuple from seven call sites. Reviewing that diff is how I caught
that I had transcribed `real_score = preds[sample][real_id]` as
`samples.labels[sample]` -- a bug I introduced and then fixed before commit.
Other real things, not just rule satisfaction: `np.in1d` is removed in NumPy 2
(NPY201) and is now `np.isin(...).ravel()`; `test_model` computed
correct_indices and incorrect_indices and used neither; two cells grew `img`
inside a loop so it was only bound by the `else` branch; and the Quick, Draw!
downloader now builds a URL whose https scheme is a literal, which is what
S310 was asking for.
The README claimed the .py was "an nbconvert export of that notebook". It is
an export of the CIFAR-10 section only -- 28 of 81 code cells -- and has been
edited since. It says that now, and says the two are separate artifacts rather
than one being regenerable from the other.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
494 ruff findings to 0, and mypy --strict is clean too -- the first course in
this campaign where that is true of both.
Two real bugs turned up on the way, neither of which ruff can see:
- `main.py` passed `args.environment_map` to `ray_trace`, and argparse only
ever defined `--environment`. Every `--algorithm ray_tracing0` run died with
AttributeError before rendering a pixel, so `rendering.py` -- the largest
file in the course -- had no way to be exercised at all.
- `mouse_button` in the environment-map viewer assigned `mouse_last_x/y`
without declaring them `global`, so the drag origin went to locals and was
thrown away. Measured against HEAD: a 10x5 px drag turned the camera by
11.0/20.5 degrees because it measured from (0, 0); it is 1.0/0.5 now, which
is what the 0.1 degrees-per-pixel constant says it should be.
Fixing the first one is what made the rest safe. `ray_tracing0` has no RNG
anywhere in it, so it renders byte-identically every time, and the whole
refactor of `rendering.py` is verified by one md5 comparison:
02484bdb6b6c1ddeff317d7e5566580f, 248619 rays, before and after. The
cornell_box path cannot do that -- its ray count drifts by ~20 between
identical runs.
Three files were over the 250-line cap and are now split, geometry away from
the algorithm that uses it: rendering.py 462 -> 137 + raytracer.py,
photon_mapping.py 261 -> 160 + photon_geometry.py, and
code/photonmapping/python/main.py 257 -> 203 + geometry.py. No TRAK source
file is over the cap now.
The two photon mappers do use numpy's legacy global RNG, so NPY002 could not
be bit-identical. Each was done in two steps: restructure first, verified
identical under an external seed (digests 7220370f60206a62 and
21b4771504ab2351, and an identical 828 photon traces), then the swap to
`default_rng` on its own, checked against the before/after distributions.
Verified by running, not by reading:
- cornell_box through `./run.sh run Programming/TRAK`: 24.27s, 13294 rays
- all five scenes build after `from sightpy import *` became explicit imports
- the standalone photon mapper end to end: 81.4s, 38742 photon traces
- the EXR loader byte for byte: a0c3890350ce6101247c6948, 6291456 bytes
- the GLUT viewer under xvfb-run, which drew a real 1024x512 frame
`pyproject.toml` gains a mypy override for Imath, OpenEXR, OpenGL and
matplotlib. Those four ship neither stubs nor py.typed, and without it mypy
cannot import them at all and so reports nothing else about the file.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
598 findings to none, and the suite ran through `./run.sh run` after each step
rather than at the end -- 18 passed, 27s, every time.
The shape of this one was different from the other courses. PORR is four
implementations of the same linear algebra (sequential, threads, processes,
dask), so nearly every finding came in groups of four:
- 229 missing annotations. Rather than `object`, which mypy strict rejects on
every attribute access, each module declares what these actually are:
`Matrix = np.ndarray | list[list[float]]` and the vector equivalent. The
backends really are called with both -- comparing them is the point of the
course.
- 76 names: `A` -> matrix, `Ab` -> augmented, `Ax` -> product, `I` ->
identity, `dA` -> dask_matrix, and the two CamelCase functions. Renamed
through `tokenize`, not regex, so no string or comment was touched.
- 121 missing docstrings, one per method name since the names repeat across
all four backends.
- Seven functions over the argument limit, all taking the same seven or nine
loose values. They are two frozen dataclasses now, in a new
richardson_problem.py: `Problem(matrix, rhs, lambda_min, lambda_max)` and
`Settings(max_iterations, x0, tol)`. `RichardsonMethod.__init__` also took a
`size` argument it never read.
- threads_indep_old's iteration did "split into chunks, submit, wait" four
times by hand (C901 11, PLR0915 54). One `_run_chunked` helper.
- tests.py's two bare asserts became explicit raises, per the no-ignores rule.
`assert False` was also B011: `python -O` deletes it, so the non-convergence
branch would have silently passed under optimisation.
Real fixes rather than style: np.random -> Generator, `os.path.exists` ->
Path, an `except Exception` around dask narrowed to the errors it actually
raises, and two runnable modules that logged at INFO without ever calling
basicConfig -- caught by scripts/check_refactor_safety.py, which is now doing
the job it was written for.
Repo-wide: 1,781 findings -> 1,183, all of them in the five courses left.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
PORR was the last course whose `run` did not work. Four measurements, not
guesses, explain why -- and none of them is the matrix size everyone assumes:
ProcessingType.PROCESSES 20.9s at n=2, 32.8s at n=5, 78.2s at n=10. That
is process-pool setup PER TEST, and it grows.
matrix_type poli3 densifies a sparse .mat into a 2.3 GB npz cache
(nemeth12: 723 MB) -- over the 2G ceiling before
any solving happens.
n=10000 800 MB matrix, and np.linalg.solve wants a copy.
n=5000 spd_5000.npz was a Git LFS pointer whose object
404s on GitHub, so it regenerates from scratch.
Remove those and what is left still exercises the whole Richardson
implementation at every size that fits: 18 tests, 28 seconds through the
harness, exit 0. The submitted parameter set is one documented environment
variable away. `main.py` now propagates pytest's status (verified: exit 5 when
nothing is collected) -- it discarded it before, so `run` could not tell a
passing suite from a failing one.
The rest of this commit is scripts/check_refactor_safety.py, which exists
because three regressions got past both ruff and a smoke run during this
campaign, and I would rather they could not get past the next one:
--signatures diffs two trees function by function, parameter by
parameter, default by default. This is what found the 22
EARIN functions whose defaults an annotation pass silently
dropped.
--logging a module with a __main__ guard that logs at INFO must call
basicConfig, or it prints nothing at all. Found two more
in PORR just now.
--print-keywords refuses to let an automated rewrite near print(x, end="")
or print(x, file=sys.stderr); losing those turns a
rendered board into one glyph per line, or silently moves
error output onto stdout.
All three pass on the whole archive. Also in here: 12 latent TypeErrors mypy
found in EARIN that ruff could not -- call sites still passing positionally to
parameters this campaign had made keyword-only, plus one caller of a function
whose argument I had removed. Every one would have raised at runtime.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
749 findings across six labs and two projects. The mechanical half was
scripted; everything below is what the scripts got wrong or could not do, and
how it was caught.
Three regressions I introduced and then found by RUNNING the code, not by
reading it:
- Converting print() to logger.info() made the programs SILENT. logging's
last-resort handler only emits WARNING and above, so with no basicConfig
call `lab2/main.py -t 1 1` printed nothing at all. Fifteen files across the
repo now configure logging where they log.
- The same conversion dropped `end=""`. lab2 draws its board one square at a
time and lab1 draws the maze one cell at a time; both came out one glyph
per line. Those thirteen call sites are sys.stdout.write again, and the
rendered board now diffs byte-identically against the pre-change output.
- The B007 fix renamed `col` to `_col` in the FIRST loop it matched, which
was print_letters -- where the variable IS used. That would have been a
NameError on the first call.
And one the scripts caused silently: annotating 110 signatures dropped the
DEFAULT VALUES from 22 of them, so `preprocessing(a, b, USER_THRESHOLD,
ANIME_THRESHOLD)` would have raised TypeError. Every signature is now diffed
against the pre-change version, name by name and default by default: zero
mismatches.
Real defects the lint surfaced:
- `alpha_beta` ended with an unreachable `return None` that violated its own
return type. Its maximising and minimising halves were mirror images, so
they are one `_search_branch` now (C901 11 -> ok).
- `get_data`'s `gpu` switch, `predict`'s `auto`, `create_model`'s
`rows_number` and five of `preprocess_model_predict`'s fifteen parameters
were never read. Gone, with their call sites.
- np.random.* -> Generator throughout (NPY002), `.values` -> `.to_numpy()`,
`datetime.now()` with a tzinfo, `inplace=True` removed, `except:` given a
type, `random.*` -> `SystemRandom` (S311).
- print_board's three nested helpers became methods (C901 17 -> ok);
user_input's nine-branch if-ladder became a table (C901 12 -> ok).
Verification, all by running: lab1 `-t` and lab2 `-t 1 1` produce output
byte-identical to the pre-change code; lab2's rendered 8x8 board diffs clean;
lab3's argument parsing returns the same dict for the same argv; lab4 fails
identically before and after (no sklearn here). Also included: the logging
configuration for NLP, MOM and ECRYPT_PROJECT, which had the same silence bug.
Repo-wide: 2,775 findings -> 1,883.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
384 findings to none across ten scripts. Most of it is annotation and
docstring work, but the lint surfaced two defects that had nothing to do with
style:
- `gpt_chunks.py` stripped its GPT preamble with
`input_string.strip("iSTS chunks:")`. str.strip takes a SET OF CHARACTERS,
not a prefix, so it also ate any leading i/S/T/s/c/h/u/n/k/space/colon:
"iSTS chunks: sentence one" came out as "entence one", silently losing the
first letter of every chunked sentence. Now removeprefix. ruff found it as
B005 + PLE1310.
- `is_numbered_line_format` had `for string in string` in its list branch,
which raises NameError the moment a list is passed (F821).
- `processing.test_XML()` read a module-level `alignments_data` that does not
exist at module scope; it only ever ran when pasted into a live session. It
takes the data as an argument now and is renamed `log_first_alignment` so
pytest stops collecting a non-test as a test.
Also real rather than cosmetic:
- The OpenAI key was a placeholder string literal in three files, one of them
carrying a "Replace me with os.environ['API_KEY']" comment. All three read
OPENAI_API_KEY now and say so when it is missing -- as committed, none of
these scripts could actually run.
- `rating.rate_similarity` was submitted unfinished: it built a client, never
touched its two arguments and returned None despite `-> str`. It now makes
the same call the rest of the package makes.
- The two 2 kB alignment prompts and one 344-character chunking prompt moved
to prompts/*.txt. Wrapping them in source would have put newlines into the
middle of the prompt.
- `format_alignments` parsed GPT's replies with `eval()`; the file holds one
repr()'d string per line, so ast.literal_eval is both sufficient and safe.
- NLP/__init__.py deleted -- it made the course directory a package named
"NLP", which is an invalid module name (N999) and which nothing imports;
the scripts use plain `import processing`.
INP001 is settled the same way throughout: a coursework file run as
`python x.py` gets a shebang and the executable bit, which is what it is,
rather than an __init__.py that would make a course code into a module name.
Verified: every module compiles and `./run.sh run Programming/NLP` is clean.
The scripts themselves need the SemEval corpus and a paid API key, so they
are not executed here.
Repo-wide: 3,159 findings -> 2,775.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
kuhy chose the strict branch: no per-file-ignores at all, not even the S101
block that ruff's own docs and the rest of this fleet carry for test code. So
125 test assertions were rewritten to raise instead of assert.
Rather than 125 copies of `if x != y: raise AssertionError(...)`, each course
grows a small expect module -- expect_equal / expect_in / expect_true /
expect_false. That is also strictly better output than a bare assert here:
pytest's assertion rewriting does not reach into a helper, so the helper has
to print both values, and it does. The rewrite itself went through ast, not a
regex, because seven of ECOTE's asserts were wrapped over several lines.
Everything else in the two courses is gone as well:
- 66 argument and 37 return annotations on the translator, worked out from
what each function actually returns rather than stamped on as Any --
`list[str] | str` where a function returns "Error!" or the parsed columns.
- 16 prints became logger.error with %-style arguments. The messages still
reach stderr with no basicConfig, via logging's last-resort handler, so the
translator behaves exactly as before.
- Two `open()` calls became Path.open / Path.write_text, six RET504
assign-then-return pairs collapsed, one 380-character expected-HTML literal
split into implicitly concatenated pieces (it cannot be wrapped any other
way, which is what ISC001 being off is for), and one block of commented-out
code deleted.
- ECRYPT_PROJECT/__init__.py removed. It made the COURSE directory a package,
which nothing imports, and it was the sole cause of `N999 Invalid module
name: 'ECRYPT_PROJECT'`; with it gone, sieve is the top-level package and
its name is fine.
Verified by running both, not by reading: ECOTE 26 tests pass and
`python -m translator.main --filename examples/complicated/table1.tex` still
writes the same HTML with the same three column-count warnings on stderr;
ECRYPT_PROJECT 99 tests pass.
Repo-wide: 3,686 findings -> 3,159.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
kuhy overruled the "do not fix coursework to be better than it was submitted"
rule: git history is the archive, so the working tree can be clean. This lands
the config and the mechanical half of the cleanup -- 9,281 ruff findings under
the fleet config down to 3,686, with no per-file-ignores anywhere.
What is in here:
- pyproject.toml, the first one in this repo. select = ["ALL"], mypy strict,
and the vendored list mirrored into ruff exclude and coverage omit. Five
rules are off and each says why; CPY001 is the only judgement call, because
stamping a copyright header onto 137 university assignments would assert a
licence that was never claimed. No hook is wired up yet -- a gate lands only
once its subject is clean.
- ruff format + the safe fixes, then a second pass over the docstring rules
ruff marks unsafe (they only edit docstring text). 108 files.
- TRAK/sightpy is vendored, not coursework: it is rafael-fuente/Python-Raytracer
(BSD 3-Clause). Verified against a fresh upstream clone with line endings
normalised -- 30 of 32 files byte-identical, ray.py and scene.py carrying a
12- and 13-line local patch that counts rays for the report. 35 files that
should never have been linted, now excluded and restored.
- WDWR, SPD, MOM and ECRYPT are fully clean, by hand and verified by running
them: WDWR's three plot scripts print the same statistics as before, MOM's
PuLP model solves to the same cost. WDWR's graph.py had a 342-character CSV
header embedded as a string literal, which cannot be wrapped -- the data
moved to data/pareto_front.csv. Two 0-byte .py stubs are deleted; an empty
file is not a deliverable and git still has it.
Three harness bugs found by actually running things:
- `run` now forces MPLBACKEND=Agg. TRAK finished its ray trace in 37 seconds
and then sat for 45 minutes: plt.show() had opened an image viewer and was
waiting for someone to close the window.
- TRAK's main.py only pops that viewer with --show now, and its runner writes
to a gitignored build/ via the new --entry-args. The default output path is
outputs/output.png, which is TRACKED -- a plain run overwrote a submitted
deliverable.
- course_lib.sh was 357 lines, over this repo's own cap. Split into
course_probe.sh and course_run.sh.
Verified: ECOTE 26 tests pass, ECRYPT_PROJECT 99 pass, TRAK renders
cornell_box in 37s and exits 0, every module compiles, and every import ruff
removed was checked to be genuinely unused.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
The 250-line cap applies to prose too, and the reports were the worst
offenders -- ENUME's projectA.tex alone was 1,876 lines. Each is now a
preamble plus one \input{} per section, which is how a long LaTeX document is
normally organised; not one byte of prose was rewritten.
"The PDF is unchanged" is checked by a script rather than by eye, because
hashing the PDF cannot work: pdflatex stamps a /CreationDate and an /ID into
every build, so two runs of the SAME source already differ. tex_fingerprint.sh
compares what a reader sees instead -- page count plus a hash of the extracted
text -- and it is stable across rebuilds of untouched sources, which is the
property that makes it usable as a gate.
It earned its keep immediately. Both ECOTE reports quote an example LaTeX
document inside an lstlisting block, so \end{document} appears in the middle
of the file; splitting on the FIRST one stranded everything after it. Same
page count, different text, caught automatically. EOPSY lab3 exposed a second
bug: a \section{} title that runs onto the next line.
Result: 19 .tex over the cap -> 1. The survivor,
ELAC/.../03-building-nfa-...tex, is 418 lines of consecutive tikz figures
with no subsection to split on and needs a hand-placed boundary.
Verified: 29 splits across two passes (section, then subsection for the
pieces still over), every one reporting `unchanged` from
scripts/tex_fingerprint.sh --check.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
`./run.sh run Programming/ZPOB` exited 12. A clean-room rebuild (the previous
failure was reading a cached .aux) shows the real error is bibtex, not LaTeX:
I found no \citation commands---while reading file esej.aux
I found no \bibstyle command---while reading file esej.aux
esej.tex ends with \bibliography{references} but never cites anything and
never sets a \bibliographystyle. latexmk therefore runs bibtex, bibtex exits
2, and latexmk reports 12 -- after pdflatex has already written a correct
esej.pdf.
Adding \bibliographystyle would make a References section appear that is not
in the deliverable: the submitted esej.pdf (LuaTeX, 2023-12-12, 2 pages) has
no bibliography either, so bibtex never succeeded for the author. The fix
belongs in the harness, so course_lib grows --no-bibtex and ZPOB's stub says
in full why it passes it.
Verified: `rm -rf esej/build && ./run.sh run Programming/ZPOB` exits 0 and
writes esej/build/esej.pdf.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
Running one directory (EADS) had already produced four bugs, so all of them
were run. 11 failed. Every one was real -- either in the coursework or in the
harness I had just written -- and the split was roughly even.
Coursework bugs:
- 15 .tex files use Polish ogonek characters with no fontenc, so pdflatex
dies with "Command \k unavailable in encoding OT1". Fixed where it was
actually needed, not blanket-applied.
- BSS had \documentclass{article}[12pt] -- options after the class name,
where they are not options at all.
- SDM's beamer deck had a stray } on an \item, which took out the frame.
- EOOP/Various/currency included "currency2.hpp"; the file is currency.hpp.
- ECRYPT_PROJECT had the SAME stdlib-shadowing bug as ECOTE: a package named
code/. Renamed to sieve/, and its 99 tests now run. With ECOTE's 26 that
is 125 tests that could not execute at all before today.
Harness bugs, which the run is what exposed:
- COURSE_ENTRY pointed at directories for PORR and ECRYPT_PROJECT, so python
looked for __main__.py and reported the wrong problem.
- main.py was auto-picked via find's first result. EARIN has six labs and
six main.py files, so that ran an arbitrary lab and called it the course.
Now only auto-picked when exactly one exists.
- Only the first requirements.txt was installed, which is why EARIN still
died on gymnasium after a venv was created for it. All of them now.
- The notebook probe accepted jupyter being on PATH. This machine's jupyter
is a pipx shim whose interpreter is gone, so "runnable" was a lie; it now
invokes jupyter --version and PBAD/twm_4 report blocked, honestly.
- Nothing ran a course's tests. Where a suite exists it is now the check,
which matters most for ECRYPT_PROJECT, whose program prompts on stdin and
cannot be run headlessly at all.
- Which directory pytest runs from is course-specific and unguessable, so
stubs pass --test-dir: ECOTE needs program/ for imports, ECRYPT_PROJECT
needs sieve/ because its fixtures open a CSV by relative path.
gates/known-incomplete.txt records the one file that genuinely cannot build:
EOOP/Various/set/set.cpp includes set.hpp, which `git log --all` shows has
never existed here. The course supplied it and only the .cpp was submitted.
That is a factual record with a stated check, not a lint allowlist -- and
fabricating the header would be inventing coursework.
Status is now 36 runnable / 5 no-code / 3 blocked.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
Two invariants had been established by hand and left ungated, which is the
"a remedy that only works if someone remembers" pattern:
- scripts/check_ignore_consistency.sh fails the commit if any TRACKED file
is also gitignored. .binary-allowlist used to *instruct* the reader to
re-run that check; now it is enforced. Verified both ways -- passes on the
tree, and fails when an ignored file is force-added.
DOCS-runnability.md now really has two columns. Each environment writes only
its own gates/runnability-<env>.tsv and the doc renders from both, so a CI run
cannot clobber the local results. The ci tsv is gitignored: committing a copy
generated here would label this machine's capabilities as the runner's.
Rendering it surfaced a classic bug -- awk's NR==FNR idiom silently swallows
the SECOND file when the first is empty, so the table came out with a header
and no rows. It compares FILENAME instead.
runnability.yml deliberately does NOT diff against a committed baseline. A
status is a property of the machine, so `runnable` locally and `blocked` on a
runner is correct rather than drift, and diffing would make the workflow red
by construction. It fails on `error` instead -- a runner that crashes or
reports an unknown kind.
Then five more courses were actually run, because EADS alone had produced four
real bugs and one sample proves nothing:
- ECOAR and EPFU build (nasm + 32-bit gcc, and gcc respectively).
- ECOTE's 17 test modules could not run AT ALL. Its package is named `code/`,
which shadows the standard library: pytest's debugging plugin imports pdb,
pdb imports code, and it got this directory instead, so the suite died
with an INTERNALERROR before collecting a single test. Renamed to
translator/ with all 17 imports updated -- 26 tests now pass.
- A LaTeX report builds. That one exposed a hazard rather than a bug:
latexmk wrote its PDF beside the source, and in a course where the built
name matches the submitted one it would have silently overwritten a
tracked deliverable. Both the LaTeX and C++ runners now build into a
gitignored build/, and rebuilding ECOGR leaves `git status` empty.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
All 44 directories now probe: 38 runnable, 5 no-code, 1 blocked (ENUME needs
MATLAB or Octave, neither of which is on this machine).
The runners are stubs over one shared scripts/course_lib.sh rather than 44
copies of the logic, for the same reason the gates live in ~/utils. A stub
passes its kind as ARGUMENTS -- course_main --kind python --entry ... -- "$@"
-- rather than setting globals, because globals a linter cannot see being read
mean SC2034 on every one of them, and the only cure would be 44 disables.
Fixed along the way:
- Generating the stubs overwrote Programming/PSD/run.sh, the original
coursework runner. Restored, and given a --probe early exit instead so it
is otherwise untouched: it installs packages with sudo and starts Kafka,
which is exactly what a probe must not trigger.
- The probe guard only recognised runners containing the literal --probe, so
the new stubs (which delegate to the library) all read unprobed. It now
also accepts delegation to course_lib.sh.
- Blocked probes reported "unknown" because _require_any prints the tool it
found, so probes redirected its stdout -- swallowing the failure reason
too. Split into _probe_any, and ENUME now says exactly what it needs.
- shellcheck runs with -x so it follows the sourced library; without it every
stub reports SC1091 and the only alternative is a per-file disable. The
library itself carries `# shellcheck shell=bash` because it has no shebang
on purpose -- it is sourced, never executed.
Then EADS was actually run, and three of its six C++ files did not compile:
- lab3.cpp: copyDictionary was declared taking Dictionary* but its body used
Node members, and the copy constructor called copyTree(), which does not
exist. The copy also dropped key and balanceFactor. Now compiles AND runs,
printing its AVL tests.
- MidTerm/task2.cpp: a const mismatch between declaration and definition, a
stray semicolon before a function body, findNode calling a Tree method on
a treeNode*, and a while loop that never advanced. sumNodeSub also had no
base case -- unreachable before, because the const mismatch meant its body
was never compiled.
These compiled years ago because older GCC did not diagnose uninstantiated
template bodies. Fragments with no main() are now type-checked with -c rather
than linked, since two of these files are exam answers, not programs.
Binaries go to a gitignored build/ -- an extensionless executable dropped next
to its source is not something .gitignore catches by name.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
Two things kept CI from being green, and neither was the code.
First, GitHub Actions was DISABLED at the repository level. The workflows
registered fine and simply never ran -- `gh api .../actions/permissions`
returned {"enabled": false}. Now enabled.
Second, once it ran, shellcheck failed on a file that is clean locally:
SC2236 in PSD/zin3/python/run_temperature_system.sh. `language: system` hooks
use whatever is on PATH, and Ubuntu's packaged shellcheck is not the 0.11.0
this machine has, so the build went red for a difference in tool version
rather than in the code. That is the failure mode testsAndMisc pins tools to
avoid, so the workflow now installs shellcheck v0.11.0 and verifies its
SHA256 -- checked here first: the digest matches and the archive really does
unpack to shellcheck-v0.11.0/shellcheck.
The `! -z` -> `-n` fix is applied too. Pinning stops the next version drift;
fixing the code means this instance is right under either version.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
`pre-commit run --all-files` is green. The config only carries hooks whose
subject is already clean, so it stays green at every commit -- file-length,
ruff/mypy and dependency-freshness are listed as blocked, each on its own
cleanup, and the file-REWRITING hooks (trailing-whitespace, end-of-file-fixer)
are held back for their own reviewed commit.
Turning it on found four real things:
1. A committed private key: ERSMS-project/nginx/ssl/nginx.key. It is a
self-signed CN=localhost cert that had already expired (notAfter
2025-06-16), so the risk is nil -- but it is regenerable in one openssl
call, so it is replaced by generate-cert.sh rather than excluded from the
hook. The script says plainly that removing it from HEAD does NOT
unpublish it: history is deliberately not rewritten, which is acceptable
only because nothing ever trusted this key.
2. Four files with unresolved merge conflict markers committed, left over
from the subtree merges. EARIN/.gitignore and .vscode/extensions.json are
resolved as the union of both sides. The two Unity folder .meta files each
had two candidate GUIDs; grepping showed neither is referenced anywhere
(folder GUIDs rarely are), so either is safe and the HEAD side is kept --
an arbitrary choice, but a checked one rather than a guess.
3. 62 shellcheck findings across 11 scripts, all fixed, no disables added.
The interesting pattern was `cd X; cmd &; PID=$!; cd ..`, which trips both
SC2164 and SC2103 and cannot use a plain subshell without losing the PID.
`( cd X && exec cmd ) &` keeps the directory, the PID and the semantics.
EOPSY lab1's submitted solution was verified by RUNNING it afterwards:
a->A, B->b, foldera->FOLDERA, FOLDERB->folderb, exactly as before.
4. check-json failing on 11 files, which was the wrong parser rather than a
defect -- tsconfig*.json and .vscode/*.json are JSONC, where TypeScript
and VS Code both allow comments. Those two path classes are excluded and
every other .json is still checked. Excluding them is what surfaced the
merge conflict in (2) as the one genuine failure.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
Five files were outside README / CLAUDE* / DOCS* / TODO*, and two were over
the 250-line cap. Both gates now pass repo-wide.
Renames: the two lowercase readme.md files, and three documents into DOCS-*.
The AIS requirements doc split at its own boundary into functional (101) and
non-functional (174). The WDWR report split into six numbered sections behind
an index that keeps the original filename, 38-235 lines each; 753 lines of
body plus the 18-line header account for all 771 original lines, so nothing
was dropped.
That WDWR file needed checking before splitting: it sits next to main.tex and
main.pdf and looked like a conversion of them. It is not -- main.tex is
project 25406 and the markdown is project 18402, a different document. So it
is authored content and the split is real work, not churn on a build artifact.
Filled the three empty READMEs and replaced ECOTE's, which was a stale copy of
the root README still claiming the group projects were excluded -- they were
merged in by subtree some time ago.
EOPSY/lab1 turned out not to be the code-free directory the survey suggested:
it has modify.sh and modify_examples.sh. The oddly-named empty files beside
them are the exercise's OUTPUT, committed so the before/after state is
visible, and its README now says so rather than leaving them looking like junk.
CLAUDE.md follows the testsAndMisc structure. Writing it caught a nice one:
the md-naming marker is a biconditional, so quoting the literal removal string
in a non-TODO file fails the gate. The gate was right; the text is rephrased.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
Untracked (--cached, so they stay on disk): 76 LaTeX artifacts, 7 .xdv XeTeX
intermediates, 18 -eps-converted-to.pdf from epstopdf, 32 files of .NET
obj/Debug/net6.0 output including three copies of lab2.dll, 16 .class/.o, the
9 MB Nx cache with its native .node, and an entire committed Python
virtualenv at ECOTE/program/ecote (python 3.10.10, 18 files).
The existing 407-line LaTeX .gitignore was NOT unused as I first assumed --
it already covered *.aux and *.class, and the artifacts simply predate it. So
this appends the categories it missed (*.o, obj/, .venv/, .nx/, target/)
rather than rewriting something that was working.
The second half is the more interesting bug. Auditing with
git ls-files | git check-ignore --stdin --no-index
found 130 files that were tracked AND ignored. That combination fails
silently: `git add` on such a file is not an error, the edit just never
lands. Causes were dist/ swallowing the vendored jQuery/Bootstrap bundle,
.vscode/ and .idea/, and -- the good one -- the LaTeX template's man-page
patterns *.[1-9] and *.[1-9][0-9] matching EPFU's date-named report
directories, so doc/21.11/ read as a man page section and took report.pdf
and seven screenshots with it.
32 of those 130 turned out to be more build output and were untracked too;
the rest got targeted ! overrides. The first attempt used a blanket
!NotProgramming/EMISY/** which re-enabled that course's own LaTeX artifacts,
so it is narrowed to the *.1 schematic exports that actually needed it.
TRAK's nested .gitignore excluded the .vscode DIRECTORY, which stops git
descending, so no root-level ! can reach inside -- that one had to be fixed
at its source.
Both invariants now verified: 0 tracked files ignored, and 0 of the 189
untracked artifacts leaked back into view.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
