kuhyx/WUT_Computer_Science
1
0
Fork 0
mirror of https://github.com/kuhyx/WUT_Computer_Science.git synced 2026-11-10 18:07:14 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity

Exempt 27 data txt files from line cap
Some checks are pending
md-naming / markdown naming convention (push) Waiting to run
Details
pre-commit / pre-commit (push) Waiting to run
Details
repo-contract / repo contract (push) Waiting to run
Details
runnability / runnability (push) Waiting to run
Details

Browse Source 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Krzysztof Rudnicki 2026-11-01 21:34:58 +01:00
parent dadc2b3e6f
commit a2644bace8
4 changed files with 17 additions and 112 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
.file-length-exempt Normal file
View File

@ -0,0 +1,12 @@
# Data files the 250-line cap reads as prose. In every one a line is a record,
# so splitting would corrupt it. Authored code beside them stays gated.
Programming/NLP/test_goldStandard/*/STSint.testinput.*.txt # official SemEval 2016 Task 2 iSTS test set, listed in test_goldStandard/00-README.txt
Programming/NLP/test_goldStandard/*/*-chunks-gpt-*.txt # recorded output of paid non-deterministic GPT runs (gpt_chunks.py); the experimental record
Programming/NLP/alignments_unformatted_*.txt # NLP pipeline stage data, one alignment per line, read by create_alignments.py and format_alignments.py
Programming/NLP/output.txt # NLP pipeline stage data, one record per line
Programming/NLP/reformated.txt # NLP pipeline stage data, one record per line
Programming/WDWR/projekt/old_code/data10000.txt # numeric matrix, 10000 scenario rows, solver input
Programming/WDWR/projekt/R/scenarios.txt # generated scenario vectors, one per line
Programming/WDWR/projekt/wdwr/solutions1.txt # CPLEX MILP solver log
Programming/WDWR/projekt/wdwr/KRZYSZTOF_RUDNICKI_307585_WDWR_PROJEKT/scenarios.txt # submitted copy of scenarios.txt, must stay byte-identical
Programming/WDWR/projekt/wdwr/KRZYSZTOF_RUDNICKI_307585_WDWR_PROJEKT/solutions1.txt # submitted copy of the CPLEX solver log, must stay byte-identical

2
.pre-commit-config.yaml
View File

@ -4,7 +4,7 @@
# Ordering rule for this repo: a hook lands only once the thing it gates is
# already clean, so `pre-commit run --all-files` is green at every commit.
# Hooks still to come, each blocked on its own cleanup:
# - file-length blocked on the .tex/code splits and TODO-file-length-250.md
# - file-length blocked on the .tex/code splits (27 files, TODO-refactor-remaining.md)
# - mypy the six third-party-typing errors are GONE (ERSMS moved
# to a typed DeclarativeBase; PSD has a scoped override).
# Still blocked on ECOTE/NLP/EARIN, and on the fact that

106
TODO-file-length-250.md
View File

@ -1,106 +0,0 @@
# TODO: the 27 data `.txt` files the 250-line cap cannot classify
REMOVE ME AFTER FINISH
The 250-line cap applies to prose as well as code, and the shared gate treats
`.txt` as prose unless its mean line is under 25 characters (a wordlist). These
27 files are **data** with long lines, so the heuristic reads them as prose and
demands they be split. Splitting any of them corrupts it: in every one, a line
is a record.
Nothing here blocks the refactor. It blocks exactly one step — installing the
`file-length` pre-commit hook and workflow — so it must be settled before that
lands, and not before.
Run `bash ~/src/utils/scripts/check_file_length.sh --all` to see the current list.
## Why the existing mechanism does not cover them
`VENDORED_SUBPATHS` in `~/src/utils/file_length/_tables.py` matches **directory
prefixes** (`/{repo}/{sub}/`). That is enough for the 7 entries already in
`gates/vendored.txt`, and it is not enough here:
- `Programming/NLP/test_goldStandard/images/` holds the official SemEval corpus
**and** kuhy's own GPT outputs, side by side.
- `Programming/WDWR/projekt/R/` holds `scenarios.txt` (data) **and** `model.R`
(authored, 464 lines, must stay gated and covered).
A directory entry would exempt authored work in both cases. Per-file `# noqa`
style suppression is banned outright. So this needs a decision, not a workaround.
## The four categories
### 1. Official SemEval package — 12 files, genuinely third-party
The `STSint.testinput.{headlines,images,answers-students}.{sent1,sent2}[.chunk].txt`
files. Verified against the package's own manifest in
`Programming/NLP/test_goldStandard/00-README.txt`, which enumerates exactly
these, plus `evalF1.pl` and `wellformed.pl`, as the contents of the
"STS 2016 Task 2 Interpretable Semantic Textual Similarity TEST DATASET"
(authors: Eneko Agirre, Montse Maritxalar et al.).
Unambiguously vendored. The only question is *how* to express it.
### 2. kuhy's GPT outputs living inside the corpus directory — 6 files
```
Programming/NLP/test_goldStandard/headlines/headlines-chunks-gpt-{one,two}.txt
Programming/NLP/test_goldStandard/images/images-chunks-gpt-{one,two}.txt
Programming/NLP/test_goldStandard/student/students-chunks-gpt-{one,two}.txt
```
**Not** in the package manifest — these are output from `gpt_chunks.py`, written
next to the corpus they were derived from. Regenerable in principle, but only by
re-running non-deterministic paid GPT calls, so untracking them destroys the
experimental record rather than saving space.
### 3. NLP pipeline data — 4 files
```
Programming/NLP/alignments_unformatted_headlines.txt 375 lines
Programming/NLP/alignments_unformatted_student.txt 344
Programming/NLP/output.txt 348
Programming/NLP/reformated.txt 348
```
Intermediate stage data. `alignments_unformatted_headlines.txt` is read as input
by `create_alignments.py:6` and `format_alignments.py:7`, so relocating these
costs exactly two path constants — the cheapest category to fix.
### 4. WDWR solver data and output — 5 files
```
Programming/WDWR/projekt/old_code/data10000.txt 10000 lines (numeric matrix, solver input)
Programming/WDWR/projekt/R/scenarios.txt 999 (generated scenario vectors)
Programming/WDWR/projekt/wdwr/solutions1.txt 3038 (CPLEX MILP solver log)
Programming/WDWR/projekt/wdwr/KRZYSZTOF_.../scenarios.txt 999 (submission copy)
Programming/WDWR/projekt/wdwr/KRZYSZTOF_.../solutions1.txt 3038 (submission copy)
```
Two of these live under `KRZYSZTOF_RUDNICKI_307585_WDWR_PROJEKT/`, the submitted
deliverable. Decision 6 of the approved plan says leave semantic duplicates
alone, so **relocation is not available for those two** — moving a file inside a
submission alters what was submitted.
## Options
**A. Extend the shared manifest to support file paths.** Teach
`~/src/utils/file_length/exemptions.py:is_vendored` to read `gates/vendored.txt` at
the repo root, supporting exact files as well as directories. This is the design
originally chosen (one root manifest consumed by every gate) and is the only
option that handles all four categories.
Cost: `file_length` currently has **zero tests** and no coverage job in
`.github/workflows/shared-gates-tests.yml`. Adding code to a gate 16 repos
depend on means authoring its test suite first.
**B. Relocate categories 2–4 into `data/` subdirectories** and add `data` to the
manifest per course dir. Cheap for categories 2 and 3. Cannot touch the two
WDWR submission-copy files. Does not address category 1.
**C. Accept a documented residual** — install the file-length gate with these 27
paths listed here, and treat the list as the record. Honest, but it is an
allowlist by another name, which the fleet rules reject.
Recommendation: **A**, taken as its own `~/src/utils` change with its own tests and
CI job, sequenced before Milestone 2 step 4. B as a fallback for categories 2–3
if A is judged too large, leaving categories 1 and 4 to a narrower version of A.

9
TODO-refactor-remaining.md
View File

@ -3,8 +3,8 @@
REMOVE ME AFTER FINISH
What is done, what is not, and the real numbers, re-measured 2026-08-28 after
the lint campaign started. Companion to `TODO-file-length-250.md`, which parks
one specific blocking decision.
the lint campaign started. The 27 data `.txt` files the cap misread as prose
are exempted per file in `.file-length-exempt` (reason per entry, 2026-11-01).
## Done
@ -150,12 +150,11 @@ fell out is a course that had never been run by the harness at all:
## Still open
### 1. The 250-line cap: 50 files
### 1. The 250-line cap: 27 files
| Type | Count | Note |
|---|---|---|
| `.txt` | 27 | Blocked -- see `TODO-file-length-250.md` |
| `.py` | 10 | Several GREW during the lint pass, because docstrings and annotations are lines: `NLP/gpt_chunks.py` 911 -> 973, `EARIN/lab2/main.py` 581 -> 647, `PORR/.../linear_algebra_utils.py` 422 -> 553. Split them AFTER the lint work, not during. TRAK is already clear: its three over-cap files were split in the same pass that linted them, each verified by a byte-identical render. `twm_4/TWM_KerasIntro.py` joined the list at 407 lines and should NOT be split: it is an nbconvert export, and a partial one |
| `.py` | 11 | Several GREW during the lint pass, because docstrings and annotations are lines: `NLP/gpt_chunks.py` 911 -> 973, `EARIN/lab2/main.py` 581 -> 647, `PORR/.../linear_algebra_utils.py` 422 -> 553. Split them AFTER the lint work, not during. TRAK is already clear: its three over-cap files were split in the same pass that linted them, each verified by a byte-identical render. `twm_4/TWM_KerasIntro.py` joined the list at 407 lines and should NOT be split: it is an nbconvert export, and a partial one |
| `.cpp` | 9 | |
| `.c` | 4 | |
| `.java` | 2 | |