Declare vendored subtrees and park the data-txt cap decision

First step of bringing this archive up to the fleet standard: nothing can be
linted until the gates know what is not kuhy's code.

gates/vendored.txt is the source of truth, one path per line with the reason
AND how it was verified, because an unverified entry is a suppression wearing
a costume. Seven subtrees: the ASP.NET scaffolder's jQuery/Bootstrap bundle
(~98% of all JS and CSS here), stock Unity package and template assets,
manufacturer-supplied Altium/PADS component libraries for the OLED and
HMC5883L parts, and the two EOPSY simulator trees.

The EOPSY entries were the one judgement call, and they turned out not to be:
work/ is the extraction target of `make setup`, and extracting the committed
ftp/*.tgz and diffing shows 14/14 java files byte-identical to the department
originals. ControlPanel.java (1852 lines) and Kernel.java (511) are therefore
not student work and are not split.

Deliberately NOT vendored: EMISY's README, declaration and report are kuhy's
own writing; E-PSYOPS Assets/Scripts (12 .cs) is authored; and the Unity Input
System binding is already exempt via its auto-generated banner, so it needs no
entry. Narrow entries beat one blanket directory.

TODO-file-length-250.md parks the one thing this mechanism cannot express: 27
data .txt files whose lines are long enough that the cap reads them as prose.
They resist directory-prefix exemption because the SemEval corpus dirs mix the
official distribution with kuhy's own GPT outputs, and WDWR/projekt/R holds
both scenarios.txt and the authored model.R. That blocks only the installation
of the file-length gate, which is several milestones away.

Mirrored into the shared gate as kuhyx/utils@a4a3e0a.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
This commit is contained in:
Krzysztof Rudnicki 2026-08-28 17:31:30 +02:00
parent 2d85cbbd74
commit fd81d75424
2 changed files with 170 additions and 0 deletions

106
TODO-file-length-250.md Normal file
View File

@ -0,0 +1,106 @@
# TODO: the 27 data `.txt` files the 250-line cap cannot classify
REMOVE ME AFTER FINISH
The 250-line cap applies to prose as well as code, and the shared gate treats
`.txt` as prose unless its mean line is under 25 characters (a wordlist). These
27 files are **data** with long lines, so the heuristic reads them as prose and
demands they be split. Splitting any of them corrupts it: in every one, a line
is a record.
Nothing here blocks the refactor. It blocks exactly one step — installing the
`file-length` pre-commit hook and workflow — so it must be settled before that
lands, and not before.
Run `bash ~/utils/scripts/check_file_length.sh --all` to see the current list.
## Why the existing mechanism does not cover them
`VENDORED_SUBPATHS` in `~/utils/file_length/_tables.py` matches **directory
prefixes** (`/{repo}/{sub}/`). That is enough for the 7 entries already in
`gates/vendored.txt`, and it is not enough here:
- `Programming/NLP/test_goldStandard/images/` holds the official SemEval corpus
**and** kuhy's own GPT outputs, side by side.
- `Programming/WDWR/projekt/R/` holds `scenarios.txt` (data) **and** `model.R`
(authored, 464 lines, must stay gated and covered).
A directory entry would exempt authored work in both cases. Per-file `# noqa`
style suppression is banned outright. So this needs a decision, not a workaround.
## The four categories
### 1. Official SemEval package — 12 files, genuinely third-party
The `STSint.testinput.{headlines,images,answers-students}.{sent1,sent2}[.chunk].txt`
files. Verified against the package's own manifest in
`Programming/NLP/test_goldStandard/00-README.txt`, which enumerates exactly
these, plus `evalF1.pl` and `wellformed.pl`, as the contents of the
"STS 2016 Task 2 Interpretable Semantic Textual Similarity TEST DATASET"
(authors: Eneko Agirre, Montse Maritxalar et al.).
Unambiguously vendored. The only question is *how* to express it.
### 2. kuhy's GPT outputs living inside the corpus directory — 6 files
```
Programming/NLP/test_goldStandard/headlines/headlines-chunks-gpt-{one,two}.txt
Programming/NLP/test_goldStandard/images/images-chunks-gpt-{one,two}.txt
Programming/NLP/test_goldStandard/student/students-chunks-gpt-{one,two}.txt
```
**Not** in the package manifest — these are output from `gpt_chunks.py`, written
next to the corpus they were derived from. Regenerable in principle, but only by
re-running non-deterministic paid GPT calls, so untracking them destroys the
experimental record rather than saving space.
### 3. NLP pipeline data — 4 files
```
Programming/NLP/alignments_unformatted_headlines.txt 375 lines
Programming/NLP/alignments_unformatted_student.txt 344
Programming/NLP/output.txt 348
Programming/NLP/reformated.txt 348
```
Intermediate stage data. `alignments_unformatted_headlines.txt` is read as input
by `create_alignments.py:6` and `format_alignments.py:7`, so relocating these
costs exactly two path constants — the cheapest category to fix.
### 4. WDWR solver data and output — 5 files
```
Programming/WDWR/projekt/old_code/data10000.txt 10000 lines (numeric matrix, solver input)
Programming/WDWR/projekt/R/scenarios.txt 999 (generated scenario vectors)
Programming/WDWR/projekt/wdwr/solutions1.txt 3038 (CPLEX MILP solver log)
Programming/WDWR/projekt/wdwr/KRZYSZTOF_.../scenarios.txt 999 (submission copy)
Programming/WDWR/projekt/wdwr/KRZYSZTOF_.../solutions1.txt 3038 (submission copy)
```
Two of these live under `KRZYSZTOF_RUDNICKI_307585_WDWR_PROJEKT/`, the submitted
deliverable. Decision 6 of the approved plan says leave semantic duplicates
alone, so **relocation is not available for those two** — moving a file inside a
submission alters what was submitted.
## Options
**A. Extend the shared manifest to support file paths.** Teach
`~/utils/file_length/exemptions.py:is_vendored` to read `gates/vendored.txt` at
the repo root, supporting exact files as well as directories. This is the design
originally chosen (one root manifest consumed by every gate) and is the only
option that handles all four categories.
Cost: `file_length` currently has **zero tests** and no coverage job in
`.github/workflows/shared-gates-tests.yml`. Adding code to a gate 16 repos
depend on means authoring its test suite first.
**B. Relocate categories 2–4 into `data/` subdirectories** and add `data` to the
manifest per course dir. Cheap for categories 2 and 3. Cannot touch the two
WDWR submission-copy files. Does not address category 1.
**C. Accept a documented residual** — install the file-length gate with these 27
paths listed here, and treat the list as the record. Honest, but it is an
allowlist by another name, which the fleet rules reject.
Recommendation: **A**, taken as its own `~/utils` change with its own tests and
CI job, sequenced before Milestone 2 step 4. B as a fallback for categories 2–3
if A is judged too large, leaving categories 1 and 4 to a narrower version of A.

64
gates/vendored.txt Normal file
View File

@ -0,0 +1,64 @@
# gates/vendored.txt -- third-party and department-provided content.
#
# THE source of truth for "not kuhy's code". Every gate reads this list:
# - the 250-line cap and the md-naming gate, via the "WUT_Computer_Science"
# key in VENDORED_SUBPATHS (~/utils/file_length/_tables.py)
# - ruff, via [tool.ruff] exclude in pyproject.toml
# - coverage, via [tool.coverage.run] omit
# - prettier, via .prettierignore
# Those four are MIRRORS. Edit this file, then re-run ./run.sh sync-vendored.
#
# Vendored content is exempt from lint, coverage and the line cap. It is NOT
# exempt from the no-binaries gate -- a vendored .png is still a binary.
#
# One path per line, repo-relative, no trailing slash. Every entry carries the
# reason AND how it was verified, because an unverified entry is a suppression
# wearing a costume.
# --- Third-party web libraries -------------------------------------------
# jQuery, Bootstrap, jquery-validation, jquery-validation-unobtrusive as
# shipped by the ASP.NET MVC scaffolder. 14 JS files (30,875 lines) and 16 CSS
# files (43,027 lines) -- 98.6% of all JS and 98.9% of all CSS in the repo.
# Verified: directory is the scaffolder's wwwroot/lib layout; files carry
# upstream copyright headers and match released dist/ builds.
Programming/EGUI/lab2/lab2/wwwroot/lib
# --- Unity engine and template content ------------------------------------
# Stock TextMesh Pro package assets shipped by Unity, not authored here.
# Verified: matches the TMP package's Assets payload; no file under it is
# referenced from Assets/Scripts.
Programming/E-PSYOPS/theProject/Assets/TextMesh Pro
# Unity's "Tutorial Info" new-project template (Readme asset + its editor
# script). Verified: unmodified from the Unity 2D template.
Programming/E-PSYOPS/theProject/Assets/TutorialInfo
#
# NOTE: Assets/Input System/BetterInput.cs is NOT listed. It is generated by
# com.unity.inputsystem's InputActionCodeGenerator and already carries an
# "auto-generated by" banner in its first 5 lines, so the shared gate's
# GENERATED_MARKERS check exempts it without an entry here. Assets/Scripts/**
# (12 .cs files) is authored and stays fully gated.
# --- Electronic component libraries ---------------------------------------
# Manufacturer-supplied Altium / EAGLE / PADS / Proteus / SOLIDWORKS symbol
# and footprint libraries for the OLED display and the HMC5883L magnetometer,
# downloaded from the vendors. Includes their own license.txt and part_info.txt.
# Verified: each tree contains the vendor's PADS/ export and license file.
# Only these two subtrees -- EMISY's README.md, declaration/ and report/ are
# kuhy's own writing and stay capped.
NotProgramming/EMISY/componentSchematic
NotProgramming/EMISY/schematic
# --- Department-provided simulator source ---------------------------------
# The EOPSY course ships its scheduler and memory-management simulators as
# tarballs; the lab is to run and analyse them, and the report is the
# deliverable. work/ is not hand-written -- it is the extraction target of
# `make setup` (see each task's README).
#
# Verified 2026-08-28 by extracting the committed tarballs and diffing:
# ftp/sched.tgz -> lab3 work/: 6/6 java files BYTE-IDENTICAL
# ftp/memory.tgz -> lab4 work/: 8/8 java files BYTE-IDENTICAL
# Zero student modification, and fully regenerable from ftp/. That is what
# makes this vendored rather than a judgement call -- and it is why
# ControlPanel.java (1852 lines) and Kernel.java (511) are not split.
Programming/EOPSY/lab3/task3/work
Programming/EOPSY/lab4/task4/work