WUT_Computer_Science/Programming/NLP/compare_chunks.py
Krzysztof kuhy Rudnicki bd24dd57ea Hold the coursework to the same lint bar as the tooling
kuhy overruled the "do not fix coursework to be better than it was submitted"
rule: git history is the archive, so the working tree can be clean. This lands
the config and the mechanical half of the cleanup -- 9,281 ruff findings under
the fleet config down to 3,686, with no per-file-ignores anywhere.

What is in here:

- pyproject.toml, the first one in this repo. select = ["ALL"], mypy strict,
  and the vendored list mirrored into ruff exclude and coverage omit. Five
  rules are off and each says why; CPY001 is the only judgement call, because
  stamping a copyright header onto 137 university assignments would assert a
  licence that was never claimed. No hook is wired up yet -- a gate lands only
  once its subject is clean.

- ruff format + the safe fixes, then a second pass over the docstring rules
  ruff marks unsafe (they only edit docstring text). 108 files.

- TRAK/sightpy is vendored, not coursework: it is rafael-fuente/Python-Raytracer
  (BSD 3-Clause). Verified against a fresh upstream clone with line endings
  normalised -- 30 of 32 files byte-identical, ray.py and scene.py carrying a
  12- and 13-line local patch that counts rays for the report. 35 files that
  should never have been linted, now excluded and restored.

- WDWR, SPD, MOM and ECRYPT are fully clean, by hand and verified by running
  them: WDWR's three plot scripts print the same statistics as before, MOM's
  PuLP model solves to the same cost. WDWR's graph.py had a 342-character CSV
  header embedded as a string literal, which cannot be wrapped -- the data
  moved to data/pareto_front.csv. Two 0-byte .py stubs are deleted; an empty
  file is not a deliverable and git still has it.

Three harness bugs found by actually running things:

- `run` now forces MPLBACKEND=Agg. TRAK finished its ray trace in 37 seconds
  and then sat for 45 minutes: plt.show() had opened an image viewer and was
  waiting for someone to close the window.
- TRAK's main.py only pops that viewer with --show now, and its runner writes
  to a gitignored build/ via the new --entry-args. The default output path is
  outputs/output.png, which is TRACKED -- a plain run overwrote a submitted
  deliverable.
- course_lib.sh was 357 lines, over this repo's own cap. Split into
  course_probe.sh and course_run.sh.

Verified: ECOTE 26 tests pass, ECRYPT_PROJECT 99 pass, TRAK renders
cornell_box in 37s and exits 0, every module compiles, and every import ruff
removed was checked to be genuinely unused.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
2026-08-28 19:53:04 +02:00

83 lines
2.9 KiB
Python

def reformat_sentence(sentence):
"""Reformats a sentence by replacing slashes '/' with ' ]'."""
return sentence.replace("/", " ]")
def insert_commas(sentence, reference_sentence):
"""Inserts commas into 'sentence' at the same positions as they appear in 'reference_sentence'."""
# Splitting the sentences into chunks
sentence_chunks = sentence.split(" ] [ ")
reference_chunks = reference_sentence.split(" ] [ ")
# Insert commas into the original sentence based on the reference sentence
for i, chunk in enumerate(reference_chunks):
if "," in chunk and i < len(sentence_chunks):
sentence_chunks[i] += ","
# Reconstruct the sentence with inserted commas
return " ] [ ".join(sentence_chunks)
def process_sentences(sentences1, sentences2):
"""Processes two lists of sentences according to the specified rules."""
processed_sentences = []
for sentence1, sentence2 in zip(sentences1, sentences2):
# Reformat the first sentence
reformatted_sentence1 = reformat_sentence(sentence1)
reformatted_sentence1 = insert_commas(reformatted_sentence1, sentence2)
# Splitting the sentences into words, ignoring the square brackets
words1 = set(
" ".join(reformatted_sentence1.strip().split("] ["))
.replace("[", "")
.replace("]", "")
.lower()
.split()
)
words2 = set(
" ".join(sentence2.strip().split("] ["))
.replace("[", "")
.replace("]", "")
.lower()
.split()
)
# Finding differences in words
diff = words2 - words1
# Preparing the output sentence
output_sentence = reformatted_sentence1.strip()
if "[ . ]" in sentence2:
output_sentence += " [ . ]"
# Adding the differences and the output sentence to the processed_sentences list
processed_sentences.append(
{
"sentence1": reformatted_sentence1.strip(),
"sentence2": sentence2.strip(),
"differences": diff,
"merged_sentence": output_sentence,
}
)
if diff and diff != {"."}:
print("Difference found!")
print("Sentence 1:", reformatted_sentence1.strip())
print("Sentence 2:", sentence2.strip())
print("Differences:", diff)
print("\n")
return processed_sentences
# First, let's open and read the contents of both files to understand their structure and content.
file_path_1 = "test_goldStandard/images/images-chunks-gpt-one.txt"
file_path_2 = "test_goldStandard/images/STSint.testinput.images.sent1.chunk.txt"
# Reading the files
with open(file_path_1) as file1, open(file_path_2) as file2:
sentences_file_1 = file1.readlines()
sentences_file_2 = file2.readlines()
# Process the sentences
processed_sentences = process_sentences(sentences_file_1, sentences_file_2)