WUT_Computer_Science/Programming/NLP/test_goldStandard/chunks.py
Krzysztof kuhy Rudnicki bd24dd57ea Hold the coursework to the same lint bar as the tooling
kuhy overruled the "do not fix coursework to be better than it was submitted"
rule: git history is the archive, so the working tree can be clean. This lands
the config and the mechanical half of the cleanup -- 9,281 ruff findings under
the fleet config down to 3,686, with no per-file-ignores anywhere.

What is in here:

- pyproject.toml, the first one in this repo. select = ["ALL"], mypy strict,
  and the vendored list mirrored into ruff exclude and coverage omit. Five
  rules are off and each says why; CPY001 is the only judgement call, because
  stamping a copyright header onto 137 university assignments would assert a
  licence that was never claimed. No hook is wired up yet -- a gate lands only
  once its subject is clean.

- ruff format + the safe fixes, then a second pass over the docstring rules
  ruff marks unsafe (they only edit docstring text). 108 files.

- TRAK/sightpy is vendored, not coursework: it is rafael-fuente/Python-Raytracer
  (BSD 3-Clause). Verified against a fresh upstream clone with line endings
  normalised -- 30 of 32 files byte-identical, ray.py and scene.py carrying a
  12- and 13-line local patch that counts rays for the report. 35 files that
  should never have been linted, now excluded and restored.

- WDWR, SPD, MOM and ECRYPT are fully clean, by hand and verified by running
  them: WDWR's three plot scripts print the same statistics as before, MOM's
  PuLP model solves to the same cost. WDWR's graph.py had a 342-character CSV
  header embedded as a string literal, which cannot be wrapped -- the data
  moved to data/pareto_front.csv. Two 0-byte .py stubs are deleted; an empty
  file is not a deliverable and git still has it.

Three harness bugs found by actually running things:

- `run` now forces MPLBACKEND=Agg. TRAK finished its ray trace in 37 seconds
  and then sat for 45 minutes: plt.show() had opened an image viewer and was
  waiting for someone to close the window.
- TRAK's main.py only pops that viewer with --show now, and its runner writes
  to a gitignored build/ via the new --entry-args. The default output path is
  outputs/output.png, which is TRACKED -- a plain run overwrote a submitted
  deliverable.
- course_lib.sh was 357 lines, over this repo's own cap. Split into
  course_probe.sh and course_run.sh.

Verified: ECOTE 26 tests pass, ECRYPT_PROJECT 99 pass, TRAK renders
cornell_box in 37s and exits 0, every module compiles, and every import ruff
removed was checked to be genuinely unused.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172dj34FDpao6wsix8ZuCzE
2026-08-28 19:53:04 +02:00

51 lines
1.7 KiB
Python

from openai import OpenAI
def sentence_loop(sentences):
# Process each sentence
chunked_sentences = []
for sentence in sentences:
chunked_sentences = process_sentence(sentence, chunked_sentences)
return chunked_sentences
def process_sentence(sentence, chunked_sentences):
response = client.chat.completions.create(
messages=[
{
"role": "user",
"content": "Please divide the following sentence into iSTS chunks. Format the response as: [[chunk 1]] [[chunk 2]] ... Ensure each chunk is enclosed in double square brackets and separated by a space. If you cannot correctly chunkify the sentence, return an error message in the format: 'ERROR: Cannot chunkify sentence: <sentence>'. Here is the sentence: "
+ sentence,
}
],
model="gpt-3.5-turbo",
)
chunked_sentence = response.choices[0].message.content
print("chunked_sentence: ", chunked_sentence)
chunked_sentences.append(chunked_sentence)
return chunked_sentences
def chunk_sentences(file_path, output_path, api_key):
# Read the sentences from the file
with open(file_path) as file:
sentences = file.readlines()
# Process each sentence
chunk_sentences = sentence_loop(sentences)
print(chunk_sentences)
# Write the chunked sentences to a new file
with open(output_path, "w") as output_file:
output_file.writelines(sentence + "\n" for sentence in chunk_sentences)
# Usage
file_path = "student/STSint.testinput.answers-students.sent2.txt"
output_path = "chunks_gpt_two.txt"
# Replace me with os.environ['API_KEY']
api_key = "REDACTED_OPENAI_API_KEY"
client = OpenAI(api_key=api_key)
chunk_sentences(file_path, output_path, api_key)