kuhyx/testsAndMisc
1
0
Fork 0
mirror of https://github.com/kuhyx/testsAndMisc.git synced 2026-11-10 18:47:04 +01:00
Code Issues Actions 6 Packages Projects Releases Wiki Activity

Split the code_tutor challenge flows out of _challenge

Browse Source 
_challenge.py 467 -> 236. The two flows and run_coding_challenge go to
_challenge_flows.py; the three static prompt strings to
_challenge_prompts.py, which covered the last eleven lines after the
first cut landed at 261.

The 13 patch strings in test_challenge_part3.py were repointed per test,
by which function is under test - the _run_user_impl tests keep
_challenge. Getting that wrong does not fail here, it HANGS: an
unintercepted _collect_and_rate_tests ran the real interactive flow and
spun the suite for two minutes. Runs are now under an explicit timeout,
and the patch targets are asserted to resolve before pytest starts.

Pass count is 216 before and after.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011w2Pj4Ek3ZNMQVYcFHgXDY
This commit is contained in:
Krzysztof Rudnicki 2026-08-16 17:42:50 +02:00
parent 40a4eaf5b5
commit 277cb9fd6f
7 changed files with 379 additions and 255 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

19
docs/superpowers/contracts/code-tutor-challenge-flows-split-2026-08-16.json Normal file
View File

@ -0,0 +1,19 @@
{
"title": "Split code_tutor/_challenge.py under the 250-line cap",
"objective": "Bring python_pkg/code_tutor/_challenge.py (467 lines) under the repo-wide 250-line cap by moving the two challenge flows and the public entry point into _challenge_flows.py, and the static prompt strings into _challenge_prompts.py.",
"acceptance_criteria": [
"_challenge.py, _challenge_flows.py and _challenge_prompts.py are each at most 250 lines.",
"The code_tutor suite still reports exactly 216 passing tests -- the same count as before the split, proving no test was dropped or silently skipped.",
"The package keeps 100% branch coverage of the code_tutor source modules.",
"Every patch string in test_challenge_part3.py names the module its function-under-test now resolves collaborators through, decided per test rather than by a blanket substitution: the _run_user_impl tests keep _challenge, the flow and entry-point tests move to _challenge_flows.",
"The suite completes in seconds. A patch that no longer intercepts here does not fail - it lets the real interactive flow run and the suite spins, so wall-clock time is part of the pass condition.",
"No assertion is changed and no test is deleted, skipped or weakened.",
"No suppression is added and no coverage exclusion is introduced."
],
"out_of_scope": [
"Changing what any test asserts, or any prompt text.",
"Splitting cli.py, _analyzer.py, _verifier.py, _llm.py or the code_tutor test modules, which are separate commits.",
"Adding a re-export shim in _challenge; _verifier.py points at _challenge_flows directly."
],
"verifier": "timeout 90 env PYTHONPATH=. python3 -m pytest python_pkg/code_tutor/tests -q (expects exactly 216 passed, well inside the timeout), plus an import-time assertion that every patched name resolves on the module its test names, plus pre-commit run --files on the touched files."
}

52
docs/superpowers/evidence/code-tutor-challenge-flows-split-2026-08-16.json Normal file
View File

@ -0,0 +1,52 @@
{
"intent": "Bring code_tutor/_challenge.py under the 250-line cap by moving the two challenge flows, the entry point and the static prompt strings into their own modules, with no behaviour change.",
"scope": [
"python_pkg/code_tutor/_challenge_flows.py, _challenge_prompts.py (new)",
"python_pkg/code_tutor/_challenge.py (467 -> 236)",
"python_pkg/code_tutor/_verifier.py (import path only)",
"python_pkg/code_tutor/tests/test_challenge_part3.py (import and patch paths only)",
"Non-goal: no assertion changed, no test deleted, skipped or weakened"
],
"changes": [
"Moved _write_tests_first_flow, _existing_tests_flow and run_coding_challenge into _challenge_flows.py, which imports the three interactive stages back from _challenge.",
"Moved _NO_AI_NOTICE, _MAX_TEST_ATTEMPTS and _TEST_JUDGE_SYSTEM into _challenge_prompts.py. The first cut left _challenge.py at 261, eleven over; these three constants are pure data with no patch targets, which made them the cheapest remaining eleven lines.",
"_verifier.py imports run_coding_challenge from _challenge_flows directly rather than through a re-export shim.",
"Repointed 13 patch strings in test_challenge_part3.py from _challenge to _challenge_flows, decided per test by which function is under test: the four _run_user_impl tests keep _challenge, the eight flow and entry-point tests move. A blanket substitution would have broken the first group.",
"_challenge.py dropped four imports the moved code solely used, and its module docstring now names where the flows went."
],
"verification": [
{
"command": "timeout 90 env PYTHONPATH=. python3 -m pytest python_pkg/code_tutor/tests -q",
"result": "pass",
"evidence": "216 passed in 2.1s - identical to the baseline count, and comfortably inside the timeout."
},
{
"command": "PYTHONPATH=. python3 -m pytest python_pkg/code_tutor/tests -q (intermediate run, before the patch strings were repointed)",
"result": "hang",
"evidence": "The suite spun for over two minutes and had to be killed. test_write_tests_first_flow_tests_none patched _collect_and_rate_tests on _challenge while _write_tests_first_flow had moved to _challenge_flows, so the patch stopped intercepting and the real interactive flow ran against a stub input function. This is the silent failure mode docs/python-split-recipes.md warns about: not a red test, but real execution. Every subsequent run was given an explicit timeout."
},
{
"command": "PYTHONPATH=. python3 -c \"<assert every patched name resolves on _challenge_flows and _challenge>\"",
"result": "pass",
"evidence": "all patch targets resolve - run before the pytest invocation specifically because a missing target here spins rather than errors."
},
{
"command": "wc -l python_pkg/code_tutor/_challenge.py python_pkg/code_tutor/_challenge_flows.py python_pkg/code_tutor/_challenge_prompts.py",
"result": "pass",
"evidence": "236, 242 and 36 lines, all under the 250-line cap."
},
{
"command": "pre-commit run --files <the touched files and artifacts>",
"result": "pass",
"evidence": "ruff caught F821 Undefined name 'Callable' - the new module's TYPE_CHECKING block needed it. Fixed; ruff, ruff format, mypy, pylint, bandit and pytest-with-coverage then all pass."
}
],
"risks": [
"This package's failure mode on a mispointed patch is a hang, not an error, because the code under test reads from an input callable in a retry loop. Any future split here should assert the patch targets resolve before running the suite, and should always run it under a timeout.",
"_challenge_flows imports from _challenge, which imports from _challenge_prompts, _challenge_support, _pytest_runner and _verdict. That direction must hold; _challenge importing from _challenge_flows would be a cycle."
],
"rollback": [
"git revert the commit; _challenge.py returns to 467 lines and the two new modules disappear.",
"After rollback, re-run the package suite and confirm it still reports 216 passed."
]
}

248
python_pkg/code_tutor/_challenge.py
View File

@ -11,9 +11,10 @@ to implement.
In both cases the original implementation is hidden during the challenge.
The lower-level discovery / pytest / verdict helpers live in
:mod:`python_pkg.code_tutor._challenge_support`; this module keeps the
interactive validation, rating and flow orchestration plus the public
``run_coding_challenge`` entry point.
:mod:`python_pkg.code_tutor._challenge_support`, and the two flows plus the
public ``run_coding_challenge`` entry point in
:mod:`python_pkg.code_tutor._challenge_flows`. This module keeps the three
interactive stages those flows are built from.
"""
from __future__ import annotations
@ -26,15 +27,16 @@ from typing import TYPE_CHECKING
from rich.panel import Panel
from rich.syntax import Syntax
from python_pkg.code_tutor._challenge_prompts import (
_MAX_TEST_ATTEMPTS,
_TEST_JUDGE_SYSTEM,
)
from python_pkg.code_tutor._challenge_support import (
_collect_lines,
_extract_signature_block,
_find_tests,
_import_hint,
_project_root,
_show_test_panels,
)
from python_pkg.code_tutor._pytest_runner import _patch_and_test, _pytest_clean
from python_pkg.code_tutor._pytest_runner import _pytest_clean
from python_pkg.code_tutor._verdict import _parse_verdict, _stream_verdict
if TYPE_CHECKING:
@ -45,34 +47,6 @@ if TYPE_CHECKING:
from python_pkg.code_tutor._analyzer import CodeItem
from python_pkg.code_tutor._llm import Backend
_NO_AI_NOTICE = (
"[bold yellow]⚠ No AI assistance -- write this yourself.[/bold yellow]\n"
"Your own explanation (and the tests below) are your only reference."
)
_MAX_TEST_ATTEMPTS = 2
_TEST_JUDGE_SYSTEM = (
"You are a code tutor evaluating whether a student's tests adequately"
" cover a function.\n\n"
"PASS if the tests:\n"
" - Include at least 2 meaningfully different test cases with distinct inputs\n"
" - Use assertions that verify the actual return value or observable behavior\n"
" - Would catch an obviously wrong implementation (e.g. one that always"
' returns "")\n\n'
"FAIL only if:\n"
" - There is only one trivial test case\n"
" - Assertions are always-true or don't check the function's real output\n"
" - All tests are essentially the same scenario with trivially different data\n\n"
"Do NOT require error/edge-case tests for pure transformation functions that"
" have no error handling. 2+ meaningful happy-path cases with distinct"
" inputs is enough.\n\n"
"Respond with valid JSON only, no other text:\n"
'{"verdict": "PASS" | "FAIL",'
' "gap": "<one sentence on the specific missing scenario,'
' or empty string on PASS>"}'
)
def _validate_tests_against_real(
test_code: str,
@ -261,207 +235,3 @@ def _run_user_impl(
return "passed"
console.print("[red]✗ Your implementation didn't pass your own tests.[/red]")
return "failed"
# ---------------------------------------------------------------------------
# "Write tests first" flow -- orchestrator
# ---------------------------------------------------------------------------
def _write_tests_first_flow(
item: CodeItem,
codebase_path: str,
user_explanation: str,
backend: Backend,
console: Console,
input_fn: Callable[[str], str],
) -> str:
"""Challenge flow used when no existing tests are found.
Asks the user to write tests first, rates them with the LLM, validates
them against the real implementation, then asks the user to write their
own implementation.
Args:
item: Code item being challenged.
codebase_path: Absolute codebase root path.
user_explanation: User's earlier explanation, shown as reference.
backend: LLM backend for test quality rating.
console: Rich console for output.
input_fn: Callable for reading user input.
Returns:
``"passed"``, ``"failed"``, or ``"skipped"``.
"""
project_root = _project_root(Path(codebase_path))
import_hint = _import_hint(item, codebase_path, project_root)
sig_block = _extract_signature_block(item, codebase_path)
console.print(
"\n[bold cyan]Coding challenge (tests first):[/bold cyan] "
"No existing tests found -- write them before implementing.\n"
)
answer = input_fn("Take the challenge? [y/N] ").strip().lower()
if answer != "y":
return "skipped"
console.print(
Panel(
user_explanation,
title="Your explanation (your only reference)",
border_style="dim",
)
)
console.print(_NO_AI_NOTICE)
console.print(
Panel(
Syntax(sig_block, "python", theme="monokai"),
title="[blue]Function signature + contract[/blue]",
border_style="blue",
)
)
console.print(
f"\n[dim]Auto-import that will be prepended to your tests:[/dim]\n"
f"[cyan]{import_hint}[/cyan]\n"
"[dim]Add any other imports (pytest, MagicMock, etc.) yourself.[/dim]\n"
)
test_code = _collect_and_rate_tests(
sig_block, user_explanation, backend, console, input_fn
)
if test_code is None:
return "skipped"
console.print(
"\n[dim]Validating your tests against the real implementation...[/dim]"
)
if not _validate_tests_against_real(test_code, import_hint, project_root, console):
console.print(
"[red]Your tests fail on the correct implementation -- "
"they may be testing the wrong behavior. Skipping.[/red]"
)
return "skipped"
console.print(
"[green]Tests look good -- they pass on the real implementation.[/green]"
)
return _run_user_impl(
item, codebase_path, test_code, import_hint, console, input_fn
)
# ---------------------------------------------------------------------------
# "Existing tests" flow
# ---------------------------------------------------------------------------
def _existing_tests_flow(
item: CodeItem,
codebase_path: str,
user_explanation: str,
test_entries: list[tuple[Path, list[str]]],
console: Console,
input_fn: Callable[[str], str],
) -> str:
"""Challenge flow used when existing tests are found.
Shows the tests, asks user to implement from scratch.
Args:
item: Code item being challenged.
codebase_path: Absolute codebase root path.
user_explanation: User's earlier explanation.
test_entries: Existing test file/node-id pairs.
console: Rich console for output.
input_fn: Callable for reading user input.
Returns:
``"passed"``, ``"failed"``, or ``"skipped"``.
"""
n_tests = sum(len(ids) for _, ids in test_entries)
console.print(
"\n[bold cyan]Coding challenge:[/bold cyan] "
f"Can you rewrite this function from scratch? "
f"[dim]({n_tests} test(s) will validate it)[/dim]"
)
answer = input_fn("Take the challenge? [y/N] ").strip().lower()
if answer != "y":
return "skipped"
console.print(
Panel(
user_explanation,
title="Your explanation (your only reference)",
border_style="dim",
)
)
console.print(_NO_AI_NOTICE)
_show_test_panels(test_entries, console)
user_code = _collect_lines(
f"\n[bold]Write [cyan]{item.name}[/cyan] from scratch.[/bold] "
"Finish with [dim]END[/dim] on a blank line, or [dim]skip[/dim] to skip.",
console,
input_fn,
)
if user_code is None:
console.print("[yellow]Challenge skipped.[/yellow]")
return "skipped"
console.print("\n[dim]Running tests against your implementation...[/dim]")
passed = _patch_and_test(item, codebase_path, user_code, test_entries, console)
if passed:
console.print(
"[green bold]✓ All tests passed -- you really understand it![/green bold]"
)
return "passed"
console.print("[red]✗ Some tests failed -- try again next session.[/red]")
return "failed"
# ---------------------------------------------------------------------------
# Public entry point
# ---------------------------------------------------------------------------
def run_coding_challenge(
item: CodeItem,
codebase_path: str,
user_explanation: str,
backend: Backend,
console: Console,
input_fn: Callable[[str], str] = input,
) -> str:
"""Offer the user a coding challenge after a PASS verdict.
Routes to one of two flows:
- **Existing tests found**: show tests, user writes implementation.
- **No tests found**: user writes tests first (LLM-rated), then implementation.
Only offered for ``.py`` files.
Args:
item: The code item to challenge on.
codebase_path: Absolute path of the codebase root.
user_explanation: The explanation the user gave during the lesson.
backend: LLM backend (used for test quality rating in the no-tests flow).
console: Rich console for output.
input_fn: Callable for reading user input.
Returns:
``"passed"``, ``"failed"``, or ``"skipped"``.
"""
if not item.file.endswith(".py"):
return "skipped"
codebase = Path(codebase_path)
test_entries = _find_tests(item, codebase)
if test_entries:
return _existing_tests_flow(
item, codebase_path, user_explanation, test_entries, console, input_fn
)
return _write_tests_first_flow(
item, codebase_path, user_explanation, backend, console, input_fn
)

242
python_pkg/code_tutor/_challenge_flows.py Normal file
View File

@ -0,0 +1,242 @@
"""The two challenge flows and the entry point that picks between them.
Split out of :mod:`python_pkg.code_tutor._challenge` to keep it under the
250-line cap. The three stage helpers stay there and are imported back, because
``tests/test_challenge_part3.py`` patches them on ``_challenge``.
"""
from __future__ import annotations
from pathlib import Path
from typing import TYPE_CHECKING
from rich.panel import Panel
from rich.syntax import Syntax
from python_pkg.code_tutor._challenge import (
_collect_and_rate_tests,
_run_user_impl,
_validate_tests_against_real,
)
from python_pkg.code_tutor._challenge_prompts import _NO_AI_NOTICE
from python_pkg.code_tutor._challenge_support import (
_collect_lines,
_extract_signature_block,
_find_tests,
_import_hint,
_project_root,
_show_test_panels,
)
from python_pkg.code_tutor._pytest_runner import _patch_and_test
if TYPE_CHECKING:
from collections.abc import Callable
from rich.console import Console
from python_pkg.code_tutor._analyzer import CodeItem
from python_pkg.code_tutor._llm import Backend
# ---------------------------------------------------------------------------
# "Write tests first" flow -- orchestrator
# ---------------------------------------------------------------------------
def _write_tests_first_flow(
item: CodeItem,
codebase_path: str,
user_explanation: str,
backend: Backend,
console: Console,
input_fn: Callable[[str], str],
) -> str:
"""Challenge flow used when no existing tests are found.
Asks the user to write tests first, rates them with the LLM, validates
them against the real implementation, then asks the user to write their
own implementation.
Args:
item: Code item being challenged.
codebase_path: Absolute codebase root path.
user_explanation: User's earlier explanation, shown as reference.
backend: LLM backend for test quality rating.
console: Rich console for output.
input_fn: Callable for reading user input.
Returns:
``"passed"``, ``"failed"``, or ``"skipped"``.
"""
project_root = _project_root(Path(codebase_path))
import_hint = _import_hint(item, codebase_path, project_root)
sig_block = _extract_signature_block(item, codebase_path)
console.print(
"\n[bold cyan]Coding challenge (tests first):[/bold cyan] "
"No existing tests found -- write them before implementing.\n"
)
answer = input_fn("Take the challenge? [y/N] ").strip().lower()
if answer != "y":
return "skipped"
console.print(
Panel(
user_explanation,
title="Your explanation (your only reference)",
border_style="dim",
)
)
console.print(_NO_AI_NOTICE)
console.print(
Panel(
Syntax(sig_block, "python", theme="monokai"),
title="[blue]Function signature + contract[/blue]",
border_style="blue",
)
)
console.print(
f"\n[dim]Auto-import that will be prepended to your tests:[/dim]\n"
f"[cyan]{import_hint}[/cyan]\n"
"[dim]Add any other imports (pytest, MagicMock, etc.) yourself.[/dim]\n"
)
test_code = _collect_and_rate_tests(
sig_block, user_explanation, backend, console, input_fn
)
if test_code is None:
return "skipped"
console.print(
"\n[dim]Validating your tests against the real implementation...[/dim]"
)
if not _validate_tests_against_real(test_code, import_hint, project_root, console):
console.print(
"[red]Your tests fail on the correct implementation -- "
"they may be testing the wrong behavior. Skipping.[/red]"
)
return "skipped"
console.print(
"[green]Tests look good -- they pass on the real implementation.[/green]"
)
return _run_user_impl(
item, codebase_path, test_code, import_hint, console, input_fn
)
# ---------------------------------------------------------------------------
# "Existing tests" flow
# ---------------------------------------------------------------------------
def _existing_tests_flow(
item: CodeItem,
codebase_path: str,
user_explanation: str,
test_entries: list[tuple[Path, list[str]]],
console: Console,
input_fn: Callable[[str], str],
) -> str:
"""Challenge flow used when existing tests are found.
Shows the tests, asks user to implement from scratch.
Args:
item: Code item being challenged.
codebase_path: Absolute codebase root path.
user_explanation: User's earlier explanation.
test_entries: Existing test file/node-id pairs.
console: Rich console for output.
input_fn: Callable for reading user input.
Returns:
``"passed"``, ``"failed"``, or ``"skipped"``.
"""
n_tests = sum(len(ids) for _, ids in test_entries)
console.print(
"\n[bold cyan]Coding challenge:[/bold cyan] "
f"Can you rewrite this function from scratch? "
f"[dim]({n_tests} test(s) will validate it)[/dim]"
)
answer = input_fn("Take the challenge? [y/N] ").strip().lower()
if answer != "y":
return "skipped"
console.print(
Panel(
user_explanation,
title="Your explanation (your only reference)",
border_style="dim",
)
)
console.print(_NO_AI_NOTICE)
_show_test_panels(test_entries, console)
user_code = _collect_lines(
f"\n[bold]Write [cyan]{item.name}[/cyan] from scratch.[/bold] "
"Finish with [dim]END[/dim] on a blank line, or [dim]skip[/dim] to skip.",
console,
input_fn,
)
if user_code is None:
console.print("[yellow]Challenge skipped.[/yellow]")
return "skipped"
console.print("\n[dim]Running tests against your implementation...[/dim]")
passed = _patch_and_test(item, codebase_path, user_code, test_entries, console)
if passed:
console.print(
"[green bold]✓ All tests passed -- you really understand it![/green bold]"
)
return "passed"
console.print("[red]✗ Some tests failed -- try again next session.[/red]")
return "failed"
# ---------------------------------------------------------------------------
# Public entry point
# ---------------------------------------------------------------------------
def run_coding_challenge(
item: CodeItem,
codebase_path: str,
user_explanation: str,
backend: Backend,
console: Console,
input_fn: Callable[[str], str] = input,
) -> str:
"""Offer the user a coding challenge after a PASS verdict.
Routes to one of two flows:
- **Existing tests found**: show tests, user writes implementation.
- **No tests found**: user writes tests first (LLM-rated), then implementation.
Only offered for ``.py`` files.
Args:
item: The code item to challenge on.
codebase_path: Absolute path of the codebase root.
user_explanation: The explanation the user gave during the lesson.
backend: LLM backend (used for test quality rating in the no-tests flow).
console: Rich console for output.
input_fn: Callable for reading user input.
Returns:
``"passed"``, ``"failed"``, or ``"skipped"``.
"""
if not item.file.endswith(".py"):
return "skipped"
codebase = Path(codebase_path)
test_entries = _find_tests(item, codebase)
if test_entries:
return _existing_tests_flow(
item, codebase_path, user_explanation, test_entries, console, input_fn
)
return _write_tests_first_flow(
item, codebase_path, user_explanation, backend, console, input_fn
)

36
python_pkg/code_tutor/_challenge_prompts.py Normal file
View File

@ -0,0 +1,36 @@
"""Static prompt and notice strings for the coding challenge.
Split out of :mod:`python_pkg.code_tutor._challenge` to keep it under the
250-line cap. These are pure data with no collaborators, so nothing patches
them and they can live anywhere.
"""
from __future__ import annotations
_NO_AI_NOTICE = (
"[bold yellow]⚠ No AI assistance -- write this yourself.[/bold yellow]\n"
"Your own explanation (and the tests below) are your only reference."
)
_MAX_TEST_ATTEMPTS = 2
_TEST_JUDGE_SYSTEM = (
"You are a code tutor evaluating whether a student's tests adequately"
" cover a function.\n\n"
"PASS if the tests:\n"
" - Include at least 2 meaningfully different test cases with distinct inputs\n"
" - Use assertions that verify the actual return value or observable behavior\n"
" - Would catch an obviously wrong implementation (e.g. one that always"
' returns "")\n\n'
"FAIL only if:\n"
" - There is only one trivial test case\n"
" - Assertions are always-true or don't check the function's real output\n"
" - All tests are essentially the same scenario with trivially different data\n\n"
"Do NOT require error/edge-case tests for pure transformation functions that"
" have no error handling. 2+ meaningful happy-path cases with distinct"
" inputs is enough.\n\n"
"Respond with valid JSON only, no other text:\n"
'{"verdict": "PASS" | "FAIL",'
' "gap": "<one sentence on the specific missing scenario,'
' or empty string on PASS>"}'
)

2
python_pkg/code_tutor/_verifier.py
View File

@ -16,7 +16,7 @@ from rich.live import Live
from rich.panel import Panel
from rich.text import Text
from python_pkg.code_tutor._challenge import run_coding_challenge
from python_pkg.code_tutor._challenge_flows import run_coding_challenge
from python_pkg.code_tutor._progress import LessonRecord
if TYPE_CHECKING:

35
python_pkg/code_tutor/tests/test_challenge_part3.py
View File

@ -10,9 +10,9 @@ from __future__ import annotations
from typing import TYPE_CHECKING
from unittest.mock import MagicMock, patch
from python_pkg.code_tutor._challenge import (
from python_pkg.code_tutor._challenge import _run_user_impl
from python_pkg.code_tutor._challenge_flows import (
_existing_tests_flow,
_run_user_impl,
_write_tests_first_flow,
run_coding_challenge,
)
@ -151,10 +151,10 @@ def test_write_tests_first_flow_tests_none(tmp_path: Path) -> None:
with (
patch(
"python_pkg.code_tutor._challenge._collect_and_rate_tests",
"python_pkg.code_tutor._challenge_flows._collect_and_rate_tests",
return_value=None,
),
patch("python_pkg.code_tutor._challenge.Syntax"),
patch("python_pkg.code_tutor._challenge_flows.Syntax"),
):
result = _write_tests_first_flow(
_item(file="mod.py"),
@ -174,14 +174,14 @@ def test_write_tests_first_flow_tests_fail_real(tmp_path: Path) -> None:
with (
patch(
"python_pkg.code_tutor._challenge._collect_and_rate_tests",
"python_pkg.code_tutor._challenge_flows._collect_and_rate_tests",
return_value="test code",
),
patch(
"python_pkg.code_tutor._challenge._validate_tests_against_real",
"python_pkg.code_tutor._challenge_flows._validate_tests_against_real",
return_value=False,
),
patch("python_pkg.code_tutor._challenge.Syntax"),
patch("python_pkg.code_tutor._challenge_flows.Syntax"),
):
result = _write_tests_first_flow(
_item(file="mod.py"),
@ -201,18 +201,18 @@ def test_write_tests_first_flow_success(tmp_path: Path) -> None:
with (
patch(
"python_pkg.code_tutor._challenge._collect_and_rate_tests",
"python_pkg.code_tutor._challenge_flows._collect_and_rate_tests",
return_value="test code",
),
patch(
"python_pkg.code_tutor._challenge._validate_tests_against_real",
"python_pkg.code_tutor._challenge_flows._validate_tests_against_real",
return_value=True,
),
patch(
"python_pkg.code_tutor._challenge._run_user_impl",
"python_pkg.code_tutor._challenge_flows._run_user_impl",
return_value="passed",
),
patch("python_pkg.code_tutor._challenge.Syntax"),
patch("python_pkg.code_tutor._challenge_flows.Syntax"),
):
result = _write_tests_first_flow(
_item(file="mod.py"),
@ -279,7 +279,9 @@ def test_existing_tests_flow_passes(tmp_path: Path) -> None:
with (
patch("python_pkg.code_tutor._challenge_support.Syntax"),
patch("python_pkg.code_tutor._challenge._patch_and_test", return_value=True),
patch(
"python_pkg.code_tutor._challenge_flows._patch_and_test", return_value=True
),
):
result = _existing_tests_flow(
_item(file="mod.py"),
@ -303,7 +305,9 @@ def test_existing_tests_flow_fails(tmp_path: Path) -> None:
with (
patch("python_pkg.code_tutor._challenge_support.Syntax"),
patch("python_pkg.code_tutor._challenge._patch_and_test", return_value=False),
patch(
"python_pkg.code_tutor._challenge_flows._patch_and_test", return_value=False
),
):
result = _existing_tests_flow(
_item(file="mod.py"),
@ -342,7 +346,8 @@ def test_run_coding_challenge_with_existing_tests(tmp_path: Path) -> None:
mock_backend = MagicMock()
with patch(
"python_pkg.code_tutor._challenge._existing_tests_flow", return_value="passed"
"python_pkg.code_tutor._challenge_flows._existing_tests_flow",
return_value="passed",
):
result = run_coding_challenge(
_item(file="mod.py"),
@ -360,7 +365,7 @@ def test_run_coding_challenge_no_tests(tmp_path: Path) -> None:
mock_backend = MagicMock()
with patch(
"python_pkg.code_tutor._challenge._write_tests_first_flow",
"python_pkg.code_tutor._challenge_flows._write_tests_first_flow",
return_value="skipped",
):
result = run_coding_challenge(