kuhyx/testsAndMisc
1
0
Fork 0
mirror of https://github.com/kuhyx/testsAndMisc.git synced 2026-11-10 18:47:04 +01:00
Code Issues Actions 6 Packages Projects Releases Wiki Activity

Clear the seven over-cap code_tutor test modules
Some checks are pending
Pre-commit checks / pre-commit (push) Waiting to run
Details
Python tests / test (push) Waiting to run
Details
Shell tests / shell-tests (push) Waiting to run
Details
Shell tests / arch-tests (push) Waiting to run
Details

Browse Source 
Partitioned at the section banners each file already used, into ten new
modules named for what they cover. python_pkg/code_tutor is now entirely
under the cap, from thirteen violations.

The splitter copies the whole original header into each new file and
lets ruff's F401 pass prune it - 59 imports removed correctly, versus a
hand-written import block failing on every earlier test split today.
That does not cover module-level helpers: _make_verifier sat between two
banners and was left behind by its callers, so it moved to conftest.py,
taking test_verifier.py's shadowing copy of _make_live_mock with it.

No patch string changed. Pass count is 217 before and after.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011w2Pj4Ek3ZNMQVYcFHgXDY
This commit is contained in:
Krzysztof Rudnicki 2026-08-16 18:06:38 +02:00
parent eb322dcf80
commit e3214c813c
21 changed files with 1835 additions and 1555 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

19
docs/superpowers/contracts/code-tutor-test-split-2026-08-16.json Normal file
View File

@ -0,0 +1,19 @@
{
"title": "Clear the seven over-cap code_tutor test modules",
"objective": "Bring every python_pkg/code_tutor test module under the repo-wide 250-line cap by partitioning them at the banner comments that already mark each function-under-test, taking code_tutor from thirteen violations to zero.",
"acceptance_criteria": [
"bash ~/utils/scripts/check_file_length.sh --all lists no file under python_pkg/code_tutor.",
"The code_tutor suite still reports exactly 217 passing tests -- the count established once the cli lint fix added its covering test -- proving no test was dropped or silently skipped.",
"The package keeps 100% branch coverage of the code_tutor source modules.",
"No patch string changes: this is a pure partition of test sections, and the source splits in the preceding commits already put every patched name in the module each file names.",
"The shared _make_verifier builder moves to tests/conftest.py rather than being duplicated, and the local copy of _make_live_mock that shadowed conftest's is deleted.",
"No assertion is changed and no test is deleted, skipped or weakened.",
"No suppression is added and no coverage exclusion is introduced."
],
"out_of_scope": [
"Changing what any test asserts.",
"Touching the code_tutor source, which is already under the cap.",
"Adding a fourth partN-style file name: each new module is named after the concern it covers."
],
"verifier": "timeout 120 env PYTHONPATH=. python3 -m pytest python_pkg/code_tutor/tests -q (expects exactly 217 passed), plus bash ~/utils/scripts/check_file_length.sh --all showing no code_tutor entry, plus pre-commit run --files on the touched files."
}

45
docs/superpowers/evidence/code-tutor-test-split-2026-08-16.json Normal file
View File

@ -0,0 +1,45 @@
{
"intent": "Clear the seven over-cap code_tutor test modules by partitioning them at their existing section banners, taking the package from thirteen violations to zero.",
"scope": [
"python_pkg/code_tutor/tests/test_cli.py (500 -> 208) + test_cli_commands.py, test_cli_status_drill.py, test_cli_discovery.py",
"python_pkg/code_tutor/tests/test_analyzer.py (454 -> 225) + test_deps.py",
"python_pkg/code_tutor/tests/test_challenge_part3.py (378 -> 123) + test_challenge_flows.py, test_run_coding_challenge.py",
"python_pkg/code_tutor/tests/test_verifier.py (361 -> 231) + test_verifier_lesson.py",
"python_pkg/code_tutor/tests/test_challenge.py (343 -> 192) + test_pytest_runner.py",
"python_pkg/code_tutor/tests/test_llm.py (324 -> 79) + test_claude_backend.py, test_llm_fallback.py",
"python_pkg/code_tutor/tests/test_challenge_part2.py (307 -> 98) + test_signature_block.py",
"python_pkg/code_tutor/tests/conftest.py (gained _make_verifier)",
"Non-goal: no assertion changed, no test deleted, skipped or weakened"
],
"changes": [
"Wrote a small splitter that cuts a module at a named section banner and copies the ENTIRE original header into the new file, then let ruff's F401 pass prune what each half does not use. That ordering is deliberate: hand-guessing the import block caused an intermediate failure in every earlier test split in this session, and here it pruned 59 imports correctly with none missed.",
"Ten new modules, each named after the concern it covers rather than a partN suffix.",
"_make_verifier was defined between two banners in test_verifier.py, so the cut left it behind while its callers moved. Rather than duplicate it, it moved to tests/conftest.py where _make_live_mock already lives - and test_verifier.py's local _make_live_mock, which had been shadowing conftest's identical copy, was deleted with it.",
"No patch string changed anywhere in this commit. The source splits in the preceding commits had already put every patched name in the module each test file names, so this is a pure partition."
],
"verification": [
{
"command": "timeout 120 env PYTHONPATH=. python3 -m pytest python_pkg/code_tutor/tests -q",
"result": "pass",
"evidence": "217 passed - identical to the count before the split. The single intermediate failure (4 failed, 213 passed) still summed to 217, which confirmed at a glance that the split had misplaced a helper rather than lost a test."
},
{
"command": "bash ~/utils/scripts/check_file_length.sh --all | grep code_tutor",
"result": "pass",
"evidence": "No output - python_pkg/code_tutor is entirely under the 250-line cap, down from thirteen violations at the start of the package."
},
{
"command": "pre-commit run --files <the touched test files and artifacts>",
"result": "pass",
"evidence": "ruff pruned 59 then 15 unused imports across the new modules; clean on re-run. name-tests-test, mypy, pylint and pytest-with-coverage all pass at 100%."
}
],
"risks": [
"Copying the whole header and letting ruff prune is safe for imports but not for module-level helpers: ruff has no way to know a function defined mid-file is still needed elsewhere. _make_verifier is the case that proves it, and any future banner split should check for defs sitting between sections before running the suite.",
"conftest.py now imports Verifier, so a future move of that class must update the fixture builder as well as the test modules."
],
"rollback": [
"git revert the commit; the seven originals return at their previous sizes and the ten new modules disappear.",
"After rollback, re-run the package suite and confirm it still reports 217 passed."
]
}

8
python_pkg/code_tutor/tests/conftest.py
View File

@ -11,6 +11,7 @@ from __future__ import annotations
from unittest.mock import MagicMock
from python_pkg.code_tutor._analyzer import CodeItem
from python_pkg.code_tutor._verifier import Verifier
def _item(
@ -54,3 +55,10 @@ def _make_live_mock() -> MagicMock:
live.__enter__ = MagicMock(return_value=live)
live.__exit__ = MagicMock(return_value=False)
return live
def _make_verifier() -> tuple[Verifier, MagicMock, MagicMock]:
mock_backend = MagicMock()
mock_console = MagicMock()
verifier = Verifier(mock_backend, mock_console)
return verifier, mock_backend, mock_console

242
python_pkg/code_tutor/tests/test_analyzer.py
View File

@ -3,11 +3,10 @@
from __future__ import annotations
import ast
from pathlib import Path
from typing import TYPE_CHECKING
from unittest.mock import patch
from python_pkg.code_tutor._analyzer import (
CodeItem,
_extract_other,
_extract_python,
_FunctionVisitor,
@ -16,13 +15,9 @@ from python_pkg.code_tutor._analyzer import (
_should_skip,
extract_items,
)
from python_pkg.code_tutor._deps import (
_build_dotted_map,
_match_deps,
codebase_fingerprint,
get_file_imports,
get_python_files,
)
if TYPE_CHECKING:
from pathlib import Path
# ---------------------------------------------------------------------------
# _is_binary
@ -223,232 +218,3 @@ def test_extract_items_is_binary_oserror(tmp_path: Path) -> None:
):
items = extract_items(tmp_path)
assert items == []
# ---------------------------------------------------------------------------
# get_python_files
# ---------------------------------------------------------------------------
def test_get_python_files(tmp_path: Path) -> None:
(tmp_path / "a.py").write_text("x = 1", encoding="utf-8")
(tmp_path / "b.py").write_text("y = 2", encoding="utf-8")
skip_dir = tmp_path / ".venv"
skip_dir.mkdir()
(skip_dir / "c.py").write_text("z = 3", encoding="utf-8")
files = get_python_files(tmp_path)
names = {f.name for f in files}
assert "a.py" in names
assert "b.py" in names
assert "c.py" not in names
# ---------------------------------------------------------------------------
# _build_dotted_map
# ---------------------------------------------------------------------------
def test_build_dotted_map_normal(tmp_path: Path) -> None:
f = tmp_path / "pkg" / "mod.py"
f.parent.mkdir()
f.write_text("x = 1", encoding="utf-8")
result = _build_dotted_map(tmp_path, [f])
assert result[f] == "pkg.mod"
def test_build_dotted_map_value_error(tmp_path: Path) -> None:
other = tmp_path.parent / "other.py"
result = _build_dotted_map(tmp_path, [other])
assert other not in result
# ---------------------------------------------------------------------------
# _match_deps
# ---------------------------------------------------------------------------
def test_match_deps_exact(tmp_path: Path) -> None:
f = tmp_path / "mod.py"
dotted_map = {f: "mod"}
path = tmp_path / "main.py"
result = _match_deps({"mod"}, dotted_map, path)
assert f in result
def test_match_deps_prefix(tmp_path: Path) -> None:
f = tmp_path / "mod.py"
dotted_map = {f: "mod"}
path = tmp_path / "main.py"
result = _match_deps({"pkg.mod"}, dotted_map, path)
assert f in result
def test_match_deps_suffix(tmp_path: Path) -> None:
f = tmp_path / "mod.py"
dotted_map = {f: "pkg.mod"}
path = tmp_path / "main.py"
result = _match_deps({"mod"}, dotted_map, path)
assert f in result
def test_match_deps_excludes_self(tmp_path: Path) -> None:
path = tmp_path / "mod.py"
dotted_map = {path: "mod"}
result = _match_deps({"mod"}, dotted_map, path)
assert path not in result
def test_match_deps_no_match(tmp_path: Path) -> None:
f = tmp_path / "other.py"
dotted_map = {f: "other"}
path = tmp_path / "main.py"
result = _match_deps({"unrelated"}, dotted_map, path)
assert f not in result
# ---------------------------------------------------------------------------
# Tests for get_file_imports
# ---------------------------------------------------------------------------
def test_get_file_imports_import(tmp_path: Path) -> None:
f = tmp_path / "main.py"
dep = tmp_path / "utils.py"
f.write_text("import utils\n", encoding="utf-8")
dep.write_text("x = 1", encoding="utf-8")
result = get_file_imports(f, tmp_path, [f, dep])
assert dep in result
def test_get_file_imports_from_import(tmp_path: Path) -> None:
f = tmp_path / "main.py"
dep = tmp_path / "utils.py"
f.write_text("from utils import helper\n", encoding="utf-8")
dep.write_text("def helper(): pass\n", encoding="utf-8")
result = get_file_imports(f, tmp_path, [f, dep])
assert dep in result
def test_get_file_imports_from_import_none_module(tmp_path: Path) -> None:
# Relative import: node.module is None
f = tmp_path / "main.py"
f.write_text("from . import helper\n", encoding="utf-8")
result = get_file_imports(f, tmp_path, [f])
assert result == set()
def test_get_file_imports_syntax_error(tmp_path: Path) -> None:
f = tmp_path / "bad.py"
f.write_text("def (broken):\n", encoding="utf-8")
result = get_file_imports(f, tmp_path, [f])
assert result == set()
# ---------------------------------------------------------------------------
# codebase_fingerprint
# ---------------------------------------------------------------------------
def test_codebase_fingerprint_basic(tmp_path: Path) -> None:
(tmp_path / "a.py").write_text("x = 1", encoding="utf-8")
result = codebase_fingerprint(tmp_path)
assert len(result) == 16
assert result.isalnum()
def test_codebase_fingerprint_skips_dir(tmp_path: Path) -> None:
sub = tmp_path / "sub"
sub.mkdir()
result = codebase_fingerprint(tmp_path)
assert len(result) == 16
def test_codebase_fingerprint_skips_should_skip(tmp_path: Path) -> None:
venv = tmp_path / ".venv"
venv.mkdir()
(venv / "lib.py").write_text("x = 1", encoding="utf-8")
result_without = codebase_fingerprint(tmp_path)
(tmp_path / "real.py").write_text("y = 2", encoding="utf-8")
result_with = codebase_fingerprint(tmp_path)
assert result_without != result_with
def test_codebase_fingerprint_stat_oserror(tmp_path: Path) -> None:
(tmp_path / "source.py").write_text("x = 1", encoding="utf-8")
stat_calls: dict[str, int] = {}
real_stat = Path.stat
def patched_stat(self: Path, **kwargs: object) -> object:
key = str(self)
stat_calls[key] = stat_calls.get(key, 0) + 1
if stat_calls[key] >= 2 and key.endswith(".py"):
msg = "simulated stat failure"
raise OSError(msg)
return real_stat(self, **kwargs)
with patch.object(Path, "stat", patched_stat):
result = codebase_fingerprint(tmp_path)
assert len(result) == 16
def test_codebase_fingerprint_changes_on_modification(tmp_path: Path) -> None:
f = tmp_path / "mod.py"
f.write_text("x = 1", encoding="utf-8")
fp1 = codebase_fingerprint(tmp_path)
f.write_text("x = 2", encoding="utf-8")
fp2 = codebase_fingerprint(tmp_path)
assert fp1 != fp2
def test_extract_items_image_suffix(tmp_path: Path) -> None:
(tmp_path / "logo.png").write_bytes(b"\x89PNG\r\n")
items = extract_items(tmp_path)
assert items == []
def test_code_item_defaults() -> None:
item = CodeItem(
id="mod.fn",
file="mod.py",
type="function",
name="fn",
start_line=1,
end_line=5,
)
assert item.class_name == ""
assert item.depends_on == []
def test_codebase_fingerprint_stat_oserror_raises(tmp_path: Path) -> None:
"""Skip a file whose ``stat`` raises ``OSError`` (except branch).
``is_file()`` is forced True for the target so the loop reaches the
explicit ``path.stat()`` at fingerprint time; that stat then raises and
the ``except OSError`` handler drops the unreadable entry. Patching both
keeps this deterministic across Python versions (on some, ``is_file``
itself calls ``stat`` and would otherwise re-raise before line reached).
"""
(tmp_path / "source.py").write_text("x = 1", encoding="utf-8")
real_stat = Path.stat
real_is_file = Path.is_file
def failing_stat(self: Path, **kwargs: object) -> object:
"""Raise for the target file, delegate to the real stat otherwise."""
if str(self).endswith("source.py"):
msg = "simulated stat failure"
raise OSError(msg)
return real_stat(self, **kwargs)
def always_file(self: Path) -> bool:
"""Report the target as a regular file without calling stat."""
return str(self).endswith("source.py") or real_is_file(self)
with (
patch.object(Path, "stat", failing_stat),
patch.object(Path, "is_file", always_file),
):
result = codebase_fingerprint(tmp_path)
assert len(result) == 16

152
python_pkg/code_tutor/tests/test_challenge.py
View File

@ -18,7 +18,6 @@ from python_pkg.code_tutor._challenge_support import (
_scan_test_file,
_show_test_panels,
)
from python_pkg.code_tutor._pytest_runner import _patch_and_test, _pytest_clean
from python_pkg.code_tutor.tests.conftest import _item
# ---------------------------------------------------------------------------
@ -190,154 +189,3 @@ def test_project_root_not_found(tmp_path: Path) -> None:
result = _project_root(isolated)
# When not found, returns start.resolve()
assert result == isolated.resolve()
# ---------------------------------------------------------------------------
# _pytest_clean
# ---------------------------------------------------------------------------
def test_pytest_clean_pass() -> None:
mock_result = MagicMock()
mock_result.returncode = 0
mock_result.stdout = "1 passed"
mock_result.stderr = ""
mock_console = MagicMock()
with patch(
"python_pkg.code_tutor._pytest_runner.subprocess.run",
return_value=mock_result,
):
assert (
_pytest_clean(["test_mod.py::test_fn"], Path("/proj"), mock_console) is True
)
def test_pytest_clean_fail() -> None:
mock_result = MagicMock()
mock_result.returncode = 1
mock_result.stdout = ""
mock_result.stderr = "FAILED"
mock_console = MagicMock()
with patch(
"python_pkg.code_tutor._pytest_runner.subprocess.run",
return_value=mock_result,
):
assert (
_pytest_clean(["test_mod.py::test_fn"], Path("/proj"), mock_console)
is False
)
def test_pytest_clean_no_output() -> None:
mock_result = MagicMock()
mock_result.returncode = 0
mock_result.stdout = ""
mock_result.stderr = ""
mock_console = MagicMock()
with patch(
"python_pkg.code_tutor._pytest_runner.subprocess.run",
return_value=mock_result,
):
_pytest_clean([], Path("/proj"), mock_console)
mock_console.print.assert_not_called()
# ---------------------------------------------------------------------------
# _patch_and_test
# ---------------------------------------------------------------------------
def test_patch_and_test_syntax_error(tmp_path: Path) -> None:
source = "def fn():\n return 1\n"
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
mock_console = MagicMock()
result = _patch_and_test(
_item(file="mod.py", start=1, end=2),
str(tmp_path),
"def fn(\n broken syntax",
[],
mock_console,
)
assert result is False
# Original file should be restored
assert (tmp_path / "mod.py").read_text(encoding="utf-8") == source
def test_patch_and_test_passes(tmp_path: Path) -> None:
source = "def fn():\n return 1\n"
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
test_file = tmp_path / "test_mod.py"
test_file.write_text("def test_fn(): pass\n", encoding="utf-8")
mock_console = MagicMock()
with patch("python_pkg.code_tutor._pytest_runner._pytest_clean", return_value=True):
result = _patch_and_test(
_item(file="mod.py", start=1, end=2),
str(tmp_path),
"def fn():\n return 99",
[(test_file, ["test_fn"])],
mock_console,
)
assert result is True
# File restored
assert (tmp_path / "mod.py").read_text(encoding="utf-8") == source
def test_patch_and_test_fails(tmp_path: Path) -> None:
source = "def fn():\n return 1\n"
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
test_file = tmp_path / "test_mod.py"
test_file.write_text("def test_fn(): pass\n", encoding="utf-8")
mock_console = MagicMock()
with patch(
"python_pkg.code_tutor._pytest_runner._pytest_clean", return_value=False
):
result = _patch_and_test(
_item(file="mod.py", start=1, end=2),
str(tmp_path),
"def fn():\n return 0",
[(test_file, ["test_fn"])],
mock_console,
)
assert result is False
assert (tmp_path / "mod.py").read_text(encoding="utf-8") == source
def test_scan_test_file_non_func_non_class_toplevel() -> None:
"""Skip a top-level node that is neither a function nor a class def.
A module-level ``import`` fails both ``isinstance`` checks, exercising
the branch that falls through the ``elif ClassDef`` back to the loop.
"""
source = "import os\ndef test_fn_uses_fn():\n fn()\n"
result = _scan_test_file(source, "fn")
assert result == ["test_fn_uses_fn"]
def test_show_test_panels_non_func_non_class_toplevel(tmp_path: Path) -> None:
"""Skip a top-level node that is neither a function nor a class def.
A module-level ``import`` yields an empty ``nodes`` list, so the
``elif ClassDef`` branch is not taken before the inner render loop.
"""
source = "import os\ndef test_fn():\n fn(1)\n"
test_file = tmp_path / "test_mod.py"
test_file.write_text(source, encoding="utf-8")
mock_console = MagicMock()
with patch("python_pkg.code_tutor._challenge_support.Syntax"):
_show_test_panels([(test_file, ["test_fn"])], mock_console)
assert mock_console.print.called
def test_show_test_panels_class_non_func_member(tmp_path: Path) -> None:
"""Skip a class-body member that is not a function definition.
The class-level ``attr = 1`` assignment fails the member ``isinstance``
check, exercising the branch that loops back to the next body member.
"""
source = "class TestFoo:\n attr = 1\n def test_fn(self):\n fn()\n"
test_file = tmp_path / "test_mod.py"
test_file.write_text(source, encoding="utf-8")
mock_console = MagicMock()
with patch("python_pkg.code_tutor._challenge_support.Syntax"):
_show_test_panels([(test_file, ["TestFoo::test_fn"])], mock_console)
assert mock_console.print.called

211
python_pkg/code_tutor/tests/test_challenge_flows.py Normal file
View File

@ -0,0 +1,211 @@
"""Tests for the two code_tutor challenge flows."""
from __future__ import annotations
from typing import TYPE_CHECKING
from unittest.mock import MagicMock, patch
from python_pkg.code_tutor._challenge_flows import (
_existing_tests_flow,
_write_tests_first_flow,
)
from python_pkg.code_tutor.tests.conftest import _item
if TYPE_CHECKING:
from pathlib import Path
# ---------------------------------------------------------------------------
# _write_tests_first_flow
# ---------------------------------------------------------------------------
def test_write_tests_first_flow_user_declines(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn(): pass\n", encoding="utf-8")
mock_backend = MagicMock()
mock_console = MagicMock()
result = _write_tests_first_flow(
_item(file="mod.py"),
str(tmp_path),
"explanation",
mock_backend,
mock_console,
lambda _: "n",
)
assert result == "skipped"
def test_write_tests_first_flow_tests_none(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn(): pass\n", encoding="utf-8")
mock_backend = MagicMock()
mock_console = MagicMock()
with (
patch(
"python_pkg.code_tutor._challenge_flows._collect_and_rate_tests",
return_value=None,
),
patch("python_pkg.code_tutor._challenge_flows.Syntax"),
):
result = _write_tests_first_flow(
_item(file="mod.py"),
str(tmp_path),
"explanation",
mock_backend,
mock_console,
lambda _: "y",
)
assert result == "skipped"
def test_write_tests_first_flow_tests_fail_real(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn(): pass\n", encoding="utf-8")
mock_backend = MagicMock()
mock_console = MagicMock()
with (
patch(
"python_pkg.code_tutor._challenge_flows._collect_and_rate_tests",
return_value="test code",
),
patch(
"python_pkg.code_tutor._challenge_flows._validate_tests_against_real",
return_value=False,
),
patch("python_pkg.code_tutor._challenge_flows.Syntax"),
):
result = _write_tests_first_flow(
_item(file="mod.py"),
str(tmp_path),
"explanation",
mock_backend,
mock_console,
lambda _: "y",
)
assert result == "skipped"
def test_write_tests_first_flow_success(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn(): pass\n", encoding="utf-8")
mock_backend = MagicMock()
mock_console = MagicMock()
with (
patch(
"python_pkg.code_tutor._challenge_flows._collect_and_rate_tests",
return_value="test code",
),
patch(
"python_pkg.code_tutor._challenge_flows._validate_tests_against_real",
return_value=True,
),
patch(
"python_pkg.code_tutor._challenge_flows._run_user_impl",
return_value="passed",
),
patch("python_pkg.code_tutor._challenge_flows.Syntax"),
):
result = _write_tests_first_flow(
_item(file="mod.py"),
str(tmp_path),
"explanation",
mock_backend,
mock_console,
lambda _: "y",
)
assert result == "passed"
# ---------------------------------------------------------------------------
# _existing_tests_flow
# ---------------------------------------------------------------------------
def test_existing_tests_flow_user_declines(tmp_path: Path) -> None:
test_file = tmp_path / "test_mod.py"
test_file.write_text("def test_fn(): fn()\n", encoding="utf-8")
mock_console = MagicMock()
result = _existing_tests_flow(
_item(),
str(tmp_path),
"explanation",
[(test_file, ["test_fn"])],
mock_console,
lambda _: "n",
)
assert result == "skipped"
def test_existing_tests_flow_user_skips_impl(tmp_path: Path) -> None:
source = "def fn():\n return 1\n"
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
test_file = tmp_path / "test_mod.py"
test_file.write_text("def test_fn(): fn()\n", encoding="utf-8")
mock_console = MagicMock()
with patch("python_pkg.code_tutor._challenge_support.Syntax"):
result = _existing_tests_flow(
_item(file="mod.py"),
str(tmp_path),
"explanation",
[(test_file, ["test_fn"])],
mock_console,
lambda _: (
"y"
if "challenge" in str(_) or _ == "Take the challenge? [y/N] "
else "skip"
),
)
assert result == "skipped"
def test_existing_tests_flow_passes(tmp_path: Path) -> None:
source = "def fn():\n return 1\n"
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
test_file = tmp_path / "test_mod.py"
test_file.write_text("def test_fn(): fn()\n", encoding="utf-8")
mock_console = MagicMock()
inputs = iter(["y", "def fn():", " return 1", "END"])
with (
patch("python_pkg.code_tutor._challenge_support.Syntax"),
patch(
"python_pkg.code_tutor._challenge_flows._patch_and_test", return_value=True
),
):
result = _existing_tests_flow(
_item(file="mod.py"),
str(tmp_path),
"explanation",
[(test_file, ["test_fn"])],
mock_console,
lambda _: next(inputs),
)
assert result == "passed"
def test_existing_tests_flow_fails(tmp_path: Path) -> None:
source = "def fn():\n return 1\n"
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
test_file = tmp_path / "test_mod.py"
test_file.write_text("def test_fn(): fn()\n", encoding="utf-8")
mock_console = MagicMock()
inputs = iter(["y", "def fn():", " return 0", "END"])
with (
patch("python_pkg.code_tutor._challenge_support.Syntax"),
patch(
"python_pkg.code_tutor._challenge_flows._patch_and_test", return_value=False
),
):
result = _existing_tests_flow(
_item(file="mod.py"),
str(tmp_path),
"explanation",
[(test_file, ["test_fn"])],
mock_console,
lambda _: next(inputs),
)
assert result == "failed"

220
python_pkg/code_tutor/tests/test_challenge_part2.py
View File

@ -8,19 +8,10 @@ Covers the LLM verdict/signature helpers in ``_challenge_support``
from __future__ import annotations
from pathlib import Path
from unittest.mock import MagicMock, patch
from python_pkg.code_tutor._challenge import (
_collect_and_rate_tests,
_validate_tests_against_real,
)
from python_pkg.code_tutor._challenge_support import (
_extract_signature_block,
_import_hint,
)
from python_pkg.code_tutor._verdict import _parse_verdict, _stream_verdict
from python_pkg.code_tutor.tests.conftest import _item, _make_live_mock
from python_pkg.code_tutor.tests.conftest import _make_live_mock
# ---------------------------------------------------------------------------
# _stream_verdict
@ -96,212 +87,3 @@ def test_challenge_parse_verdict_json_error() -> None:
def test_challenge_parse_verdict_invalid() -> None:
verdict, _gap = _parse_verdict('{"verdict": "UNKNOWN", "gap": "x"}')
assert verdict == "FAIL"
# ---------------------------------------------------------------------------
# _extract_signature_block
# ---------------------------------------------------------------------------
def test_extract_signature_block_with_docstring(tmp_path: Path) -> None:
source = 'def fn(x):\n """Do thing."""\n return x\n'
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
result = _extract_signature_block(_item(file="mod.py"), str(tmp_path))
assert "def fn" in result
assert '"""Do thing."""' in result
def test_extract_signature_block_without_docstring(tmp_path: Path) -> None:
source = "def fn(x):\n return x\n"
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
result = _extract_signature_block(_item(file="mod.py"), str(tmp_path))
assert "def fn" in result
assert "return" not in result
def test_extract_signature_block_oserror(tmp_path: Path) -> None:
result = _extract_signature_block(_item(file="missing.py"), str(tmp_path))
assert "def fn(...):" in result
def test_extract_signature_block_no_match(tmp_path: Path) -> None:
source = "def other(): pass\n"
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
result = _extract_signature_block(_item(file="mod.py", name="fn"), str(tmp_path))
assert "def fn(...):" in result
def test_extract_signature_block_syntax_error(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def (broken):\n", encoding="utf-8")
result = _extract_signature_block(_item(file="mod.py"), str(tmp_path))
assert "def fn(...):" in result
# ---------------------------------------------------------------------------
# Tests for _import_hint
# ---------------------------------------------------------------------------
def test_import_hint_relative_to_project(tmp_path: Path) -> None:
(tmp_path / "src").mkdir()
result = _import_hint(
_item(file="src/mod.py", name="fn"),
str(tmp_path),
tmp_path,
)
assert result == "from src.mod import fn"
def test_import_hint_value_error(tmp_path: Path) -> None:
# project_root is different from codebase_path
other_root = tmp_path / "other"
other_root.mkdir()
result = _import_hint(
_item(file="src/mod.py", name="fn"),
str(tmp_path),
other_root,
)
assert result == "from mod import fn"
# ---------------------------------------------------------------------------
# _validate_tests_against_real
# ---------------------------------------------------------------------------
def test_validate_tests_syntax_error() -> None:
mock_console = MagicMock()
result = _validate_tests_against_real(
"def (broken):", "import fn", Path("/proj"), mock_console
)
assert result is False
def test_validate_tests_passes() -> None:
mock_console = MagicMock()
with patch("python_pkg.code_tutor._challenge._pytest_clean", return_value=True):
result = _validate_tests_against_real(
"def test_fn():\n assert True\n",
"# import",
Path("/proj"),
mock_console,
)
assert result is True
def test_validate_tests_fails() -> None:
mock_console = MagicMock()
with patch("python_pkg.code_tutor._challenge._pytest_clean", return_value=False):
result = _validate_tests_against_real(
"def test_fn():\n assert False\n",
"# import",
Path("/proj"),
mock_console,
)
assert result is False
# ---------------------------------------------------------------------------
# _collect_and_rate_tests
# ---------------------------------------------------------------------------
def test_collect_and_rate_tests_skip() -> None:
mock_backend = MagicMock()
mock_console = MagicMock()
live_mock = _make_live_mock()
with patch("python_pkg.code_tutor._verdict.Live", return_value=live_mock):
result = _collect_and_rate_tests(
"def fn():",
"explanation",
mock_backend,
mock_console,
lambda _: "skip",
)
assert result is None
def test_collect_and_rate_tests_pass_first() -> None:
mock_backend = MagicMock()
mock_console = MagicMock()
live_mock = _make_live_mock()
def fake_stream(system: str, user: str, on_token: object) -> str:
assert callable(on_token)
on_token('{"verdict": "PASS", "gap": ""}')
return ""
mock_backend.stream.side_effect = fake_stream
lines_iter = iter(["test code line", "END"])
with patch("python_pkg.code_tutor._verdict.Live", return_value=live_mock):
result = _collect_and_rate_tests(
"def fn():",
"explanation",
mock_backend,
mock_console,
lambda _: next(lines_iter),
)
assert result == "test code line"
def test_collect_and_rate_tests_fail_twice() -> None:
mock_backend = MagicMock()
mock_console = MagicMock()
live_mock = _make_live_mock()
mock_backend.stream.return_value = '{"verdict": "FAIL", "gap": "bad"}'
attempt_inputs = [
# attempt 1: provide code, then END
"test code",
"END",
# attempt 2: provide code, then END
"test code again",
"END",
]
inputs_iter = iter(attempt_inputs)
with patch("python_pkg.code_tutor._verdict.Live", return_value=live_mock):
result = _collect_and_rate_tests(
"def fn():",
"explanation",
mock_backend,
mock_console,
lambda _: next(inputs_iter),
)
assert result is None
def test_collect_and_rate_tests_fail_then_pass() -> None:
mock_backend = MagicMock()
mock_console = MagicMock()
live_mock = _make_live_mock()
call_count = [0]
def fake_stream(system: str, user: str, on_token: object) -> str:
assert callable(on_token)
call_count[0] += 1
verdict = (
'{"verdict": "FAIL", "gap": "not enough"}'
if call_count[0] == 1
else '{"verdict": "PASS", "gap": ""}'
)
on_token(verdict)
return ""
mock_backend.stream.side_effect = fake_stream
inputs = iter(["bad test", "END", "good test", "END"])
with patch("python_pkg.code_tutor._verdict.Live", return_value=live_mock):
result = _collect_and_rate_tests(
"def fn():",
"explanation",
mock_backend,
mock_console,
lambda _: next(inputs),
)
assert result == "good test"

260
python_pkg/code_tutor/tests/test_challenge_part3.py
View File

@ -11,11 +11,6 @@ from typing import TYPE_CHECKING
from unittest.mock import MagicMock, patch
from python_pkg.code_tutor._challenge import _run_user_impl
from python_pkg.code_tutor._challenge_flows import (
_existing_tests_flow,
_write_tests_first_flow,
run_coding_challenge,
)
from python_pkg.code_tutor.tests.conftest import _item, _make_live_mock
if TYPE_CHECKING:
@ -121,258 +116,3 @@ def test_run_user_impl_failed(tmp_path: Path) -> None:
)
assert result == "failed"
assert (tmp_path / "mod.py").read_text(encoding="utf-8") == source
# ---------------------------------------------------------------------------
# _write_tests_first_flow
# ---------------------------------------------------------------------------
def test_write_tests_first_flow_user_declines(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn(): pass\n", encoding="utf-8")
mock_backend = MagicMock()
mock_console = MagicMock()
result = _write_tests_first_flow(
_item(file="mod.py"),
str(tmp_path),
"explanation",
mock_backend,
mock_console,
lambda _: "n",
)
assert result == "skipped"
def test_write_tests_first_flow_tests_none(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn(): pass\n", encoding="utf-8")
mock_backend = MagicMock()
mock_console = MagicMock()
with (
patch(
"python_pkg.code_tutor._challenge_flows._collect_and_rate_tests",
return_value=None,
),
patch("python_pkg.code_tutor._challenge_flows.Syntax"),
):
result = _write_tests_first_flow(
_item(file="mod.py"),
str(tmp_path),
"explanation",
mock_backend,
mock_console,
lambda _: "y",
)
assert result == "skipped"
def test_write_tests_first_flow_tests_fail_real(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn(): pass\n", encoding="utf-8")
mock_backend = MagicMock()
mock_console = MagicMock()
with (
patch(
"python_pkg.code_tutor._challenge_flows._collect_and_rate_tests",
return_value="test code",
),
patch(
"python_pkg.code_tutor._challenge_flows._validate_tests_against_real",
return_value=False,
),
patch("python_pkg.code_tutor._challenge_flows.Syntax"),
):
result = _write_tests_first_flow(
_item(file="mod.py"),
str(tmp_path),
"explanation",
mock_backend,
mock_console,
lambda _: "y",
)
assert result == "skipped"
def test_write_tests_first_flow_success(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn(): pass\n", encoding="utf-8")
mock_backend = MagicMock()
mock_console = MagicMock()
with (
patch(
"python_pkg.code_tutor._challenge_flows._collect_and_rate_tests",
return_value="test code",
),
patch(
"python_pkg.code_tutor._challenge_flows._validate_tests_against_real",
return_value=True,
),
patch(
"python_pkg.code_tutor._challenge_flows._run_user_impl",
return_value="passed",
),
patch("python_pkg.code_tutor._challenge_flows.Syntax"),
):
result = _write_tests_first_flow(
_item(file="mod.py"),
str(tmp_path),
"explanation",
mock_backend,
mock_console,
lambda _: "y",
)
assert result == "passed"
# ---------------------------------------------------------------------------
# _existing_tests_flow
# ---------------------------------------------------------------------------
def test_existing_tests_flow_user_declines(tmp_path: Path) -> None:
test_file = tmp_path / "test_mod.py"
test_file.write_text("def test_fn(): fn()\n", encoding="utf-8")
mock_console = MagicMock()
result = _existing_tests_flow(
_item(),
str(tmp_path),
"explanation",
[(test_file, ["test_fn"])],
mock_console,
lambda _: "n",
)
assert result == "skipped"
def test_existing_tests_flow_user_skips_impl(tmp_path: Path) -> None:
source = "def fn():\n return 1\n"
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
test_file = tmp_path / "test_mod.py"
test_file.write_text("def test_fn(): fn()\n", encoding="utf-8")
mock_console = MagicMock()
with patch("python_pkg.code_tutor._challenge_support.Syntax"):
result = _existing_tests_flow(
_item(file="mod.py"),
str(tmp_path),
"explanation",
[(test_file, ["test_fn"])],
mock_console,
lambda _: (
"y"
if "challenge" in str(_) or _ == "Take the challenge? [y/N] "
else "skip"
),
)
assert result == "skipped"
def test_existing_tests_flow_passes(tmp_path: Path) -> None:
source = "def fn():\n return 1\n"
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
test_file = tmp_path / "test_mod.py"
test_file.write_text("def test_fn(): fn()\n", encoding="utf-8")
mock_console = MagicMock()
inputs = iter(["y", "def fn():", " return 1", "END"])
with (
patch("python_pkg.code_tutor._challenge_support.Syntax"),
patch(
"python_pkg.code_tutor._challenge_flows._patch_and_test", return_value=True
),
):
result = _existing_tests_flow(
_item(file="mod.py"),
str(tmp_path),
"explanation",
[(test_file, ["test_fn"])],
mock_console,
lambda _: next(inputs),
)
assert result == "passed"
def test_existing_tests_flow_fails(tmp_path: Path) -> None:
source = "def fn():\n return 1\n"
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
test_file = tmp_path / "test_mod.py"
test_file.write_text("def test_fn(): fn()\n", encoding="utf-8")
mock_console = MagicMock()
inputs = iter(["y", "def fn():", " return 0", "END"])
with (
patch("python_pkg.code_tutor._challenge_support.Syntax"),
patch(
"python_pkg.code_tutor._challenge_flows._patch_and_test", return_value=False
),
):
result = _existing_tests_flow(
_item(file="mod.py"),
str(tmp_path),
"explanation",
[(test_file, ["test_fn"])],
mock_console,
lambda _: next(inputs),
)
assert result == "failed"
# ---------------------------------------------------------------------------
# run_coding_challenge
# ---------------------------------------------------------------------------
def test_run_coding_challenge_non_python() -> None:
mock_console = MagicMock()
mock_backend = MagicMock()
result = run_coding_challenge(
_item(file="main.go"),
"/codebase",
"explanation",
mock_backend,
mock_console,
)
assert result == "skipped"
def test_run_coding_challenge_with_existing_tests(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn(): pass\n", encoding="utf-8")
test_file = tmp_path / "test_mod.py"
test_file.write_text("def test_fn(): fn()\n", encoding="utf-8")
mock_console = MagicMock()
mock_backend = MagicMock()
with patch(
"python_pkg.code_tutor._challenge_flows._existing_tests_flow",
return_value="passed",
):
result = run_coding_challenge(
_item(file="mod.py"),
str(tmp_path),
"explanation",
mock_backend,
mock_console,
)
assert result == "passed"
def test_run_coding_challenge_no_tests(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn(): pass\n", encoding="utf-8")
mock_console = MagicMock()
mock_backend = MagicMock()
with patch(
"python_pkg.code_tutor._challenge_flows._write_tests_first_flow",
return_value="skipped",
):
result = run_coding_challenge(
_item(file="mod.py"),
str(tmp_path),
"explanation",
mock_backend,
mock_console,
)
assert result == "skipped"

100
python_pkg/code_tutor/tests/test_claude_backend.py Normal file
View File

@ -0,0 +1,100 @@
"""Tests for python_pkg.code_tutor._claude_backend."""
from __future__ import annotations
from unittest.mock import MagicMock, patch
from python_pkg.code_tutor._claude_backend import ClaudeBackend
# ---------------------------------------------------------------------------
# ClaudeBackend.complete
# ---------------------------------------------------------------------------
def test_claude_complete_with_text_block() -> None:
mock_block = MagicMock()
mock_block.text = "answer"
mock_message = MagicMock()
mock_message.content = [mock_block]
mock_client = MagicMock()
mock_client.messages.create.return_value = mock_message
with patch(
"python_pkg.code_tutor._claude_backend.anthropic.Anthropic",
return_value=mock_client,
):
backend = ClaudeBackend()
result = backend.complete("sys", "user")
assert result == "answer"
def test_claude_complete_no_text_attr() -> None:
mock_block = MagicMock(spec=[]) # no .text attribute
mock_message = MagicMock()
mock_message.content = [mock_block]
mock_client = MagicMock()
mock_client.messages.create.return_value = mock_message
with patch(
"python_pkg.code_tutor._claude_backend.anthropic.Anthropic",
return_value=mock_client,
):
backend = ClaudeBackend()
result = backend.complete("sys", "user")
assert result == ""
# ---------------------------------------------------------------------------
# ClaudeBackend.stream
# ---------------------------------------------------------------------------
def test_claude_stream_success() -> None:
mock_stream_ctx = MagicMock()
mock_stream_ctx.__enter__ = MagicMock(return_value=mock_stream_ctx)
mock_stream_ctx.__exit__ = MagicMock(return_value=False)
mock_stream_ctx.text_stream = iter(["hello", " world"])
mock_client = MagicMock()
mock_client.messages.stream.return_value = mock_stream_ctx
tokens: list[str] = []
with patch(
"python_pkg.code_tutor._claude_backend.anthropic.Anthropic",
return_value=mock_client,
):
backend = ClaudeBackend()
result = backend.stream("sys", "user", tokens.append)
assert result == "hello world"
assert tokens == ["hello", " world"]
def test_claude_stream_api_error_fallback() -> None:
import anthropic as anthropic_lib
mock_client = MagicMock()
mock_client.messages.stream.side_effect = anthropic_lib.APIError(
message="fail",
request=MagicMock(),
body=None,
)
mock_block = MagicMock()
mock_block.text = "fallback"
mock_message = MagicMock()
mock_message.content = [mock_block]
mock_client.messages.create.return_value = mock_message
tokens: list[str] = []
with patch(
"python_pkg.code_tutor._claude_backend.anthropic.Anthropic",
return_value=mock_client,
):
backend = ClaudeBackend()
result = backend.stream("sys", "user", tokens.append)
assert result == "fallback"

296
python_pkg/code_tutor/tests/test_cli.py
View File

@ -2,7 +2,6 @@
from __future__ import annotations
import json
from typing import TYPE_CHECKING
from unittest.mock import MagicMock, patch
@ -10,12 +9,9 @@ import requests
from typer.testing import CliRunner
from python_pkg.code_tutor._cli_checks import (
_check_plan_file,
_ensure_fresh_plan,
_ensure_ollama_running,
_find_codebase_for_file,
)
from python_pkg.code_tutor.cli import app
if TYPE_CHECKING:
from pathlib import Path
@ -206,295 +202,3 @@ def test_ensure_fresh_plan_stale_rebuilds(tmp_path: Path) -> None:
):
result = _ensure_fresh_plan(tmp_path, plan, mock_console)
assert result["source_fingerprint"] == "new"
# ---------------------------------------------------------------------------
# analyze command
# ---------------------------------------------------------------------------
def test_analyze_not_dir(tmp_path: Path) -> None:
result = runner.invoke(app, ["analyze", str(tmp_path / "no_such_dir")])
assert result.exit_code == 1
def test_analyze_no_items(tmp_path: Path) -> None:
with patch("python_pkg.code_tutor.cli.extract_items", return_value=[]):
result = runner.invoke(app, ["analyze", str(tmp_path)])
assert result.exit_code == 0
def test_analyze_with_items(tmp_path: Path) -> None:
mock_items = [MagicMock()]
mock_plan = _make_plan(tmp_path)
with (
patch("python_pkg.code_tutor.cli.extract_items", return_value=mock_items),
patch("python_pkg.code_tutor.cli.build_plan", return_value=mock_plan),
patch("python_pkg.code_tutor.cli.save_plan"),
patch("python_pkg.code_tutor.cli.config_dir", return_value=tmp_path),
):
result = runner.invoke(app, ["analyze", str(tmp_path)])
assert result.exit_code == 0
# ---------------------------------------------------------------------------
# study command
# ---------------------------------------------------------------------------
def test_study_not_dir(tmp_path: Path) -> None:
result = runner.invoke(app, ["study", str(tmp_path / "no_such_dir")])
assert result.exit_code == 1
def test_study_no_plan(tmp_path: Path) -> None:
with patch("python_pkg.code_tutor.cli.load_plan", return_value=None):
result = runner.invoke(app, ["study", str(tmp_path)])
assert result.exit_code == 1
def test_study_ollama_fails(tmp_path: Path) -> None:
plan = _make_plan(tmp_path)
with (
patch("python_pkg.code_tutor.cli.load_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_fresh_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_ollama_running", return_value=False),
):
result = runner.invoke(app, ["study", str(tmp_path)])
assert result.exit_code == 1
def test_study_connection_error(tmp_path: Path) -> None:
plan = _make_plan(tmp_path)
with (
patch("python_pkg.code_tutor.cli.load_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_fresh_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_ollama_running", return_value=True),
patch(
"python_pkg.code_tutor.cli.run_session",
side_effect=requests.exceptions.ConnectionError(),
),
):
result = runner.invoke(app, ["study", str(tmp_path)])
assert result.exit_code == 1
def test_study_success(tmp_path: Path) -> None:
plan = _make_plan(tmp_path)
with (
patch("python_pkg.code_tutor.cli.load_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_fresh_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_ollama_running", return_value=True),
patch("python_pkg.code_tutor.cli.run_session"),
):
result = runner.invoke(app, ["study", str(tmp_path)])
assert result.exit_code == 0
# ---------------------------------------------------------------------------
# status command
# ---------------------------------------------------------------------------
def test_status_no_plan(tmp_path: Path) -> None:
with patch("python_pkg.code_tutor.cli.load_plan", return_value=None):
result = runner.invoke(app, ["status", str(tmp_path)])
assert result.exit_code == 1
def test_status_with_plan(tmp_path: Path) -> None:
plan = _make_plan(tmp_path)
progress = {"learned": [], "struggled": [], "skipped": [], "last_session": ""}
with (
patch("python_pkg.code_tutor.cli.load_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_fresh_plan", return_value=plan),
patch("python_pkg.code_tutor.cli.load_progress", return_value=progress),
):
result = runner.invoke(app, ["status", str(tmp_path)])
assert result.exit_code == 0
# ---------------------------------------------------------------------------
# drill command
# ---------------------------------------------------------------------------
def test_drill_no_plan_for_file(tmp_path: Path) -> None:
f = tmp_path / "mod.py"
f.write_text("def fn(): pass\n", encoding="utf-8")
with patch("python_pkg.code_tutor.cli._find_codebase_for_file", return_value=None):
result = runner.invoke(app, ["drill", str(f)])
assert result.exit_code == 1
def test_drill_plan_disappears(tmp_path: Path) -> None:
f = tmp_path / "mod.py"
f.write_text("def fn(): pass\n", encoding="utf-8")
with (
patch(
"python_pkg.code_tutor.cli._find_codebase_for_file", return_value=tmp_path
),
patch("python_pkg.code_tutor.cli.load_plan", return_value=None),
):
result = runner.invoke(app, ["drill", str(f)])
assert result.exit_code == 1
def test_drill_no_items_in_plan(tmp_path: Path) -> None:
f = tmp_path / "other.py"
f.write_text("def fn(): pass\n", encoding="utf-8")
plan = _make_plan(tmp_path) # plan has mod.py items, not other.py
with (
patch(
"python_pkg.code_tutor.cli._find_codebase_for_file", return_value=tmp_path
),
patch("python_pkg.code_tutor.cli.load_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_fresh_plan", return_value=plan),
):
result = runner.invoke(app, ["drill", str(f)])
assert result.exit_code == 0
def test_drill_ollama_fails(tmp_path: Path) -> None:
f = tmp_path / "mod.py"
f.write_text("def fn(): pass\n", encoding="utf-8")
plan = _make_plan(tmp_path)
with (
patch(
"python_pkg.code_tutor.cli._find_codebase_for_file", return_value=tmp_path
),
patch("python_pkg.code_tutor.cli.load_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_fresh_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_ollama_running", return_value=False),
):
result = runner.invoke(app, ["drill", str(f)])
assert result.exit_code == 1
def test_drill_runs_lesson(tmp_path: Path) -> None:
from python_pkg.code_tutor._progress import LessonRecord
f = tmp_path / "mod.py"
f.write_text("def fn(): pass\n", encoding="utf-8")
plan = _make_plan(tmp_path)
record = LessonRecord(
timestamp="t",
item_id="mod.fn",
file="mod.py",
lines="1-3",
snippet="code",
outcome="learned",
answers={},
improvement="",
verdict="PASS",
attempt=1,
)
mock_verifier = MagicMock()
mock_verifier.run_lesson.return_value = record
with (
patch(
"python_pkg.code_tutor.cli._find_codebase_for_file", return_value=tmp_path
),
patch("python_pkg.code_tutor.cli.load_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_fresh_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_ollama_running", return_value=True),
patch("python_pkg.code_tutor.cli.Verifier", return_value=mock_verifier),
patch("python_pkg.code_tutor.cli.append_session_record"),
):
result = runner.invoke(app, ["drill", str(f)])
assert result.exit_code == 0
mock_verifier.run_lesson.assert_called_once()
# ---------------------------------------------------------------------------
# _find_codebase_for_file
# ---------------------------------------------------------------------------
def test_find_codebase_for_file_no_config_dir(tmp_path: Path) -> None:
with patch(
"python_pkg.code_tutor.cli.Path.home",
return_value=tmp_path,
):
result = _find_codebase_for_file(tmp_path / "mod.py")
assert result is None
def test_find_codebase_for_file_found(tmp_path: Path) -> None:
home = tmp_path / "home"
config_root = home / ".config" / "code_tutor" / "hash"
config_root.mkdir(parents=True)
codebase = tmp_path / "codebase"
codebase.mkdir()
f = codebase / "mod.py"
f.write_text("def fn(): pass\n", encoding="utf-8")
plan_data = {
"codebase_path": str(codebase),
"sessions": [],
}
(config_root / "plan.json").write_text(json.dumps(plan_data), encoding="utf-8")
with patch("python_pkg.code_tutor.cli.Path.home", return_value=home):
result = _find_codebase_for_file(f)
assert result == codebase
def test_find_codebase_for_file_not_found(tmp_path: Path) -> None:
home = tmp_path / "home"
config_root = home / ".config" / "code_tutor" / "hash"
config_root.mkdir(parents=True)
other_codebase = tmp_path / "other"
other_codebase.mkdir()
plan_data = {"codebase_path": str(other_codebase), "sessions": []}
(config_root / "plan.json").write_text(json.dumps(plan_data), encoding="utf-8")
target = tmp_path / "unrelated" / "mod.py"
with patch("python_pkg.code_tutor.cli.Path.home", return_value=home):
result = _find_codebase_for_file(target)
assert result is None
# ---------------------------------------------------------------------------
# _check_plan_file
# ---------------------------------------------------------------------------
def test_check_plan_file_found(tmp_path: Path) -> None:
codebase = tmp_path / "cb"
codebase.mkdir()
f = codebase / "mod.py"
plan_file = tmp_path / "plan.json"
plan_file.write_text(json.dumps({"codebase_path": str(codebase)}), encoding="utf-8")
result = _check_plan_file(plan_file, f)
assert result == codebase
def test_check_plan_file_value_error(tmp_path: Path) -> None:
codebase = tmp_path / "cb"
codebase.mkdir()
other_file = tmp_path / "other" / "mod.py"
plan_file = tmp_path / "plan.json"
plan_file.write_text(json.dumps({"codebase_path": str(codebase)}), encoding="utf-8")
# other_file is not relative to codebase → ValueError
result = _check_plan_file(plan_file, other_file)
assert result is None
def test_check_plan_file_oserror(tmp_path: Path) -> None:
plan_file = tmp_path / "missing_plan.json"
result = _check_plan_file(plan_file, tmp_path / "mod.py")
assert result is None
def test_check_plan_file_json_error(tmp_path: Path) -> None:
plan_file = tmp_path / "plan.json"
plan_file.write_text("not valid json {{{", encoding="utf-8")
result = _check_plan_file(plan_file, tmp_path / "mod.py")
assert result is None

126
python_pkg/code_tutor/tests/test_cli_commands.py Normal file
View File

@ -0,0 +1,126 @@
"""Tests for the code_tutor CLI commands themselves."""
from __future__ import annotations
from typing import TYPE_CHECKING
from unittest.mock import MagicMock, patch
import requests
from typer.testing import CliRunner
from python_pkg.code_tutor.cli import app
if TYPE_CHECKING:
from pathlib import Path
runner = CliRunner()
def _make_plan(codebase: Path, fingerprint: str = "abc") -> dict:
return {
"codebase_path": str(codebase),
"created_at": "2026-01-01T00:00:00+00:00",
"total_items": 2,
"source_fingerprint": fingerprint,
"sessions": [
{
"id": 1,
"title": "mod",
"items": [
{
"id": "mod.fn",
"file": "mod.py",
"type": "function",
"name": "fn",
"start_line": 1,
"end_line": 3,
"class_name": "",
"depends_on": [],
}
],
}
],
}
# ---------------------------------------------------------------------------
# analyze command
# ---------------------------------------------------------------------------
def test_analyze_not_dir(tmp_path: Path) -> None:
result = runner.invoke(app, ["analyze", str(tmp_path / "no_such_dir")])
assert result.exit_code == 1
def test_analyze_no_items(tmp_path: Path) -> None:
with patch("python_pkg.code_tutor.cli.extract_items", return_value=[]):
result = runner.invoke(app, ["analyze", str(tmp_path)])
assert result.exit_code == 0
def test_analyze_with_items(tmp_path: Path) -> None:
mock_items = [MagicMock()]
mock_plan = _make_plan(tmp_path)
with (
patch("python_pkg.code_tutor.cli.extract_items", return_value=mock_items),
patch("python_pkg.code_tutor.cli.build_plan", return_value=mock_plan),
patch("python_pkg.code_tutor.cli.save_plan"),
patch("python_pkg.code_tutor.cli.config_dir", return_value=tmp_path),
):
result = runner.invoke(app, ["analyze", str(tmp_path)])
assert result.exit_code == 0
# ---------------------------------------------------------------------------
# study command
# ---------------------------------------------------------------------------
def test_study_not_dir(tmp_path: Path) -> None:
result = runner.invoke(app, ["study", str(tmp_path / "no_such_dir")])
assert result.exit_code == 1
def test_study_no_plan(tmp_path: Path) -> None:
with patch("python_pkg.code_tutor.cli.load_plan", return_value=None):
result = runner.invoke(app, ["study", str(tmp_path)])
assert result.exit_code == 1
def test_study_ollama_fails(tmp_path: Path) -> None:
plan = _make_plan(tmp_path)
with (
patch("python_pkg.code_tutor.cli.load_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_fresh_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_ollama_running", return_value=False),
):
result = runner.invoke(app, ["study", str(tmp_path)])
assert result.exit_code == 1
def test_study_connection_error(tmp_path: Path) -> None:
plan = _make_plan(tmp_path)
with (
patch("python_pkg.code_tutor.cli.load_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_fresh_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_ollama_running", return_value=True),
patch(
"python_pkg.code_tutor.cli.run_session",
side_effect=requests.exceptions.ConnectionError(),
),
):
result = runner.invoke(app, ["study", str(tmp_path)])
assert result.exit_code == 1
def test_study_success(tmp_path: Path) -> None:
plan = _make_plan(tmp_path)
with (
patch("python_pkg.code_tutor.cli.load_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_fresh_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_ollama_running", return_value=True),
patch("python_pkg.code_tutor.cli.run_session"),
):
result = runner.invoke(app, ["study", str(tmp_path)])
assert result.exit_code == 0

138
python_pkg/code_tutor/tests/test_cli_discovery.py Normal file
View File

@ -0,0 +1,138 @@
"""Tests for locating a codebase from a single file."""
from __future__ import annotations
import json
from typing import TYPE_CHECKING
from unittest.mock import patch
from typer.testing import CliRunner
from python_pkg.code_tutor._cli_checks import (
_check_plan_file,
_find_codebase_for_file,
)
if TYPE_CHECKING:
from pathlib import Path
runner = CliRunner()
def _make_plan(codebase: Path, fingerprint: str = "abc") -> dict:
return {
"codebase_path": str(codebase),
"created_at": "2026-01-01T00:00:00+00:00",
"total_items": 2,
"source_fingerprint": fingerprint,
"sessions": [
{
"id": 1,
"title": "mod",
"items": [
{
"id": "mod.fn",
"file": "mod.py",
"type": "function",
"name": "fn",
"start_line": 1,
"end_line": 3,
"class_name": "",
"depends_on": [],
}
],
}
],
}
# ---------------------------------------------------------------------------
# _find_codebase_for_file
# ---------------------------------------------------------------------------
def test_find_codebase_for_file_no_config_dir(tmp_path: Path) -> None:
with patch(
"python_pkg.code_tutor.cli.Path.home",
return_value=tmp_path,
):
result = _find_codebase_for_file(tmp_path / "mod.py")
assert result is None
def test_find_codebase_for_file_found(tmp_path: Path) -> None:
home = tmp_path / "home"
config_root = home / ".config" / "code_tutor" / "hash"
config_root.mkdir(parents=True)
codebase = tmp_path / "codebase"
codebase.mkdir()
f = codebase / "mod.py"
f.write_text("def fn(): pass\n", encoding="utf-8")
plan_data = {
"codebase_path": str(codebase),
"sessions": [],
}
(config_root / "plan.json").write_text(json.dumps(plan_data), encoding="utf-8")
with patch("python_pkg.code_tutor.cli.Path.home", return_value=home):
result = _find_codebase_for_file(f)
assert result == codebase
def test_find_codebase_for_file_not_found(tmp_path: Path) -> None:
home = tmp_path / "home"
config_root = home / ".config" / "code_tutor" / "hash"
config_root.mkdir(parents=True)
other_codebase = tmp_path / "other"
other_codebase.mkdir()
plan_data = {"codebase_path": str(other_codebase), "sessions": []}
(config_root / "plan.json").write_text(json.dumps(plan_data), encoding="utf-8")
target = tmp_path / "unrelated" / "mod.py"
with patch("python_pkg.code_tutor.cli.Path.home", return_value=home):
result = _find_codebase_for_file(target)
assert result is None
# ---------------------------------------------------------------------------
# _check_plan_file
# ---------------------------------------------------------------------------
def test_check_plan_file_found(tmp_path: Path) -> None:
codebase = tmp_path / "cb"
codebase.mkdir()
f = codebase / "mod.py"
plan_file = tmp_path / "plan.json"
plan_file.write_text(json.dumps({"codebase_path": str(codebase)}), encoding="utf-8")
result = _check_plan_file(plan_file, f)
assert result == codebase
def test_check_plan_file_value_error(tmp_path: Path) -> None:
codebase = tmp_path / "cb"
codebase.mkdir()
other_file = tmp_path / "other" / "mod.py"
plan_file = tmp_path / "plan.json"
plan_file.write_text(json.dumps({"codebase_path": str(codebase)}), encoding="utf-8")
# other_file is not relative to codebase → ValueError
result = _check_plan_file(plan_file, other_file)
assert result is None
def test_check_plan_file_oserror(tmp_path: Path) -> None:
plan_file = tmp_path / "missing_plan.json"
result = _check_plan_file(plan_file, tmp_path / "mod.py")
assert result is None
def test_check_plan_file_json_error(tmp_path: Path) -> None:
plan_file = tmp_path / "plan.json"
plan_file.write_text("not valid json {{{", encoding="utf-8")
result = _check_plan_file(plan_file, tmp_path / "mod.py")
assert result is None

159
python_pkg/code_tutor/tests/test_cli_status_drill.py Normal file
View File

@ -0,0 +1,159 @@
"""Tests for the code_tutor status and drill commands."""
from __future__ import annotations
from typing import TYPE_CHECKING
from unittest.mock import MagicMock, patch
from typer.testing import CliRunner
from python_pkg.code_tutor.cli import app
if TYPE_CHECKING:
from pathlib import Path
runner = CliRunner()
def _make_plan(codebase: Path, fingerprint: str = "abc") -> dict:
return {
"codebase_path": str(codebase),
"created_at": "2026-01-01T00:00:00+00:00",
"total_items": 2,
"source_fingerprint": fingerprint,
"sessions": [
{
"id": 1,
"title": "mod",
"items": [
{
"id": "mod.fn",
"file": "mod.py",
"type": "function",
"name": "fn",
"start_line": 1,
"end_line": 3,
"class_name": "",
"depends_on": [],
}
],
}
],
}
# ---------------------------------------------------------------------------
# status command
# ---------------------------------------------------------------------------
def test_status_no_plan(tmp_path: Path) -> None:
with patch("python_pkg.code_tutor.cli.load_plan", return_value=None):
result = runner.invoke(app, ["status", str(tmp_path)])
assert result.exit_code == 1
def test_status_with_plan(tmp_path: Path) -> None:
plan = _make_plan(tmp_path)
progress = {"learned": [], "struggled": [], "skipped": [], "last_session": ""}
with (
patch("python_pkg.code_tutor.cli.load_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_fresh_plan", return_value=plan),
patch("python_pkg.code_tutor.cli.load_progress", return_value=progress),
):
result = runner.invoke(app, ["status", str(tmp_path)])
assert result.exit_code == 0
# ---------------------------------------------------------------------------
# drill command
# ---------------------------------------------------------------------------
def test_drill_no_plan_for_file(tmp_path: Path) -> None:
f = tmp_path / "mod.py"
f.write_text("def fn(): pass\n", encoding="utf-8")
with patch("python_pkg.code_tutor.cli._find_codebase_for_file", return_value=None):
result = runner.invoke(app, ["drill", str(f)])
assert result.exit_code == 1
def test_drill_plan_disappears(tmp_path: Path) -> None:
f = tmp_path / "mod.py"
f.write_text("def fn(): pass\n", encoding="utf-8")
with (
patch(
"python_pkg.code_tutor.cli._find_codebase_for_file", return_value=tmp_path
),
patch("python_pkg.code_tutor.cli.load_plan", return_value=None),
):
result = runner.invoke(app, ["drill", str(f)])
assert result.exit_code == 1
def test_drill_no_items_in_plan(tmp_path: Path) -> None:
f = tmp_path / "other.py"
f.write_text("def fn(): pass\n", encoding="utf-8")
plan = _make_plan(tmp_path) # plan has mod.py items, not other.py
with (
patch(
"python_pkg.code_tutor.cli._find_codebase_for_file", return_value=tmp_path
),
patch("python_pkg.code_tutor.cli.load_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_fresh_plan", return_value=plan),
):
result = runner.invoke(app, ["drill", str(f)])
assert result.exit_code == 0
def test_drill_ollama_fails(tmp_path: Path) -> None:
f = tmp_path / "mod.py"
f.write_text("def fn(): pass\n", encoding="utf-8")
plan = _make_plan(tmp_path)
with (
patch(
"python_pkg.code_tutor.cli._find_codebase_for_file", return_value=tmp_path
),
patch("python_pkg.code_tutor.cli.load_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_fresh_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_ollama_running", return_value=False),
):
result = runner.invoke(app, ["drill", str(f)])
assert result.exit_code == 1
def test_drill_runs_lesson(tmp_path: Path) -> None:
from python_pkg.code_tutor._progress import LessonRecord
f = tmp_path / "mod.py"
f.write_text("def fn(): pass\n", encoding="utf-8")
plan = _make_plan(tmp_path)
record = LessonRecord(
timestamp="t",
item_id="mod.fn",
file="mod.py",
lines="1-3",
snippet="code",
outcome="learned",
answers={},
improvement="",
verdict="PASS",
attempt=1,
)
mock_verifier = MagicMock()
mock_verifier.run_lesson.return_value = record
with (
patch(
"python_pkg.code_tutor.cli._find_codebase_for_file", return_value=tmp_path
),
patch("python_pkg.code_tutor.cli.load_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_fresh_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_ollama_running", return_value=True),
patch("python_pkg.code_tutor.cli.Verifier", return_value=mock_verifier),
patch("python_pkg.code_tutor.cli.append_session_record"),
):
result = runner.invoke(app, ["drill", str(f)])
assert result.exit_code == 0
mock_verifier.run_lesson.assert_called_once()

246
python_pkg/code_tutor/tests/test_deps.py Normal file
View File

@ -0,0 +1,246 @@
"""Tests for python_pkg.code_tutor._deps."""
from __future__ import annotations
from pathlib import Path
from unittest.mock import patch
from python_pkg.code_tutor._analyzer import (
CodeItem,
extract_items,
)
from python_pkg.code_tutor._deps import (
_build_dotted_map,
_match_deps,
codebase_fingerprint,
get_file_imports,
get_python_files,
)
# ---------------------------------------------------------------------------
# get_python_files
# ---------------------------------------------------------------------------
def test_get_python_files(tmp_path: Path) -> None:
(tmp_path / "a.py").write_text("x = 1", encoding="utf-8")
(tmp_path / "b.py").write_text("y = 2", encoding="utf-8")
skip_dir = tmp_path / ".venv"
skip_dir.mkdir()
(skip_dir / "c.py").write_text("z = 3", encoding="utf-8")
files = get_python_files(tmp_path)
names = {f.name for f in files}
assert "a.py" in names
assert "b.py" in names
assert "c.py" not in names
# ---------------------------------------------------------------------------
# _build_dotted_map
# ---------------------------------------------------------------------------
def test_build_dotted_map_normal(tmp_path: Path) -> None:
f = tmp_path / "pkg" / "mod.py"
f.parent.mkdir()
f.write_text("x = 1", encoding="utf-8")
result = _build_dotted_map(tmp_path, [f])
assert result[f] == "pkg.mod"
def test_build_dotted_map_value_error(tmp_path: Path) -> None:
other = tmp_path.parent / "other.py"
result = _build_dotted_map(tmp_path, [other])
assert other not in result
# ---------------------------------------------------------------------------
# _match_deps
# ---------------------------------------------------------------------------
def test_match_deps_exact(tmp_path: Path) -> None:
f = tmp_path / "mod.py"
dotted_map = {f: "mod"}
path = tmp_path / "main.py"
result = _match_deps({"mod"}, dotted_map, path)
assert f in result
def test_match_deps_prefix(tmp_path: Path) -> None:
f = tmp_path / "mod.py"
dotted_map = {f: "mod"}
path = tmp_path / "main.py"
result = _match_deps({"pkg.mod"}, dotted_map, path)
assert f in result
def test_match_deps_suffix(tmp_path: Path) -> None:
f = tmp_path / "mod.py"
dotted_map = {f: "pkg.mod"}
path = tmp_path / "main.py"
result = _match_deps({"mod"}, dotted_map, path)
assert f in result
def test_match_deps_excludes_self(tmp_path: Path) -> None:
path = tmp_path / "mod.py"
dotted_map = {path: "mod"}
result = _match_deps({"mod"}, dotted_map, path)
assert path not in result
def test_match_deps_no_match(tmp_path: Path) -> None:
f = tmp_path / "other.py"
dotted_map = {f: "other"}
path = tmp_path / "main.py"
result = _match_deps({"unrelated"}, dotted_map, path)
assert f not in result
# ---------------------------------------------------------------------------
# Tests for get_file_imports
# ---------------------------------------------------------------------------
def test_get_file_imports_import(tmp_path: Path) -> None:
f = tmp_path / "main.py"
dep = tmp_path / "utils.py"
f.write_text("import utils\n", encoding="utf-8")
dep.write_text("x = 1", encoding="utf-8")
result = get_file_imports(f, tmp_path, [f, dep])
assert dep in result
def test_get_file_imports_from_import(tmp_path: Path) -> None:
f = tmp_path / "main.py"
dep = tmp_path / "utils.py"
f.write_text("from utils import helper\n", encoding="utf-8")
dep.write_text("def helper(): pass\n", encoding="utf-8")
result = get_file_imports(f, tmp_path, [f, dep])
assert dep in result
def test_get_file_imports_from_import_none_module(tmp_path: Path) -> None:
# Relative import: node.module is None
f = tmp_path / "main.py"
f.write_text("from . import helper\n", encoding="utf-8")
result = get_file_imports(f, tmp_path, [f])
assert result == set()
def test_get_file_imports_syntax_error(tmp_path: Path) -> None:
f = tmp_path / "bad.py"
f.write_text("def (broken):\n", encoding="utf-8")
result = get_file_imports(f, tmp_path, [f])
assert result == set()
# ---------------------------------------------------------------------------
# codebase_fingerprint
# ---------------------------------------------------------------------------
def test_codebase_fingerprint_basic(tmp_path: Path) -> None:
(tmp_path / "a.py").write_text("x = 1", encoding="utf-8")
result = codebase_fingerprint(tmp_path)
assert len(result) == 16
assert result.isalnum()
def test_codebase_fingerprint_skips_dir(tmp_path: Path) -> None:
sub = tmp_path / "sub"
sub.mkdir()
result = codebase_fingerprint(tmp_path)
assert len(result) == 16
def test_codebase_fingerprint_skips_should_skip(tmp_path: Path) -> None:
venv = tmp_path / ".venv"
venv.mkdir()
(venv / "lib.py").write_text("x = 1", encoding="utf-8")
result_without = codebase_fingerprint(tmp_path)
(tmp_path / "real.py").write_text("y = 2", encoding="utf-8")
result_with = codebase_fingerprint(tmp_path)
assert result_without != result_with
def test_codebase_fingerprint_stat_oserror(tmp_path: Path) -> None:
(tmp_path / "source.py").write_text("x = 1", encoding="utf-8")
stat_calls: dict[str, int] = {}
real_stat = Path.stat
def patched_stat(self: Path, **kwargs: object) -> object:
key = str(self)
stat_calls[key] = stat_calls.get(key, 0) + 1
if stat_calls[key] >= 2 and key.endswith(".py"):
msg = "simulated stat failure"
raise OSError(msg)
return real_stat(self, **kwargs)
with patch.object(Path, "stat", patched_stat):
result = codebase_fingerprint(tmp_path)
assert len(result) == 16
def test_codebase_fingerprint_changes_on_modification(tmp_path: Path) -> None:
f = tmp_path / "mod.py"
f.write_text("x = 1", encoding="utf-8")
fp1 = codebase_fingerprint(tmp_path)
f.write_text("x = 2", encoding="utf-8")
fp2 = codebase_fingerprint(tmp_path)
assert fp1 != fp2
def test_extract_items_image_suffix(tmp_path: Path) -> None:
(tmp_path / "logo.png").write_bytes(b"\x89PNG\r\n")
items = extract_items(tmp_path)
assert items == []
def test_code_item_defaults() -> None:
item = CodeItem(
id="mod.fn",
file="mod.py",
type="function",
name="fn",
start_line=1,
end_line=5,
)
assert item.class_name == ""
assert item.depends_on == []
def test_codebase_fingerprint_stat_oserror_raises(tmp_path: Path) -> None:
"""Skip a file whose ``stat`` raises ``OSError`` (except branch).
``is_file()`` is forced True for the target so the loop reaches the
explicit ``path.stat()`` at fingerprint time; that stat then raises and
the ``except OSError`` handler drops the unreadable entry. Patching both
keeps this deterministic across Python versions (on some, ``is_file``
itself calls ``stat`` and would otherwise re-raise before line reached).
"""
(tmp_path / "source.py").write_text("x = 1", encoding="utf-8")
real_stat = Path.stat
real_is_file = Path.is_file
def failing_stat(self: Path, **kwargs: object) -> object:
"""Raise for the target file, delegate to the real stat otherwise."""
if str(self).endswith("source.py"):
msg = "simulated stat failure"
raise OSError(msg)
return real_stat(self, **kwargs)
def always_file(self: Path) -> bool:
"""Report the target as a regular file without calling stat."""
return str(self).endswith("source.py") or real_is_file(self)
with (
patch.object(Path, "stat", failing_stat),
patch.object(Path, "is_file", always_file),
):
result = codebase_fingerprint(tmp_path)
assert len(result) == 16

250
python_pkg/code_tutor/tests/test_llm.py
View File

@ -5,13 +5,8 @@ from __future__ import annotations
import json
from unittest.mock import MagicMock, patch
import pytest
from python_pkg.code_tutor._claude_backend import ClaudeBackend
from python_pkg.code_tutor._llm import (
OllamaBackend,
_is_weak,
complete_with_fallback,
)
# ---------------------------------------------------------------------------
@ -77,248 +72,3 @@ def test_ollama_stream_empty_token_not_appended() -> None:
assert result == "ok"
assert tokens == ["ok"]
# ---------------------------------------------------------------------------
# ClaudeBackend.complete
# ---------------------------------------------------------------------------
def test_claude_complete_with_text_block() -> None:
mock_block = MagicMock()
mock_block.text = "answer"
mock_message = MagicMock()
mock_message.content = [mock_block]
mock_client = MagicMock()
mock_client.messages.create.return_value = mock_message
with patch(
"python_pkg.code_tutor._claude_backend.anthropic.Anthropic",
return_value=mock_client,
):
backend = ClaudeBackend()
result = backend.complete("sys", "user")
assert result == "answer"
def test_claude_complete_no_text_attr() -> None:
mock_block = MagicMock(spec=[]) # no .text attribute
mock_message = MagicMock()
mock_message.content = [mock_block]
mock_client = MagicMock()
mock_client.messages.create.return_value = mock_message
with patch(
"python_pkg.code_tutor._claude_backend.anthropic.Anthropic",
return_value=mock_client,
):
backend = ClaudeBackend()
result = backend.complete("sys", "user")
assert result == ""
# ---------------------------------------------------------------------------
# ClaudeBackend.stream
# ---------------------------------------------------------------------------
def test_claude_stream_success() -> None:
mock_stream_ctx = MagicMock()
mock_stream_ctx.__enter__ = MagicMock(return_value=mock_stream_ctx)
mock_stream_ctx.__exit__ = MagicMock(return_value=False)
mock_stream_ctx.text_stream = iter(["hello", " world"])
mock_client = MagicMock()
mock_client.messages.stream.return_value = mock_stream_ctx
tokens: list[str] = []
with patch(
"python_pkg.code_tutor._claude_backend.anthropic.Anthropic",
return_value=mock_client,
):
backend = ClaudeBackend()
result = backend.stream("sys", "user", tokens.append)
assert result == "hello world"
assert tokens == ["hello", " world"]
def test_claude_stream_api_error_fallback() -> None:
import anthropic as anthropic_lib
mock_client = MagicMock()
mock_client.messages.stream.side_effect = anthropic_lib.APIError(
message="fail",
request=MagicMock(),
body=None,
)
mock_block = MagicMock()
mock_block.text = "fallback"
mock_message = MagicMock()
mock_message.content = [mock_block]
mock_client.messages.create.return_value = mock_message
tokens: list[str] = []
with patch(
"python_pkg.code_tutor._claude_backend.anthropic.Anthropic",
return_value=mock_client,
):
backend = ClaudeBackend()
result = backend.stream("sys", "user", tokens.append)
assert result == "fallback"
# ---------------------------------------------------------------------------
# _is_weak
# ---------------------------------------------------------------------------
def test_is_weak_short() -> None:
assert _is_weak("ok") is True
def test_is_weak_phrase() -> None:
assert _is_weak("I don't know what this code does at all honestly") is True
def test_is_weak_strong_response() -> None:
long_response = (
"This function reads the first 512 bytes of a file and checks for"
" null bytes to determine if the file is binary."
)
assert _is_weak(long_response) is False
# ---------------------------------------------------------------------------
# complete_with_fallback
# ---------------------------------------------------------------------------
def test_complete_with_fallback_strong() -> None:
mock_primary = MagicMock(spec=OllamaBackend)
mock_primary.complete.return_value = (
"This is a detailed response explaining the code in full."
)
mock_console = MagicMock()
result = complete_with_fallback(
mock_primary, "sys", "user", console=mock_console, input_fn=lambda _: ""
)
assert result == "This is a detailed response explaining the code in full."
mock_primary.complete.assert_called_once()
def test_complete_with_fallback_weak_user_says_yes_claude_ok() -> None:
mock_primary = MagicMock(spec=OllamaBackend)
mock_primary.complete.return_value = "ok" # short → weak
mock_block = MagicMock()
mock_block.text = "claude response is great"
mock_message = MagicMock()
mock_message.content = [mock_block]
mock_client = MagicMock()
mock_client.messages.create.return_value = mock_message
mock_console = MagicMock()
with patch(
"python_pkg.code_tutor._llm.anthropic.Anthropic", return_value=mock_client
):
result = complete_with_fallback(
mock_primary, "sys", "user", console=mock_console, input_fn=lambda _: "y"
)
assert result == "claude response is great"
def test_complete_with_fallback_weak_user_says_yes_claude_fails() -> None:
import anthropic as anthropic_lib
mock_primary = MagicMock(spec=OllamaBackend)
mock_primary.complete.side_effect = [
"ok", # first call → weak
"retry response from ollama", # retry after claude failure
]
mock_client = MagicMock()
mock_client.messages.create.side_effect = anthropic_lib.APIError(
message="fail",
request=MagicMock(),
body=None,
)
mock_console = MagicMock()
with patch(
"python_pkg.code_tutor._llm.anthropic.Anthropic", return_value=mock_client
):
result = complete_with_fallback(
mock_primary, "sys", "user", console=mock_console, input_fn=lambda _: "y"
)
assert result == "retry response from ollama"
assert mock_primary.complete.call_count == 2
def test_complete_with_fallback_weak_user_says_no() -> None:
mock_primary = MagicMock(spec=OllamaBackend)
mock_primary.complete.side_effect = [
"ok", # first call → weak
"second ollama response here too", # retry
]
mock_console = MagicMock()
result = complete_with_fallback(
mock_primary, "sys", "user", console=mock_console, input_fn=lambda _: "n"
)
assert result == "second ollama response here too"
assert mock_primary.complete.call_count == 2
def test_ollama_backend_defaults() -> None:
backend = OllamaBackend()
assert "11434" in backend._url
assert backend._timeout > 0
@pytest.mark.parametrize(
"phrase",
[
"i cannot help with that request really",
"it is unclear what you mean by that",
"i'm not sure about this code",
],
)
def test_is_weak_various_phrases(phrase: str) -> None:
assert _is_weak(phrase) is True
def test_ollama_stream_no_done_marker() -> None:
"""Exhaust the stream loop without a ``done`` marker (no ``break``).
When no line carries ``done: true`` the loop runs to exhaustion and
falls through to ``return "".join(parts)`` rather than breaking early.
"""
lines = [
json.dumps({"response": "foo", "done": False}).encode(),
json.dumps({"response": "bar", "done": False}).encode(),
]
mock_resp = MagicMock()
mock_resp.iter_lines.return_value = iter(lines)
tokens: list[str] = []
with patch("python_pkg.code_tutor._llm.requests.post", return_value=mock_resp):
backend = OllamaBackend()
result = backend.stream("sys", "user", tokens.append)
assert result == "foobar"
assert tokens == ["foo", "bar"]

164
python_pkg/code_tutor/tests/test_llm_fallback.py Normal file
View File

@ -0,0 +1,164 @@
"""Tests for weak-response detection and the Claude fallback."""
from __future__ import annotations
import json
from unittest.mock import MagicMock, patch
import pytest
from python_pkg.code_tutor._llm import (
OllamaBackend,
_is_weak,
complete_with_fallback,
)
# ---------------------------------------------------------------------------
# _is_weak
# ---------------------------------------------------------------------------
def test_is_weak_short() -> None:
assert _is_weak("ok") is True
def test_is_weak_phrase() -> None:
assert _is_weak("I don't know what this code does at all honestly") is True
def test_is_weak_strong_response() -> None:
long_response = (
"This function reads the first 512 bytes of a file and checks for"
" null bytes to determine if the file is binary."
)
assert _is_weak(long_response) is False
# ---------------------------------------------------------------------------
# complete_with_fallback
# ---------------------------------------------------------------------------
def test_complete_with_fallback_strong() -> None:
mock_primary = MagicMock(spec=OllamaBackend)
mock_primary.complete.return_value = (
"This is a detailed response explaining the code in full."
)
mock_console = MagicMock()
result = complete_with_fallback(
mock_primary, "sys", "user", console=mock_console, input_fn=lambda _: ""
)
assert result == "This is a detailed response explaining the code in full."
mock_primary.complete.assert_called_once()
def test_complete_with_fallback_weak_user_says_yes_claude_ok() -> None:
mock_primary = MagicMock(spec=OllamaBackend)
mock_primary.complete.return_value = "ok" # short → weak
mock_block = MagicMock()
mock_block.text = "claude response is great"
mock_message = MagicMock()
mock_message.content = [mock_block]
mock_client = MagicMock()
mock_client.messages.create.return_value = mock_message
mock_console = MagicMock()
with patch(
"python_pkg.code_tutor._llm.anthropic.Anthropic", return_value=mock_client
):
result = complete_with_fallback(
mock_primary, "sys", "user", console=mock_console, input_fn=lambda _: "y"
)
assert result == "claude response is great"
def test_complete_with_fallback_weak_user_says_yes_claude_fails() -> None:
import anthropic as anthropic_lib
mock_primary = MagicMock(spec=OllamaBackend)
mock_primary.complete.side_effect = [
"ok", # first call → weak
"retry response from ollama", # retry after claude failure
]
mock_client = MagicMock()
mock_client.messages.create.side_effect = anthropic_lib.APIError(
message="fail",
request=MagicMock(),
body=None,
)
mock_console = MagicMock()
with patch(
"python_pkg.code_tutor._llm.anthropic.Anthropic", return_value=mock_client
):
result = complete_with_fallback(
mock_primary, "sys", "user", console=mock_console, input_fn=lambda _: "y"
)
assert result == "retry response from ollama"
assert mock_primary.complete.call_count == 2
def test_complete_with_fallback_weak_user_says_no() -> None:
mock_primary = MagicMock(spec=OllamaBackend)
mock_primary.complete.side_effect = [
"ok", # first call → weak
"second ollama response here too", # retry
]
mock_console = MagicMock()
result = complete_with_fallback(
mock_primary, "sys", "user", console=mock_console, input_fn=lambda _: "n"
)
assert result == "second ollama response here too"
assert mock_primary.complete.call_count == 2
def test_ollama_backend_defaults() -> None:
backend = OllamaBackend()
assert "11434" in backend._url
assert backend._timeout > 0
@pytest.mark.parametrize(
"phrase",
[
"i cannot help with that request really",
"it is unclear what you mean by that",
"i'm not sure about this code",
],
)
def test_is_weak_various_phrases(phrase: str) -> None:
assert _is_weak(phrase) is True
def test_ollama_stream_no_done_marker() -> None:
"""Exhaust the stream loop without a ``done`` marker (no ``break``).
When no line carries ``done: true`` the loop runs to exhaustion and
falls through to ``return "".join(parts)`` rather than breaking early.
"""
lines = [
json.dumps({"response": "foo", "done": False}).encode(),
json.dumps({"response": "bar", "done": False}).encode(),
]
mock_resp = MagicMock()
mock_resp.iter_lines.return_value = iter(lines)
tokens: list[str] = []
with patch("python_pkg.code_tutor._llm.requests.post", return_value=mock_resp):
backend = OllamaBackend()
result = backend.stream("sys", "user", tokens.append)
assert result == "foobar"
assert tokens == ["foo", "bar"]

163
python_pkg/code_tutor/tests/test_pytest_runner.py Normal file
View File

@ -0,0 +1,163 @@
"""Tests for python_pkg.code_tutor._pytest_runner."""
from __future__ import annotations
from pathlib import Path
from unittest.mock import MagicMock, patch
from python_pkg.code_tutor._challenge_support import (
_scan_test_file,
_show_test_panels,
)
from python_pkg.code_tutor._pytest_runner import _patch_and_test, _pytest_clean
from python_pkg.code_tutor.tests.conftest import _item
# ---------------------------------------------------------------------------
# _pytest_clean
# ---------------------------------------------------------------------------
def test_pytest_clean_pass() -> None:
mock_result = MagicMock()
mock_result.returncode = 0
mock_result.stdout = "1 passed"
mock_result.stderr = ""
mock_console = MagicMock()
with patch(
"python_pkg.code_tutor._pytest_runner.subprocess.run",
return_value=mock_result,
):
assert (
_pytest_clean(["test_mod.py::test_fn"], Path("/proj"), mock_console) is True
)
def test_pytest_clean_fail() -> None:
mock_result = MagicMock()
mock_result.returncode = 1
mock_result.stdout = ""
mock_result.stderr = "FAILED"
mock_console = MagicMock()
with patch(
"python_pkg.code_tutor._pytest_runner.subprocess.run",
return_value=mock_result,
):
assert (
_pytest_clean(["test_mod.py::test_fn"], Path("/proj"), mock_console)
is False
)
def test_pytest_clean_no_output() -> None:
mock_result = MagicMock()
mock_result.returncode = 0
mock_result.stdout = ""
mock_result.stderr = ""
mock_console = MagicMock()
with patch(
"python_pkg.code_tutor._pytest_runner.subprocess.run",
return_value=mock_result,
):
_pytest_clean([], Path("/proj"), mock_console)
mock_console.print.assert_not_called()
# ---------------------------------------------------------------------------
# _patch_and_test
# ---------------------------------------------------------------------------
def test_patch_and_test_syntax_error(tmp_path: Path) -> None:
source = "def fn():\n return 1\n"
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
mock_console = MagicMock()
result = _patch_and_test(
_item(file="mod.py", start=1, end=2),
str(tmp_path),
"def fn(\n broken syntax",
[],
mock_console,
)
assert result is False
# Original file should be restored
assert (tmp_path / "mod.py").read_text(encoding="utf-8") == source
def test_patch_and_test_passes(tmp_path: Path) -> None:
source = "def fn():\n return 1\n"
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
test_file = tmp_path / "test_mod.py"
test_file.write_text("def test_fn(): pass\n", encoding="utf-8")
mock_console = MagicMock()
with patch("python_pkg.code_tutor._pytest_runner._pytest_clean", return_value=True):
result = _patch_and_test(
_item(file="mod.py", start=1, end=2),
str(tmp_path),
"def fn():\n return 99",
[(test_file, ["test_fn"])],
mock_console,
)
assert result is True
# File restored
assert (tmp_path / "mod.py").read_text(encoding="utf-8") == source
def test_patch_and_test_fails(tmp_path: Path) -> None:
source = "def fn():\n return 1\n"
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
test_file = tmp_path / "test_mod.py"
test_file.write_text("def test_fn(): pass\n", encoding="utf-8")
mock_console = MagicMock()
with patch(
"python_pkg.code_tutor._pytest_runner._pytest_clean", return_value=False
):
result = _patch_and_test(
_item(file="mod.py", start=1, end=2),
str(tmp_path),
"def fn():\n return 0",
[(test_file, ["test_fn"])],
mock_console,
)
assert result is False
assert (tmp_path / "mod.py").read_text(encoding="utf-8") == source
def test_scan_test_file_non_func_non_class_toplevel() -> None:
"""Skip a top-level node that is neither a function nor a class def.
A module-level ``import`` fails both ``isinstance`` checks, exercising
the branch that falls through the ``elif ClassDef`` back to the loop.
"""
source = "import os\ndef test_fn_uses_fn():\n fn()\n"
result = _scan_test_file(source, "fn")
assert result == ["test_fn_uses_fn"]
def test_show_test_panels_non_func_non_class_toplevel(tmp_path: Path) -> None:
"""Skip a top-level node that is neither a function nor a class def.
A module-level ``import`` yields an empty ``nodes`` list, so the
``elif ClassDef`` branch is not taken before the inner render loop.
"""
source = "import os\ndef test_fn():\n fn(1)\n"
test_file = tmp_path / "test_mod.py"
test_file.write_text(source, encoding="utf-8")
mock_console = MagicMock()
with patch("python_pkg.code_tutor._challenge_support.Syntax"):
_show_test_panels([(test_file, ["test_fn"])], mock_console)
assert mock_console.print.called
def test_show_test_panels_class_non_func_member(tmp_path: Path) -> None:
"""Skip a class-body member that is not a function definition.
The class-level ``attr = 1`` assignment fails the member ``isinstance``
check, exercising the branch that loops back to the next body member.
"""
source = "class TestFoo:\n attr = 1\n def test_fn(self):\n fn()\n"
test_file = tmp_path / "test_mod.py"
test_file.write_text(source, encoding="utf-8")
mock_console = MagicMock()
with patch("python_pkg.code_tutor._challenge_support.Syntax"):
_show_test_panels([(test_file, ["TestFoo::test_fn"])], mock_console)
assert mock_console.print.called

71
python_pkg/code_tutor/tests/test_run_coding_challenge.py Normal file
View File

@ -0,0 +1,71 @@
"""Tests for the run_coding_challenge entry point."""
from __future__ import annotations
from typing import TYPE_CHECKING
from unittest.mock import MagicMock, patch
from python_pkg.code_tutor._challenge_flows import (
run_coding_challenge,
)
from python_pkg.code_tutor.tests.conftest import _item
if TYPE_CHECKING:
from pathlib import Path
# ---------------------------------------------------------------------------
# run_coding_challenge
# ---------------------------------------------------------------------------
def test_run_coding_challenge_non_python() -> None:
mock_console = MagicMock()
mock_backend = MagicMock()
result = run_coding_challenge(
_item(file="main.go"),
"/codebase",
"explanation",
mock_backend,
mock_console,
)
assert result == "skipped"
def test_run_coding_challenge_with_existing_tests(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn(): pass\n", encoding="utf-8")
test_file = tmp_path / "test_mod.py"
test_file.write_text("def test_fn(): fn()\n", encoding="utf-8")
mock_console = MagicMock()
mock_backend = MagicMock()
with patch(
"python_pkg.code_tutor._challenge_flows._existing_tests_flow",
return_value="passed",
):
result = run_coding_challenge(
_item(file="mod.py"),
str(tmp_path),
"explanation",
mock_backend,
mock_console,
)
assert result == "passed"
def test_run_coding_challenge_no_tests(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn(): pass\n", encoding="utf-8")
mock_console = MagicMock()
mock_backend = MagicMock()
with patch(
"python_pkg.code_tutor._challenge_flows._write_tests_first_flow",
return_value="skipped",
):
result = run_coding_challenge(
_item(file="mod.py"),
str(tmp_path),
"explanation",
mock_backend,
mock_console,
)
assert result == "skipped"

224
python_pkg/code_tutor/tests/test_signature_block.py Normal file
View File

@ -0,0 +1,224 @@
"""Tests for signature-block extraction and import hints."""
from __future__ import annotations
from pathlib import Path
from unittest.mock import MagicMock, patch
from python_pkg.code_tutor._challenge import (
_collect_and_rate_tests,
_validate_tests_against_real,
)
from python_pkg.code_tutor._challenge_support import (
_extract_signature_block,
_import_hint,
)
from python_pkg.code_tutor.tests.conftest import _item, _make_live_mock
# ---------------------------------------------------------------------------
# _extract_signature_block
# ---------------------------------------------------------------------------
def test_extract_signature_block_with_docstring(tmp_path: Path) -> None:
source = 'def fn(x):\n """Do thing."""\n return x\n'
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
result = _extract_signature_block(_item(file="mod.py"), str(tmp_path))
assert "def fn" in result
assert '"""Do thing."""' in result
def test_extract_signature_block_without_docstring(tmp_path: Path) -> None:
source = "def fn(x):\n return x\n"
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
result = _extract_signature_block(_item(file="mod.py"), str(tmp_path))
assert "def fn" in result
assert "return" not in result
def test_extract_signature_block_oserror(tmp_path: Path) -> None:
result = _extract_signature_block(_item(file="missing.py"), str(tmp_path))
assert "def fn(...):" in result
def test_extract_signature_block_no_match(tmp_path: Path) -> None:
source = "def other(): pass\n"
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
result = _extract_signature_block(_item(file="mod.py", name="fn"), str(tmp_path))
assert "def fn(...):" in result
def test_extract_signature_block_syntax_error(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def (broken):\n", encoding="utf-8")
result = _extract_signature_block(_item(file="mod.py"), str(tmp_path))
assert "def fn(...):" in result
# ---------------------------------------------------------------------------
# Tests for _import_hint
# ---------------------------------------------------------------------------
def test_import_hint_relative_to_project(tmp_path: Path) -> None:
(tmp_path / "src").mkdir()
result = _import_hint(
_item(file="src/mod.py", name="fn"),
str(tmp_path),
tmp_path,
)
assert result == "from src.mod import fn"
def test_import_hint_value_error(tmp_path: Path) -> None:
# project_root is different from codebase_path
other_root = tmp_path / "other"
other_root.mkdir()
result = _import_hint(
_item(file="src/mod.py", name="fn"),
str(tmp_path),
other_root,
)
assert result == "from mod import fn"
# ---------------------------------------------------------------------------
# _validate_tests_against_real
# ---------------------------------------------------------------------------
def test_validate_tests_syntax_error() -> None:
mock_console = MagicMock()
result = _validate_tests_against_real(
"def (broken):", "import fn", Path("/proj"), mock_console
)
assert result is False
def test_validate_tests_passes() -> None:
mock_console = MagicMock()
with patch("python_pkg.code_tutor._challenge._pytest_clean", return_value=True):
result = _validate_tests_against_real(
"def test_fn():\n assert True\n",
"# import",
Path("/proj"),
mock_console,
)
assert result is True
def test_validate_tests_fails() -> None:
mock_console = MagicMock()
with patch("python_pkg.code_tutor._challenge._pytest_clean", return_value=False):
result = _validate_tests_against_real(
"def test_fn():\n assert False\n",
"# import",
Path("/proj"),
mock_console,
)
assert result is False
# ---------------------------------------------------------------------------
# _collect_and_rate_tests
# ---------------------------------------------------------------------------
def test_collect_and_rate_tests_skip() -> None:
mock_backend = MagicMock()
mock_console = MagicMock()
live_mock = _make_live_mock()
with patch("python_pkg.code_tutor._verdict.Live", return_value=live_mock):
result = _collect_and_rate_tests(
"def fn():",
"explanation",
mock_backend,
mock_console,
lambda _: "skip",
)
assert result is None
def test_collect_and_rate_tests_pass_first() -> None:
mock_backend = MagicMock()
mock_console = MagicMock()
live_mock = _make_live_mock()
def fake_stream(system: str, user: str, on_token: object) -> str:
assert callable(on_token)
on_token('{"verdict": "PASS", "gap": ""}')
return ""
mock_backend.stream.side_effect = fake_stream
lines_iter = iter(["test code line", "END"])
with patch("python_pkg.code_tutor._verdict.Live", return_value=live_mock):
result = _collect_and_rate_tests(
"def fn():",
"explanation",
mock_backend,
mock_console,
lambda _: next(lines_iter),
)
assert result == "test code line"
def test_collect_and_rate_tests_fail_twice() -> None:
mock_backend = MagicMock()
mock_console = MagicMock()
live_mock = _make_live_mock()
mock_backend.stream.return_value = '{"verdict": "FAIL", "gap": "bad"}'
attempt_inputs = [
# attempt 1: provide code, then END
"test code",
"END",
# attempt 2: provide code, then END
"test code again",
"END",
]
inputs_iter = iter(attempt_inputs)
with patch("python_pkg.code_tutor._verdict.Live", return_value=live_mock):
result = _collect_and_rate_tests(
"def fn():",
"explanation",
mock_backend,
mock_console,
lambda _: next(inputs_iter),
)
assert result is None
def test_collect_and_rate_tests_fail_then_pass() -> None:
mock_backend = MagicMock()
mock_console = MagicMock()
live_mock = _make_live_mock()
call_count = [0]
def fake_stream(system: str, user: str, on_token: object) -> str:
assert callable(on_token)
call_count[0] += 1
verdict = (
'{"verdict": "FAIL", "gap": "not enough"}'
if call_count[0] == 1
else '{"verdict": "PASS", "gap": ""}'
)
on_token(verdict)
return ""
mock_backend.stream.side_effect = fake_stream
inputs = iter(["bad test", "END", "good test", "END"])
with patch("python_pkg.code_tutor._verdict.Live", return_value=live_mock):
result = _collect_and_rate_tests(
"def fn():",
"explanation",
mock_backend,
mock_console,
lambda _: next(inputs),
)
assert result == "good test"

142
python_pkg/code_tutor/tests/test_verifier.py
View File

@ -3,15 +3,15 @@
from __future__ import annotations
from typing import TYPE_CHECKING
from unittest.mock import MagicMock, patch
from unittest.mock import patch
from python_pkg.code_tutor._analyzer import CodeItem
from python_pkg.code_tutor._verifier import Verifier
from python_pkg.code_tutor._verifier_support import (
_class_header,
_parse_verdict,
_read_snippet,
)
from python_pkg.code_tutor.tests.conftest import _make_live_mock, _make_verifier
if TYPE_CHECKING:
from pathlib import Path
@ -152,20 +152,6 @@ def test_parse_verdict_strips_markdown_fence() -> None:
# ---------------------------------------------------------------------------
def _make_live_mock() -> MagicMock:
live = MagicMock()
live.__enter__ = MagicMock(return_value=live)
live.__exit__ = MagicMock(return_value=False)
return live
def _make_verifier() -> tuple[Verifier, MagicMock, MagicMock]:
mock_backend = MagicMock()
mock_console = MagicMock()
verifier = Verifier(mock_backend, mock_console)
return verifier, mock_backend, mock_console
def test_judge_calls_stream_and_parses(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn():\n pass\n", encoding="utf-8")
verifier, mock_backend, _ = _make_verifier()
@ -235,127 +221,3 @@ def test_ask_improvement_empty() -> None:
verifier, _, _ = _make_verifier()
result = verifier._ask_improvement(lambda _: "")
assert result == ""
# ---------------------------------------------------------------------------
# Verifier.run_lesson
# ---------------------------------------------------------------------------
def _four_answers() -> list[str]:
return ["purpose", "inputs", "outputs", "why"]
def test_run_lesson_pass_first_attempt(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn():\n pass\n", encoding="utf-8")
verifier, mock_backend, _ = _make_verifier()
def fake_stream(system: str, user: str, on_token: object) -> str:
assert callable(on_token)
on_token('{"verdict": "PASS", "gap": ""}')
return ""
mock_backend.stream.side_effect = fake_stream
answers_iter = iter(_four_answers())
live_mock = _make_live_mock()
with (
patch("python_pkg.code_tutor._verifier.Live", return_value=live_mock),
patch(
"python_pkg.code_tutor._verifier.run_coding_challenge",
return_value="skipped",
),
):
record = verifier.run_lesson(
_make_item(file="mod.py"),
str(tmp_path),
input_fn=lambda _: next(answers_iter, ""),
)
assert record.outcome == "learned"
assert record.verdict == "PASS"
assert record.attempt == 1
def test_run_lesson_skip_on_first_question(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn():\n pass\n", encoding="utf-8")
verifier, _, _ = _make_verifier()
live_mock = _make_live_mock()
with patch("python_pkg.code_tutor._verifier.Live", return_value=live_mock):
record = verifier.run_lesson(
_make_item(file="mod.py"),
str(tmp_path),
input_fn=lambda _: "skip",
)
assert record.outcome == "skipped"
assert record.verdict == "skipped"
def test_run_lesson_all_attempts_fail(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn():\n pass\n", encoding="utf-8")
verifier, mock_backend, _ = _make_verifier()
call_count = [0]
def fail_stream(system: str, user: str, on_token: object) -> str:
assert callable(on_token)
on_token('{"verdict": "FAIL", "gap": "Missing detail."}')
return ""
mock_backend.stream.side_effect = fail_stream
def input_fn(_: str) -> str:
call_count[0] += 1
# Each "round" of 4 questions: return an answer
return "my answer"
live_mock = _make_live_mock()
with patch("python_pkg.code_tutor._verifier.Live", return_value=live_mock):
record = verifier.run_lesson(
_make_item(file="mod.py"),
str(tmp_path),
input_fn=input_fn,
)
assert record.outcome == "struggled"
assert record.verdict == "FAIL"
assert record.attempt == 3
def test_run_lesson_pass_on_second_attempt(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn():\n pass\n", encoding="utf-8")
verifier, mock_backend, _ = _make_verifier()
stream_calls = [0]
def fake_stream(system: str, user: str, on_token: object) -> str:
assert callable(on_token)
stream_calls[0] += 1
verdict = (
'{"verdict": "FAIL", "gap": "gap"}'
if stream_calls[0] == 1
else '{"verdict": "PASS", "gap": ""}'
)
on_token(verdict)
return ""
mock_backend.stream.side_effect = fake_stream
live_mock = _make_live_mock()
with (
patch("python_pkg.code_tutor._verifier.Live", return_value=live_mock),
patch(
"python_pkg.code_tutor._verifier.run_coding_challenge",
return_value="passed",
),
):
record = verifier.run_lesson(
_make_item(file="mod.py"),
str(tmp_path),
input_fn=lambda _: "my answer",
)
assert record.outcome == "learned"
assert record.attempt == 2

154
python_pkg/code_tutor/tests/test_verifier_lesson.py Normal file
View File

@ -0,0 +1,154 @@
"""Tests for Verifier.run_lesson."""
from __future__ import annotations
from typing import TYPE_CHECKING
from unittest.mock import patch
from python_pkg.code_tutor._analyzer import CodeItem
from python_pkg.code_tutor.tests.conftest import _make_live_mock, _make_verifier
if TYPE_CHECKING:
from pathlib import Path
def _make_item(
file: str = "mod.py",
name: str = "fn",
start: int = 1,
end: int = 3,
class_name: str = "",
) -> CodeItem:
return CodeItem(
id=f"{file}.{name}",
file=file,
type="function",
name=name,
start_line=start,
end_line=end,
class_name=class_name,
)
# ---------------------------------------------------------------------------
# Verifier.run_lesson
# ---------------------------------------------------------------------------
def _four_answers() -> list[str]:
return ["purpose", "inputs", "outputs", "why"]
def test_run_lesson_pass_first_attempt(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn():\n pass\n", encoding="utf-8")
verifier, mock_backend, _ = _make_verifier()
def fake_stream(system: str, user: str, on_token: object) -> str:
assert callable(on_token)
on_token('{"verdict": "PASS", "gap": ""}')
return ""
mock_backend.stream.side_effect = fake_stream
answers_iter = iter(_four_answers())
live_mock = _make_live_mock()
with (
patch("python_pkg.code_tutor._verifier.Live", return_value=live_mock),
patch(
"python_pkg.code_tutor._verifier.run_coding_challenge",
return_value="skipped",
),
):
record = verifier.run_lesson(
_make_item(file="mod.py"),
str(tmp_path),
input_fn=lambda _: next(answers_iter, ""),
)
assert record.outcome == "learned"
assert record.verdict == "PASS"
assert record.attempt == 1
def test_run_lesson_skip_on_first_question(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn():\n pass\n", encoding="utf-8")
verifier, _, _ = _make_verifier()
live_mock = _make_live_mock()
with patch("python_pkg.code_tutor._verifier.Live", return_value=live_mock):
record = verifier.run_lesson(
_make_item(file="mod.py"),
str(tmp_path),
input_fn=lambda _: "skip",
)
assert record.outcome == "skipped"
assert record.verdict == "skipped"
def test_run_lesson_all_attempts_fail(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn():\n pass\n", encoding="utf-8")
verifier, mock_backend, _ = _make_verifier()
call_count = [0]
def fail_stream(system: str, user: str, on_token: object) -> str:
assert callable(on_token)
on_token('{"verdict": "FAIL", "gap": "Missing detail."}')
return ""
mock_backend.stream.side_effect = fail_stream
def input_fn(_: str) -> str:
call_count[0] += 1
# Each "round" of 4 questions: return an answer
return "my answer"
live_mock = _make_live_mock()
with patch("python_pkg.code_tutor._verifier.Live", return_value=live_mock):
record = verifier.run_lesson(
_make_item(file="mod.py"),
str(tmp_path),
input_fn=input_fn,
)
assert record.outcome == "struggled"
assert record.verdict == "FAIL"
assert record.attempt == 3
def test_run_lesson_pass_on_second_attempt(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn():\n pass\n", encoding="utf-8")
verifier, mock_backend, _ = _make_verifier()
stream_calls = [0]
def fake_stream(system: str, user: str, on_token: object) -> str:
assert callable(on_token)
stream_calls[0] += 1
verdict = (
'{"verdict": "FAIL", "gap": "gap"}'
if stream_calls[0] == 1
else '{"verdict": "PASS", "gap": ""}'
)
on_token(verdict)
return ""
mock_backend.stream.side_effect = fake_stream
live_mock = _make_live_mock()
with (
patch("python_pkg.code_tutor._verifier.Live", return_value=live_mock),
patch(
"python_pkg.code_tutor._verifier.run_coding_challenge",
return_value="passed",
),
):
record = verifier.run_lesson(
_make_item(file="mod.py"),
str(tmp_path),
input_fn=lambda _: "my answer",
)
assert record.outcome == "learned"
assert record.attempt == 2