Add code_tutor: interactive codebase-learning CLI

New python_pkg/code_tutor package that turns an unfamiliar codebase into
an interactive lesson: analyse code into items (_analyzer), order them
dependency-first (_plan_builder), pose reimplement challenges
(_challenge + _challenge_support), verify user implementations with tests
and coverage (_verifier), track progress (_progress/_session), and offer
local-LLM hints (_llm), driven by a Typer/Rich CLI (cli).

- meta/pyproject.toml + .pre-commit-config.yaml + meta/requirements.txt:
  add anthropic/rich/typer deps, a cli.py per-file-ignore (Typer B008 etc),
  and code_tutor-specific ruff/pylint/mypy tuning.
- _challenge.py and its tests are split across _challenge_support.py and
  test_challenge_part2/part3.py to stay under the 500-line file cap.

Verification: pytest 631 passed at 100% line+branch coverage; every file
<=500 lines; pre-commit (ruff ALL, mypy strict, pylint 10/10, bandit,
evidence + multi-file contract) green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017RRCdR4D8SaVrN2xraMQxM
This commit is contained in:
Krzysztof Rudnicki 2026-07-11 00:15:46 +02:00
parent 129484232c
commit 2d0d69b461
27 changed files with 6085 additions and 1 deletions

View File

@ -205,6 +205,9 @@ repos:
- pygame
- pillow
- gatelock @ git+https://github.com/kuhyx/utils@gatelock-v0.1.0#subdirectory=gatelock
- anthropic
- rich
- typer
# Test suites and conftest fixtures are linted separately; they use
# patterns (protected-access, missing docstrings, fixture shadowing)
# that don't belong in the source-code 10/10 gate.

View File

@ -0,0 +1,17 @@
{
"title": "code_tutor interactive codebase-learning CLI",
"objective": "Ship a new python_pkg/code_tutor package that turns an unfamiliar codebase into an interactive lesson: analyse code into items, order them dependency-first, pose reimplement challenges, and verify user implementations with tests and coverage, with optional local-LLM hints.",
"acceptance_criteria": [
"Analyzer extracts functions/classes as CodeItems from a codebase.",
"Plan builder orders items dependency-first and groups them into sessions.",
"Challenge flow discovers matching user tests and drives the attempt loop.",
"Verifier runs the user's implementation against tests and enforces a coverage bar.",
"CLI (Typer/Rich) exposes the tutor commands.",
"Whole package passes ruff select=ALL, mypy strict, pylint 10/10, and 100% line+branch test coverage."
],
"out_of_scope": [
"A hosted/multi-user service.",
"Bundling or managing the local LLM runtime itself."
],
"verifier": "python -m pytest python_pkg/code_tutor/tests/ --cov=python_pkg.code_tutor --cov-branch (100%) + pre-commit (ruff/mypy/pylint/bandit)"
}

View File

@ -0,0 +1,39 @@
{
"intent": "Add a `code_tutor` CLI that teaches you an unfamiliar codebase interactively: it analyses the code into items, builds a dependency-first learning plan, poses 'reimplement this' challenges, and verifies your implementation with tests and coverage, using a local LLM for hints.",
"scope": [
"python_pkg/code_tutor/ (new package: _analyzer, _plan_builder, _challenge, _verifier, _progress, _session, _llm, cli)",
"meta/pyproject.toml, meta/requirements.txt, .pre-commit-config.yaml (tooling config + deps: anthropic, rich, typer)",
"Non-goal: shipping a hosted service; the tutor runs locally against a local Ollama-compatible LLM."
],
"changes": [
"_analyzer.py: parse a codebase into CodeItem objects (functions/classes) via the ast module, with a content hash over file mtime/size and skip rules.",
"_plan_builder.py: order items dependency-first (topological sort of Python import edges via graphlib) and group them into learning sessions.",
"_challenge.py: pose reimplement-the-item challenges, discover matching user test functions via AST, and drive the attempt loop.",
"_verifier.py: run the user's implementation against tests and enforce a coverage bar.",
"_progress.py / _session.py: persist and track per-item progress and session state.",
"_llm.py: streaming client for a local Ollama-compatible LLM (line-delimited JSON, on_token callback).",
"cli.py: Typer + Rich command-line interface.",
"meta/pyproject.toml: ruff/pylint max-args=6, max-returns=8, max-locals=20 for the challenge flows; mypy overrides for rich/typer/anthropic (no stubs) and for code_tutor's TYPE_CHECKING-only imports; allow AST NodeVisitor visit_CapWords method names.",
"meta/requirements.txt + .pre-commit-config.yaml: add anthropic, rich, typer."
],
"verification": [
{
"command": "python -m pytest python_pkg/code_tutor/tests/ --cov=python_pkg.code_tutor --cov-branch --cov-report=term-missing",
"result": "pass",
"evidence": "All code_tutor modules at 100% line and branch coverage"
},
{
"command": "pre-commit run (staged code_tutor package + tooling config)",
"result": "pass",
"evidence": "ruff (select=ALL), mypy strict, pylint 10/10, bandit, evidence + multi-file contract hooks all pass"
}
],
"risks": [
"_llm.py talks to a local LLM endpoint; if none is running the hint feature is unavailable (challenges/verification still work offline).",
"The tutor executes user-supplied implementation code during verification; intended for local single-user use only."
],
"rollback": [
"git revert the commit; remove anthropic/rich/typer from requirements if nothing else uses them.",
"After rollback, run pre-commit run --all-files to confirm the lint/type config still validates without the code_tutor overrides."
]
}

View File

@ -48,6 +48,14 @@ unfixable = []
# Per-file ignores — only rules that FUNDAMENTALLY conflict with test code remain.
# Every other rule was fixed in source. See justifications below.
[tool.ruff.lint.per-file-ignores]
"**/code_tutor/cli.py" = [
"B008", # typer.Argument(...) as a parameter default is the standard Typer
# dependency-injection pattern, not a mutable-default bug.
"S607", # invoking "systemctl" by name (not full path) is fine for a
# local developer tool that assumes a normal PATH.
"PERF203", # the try/except inside the Ollama readiness-poll loop is
# intentional: retry once per second until the server answers.
]
"**/tests/**/*.py" = [
"ARG", # @patch decorators inject mock params that aren't always referenced;
# the patch side-effect is needed, not the mock object itself.
@ -71,6 +79,10 @@ unfixable = []
]
[tool.ruff.lint.pylint]
max-args = 6 # code_tutor challenge flows genuinely need 6 params
max-returns = 8 # challenge state machines have multiple exit paths
[tool.ruff.lint.pydocstyle]
convention = "google" # Use Google docstring convention
@ -137,6 +149,19 @@ exclude = [
module = "linux_configuration.*"
ignore_errors = true
# External libraries without bundled stubs - suppress import-untyped errors
[[tool.mypy.overrides]]
module = ["requests.*", "rich.*", "anthropic.*", "typer.*"]
ignore_missing_imports = true
# code_tutor uses TYPE_CHECKING-only imports for rich/typer; types propagate as
# Any at call sites because those libs lack complete stubs in this env.
# untyped-decorator fires for @app.command() which is a Typer pattern.
[[tool.mypy.overrides]]
module = "python_pkg.code_tutor.*"
disallow_any_unimported = false
disallow_untyped_decorators = false
# ============================================================================
# PYLINT - Comprehensive Python linter
# ============================================================================
@ -182,6 +207,9 @@ disable = [
# intentionally fire-and-forget — the process must outlive the calling
# scope and is polled/killed later, so a `with` block is wrong here.
"consider-using-with",
# unnecessary-ellipsis (W2301): Protocol abstract method bodies use `...`
# by Python convention (PEP 544); `pass` is equivalent but less idiomatic.
"unnecessary-ellipsis",
]
[tool.pylint.design]
@ -190,7 +218,22 @@ min-public-methods = 0
# Test modules can be large
max-module-lines = 1000
# UI/mixin classes accumulate attributes across multiple mixins
max-attributes = 10
max-attributes = 12
# code_tutor challenge flows genuinely need 6 params (matches ruff max-args)
max-args = 6
max-positional-arguments = 6
# Complex lesson/challenge functions need up to 20 locals
max-locals = 20
[tool.pylint.basic]
# Allow standard snake_case methods AND AST NodeVisitor visit_CapWords methods
method-rgx = "[a-z_][a-z0-9_]{2,}$|visit_[A-Za-z][A-Za-z0-9_]*"
[tool.pylint.similarities]
# 6 lines: CodeItem and ItemData share id/file/type/name/start_line/end_line fields
# intentionally (mirror the same domain schema). Raise the threshold so pylint
# doesn't flag this intentional structural duplication.
min-similarity-lines = 10
[tool.pylint.spelling]
# No spelling dictionary to avoid false positives

View File

@ -6,6 +6,7 @@
add-trailing-comma>=3.1.0
aiohttp>=3.9
anthropic>=0.25.0
autoflake>=2.2.0
autopep8>=2.0.0
bandit>=1.7.0
@ -70,9 +71,11 @@ pyupgrade>=3.15.0
radon>=6.0.0
reorder-python-imports>=3.12.0
requests>=2.0
rich>=13.0
ruff>=0.8.0
safety>=2.3.0
selenium>=4.0
typer>=0.12.0
types-python-dateutil>=2.8.0
types-PyYAML>=6.0.0
types-requests>=2.31.0

View File

@ -0,0 +1,6 @@
"""Code Tutor: Socratic codebase understanding tool.
Asks you to explain code before offering any hints — the Socratic ordering
that prevents parroting. Use ``code_tutor analyze`` to build a study plan,
then ``code_tutor study`` to run interactive lessons.
"""

View File

@ -0,0 +1,388 @@
"""Language-agnostic code item extraction from source files.
Supports Python via the ``ast`` module (exact line ranges) and other languages
via a regex heuristic. Binary files and vendored directories are skipped.
"""
from __future__ import annotations
import ast
from dataclasses import dataclass, field
import hashlib
from pathlib import Path
import re
_SKIP_DIRS: frozenset[str] = frozenset(
{
"third_party",
".venv",
"node_modules",
"__pycache__",
".git",
"dist",
"build",
}
)
_SKIP_SUFFIXES: frozenset[str] = frozenset(
{
".geojson",
".png",
".jpg",
".jpeg",
".gif",
".ico",
".svg",
".webp",
".pdf",
".zip",
".tar",
".gz",
".bin",
}
)
_OTHER_LANGS: frozenset[str] = frozenset(
{".js", ".ts", ".go", ".rs", ".c", ".cpp", ".dart"}
)
# Matches leading keyword(s) followed by an identifier and opening paren.
_FUNCTION_RE: re.Pattern[str] = re.compile(
r"^[ \t]*(?:(?:pub(?:lic)?|priv(?:ate)?|prot(?:ected)?|static|async)\s+)*"
r"(?:def|function|fn|func|void|int|float|bool|string)\s+(\w+)\s*\(",
re.MULTILINE,
)
@dataclass
class CodeItem:
"""A single extractable code unit (function or method).
Attributes:
id: Dotted identifier, e.g. ``module.submodule.function_name``.
file: Relative path from the codebase root.
type: ``"function"`` or ``"async_function"``.
name: Bare function or method name.
start_line: 1-based first line of the definition.
end_line: 1-based last line of the definition.
class_name: Enclosing class name, or ``""`` for module-level functions.
depends_on: IDs of items that must be understood first.
"""
id: str
file: str
type: str
name: str
start_line: int
end_line: int
class_name: str = ""
depends_on: list[str] = field(default_factory=list)
def _is_binary(path: Path) -> bool:
"""Return True if *path* appears to be a binary file (null-byte heuristic).
Uses the same heuristic as ``git`` and GNU ``grep``: a file is binary if
its first 512 bytes contain a null byte. Raises ``OSError`` on read
failure so callers can handle unreadable files separately.
Args:
path: File to inspect.
Returns:
True when the first 512 bytes contain a null byte; False otherwise.
Raises:
OSError: When the file cannot be read.
"""
chunk = path.read_bytes()[:512]
return b"\x00" in chunk
def _should_skip(rel: str) -> bool:
"""Return True if this relative path should be excluded from analysis.
Args:
rel: Path relative to the codebase root.
Returns:
True when any path component is in ``_SKIP_DIRS`` or the file
extension is in ``_SKIP_SUFFIXES``.
"""
parts = Path(rel).parts
if any(p in _SKIP_DIRS for p in parts):
return True
return Path(rel).suffix.lower() in _SKIP_SUFFIXES
def _make_id(rel_path: str, name: str) -> str:
"""Build a dotted item identifier from a relative path and function name.
Args:
rel_path: Path relative to the codebase root.
name: Function or method name.
Returns:
Dotted identifier like ``module.submodule.function_name``.
"""
without_ext = Path(rel_path).with_suffix("")
return ".".join(without_ext.parts) + "." + name
class _FunctionVisitor(ast.NodeVisitor):
"""AST visitor that extracts function/method items with class context.
Args:
rel_path: Relative path of the file being visited.
"""
def __init__(self, rel_path: str) -> None:
"""Initialise with file path and empty state."""
self._rel_path = rel_path
self._class_stack: list[str] = []
self.items: list[CodeItem] = []
def visit_ClassDef(self, node: ast.ClassDef) -> None:
"""Push class name onto stack, recurse, then pop."""
self._class_stack.append(node.name)
self.generic_visit(node)
self._class_stack.pop()
def _add(self, node: ast.FunctionDef | ast.AsyncFunctionDef, kind: str) -> None:
end = getattr(node, "end_lineno", node.lineno)
self.items.append(
CodeItem(
id=_make_id(self._rel_path, node.name),
file=self._rel_path,
type=kind,
name=node.name,
start_line=node.lineno,
end_line=end,
class_name=self._class_stack[-1] if self._class_stack else "",
)
)
def visit_FunctionDef(self, node: ast.FunctionDef) -> None:
"""Record function and recurse into body."""
self._add(node, "function")
self.generic_visit(node)
def visit_AsyncFunctionDef(self, node: ast.AsyncFunctionDef) -> None:
"""Record async function and recurse into body."""
self._add(node, "async_function")
self.generic_visit(node)
def _extract_python(path: Path, rel_path: str) -> list[CodeItem]:
"""Extract function/method items from a Python file using the AST.
Args:
path: Absolute path to the Python file.
rel_path: Relative path used to build item IDs.
Returns:
List of ``CodeItem`` instances, one per function definition found.
Returns an empty list on ``SyntaxError``.
"""
try:
source = path.read_text(encoding="utf-8", errors="replace")
tree = ast.parse(source)
except SyntaxError:
return []
visitor = _FunctionVisitor(rel_path)
visitor.visit(tree)
return visitor.items
def _extract_other(path: Path, rel_path: str) -> list[CodeItem]:
"""Extract function-like items from non-Python files via regex.
Scans each line for a function-definition keyword followed by an
identifier and opening parenthesis. End line is estimated at
``start + 40`` (clamped to file length).
Args:
path: Absolute path to the source file.
rel_path: Relative path used to build item IDs.
Returns:
List of ``CodeItem`` instances. Returns an empty list on ``OSError``.
"""
try:
lines = path.read_text(encoding="utf-8", errors="replace").splitlines()
except OSError:
return []
items: list[CodeItem] = []
for lineno, line in enumerate(lines, start=1):
match = _FUNCTION_RE.match(line)
if match:
name = match.group(1)
end = min(lineno + 40, len(lines))
items.append(
CodeItem(
id=_make_id(rel_path, name),
file=rel_path,
type="function",
name=name,
start_line=lineno,
end_line=end,
)
)
return items
def extract_items(codebase: Path) -> list[CodeItem]:
"""Walk *codebase* and return all extractable code items.
Args:
codebase: Root directory to analyse.
Returns:
Flat list of ``CodeItem`` instances, unsorted.
"""
items: list[CodeItem] = []
for path in sorted(codebase.rglob("*")):
if not path.is_file():
continue
rel = str(path.relative_to(codebase))
if _should_skip(rel):
continue
try:
binary = _is_binary(path)
except OSError:
continue
if binary:
continue
if path.suffix == ".py":
items.extend(_extract_python(path, rel))
elif path.suffix in _OTHER_LANGS:
items.extend(_extract_other(path, rel))
return items
def get_python_files(codebase: Path) -> list[Path]:
"""Return all non-skipped Python files under *codebase*.
Args:
codebase: Root directory to search.
Returns:
Sorted list of absolute Python file paths.
"""
return [
path
for path in sorted(codebase.rglob("*.py"))
if path.is_file() and not _should_skip(str(path.relative_to(codebase)))
]
def _build_dotted_map(codebase: Path, all_files: list[Path]) -> dict[Path, str]:
"""Build a mapping from file to its dotted module path relative to *codebase*.
Args:
codebase: Root directory used for relative path computation.
all_files: All Python files within the codebase.
Returns:
Dict mapping each file in *all_files* to its dotted module string.
Files that cannot be made relative to *codebase* are excluded.
"""
result: dict[Path, str] = {}
for f in all_files:
try:
rel = f.relative_to(codebase)
except ValueError:
continue
result[f] = ".".join(rel.with_suffix("").parts)
return result
def _match_deps(
imported: set[str],
dotted_map: dict[Path, str],
path: Path,
) -> set[Path]:
"""Return the subset of *dotted_map* files that *path* appears to import.
A file ``f`` is included when any imported module name equals, contains,
or is contained by ``f``'s dotted path.
Args:
imported: Set of module name strings collected from import statements.
dotted_map: Mapping of file to dotted module path.
path: The file being analysed (excluded from its own dependency set).
Returns:
Set of dependency files.
"""
deps: set[Path] = set()
for imp in imported:
for f, dotted in dotted_map.items():
if f != path and (
imp == dotted
or imp.endswith("." + dotted)
or dotted.endswith("." + imp)
):
deps.add(f)
return deps
def get_file_imports(path: Path, codebase: Path, all_files: list[Path]) -> set[Path]:
"""Parse Python imports and return the codebase files this file depends on.
Uses a suffix-match heuristic: if an imported module name ends with (or
matches) the dotted path of a file in *all_files*, that file is treated as
a dependency.
Args:
path: The Python file to analyse.
codebase: Root directory used for relative path computation.
all_files: All Python files within the codebase.
Returns:
Subset of *all_files* that *path* imports from.
"""
try:
source = path.read_text(encoding="utf-8", errors="replace")
tree = ast.parse(source)
except SyntaxError:
return set()
imported: set[str] = set()
for node in ast.walk(tree):
if isinstance(node, ast.Import):
for alias in node.names:
imported.add(alias.name)
elif isinstance(node, ast.ImportFrom) and node.module:
imported.add(node.module)
dotted_map = _build_dotted_map(codebase, all_files)
return _match_deps(imported, dotted_map, path)
def codebase_fingerprint(codebase: Path) -> str:
"""Compute a short fingerprint of all source files under *codebase*.
Uses each file's mtime and size rather than content, so it is fast
even on large repositories. Returns a 16-character hex string that
changes whenever any tracked file is added, removed, or modified.
Args:
codebase: Root directory of the codebase.
Returns:
16-character hex fingerprint string.
"""
parts: list[str] = []
for path in sorted(codebase.rglob("*")):
if not path.is_file():
continue
rel = str(path.relative_to(codebase))
if _should_skip(rel):
continue
try:
s = path.stat()
parts.append(f"{rel}:{s.st_mtime_ns}:{s.st_size}")
except OSError:
pass
return hashlib.sha256("\n".join(parts).encode()).hexdigest()[:16]

View File

@ -0,0 +1,469 @@
"""Coding challenge: reproduce a function from scratch, validated by tests.
After a PASS verdict the user is offered a challenge: rewrite the function
without seeing its body, validated either by existing tests or by tests they
write themselves.
If existing tests exist for the function -> show them, ask user to implement.
If no tests exist -> ask user to write tests first, rate them, then ask user
to implement.
In both cases the original implementation is hidden during the challenge.
The lower-level discovery / pytest / verdict helpers live in
:mod:`python_pkg.code_tutor._challenge_support`; this module keeps the
interactive validation, rating and flow orchestration plus the public
``run_coding_challenge`` entry point.
"""
from __future__ import annotations
import ast
from pathlib import Path
import tempfile
from typing import TYPE_CHECKING
from rich.panel import Panel
from rich.syntax import Syntax
from python_pkg.code_tutor._challenge_support import (
_collect_lines,
_extract_signature_block,
_find_tests,
_import_hint,
_parse_verdict,
_patch_and_test,
_project_root,
_pytest_clean,
_show_test_panels,
_stream_verdict,
)
if TYPE_CHECKING:
from collections.abc import Callable
from rich.console import Console
from python_pkg.code_tutor._analyzer import CodeItem
from python_pkg.code_tutor._llm import Backend
_NO_AI_NOTICE = (
"[bold yellow]⚠ No AI assistance -- write this yourself.[/bold yellow]\n"
"Your own explanation (and the tests below) are your only reference."
)
_MAX_TEST_ATTEMPTS = 2
_TEST_JUDGE_SYSTEM = (
"You are a code tutor evaluating whether a student's tests adequately"
" cover a function.\n\n"
"PASS if the tests:\n"
" - Include at least 2 meaningfully different test cases with distinct inputs\n"
" - Use assertions that verify the actual return value or observable behavior\n"
" - Would catch an obviously wrong implementation (e.g. one that always"
' returns "")\n\n'
"FAIL only if:\n"
" - There is only one trivial test case\n"
" - Assertions are always-true or don't check the function's real output\n"
" - All tests are essentially the same scenario with trivially different data\n\n"
"Do NOT require error/edge-case tests for pure transformation functions that"
" have no error handling. 2+ meaningful happy-path cases with distinct"
" inputs is enough.\n\n"
"Respond with valid JSON only, no other text:\n"
'{"verdict": "PASS" | "FAIL",'
' "gap": "<one sentence on the specific missing scenario,'
' or empty string on PASS>"}'
)
def _validate_tests_against_real(
test_code: str,
import_header: str,
project_root: Path,
console: Console,
) -> bool:
"""Write *test_code* to a temp file and run it against the real implementation.
If the tests fail on the real (correct) implementation, they are wrong.
Args:
test_code: The user's pytest test code.
import_header: Auto-generated import line prepended to the test file.
project_root: Project root used for PYTHONPATH and cwd.
console: Rich console for output.
Returns:
``True`` when all tests pass on the real implementation.
"""
full_code = import_header + "\n\n" + test_code
try:
ast.parse(full_code)
except SyntaxError as exc:
console.print(f"[red]Syntax error in your tests: {exc}[/red]")
return False
with tempfile.NamedTemporaryFile(
mode="w",
suffix="_ct_test.py",
dir=None,
delete=False,
encoding="utf-8",
) as fh:
fh.write(full_code)
temp_path = Path(fh.name)
try:
return _pytest_clean(
[str(temp_path)],
project_root,
console,
extra_env={"PYTHONPATH": str(project_root)},
)
finally:
temp_path.unlink(missing_ok=True)
def _collect_and_rate_tests(
sig_block: str,
user_explanation: str,
backend: Backend,
console: Console,
input_fn: Callable[[str], str],
) -> str | None:
"""Prompt the user to write tests and rate them up to *_MAX_TEST_ATTEMPTS* times.
Args:
sig_block: Function signature + docstring shown as context.
user_explanation: Student's earlier explanation (shown as reference).
backend: LLM backend for test quality rating.
console: Rich console for output.
input_fn: Callable for reading user input.
Returns:
The approved test code string, or ``None`` if the user skipped.
"""
for attempt in range(1, _MAX_TEST_ATTEMPTS + 1):
collected = _collect_lines(
f"\n[bold]Write your tests (attempt {attempt}/{_MAX_TEST_ATTEMPTS}).[/bold]"
" Finish with [dim]END[/dim] on a blank line,"
" or [dim]skip[/dim] to exit.",
console,
input_fn,
)
if collected is None:
console.print("[yellow]Challenge skipped.[/yellow]")
return None
sig_and_exp = (
f"Function contract:\n{sig_block}"
f"\n\nStudent explanation:\n{user_explanation}"
)
raw = _stream_verdict(
_TEST_JUDGE_SYSTEM,
f"{sig_and_exp}\n\nStudent's tests:\n{collected}",
backend,
console,
label="Rating tests",
)
verdict, gap = _parse_verdict(raw)
if verdict == "PASS":
return collected
console.print(f"[red]Tests need improvement[/red] -- {gap}")
if attempt == _MAX_TEST_ATTEMPTS:
console.print(
f"[yellow]Skipping challenge after"
f" {_MAX_TEST_ATTEMPTS} attempts.[/yellow]"
)
return None
console.print("Try again with better coverage:\n")
return None # pragma: no cover - loop always returns on the final attempt
def _run_user_impl(
item: CodeItem,
codebase_path: str,
test_code: str,
import_hint: str,
console: Console,
input_fn: Callable[[str], str],
) -> str:
"""Prompt user to write an implementation and run it against *test_code*.
Patches the source file, runs a temp test file, then always restores.
Args:
item: Code item to implement.
codebase_path: Absolute codebase root path.
test_code: Approved test code (without import header).
import_hint: Auto-generated import line prepended to the test file.
console: Rich console for output.
input_fn: Callable for reading user input.
Returns:
``"passed"``, ``"failed"``, or ``"skipped"``.
"""
project_root = _project_root(Path(codebase_path))
sig_block = _extract_signature_block(item, codebase_path)
console.print(
Panel(
Syntax(sig_block, "python", theme="monokai"),
title="[blue]Function signature (implement this)[/blue]",
border_style="blue",
)
)
user_impl = _collect_lines(
f"\n[bold]Now write the implementation of [cyan]{item.name}[/cyan].[/bold]"
" Finish with [dim]END[/dim] on a blank line, or [dim]skip[/dim] to exit.",
console,
input_fn,
)
if user_impl is None:
console.print("[yellow]Challenge skipped.[/yellow]")
return "skipped"
try:
ast.parse(user_impl)
except SyntaxError as exc:
console.print(f"[red]Syntax error in your implementation: {exc}[/red]")
return "failed"
full_test_code = import_hint + "\n\n" + test_code
with tempfile.NamedTemporaryFile(
mode="w", suffix="_ct_test.py", delete=False, encoding="utf-8"
) as fh:
fh.write(full_test_code)
temp_path = Path(fh.name)
source_file = Path(codebase_path) / item.file
original = source_file.read_text(encoding="utf-8")
orig_lines = original.splitlines()
before = orig_lines[: item.start_line - 1]
after = orig_lines[item.end_line :]
new_source = "\n".join(before + user_impl.splitlines() + after) + "\n"
try:
source_file.write_text(new_source, encoding="utf-8")
passed = _pytest_clean(
[str(temp_path)],
project_root,
console,
extra_env={"PYTHONPATH": str(project_root)},
)
finally:
source_file.write_text(original, encoding="utf-8")
temp_path.unlink(missing_ok=True)
if passed:
console.print(
"[green bold]✓ All your tests passed"
" -- you can write it and test it![/green bold]"
)
return "passed"
console.print("[red]✗ Your implementation didn't pass your own tests.[/red]")
return "failed"
# ---------------------------------------------------------------------------
# "Write tests first" flow -- orchestrator
# ---------------------------------------------------------------------------
def _write_tests_first_flow(
item: CodeItem,
codebase_path: str,
user_explanation: str,
backend: Backend,
console: Console,
input_fn: Callable[[str], str],
) -> str:
"""Challenge flow used when no existing tests are found.
Asks the user to write tests first, rates them with the LLM, validates
them against the real implementation, then asks the user to write their
own implementation.
Args:
item: Code item being challenged.
codebase_path: Absolute codebase root path.
user_explanation: User's earlier explanation, shown as reference.
backend: LLM backend for test quality rating.
console: Rich console for output.
input_fn: Callable for reading user input.
Returns:
``"passed"``, ``"failed"``, or ``"skipped"``.
"""
project_root = _project_root(Path(codebase_path))
import_hint = _import_hint(item, codebase_path, project_root)
sig_block = _extract_signature_block(item, codebase_path)
console.print(
"\n[bold cyan]Coding challenge (tests first):[/bold cyan] "
"No existing tests found -- write them before implementing.\n"
)
answer = input_fn("Take the challenge? [y/N] ").strip().lower()
if answer != "y":
return "skipped"
console.print(
Panel(
user_explanation,
title="Your explanation (your only reference)",
border_style="dim",
)
)
console.print(_NO_AI_NOTICE)
console.print(
Panel(
Syntax(sig_block, "python", theme="monokai"),
title="[blue]Function signature + contract[/blue]",
border_style="blue",
)
)
console.print(
f"\n[dim]Auto-import that will be prepended to your tests:[/dim]\n"
f"[cyan]{import_hint}[/cyan]\n"
"[dim]Add any other imports (pytest, MagicMock, etc.) yourself.[/dim]\n"
)
test_code = _collect_and_rate_tests(
sig_block, user_explanation, backend, console, input_fn
)
if test_code is None:
return "skipped"
console.print(
"\n[dim]Validating your tests against the real implementation...[/dim]"
)
if not _validate_tests_against_real(test_code, import_hint, project_root, console):
console.print(
"[red]Your tests fail on the correct implementation -- "
"they may be testing the wrong behavior. Skipping.[/red]"
)
return "skipped"
console.print(
"[green]Tests look good -- they pass on the real implementation.[/green]"
)
return _run_user_impl(
item, codebase_path, test_code, import_hint, console, input_fn
)
# ---------------------------------------------------------------------------
# "Existing tests" flow
# ---------------------------------------------------------------------------
def _existing_tests_flow(
item: CodeItem,
codebase_path: str,
user_explanation: str,
test_entries: list[tuple[Path, list[str]]],
console: Console,
input_fn: Callable[[str], str],
) -> str:
"""Challenge flow used when existing tests are found.
Shows the tests, asks user to implement from scratch.
Args:
item: Code item being challenged.
codebase_path: Absolute codebase root path.
user_explanation: User's earlier explanation.
test_entries: Existing test file/node-id pairs.
console: Rich console for output.
input_fn: Callable for reading user input.
Returns:
``"passed"``, ``"failed"``, or ``"skipped"``.
"""
n_tests = sum(len(ids) for _, ids in test_entries)
console.print(
"\n[bold cyan]Coding challenge:[/bold cyan] "
f"Can you rewrite this function from scratch? "
f"[dim]({n_tests} test(s) will validate it)[/dim]"
)
answer = input_fn("Take the challenge? [y/N] ").strip().lower()
if answer != "y":
return "skipped"
console.print(
Panel(
user_explanation,
title="Your explanation (your only reference)",
border_style="dim",
)
)
console.print(_NO_AI_NOTICE)
_show_test_panels(test_entries, console)
user_code = _collect_lines(
f"\n[bold]Write [cyan]{item.name}[/cyan] from scratch.[/bold] "
"Finish with [dim]END[/dim] on a blank line, or [dim]skip[/dim] to skip.",
console,
input_fn,
)
if user_code is None:
console.print("[yellow]Challenge skipped.[/yellow]")
return "skipped"
console.print("\n[dim]Running tests against your implementation...[/dim]")
passed = _patch_and_test(item, codebase_path, user_code, test_entries, console)
if passed:
console.print(
"[green bold]✓ All tests passed -- you really understand it![/green bold]"
)
return "passed"
console.print("[red]✗ Some tests failed -- try again next session.[/red]")
return "failed"
# ---------------------------------------------------------------------------
# Public entry point
# ---------------------------------------------------------------------------
def run_coding_challenge(
item: CodeItem,
codebase_path: str,
user_explanation: str,
backend: Backend,
console: Console,
input_fn: Callable[[str], str] = input,
) -> str:
"""Offer the user a coding challenge after a PASS verdict.
Routes to one of two flows:
- **Existing tests found**: show tests, user writes implementation.
- **No tests found**: user writes tests first (LLM-rated), then implementation.
Only offered for ``.py`` files.
Args:
item: The code item to challenge on.
codebase_path: Absolute path of the codebase root.
user_explanation: The explanation the user gave during the lesson.
backend: LLM backend (used for test quality rating in the no-tests flow).
console: Rich console for output.
input_fn: Callable for reading user input.
Returns:
``"passed"``, ``"failed"``, or ``"skipped"``.
"""
if not item.file.endswith(".py"):
return "skipped"
codebase = Path(codebase_path)
test_entries = _find_tests(item, codebase)
if test_entries:
return _existing_tests_flow(
item, codebase_path, user_explanation, test_entries, console, input_fn
)
return _write_tests_first_flow(
item, codebase_path, user_explanation, backend, console, input_fn
)

View File

@ -0,0 +1,402 @@
"""Lower-level helpers shared by the coding-challenge flows.
This module holds the test-discovery, pytest-execution and LLM
verdict/signature helpers used by :mod:`python_pkg.code_tutor._challenge`.
Splitting them out keeps each module comfortably under the repo's
500-line-per-file limit while preserving a clean, single-responsibility seam:
everything here is a stateless helper with no knowledge of the interactive
challenge flows that consume it.
"""
from __future__ import annotations
import ast
import json
import os
from pathlib import Path
import re
import subprocess
import sys
import time
from typing import TYPE_CHECKING
from rich.live import Live
from rich.panel import Panel
from rich.syntax import Syntax
from rich.text import Text
if TYPE_CHECKING:
from collections.abc import Callable
from rich.console import Console
from python_pkg.code_tutor._analyzer import CodeItem
from python_pkg.code_tutor._llm import Backend
def _scan_test_file(source: str, item_name: str) -> list[str]:
"""Return pytest node IDs that reference *item_name*.
Handles both module-level functions and class methods. Class IDs use the
``ClassName::test_name`` format expected by pytest.
Args:
source: Full text of the test file.
item_name: Name of the function under test.
Returns:
List of qualified pytest node IDs.
"""
try:
tree = ast.parse(source)
except SyntaxError:
return []
src_lines = source.splitlines()
matching: list[str] = []
def _body_has(node: ast.FunctionDef | ast.AsyncFunctionDef) -> bool:
end = getattr(node, "end_lineno", node.lineno)
return item_name in "\n".join(src_lines[node.lineno - 1 : end])
for top in tree.body:
if isinstance(top, (ast.FunctionDef, ast.AsyncFunctionDef)):
if top.name.startswith("test_") and _body_has(top):
matching.append(top.name)
elif isinstance(top, ast.ClassDef):
matching.extend(
f"{top.name}::{method.name}"
for method in top.body
if (
isinstance(method, (ast.FunctionDef, ast.AsyncFunctionDef))
and method.name.startswith("test_")
and _body_has(method)
)
)
return matching
def _find_tests(item: CodeItem, codebase: Path) -> list[tuple[Path, list[str]]]:
"""Find existing test node IDs that reference *item.name* in *codebase*.
Args:
item: The code item being challenged.
codebase: Root directory of the codebase to search.
Returns:
List of ``(test_file, [node_ids])`` pairs.
"""
candidates = list(codebase.rglob("test_*.py")) + list(codebase.rglob("*_test.py"))
results: list[tuple[Path, list[str]]] = []
for test_file in sorted(set(candidates)):
try:
source = test_file.read_text(encoding="utf-8", errors="replace")
except OSError:
continue
if item.name not in source:
continue
node_ids = _scan_test_file(source, item.name)
if node_ids:
results.append((test_file, node_ids))
return results
def _show_test_panels(
test_entries: list[tuple[Path, list[str]]],
console: Console,
) -> None:
"""Display each test function in a syntax-highlighted panel.
Args:
test_entries: List of ``(test_file, node_ids)`` pairs.
console: Rich console for output.
"""
for test_file, node_ids in test_entries:
try:
source = test_file.read_text(encoding="utf-8", errors="replace")
tree = ast.parse(source)
except (OSError, SyntaxError):
continue
src_lines = source.splitlines()
func_names = {nid.split("::")[-1] for nid in node_ids}
for top in tree.body:
nodes: list[tuple[str, ast.FunctionDef | ast.AsyncFunctionDef]] = []
if isinstance(top, (ast.FunctionDef, ast.AsyncFunctionDef)):
nodes = [(top.name, top)]
elif isinstance(top, ast.ClassDef):
for m in top.body:
if isinstance(m, (ast.FunctionDef, ast.AsyncFunctionDef)):
nodes.append((m.name, m))
for name, node in nodes:
if name not in func_names:
continue
end = getattr(node, "end_lineno", node.lineno)
snippet = "\n".join(src_lines[node.lineno - 1 : end])
console.print(
Panel(
Syntax(snippet, "python", theme="monokai", line_numbers=True),
title=f"[blue]Test: {name}[/blue]",
border_style="blue",
)
)
def _collect_lines(
prompt: str,
console: Console,
input_fn: Callable[[str], str],
) -> str | None:
"""Collect multi-line input until the user types END or skip.
Args:
prompt: Message to display before input begins.
console: Rich console for output.
input_fn: Callable for reading user input.
Returns:
The collected text as a single string, or ``None`` if skipped.
"""
console.print(prompt)
lines: list[str] = []
while True:
line = input_fn("")
if line.strip().lower() == "skip":
return None
if line.strip() == "END":
break
lines.append(line)
return "\n".join(lines)
def _project_root(start: Path) -> Path:
"""Walk up from *start* to find the nearest ``pyproject.toml`` / ``setup.py``.
Args:
start: Directory to begin the search from.
Returns:
Project root, or *start* if none found.
"""
current = start.resolve()
while current != current.parent:
if (current / "pyproject.toml").exists() or (current / "setup.py").exists():
return current
current = current.parent
return start.resolve()
def _pytest_clean(
test_ids_or_file: list[str],
project_root: Path,
console: Console,
extra_env: dict[str, str] | None = None,
) -> bool:
"""Run pytest on *test_ids_or_file* and return True if all pass.
Coverage is disabled so the project's ``fail_under`` threshold does not
interfere with isolated challenge runs.
Args:
test_ids_or_file: Pytest node IDs or a single temp-file path.
project_root: Working directory for the pytest subprocess.
console: Rich console for output.
extra_env: Extra environment variables (e.g. PYTHONPATH override).
Returns:
``True`` when all collected tests pass.
"""
env = {**os.environ, **(extra_env or {})}
result = subprocess.run(
[
sys.executable,
"-m",
"pytest",
*test_ids_or_file,
"-v",
"--tb=short",
"--no-header",
"-p",
"no:cov",
"--override-ini=addopts=",
],
capture_output=True,
text=True,
cwd=str(project_root),
env=env,
check=False,
)
output = (result.stdout + result.stderr).strip()
if output:
console.print(output)
return result.returncode == 0
def _patch_and_test(
item: CodeItem,
codebase_path: str,
user_code: str,
test_entries: list[tuple[Path, list[str]]],
console: Console,
) -> bool:
"""Replace *item* in source with *user_code*, run tests, then restore.
Args:
item: Code item to patch.
codebase_path: Absolute codebase root path.
user_code: User's implementation text.
test_entries: ``(test_file, node_ids)`` pairs to run.
console: Rich console for output.
Returns:
``True`` if all tests passed.
"""
source_file = Path(codebase_path) / item.file
original = source_file.read_text(encoding="utf-8")
orig_lines = original.splitlines()
before = orig_lines[: item.start_line - 1]
after = orig_lines[item.end_line :]
new_source = "\n".join(before + user_code.splitlines() + after) + "\n"
try:
ast.parse(new_source)
except SyntaxError as exc:
console.print(f"[red]Syntax error in your implementation: {exc}[/red]")
return False
test_ids = [f"{tf}::{nid}" for tf, nids in test_entries for nid in nids]
project_root = _project_root(Path(codebase_path))
try:
source_file.write_text(new_source, encoding="utf-8")
return _pytest_clean(test_ids, project_root, console)
finally:
source_file.write_text(original, encoding="utf-8")
def _stream_verdict(
system: str,
user_msg: str,
backend: Backend,
console: Console,
label: str = "Rating",
) -> str:
"""Stream an LLM call silently while showing an elapsed timer.
Args:
system: System prompt for the judge.
user_msg: User message to judge.
backend: LLM backend with a ``stream(system, user, callback)`` method.
console: Rich console for the live timer.
label: Label prefix shown in the timer (e.g. ``"Rating"``).
Returns:
Accumulated response text.
"""
parts: list[str] = []
start = time.monotonic()
def _on_token(token: str) -> None:
parts.append(token)
elapsed = int(time.monotonic() - start)
live.update(Text(f"{label}... {elapsed}s", style="yellow"))
with Live(
Text(f"{label}... 0s", style="yellow"),
console=console,
refresh_per_second=4,
transient=True,
) as live:
backend.stream(system, user_msg, _on_token)
return "".join(parts)
def _parse_verdict(raw: str) -> tuple[str, str]:
"""Parse ``{"verdict": ..., "gap": ...}`` JSON from *raw*, tolerating fences.
Args:
raw: Raw LLM response text.
Returns:
``(verdict, gap)`` where verdict is ``"PASS"`` or ``"FAIL"``.
"""
clean = re.sub(r"```(?:json)?\s*", "", raw, flags=re.DOTALL).strip()
start = clean.find("{")
end = clean.rfind("}") + 1
if start == -1 or end == 0:
return "FAIL", "Could not parse judge response."
try:
data = json.loads(clean[start:end])
except json.JSONDecodeError:
return "FAIL", "Could not parse judge response."
verdict = str(data.get("verdict", "FAIL")).upper()
if verdict not in {"PASS", "FAIL"}:
verdict = "FAIL"
return verdict, str(data.get("gap", ""))
# ---------------------------------------------------------------------------
# "Write tests first" flow -- helpers
# ---------------------------------------------------------------------------
def _extract_signature_block(item: CodeItem, codebase_path: str) -> str:
"""Return the function signature plus docstring, without the body.
Gives the user the contract of the function so they can write tests
without seeing the implementation.
Args:
item: Code item to extract from.
codebase_path: Absolute codebase root path.
Returns:
Multi-line string ending after the closing ``\"\"\"`` of the docstring,
or just the ``def`` line when no docstring is present.
"""
path = Path(codebase_path) / item.file
try:
source = path.read_text(encoding="utf-8", errors="replace")
tree = ast.parse(source)
except (OSError, SyntaxError):
return f"def {item.name}(...):"
lines = source.splitlines()
for node in ast.walk(tree):
if not isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
continue
if node.name != item.name:
continue
sig_end = node.lineno
if (
node.body
and isinstance(node.body[0], ast.Expr)
and isinstance(node.body[0].value, ast.Constant)
and isinstance(node.body[0].value.value, str)
):
sig_end = getattr(node.body[0], "end_lineno", node.lineno + 1)
return "\n".join(lines[node.lineno - 1 : sig_end])
return f"def {item.name}(...):"
def _import_hint(item: CodeItem, codebase_path: str, project_root: Path) -> str:
"""Build the Python import statement for *item*.
Args:
item: Code item to import.
codebase_path: Absolute codebase root path.
project_root: Project root (where pyproject.toml lives).
Returns:
A ``from <module> import <name>`` string.
"""
abs_file = (Path(codebase_path) / item.file).resolve()
try:
rel = abs_file.relative_to(project_root).with_suffix("")
module = ".".join(rel.parts)
except ValueError:
module = Path(item.file).stem
return f"from {module} import {item.name}"

View File

@ -0,0 +1,295 @@
"""LLM backend abstraction: Ollama (primary) and Claude (fallback).
Backends implement the ``Backend`` protocol. The ``complete_with_fallback``
helper calls Ollama, detects weak responses, and offers to switch to Claude.
"""
from __future__ import annotations
import json
from typing import TYPE_CHECKING, Protocol
import anthropic
import requests
if TYPE_CHECKING:
from collections.abc import Callable
from rich.console import Console
_OLLAMA_URL = "http://localhost:11434/api/generate"
_OLLAMA_MODEL = "qwen3-32b-dev:latest"
_OLLAMA_TIMEOUT = 120
_WEAK_PHRASES: frozenset[str] = frozenset(
{
"i don't know",
"i cannot",
"unclear",
"i'm not sure",
}
)
_WEAK_MIN_LEN = 30
class Backend(Protocol):
"""Structural protocol for an LLM backend."""
def complete(self, system: str, user: str) -> str:
"""Send *system* + *user* prompts and return the model's response.
Args:
system: System prompt (role / instructions).
user: User message (the actual query).
Returns:
Raw response text from the model.
"""
... # pragma: no cover
def stream(
self,
system: str,
user: str,
on_token: Callable[[str], None],
) -> str:
"""Stream the response token-by-token via *on_token*, return full text.
Args:
system: System prompt.
user: User message.
on_token: Called with each text token as it arrives.
Returns:
Full accumulated response text.
"""
... # pragma: no cover
class OllamaBackend:
"""LLM backend that calls the local Ollama HTTP API.
Args:
model: Ollama model tag to use.
url: Base URL of the Ollama API.
timeout: Request timeout in seconds.
"""
def __init__(
self,
model: str = _OLLAMA_MODEL,
url: str = _OLLAMA_URL,
timeout: int = _OLLAMA_TIMEOUT,
) -> None:
"""Initialise with model/URL/timeout settings."""
self._model = model
self._url = url
self._timeout = timeout
def complete(self, system: str, user: str) -> str:
"""Call Ollama and return the response text (non-streaming).
Args:
system: System prompt.
user: User message / prompt.
Returns:
Response string from the model.
Raises:
requests.exceptions.Timeout: When the request exceeds *timeout*.
requests.exceptions.ConnectionError: When Ollama is unreachable.
requests.exceptions.HTTPError: On non-2xx HTTP responses.
"""
payload = {
"model": self._model,
"system": system,
"prompt": user,
"stream": False,
}
response = requests.post(self._url, json=payload, timeout=self._timeout)
response.raise_for_status()
data = response.json()
return str(data.get("response", ""))
def stream(
self,
system: str,
user: str,
on_token: Callable[[str], None],
) -> str:
"""Call Ollama with streaming enabled, invoking *on_token* per token.
Args:
system: System prompt.
user: User message.
on_token: Callback invoked with each response token as it arrives.
Returns:
Full accumulated response text.
Raises:
requests.exceptions.Timeout: When the request exceeds *timeout*.
requests.exceptions.ConnectionError: When Ollama is unreachable.
requests.exceptions.HTTPError: On non-2xx HTTP responses.
"""
payload = {
"model": self._model,
"system": system,
"prompt": user,
"stream": True,
}
response = requests.post(
self._url, json=payload, timeout=self._timeout, stream=True
)
response.raise_for_status()
parts: list[str] = []
for raw_line in response.iter_lines():
if not raw_line:
continue
data = json.loads(raw_line)
token = str(data.get("response", ""))
if token:
on_token(token)
parts.append(token)
if data.get("done"):
break
return "".join(parts)
class ClaudeBackend:
"""LLM backend that calls the Anthropic Claude API.
Reads ``ANTHROPIC_API_KEY`` from the environment automatically.
Args:
model: Claude model identifier.
max_tokens: Maximum tokens in the response.
"""
def __init__(
self,
model: str = "claude-sonnet-4-6",
max_tokens: int = 1024,
) -> None:
"""Initialise Anthropic client."""
self._client = anthropic.Anthropic()
self._model = model
self._max_tokens = max_tokens
def complete(self, system: str, user: str) -> str:
"""Call Claude and return the response text.
Args:
system: System prompt.
user: User message.
Returns:
Response string from Claude.
Raises:
anthropic.APIError: On any Anthropic API failure.
"""
message = self._client.messages.create(
model=self._model,
max_tokens=self._max_tokens,
system=system,
messages=[{"role": "user", "content": user}],
)
block = message.content[0]
if hasattr(block, "text"):
return str(block.text)
return ""
def stream(
self,
system: str,
user: str,
on_token: Callable[[str], None],
) -> str:
"""Call Claude with streaming, invoking *on_token* per text delta.
Falls back to ``complete`` on ``APIError``.
Args:
system: System prompt.
user: User message.
on_token: Callback invoked with each text delta as it arrives.
Returns:
Full accumulated response text.
"""
try:
parts: list[str] = []
with self._client.messages.stream(
model=self._model,
max_tokens=self._max_tokens,
system=system,
messages=[{"role": "user", "content": user}],
) as s:
for text in s.text_stream:
on_token(text)
parts.append(text)
return "".join(parts)
except anthropic.APIError:
return self.complete(system, user)
def _is_weak(response: str) -> bool:
"""Return True if *response* looks too short or expresses uncertainty.
Args:
response: Raw text from the LLM.
Returns:
True when the response is shorter than ``_WEAK_MIN_LEN`` characters
or contains one of the ``_WEAK_PHRASES`` (case-insensitive).
"""
if len(response) < _WEAK_MIN_LEN:
return True
lower = response.lower()
return any(phrase in lower for phrase in _WEAK_PHRASES)
def complete_with_fallback(
primary: OllamaBackend,
system: str,
user: str,
*,
console: Console,
input_fn: Callable[[str], str] = input,
) -> str:
"""Call *primary*, detect weakness, and offer a Claude fallback.
If the primary's response is weak, the user is asked whether to switch
to Claude for this item. If they decline (or Claude fails), Ollama is
retried once.
Args:
primary: The Ollama backend to call first.
system: System prompt.
user: User message.
console: Rich console for status messages.
input_fn: Callable used for user input (injectable for testing).
Returns:
Final response string.
"""
response = primary.complete(system, user)
if not _is_weak(response):
return response
console.print(
"[yellow][!] Qwen3's response looks weak. Switch to Claude for this item?\n"
" (This will use your ANTHROPIC_API_KEY) [y/N]:[/yellow]"
)
choice = input_fn("").strip().lower()
if choice == "y":
try:
claude = ClaudeBackend()
return claude.complete(system, user)
except anthropic.APIError:
console.print("[yellow]Claude unavailable, retrying Qwen3...[/yellow]")
return primary.complete(system, user)

View File

@ -0,0 +1,139 @@
"""Study plan construction: dependency ordering and session grouping."""
from __future__ import annotations
import dataclasses
from datetime import datetime, timezone
from graphlib import CycleError, TopologicalSorter
from pathlib import Path
from python_pkg.code_tutor._analyzer import (
CodeItem,
codebase_fingerprint,
get_file_imports,
get_python_files,
)
def _toposort_files(file_deps: dict[Path, set[Path]]) -> list[Path]:
"""Return files in topological order so dependencies come first.
Args:
file_deps: Mapping of file → set of files it depends on.
Returns:
Files ordered for learning (dependencies before dependents).
Cycles are broken by appending remaining nodes in arbitrary order.
"""
str_deps: dict[str, set[str]] = {
str(k): {str(v) for v in vs} for k, vs in file_deps.items()
}
try:
ordered_strs = list(TopologicalSorter(str_deps).static_order())
except CycleError:
ordered_strs = list(str_deps)
return [Path(s) for s in ordered_strs]
def _order_items(items: list[CodeItem], codebase: Path) -> list[CodeItem]:
"""Sort *items* in dependency-first order.
Python files are ordered by import relationships. Other files are
sorted by directory depth (shallowest first) then filename.
Args:
items: Unsorted list of code items.
codebase: Root directory of the codebase.
Returns:
Items in recommended learning order.
"""
if not items:
return []
py_files = get_python_files(codebase)
py_rel_set: set[str] = set()
file_order: dict[str, int] = {}
if py_files:
file_deps: dict[Path, set[Path]] = {
f: get_file_imports(f, codebase, py_files) for f in py_files
}
ordered_files = _toposort_files(file_deps)
for i, f in enumerate(ordered_files):
if f.is_relative_to(codebase):
rel = str(f.relative_to(codebase))
file_order[rel] = i
py_rel_set.add(rel)
def _sort_key(item: CodeItem) -> tuple[int, int, str, int]:
if item.file in py_rel_set:
order = file_order.get(item.file, 9999)
else:
order = len(Path(item.file).parts) * 1000
return (order, len(Path(item.file).parts), item.file, item.start_line)
return sorted(items, key=_sort_key)
def _session_title(items: list[CodeItem]) -> str:
"""Derive a human-readable session title from items' file paths.
Args:
items: Items that will belong to the session.
Returns:
A short title string based on the common directory or first file stem.
"""
if not items:
return "untitled"
dirs = [str(Path(it.file).parent) for it in items]
common = dirs[0]
for d in dirs[1:]:
if d != common:
parent = Path(common).parent
common = "." if parent == Path() or str(parent) == common else str(parent)
if common in {".", ""}:
return Path(items[0].file).stem
return Path(common).name or Path(items[0].file).stem
def build_plan(
codebase: Path,
items: list[CodeItem],
*,
session_size: int = 10,
) -> dict[str, object]:
"""Assemble a learning plan from extracted code items.
Args:
codebase: Root directory of the codebase.
items: All extracted items (sorted internally by dependency order).
session_size: Maximum number of items per session.
Returns:
Dict matching the ``plan.json`` schema ready for JSON serialisation.
"""
ordered = _order_items(items, codebase)
sessions: list[dict[str, object]] = []
for session_idx in range(0, max(len(ordered), 1), session_size):
chunk = ordered[session_idx : session_idx + session_size]
if not chunk:
break
session_id = session_idx // session_size + 1
sessions.append(
{
"id": session_id,
"title": _session_title(chunk),
"items": [dataclasses.asdict(it) for it in chunk],
}
)
return {
"codebase_path": str(codebase.resolve()),
"created_at": datetime.now(tz=timezone.utc).isoformat(timespec="seconds"),
"total_items": len(ordered),
"source_fingerprint": codebase_fingerprint(codebase),
"sessions": sessions,
}

View File

@ -0,0 +1,227 @@
"""Persist and load study plans and progress state.
Storage layout::
~/.config/code_tutor/
└── <sha256 of absolute codebase path>/
├── plan.json # created by ``analyze``
├── progress.json # updated after each lesson
└── sessions/
└── YYYY-MM-DD.jsonl # full lesson records for analysis
"""
from __future__ import annotations
from dataclasses import asdict, dataclass
import datetime
import hashlib
import json
from pathlib import Path
from typing import TypedDict, cast
from python_pkg.code_tutor._analyzer import CodeItem
# ---------------------------------------------------------------------------
# TypedDicts matching the plan.json / progress.json schemas
# ---------------------------------------------------------------------------
class ItemData(TypedDict):
"""Schema for a single code item entry in ``plan.json``."""
id: str
file: str
type: str
name: str
start_line: int
end_line: int
class_name: str
depends_on: list[str]
class SessionData(TypedDict):
"""Schema for a session block in ``plan.json``."""
id: int
title: str
items: list[ItemData]
class PlanData(TypedDict):
"""Top-level schema of ``plan.json``."""
codebase_path: str
created_at: str
total_items: int
source_fingerprint: str
sessions: list[SessionData]
class ProgressData(TypedDict):
"""Schema of ``progress.json`` tracking what the user has studied."""
learned: list[str]
struggled: list[str]
skipped: list[str]
last_session: str
# ---------------------------------------------------------------------------
# Full lesson record — written to sessions/YYYY-MM-DD.jsonl for analysis
# ---------------------------------------------------------------------------
@dataclass
class LessonRecord:
"""Complete transcript of one Socratic lesson.
Stored as a JSON line so the file can be streamed / analysed later.
Attributes:
timestamp: ISO-8601 UTC timestamp of the lesson.
item_id: Dotted ID of the code item studied.
file: Relative path of the source file.
lines: ``"start-end"`` line range string.
snippet: The exact source text shown to the user.
outcome: ``"learned"``, ``"struggled"``, or ``"skipped"``.
answers: Mapping of question label to the user's answer.
improvement: User's free-text code improvement idea (may be empty).
verdict: Final LLM verdict: ``"PASS"``, ``"FAIL"``, or ``"skipped"``.
attempt: Which attempt produced the final verdict (1-3).
"""
timestamp: str
item_id: str
file: str
lines: str
snippet: str
outcome: str
answers: dict[str, str]
improvement: str
verdict: str
attempt: int
challenge_result: str = ""
def append_session_record(codebase: Path, record: LessonRecord) -> None:
"""Append *record* as a JSON line to today's session log for *codebase*.
Creates ``sessions/`` if it does not exist. Each line is a self-contained
JSON object so the file can be streamed with ``jq``.
Args:
codebase: Root directory of the codebase being studied.
record: Lesson record to persist.
"""
sessions_dir = config_dir(codebase) / "sessions"
sessions_dir.mkdir(parents=True, exist_ok=True)
today = datetime.datetime.now(tz=datetime.timezone.utc).date().isoformat()
log_file = sessions_dir / f"{today}.jsonl"
line = json.dumps(asdict(record), ensure_ascii=False)
with log_file.open("a", encoding="utf-8") as fh:
fh.write(line + "\n")
# ---------------------------------------------------------------------------
# Config directory helpers
# ---------------------------------------------------------------------------
def config_dir(codebase: Path) -> Path:
"""Return the config directory for *codebase*.
The directory name is the SHA-256 hex digest of the absolute path string,
so different codebases never collide.
Args:
codebase: Root directory of the codebase being studied.
Returns:
``~/.config/code_tutor/<hex>/`` (not guaranteed to exist).
"""
path_hash = hashlib.sha256(str(codebase.resolve()).encode()).hexdigest()
return Path.home() / ".config" / "code_tutor" / path_hash
def load_plan(codebase: Path) -> PlanData | None:
"""Load the saved plan for *codebase*, or return ``None`` if absent.
Args:
codebase: Root directory of the codebase.
Returns:
Parsed ``PlanData`` dict, or ``None`` when no plan file exists.
"""
plan_file = config_dir(codebase) / "plan.json"
if not plan_file.exists():
return None
return cast("PlanData", json.loads(plan_file.read_text(encoding="utf-8")))
def save_plan(codebase: Path, plan: dict[str, object]) -> None:
"""Persist *plan* to ``plan.json`` for *codebase*.
Creates the config directory if it does not already exist.
Args:
codebase: Root directory of the codebase.
plan: Plan dict (as returned by ``build_plan``).
"""
dest = config_dir(codebase)
dest.mkdir(parents=True, exist_ok=True)
(dest / "plan.json").write_text(json.dumps(plan, indent=2), encoding="utf-8")
def load_progress(codebase: Path) -> ProgressData:
"""Load progress for *codebase*, returning an empty record if none exists.
Args:
codebase: Root directory of the codebase.
Returns:
``ProgressData`` dict with learned/struggled/skipped lists.
"""
progress_file = config_dir(codebase) / "progress.json"
if not progress_file.exists():
return {
"learned": [],
"struggled": [],
"skipped": [],
"last_session": "",
}
return cast("ProgressData", json.loads(progress_file.read_text(encoding="utf-8")))
def save_progress(codebase: Path, progress: ProgressData) -> None:
"""Write *progress* to ``progress.json`` for *codebase*.
Args:
codebase: Root directory of the codebase.
progress: Progress dict to persist.
"""
dest = config_dir(codebase)
dest.mkdir(parents=True, exist_ok=True)
(dest / "progress.json").write_text(
json.dumps(progress, indent=2), encoding="utf-8"
)
def item_from_data(data: ItemData) -> CodeItem:
"""Convert an ``ItemData`` TypedDict to a ``CodeItem`` dataclass.
Args:
data: Dict loaded from ``plan.json``.
Returns:
Corresponding ``CodeItem`` with all fields populated.
"""
return CodeItem(
id=data["id"],
file=data["file"],
type=data["type"],
name=data["name"],
start_line=data["start_line"],
end_line=data["end_line"],
class_name=data["class_name"],
depends_on=data["depends_on"],
)

View File

@ -0,0 +1,133 @@
"""Interactive study session orchestrator.
Thin coordination layer: loads plan + progress, calls the Verifier per item,
saves progress after each lesson, and shows a summary table at the end.
No LLM calls or judgment logic live here.
"""
from __future__ import annotations
from datetime import datetime, timezone
from typing import TYPE_CHECKING
from rich.table import Table
from python_pkg.code_tutor._progress import (
append_session_record,
item_from_data,
load_plan,
load_progress,
save_progress,
)
from python_pkg.code_tutor._verifier import Verifier
if TYPE_CHECKING:
from collections.abc import Callable
from pathlib import Path
from rich.console import Console
from python_pkg.code_tutor._analyzer import CodeItem
from python_pkg.code_tutor._llm import Backend
from python_pkg.code_tutor._progress import PlanData, ProgressData
def _items_from_plan(plan: PlanData) -> list[CodeItem]:
"""Reconstruct ``CodeItem`` objects from a loaded ``PlanData`` dict.
Args:
plan: Plan as loaded from ``plan.json``.
Returns:
Flat list of ``CodeItem`` instances in session order.
"""
return [
item_from_data(data)
for session in plan["sessions"]
for data in session["items"]
]
def _show_summary(
results: dict[str, list[str]],
console: Console,
) -> None:
"""Print a session summary table to *console*.
Args:
results: Mapping of outcome (``"learned"``, ``"struggled"``,
``"skipped"``) to lists of item IDs.
console: Rich console for output.
"""
table = Table(title="Session summary", show_header=True, header_style="bold")
table.add_column("Outcome")
table.add_column("Count", justify="right")
table.add_row("[green]Learned[/green]", str(len(results.get("learned", []))))
table.add_row("[red]Struggled[/red]", str(len(results.get("struggled", []))))
table.add_row("[yellow]Skipped[/yellow]", str(len(results.get("skipped", []))))
console.print(table)
def run_session(
codebase: Path,
backend: Backend,
*,
console: Console,
input_fn: Callable[[str], str] = input,
) -> None:
"""Run an interactive study session for *codebase*.
Loads the saved plan and progress, iterates over unfinished items, calls
the Verifier per item, saves progress after each lesson, and shows a
summary table on completion.
Args:
codebase: Root directory of the codebase being studied.
backend: LLM backend to pass to the Verifier.
console: Rich console for all output.
input_fn: Callable used for user input (injectable for testing).
"""
plan = load_plan(codebase)
if plan is None:
console.print(
"[red]No plan found. "
"Run [bold]code_tutor analyze <path>[/bold] first.[/red]"
)
return
progress: ProgressData = load_progress(codebase)
done_ids: set[str] = (
set(progress["learned"]) | set(progress["struggled"]) | set(progress["skipped"])
)
all_items = _items_from_plan(plan)
pending = [it for it in all_items if it.id not in done_ids]
if not pending:
console.print(
"[green]All items complete! "
"Run [bold]code_tutor status[/bold] to see your progress.[/green]"
)
return
verifier = Verifier(backend, console)
results: dict[str, list[str]] = {"learned": [], "struggled": [], "skipped": []}
codebase_str = str(codebase.resolve())
for item in pending:
record = verifier.run_lesson(item, codebase_str, input_fn=input_fn)
outcome = record.outcome
results[outcome].append(item.id)
append_session_record(codebase, record)
if outcome == "learned":
progress["learned"].append(item.id)
elif outcome == "struggled":
progress["struggled"].append(item.id)
else:
progress["skipped"].append(item.id)
progress["last_session"] = datetime.now(tz=timezone.utc).isoformat(
timespec="seconds"
)
save_progress(codebase, progress)
_show_summary(results, console)

View File

@ -0,0 +1,347 @@
"""Socratic lesson loop: ask-first, then judge, then explain.
The LLM explanation is NEVER shown before the user attempts to explain.
"""
from __future__ import annotations
from datetime import datetime, timezone
import json
from pathlib import Path
import re
import time
from typing import TYPE_CHECKING
from rich.live import Live
from rich.panel import Panel
from rich.text import Text
from python_pkg.code_tutor._challenge import run_coding_challenge
from python_pkg.code_tutor._progress import LessonRecord
if TYPE_CHECKING:
from collections.abc import Callable
from rich.console import Console
from python_pkg.code_tutor._analyzer import CodeItem
from python_pkg.code_tutor._llm import Backend
_MAX_ATTEMPTS = 3
_SYSTEM_PROMPT = (
"You are a fair but rigorous code tutor assessing whether a student"
" genuinely understands a piece of code.\n\n"
"Grade their explanation PASS or FAIL.\n\n"
"PASS when the student demonstrates understanding of:\n"
" - what problem the code solves (purpose)\n"
" - what goes in (inputs/parameters and their roles)\n"
" - what comes out or happens (return value or side effect)\n"
" - any key behavior that would surprise a reader"
" (edge cases, error handling, non-obvious constraints)\n\n"
"You do NOT require verbatim phrasing from the code or docstring.\n"
"Correct paraphrasing counts as understanding.\n"
"ONLY cite a gap if it is genuinely absent from the student's answer"
" -- do NOT cite a gap the student already covered in different words.\n\n"
"FAIL only if the student is vague, factually wrong, or missed a key"
" behavioral detail that materially changes how the code is used.\n"
'"It does some stuff" = FAIL. "It processes data" = FAIL.'
' But "it reads the first 512 bytes and checks for a null byte" = PASS'
" on the behavior criterion, even if they didn't mention OSError.\n\n"
"Respond with valid JSON only, no other text:\n"
'{"verdict": "PASS" | "FAIL",'
' "gap": "<one sentence on the specific missing piece, or empty string on PASS>"}'
)
_EXPLAIN_SYSTEM = (
"You are a code tutor."
" Explain the following code clearly and concisely in 3-5 sentences."
)
_QUESTIONS: tuple[tuple[str, str], ...] = (
("Purpose", "What does this code do?"),
("Inputs", "What are the inputs (parameters / arguments)?"),
("Outputs", "What does it output or do as a side effect?"),
("Why", "Why does it exist? What problem does it solve?"),
)
_IMPROVEMENT_Q = "What would you improve or simplify here? (Enter to skip)"
def _class_header(lines: list[str], class_name: str, before_line: int) -> str:
"""Find the class definition for *class_name* and return a short header.
Searches backward from *before_line* for ``class <class_name>``. Returns
the class signature plus up to 4 following lines (docstring / key attrs).
Args:
lines: All source lines of the file (0-indexed).
class_name: Name of the enclosing class.
before_line: 1-based line number of the method; search stops here.
Returns:
A short string summary of the class, or ``""`` if not found.
"""
pattern = re.compile(rf"^class\s+{re.escape(class_name)}\b")
for i in range(min(before_line - 1, len(lines)) - 1, -1, -1):
if pattern.match(lines[i]):
snippet_end = min(i + 5, before_line - 1)
return "\n".join(lines[i:snippet_end])
return ""
def _read_snippet(item: CodeItem, codebase_path: str) -> str:
"""Read source lines for *item* from *codebase_path*.
When the item belongs to a class, a short class header is prepended so
the user has context about what ``self`` refers to.
Args:
item: Code item whose file and line range to read.
codebase_path: Absolute path of the codebase root.
Returns:
The extracted source lines as a single string, or a placeholder
message when the file cannot be read.
"""
method_src = ""
try:
text = (Path(codebase_path) / item.file).read_text(
encoding="utf-8", errors="replace"
)
lines = text.splitlines()
start = max(0, item.start_line - 1)
end = min(len(lines), item.end_line)
method_src = "\n".join(lines[start:end])
if item.class_name:
header = _class_header(lines, item.class_name, item.start_line)
if header:
return (
f"# class {item.class_name} (context):\n{header}"
f"\n\n# method:\n{method_src}"
)
except OSError:
return f"(source unavailable for {item.file})"
return method_src
def _parse_verdict(raw: str) -> tuple[str, str]:
"""Parse the LLM's JSON verdict, tolerating markdown code fences.
Args:
raw: Raw text returned by the judge LLM.
Returns:
Tuple of ``(verdict, gap)`` where *verdict* is ``"PASS"`` or
``"FAIL"`` and *gap* is a one-sentence explanation of what was wrong.
Returns ``("FAIL", ...)`` on any parse failure.
"""
clean = re.sub(r"```(?:json)?\s*", "", raw, flags=re.DOTALL).strip()
start = clean.find("{")
end = clean.rfind("}") + 1
if start == -1 or end == 0:
return "FAIL", "Could not parse response from judge."
try:
data = json.loads(clean[start:end])
except json.JSONDecodeError:
return "FAIL", "Could not parse response from judge."
verdict = str(data.get("verdict", "FAIL")).upper()
if verdict not in {"PASS", "FAIL"}:
verdict = "FAIL"
gap = str(data.get("gap", "No specific gap identified."))
return verdict, gap
class Verifier:
"""Runs Socratic lessons: user explains first, then the LLM judges.
Args:
backend: LLM backend used for both judging and generating explanations.
console: Rich console for all output.
"""
def __init__(self, backend: Backend, console: Console) -> None:
"""Store backend and console."""
self._backend = backend
self._console = console
def _judge(self, snippet: str, explanation: str) -> tuple[str, str]:
"""Stream the judge call, show elapsed time, accumulate and parse.
The raw JSON is streamed silently; a live elapsed-time counter shows
the user that inference is progressing.
Args:
snippet: Source code the user was shown.
explanation: The user's explanation text.
Returns:
Tuple ``(verdict, gap)`` from ``_parse_verdict``.
"""
user_msg = f"Code:\n{snippet}\n\nUser's explanation:\n{explanation}"
start = time.monotonic()
parts: list[str] = []
def _on_token(token: str) -> None:
parts.append(token)
elapsed = int(time.monotonic() - start)
live.update(Text(f"Judging... {elapsed}s", style="yellow"))
with Live(
Text("Judging... 0s", style="yellow"),
console=self._console,
refresh_per_second=4,
transient=True,
) as live:
self._backend.stream(_SYSTEM_PROMPT, user_msg, _on_token)
return _parse_verdict("".join(parts))
def _collect_answers(
self, input_fn: Callable[[str], str]
) -> tuple[dict[str, str], bool]:
"""Ask the four core questions and collect answers.
Args:
input_fn: Callable used for reading user input.
Returns:
Tuple of ``(answers, skipped)`` where *answers* maps question
labels to the user's text. *skipped* is ``True`` when the user
typed ``skip`` on any question.
"""
answers: dict[str, str] = {}
for label, question in _QUESTIONS:
self._console.print(f"[bold]{question}[/bold]")
answer = input_fn("> ").strip()
if answer.lower() == "skip":
self._console.print("[yellow]Skipped.[/yellow]\n")
return {}, True
answers[label] = answer
return answers, False
def _ask_improvement(self, input_fn: Callable[[str], str]) -> str:
"""Prompt for an optional code-improvement note.
Args:
input_fn: Callable used for reading user input.
Returns:
The user's improvement idea, or ``""`` if they pressed Enter.
"""
self._console.print(f"\n[dim]{_IMPROVEMENT_Q}[/dim]")
return input_fn("> ").strip()
def run_lesson(
self,
item: CodeItem,
codebase_path: str,
*,
input_fn: Callable[[str], str] = input,
) -> LessonRecord:
"""Run one Socratic lesson for *item* and return a full transcript.
Shows the code, asks four sequential questions, judges the answer, and
repeats up to ``_MAX_ATTEMPTS`` times. On PASS, offers an optional
coding challenge (rewrite from scratch, validated by tests) then asks
the improvement question. Never reveals the correct explanation before
the user has tried.
Args:
item: The code item to study.
codebase_path: Absolute path of the codebase root.
input_fn: Callable used for reading user input.
Returns:
``LessonRecord`` with ``outcome`` of ``"learned"``,
``"struggled"``, or ``"skipped"``.
"""
snippet = _read_snippet(item, codebase_path)
timestamp = datetime.now(tz=timezone.utc).isoformat(timespec="seconds")
lines_str = f"{item.start_line}-{item.end_line}"
title = f"{item.file} lines {lines_str}"
self._console.print(Panel(snippet, title=title, border_style="blue"))
self._console.print(
"\nAnswer each question before I say anything. "
"Type [dim]skip[/dim] on any question to skip this item.\n"
)
answers: dict[str, str] = {}
for attempt in range(1, _MAX_ATTEMPTS + 1):
collected, skipped = self._collect_answers(input_fn)
if skipped:
return LessonRecord(
timestamp=timestamp,
item_id=item.id,
file=item.file,
lines=lines_str,
snippet=snippet,
outcome="skipped",
answers={},
improvement="",
verdict="skipped",
attempt=attempt,
)
answers = collected
explanation = "\n".join(f"{k}: {v}" for k, v in answers.items())
verdict, gap = self._judge(snippet, explanation)
if verdict == "PASS":
self._console.print(
f"[green]✓ PASS[/green] (attempt {attempt}/{_MAX_ATTEMPTS})\n"
)
challenge_result = run_coding_challenge(
item,
codebase_path,
explanation,
self._backend,
self._console,
input_fn,
)
improvement = self._ask_improvement(input_fn)
return LessonRecord(
timestamp=timestamp,
item_id=item.id,
file=item.file,
lines=lines_str,
snippet=snippet,
outcome="learned",
answers=answers,
improvement=improvement,
verdict="PASS",
attempt=attempt,
challenge_result=challenge_result,
)
if attempt < _MAX_ATTEMPTS:
self._console.print(f"[red]✗ FAIL[/red] -- {gap}")
self._console.print(f"Try again ({attempt}/{_MAX_ATTEMPTS}):\n")
self._console.print(
f"[red]After {_MAX_ATTEMPTS} attempts -- correct explanation:[/red]\n"
)
self._backend.stream(
_EXPLAIN_SYSTEM,
f"Explain:\n{snippet}",
lambda token: self._console.print(
token, end="", markup=False, highlight=False
),
)
self._console.print("\n")
improvement = self._ask_improvement(input_fn)
return LessonRecord(
timestamp=timestamp,
item_id=item.id,
file=item.file,
lines=lines_str,
snippet=snippet,
outcome="struggled",
answers=answers,
improvement=improvement,
verdict="FAIL",
attempt=_MAX_ATTEMPTS,
)

View File

@ -0,0 +1,345 @@
"""CLI entry point for code_tutor.
Commands:
analyze -- walk a codebase and produce a study plan
study -- run (or resume) an interactive study session
status -- show progress dashboard
drill -- force a lesson on a specific file
"""
from __future__ import annotations
import json
from pathlib import Path
import subprocess
import sys
import time
from typing import cast
import requests
from rich.console import Console
from rich.table import Table
import typer
from python_pkg.code_tutor._analyzer import codebase_fingerprint, extract_items
from python_pkg.code_tutor._llm import OllamaBackend
from python_pkg.code_tutor._plan_builder import build_plan
from python_pkg.code_tutor._progress import (
PlanData,
append_session_record,
config_dir,
item_from_data,
load_plan,
load_progress,
save_plan,
)
from python_pkg.code_tutor._session import run_session
from python_pkg.code_tutor._verifier import Verifier
app = typer.Typer(help="Socratic codebase understanding tutor.")
_console = Console()
_OLLAMA_API = "http://localhost:11434/api/tags"
_OLLAMA_START_TIMEOUT = 30
def _ensure_ollama_running(console: Console) -> bool:
"""Start the Ollama systemd service if it is not already reachable.
Tries ``systemctl start ollama`` and polls the API for up to
``_OLLAMA_START_TIMEOUT`` seconds.
Args:
console: Rich console for status messages.
Returns:
True when Ollama is reachable, False after timeout.
"""
try:
requests.get(_OLLAMA_API, timeout=2)
except requests.exceptions.RequestException:
pass
else:
return True
console.print("[yellow]Ollama not running -- starting via systemctl...[/yellow]")
try:
subprocess.run(
["systemctl", "start", "ollama"],
check=True,
capture_output=True,
)
except subprocess.CalledProcessError as exc:
console.print(
f"[red]systemctl start ollama failed: {exc.stderr.decode().strip()}[/red]"
)
return False
deadline = time.monotonic() + _OLLAMA_START_TIMEOUT
while time.monotonic() < deadline:
try:
requests.get(_OLLAMA_API, timeout=2)
except requests.exceptions.RequestException:
time.sleep(1)
else:
console.print("[green]Ollama is up.[/green]")
return True
console.print("[red]Ollama did not become ready in time.[/red]")
return False
def _ensure_fresh_plan(codebase: Path, plan: PlanData, console: Console) -> PlanData:
"""Rebuild *plan* if the codebase source files have changed since it was created.
Compares the plan's stored fingerprint against the current one. When they
differ, re-runs the full analyze pipeline, saves the new plan to disk, and
returns it. Plans without a stored fingerprint (created before this feature)
are returned unchanged.
Args:
codebase: Root directory of the codebase.
plan: Plan dict loaded from ``plan.json``.
console: Rich console for status messages.
Returns:
The original plan when up-to-date, or a freshly built ``PlanData`` when
the fingerprint has changed.
"""
saved = plan.get("source_fingerprint", "")
if not saved:
return plan
current = codebase_fingerprint(codebase)
if current == saved:
return plan
console.print(
"[yellow]Plan is stale -- source files changed. Rebuilding...[/yellow]"
)
items = extract_items(codebase)
if not items:
console.print("[red]No extractable items found -- keeping existing plan.[/red]")
return plan
new_plan = build_plan(codebase, items)
save_plan(codebase, new_plan)
n = new_plan["total_items"]
console.print(f"[green]Plan rebuilt: {n} items.[/green]")
return cast("PlanData", new_plan)
@app.command()
def analyze(
codebase: Path = typer.Argument(
..., help="Root directory of the codebase to study."
),
session_size: int = typer.Option(10, help="Items per study session."),
) -> None:
"""Analyse *codebase* and create a dependency-ordered study plan.
The plan is saved to ``~/.config/code_tutor/<hash>/plan.json``.
Args:
codebase: Root directory of the codebase.
session_size: Maximum number of items per study session.
"""
if not codebase.is_dir():
_console.print(f"[red]Error: {codebase} is not a directory.[/red]")
raise typer.Exit(1)
_console.print(f"Analysing [bold]{codebase}[/bold] ...")
items = extract_items(codebase)
if not items:
_console.print("[yellow]No extractable items found in this directory.[/yellow]")
raise typer.Exit(0)
n_items = len(items)
plan = build_plan(codebase, items, session_size=session_size)
save_plan(codebase, plan)
n_sessions = (n_items + session_size - 1) // session_size
dest = config_dir(codebase) / "plan.json"
_console.print(
f"Found [bold]{n_items}[/bold] items across {n_sessions} session(s).\n"
f"Plan saved to [dim]{dest}[/dim]"
)
@app.command()
def study(
codebase: Path = typer.Argument(
Path(),
help="Root directory of the codebase. Defaults to the current directory.",
),
) -> None:
"""Start or resume an interactive study session for *codebase*.
Automatically rebuilds the plan if source files have changed since the last
analyze run. Starts Ollama via systemctl if it is not already running.
Args:
codebase: Root directory of the codebase.
"""
if not codebase.is_dir():
_console.print(f"[red]Error: {codebase} is not a directory.[/red]")
raise typer.Exit(1)
plan = load_plan(codebase)
if plan is None:
_console.print(
"[yellow]No plan found. "
"Run [bold]code_tutor analyze <path>[/bold] first.[/yellow]"
)
raise typer.Exit(1)
_ensure_fresh_plan(codebase, plan, _console)
if not _ensure_ollama_running(_console):
raise typer.Exit(1)
backend = OllamaBackend()
try:
run_session(codebase, backend, console=_console)
except requests.exceptions.ConnectionError as exc:
_console.print(
"[red]Cannot connect to Ollama. Is it running at localhost:11434?[/red]"
)
raise typer.Exit(1) from exc
@app.command()
def status(
codebase: Path = typer.Argument(
Path(),
help="Root directory of the codebase.",
),
) -> None:
"""Show a progress dashboard for *codebase*.
Automatically rebuilds the plan if source files have changed.
Args:
codebase: Root directory of the codebase.
"""
plan = load_plan(codebase)
if plan is None:
_console.print(
"[yellow]No plan found. "
"Run [bold]code_tutor analyze <path>[/bold] first.[/yellow]"
)
raise typer.Exit(1)
plan = _ensure_fresh_plan(codebase, plan, _console)
progress = load_progress(codebase)
total = plan["total_items"]
learned = len(progress["learned"])
struggled = len(progress["struggled"])
skipped = len(progress["skipped"])
remaining = total - learned - struggled - skipped
table = Table(title=f"Progress: {codebase}", show_header=True, header_style="bold")
table.add_column("Category")
table.add_column("Count", justify="right")
table.add_row("[green]Learned[/green]", str(learned))
table.add_row("[red]Struggled[/red]", str(struggled))
table.add_row("[yellow]Skipped[/yellow]", str(skipped))
table.add_row("Remaining", str(remaining))
table.add_row("[bold]Total[/bold]", str(total))
_console.print(table)
@app.command()
def drill(
file: Path = typer.Argument(..., help="Source file to drill."),
) -> None:
"""Force a study lesson on every item in *file*.
Searches all saved plans to find which codebase contains *file*.
Automatically rebuilds the plan if source files have changed.
Starts Ollama via systemctl if it is not already running.
Args:
file: Absolute or relative path to the source file.
"""
file = file.resolve()
codebase = _find_codebase_for_file(file)
if codebase is None:
_console.print(
"[red]No saved plan contains this file. "
"Run [bold]code_tutor analyze <codebase>[/bold] first.[/red]"
)
raise typer.Exit(1)
plan = load_plan(codebase)
if plan is None:
_console.print("[red]Plan disappeared unexpectedly.[/red]")
raise typer.Exit(1)
plan = _ensure_fresh_plan(codebase, plan, _console)
rel = str(file.relative_to(codebase))
target_items = [
item
for session in plan["sessions"]
for item in session["items"]
if item["file"] == rel
]
if not target_items:
_console.print(f"[yellow]No items found for {rel} in the plan.[/yellow]")
raise typer.Exit(0)
if not _ensure_ollama_running(_console):
raise typer.Exit(1)
backend = OllamaBackend()
verifier = Verifier(backend, _console)
codebase_str = str(codebase)
for item_data in target_items:
item = item_from_data(item_data)
record = verifier.run_lesson(item, codebase_str)
append_session_record(codebase, record)
def _find_codebase_for_file(file: Path) -> Path | None:
"""Search all saved plans and return the codebase that contains *file*.
Args:
file: Absolute path to the source file.
Returns:
The codebase ``Path`` whose plan contains *file*, or ``None``.
"""
config_root = Path.home() / ".config" / "code_tutor"
if not config_root.exists():
return None
for plan_file in sorted(config_root.glob("*/plan.json")):
result = _check_plan_file(plan_file, file)
if result is not None:
return result
return None
def _check_plan_file(plan_file: Path, file: Path) -> Path | None:
"""Return the codebase path from *plan_file* if it contains *file*, else None.
Args:
plan_file: Path to a ``plan.json`` file.
file: Absolute path to look up.
Returns:
The codebase ``Path`` when *file* is relative to it, or ``None``.
"""
try:
data = json.loads(plan_file.read_text(encoding="utf-8"))
codebase = Path(str(data.get("codebase_path", "")))
file.relative_to(codebase)
except (ValueError, KeyError, OSError):
return None
return codebase
if __name__ == "__main__":
sys.exit(app())

View File

@ -0,0 +1 @@
"""Tests for the code_tutor package."""

View File

@ -0,0 +1,56 @@
"""Shared factory helpers for the ``_challenge`` / ``_challenge_support`` tests.
Kept in ``conftest.py`` (a single, import-safe location) so the split test
modules (``test_challenge``, ``test_challenge_part2``, ``test_challenge_part3``)
reuse one definition instead of duplicating the factories, which would trip
pylint's ``duplicate-code`` check.
"""
from __future__ import annotations
from unittest.mock import MagicMock
from python_pkg.code_tutor._analyzer import CodeItem
def _item(
file: str = "mod.py",
name: str = "fn",
start: int = 1,
end: int = 3,
class_name: str = "",
) -> CodeItem:
"""Build a ``CodeItem`` with sensible defaults for tests.
Args:
file: Relative source-file path for the item.
name: Function name.
start: 1-based start line of the item.
end: 1-based end line of the item.
class_name: Owning class name, or empty for module-level functions.
Returns:
A populated ``CodeItem`` instance.
"""
return CodeItem(
id=f"{file}.{name}",
file=file,
type="function",
name=name,
start_line=start,
end_line=end,
class_name=class_name,
)
def _make_live_mock() -> MagicMock:
"""Return a ``MagicMock`` that behaves like a ``rich.live.Live`` context.
Returns:
A mock whose ``__enter__`` yields itself and whose ``__exit__``
returns ``False`` so exceptions propagate normally.
"""
live = MagicMock()
live.__enter__ = MagicMock(return_value=live)
live.__exit__ = MagicMock(return_value=False)
return live

View File

@ -0,0 +1,442 @@
"""Tests for python_pkg.code_tutor._analyzer."""
from __future__ import annotations
import ast
from pathlib import Path
from unittest.mock import patch
from python_pkg.code_tutor._analyzer import (
CodeItem,
_build_dotted_map,
_extract_other,
_extract_python,
_FunctionVisitor,
_is_binary,
_make_id,
_match_deps,
_should_skip,
codebase_fingerprint,
extract_items,
get_file_imports,
get_python_files,
)
# ---------------------------------------------------------------------------
# _is_binary
# ---------------------------------------------------------------------------
def test_is_binary_true(tmp_path: Path) -> None:
f = tmp_path / "binary.bin"
f.write_bytes(b"hello\x00world")
assert _is_binary(f) is True
def test_is_binary_false(tmp_path: Path) -> None:
f = tmp_path / "text.py"
f.write_text("print('hello')", encoding="utf-8")
assert _is_binary(f) is False
# ---------------------------------------------------------------------------
# _should_skip
# ---------------------------------------------------------------------------
def test_should_skip_dir(tmp_path: Path) -> None:
assert _should_skip(".venv/lib/site-packages/foo.py") is True
def test_should_skip_node_modules(tmp_path: Path) -> None:
assert _should_skip("node_modules/react/index.js") is True
def test_should_skip_suffix(tmp_path: Path) -> None:
assert _should_skip("assets/logo.png") is True
def test_should_skip_false(tmp_path: Path) -> None:
assert _should_skip("src/main.py") is False
def test_should_skip_pycache(tmp_path: Path) -> None:
assert _should_skip("__pycache__/foo.pyc") is True
# ---------------------------------------------------------------------------
# _make_id
# ---------------------------------------------------------------------------
def test_make_id_basic() -> None:
assert _make_id("src/utils.py", "helper") == "src.utils.helper"
def test_make_id_nested() -> None:
assert _make_id("pkg/sub/mod.py", "fn") == "pkg.sub.mod.fn"
# ---------------------------------------------------------------------------
# _FunctionVisitor
# ---------------------------------------------------------------------------
def test_function_visitor_module_level() -> None:
source = "def my_func(x):\n return x\n"
tree = ast.parse(source)
visitor = _FunctionVisitor("mod.py")
visitor.visit(tree)
assert len(visitor.items) == 1
item = visitor.items[0]
assert item.name == "my_func"
assert item.class_name == ""
assert item.type == "function"
def test_function_visitor_class_method() -> None:
source = "class Foo:\n def bar(self):\n pass\n"
tree = ast.parse(source)
visitor = _FunctionVisitor("mod.py")
visitor.visit(tree)
assert len(visitor.items) == 1
item = visitor.items[0]
assert item.name == "bar"
assert item.class_name == "Foo"
def test_function_visitor_async() -> None:
source = "async def do_thing():\n pass\n"
tree = ast.parse(source)
visitor = _FunctionVisitor("mod.py")
visitor.visit(tree)
assert len(visitor.items) == 1
assert visitor.items[0].type == "async_function"
def test_function_visitor_class_stack_restored() -> None:
source = "class A:\n def m(self): pass\nclass B:\n def n(self): pass\n"
tree = ast.parse(source)
visitor = _FunctionVisitor("mod.py")
visitor.visit(tree)
names = {it.name: it.class_name for it in visitor.items}
assert names["m"] == "A"
assert names["n"] == "B"
# ---------------------------------------------------------------------------
# _extract_python
# ---------------------------------------------------------------------------
def test_extract_python_valid(tmp_path: Path) -> None:
f = tmp_path / "mod.py"
f.write_text("def foo():\n pass\n", encoding="utf-8")
items = _extract_python(f, "mod.py")
assert len(items) == 1
assert items[0].name == "foo"
def test_extract_python_syntax_error(tmp_path: Path) -> None:
f = tmp_path / "bad.py"
f.write_text("def (broken):\n", encoding="utf-8")
items = _extract_python(f, "bad.py")
assert items == []
# ---------------------------------------------------------------------------
# _extract_other
# ---------------------------------------------------------------------------
def test_extract_other_match(tmp_path: Path) -> None:
f = tmp_path / "main.go"
f.write_text("func doThing(x int) {\n}\n", encoding="utf-8")
items = _extract_other(f, "main.go")
assert any(it.name == "doThing" for it in items)
def test_extract_other_no_match(tmp_path: Path) -> None:
f = tmp_path / "empty.go"
f.write_text("package main\n", encoding="utf-8")
items = _extract_other(f, "empty.go")
assert items == []
def test_extract_other_oserror(tmp_path: Path) -> None:
missing = tmp_path / "ghost.go"
items = _extract_other(missing, "ghost.go")
assert items == []
# ---------------------------------------------------------------------------
# extract_items
# ---------------------------------------------------------------------------
def test_extract_items_py_file(tmp_path: Path) -> None:
(tmp_path / "a.py").write_text("def hello(): pass\n", encoding="utf-8")
items = extract_items(tmp_path)
assert any(it.name == "hello" for it in items)
def test_extract_items_other_lang(tmp_path: Path) -> None:
(tmp_path / "main.go").write_text("func run(x int) {}\n", encoding="utf-8")
items = extract_items(tmp_path)
assert any(it.name == "run" for it in items)
def test_extract_items_skips_binary(tmp_path: Path) -> None:
(tmp_path / "img.py").write_bytes(b"\x00\x01\x02\x03")
items = extract_items(tmp_path)
assert items == []
def test_extract_items_skips_should_skip(tmp_path: Path) -> None:
skip_dir = tmp_path / ".venv"
skip_dir.mkdir()
(skip_dir / "lib.py").write_text("def x(): pass\n", encoding="utf-8")
items = extract_items(tmp_path)
assert items == []
def test_extract_items_skips_directories(tmp_path: Path) -> None:
sub = tmp_path / "sub"
sub.mkdir()
(sub / "mod.py").write_text("def f(): pass\n", encoding="utf-8")
items = extract_items(tmp_path)
assert any(it.name == "f" for it in items)
def test_extract_items_skips_unhandled_extension(tmp_path: Path) -> None:
(tmp_path / "readme.txt").write_text("hello world", encoding="utf-8")
items = extract_items(tmp_path)
assert items == []
def test_extract_items_is_binary_oserror(tmp_path: Path) -> None:
(tmp_path / "a.py").write_text("def f(): pass\n", encoding="utf-8")
with patch(
"python_pkg.code_tutor._analyzer._is_binary", side_effect=OSError("perm")
):
items = extract_items(tmp_path)
assert items == []
# ---------------------------------------------------------------------------
# get_python_files
# ---------------------------------------------------------------------------
def test_get_python_files(tmp_path: Path) -> None:
(tmp_path / "a.py").write_text("x = 1", encoding="utf-8")
(tmp_path / "b.py").write_text("y = 2", encoding="utf-8")
skip_dir = tmp_path / ".venv"
skip_dir.mkdir()
(skip_dir / "c.py").write_text("z = 3", encoding="utf-8")
files = get_python_files(tmp_path)
names = {f.name for f in files}
assert "a.py" in names
assert "b.py" in names
assert "c.py" not in names
# ---------------------------------------------------------------------------
# _build_dotted_map
# ---------------------------------------------------------------------------
def test_build_dotted_map_normal(tmp_path: Path) -> None:
f = tmp_path / "pkg" / "mod.py"
f.parent.mkdir()
f.write_text("x = 1", encoding="utf-8")
result = _build_dotted_map(tmp_path, [f])
assert result[f] == "pkg.mod"
def test_build_dotted_map_value_error(tmp_path: Path) -> None:
other = tmp_path.parent / "other.py"
result = _build_dotted_map(tmp_path, [other])
assert other not in result
# ---------------------------------------------------------------------------
# _match_deps
# ---------------------------------------------------------------------------
def test_match_deps_exact(tmp_path: Path) -> None:
f = tmp_path / "mod.py"
dotted_map = {f: "mod"}
path = tmp_path / "main.py"
result = _match_deps({"mod"}, dotted_map, path)
assert f in result
def test_match_deps_prefix(tmp_path: Path) -> None:
f = tmp_path / "mod.py"
dotted_map = {f: "mod"}
path = tmp_path / "main.py"
result = _match_deps({"pkg.mod"}, dotted_map, path)
assert f in result
def test_match_deps_suffix(tmp_path: Path) -> None:
f = tmp_path / "mod.py"
dotted_map = {f: "pkg.mod"}
path = tmp_path / "main.py"
result = _match_deps({"mod"}, dotted_map, path)
assert f in result
def test_match_deps_excludes_self(tmp_path: Path) -> None:
path = tmp_path / "mod.py"
dotted_map = {path: "mod"}
result = _match_deps({"mod"}, dotted_map, path)
assert path not in result
def test_match_deps_no_match(tmp_path: Path) -> None:
f = tmp_path / "other.py"
dotted_map = {f: "other"}
path = tmp_path / "main.py"
result = _match_deps({"unrelated"}, dotted_map, path)
assert f not in result
# ---------------------------------------------------------------------------
# Tests for get_file_imports
# ---------------------------------------------------------------------------
def test_get_file_imports_import(tmp_path: Path) -> None:
f = tmp_path / "main.py"
dep = tmp_path / "utils.py"
f.write_text("import utils\n", encoding="utf-8")
dep.write_text("x = 1", encoding="utf-8")
result = get_file_imports(f, tmp_path, [f, dep])
assert dep in result
def test_get_file_imports_from_import(tmp_path: Path) -> None:
f = tmp_path / "main.py"
dep = tmp_path / "utils.py"
f.write_text("from utils import helper\n", encoding="utf-8")
dep.write_text("def helper(): pass\n", encoding="utf-8")
result = get_file_imports(f, tmp_path, [f, dep])
assert dep in result
def test_get_file_imports_from_import_none_module(tmp_path: Path) -> None:
# Relative import: node.module is None
f = tmp_path / "main.py"
f.write_text("from . import helper\n", encoding="utf-8")
result = get_file_imports(f, tmp_path, [f])
assert result == set()
def test_get_file_imports_syntax_error(tmp_path: Path) -> None:
f = tmp_path / "bad.py"
f.write_text("def (broken):\n", encoding="utf-8")
result = get_file_imports(f, tmp_path, [f])
assert result == set()
# ---------------------------------------------------------------------------
# codebase_fingerprint
# ---------------------------------------------------------------------------
def test_codebase_fingerprint_basic(tmp_path: Path) -> None:
(tmp_path / "a.py").write_text("x = 1", encoding="utf-8")
result = codebase_fingerprint(tmp_path)
assert len(result) == 16
assert result.isalnum()
def test_codebase_fingerprint_skips_dir(tmp_path: Path) -> None:
sub = tmp_path / "sub"
sub.mkdir()
result = codebase_fingerprint(tmp_path)
assert len(result) == 16
def test_codebase_fingerprint_skips_should_skip(tmp_path: Path) -> None:
venv = tmp_path / ".venv"
venv.mkdir()
(venv / "lib.py").write_text("x = 1", encoding="utf-8")
result_without = codebase_fingerprint(tmp_path)
(tmp_path / "real.py").write_text("y = 2", encoding="utf-8")
result_with = codebase_fingerprint(tmp_path)
assert result_without != result_with
def test_codebase_fingerprint_stat_oserror(tmp_path: Path) -> None:
(tmp_path / "source.py").write_text("x = 1", encoding="utf-8")
stat_calls: dict[str, int] = {}
real_stat = Path.stat
def patched_stat(self: Path, **kwargs: object) -> object:
key = str(self)
stat_calls[key] = stat_calls.get(key, 0) + 1
if stat_calls[key] >= 2 and key.endswith(".py"):
msg = "simulated stat failure"
raise OSError(msg)
return real_stat(self, **kwargs)
with patch.object(Path, "stat", patched_stat):
result = codebase_fingerprint(tmp_path)
assert len(result) == 16
def test_codebase_fingerprint_changes_on_modification(tmp_path: Path) -> None:
f = tmp_path / "mod.py"
f.write_text("x = 1", encoding="utf-8")
fp1 = codebase_fingerprint(tmp_path)
f.write_text("x = 2", encoding="utf-8")
fp2 = codebase_fingerprint(tmp_path)
assert fp1 != fp2
def test_extract_items_image_suffix(tmp_path: Path) -> None:
(tmp_path / "logo.png").write_bytes(b"\x89PNG\r\n")
items = extract_items(tmp_path)
assert items == []
def test_code_item_defaults() -> None:
item = CodeItem(
id="mod.fn",
file="mod.py",
type="function",
name="fn",
start_line=1,
end_line=5,
)
assert item.class_name == ""
assert item.depends_on == []
def test_codebase_fingerprint_stat_oserror_raises(tmp_path: Path) -> None:
"""Skip a file whose ``stat`` raises ``OSError`` (except branch).
On this Python ``Path.is_file`` uses ``os.path.isfile`` rather than
``Path.stat``, so a stat that always fails for the source file reaches
the ``except OSError`` handler that drops the unreadable entry.
"""
(tmp_path / "source.py").write_text("x = 1", encoding="utf-8")
real_stat = Path.stat
def failing_stat(self: Path, **kwargs: object) -> object:
"""Raise for the target file, delegate to the real stat otherwise."""
if str(self).endswith("source.py"):
msg = "simulated stat failure"
raise OSError(msg)
return real_stat(self, **kwargs)
with patch.object(Path, "stat", failing_stat):
result = codebase_fingerprint(tmp_path)
assert len(result) == 16

View File

@ -0,0 +1,346 @@
"""Tests for python_pkg.code_tutor._challenge_support -- part 1.
Covers the test-discovery and pytest-execution helpers:
``_scan_test_file``, ``_find_tests``, ``_show_test_panels``,
``_collect_lines``, ``_project_root``, ``_pytest_clean`` and
``_patch_and_test``.
"""
from __future__ import annotations
from pathlib import Path
from unittest.mock import MagicMock, patch
from python_pkg.code_tutor._challenge_support import (
_collect_lines,
_find_tests,
_patch_and_test,
_project_root,
_pytest_clean,
_scan_test_file,
_show_test_panels,
)
from python_pkg.code_tutor.tests.conftest import _item
# ---------------------------------------------------------------------------
# _scan_test_file
# ---------------------------------------------------------------------------
def test_scan_test_file_module_function_match() -> None:
source = "def test_fn_does_thing():\n fn(1)\n"
result = _scan_test_file(source, "fn")
assert "test_fn_does_thing" in result
def test_scan_test_file_class_method_match() -> None:
source = "class TestFoo:\n def test_fn_works(self):\n fn()\n"
result = _scan_test_file(source, "fn")
assert any("test_fn_works" in r for r in result)
def test_scan_test_file_no_match() -> None:
source = "def test_other():\n other()\n"
result = _scan_test_file(source, "fn")
assert result == []
def test_scan_test_file_syntax_error() -> None:
result = _scan_test_file("def (broken):", "fn")
assert result == []
def test_scan_test_file_non_test_function_skipped() -> None:
source = "def helper_fn():\n fn()\n"
result = _scan_test_file(source, "fn")
assert result == []
# ---------------------------------------------------------------------------
# _find_tests
# ---------------------------------------------------------------------------
def test_find_tests_found(tmp_path: Path) -> None:
(tmp_path / "test_mod.py").write_text(
"def test_fn_works():\n fn()\n", encoding="utf-8"
)
result = _find_tests(_item(), tmp_path)
assert len(result) == 1
assert result[0][1] == ["test_fn_works"]
def test_find_tests_no_name_in_source(tmp_path: Path) -> None:
(tmp_path / "test_other.py").write_text(
"def test_other_works():\n other()\n", encoding="utf-8"
)
result = _find_tests(_item(), tmp_path)
assert result == []
def test_find_tests_oserror(tmp_path: Path) -> None:
test_file = tmp_path / "test_mod.py"
test_file.write_text("def test_fn(): fn()\n", encoding="utf-8")
with patch("pathlib.Path.read_text", side_effect=OSError("perm")):
result = _find_tests(_item(), tmp_path)
assert result == []
def test_find_tests_no_matching_node_ids(tmp_path: Path) -> None:
# File mentions 'fn' in source but not in test function bodies
(tmp_path / "test_mod.py").write_text(
"# fn is mentioned here\ndef test_other():\n pass\n", encoding="utf-8"
)
result = _find_tests(_item(), tmp_path)
assert result == []
# ---------------------------------------------------------------------------
# _show_test_panels
# ---------------------------------------------------------------------------
def test_show_test_panels_normal(tmp_path: Path) -> None:
source = "def test_fn():\n fn(1)\n assert True\n"
test_file = tmp_path / "test_mod.py"
test_file.write_text(source, encoding="utf-8")
mock_console = MagicMock()
with patch("python_pkg.code_tutor._challenge_support.Syntax"):
_show_test_panels([(test_file, ["test_fn"])], mock_console)
assert mock_console.print.called
def test_show_test_panels_class_method(tmp_path: Path) -> None:
source = "class TestFoo:\n def test_fn(self):\n fn()\n"
test_file = tmp_path / "test_mod.py"
test_file.write_text(source, encoding="utf-8")
mock_console = MagicMock()
with patch("python_pkg.code_tutor._challenge_support.Syntax"):
_show_test_panels([(test_file, ["TestFoo::test_fn"])], mock_console)
assert mock_console.print.called
def test_show_test_panels_oserror(tmp_path: Path) -> None:
test_file = tmp_path / "test_mod.py"
test_file.write_text("def test_fn(): pass\n", encoding="utf-8")
mock_console = MagicMock()
with patch("pathlib.Path.read_text", side_effect=OSError("perm")):
_show_test_panels([(test_file, ["test_fn"])], mock_console)
mock_console.print.assert_not_called()
def test_show_test_panels_node_not_in_func_names(tmp_path: Path) -> None:
source = "def test_fn():\n fn()\n"
test_file = tmp_path / "test_mod.py"
test_file.write_text(source, encoding="utf-8")
mock_console = MagicMock()
with patch("python_pkg.code_tutor._challenge_support.Syntax"):
# Pass a node_id that doesn't match any function name
_show_test_panels([(test_file, ["test_other"])], mock_console)
mock_console.print.assert_not_called()
# ---------------------------------------------------------------------------
# _collect_lines
# ---------------------------------------------------------------------------
def test_collect_lines_end() -> None:
responses = iter(["line one", "line two", "END"])
mock_console = MagicMock()
result = _collect_lines("prompt", mock_console, lambda _: next(responses))
assert result == "line one\nline two"
def test_collect_lines_skip() -> None:
mock_console = MagicMock()
result = _collect_lines("prompt", mock_console, lambda _: "skip")
assert result is None
# ---------------------------------------------------------------------------
# _project_root
# ---------------------------------------------------------------------------
def test_project_root_finds_pyproject(tmp_path: Path) -> None:
(tmp_path / "pyproject.toml").write_text("[project]", encoding="utf-8")
sub = tmp_path / "src" / "pkg"
sub.mkdir(parents=True)
result = _project_root(sub)
assert result == tmp_path
def test_project_root_finds_setup_py(tmp_path: Path) -> None:
(tmp_path / "setup.py").write_text("# setup", encoding="utf-8")
result = _project_root(tmp_path)
assert result == tmp_path
def test_project_root_not_found(tmp_path: Path) -> None:
# Create a dir deep inside tmp_path with no pyproject/setup
deep = tmp_path / "a" / "b" / "c"
deep.mkdir(parents=True)
# Walk up to root (tmp_path's parent and above) without finding the files
# We need a completely isolated path — create one without any pyproject.toml
import tempfile
with tempfile.TemporaryDirectory() as td:
isolated = Path(td) / "pkg"
isolated.mkdir()
result = _project_root(isolated)
# When not found, returns start.resolve()
assert result == isolated.resolve()
# ---------------------------------------------------------------------------
# _pytest_clean
# ---------------------------------------------------------------------------
def test_pytest_clean_pass() -> None:
mock_result = MagicMock()
mock_result.returncode = 0
mock_result.stdout = "1 passed"
mock_result.stderr = ""
mock_console = MagicMock()
with patch(
"python_pkg.code_tutor._challenge_support.subprocess.run",
return_value=mock_result,
):
assert (
_pytest_clean(["test_mod.py::test_fn"], Path("/proj"), mock_console) is True
)
def test_pytest_clean_fail() -> None:
mock_result = MagicMock()
mock_result.returncode = 1
mock_result.stdout = ""
mock_result.stderr = "FAILED"
mock_console = MagicMock()
with patch(
"python_pkg.code_tutor._challenge_support.subprocess.run",
return_value=mock_result,
):
assert (
_pytest_clean(["test_mod.py::test_fn"], Path("/proj"), mock_console)
is False
)
def test_pytest_clean_no_output() -> None:
mock_result = MagicMock()
mock_result.returncode = 0
mock_result.stdout = ""
mock_result.stderr = ""
mock_console = MagicMock()
with patch(
"python_pkg.code_tutor._challenge_support.subprocess.run",
return_value=mock_result,
):
_pytest_clean([], Path("/proj"), mock_console)
mock_console.print.assert_not_called()
# ---------------------------------------------------------------------------
# _patch_and_test
# ---------------------------------------------------------------------------
def test_patch_and_test_syntax_error(tmp_path: Path) -> None:
source = "def fn():\n return 1\n"
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
mock_console = MagicMock()
result = _patch_and_test(
_item(file="mod.py", start=1, end=2),
str(tmp_path),
"def fn(\n broken syntax",
[],
mock_console,
)
assert result is False
# Original file should be restored
assert (tmp_path / "mod.py").read_text(encoding="utf-8") == source
def test_patch_and_test_passes(tmp_path: Path) -> None:
source = "def fn():\n return 1\n"
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
test_file = tmp_path / "test_mod.py"
test_file.write_text("def test_fn(): pass\n", encoding="utf-8")
mock_console = MagicMock()
with patch(
"python_pkg.code_tutor._challenge_support._pytest_clean", return_value=True
):
result = _patch_and_test(
_item(file="mod.py", start=1, end=2),
str(tmp_path),
"def fn():\n return 99",
[(test_file, ["test_fn"])],
mock_console,
)
assert result is True
# File restored
assert (tmp_path / "mod.py").read_text(encoding="utf-8") == source
def test_patch_and_test_fails(tmp_path: Path) -> None:
source = "def fn():\n return 1\n"
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
test_file = tmp_path / "test_mod.py"
test_file.write_text("def test_fn(): pass\n", encoding="utf-8")
mock_console = MagicMock()
with patch(
"python_pkg.code_tutor._challenge_support._pytest_clean", return_value=False
):
result = _patch_and_test(
_item(file="mod.py", start=1, end=2),
str(tmp_path),
"def fn():\n return 0",
[(test_file, ["test_fn"])],
mock_console,
)
assert result is False
assert (tmp_path / "mod.py").read_text(encoding="utf-8") == source
def test_scan_test_file_non_func_non_class_toplevel() -> None:
"""Skip a top-level node that is neither a function nor a class def.
A module-level ``import`` fails both ``isinstance`` checks, exercising
the branch that falls through the ``elif ClassDef`` back to the loop.
"""
source = "import os\ndef test_fn_uses_fn():\n fn()\n"
result = _scan_test_file(source, "fn")
assert result == ["test_fn_uses_fn"]
def test_show_test_panels_non_func_non_class_toplevel(tmp_path: Path) -> None:
"""Skip a top-level node that is neither a function nor a class def.
A module-level ``import`` yields an empty ``nodes`` list, so the
``elif ClassDef`` branch is not taken before the inner render loop.
"""
source = "import os\ndef test_fn():\n fn(1)\n"
test_file = tmp_path / "test_mod.py"
test_file.write_text(source, encoding="utf-8")
mock_console = MagicMock()
with patch("python_pkg.code_tutor._challenge_support.Syntax"):
_show_test_panels([(test_file, ["test_fn"])], mock_console)
assert mock_console.print.called
def test_show_test_panels_class_non_func_member(tmp_path: Path) -> None:
"""Skip a class-body member that is not a function definition.
The class-level ``attr = 1`` assignment fails the member ``isinstance``
check, exercising the branch that loops back to the next body member.
"""
source = "class TestFoo:\n attr = 1\n def test_fn(self):\n fn()\n"
test_file = tmp_path / "test_mod.py"
test_file.write_text(source, encoding="utf-8")
mock_console = MagicMock()
with patch("python_pkg.code_tutor._challenge_support.Syntax"):
_show_test_panels([(test_file, ["TestFoo::test_fn"])], mock_console)
assert mock_console.print.called

View File

@ -0,0 +1,308 @@
"""Tests for the code_tutor challenge helpers -- part 2.
Covers the LLM verdict/signature helpers in ``_challenge_support``
(``_stream_verdict``, ``_parse_verdict``, ``_extract_signature_block``,
``_import_hint``) and the test-validation/rating helpers in ``_challenge``
(``_validate_tests_against_real``, ``_collect_and_rate_tests``).
"""
from __future__ import annotations
from pathlib import Path
from unittest.mock import MagicMock, patch
from python_pkg.code_tutor._challenge import (
_collect_and_rate_tests,
_validate_tests_against_real,
)
from python_pkg.code_tutor._challenge_support import (
_extract_signature_block,
_import_hint,
_parse_verdict,
_stream_verdict,
)
from python_pkg.code_tutor.tests.conftest import _item, _make_live_mock
# ---------------------------------------------------------------------------
# _stream_verdict
# ---------------------------------------------------------------------------
def test_stream_verdict_accumulates_tokens() -> None:
mock_backend = MagicMock()
mock_console = MagicMock()
live_mock = _make_live_mock()
def fake_stream(system: str, user: str, on_token: object) -> str:
assert callable(on_token)
return '{"verdict": "PASS", "gap": ""}'
mock_backend.stream.side_effect = fake_stream
with patch("python_pkg.code_tutor._challenge_support.Live", return_value=live_mock):
result = _stream_verdict("sys", "user", mock_backend, mock_console)
assert result == "" # parts is empty since on_token was never called by stream
mock_backend.stream.assert_called_once()
def test_stream_verdict_on_token_called() -> None:
mock_backend = MagicMock()
mock_console = MagicMock()
live_mock = _make_live_mock()
def fake_stream(system: str, user: str, on_token: object) -> str:
assert callable(on_token)
on_token('{"verdict":')
on_token(' "PASS", "gap": ""}')
return ""
mock_backend.stream.side_effect = fake_stream
with patch("python_pkg.code_tutor._challenge_support.Live", return_value=live_mock):
result = _stream_verdict(
"sys", "user", mock_backend, mock_console, label="Test"
)
assert '{"verdict":' in result
# ---------------------------------------------------------------------------
# _parse_verdict (challenge version)
# ---------------------------------------------------------------------------
def test_challenge_parse_verdict_pass() -> None:
verdict, gap = _parse_verdict('{"verdict": "PASS", "gap": ""}')
assert verdict == "PASS"
assert gap == ""
def test_challenge_parse_verdict_fail() -> None:
verdict, gap = _parse_verdict('{"verdict": "FAIL", "gap": "missing"}')
assert verdict == "FAIL"
assert gap == "missing"
def test_challenge_parse_verdict_no_braces() -> None:
verdict, _gap = _parse_verdict("no json")
assert verdict == "FAIL"
def test_challenge_parse_verdict_json_error() -> None:
verdict, _gap = _parse_verdict("{bad}")
assert verdict == "FAIL"
def test_challenge_parse_verdict_invalid() -> None:
verdict, _gap = _parse_verdict('{"verdict": "UNKNOWN", "gap": "x"}')
assert verdict == "FAIL"
# ---------------------------------------------------------------------------
# _extract_signature_block
# ---------------------------------------------------------------------------
def test_extract_signature_block_with_docstring(tmp_path: Path) -> None:
source = 'def fn(x):\n """Do thing."""\n return x\n'
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
result = _extract_signature_block(_item(file="mod.py"), str(tmp_path))
assert "def fn" in result
assert '"""Do thing."""' in result
def test_extract_signature_block_without_docstring(tmp_path: Path) -> None:
source = "def fn(x):\n return x\n"
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
result = _extract_signature_block(_item(file="mod.py"), str(tmp_path))
assert "def fn" in result
assert "return" not in result
def test_extract_signature_block_oserror(tmp_path: Path) -> None:
result = _extract_signature_block(_item(file="missing.py"), str(tmp_path))
assert "def fn(...):" in result
def test_extract_signature_block_no_match(tmp_path: Path) -> None:
source = "def other(): pass\n"
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
result = _extract_signature_block(_item(file="mod.py", name="fn"), str(tmp_path))
assert "def fn(...):" in result
def test_extract_signature_block_syntax_error(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def (broken):\n", encoding="utf-8")
result = _extract_signature_block(_item(file="mod.py"), str(tmp_path))
assert "def fn(...):" in result
# ---------------------------------------------------------------------------
# Tests for _import_hint
# ---------------------------------------------------------------------------
def test_import_hint_relative_to_project(tmp_path: Path) -> None:
(tmp_path / "src").mkdir()
result = _import_hint(
_item(file="src/mod.py", name="fn"),
str(tmp_path),
tmp_path,
)
assert result == "from src.mod import fn"
def test_import_hint_value_error(tmp_path: Path) -> None:
# project_root is different from codebase_path
other_root = tmp_path / "other"
other_root.mkdir()
result = _import_hint(
_item(file="src/mod.py", name="fn"),
str(tmp_path),
other_root,
)
assert result == "from mod import fn"
# ---------------------------------------------------------------------------
# _validate_tests_against_real
# ---------------------------------------------------------------------------
def test_validate_tests_syntax_error() -> None:
mock_console = MagicMock()
result = _validate_tests_against_real(
"def (broken):", "import fn", Path("/proj"), mock_console
)
assert result is False
def test_validate_tests_passes() -> None:
mock_console = MagicMock()
with patch("python_pkg.code_tutor._challenge._pytest_clean", return_value=True):
result = _validate_tests_against_real(
"def test_fn():\n assert True\n",
"# import",
Path("/proj"),
mock_console,
)
assert result is True
def test_validate_tests_fails() -> None:
mock_console = MagicMock()
with patch("python_pkg.code_tutor._challenge._pytest_clean", return_value=False):
result = _validate_tests_against_real(
"def test_fn():\n assert False\n",
"# import",
Path("/proj"),
mock_console,
)
assert result is False
# ---------------------------------------------------------------------------
# _collect_and_rate_tests
# ---------------------------------------------------------------------------
def test_collect_and_rate_tests_skip() -> None:
mock_backend = MagicMock()
mock_console = MagicMock()
live_mock = _make_live_mock()
with patch("python_pkg.code_tutor._challenge_support.Live", return_value=live_mock):
result = _collect_and_rate_tests(
"def fn():",
"explanation",
mock_backend,
mock_console,
lambda _: "skip",
)
assert result is None
def test_collect_and_rate_tests_pass_first() -> None:
mock_backend = MagicMock()
mock_console = MagicMock()
live_mock = _make_live_mock()
def fake_stream(system: str, user: str, on_token: object) -> str:
assert callable(on_token)
on_token('{"verdict": "PASS", "gap": ""}')
return ""
mock_backend.stream.side_effect = fake_stream
lines_iter = iter(["test code line", "END"])
with patch("python_pkg.code_tutor._challenge_support.Live", return_value=live_mock):
result = _collect_and_rate_tests(
"def fn():",
"explanation",
mock_backend,
mock_console,
lambda _: next(lines_iter),
)
assert result == "test code line"
def test_collect_and_rate_tests_fail_twice() -> None:
mock_backend = MagicMock()
mock_console = MagicMock()
live_mock = _make_live_mock()
mock_backend.stream.return_value = '{"verdict": "FAIL", "gap": "bad"}'
attempt_inputs = [
# attempt 1: provide code, then END
"test code",
"END",
# attempt 2: provide code, then END
"test code again",
"END",
]
inputs_iter = iter(attempt_inputs)
with patch("python_pkg.code_tutor._challenge_support.Live", return_value=live_mock):
result = _collect_and_rate_tests(
"def fn():",
"explanation",
mock_backend,
mock_console,
lambda _: next(inputs_iter),
)
assert result is None
def test_collect_and_rate_tests_fail_then_pass() -> None:
mock_backend = MagicMock()
mock_console = MagicMock()
live_mock = _make_live_mock()
call_count = [0]
def fake_stream(system: str, user: str, on_token: object) -> str:
assert callable(on_token)
call_count[0] += 1
verdict = (
'{"verdict": "FAIL", "gap": "not enough"}'
if call_count[0] == 1
else '{"verdict": "PASS", "gap": ""}'
)
on_token(verdict)
return ""
mock_backend.stream.side_effect = fake_stream
inputs = iter(["bad test", "END", "good test", "END"])
with patch("python_pkg.code_tutor._challenge_support.Live", return_value=live_mock):
result = _collect_and_rate_tests(
"def fn():",
"explanation",
mock_backend,
mock_console,
lambda _: next(inputs),
)
assert result == "good test"

View File

@ -0,0 +1,373 @@
"""Tests for python_pkg.code_tutor._challenge -- part 3.
Covers the user-implementation runner and the two challenge flows plus the
public entry point: ``_run_user_impl``, ``_write_tests_first_flow``,
``_existing_tests_flow`` and ``run_coding_challenge``.
"""
from __future__ import annotations
from typing import TYPE_CHECKING
from unittest.mock import MagicMock, patch
from python_pkg.code_tutor._challenge import (
_existing_tests_flow,
_run_user_impl,
_write_tests_first_flow,
run_coding_challenge,
)
from python_pkg.code_tutor.tests.conftest import _item, _make_live_mock
if TYPE_CHECKING:
from pathlib import Path
# ---------------------------------------------------------------------------
# _run_user_impl
# ---------------------------------------------------------------------------
def test_run_user_impl_skip(tmp_path: Path) -> None:
source = "def fn():\n return 1\n"
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
mock_console = MagicMock()
live_mock = _make_live_mock()
with (
patch("python_pkg.code_tutor._challenge_support.Live", return_value=live_mock),
patch("python_pkg.code_tutor._challenge.Syntax"),
):
result = _run_user_impl(
_item(file="mod.py"),
str(tmp_path),
"def test_fn(): pass",
"# import",
mock_console,
lambda _: "skip",
)
assert result == "skipped"
def test_run_user_impl_syntax_error(tmp_path: Path) -> None:
source = "def fn():\n return 1\n"
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
mock_console = MagicMock()
inputs = iter(["def (broken):", ""])
with patch("python_pkg.code_tutor._challenge.Syntax"):
result = _run_user_impl(
_item(file="mod.py"),
str(tmp_path),
"def test_fn(): pass",
"# import",
mock_console,
lambda _: next(inputs, "END") if _ == "" else "END",
)
# Provide the END and the broken implementation
inputs2 = iter(["def (broken):", "END"])
with patch("python_pkg.code_tutor._challenge.Syntax"):
result = _run_user_impl(
_item(file="mod.py"),
str(tmp_path),
"def test_fn(): pass",
"# import",
mock_console,
lambda _: next(inputs2),
)
assert result == "failed"
def test_run_user_impl_passed(tmp_path: Path) -> None:
source = "def fn():\n return 1\n"
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
mock_console = MagicMock()
inputs = iter(["def fn():", " return 42", "END"])
with (
patch("python_pkg.code_tutor._challenge.Syntax"),
patch("python_pkg.code_tutor._challenge._pytest_clean", return_value=True),
):
result = _run_user_impl(
_item(file="mod.py"),
str(tmp_path),
"def test_fn(): pass",
"# import",
mock_console,
lambda _: next(inputs),
)
assert result == "passed"
# File restored
assert (tmp_path / "mod.py").read_text(encoding="utf-8") == source
def test_run_user_impl_failed(tmp_path: Path) -> None:
source = "def fn():\n return 1\n"
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
mock_console = MagicMock()
inputs = iter(["def fn():", " return 0", "END"])
with (
patch("python_pkg.code_tutor._challenge.Syntax"),
patch("python_pkg.code_tutor._challenge._pytest_clean", return_value=False),
):
result = _run_user_impl(
_item(file="mod.py"),
str(tmp_path),
"def test_fn(): pass",
"# import",
mock_console,
lambda _: next(inputs),
)
assert result == "failed"
assert (tmp_path / "mod.py").read_text(encoding="utf-8") == source
# ---------------------------------------------------------------------------
# _write_tests_first_flow
# ---------------------------------------------------------------------------
def test_write_tests_first_flow_user_declines(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn(): pass\n", encoding="utf-8")
mock_backend = MagicMock()
mock_console = MagicMock()
result = _write_tests_first_flow(
_item(file="mod.py"),
str(tmp_path),
"explanation",
mock_backend,
mock_console,
lambda _: "n",
)
assert result == "skipped"
def test_write_tests_first_flow_tests_none(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn(): pass\n", encoding="utf-8")
mock_backend = MagicMock()
mock_console = MagicMock()
with (
patch(
"python_pkg.code_tutor._challenge._collect_and_rate_tests",
return_value=None,
),
patch("python_pkg.code_tutor._challenge.Syntax"),
):
result = _write_tests_first_flow(
_item(file="mod.py"),
str(tmp_path),
"explanation",
mock_backend,
mock_console,
lambda _: "y",
)
assert result == "skipped"
def test_write_tests_first_flow_tests_fail_real(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn(): pass\n", encoding="utf-8")
mock_backend = MagicMock()
mock_console = MagicMock()
with (
patch(
"python_pkg.code_tutor._challenge._collect_and_rate_tests",
return_value="test code",
),
patch(
"python_pkg.code_tutor._challenge._validate_tests_against_real",
return_value=False,
),
patch("python_pkg.code_tutor._challenge.Syntax"),
):
result = _write_tests_first_flow(
_item(file="mod.py"),
str(tmp_path),
"explanation",
mock_backend,
mock_console,
lambda _: "y",
)
assert result == "skipped"
def test_write_tests_first_flow_success(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn(): pass\n", encoding="utf-8")
mock_backend = MagicMock()
mock_console = MagicMock()
with (
patch(
"python_pkg.code_tutor._challenge._collect_and_rate_tests",
return_value="test code",
),
patch(
"python_pkg.code_tutor._challenge._validate_tests_against_real",
return_value=True,
),
patch(
"python_pkg.code_tutor._challenge._run_user_impl",
return_value="passed",
),
patch("python_pkg.code_tutor._challenge.Syntax"),
):
result = _write_tests_first_flow(
_item(file="mod.py"),
str(tmp_path),
"explanation",
mock_backend,
mock_console,
lambda _: "y",
)
assert result == "passed"
# ---------------------------------------------------------------------------
# _existing_tests_flow
# ---------------------------------------------------------------------------
def test_existing_tests_flow_user_declines(tmp_path: Path) -> None:
test_file = tmp_path / "test_mod.py"
test_file.write_text("def test_fn(): fn()\n", encoding="utf-8")
mock_console = MagicMock()
result = _existing_tests_flow(
_item(),
str(tmp_path),
"explanation",
[(test_file, ["test_fn"])],
mock_console,
lambda _: "n",
)
assert result == "skipped"
def test_existing_tests_flow_user_skips_impl(tmp_path: Path) -> None:
source = "def fn():\n return 1\n"
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
test_file = tmp_path / "test_mod.py"
test_file.write_text("def test_fn(): fn()\n", encoding="utf-8")
mock_console = MagicMock()
with patch("python_pkg.code_tutor._challenge_support.Syntax"):
result = _existing_tests_flow(
_item(file="mod.py"),
str(tmp_path),
"explanation",
[(test_file, ["test_fn"])],
mock_console,
lambda _: (
"y"
if "challenge" in str(_) or _ == "Take the challenge? [y/N] "
else "skip"
),
)
assert result == "skipped"
def test_existing_tests_flow_passes(tmp_path: Path) -> None:
source = "def fn():\n return 1\n"
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
test_file = tmp_path / "test_mod.py"
test_file.write_text("def test_fn(): fn()\n", encoding="utf-8")
mock_console = MagicMock()
inputs = iter(["y", "def fn():", " return 1", "END"])
with (
patch("python_pkg.code_tutor._challenge_support.Syntax"),
patch("python_pkg.code_tutor._challenge._patch_and_test", return_value=True),
):
result = _existing_tests_flow(
_item(file="mod.py"),
str(tmp_path),
"explanation",
[(test_file, ["test_fn"])],
mock_console,
lambda _: next(inputs),
)
assert result == "passed"
def test_existing_tests_flow_fails(tmp_path: Path) -> None:
source = "def fn():\n return 1\n"
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
test_file = tmp_path / "test_mod.py"
test_file.write_text("def test_fn(): fn()\n", encoding="utf-8")
mock_console = MagicMock()
inputs = iter(["y", "def fn():", " return 0", "END"])
with (
patch("python_pkg.code_tutor._challenge_support.Syntax"),
patch("python_pkg.code_tutor._challenge._patch_and_test", return_value=False),
):
result = _existing_tests_flow(
_item(file="mod.py"),
str(tmp_path),
"explanation",
[(test_file, ["test_fn"])],
mock_console,
lambda _: next(inputs),
)
assert result == "failed"
# ---------------------------------------------------------------------------
# run_coding_challenge
# ---------------------------------------------------------------------------
def test_run_coding_challenge_non_python() -> None:
mock_console = MagicMock()
mock_backend = MagicMock()
result = run_coding_challenge(
_item(file="main.go"),
"/codebase",
"explanation",
mock_backend,
mock_console,
)
assert result == "skipped"
def test_run_coding_challenge_with_existing_tests(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn(): pass\n", encoding="utf-8")
test_file = tmp_path / "test_mod.py"
test_file.write_text("def test_fn(): fn()\n", encoding="utf-8")
mock_console = MagicMock()
mock_backend = MagicMock()
with patch(
"python_pkg.code_tutor._challenge._existing_tests_flow", return_value="passed"
):
result = run_coding_challenge(
_item(file="mod.py"),
str(tmp_path),
"explanation",
mock_backend,
mock_console,
)
assert result == "passed"
def test_run_coding_challenge_no_tests(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn(): pass\n", encoding="utf-8")
mock_console = MagicMock()
mock_backend = MagicMock()
with patch(
"python_pkg.code_tutor._challenge._write_tests_first_flow",
return_value="skipped",
):
result = run_coding_challenge(
_item(file="mod.py"),
str(tmp_path),
"explanation",
mock_backend,
mock_console,
)
assert result == "skipped"

View File

@ -0,0 +1,470 @@
"""Tests for python_pkg.code_tutor.cli."""
from __future__ import annotations
import json
from typing import TYPE_CHECKING
from unittest.mock import MagicMock, patch
import requests
from typer.testing import CliRunner
from python_pkg.code_tutor.cli import (
_check_plan_file,
_ensure_fresh_plan,
_ensure_ollama_running,
_find_codebase_for_file,
app,
)
if TYPE_CHECKING:
from pathlib import Path
runner = CliRunner()
def _make_plan(codebase: Path, fingerprint: str = "abc") -> dict:
return {
"codebase_path": str(codebase),
"created_at": "2026-01-01T00:00:00+00:00",
"total_items": 2,
"source_fingerprint": fingerprint,
"sessions": [
{
"id": 1,
"title": "mod",
"items": [
{
"id": "mod.fn",
"file": "mod.py",
"type": "function",
"name": "fn",
"start_line": 1,
"end_line": 3,
"class_name": "",
"depends_on": [],
}
],
}
],
}
# ---------------------------------------------------------------------------
# _ensure_ollama_running
# ---------------------------------------------------------------------------
def test_ensure_ollama_running_already_up() -> None:
mock_console = MagicMock()
mock_resp = MagicMock()
with patch("python_pkg.code_tutor.cli.requests.get", return_value=mock_resp):
result = _ensure_ollama_running(mock_console)
assert result is True
def test_ensure_ollama_running_systemctl_fails() -> None:
import subprocess
mock_console = MagicMock()
with (
patch(
"python_pkg.code_tutor.cli.requests.get",
side_effect=requests.exceptions.ConnectionError(),
),
patch(
"python_pkg.code_tutor.cli.subprocess.run",
side_effect=subprocess.CalledProcessError(1, "systemctl", stderr=b"failed"),
),
):
result = _ensure_ollama_running(mock_console)
assert result is False
def test_ensure_ollama_running_starts_then_up() -> None:
mock_console = MagicMock()
mock_subprocess = MagicMock()
request_calls = [0]
def fake_get(*args: object, **kwargs: object) -> MagicMock:
request_calls[0] += 1
if request_calls[0] == 1:
raise requests.exceptions.ConnectionError
return MagicMock()
with (
patch("python_pkg.code_tutor.cli.requests.get", side_effect=fake_get),
patch("python_pkg.code_tutor.cli.subprocess.run", return_value=mock_subprocess),
patch("python_pkg.code_tutor.cli.time.sleep"),
):
result = _ensure_ollama_running(mock_console)
assert result is True
def test_ensure_ollama_running_times_out() -> None:
mock_console = MagicMock()
mock_subprocess = MagicMock()
time_values = iter(
[0, 0, 100]
) # deadline=30, first while check→0 (enter), next→100 (exit)
def fake_get(*args: object, **kwargs: object) -> MagicMock:
raise requests.exceptions.ConnectionError
with (
patch("python_pkg.code_tutor.cli.requests.get", side_effect=fake_get),
patch("python_pkg.code_tutor.cli.subprocess.run", return_value=mock_subprocess),
patch(
"python_pkg.code_tutor.cli.time.monotonic",
side_effect=time_values,
),
patch("python_pkg.code_tutor.cli.time.sleep"),
):
result = _ensure_ollama_running(mock_console)
assert result is False
# ---------------------------------------------------------------------------
# _ensure_fresh_plan
# ---------------------------------------------------------------------------
def test_ensure_fresh_plan_no_fingerprint(tmp_path: Path) -> None:
plan = {
"codebase_path": str(tmp_path),
"created_at": "t",
"total_items": 0,
"source_fingerprint": "",
"sessions": [],
}
mock_console = MagicMock()
result = _ensure_fresh_plan(tmp_path, plan, mock_console)
assert result is plan
def test_ensure_fresh_plan_up_to_date(tmp_path: Path) -> None:
plan = _make_plan(tmp_path, fingerprint="same")
mock_console = MagicMock()
with patch("python_pkg.code_tutor.cli.codebase_fingerprint", return_value="same"):
result = _ensure_fresh_plan(tmp_path, plan, mock_console)
assert result is plan
def test_ensure_fresh_plan_stale_no_items(tmp_path: Path) -> None:
plan = _make_plan(tmp_path, fingerprint="old")
mock_console = MagicMock()
with (
patch("python_pkg.code_tutor.cli.codebase_fingerprint", return_value="new"),
patch("python_pkg.code_tutor.cli.extract_items", return_value=[]),
):
result = _ensure_fresh_plan(tmp_path, plan, mock_console)
assert result is plan
def test_ensure_fresh_plan_stale_rebuilds(tmp_path: Path) -> None:
plan = _make_plan(tmp_path, fingerprint="old")
new_plan = _make_plan(tmp_path, fingerprint="new")
mock_console = MagicMock()
mock_items = [MagicMock()]
with (
patch("python_pkg.code_tutor.cli.codebase_fingerprint", return_value="new"),
patch("python_pkg.code_tutor.cli.extract_items", return_value=mock_items),
patch("python_pkg.code_tutor.cli.build_plan", return_value=new_plan),
patch("python_pkg.code_tutor.cli.save_plan"),
):
result = _ensure_fresh_plan(tmp_path, plan, mock_console)
assert result["source_fingerprint"] == "new"
# ---------------------------------------------------------------------------
# analyze command
# ---------------------------------------------------------------------------
def test_analyze_not_dir(tmp_path: Path) -> None:
result = runner.invoke(app, ["analyze", str(tmp_path / "no_such_dir")])
assert result.exit_code == 1
def test_analyze_no_items(tmp_path: Path) -> None:
with patch("python_pkg.code_tutor.cli.extract_items", return_value=[]):
result = runner.invoke(app, ["analyze", str(tmp_path)])
assert result.exit_code == 0
def test_analyze_with_items(tmp_path: Path) -> None:
mock_items = [MagicMock()]
mock_plan = _make_plan(tmp_path)
with (
patch("python_pkg.code_tutor.cli.extract_items", return_value=mock_items),
patch("python_pkg.code_tutor.cli.build_plan", return_value=mock_plan),
patch("python_pkg.code_tutor.cli.save_plan"),
patch("python_pkg.code_tutor.cli.config_dir", return_value=tmp_path),
):
result = runner.invoke(app, ["analyze", str(tmp_path)])
assert result.exit_code == 0
# ---------------------------------------------------------------------------
# study command
# ---------------------------------------------------------------------------
def test_study_not_dir(tmp_path: Path) -> None:
result = runner.invoke(app, ["study", str(tmp_path / "no_such_dir")])
assert result.exit_code == 1
def test_study_no_plan(tmp_path: Path) -> None:
with patch("python_pkg.code_tutor.cli.load_plan", return_value=None):
result = runner.invoke(app, ["study", str(tmp_path)])
assert result.exit_code == 1
def test_study_ollama_fails(tmp_path: Path) -> None:
plan = _make_plan(tmp_path)
with (
patch("python_pkg.code_tutor.cli.load_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_fresh_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_ollama_running", return_value=False),
):
result = runner.invoke(app, ["study", str(tmp_path)])
assert result.exit_code == 1
def test_study_connection_error(tmp_path: Path) -> None:
plan = _make_plan(tmp_path)
with (
patch("python_pkg.code_tutor.cli.load_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_fresh_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_ollama_running", return_value=True),
patch(
"python_pkg.code_tutor.cli.run_session",
side_effect=requests.exceptions.ConnectionError(),
),
):
result = runner.invoke(app, ["study", str(tmp_path)])
assert result.exit_code == 1
def test_study_success(tmp_path: Path) -> None:
plan = _make_plan(tmp_path)
with (
patch("python_pkg.code_tutor.cli.load_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_fresh_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_ollama_running", return_value=True),
patch("python_pkg.code_tutor.cli.run_session"),
):
result = runner.invoke(app, ["study", str(tmp_path)])
assert result.exit_code == 0
# ---------------------------------------------------------------------------
# status command
# ---------------------------------------------------------------------------
def test_status_no_plan(tmp_path: Path) -> None:
with patch("python_pkg.code_tutor.cli.load_plan", return_value=None):
result = runner.invoke(app, ["status", str(tmp_path)])
assert result.exit_code == 1
def test_status_with_plan(tmp_path: Path) -> None:
plan = _make_plan(tmp_path)
progress = {"learned": [], "struggled": [], "skipped": [], "last_session": ""}
with (
patch("python_pkg.code_tutor.cli.load_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_fresh_plan", return_value=plan),
patch("python_pkg.code_tutor.cli.load_progress", return_value=progress),
):
result = runner.invoke(app, ["status", str(tmp_path)])
assert result.exit_code == 0
# ---------------------------------------------------------------------------
# drill command
# ---------------------------------------------------------------------------
def test_drill_no_plan_for_file(tmp_path: Path) -> None:
f = tmp_path / "mod.py"
f.write_text("def fn(): pass\n", encoding="utf-8")
with patch("python_pkg.code_tutor.cli._find_codebase_for_file", return_value=None):
result = runner.invoke(app, ["drill", str(f)])
assert result.exit_code == 1
def test_drill_plan_disappears(tmp_path: Path) -> None:
f = tmp_path / "mod.py"
f.write_text("def fn(): pass\n", encoding="utf-8")
with (
patch(
"python_pkg.code_tutor.cli._find_codebase_for_file", return_value=tmp_path
),
patch("python_pkg.code_tutor.cli.load_plan", return_value=None),
):
result = runner.invoke(app, ["drill", str(f)])
assert result.exit_code == 1
def test_drill_no_items_in_plan(tmp_path: Path) -> None:
f = tmp_path / "other.py"
f.write_text("def fn(): pass\n", encoding="utf-8")
plan = _make_plan(tmp_path) # plan has mod.py items, not other.py
with (
patch(
"python_pkg.code_tutor.cli._find_codebase_for_file", return_value=tmp_path
),
patch("python_pkg.code_tutor.cli.load_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_fresh_plan", return_value=plan),
):
result = runner.invoke(app, ["drill", str(f)])
assert result.exit_code == 0
def test_drill_ollama_fails(tmp_path: Path) -> None:
f = tmp_path / "mod.py"
f.write_text("def fn(): pass\n", encoding="utf-8")
plan = _make_plan(tmp_path)
with (
patch(
"python_pkg.code_tutor.cli._find_codebase_for_file", return_value=tmp_path
),
patch("python_pkg.code_tutor.cli.load_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_fresh_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_ollama_running", return_value=False),
):
result = runner.invoke(app, ["drill", str(f)])
assert result.exit_code == 1
def test_drill_runs_lesson(tmp_path: Path) -> None:
from python_pkg.code_tutor._progress import LessonRecord
f = tmp_path / "mod.py"
f.write_text("def fn(): pass\n", encoding="utf-8")
plan = _make_plan(tmp_path)
record = LessonRecord(
timestamp="t",
item_id="mod.fn",
file="mod.py",
lines="1-3",
snippet="code",
outcome="learned",
answers={},
improvement="",
verdict="PASS",
attempt=1,
)
mock_verifier = MagicMock()
mock_verifier.run_lesson.return_value = record
with (
patch(
"python_pkg.code_tutor.cli._find_codebase_for_file", return_value=tmp_path
),
patch("python_pkg.code_tutor.cli.load_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_fresh_plan", return_value=plan),
patch("python_pkg.code_tutor.cli._ensure_ollama_running", return_value=True),
patch("python_pkg.code_tutor.cli.Verifier", return_value=mock_verifier),
patch("python_pkg.code_tutor.cli.append_session_record"),
):
result = runner.invoke(app, ["drill", str(f)])
assert result.exit_code == 0
mock_verifier.run_lesson.assert_called_once()
# ---------------------------------------------------------------------------
# _find_codebase_for_file
# ---------------------------------------------------------------------------
def test_find_codebase_for_file_no_config_dir(tmp_path: Path) -> None:
with patch(
"python_pkg.code_tutor.cli.Path.home",
return_value=tmp_path,
):
result = _find_codebase_for_file(tmp_path / "mod.py")
assert result is None
def test_find_codebase_for_file_found(tmp_path: Path) -> None:
home = tmp_path / "home"
config_root = home / ".config" / "code_tutor" / "hash"
config_root.mkdir(parents=True)
codebase = tmp_path / "codebase"
codebase.mkdir()
f = codebase / "mod.py"
f.write_text("def fn(): pass\n", encoding="utf-8")
plan_data = {
"codebase_path": str(codebase),
"sessions": [],
}
(config_root / "plan.json").write_text(json.dumps(plan_data), encoding="utf-8")
with patch("python_pkg.code_tutor.cli.Path.home", return_value=home):
result = _find_codebase_for_file(f)
assert result == codebase
def test_find_codebase_for_file_not_found(tmp_path: Path) -> None:
home = tmp_path / "home"
config_root = home / ".config" / "code_tutor" / "hash"
config_root.mkdir(parents=True)
other_codebase = tmp_path / "other"
other_codebase.mkdir()
plan_data = {"codebase_path": str(other_codebase), "sessions": []}
(config_root / "plan.json").write_text(json.dumps(plan_data), encoding="utf-8")
target = tmp_path / "unrelated" / "mod.py"
with patch("python_pkg.code_tutor.cli.Path.home", return_value=home):
result = _find_codebase_for_file(target)
assert result is None
# ---------------------------------------------------------------------------
# _check_plan_file
# ---------------------------------------------------------------------------
def test_check_plan_file_found(tmp_path: Path) -> None:
codebase = tmp_path / "cb"
codebase.mkdir()
f = codebase / "mod.py"
plan_file = tmp_path / "plan.json"
plan_file.write_text(json.dumps({"codebase_path": str(codebase)}), encoding="utf-8")
result = _check_plan_file(plan_file, f)
assert result == codebase
def test_check_plan_file_value_error(tmp_path: Path) -> None:
codebase = tmp_path / "cb"
codebase.mkdir()
other_file = tmp_path / "other" / "mod.py"
plan_file = tmp_path / "plan.json"
plan_file.write_text(json.dumps({"codebase_path": str(codebase)}), encoding="utf-8")
# other_file is not relative to codebase → ValueError
result = _check_plan_file(plan_file, other_file)
assert result is None
def test_check_plan_file_oserror(tmp_path: Path) -> None:
plan_file = tmp_path / "missing_plan.json"
result = _check_plan_file(plan_file, tmp_path / "mod.py")
assert result is None
def test_check_plan_file_json_error(tmp_path: Path) -> None:
plan_file = tmp_path / "plan.json"
plan_file.write_text("not valid json {{{", encoding="utf-8")
result = _check_plan_file(plan_file, tmp_path / "mod.py")
assert result is None

View File

@ -0,0 +1,320 @@
"""Tests for python_pkg.code_tutor._llm."""
from __future__ import annotations
import json
from unittest.mock import MagicMock, patch
import pytest
from python_pkg.code_tutor._llm import (
ClaudeBackend,
OllamaBackend,
_is_weak,
complete_with_fallback,
)
# ---------------------------------------------------------------------------
# OllamaBackend.complete
# ---------------------------------------------------------------------------
def test_ollama_complete_success() -> None:
mock_resp = MagicMock()
mock_resp.json.return_value = {"response": "hello world"}
with patch("python_pkg.code_tutor._llm.requests.post", return_value=mock_resp):
backend = OllamaBackend()
result = backend.complete("sys", "user")
assert result == "hello world"
mock_resp.raise_for_status.assert_called_once()
def test_ollama_complete_missing_key() -> None:
mock_resp = MagicMock()
mock_resp.json.return_value = {}
with patch("python_pkg.code_tutor._llm.requests.post", return_value=mock_resp):
backend = OllamaBackend()
result = backend.complete("sys", "user")
assert result == ""
# ---------------------------------------------------------------------------
# OllamaBackend.stream
# ---------------------------------------------------------------------------
def test_ollama_stream_basic() -> None:
lines = [
json.dumps({"response": "hello", "done": False}).encode(),
b"", # empty line -> skipped
json.dumps({"response": " world", "done": False}).encode(),
json.dumps({"response": "", "done": True}).encode(),
]
mock_resp = MagicMock()
mock_resp.iter_lines.return_value = iter(lines)
tokens: list[str] = []
with patch("python_pkg.code_tutor._llm.requests.post", return_value=mock_resp):
backend = OllamaBackend()
result = backend.stream("sys", "user", tokens.append)
assert result == "hello world"
assert tokens == ["hello", " world"]
def test_ollama_stream_empty_token_not_appended() -> None:
lines = [
json.dumps({"response": "", "done": False}).encode(),
json.dumps({"response": "ok", "done": True}).encode(),
]
mock_resp = MagicMock()
mock_resp.iter_lines.return_value = iter(lines)
tokens: list[str] = []
with patch("python_pkg.code_tutor._llm.requests.post", return_value=mock_resp):
backend = OllamaBackend()
result = backend.stream("sys", "user", tokens.append)
assert result == "ok"
assert tokens == ["ok"]
# ---------------------------------------------------------------------------
# ClaudeBackend.complete
# ---------------------------------------------------------------------------
def test_claude_complete_with_text_block() -> None:
mock_block = MagicMock()
mock_block.text = "answer"
mock_message = MagicMock()
mock_message.content = [mock_block]
mock_client = MagicMock()
mock_client.messages.create.return_value = mock_message
with patch(
"python_pkg.code_tutor._llm.anthropic.Anthropic", return_value=mock_client
):
backend = ClaudeBackend()
result = backend.complete("sys", "user")
assert result == "answer"
def test_claude_complete_no_text_attr() -> None:
mock_block = MagicMock(spec=[]) # no .text attribute
mock_message = MagicMock()
mock_message.content = [mock_block]
mock_client = MagicMock()
mock_client.messages.create.return_value = mock_message
with patch(
"python_pkg.code_tutor._llm.anthropic.Anthropic", return_value=mock_client
):
backend = ClaudeBackend()
result = backend.complete("sys", "user")
assert result == ""
# ---------------------------------------------------------------------------
# ClaudeBackend.stream
# ---------------------------------------------------------------------------
def test_claude_stream_success() -> None:
mock_stream_ctx = MagicMock()
mock_stream_ctx.__enter__ = MagicMock(return_value=mock_stream_ctx)
mock_stream_ctx.__exit__ = MagicMock(return_value=False)
mock_stream_ctx.text_stream = iter(["hello", " world"])
mock_client = MagicMock()
mock_client.messages.stream.return_value = mock_stream_ctx
tokens: list[str] = []
with patch(
"python_pkg.code_tutor._llm.anthropic.Anthropic", return_value=mock_client
):
backend = ClaudeBackend()
result = backend.stream("sys", "user", tokens.append)
assert result == "hello world"
assert tokens == ["hello", " world"]
def test_claude_stream_api_error_fallback() -> None:
import anthropic as anthropic_lib
mock_client = MagicMock()
mock_client.messages.stream.side_effect = anthropic_lib.APIError(
message="fail",
request=MagicMock(),
body=None,
)
mock_block = MagicMock()
mock_block.text = "fallback"
mock_message = MagicMock()
mock_message.content = [mock_block]
mock_client.messages.create.return_value = mock_message
tokens: list[str] = []
with patch(
"python_pkg.code_tutor._llm.anthropic.Anthropic", return_value=mock_client
):
backend = ClaudeBackend()
result = backend.stream("sys", "user", tokens.append)
assert result == "fallback"
# ---------------------------------------------------------------------------
# _is_weak
# ---------------------------------------------------------------------------
def test_is_weak_short() -> None:
assert _is_weak("ok") is True
def test_is_weak_phrase() -> None:
assert _is_weak("I don't know what this code does at all honestly") is True
def test_is_weak_strong_response() -> None:
long_response = (
"This function reads the first 512 bytes of a file and checks for"
" null bytes to determine if the file is binary."
)
assert _is_weak(long_response) is False
# ---------------------------------------------------------------------------
# complete_with_fallback
# ---------------------------------------------------------------------------
def test_complete_with_fallback_strong() -> None:
mock_primary = MagicMock(spec=OllamaBackend)
mock_primary.complete.return_value = (
"This is a detailed response explaining the code in full."
)
mock_console = MagicMock()
result = complete_with_fallback(
mock_primary, "sys", "user", console=mock_console, input_fn=lambda _: ""
)
assert result == "This is a detailed response explaining the code in full."
mock_primary.complete.assert_called_once()
def test_complete_with_fallback_weak_user_says_yes_claude_ok() -> None:
mock_primary = MagicMock(spec=OllamaBackend)
mock_primary.complete.return_value = "ok" # short → weak
mock_block = MagicMock()
mock_block.text = "claude response is great"
mock_message = MagicMock()
mock_message.content = [mock_block]
mock_client = MagicMock()
mock_client.messages.create.return_value = mock_message
mock_console = MagicMock()
with patch(
"python_pkg.code_tutor._llm.anthropic.Anthropic", return_value=mock_client
):
result = complete_with_fallback(
mock_primary, "sys", "user", console=mock_console, input_fn=lambda _: "y"
)
assert result == "claude response is great"
def test_complete_with_fallback_weak_user_says_yes_claude_fails() -> None:
import anthropic as anthropic_lib
mock_primary = MagicMock(spec=OllamaBackend)
mock_primary.complete.side_effect = [
"ok", # first call → weak
"retry response from ollama", # retry after claude failure
]
mock_client = MagicMock()
mock_client.messages.create.side_effect = anthropic_lib.APIError(
message="fail",
request=MagicMock(),
body=None,
)
mock_console = MagicMock()
with patch(
"python_pkg.code_tutor._llm.anthropic.Anthropic", return_value=mock_client
):
result = complete_with_fallback(
mock_primary, "sys", "user", console=mock_console, input_fn=lambda _: "y"
)
assert result == "retry response from ollama"
assert mock_primary.complete.call_count == 2
def test_complete_with_fallback_weak_user_says_no() -> None:
mock_primary = MagicMock(spec=OllamaBackend)
mock_primary.complete.side_effect = [
"ok", # first call → weak
"second ollama response here too", # retry
]
mock_console = MagicMock()
result = complete_with_fallback(
mock_primary, "sys", "user", console=mock_console, input_fn=lambda _: "n"
)
assert result == "second ollama response here too"
assert mock_primary.complete.call_count == 2
def test_ollama_backend_defaults() -> None:
backend = OllamaBackend()
assert "11434" in backend._url
assert backend._timeout > 0
@pytest.mark.parametrize(
"phrase",
[
"i cannot help with that request really",
"it is unclear what you mean by that",
"i'm not sure about this code",
],
)
def test_is_weak_various_phrases(phrase: str) -> None:
assert _is_weak(phrase) is True
def test_ollama_stream_no_done_marker() -> None:
"""Exhaust the stream loop without a ``done`` marker (no ``break``).
When no line carries ``done: true`` the loop runs to exhaustion and
falls through to ``return "".join(parts)`` rather than breaking early.
"""
lines = [
json.dumps({"response": "foo", "done": False}).encode(),
json.dumps({"response": "bar", "done": False}).encode(),
]
mock_resp = MagicMock()
mock_resp.iter_lines.return_value = iter(lines)
tokens: list[str] = []
with patch("python_pkg.code_tutor._llm.requests.post", return_value=mock_resp):
backend = OllamaBackend()
result = backend.stream("sys", "user", tokens.append)
assert result == "foobar"
assert tokens == ["foo", "bar"]

View File

@ -0,0 +1,177 @@
"""Tests for python_pkg.code_tutor._plan_builder."""
from __future__ import annotations
from pathlib import Path
from unittest.mock import patch
from python_pkg.code_tutor._analyzer import CodeItem
from python_pkg.code_tutor._plan_builder import (
_order_items,
_session_title,
_toposort_files,
build_plan,
)
def _item(file: str, name: str = "fn", start: int = 1, end: int = 5) -> CodeItem:
return CodeItem(
id=f"{file}.{name}",
file=file,
type="function",
name=name,
start_line=start,
end_line=end,
)
# ---------------------------------------------------------------------------
# _toposort_files
# ---------------------------------------------------------------------------
def test_toposort_files_no_deps() -> None:
a = Path("a.py")
b = Path("b.py")
deps = {a: set(), b: set()}
result = _toposort_files(deps)
assert set(result) == {a, b}
def test_toposort_files_with_deps() -> None:
a = Path("a.py")
b = Path("b.py")
deps = {a: set(), b: {a}}
result = _toposort_files(deps)
assert result.index(a) < result.index(b)
def test_toposort_files_cycle() -> None:
a = Path("a.py")
b = Path("b.py")
deps = {a: {b}, b: {a}}
result = _toposort_files(deps)
assert set(result) == {a, b}
# ---------------------------------------------------------------------------
# _order_items
# ---------------------------------------------------------------------------
def test_order_items_empty(tmp_path: Path) -> None:
assert _order_items([], tmp_path) == []
def test_order_items_no_py_files(tmp_path: Path) -> None:
items = [_item("main.go"), _item("lib.go")]
result = _order_items(items, tmp_path)
assert len(result) == 2
def test_order_items_py_files(tmp_path: Path) -> None:
(tmp_path / "a.py").write_text("def fn(): pass\n", encoding="utf-8")
(tmp_path / "b.py").write_text("import a\ndef fn2(): pass\n", encoding="utf-8")
items = [_item("a.py"), _item("b.py", name="fn2")]
result = _order_items(items, tmp_path)
assert len(result) == 2
def test_order_items_mixed(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn(): pass\n", encoding="utf-8")
items = [_item("mod.py"), _item("script.go")]
result = _order_items(items, tmp_path)
assert len(result) == 2
def test_order_items_toposort_file_outside_codebase(tmp_path: Path) -> None:
"""A toposorted file outside the codebase root is skipped.
Covers the ``is_relative_to`` False branch: an import can resolve to a
path outside the codebase, which must not be added to the ordering map.
"""
module = "python_pkg.code_tutor._plan_builder"
outside = Path("/somewhere/else/x.py")
items = [_item("a.py")]
with (
patch(f"{module}.get_python_files", return_value=[tmp_path / "a.py"]),
patch(f"{module}.get_file_imports", return_value=set()),
patch(f"{module}._toposort_files", return_value=[outside]),
):
result = _order_items(items, tmp_path)
assert len(result) == 1
# ---------------------------------------------------------------------------
# _session_title
# ---------------------------------------------------------------------------
def test_session_title_empty() -> None:
assert _session_title([]) == "untitled"
def test_session_title_single() -> None:
items = [_item("src/main.py")]
assert _session_title(items) == "src"
def test_session_title_same_dir() -> None:
items = [_item("src/a.py"), _item("src/b.py")]
assert _session_title(items) == "src"
def test_session_title_common_parent_root(tmp_path: Path) -> None:
# parent of "src" is ".", so common becomes "."
items = [_item("src/a.py"), _item("src/b/c.py")]
result = _session_title(items)
# common becomes "." because parent of "src" is Path(".")
assert result == "a"
def test_session_title_common_parent_equals_common(tmp_path: Path) -> None:
# dirs differ: common goes up from "pkg/sub" to parent "pkg"
items = [_item("pkg/sub/a.py"), _item("pkg/sub/deeper/b.py")]
result = _session_title(items)
assert result == "pkg"
def test_session_title_deeper_common(tmp_path: Path) -> None:
# Two items in different subdirs share a common parent that is not "."
items = [_item("pkg/a/x.py"), _item("pkg/b/y.py")]
result = _session_title(items)
assert result == "pkg"
def test_session_title_root_files() -> None:
# Both files at root level: parent = ".", common = "."
items = [_item("a.py"), _item("b.py")]
result = _session_title(items)
assert result == "a"
# ---------------------------------------------------------------------------
# build_plan
# ---------------------------------------------------------------------------
def test_build_plan_basic(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn(): pass\n", encoding="utf-8")
items = [_item("mod.py")]
plan = build_plan(tmp_path, items)
assert plan["total_items"] == 1
assert len(plan["sessions"]) == 1
assert "source_fingerprint" in plan
def test_build_plan_session_size(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn(): pass\n", encoding="utf-8")
items = [_item("mod.py", name=f"fn{i}", start=i, end=i + 1) for i in range(5)]
plan = build_plan(tmp_path, items, session_size=2)
assert len(plan["sessions"]) == 3
def test_build_plan_empty_items(tmp_path: Path) -> None:
plan = build_plan(tmp_path, [])
assert plan["total_items"] == 0
assert plan["sessions"] == []

View File

@ -0,0 +1,173 @@
"""Tests for python_pkg.code_tutor._progress."""
from __future__ import annotations
from pathlib import Path
from unittest.mock import patch
from python_pkg.code_tutor._progress import (
LessonRecord,
append_session_record,
config_dir,
item_from_data,
load_plan,
load_progress,
save_plan,
save_progress,
)
def _fake_home(tmp_path: Path) -> Path:
home = tmp_path / "home"
home.mkdir()
return home
# ---------------------------------------------------------------------------
# config_dir
# ---------------------------------------------------------------------------
def test_config_dir_returns_path(tmp_path: Path) -> None:
with patch(
"python_pkg.code_tutor._progress.Path.home", return_value=_fake_home(tmp_path)
):
result = config_dir(tmp_path / "codebase")
assert isinstance(result, Path)
assert "code_tutor" in str(result)
# ---------------------------------------------------------------------------
# load_plan / save_plan
# ---------------------------------------------------------------------------
def test_load_plan_missing(tmp_path: Path) -> None:
with patch(
"python_pkg.code_tutor._progress.Path.home", return_value=_fake_home(tmp_path)
):
result = load_plan(tmp_path / "cb")
assert result is None
def test_save_and_load_plan(tmp_path: Path) -> None:
home = _fake_home(tmp_path)
codebase = tmp_path / "cb"
plan: dict[str, object] = {
"codebase_path": str(codebase),
"created_at": "2026-01-01T00:00:00+00:00",
"total_items": 0,
"source_fingerprint": "abc",
"sessions": [],
}
with patch("python_pkg.code_tutor._progress.Path.home", return_value=home):
save_plan(codebase, plan)
loaded = load_plan(codebase)
assert loaded is not None
assert loaded["total_items"] == 0
# ---------------------------------------------------------------------------
# load_progress / save_progress
# ---------------------------------------------------------------------------
def test_load_progress_missing(tmp_path: Path) -> None:
with patch(
"python_pkg.code_tutor._progress.Path.home", return_value=_fake_home(tmp_path)
):
result = load_progress(tmp_path / "cb")
assert result["learned"] == []
assert result["struggled"] == []
assert result["skipped"] == []
assert result["last_session"] == ""
def test_save_and_load_progress(tmp_path: Path) -> None:
home = _fake_home(tmp_path)
codebase = tmp_path / "cb"
progress = {
"learned": ["a.fn"],
"struggled": [],
"skipped": ["b.fn"],
"last_session": "2026-01-01T00:00:00+00:00",
}
with patch("python_pkg.code_tutor._progress.Path.home", return_value=home):
save_progress(codebase, progress)
loaded = load_progress(codebase)
assert loaded["learned"] == ["a.fn"]
assert loaded["skipped"] == ["b.fn"]
# ---------------------------------------------------------------------------
# append_session_record
# ---------------------------------------------------------------------------
def test_append_session_record(tmp_path: Path) -> None:
home = _fake_home(tmp_path)
codebase = tmp_path / "cb"
record = LessonRecord(
timestamp="2026-01-01T00:00:00+00:00",
item_id="mod.fn",
file="mod.py",
lines="1-10",
snippet="def fn(): pass",
outcome="learned",
answers={"Purpose": "does stuff"},
improvement="",
verdict="PASS",
attempt=1,
)
with patch("python_pkg.code_tutor._progress.Path.home", return_value=home):
append_session_record(codebase, record)
cfg = config_dir(codebase)
sessions_dir = cfg / "sessions"
assert sessions_dir.exists()
jsonl_files = list(sessions_dir.glob("*.jsonl"))
assert len(jsonl_files) == 1
content = jsonl_files[0].read_text(encoding="utf-8")
assert "mod.fn" in content
# ---------------------------------------------------------------------------
# item_from_data
# ---------------------------------------------------------------------------
def test_item_from_data() -> None:
data = {
"id": "mod.fn",
"file": "mod.py",
"type": "function",
"name": "fn",
"start_line": 1,
"end_line": 5,
"class_name": "",
"depends_on": [],
}
item = item_from_data(data)
assert item.id == "mod.fn"
assert item.name == "fn"
assert item.start_line == 1
# ---------------------------------------------------------------------------
# LessonRecord default challenge_result
# ---------------------------------------------------------------------------
def test_lesson_record_defaults() -> None:
record = LessonRecord(
timestamp="t",
item_id="i",
file="f.py",
lines="1-2",
snippet="code",
outcome="learned",
answers={},
improvement="",
verdict="PASS",
attempt=1,
)
assert record.challenge_result == ""

View File

@ -0,0 +1,201 @@
"""Tests for python_pkg.code_tutor._session."""
from __future__ import annotations
from typing import TYPE_CHECKING
from unittest.mock import MagicMock, patch
from python_pkg.code_tutor._progress import LessonRecord
from python_pkg.code_tutor._session import _items_from_plan, _show_summary, run_session
if TYPE_CHECKING:
from pathlib import Path
def _make_item_data(name: str = "fn", file: str = "mod.py") -> dict:
return {
"id": f"{file}.{name}",
"file": file,
"type": "function",
"name": name,
"start_line": 1,
"end_line": 5,
"class_name": "",
"depends_on": [],
}
def _make_plan(items: list[dict]) -> dict:
return {
"codebase_path": "/cb",
"created_at": "2026-01-01T00:00:00+00:00",
"total_items": len(items),
"source_fingerprint": "abc",
"sessions": [{"id": 1, "title": "mod", "items": items}],
}
def _make_record(item_id: str, outcome: str) -> LessonRecord:
return LessonRecord(
timestamp="2026-01-01T00:00:00+00:00",
item_id=item_id,
file="mod.py",
lines="1-5",
snippet="code",
outcome=outcome,
answers={},
improvement="",
verdict="PASS" if outcome == "learned" else "FAIL",
attempt=1,
)
# ---------------------------------------------------------------------------
# _items_from_plan
# ---------------------------------------------------------------------------
def test_items_from_plan_basic() -> None:
plan = _make_plan([_make_item_data("fn")])
items = _items_from_plan(plan)
assert len(items) == 1
assert items[0].name == "fn"
def test_items_from_plan_multiple_sessions() -> None:
plan = {
"codebase_path": "/cb",
"created_at": "t",
"total_items": 2,
"source_fingerprint": "x",
"sessions": [
{"id": 1, "title": "a", "items": [_make_item_data("fn1")]},
{"id": 2, "title": "b", "items": [_make_item_data("fn2")]},
],
}
items = _items_from_plan(plan)
assert len(items) == 2
# ---------------------------------------------------------------------------
# _show_summary
# ---------------------------------------------------------------------------
def test_show_summary_renders_table() -> None:
mock_console = MagicMock()
_show_summary(
{"learned": ["a", "b"], "struggled": ["c"], "skipped": []},
mock_console,
)
mock_console.print.assert_called_once()
def test_show_summary_empty() -> None:
mock_console = MagicMock()
_show_summary({}, mock_console)
mock_console.print.assert_called_once()
# ---------------------------------------------------------------------------
# run_session
# ---------------------------------------------------------------------------
def test_run_session_no_plan(tmp_path: Path) -> None:
mock_backend = MagicMock()
mock_console = MagicMock()
with patch("python_pkg.code_tutor._session.load_plan", return_value=None):
run_session(tmp_path, mock_backend, console=mock_console)
mock_console.print.assert_called()
def test_run_session_all_done(tmp_path: Path) -> None:
plan = _make_plan([_make_item_data("fn")])
progress = {
"learned": ["mod.py.fn"],
"struggled": [],
"skipped": [],
"last_session": "",
}
mock_backend = MagicMock()
mock_console = MagicMock()
with (
patch("python_pkg.code_tutor._session.load_plan", return_value=plan),
patch("python_pkg.code_tutor._session.load_progress", return_value=progress),
):
run_session(tmp_path, mock_backend, console=mock_console)
mock_console.print.assert_called()
def test_run_session_learned(tmp_path: Path) -> None:
item_data = _make_item_data("fn")
plan = _make_plan([item_data])
progress = {"learned": [], "struggled": [], "skipped": [], "last_session": ""}
record = _make_record("mod.py.fn", "learned")
mock_backend = MagicMock()
mock_console = MagicMock()
mock_verifier = MagicMock()
mock_verifier.run_lesson.return_value = record
with (
patch("python_pkg.code_tutor._session.load_plan", return_value=plan),
patch("python_pkg.code_tutor._session.load_progress", return_value=progress),
patch("python_pkg.code_tutor._session.Verifier", return_value=mock_verifier),
patch("python_pkg.code_tutor._session.append_session_record"),
patch("python_pkg.code_tutor._session.save_progress"),
):
run_session(tmp_path, mock_backend, console=mock_console)
assert "mod.py.fn" in progress["learned"]
def test_run_session_struggled(tmp_path: Path) -> None:
item_data = _make_item_data("fn")
plan = _make_plan([item_data])
progress = {"learned": [], "struggled": [], "skipped": [], "last_session": ""}
record = _make_record("mod.py.fn", "struggled")
mock_backend = MagicMock()
mock_console = MagicMock()
mock_verifier = MagicMock()
mock_verifier.run_lesson.return_value = record
with (
patch("python_pkg.code_tutor._session.load_plan", return_value=plan),
patch("python_pkg.code_tutor._session.load_progress", return_value=progress),
patch("python_pkg.code_tutor._session.Verifier", return_value=mock_verifier),
patch("python_pkg.code_tutor._session.append_session_record"),
patch("python_pkg.code_tutor._session.save_progress"),
):
run_session(tmp_path, mock_backend, console=mock_console)
assert "mod.py.fn" in progress["struggled"]
def test_run_session_skipped(tmp_path: Path) -> None:
item_data = _make_item_data("fn")
plan = _make_plan([item_data])
progress = {"learned": [], "struggled": [], "skipped": [], "last_session": ""}
record = _make_record("mod.py.fn", "skipped")
mock_backend = MagicMock()
mock_console = MagicMock()
mock_verifier = MagicMock()
mock_verifier.run_lesson.return_value = record
with (
patch("python_pkg.code_tutor._session.load_plan", return_value=plan),
patch("python_pkg.code_tutor._session.load_progress", return_value=progress),
patch("python_pkg.code_tutor._session.Verifier", return_value=mock_verifier),
patch("python_pkg.code_tutor._session.append_session_record"),
patch("python_pkg.code_tutor._session.save_progress"),
):
run_session(tmp_path, mock_backend, console=mock_console)
assert "mod.py.fn" in progress["skipped"]

View File

@ -0,0 +1,361 @@
"""Tests for python_pkg.code_tutor._verifier."""
from __future__ import annotations
from typing import TYPE_CHECKING
from unittest.mock import MagicMock, patch
from python_pkg.code_tutor._analyzer import CodeItem
from python_pkg.code_tutor._verifier import (
Verifier,
_class_header,
_parse_verdict,
_read_snippet,
)
if TYPE_CHECKING:
from pathlib import Path
def _make_item(
file: str = "mod.py",
name: str = "fn",
start: int = 1,
end: int = 3,
class_name: str = "",
) -> CodeItem:
return CodeItem(
id=f"{file}.{name}",
file=file,
type="function",
name=name,
start_line=start,
end_line=end,
class_name=class_name,
)
# ---------------------------------------------------------------------------
# _class_header
# ---------------------------------------------------------------------------
def test_class_header_found() -> None:
lines = [
"class Foo:",
' """Docstring."""',
" x: int",
"",
" def bar(self):",
" pass",
]
result = _class_header(lines, "Foo", 5)
assert "class Foo:" in result
def test_class_header_not_found() -> None:
lines = ["def bar():", " pass"]
result = _class_header(lines, "Missing", 2)
assert result == ""
def test_class_header_snippet_end_clamped() -> None:
# Only 3 lines before the method, so snippet_end is clamped to before_line - 1
lines = ["class A:", " x = 1", " def m(self):"]
result = _class_header(lines, "A", 3)
assert "class A:" in result
# ---------------------------------------------------------------------------
# _read_snippet
# ---------------------------------------------------------------------------
def test_read_snippet_oserror(tmp_path: Path) -> None:
item = _make_item(file="missing.py")
result = _read_snippet(item, str(tmp_path))
assert "source unavailable" in result
def test_read_snippet_no_class(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text(
"def fn():\n pass\n return 1\n", encoding="utf-8"
)
item = _make_item(file="mod.py", start=1, end=3)
result = _read_snippet(item, str(tmp_path))
assert "def fn():" in result
def test_read_snippet_with_class_header_found(tmp_path: Path) -> None:
source = "class Foo:\n def fn(self):\n pass\n"
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
item = _make_item(file="mod.py", start=2, end=3, class_name="Foo")
result = _read_snippet(item, str(tmp_path))
assert "class Foo" in result
assert "method" in result
def test_read_snippet_with_class_header_not_found(tmp_path: Path) -> None:
source = "def fn():\n pass\n"
(tmp_path / "mod.py").write_text(source, encoding="utf-8")
# class_name is set but not present in file => no header
item = _make_item(file="mod.py", start=1, end=2, class_name="NoSuchClass")
result = _read_snippet(item, str(tmp_path))
assert "def fn():" in result
# ---------------------------------------------------------------------------
# _parse_verdict
# ---------------------------------------------------------------------------
def test_parse_verdict_pass() -> None:
raw = '{"verdict": "PASS", "gap": ""}'
verdict, gap = _parse_verdict(raw)
assert verdict == "PASS"
assert gap == ""
def test_parse_verdict_fail_with_gap() -> None:
raw = '{"verdict": "FAIL", "gap": "Missing edge case."}'
verdict, gap = _parse_verdict(raw)
assert verdict == "FAIL"
assert gap == "Missing edge case."
def test_parse_verdict_no_braces() -> None:
verdict, gap = _parse_verdict("no json here at all")
assert verdict == "FAIL"
assert "parse" in gap.lower()
def test_parse_verdict_json_decode_error() -> None:
verdict, gap = _parse_verdict("{bad json}")
assert verdict == "FAIL"
assert "parse" in gap.lower()
def test_parse_verdict_invalid_verdict() -> None:
raw = '{"verdict": "MAYBE", "gap": "unclear"}'
verdict, _gap = _parse_verdict(raw)
assert verdict == "FAIL"
def test_parse_verdict_strips_markdown_fence() -> None:
raw = '```json\n{"verdict": "PASS", "gap": ""}\n```'
verdict, _gap = _parse_verdict(raw)
assert verdict == "PASS"
# ---------------------------------------------------------------------------
# Verifier._judge (via run_lesson)
# ---------------------------------------------------------------------------
def _make_live_mock() -> MagicMock:
live = MagicMock()
live.__enter__ = MagicMock(return_value=live)
live.__exit__ = MagicMock(return_value=False)
return live
def _make_verifier() -> tuple[Verifier, MagicMock, MagicMock]:
mock_backend = MagicMock()
mock_console = MagicMock()
verifier = Verifier(mock_backend, mock_console)
return verifier, mock_backend, mock_console
def test_judge_calls_stream_and_parses(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn():\n pass\n", encoding="utf-8")
verifier, mock_backend, _ = _make_verifier()
def fake_stream(system: str, user: str, on_token: object) -> str:
assert callable(on_token)
on_token('{"verdict": "PASS", "gap": ""}')
return ""
mock_backend.stream.side_effect = fake_stream
live_mock = _make_live_mock()
with patch("python_pkg.code_tutor._verifier.Live", return_value=live_mock):
verdict, _gap = verifier._judge("code snippet", "my explanation")
assert verdict == "PASS"
# ---------------------------------------------------------------------------
# Verifier._collect_answers
# ---------------------------------------------------------------------------
def test_collect_answers_all_answered() -> None:
verifier, _, _ = _make_verifier()
responses = iter(
["purpose answer", "inputs answer", "outputs answer", "why answer"]
)
answers, skipped = verifier._collect_answers(lambda _: next(responses))
assert not skipped
assert len(answers) == 4
def test_collect_answers_skip_on_first() -> None:
verifier, _, _ = _make_verifier()
answers, skipped = verifier._collect_answers(lambda _: "skip")
assert skipped
assert answers == {}
def test_collect_answers_skip_on_second() -> None:
verifier, _, _ = _make_verifier()
call_count = [0]
def input_fn(_: str) -> str:
call_count[0] += 1
if call_count[0] == 2:
return "skip"
return "answer"
_answers, skipped = verifier._collect_answers(input_fn)
assert skipped
# ---------------------------------------------------------------------------
# Verifier._ask_improvement
# ---------------------------------------------------------------------------
def test_ask_improvement_returns_input() -> None:
verifier, _, _ = _make_verifier()
result = verifier._ask_improvement(lambda _: " make it faster ")
assert result == "make it faster"
def test_ask_improvement_empty() -> None:
verifier, _, _ = _make_verifier()
result = verifier._ask_improvement(lambda _: "")
assert result == ""
# ---------------------------------------------------------------------------
# Verifier.run_lesson
# ---------------------------------------------------------------------------
def _four_answers() -> list[str]:
return ["purpose", "inputs", "outputs", "why"]
def test_run_lesson_pass_first_attempt(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn():\n pass\n", encoding="utf-8")
verifier, mock_backend, _ = _make_verifier()
def fake_stream(system: str, user: str, on_token: object) -> str:
assert callable(on_token)
on_token('{"verdict": "PASS", "gap": ""}')
return ""
mock_backend.stream.side_effect = fake_stream
answers_iter = iter(_four_answers())
live_mock = _make_live_mock()
with (
patch("python_pkg.code_tutor._verifier.Live", return_value=live_mock),
patch(
"python_pkg.code_tutor._verifier.run_coding_challenge",
return_value="skipped",
),
):
record = verifier.run_lesson(
_make_item(file="mod.py"),
str(tmp_path),
input_fn=lambda _: next(answers_iter, ""),
)
assert record.outcome == "learned"
assert record.verdict == "PASS"
assert record.attempt == 1
def test_run_lesson_skip_on_first_question(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn():\n pass\n", encoding="utf-8")
verifier, _, _ = _make_verifier()
live_mock = _make_live_mock()
with patch("python_pkg.code_tutor._verifier.Live", return_value=live_mock):
record = verifier.run_lesson(
_make_item(file="mod.py"),
str(tmp_path),
input_fn=lambda _: "skip",
)
assert record.outcome == "skipped"
assert record.verdict == "skipped"
def test_run_lesson_all_attempts_fail(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn():\n pass\n", encoding="utf-8")
verifier, mock_backend, _ = _make_verifier()
call_count = [0]
def fail_stream(system: str, user: str, on_token: object) -> str:
assert callable(on_token)
on_token('{"verdict": "FAIL", "gap": "Missing detail."}')
return ""
mock_backend.stream.side_effect = fail_stream
def input_fn(_: str) -> str:
call_count[0] += 1
# Each "round" of 4 questions: return an answer
return "my answer"
live_mock = _make_live_mock()
with patch("python_pkg.code_tutor._verifier.Live", return_value=live_mock):
record = verifier.run_lesson(
_make_item(file="mod.py"),
str(tmp_path),
input_fn=input_fn,
)
assert record.outcome == "struggled"
assert record.verdict == "FAIL"
assert record.attempt == 3
def test_run_lesson_pass_on_second_attempt(tmp_path: Path) -> None:
(tmp_path / "mod.py").write_text("def fn():\n pass\n", encoding="utf-8")
verifier, mock_backend, _ = _make_verifier()
stream_calls = [0]
def fake_stream(system: str, user: str, on_token: object) -> str:
assert callable(on_token)
stream_calls[0] += 1
verdict = (
'{"verdict": "FAIL", "gap": "gap"}'
if stream_calls[0] == 1
else '{"verdict": "PASS", "gap": ""}'
)
on_token(verdict)
return ""
mock_backend.stream.side_effect = fake_stream
live_mock = _make_live_mock()
with (
patch("python_pkg.code_tutor._verifier.Live", return_value=live_mock),
patch(
"python_pkg.code_tutor._verifier.run_coding_challenge",
return_value="passed",
),
):
record = verifier.run_lesson(
_make_item(file="mod.py"),
str(tmp_path),
input_fn=lambda _: "my answer",
)
assert record.outcome == "learned"
assert record.attempt == 2