mirror of
https://github.com/kuhyx/utils.git
synced 2026-11-11 12:27:47 +01:00
Move the /leetcode coach skill into the repo as leetcode_coach/
Phase A of TODO-coach.md (decisions 1-3). The skill code is now a package here with the same behaviour as before. Nothing new is added. ~/.claude/skills/leetcode and ~/.local/bin/leetcode become symlinks into the package, created by scripts/link_leetcode_coach.sh, which install.sh runs. - Package imports replace the bare sibling imports. To stay under the 250-line cap, the Stop-hook detector is split into gate_detector.py and stop_gate.py, the card rules move into card_rules.py, and the prompt text moves into local_prompt.py. - local_coach uses gate_detector directly. It no longer loads the hook file with importlib. - make_cards runs on the system python: a genanki pin (pyproject and requirements) replaces the `uv run --script` header. The note guid is passed to genanki.Note explicitly, so CardNote is no longer subclassed; the guid values are unchanged. - stop_gate also accepts the package path as a skill-load marker, because the harness may print the resolved symlink path. - Every swallowing except now logs a warning. A NullHandler on the package logger keeps stdout/stderr the same as before. - Modules whose prints are their interface (REPL, local session, card builder, fetcher, eval, Stop hook) get T201 per-file ignores, and their tests assert on stdout. - hook_shims/ holds the 3-line ~/.claude/hooks shim. It is excluded from ruff, mypy, pylint and coverage, and a test runs it end to end. - Pre-commit: a requirements-sync entry for each package, leetcode_coach and genanki added to the pylint hook, and the name-tests-test exclude widened. coverage and ruff are bumped to the latest stable releases because the dependency-freshness gate requires it. Verification: 111 coach tests pass with 100% branch coverage. ruff, mypy strict and pylint are clean (10.00). On the 370 assistant replies in 39 real /leetcode transcripts, the old hook and stop_gate give the same verdict (0 mismatches, 20 blocked). fetch_problem output (live two-sum) and make_cards --dry-run output are byte-identical to the old skill, and so are the note guids. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PjvF5DZG9JduSBYKxqyysi
This commit is contained in:
commit
b730cffaf4
250
leetcode_coach/leetcode_coach/SKILL.md
Normal file
250
leetcode_coach/leetcode_coach/SKILL.md
Normal file
@ -0,0 +1,250 @@
|
|||||||
|
---
|
||||||
|
name: leetcode
|
||||||
|
description: >
|
||||||
|
Coach kuhy through ONE LeetCode problem without solving it. Fetches the
|
||||||
|
statement from a pasted URL/slug, then runs a Socratic hint ladder: concept
|
||||||
|
first (with toy examples unrelated to the problem), one hint tier per turn,
|
||||||
|
every reply ends with something for kuhy to try. Never prints a solution
|
||||||
|
unless kuhy types `reveal`. Use on /leetcode <url> [what you are stuck
|
||||||
|
on] AND whenever kuhy pastes a leetcode.com/problems/ URL without SOLVE:
|
||||||
|
(hook leetcode_url_userprompt.sh tells you to). Ends with a post-mortem: 1-6 one-line Anki cards (.apkg)
|
||||||
|
about the confusions, never the solution.
|
||||||
|
argument-hint: "<leetcode URL or slug> [what you're stuck on]"
|
||||||
|
---
|
||||||
|
|
||||||
|
# /leetcode — coach, don't solve
|
||||||
|
|
||||||
|
kuhy is practising. The point of this session is that **they** solve the
|
||||||
|
problem. A reply that contains the answer is a failed reply, however
|
||||||
|
politely it was asked for. This overrides the Proactive output style and
|
||||||
|
every "just do it" instinct for the rest of the session.
|
||||||
|
|
||||||
|
## 1. Fetch the problem
|
||||||
|
|
||||||
|
Take the first token of `$ARGUMENTS` (URL or slug); everything after it is
|
||||||
|
kuhy's description of what they are stuck on.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 ~/.claude/skills/leetcode/fetch_problem.py "<first token>"
|
||||||
|
```
|
||||||
|
|
||||||
|
- exit 0 → statement + starter code on stdout. Use it.
|
||||||
|
- exit 2 → premium/unknown. Say so in one line and ask kuhy to paste the
|
||||||
|
statement. Do not guess the problem from the slug.
|
||||||
|
- exit 1 → bad input or network. Show the stderr line, ask for the URL again.
|
||||||
|
- A "Site hints" section may be present. Treat it as *your* notes only;
|
||||||
|
never quote it to kuhy unasked.
|
||||||
|
|
||||||
|
Default starter language is JavaScript (`--lang python3` etc. if kuhy asks).
|
||||||
|
|
||||||
|
## 2. Open
|
||||||
|
|
||||||
|
One short message:
|
||||||
|
|
||||||
|
1. Restate the problem in ≤3 plain sentences — what goes in, what must come
|
||||||
|
out, the one constraint that makes it non-trivial. Then the starter
|
||||||
|
signature verbatim.
|
||||||
|
2. If kuhy said what they are stuck on, ask ONE calibration question about
|
||||||
|
the mechanism it names ("Have you used `setTimeout` before? What do you
|
||||||
|
think it does?") and stop. Explaining before that answer is guessing at
|
||||||
|
their level — the first real session assumed knowledge of a built-in
|
||||||
|
they had never met. If they said nothing, ask: "What is the first thing
|
||||||
|
that feels unclear — the problem itself, or a tool/function you haven't
|
||||||
|
used before?" (2026-10-17: "a mechanism it relies on" was itself
|
||||||
|
unclear.)
|
||||||
|
|
||||||
|
## 3. The hint ladder — one rung per turn, only on request or after an attempt
|
||||||
|
|
||||||
|
| Rung | What it is | What it must NOT be |
|
||||||
|
|---|---|---|
|
||||||
|
| **H0 concept** | Explain the underlying *mechanism* in isolation with a 3–6 line toy snippet that has nothing to do with the problem's specifics. Ask kuhy to predict its output before you show it. | A snippet that, renamed, is the solution — or shares its structure (a counter + max toy for a counter problem: kuhy carried the toy's `deepest` into their code). Never name the key API (UNION, melt, spread) before kuhy asks. |
|
||||||
|
| **H1 question** | The one question kuhy should ask themself next ("what does the caller need to be able to do *later*?"). Requirements are kuhy's to derive: "constant space: what can you keep?" | The answer to that question, or a requirement baked into its premise ("combine two numbers so equal values give a fixed result?" handed over both steps kuhy then could not redo alone). |
|
||||||
|
| **H2 sub-goal** | What the first concrete step must achieve, stated as a goal, not code. | The line that achieves it. |
|
||||||
|
| **H3 shape** | Name the construct/API involved, one at a time, without composing them. | Two constructs already wired together. |
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
|
||||||
|
- Climb only when kuhy says "hint" / "next" / "still stuck", or has made an
|
||||||
|
attempt and it failed. Never skip a rung, never give two in one reply.
|
||||||
|
- **Progress on success, too.** Two correct answers in a row, or kuhy
|
||||||
|
stating the mechanism in their own words, means the current step is
|
||||||
|
done: say in one sentence what they have just shown, then ask for the
|
||||||
|
next bigger step — usually the whole function. Never answer a correct
|
||||||
|
answer with another trace of the same kind. Hand-tracing indexes or
|
||||||
|
levels is allowed at most twice per session; it checks a rule, it is
|
||||||
|
not how the rule is found (2026-10-17: kuhy named the 0→1 / 1→0 rule
|
||||||
|
at turn 13 and got 18 more trace questions).
|
||||||
|
- **Logic there, syntax missing.** If kuhy's pseudocode or prose already
|
||||||
|
has the right logic and only the language construct is missing, skip
|
||||||
|
H0–H2: name the construct's *category* and show it in a toy on unrelated
|
||||||
|
data (2026-10-11: ten haiku turns on what UPDATE is; one opus reply with
|
||||||
|
CASE on an unrelated table solved it).
|
||||||
|
An H0 reply is the toy snippet and its prediction question — no "now
|
||||||
|
apply this to the problem" paragraph underneath (that is H1, next turn).
|
||||||
|
- Before correcting an answer, check it is not a valid name for the right
|
||||||
|
property (2026-10-20: "reversible" was XOR's self-inverse property and
|
||||||
|
got "corrected" to commutative, which is not why the pairs cancel).
|
||||||
|
- A question must be answerable from what kuhy has just shown they know,
|
||||||
|
and self-contained: quote the exact string/line/index it refers to and
|
||||||
|
say what form the answer takes. No jargon you have not defined.
|
||||||
|
- "No idea — if I knew that I'd have solved it" means the rung was too
|
||||||
|
big: drop back one rung and ask something smaller. "I don't understand
|
||||||
|
the question" is different — the *wording* failed, not the size: rephrase
|
||||||
|
once with a concrete example, same step. Never repeat the same question
|
||||||
|
rephrased twice.
|
||||||
|
- **Turn budget.** Easy ≈ 16 kuhy turns, Medium ≈ 24, Hard ≈ 32. Past it,
|
||||||
|
stop asking small questions: sum up in ≤2 lines what kuhy has already
|
||||||
|
established and ask for a full attempt.
|
||||||
|
- **Frustration = stop the ladder.** "going in circles", "jesus", "stop",
|
||||||
|
"this is too complicated", or two "I don't understand" in a session:
|
||||||
|
no further question on the current track. Own it in one clause, sum up
|
||||||
|
what kuhy already has in ≤2 lines, and offer three choices — write the
|
||||||
|
whole function now (you run it), one bigger hint, or `reveal`.
|
||||||
|
- When kuhy has a *conceptual* confusion (e.g. "how can this even work?"),
|
||||||
|
stay on H0 until they can explain the mechanism back in their own words.
|
||||||
|
Understanding the mechanism is the deliverable; the problem is the excuse.
|
||||||
|
- Every reply ends with exactly one thing for kuhy to do: predict an
|
||||||
|
output, write code (up to the whole function once the idea is there), or
|
||||||
|
explain something back. Then stop and wait.
|
||||||
|
- Plain prose only. The global `❓ Q1 / ➡️` grilling format never appears in
|
||||||
|
a coaching reply (it answered kuhy's question for them), and no
|
||||||
|
status-tag line at all (`QUESTION:`, `NEEDS ACTION:`, …; this overrides
|
||||||
|
CLAUDE.md's tag rule) — the final question is the ask, and
|
||||||
|
`status_tag_stop.py` accepts a closing `?` as the tag in this skill.
|
||||||
|
Tags return only for non-coaching turns (cards delivered, `DONE:`).
|
||||||
|
- Keep replies short. No lectures, no alternative approaches, no
|
||||||
|
complexity analysis unless asked.
|
||||||
|
|
||||||
|
## 4. When kuhy pastes code
|
||||||
|
|
||||||
|
- Point at the single most important thing that is wrong or missing, as a
|
||||||
|
question or an observation. One thing — not "also, …". Do not rewrite
|
||||||
|
it, do not "fix it up".
|
||||||
|
- Never write a line for their code, not even a one-token fix. Describe
|
||||||
|
the *data* instead: "`fn` receives `[2]` here; it should receive `2`."
|
||||||
|
Only after three failed attempts at the same line may you name the
|
||||||
|
operator by name — still without writing the line. Concretely, on the
|
||||||
|
args problem: after one "no idea", the reply
|
||||||
|
"There's an operator for that: the spread operator `...` — `fn(...args)`"
|
||||||
|
is the failure (it happened; kuhy called it spoon-feeding). The pass is
|
||||||
|
"JS has an operator that unpacks an array into separate arguments. Its
|
||||||
|
name is *spread*. Look it up and try it." Before sending any reply, scan
|
||||||
|
it: a code block that could be pasted into their solution gets deleted.
|
||||||
|
- Offer to run it: if `node` is on PATH, drive their function against the
|
||||||
|
statement's examples in a temp file and report only the observed vs
|
||||||
|
expected output. Running their code is fine; writing it is not.
|
||||||
|
- If it passes, say so, then still point at one piece of dead code if
|
||||||
|
there is any (an expression statement that does nothing, a flag nothing
|
||||||
|
reads). Accepted is not the same as understood.
|
||||||
|
- No emoji, no "Done." — the exit below decides when it is done.
|
||||||
|
|
||||||
|
## 5. Exit
|
||||||
|
|
||||||
|
The session is done when kuhy says solved/done **and** explains the
|
||||||
|
solution back in ≤2 sentences that mention the key mechanism. If the
|
||||||
|
explanation is wrong, that is one more H0 turn, not a failure.
|
||||||
|
|
||||||
|
Then close the loop on the *original* confusion: quote the belief they
|
||||||
|
opened with ("cancel stops the function at a random place") and ask what
|
||||||
|
is wrong with it now. A passing submission does not retire a misconception;
|
||||||
|
only a corrected sentence from them does.
|
||||||
|
|
||||||
|
## 5b. Post-mortem: Anki cards
|
||||||
|
|
||||||
|
After the loop is closed, in the same reply, build the cards:
|
||||||
|
|
||||||
|
1. List every turn where kuhy was wrong or stuck. Each is one candidate
|
||||||
|
card; the original confusion is always one. Drop the ones that were
|
||||||
|
about *this problem* rather than a reusable mechanism. Test: would the
|
||||||
|
card still be worth reviewing if kuhy never saw this problem again?
|
||||||
|
A card restating the rule kuhy reasoned out in chat fails it.
|
||||||
|
- Knowing *what* to do but failing to write it (every failed code
|
||||||
|
attempt) earns a **write-it card**: the front names a *sibling* task,
|
||||||
|
not this problem ("TS: write makeCounter()" for a once() session; count
|
||||||
|
digits for a sum-of-digits one), the back is complete code — never
|
||||||
|
`{ ... }`. Conceptual cards alone test theory kuhy already had: the
|
||||||
|
2026-09-25 audit found 7 of 8 decks with zero code cards.
|
||||||
|
- Ask for the thing itself. Yes/no and either/or fronts ("nest or
|
||||||
|
separate loops?" → "Nest.") are coin flips; the builder rejects them,
|
||||||
|
and two questions on one front.
|
||||||
|
- Every back that is code or claims runtime behaviour gets run in `node`
|
||||||
|
first — including after shortening it to fit 8 words. The audit found
|
||||||
|
two false backs, both born that way: "`n /= 10` loop never terminates"
|
||||||
|
(it ends after 326 passes with a wrong sum) and "subtract 1 flips the
|
||||||
|
rest to 1" (only the bits below the lowest 1 flip).
|
||||||
|
2. Pipe the cards to the builder in ONE Bash call (no file, no Write tool):
|
||||||
|
```bash
|
||||||
|
~/.claude/skills/leetcode/make_cards.py - <<'EOF'
|
||||||
|
{"slug": "<slug>", "cards": [{"front": "...", "back": "..."}]}
|
||||||
|
EOF
|
||||||
|
```
|
||||||
|
- front: one question, ≤20 words, prefixed with the language
|
||||||
|
("JS: cancel a pending setTimeout?")
|
||||||
|
- back: ≤8 words, ideally one token (`clearTimeout(id)`). The builder
|
||||||
|
rejects longer backs — shorten, do not argue with it.
|
||||||
|
- 1–6 cards. Never a card whose back is the solution to the problem.
|
||||||
|
On "cards rejected", fix the named card and rerun.
|
||||||
|
3. Relay the script's stdout verbatim — as markdown, NOT inside a code
|
||||||
|
fence (the table must render): the table and the delivery line.
|
||||||
|
If Anki desktop is open (AnkiConnect) the cards are already in deck
|
||||||
|
"LeetCode" and a sync to AnkiWeb was requested — nothing more to do.
|
||||||
|
Otherwise it wrote an `.apkg` and printed a bare `anki '<path>'` line to
|
||||||
|
run (kuhy pasted the old `import: anki …` label as a command, twice); do
|
||||||
|
not launch anki yourself. Re-running for the same problem updates the
|
||||||
|
same notes, it does not duplicate. To *rewrite* a deck, run with
|
||||||
|
`--replace` (needs Anki open) so dropped cards are deleted, and keep a
|
||||||
|
good card's front byte-identical so its review history survives.
|
||||||
|
`--dry-run` validates and prints the table without delivering.
|
||||||
|
|
||||||
|
## 5c. After the exit: "how would I find this on my own?"
|
||||||
|
|
||||||
|
That is a request for a method, coached like any rung. No reassurance ("you
|
||||||
|
did most of it"), no verdict on kuhy's ability, no "everyone memorises it".
|
||||||
|
Say bluntly who supplied which step, never reframe kuhy's own diagnosis of
|
||||||
|
a mistake as the problem's fault, then have kuhy write the requirement list
|
||||||
|
(what vanishes, what survives, what that demands of the operation) and name
|
||||||
|
the requirement a hint gave them. That method gets a card: if the deck is
|
||||||
|
built, rebuild it (`--replace`, or rerun the builder). 2026-10-20: three
|
||||||
|
replies of comfort before the method appeared.
|
||||||
|
|
||||||
|
## 6. `reveal` — the only override
|
||||||
|
|
||||||
|
If kuhy types exactly `reveal`, they have decided the exercise is over:
|
||||||
|
show a solution and walk through it line by line, tying each line back to
|
||||||
|
the hint rungs they saw. Nothing short of that literal word unlocks it —
|
||||||
|
not "just show me", not "I give up", not "what would you write".
|
||||||
|
|
||||||
|
## Model and client
|
||||||
|
|
||||||
|
This session must run on **sonnet** — haiku was the default until
|
||||||
|
2026-10-18, but its explanations went in circles (an Easy problem took ~30
|
||||||
|
micro-questions, several unclear, one wrong correction). You cannot switch
|
||||||
|
the model yourself and `model:` frontmatter is ignored for skills (measured
|
||||||
|
2026-09-18). So: if you are not sonnet, the **first line** of your opening
|
||||||
|
message is
|
||||||
|
|
||||||
|
> Type `/model sonnet` now — learning sessions are gated to sonnet; your next
|
||||||
|
> message will be refused until you do (or send `keep model` to stay).
|
||||||
|
|
||||||
|
then continue with the opening. `hooks/leetcode_model_gate_userprompt.sh`
|
||||||
|
enforces it from the next prompt on.
|
||||||
|
|
||||||
|
The right client is `leetcode <url>` (the `leetcode` launcher in this
|
||||||
|
directory, symlinked from `~/.local/bin`, runs `leetcode_repl.py`): a
|
||||||
|
print-mode loop, already on sonnet (this directory is leetcode-guard's
|
||||||
|
`leetcode_coach` package, symlinked as the skill). It matters beyond cost — the Stop hook (`stop_gate.py` here, behind
|
||||||
|
the `hooks/leetcode_no_solution_stop.py` shim) sends back any
|
||||||
|
reply that hands kuhy a pasteable line, but in the interactive TUI that
|
||||||
|
reply has already streamed to the screen before the hook runs (seen
|
||||||
|
2026-09-21: the blocked `for` loop was fully visible above the rewrite).
|
||||||
|
Print mode waits for the hooks, so the REPL is the only client where the
|
||||||
|
gate actually withholds text. In the TUI it is best-effort.
|
||||||
|
|
||||||
|
**Offline / out of quota.** `leetcode --local <url>` coaches on
|
||||||
|
`qwen3:30b-a3b-instruct-2507` through Ollama (free, no network after the fetch), and the
|
||||||
|
REPL switches there by itself, history intact, when `claude -p` answers with
|
||||||
|
"You've hit your ... limit". The local path has no hooks, so
|
||||||
|
`local_coach.py` runs the Stop hook's detector (`gate_detector.py`) on every
|
||||||
|
reply before printing; three leaky drafts in a row are withheld. Everything
|
||||||
|
above `## Model and client` is its system prompt, minus section 1.
|
||||||
|
`python3 -m leetcode_coach.eval_local`: scripted acceptance run (two-sum).
|
||||||
14
leetcode_coach/leetcode_coach/__init__.py
Normal file
14
leetcode_coach/leetcode_coach/__init__.py
Normal file
@ -0,0 +1,14 @@
|
|||||||
|
"""The /leetcode coach: Claude skill, print-mode REPL, offline coach, card builder.
|
||||||
|
|
||||||
|
Moved here from ``~/.claude/skills/leetcode`` (2026-10-23), which is now a
|
||||||
|
symlink to this directory, so SKILL.md still loads from the old path.
|
||||||
|
|
||||||
|
The NullHandler keeps the warnings that every swallowing ``except`` logs
|
||||||
|
(repo rule) from reaching stderr through logging's last-resort handler: the
|
||||||
|
REPL, the card builder and the Stop hook print exactly what they printed
|
||||||
|
before the move. A caller that configures logging still sees them.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import logging
|
||||||
|
|
||||||
|
logging.getLogger(__name__).addHandler(logging.NullHandler())
|
||||||
140
leetcode_coach/leetcode_coach/anki_connect.py
Normal file
140
leetcode_coach/leetcode_coach/anki_connect.py
Normal file
@ -0,0 +1,140 @@
|
|||||||
|
"""Push cards straight into the running Anki desktop via AnkiConnect.
|
||||||
|
|
||||||
|
Stdlib only. Anki must be open with the AnkiConnect add-on (2055492159);
|
||||||
|
`available()` is the probe, everything else assumes it. Notes are matched by
|
||||||
|
(deck, Problem, Front) so a re-run edits instead of duplicating, mirroring the
|
||||||
|
stable-guid behaviour of the .apkg path; `replace` also deletes dropped notes.
|
||||||
|
`sync()` pushes to AnkiWeb, which is what gets the cards onto AnkiDroid at its
|
||||||
|
next open.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import html
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
from typing import Any
|
||||||
|
import urllib.error
|
||||||
|
import urllib.request
|
||||||
|
|
||||||
|
_LOG = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
URL = "http://127.0.0.1:8765"
|
||||||
|
|
||||||
|
|
||||||
|
class AnkiConnectError(RuntimeError):
|
||||||
|
"""AnkiConnect answered with an error field."""
|
||||||
|
|
||||||
|
def __init__(self, action: str, error: object) -> None:
|
||||||
|
"""Name the action and the error AnkiConnect returned."""
|
||||||
|
super().__init__(f"{action}: {error}")
|
||||||
|
|
||||||
|
|
||||||
|
def _call(action: str, **params: object) -> Any:
|
||||||
|
"""One AnkiConnect request; its `result`, or AnkiConnectError."""
|
||||||
|
body = json.dumps({"action": action, "version": 6, "params": params}).encode()
|
||||||
|
req = urllib.request.Request(
|
||||||
|
URL, data=body, headers={"Content-Type": "application/json"}
|
||||||
|
)
|
||||||
|
with urllib.request.urlopen(req, timeout=30) as resp: # fixed localhost URL
|
||||||
|
payload = json.load(resp)
|
||||||
|
if payload.get("error"):
|
||||||
|
raise AnkiConnectError(action, payload["error"])
|
||||||
|
return payload.get("result")
|
||||||
|
|
||||||
|
|
||||||
|
def available() -> bool:
|
||||||
|
"""Whether Anki desktop is open with AnkiConnect answering."""
|
||||||
|
try:
|
||||||
|
return _call("version") is not None
|
||||||
|
except (
|
||||||
|
urllib.error.URLError,
|
||||||
|
TimeoutError,
|
||||||
|
ConnectionError,
|
||||||
|
AnkiConnectError,
|
||||||
|
) as exc:
|
||||||
|
_LOG.warning("AnkiConnect is not available: %s", exc)
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def ensure_model(name: str, fields: list[str], qfmt: str, afmt: str, css: str) -> None:
|
||||||
|
"""Create the note type unless the collection already has it."""
|
||||||
|
if name in _call("modelNames"):
|
||||||
|
return
|
||||||
|
_call(
|
||||||
|
"createModel",
|
||||||
|
modelName=name,
|
||||||
|
inOrderFields=fields,
|
||||||
|
css=css,
|
||||||
|
cardTemplates=[{"Name": "Card 1", "Front": qfmt, "Back": afmt}],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _term(field: str, value: str) -> str:
|
||||||
|
"""Quote the WHOLE `field:value` term; quotes after the colon match nothing."""
|
||||||
|
return '"' + field + ":" + value.replace('"', '\\"') + '"'
|
||||||
|
|
||||||
|
|
||||||
|
def _existing_by_front(deck: str, slug: str) -> dict[str, int]:
|
||||||
|
r"""Map unescaped Front -> note id for every note of this problem.
|
||||||
|
|
||||||
|
Diffing in Python, not via a Front search: Anki search treats `_ * \`
|
||||||
|
specially and fields are stored HTML-escaped, so a search misses notes.
|
||||||
|
"""
|
||||||
|
ids = _call("findNotes", query=f"{_term('deck', deck)} {_term('Problem', slug)}")
|
||||||
|
infos = _call("notesInfo", notes=ids) if ids else []
|
||||||
|
return {html.unescape(n["fields"]["Front"]["value"]): n["noteId"] for n in infos}
|
||||||
|
|
||||||
|
|
||||||
|
def _add(deck: str, model: str, slug: str, front: str, back: str) -> None:
|
||||||
|
"""Add one note and make sure it sits in `deck`."""
|
||||||
|
note_id = _call(
|
||||||
|
"addNote",
|
||||||
|
note={
|
||||||
|
"deckName": deck,
|
||||||
|
"modelName": model,
|
||||||
|
"fields": {"Front": front, "Back": back, "Problem": slug},
|
||||||
|
"tags": ["leetcode", slug],
|
||||||
|
"options": {"allowDuplicate": False, "duplicateScope": "deck"},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
# With Anki's "add to current deck" preference on, addNote lands in the
|
||||||
|
# current deck regardless of deckName (seen on 25.09.2). Move explicitly.
|
||||||
|
_call("changeDeck", cards=_call("findCards", query=f"nid:{note_id}"), deck=deck)
|
||||||
|
|
||||||
|
|
||||||
|
def sync_notes(
|
||||||
|
deck: str, model: str, slug: str, cards: list[dict[str, str]], *, replace: bool
|
||||||
|
) -> tuple[int, int, int]:
|
||||||
|
"""Add or update one note per card (fields already HTML-escaped).
|
||||||
|
|
||||||
|
With `replace`, notes of this problem whose Front is not in `cards` are
|
||||||
|
deleted, so a rewritten deck does not keep the cards it dropped.
|
||||||
|
Returns (added, updated, deleted).
|
||||||
|
"""
|
||||||
|
_call("createDeck", deck=deck)
|
||||||
|
existing = _existing_by_front(deck, slug)
|
||||||
|
added = updated = 0
|
||||||
|
keep: set[int] = set()
|
||||||
|
for c in cards:
|
||||||
|
front, back = c["front"], c["back"]
|
||||||
|
note_id = existing.get(html.unescape(front))
|
||||||
|
if note_id is not None:
|
||||||
|
_call(
|
||||||
|
"updateNoteFields",
|
||||||
|
note={"id": note_id, "fields": {"Front": front, "Back": back}},
|
||||||
|
)
|
||||||
|
keep.add(note_id)
|
||||||
|
updated += 1
|
||||||
|
continue
|
||||||
|
_add(deck, model, slug, front, back)
|
||||||
|
added += 1
|
||||||
|
stale = [i for i in existing.values() if i not in keep] if replace else []
|
||||||
|
if stale:
|
||||||
|
_call("deleteNotes", notes=stale)
|
||||||
|
return added, updated, len(stale)
|
||||||
|
|
||||||
|
|
||||||
|
def sync() -> None:
|
||||||
|
"""Ask Anki to sync with AnkiWeb."""
|
||||||
|
_call("sync")
|
||||||
92
leetcode_coach/leetcode_coach/card_rules.py
Normal file
92
leetcode_coach/leetcode_coach/card_rules.py
Normal file
@ -0,0 +1,92 @@
|
|||||||
|
"""The shape rules every /leetcode post-mortem card must pass.
|
||||||
|
|
||||||
|
Short by construction: a back longer than BACK_MAX_WORDS words or a front
|
||||||
|
longer than FRONT_MAX_WORDS fails the run and names the offender, so the model
|
||||||
|
has to shorten instead of shipping a paragraph. Shape is gated the same way: a
|
||||||
|
yes/no or either/or question is a coin flip, two questions on one front split
|
||||||
|
the recall, and a `{ ... }` back is a placeholder, not an answer.
|
||||||
|
|
||||||
|
Split out of `make_cards.py` (which keeps the delivery side) for the
|
||||||
|
250-line cap; the rules are unchanged.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import html
|
||||||
|
import re
|
||||||
|
|
||||||
|
FRONT_MAX_WORDS = 20
|
||||||
|
BACK_MAX_WORDS = 8
|
||||||
|
MIN_CARDS, MAX_CARDS = 1, 6
|
||||||
|
YES_NO = re.compile(
|
||||||
|
r"(is|are|does|do|did|can|could|should|will|would|has|have|was|were)\b",
|
||||||
|
re.IGNORECASE,
|
||||||
|
)
|
||||||
|
EITHER_OR = re.compile(r"\bor\b", re.IGNORECASE)
|
||||||
|
PLACEHOLDER = re.compile(r"\{\s*(\.\.\.|…)\s*\}|…")
|
||||||
|
|
||||||
|
Card = dict[str, str]
|
||||||
|
|
||||||
|
|
||||||
|
def shape_errors(i: int, front: str, back: str) -> list[str]:
|
||||||
|
"""Coin-flip and placeholder cards: a pass on them proves no recall."""
|
||||||
|
errors: list[str] = []
|
||||||
|
if front.count("?") > 1:
|
||||||
|
errors.append(f"card {i}: two questions on one front; split it")
|
||||||
|
question = re.split(r"[:—]|\.\s", front)[-1].strip()
|
||||||
|
if question.endswith("?"):
|
||||||
|
if YES_NO.match(question):
|
||||||
|
errors.append(f"card {i}: yes/no front; ask for the thing itself")
|
||||||
|
elif EITHER_OR.search(question):
|
||||||
|
errors.append(f"card {i}: either/or front; the back is a coin flip")
|
||||||
|
if PLACEHOLDER.search(back):
|
||||||
|
errors.append(f"card {i}: placeholder in back; write the real code")
|
||||||
|
return errors
|
||||||
|
|
||||||
|
|
||||||
|
def validate(slug: str, cards: list[Card]) -> list[str]:
|
||||||
|
"""Return every rule violation as a human-readable line (empty = ok)."""
|
||||||
|
errors: list[str] = []
|
||||||
|
if not MIN_CARDS <= len(cards) <= MAX_CARDS:
|
||||||
|
errors.append(f"{len(cards)} cards; want {MIN_CARDS}-{MAX_CARDS}")
|
||||||
|
for i, c in enumerate(cards, 1):
|
||||||
|
front, back = str(c.get("front", "")).strip(), str(c.get("back", "")).strip()
|
||||||
|
if not front or not back:
|
||||||
|
errors.append(f"card {i}: empty front or back")
|
||||||
|
continue
|
||||||
|
if "\n" in front or "\n" in back:
|
||||||
|
errors.append(f"card {i}: newlines are not allowed")
|
||||||
|
if len(front.split()) > FRONT_MAX_WORDS:
|
||||||
|
errors.append(
|
||||||
|
f"card {i}: front is {len(front.split())} words (max {FRONT_MAX_WORDS})"
|
||||||
|
)
|
||||||
|
if (n := len(back.split())) > BACK_MAX_WORDS:
|
||||||
|
errors.append(
|
||||||
|
f"card {i}: back is {n} words (max {BACK_MAX_WORDS}): {back!r}"
|
||||||
|
)
|
||||||
|
errors += shape_errors(i, front, back)
|
||||||
|
if not slug:
|
||||||
|
errors.append("missing slug")
|
||||||
|
return errors
|
||||||
|
|
||||||
|
|
||||||
|
def _cell(text: str) -> str:
|
||||||
|
"""Markdown table cell: a bare pipe would split it in two."""
|
||||||
|
return text.strip().replace("|", "\\|")
|
||||||
|
|
||||||
|
|
||||||
|
def _esc(text: str) -> str:
|
||||||
|
"""HTML-escape one field (quotes are left alone)."""
|
||||||
|
return html.escape(text.strip(), quote=False)
|
||||||
|
|
||||||
|
|
||||||
|
def table(cards: list[Card]) -> str:
|
||||||
|
"""Markdown table for the chat reply, so the model relays instead of retyping."""
|
||||||
|
rows = ["| Front | Back |", "|---|---|"]
|
||||||
|
rows += [f"| {_cell(c['front'])} | {_cell(c['back'])} |" for c in cards]
|
||||||
|
return "\n".join(rows)
|
||||||
|
|
||||||
|
|
||||||
|
def escaped(cards: list[Card]) -> list[Card]:
|
||||||
|
"""Anki fields are HTML: escape so `a<b` or `&` survive rendering."""
|
||||||
|
return [{"front": _esc(c["front"]), "back": _esc(c["back"])} for c in cards]
|
||||||
105
leetcode_coach/leetcode_coach/eval_local.py
Executable file
105
leetcode_coach/leetcode_coach/eval_local.py
Executable file
@ -0,0 +1,105 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Scripted acceptance run of the local coach on two-sum (no human needed).
|
||||||
|
|
||||||
|
Six canned student turns, including a demand for the code, a buggy paste and a
|
||||||
|
give-up. PASS only if every criterion holds on every reply:
|
||||||
|
|
||||||
|
no-code the no-solution gate finds nothing pasteable in the shown reply
|
||||||
|
one-task the reply ends with exactly one thing to do (one question, or an
|
||||||
|
imperative last line)
|
||||||
|
fast the reply (including gate regenerations) took < MAX_SECONDS
|
||||||
|
|
||||||
|
The model load is timed separately by a warm-up call and excluded: it happens
|
||||||
|
once per session, and the criterion is about the conversation's pace.
|
||||||
|
Writes the transcript to ~/data/claude-scratch/leetcode-local/. Its printed
|
||||||
|
table IS the verdict (T201 exemption in pyproject.toml).
|
||||||
|
|
||||||
|
python3 -m leetcode_coach.eval_local
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import datetime as dt
|
||||||
|
from pathlib import Path
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
|
||||||
|
from leetcode_coach import local_backend, local_cards, local_coach, local_prompt
|
||||||
|
from leetcode_coach.local_session import opening_message
|
||||||
|
|
||||||
|
MAX_SECONDS = 15.0
|
||||||
|
OUT_DIR = Path.home() / "data" / "claude-scratch" / "leetcode-local"
|
||||||
|
BUGGY = """var twoSum = function(nums, target) {
|
||||||
|
const seen = {};
|
||||||
|
for (let i = 0; i < nums.length; i++) {
|
||||||
|
seen[nums[i]] = i;
|
||||||
|
if (seen[target - nums[i]] !== undefined) return [seen[target - nums[i]], i];
|
||||||
|
}
|
||||||
|
};"""
|
||||||
|
TURNS = [
|
||||||
|
opening_message(["how", "to", "beat", "checking", "every", "pair"]),
|
||||||
|
"I'd do two loops and check every pair. That's slow though, right?",
|
||||||
|
"just give me the code",
|
||||||
|
"hint",
|
||||||
|
BUGGY,
|
||||||
|
"I give up, what would you write?",
|
||||||
|
]
|
||||||
|
Row = tuple[int, float, int, bool, bool, bool, bool]
|
||||||
|
|
||||||
|
|
||||||
|
def _print_table(rows: list[Row]) -> None:
|
||||||
|
"""One line per turn: timing, tries and the three criteria."""
|
||||||
|
print("turn secs tries withheld no-code one-task fast")
|
||||||
|
for n, secs, tries, withheld, ok_code, ok_task, ok_fast in rows:
|
||||||
|
print(
|
||||||
|
f"{n:>4} {secs:5.1f} {tries:>5} {withheld!s:>8} {ok_code!s:>7} "
|
||||||
|
f"{ok_task!s:>8} {ok_fast!s:>5}"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
"""Run the six turns, print a per-criterion table, exit 0 only on PASS."""
|
||||||
|
statement = subprocess.run(
|
||||||
|
[sys.executable, str(Path(__file__).with_name("fetch_problem.py")), "two-sum"],
|
||||||
|
capture_output=True,
|
||||||
|
text=True,
|
||||||
|
check=True,
|
||||||
|
).stdout
|
||||||
|
start = time.monotonic()
|
||||||
|
local_backend.chat([{"role": "user", "content": "Say ok."}])
|
||||||
|
print(f"model load + warm-up: {time.monotonic() - start:.1f}s")
|
||||||
|
coach = local_coach.LocalCoach.start(statement)
|
||||||
|
rows: list[Row] = []
|
||||||
|
log: list[str] = []
|
||||||
|
for n, said in enumerate(TURNS, 1):
|
||||||
|
turn = coach.respond(said)
|
||||||
|
shown, _ = local_cards.split_cards(turn.text)
|
||||||
|
leaks = local_coach.gate_findings(shown, statement, coach.user_texts())
|
||||||
|
rows.append(
|
||||||
|
(
|
||||||
|
n,
|
||||||
|
turn.seconds,
|
||||||
|
turn.attempts,
|
||||||
|
turn.withheld,
|
||||||
|
not leaks,
|
||||||
|
local_prompt.ends_with_one_task(shown),
|
||||||
|
turn.seconds < MAX_SECONDS,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
log.append(
|
||||||
|
f"## Turn {n}\n\n**kuhy:** {said}\n\n**coach** "
|
||||||
|
f"({turn.seconds:.1f}s, attempts {turn.attempts}):\n\n{shown}\n"
|
||||||
|
)
|
||||||
|
OUT_DIR.mkdir(parents=True, exist_ok=True)
|
||||||
|
stamp = dt.datetime.now(tz=dt.UTC).astimezone()
|
||||||
|
out = OUT_DIR / f"eval-{stamp:%Y%m%d-%H%M%S}.md"
|
||||||
|
out.write_text("\n".join(log), encoding="utf-8")
|
||||||
|
_print_table(rows)
|
||||||
|
passed = all(all(r[4:]) for r in rows)
|
||||||
|
print(f"{'PASS' if passed else 'FAIL'} transcript: {out}")
|
||||||
|
return 0 if passed else 1
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
179
leetcode_coach/leetcode_coach/fetch_problem.py
Executable file
179
leetcode_coach/leetcode_coach/fetch_problem.py
Executable file
@ -0,0 +1,179 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Fetch a LeetCode problem statement + starter code as markdown.
|
||||||
|
|
||||||
|
Usage: fetch_problem.py <url-or-slug> [--lang javascript]
|
||||||
|
|
||||||
|
Uses LeetCode's public GraphQL endpoint (no login). Exit codes:
|
||||||
|
0 printed the problem
|
||||||
|
1 bad input / network error
|
||||||
|
2 problem not available without login (premium) or unknown slug
|
||||||
|
|
||||||
|
Stdlib only and free of package imports on purpose: SKILL.md runs it by path
|
||||||
|
(`python3 ~/.claude/skills/leetcode/fetch_problem.py`), which works without
|
||||||
|
any install. Its stdout IS its interface (T201 exemption in pyproject.toml).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
from html.parser import HTMLParser
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import re
|
||||||
|
import sys
|
||||||
|
from typing import Any
|
||||||
|
import urllib.error
|
||||||
|
import urllib.request
|
||||||
|
|
||||||
|
# A fixed name, and its own NullHandler: run by path this module is
|
||||||
|
# `__main__` and the package __init__ (which silences the package logger)
|
||||||
|
# never loads, so without both the warning below would print a second copy of
|
||||||
|
# the "fetch failed" line through logging's last-resort handler.
|
||||||
|
_LOG = logging.getLogger("leetcode_coach.fetch_problem")
|
||||||
|
_LOG.addHandler(logging.NullHandler())
|
||||||
|
|
||||||
|
GRAPHQL = "https://leetcode.com/graphql"
|
||||||
|
QUERY = """
|
||||||
|
query q($s: String!) {
|
||||||
|
question(titleSlug: $s) {
|
||||||
|
questionFrontendId title difficulty content isPaidOnly
|
||||||
|
codeSnippets { langSlug code }
|
||||||
|
topicTags { name }
|
||||||
|
hints
|
||||||
|
}
|
||||||
|
}
|
||||||
|
"""
|
||||||
|
EXIT_FAILED, EXIT_UNAVAILABLE = 1, 2
|
||||||
|
_OPEN = {"p": "\n", "div": "\n", "ul": "\n", "ol": "\n", "li": "\n- ", "br": "\n"}
|
||||||
|
_CLOSE = {"p": "\n", "div": "\n"}
|
||||||
|
_BOTH = {"strong": "**", "b": "**", "em": "_", "i": "_"}
|
||||||
|
|
||||||
|
|
||||||
|
class _HtmlToText(HTMLParser):
|
||||||
|
"""Minimal HTML -> markdown-ish text; keeps <pre>/<code> verbatim."""
|
||||||
|
|
||||||
|
def __init__(self) -> None:
|
||||||
|
super().__init__()
|
||||||
|
self.out: list[str] = []
|
||||||
|
self.in_pre = False
|
||||||
|
self.in_code = False
|
||||||
|
|
||||||
|
def handle_starttag(self, tag: str, attrs: list[tuple[str, str | None]]) -> None:
|
||||||
|
"""Open a fence, an inline code span or a markdown marker."""
|
||||||
|
del attrs # no attribute matters for the text
|
||||||
|
if tag == "pre":
|
||||||
|
self.in_pre = True
|
||||||
|
self.out.append("\n```\n")
|
||||||
|
elif tag == "code" and not self.in_pre:
|
||||||
|
self.in_code = True
|
||||||
|
self.out.append("`")
|
||||||
|
elif tag in _OPEN or tag in _BOTH:
|
||||||
|
self.out.append(_OPEN.get(tag) or _BOTH[tag])
|
||||||
|
|
||||||
|
def handle_endtag(self, tag: str) -> None:
|
||||||
|
"""Close what `handle_starttag` opened."""
|
||||||
|
if tag == "pre":
|
||||||
|
self.in_pre = False
|
||||||
|
self.out.append("\n```\n")
|
||||||
|
elif tag == "code" and self.in_code:
|
||||||
|
self.in_code = False
|
||||||
|
self.out.append("`")
|
||||||
|
elif tag in _CLOSE or tag in _BOTH:
|
||||||
|
self.out.append(_CLOSE.get(tag) or _BOTH[tag])
|
||||||
|
|
||||||
|
def handle_data(self, data: str) -> None:
|
||||||
|
"""Text: verbatim inside <pre>, whitespace-collapsed elsewhere."""
|
||||||
|
self.out.append(data if self.in_pre else re.sub(r"\s+", " ", data))
|
||||||
|
|
||||||
|
def text(self) -> str:
|
||||||
|
"""Everything fed so far, with runs of blank lines squeezed."""
|
||||||
|
return re.sub(r"\n{3,}", "\n\n", "".join(self.out)).strip()
|
||||||
|
|
||||||
|
|
||||||
|
def slug_from(arg: str) -> str:
|
||||||
|
"""Accept a full URL (with /description/, /submissions/, ?query) or a bare slug."""
|
||||||
|
m = re.search(r"leetcode\.(?:com|cn)/problems/([a-z0-9-]+)", arg)
|
||||||
|
if m:
|
||||||
|
return m.group(1)
|
||||||
|
if re.fullmatch(r"[a-z0-9-]+", arg):
|
||||||
|
return arg
|
||||||
|
msg = f"not a LeetCode problem URL or slug: {arg!r}"
|
||||||
|
raise ValueError(msg)
|
||||||
|
|
||||||
|
|
||||||
|
def fetch(slug: str) -> dict[str, Any] | None:
|
||||||
|
"""The GraphQL `question` object for `slug`, or None when LeetCode has none."""
|
||||||
|
body = json.dumps({"query": QUERY, "variables": {"s": slug}}).encode()
|
||||||
|
req = urllib.request.Request(
|
||||||
|
GRAPHQL,
|
||||||
|
data=body,
|
||||||
|
headers={
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
"Referer": f"https://leetcode.com/problems/{slug}/",
|
||||||
|
"User-Agent": "Mozilla/5.0 (leetcode-coach skill)",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
with urllib.request.urlopen(req, timeout=20) as resp: # fixed https URL
|
||||||
|
payload = json.load(resp)
|
||||||
|
question: dict[str, Any] | None = payload.get("data", {}).get("question")
|
||||||
|
return question
|
||||||
|
|
||||||
|
|
||||||
|
def render(q: dict[str, Any], lang: str) -> str:
|
||||||
|
"""Markdown: title, tags, statement, starter code, then the hidden hints."""
|
||||||
|
parser = _HtmlToText()
|
||||||
|
parser.feed(q["content"] or "")
|
||||||
|
tags = ", ".join(t["name"] for t in q["topicTags"]) or "none"
|
||||||
|
snippets = {s["langSlug"]: s["code"] for s in q["codeSnippets"]}
|
||||||
|
chosen = lang if lang in snippets else next(iter(snippets), None)
|
||||||
|
lines = [
|
||||||
|
f"# {q['questionFrontendId']}. {q['title']} ({q['difficulty']})",
|
||||||
|
f"Tags: {tags}",
|
||||||
|
f"Languages available: {', '.join(snippets) or 'none'}",
|
||||||
|
"",
|
||||||
|
parser.text(),
|
||||||
|
"",
|
||||||
|
]
|
||||||
|
if chosen:
|
||||||
|
lines += [
|
||||||
|
f"## Starter code ({chosen})",
|
||||||
|
"```",
|
||||||
|
snippets[chosen].rstrip(),
|
||||||
|
"```",
|
||||||
|
]
|
||||||
|
if q["hints"]:
|
||||||
|
lines += ["", "## Site hints (do NOT show to the user unasked)"]
|
||||||
|
lines += [f"- {h}" for h in q["hints"]]
|
||||||
|
return "\n".join(lines)
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
"""Print the problem named on argv; see the module docstring for exit codes."""
|
||||||
|
ap = argparse.ArgumentParser(description=(__doc__ or "").splitlines()[0])
|
||||||
|
ap.add_argument("problem", help="LeetCode problem URL or title slug")
|
||||||
|
ap.add_argument("--lang", default="javascript", help="starter-code language slug")
|
||||||
|
args = ap.parse_args()
|
||||||
|
try:
|
||||||
|
slug = slug_from(args.problem)
|
||||||
|
q = fetch(slug)
|
||||||
|
except (
|
||||||
|
ValueError,
|
||||||
|
urllib.error.URLError,
|
||||||
|
TimeoutError,
|
||||||
|
json.JSONDecodeError,
|
||||||
|
) as exc:
|
||||||
|
_LOG.warning("fetch failed: %s", exc)
|
||||||
|
print(f"fetch failed: {exc}", file=sys.stderr)
|
||||||
|
return EXIT_FAILED
|
||||||
|
if q is None or q.get("isPaidOnly") or not q.get("content"):
|
||||||
|
print(
|
||||||
|
f"problem {slug!r} is unknown or premium-only (no statement without login)",
|
||||||
|
file=sys.stderr,
|
||||||
|
)
|
||||||
|
return EXIT_UNAVAILABLE
|
||||||
|
print(render(q, args.lang))
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
176
leetcode_coach/leetcode_coach/gate_detector.py
Normal file
176
leetcode_coach/leetcode_coach/gate_detector.py
Normal file
@ -0,0 +1,176 @@
|
|||||||
|
"""Transcript parsing and pasteable-snippet detection for the no-solution gate.
|
||||||
|
|
||||||
|
Shared by the Stop hook (`stop_gate.py`) and the offline coach
|
||||||
|
(`local_coach.py`), so both paths judge a reply with one detector. A snippet
|
||||||
|
is pasteable when a code span or fenced block uses an identifier from the
|
||||||
|
problem's starter signature together with a call/spread/assignment -- unless
|
||||||
|
that exact snippet already appears in kuhy's own messages or the fetched
|
||||||
|
problem (quoting their code back is fine).
|
||||||
|
|
||||||
|
Moved verbatim from ``~/.claude/hooks/leetcode_no_solution_stop.py``.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import re
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
_LOG = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
KEYWORDS = {
|
||||||
|
"function",
|
||||||
|
"return",
|
||||||
|
"const",
|
||||||
|
"let",
|
||||||
|
"var",
|
||||||
|
"self",
|
||||||
|
"def",
|
||||||
|
"class",
|
||||||
|
"this",
|
||||||
|
"new",
|
||||||
|
"true",
|
||||||
|
"false",
|
||||||
|
"null",
|
||||||
|
"None",
|
||||||
|
"int",
|
||||||
|
"str",
|
||||||
|
"list",
|
||||||
|
"number",
|
||||||
|
"string",
|
||||||
|
"void",
|
||||||
|
}
|
||||||
|
CODE_RE = re.compile(r"```.*?```|`[^`\n]+`", re.DOTALL)
|
||||||
|
SIG_RE = re.compile(
|
||||||
|
r"(?:function\s*(\w*)|def\s+(\w+)|(\w+)\s*=\s*(?:function)?)\s*\(([^)]*)\)"
|
||||||
|
)
|
||||||
|
# User-role messages that are not kuhy: the injected skill body and the hook's
|
||||||
|
# own block reason (which quotes the offending snippet and must not whitelist it).
|
||||||
|
NOT_KUHY = ("Base directory for this skill", "Stop hook feedback:")
|
||||||
|
MIN_IDENT_LEN = 2 # single letters (`t`) are too common in prose and toy code
|
||||||
|
LANG_PREFIXES = ("javascript", "js", "python", "typescript")
|
||||||
|
|
||||||
|
Entry = dict[str, Any]
|
||||||
|
|
||||||
|
|
||||||
|
def _parse(line: str) -> object:
|
||||||
|
"""One JSONL line, or None when it is not JSON (e.g. a half-written tail)."""
|
||||||
|
try:
|
||||||
|
return json.loads(line)
|
||||||
|
except json.JSONDecodeError as exc:
|
||||||
|
_LOG.warning("skipping a transcript line that is not JSON: %s", exc)
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def transcript_lines(raw: str) -> list[Entry]:
|
||||||
|
"""Every JSON object in a JSONL transcript, in order."""
|
||||||
|
return [d for d in map(_parse, raw.splitlines()) if isinstance(d, dict)]
|
||||||
|
|
||||||
|
|
||||||
|
def _content(msg: Entry) -> str | list[Entry]:
|
||||||
|
"""The `message.content` field: a string or a list of content blocks."""
|
||||||
|
content: str | list[Entry] = msg.get("message", {}).get("content", "")
|
||||||
|
return content
|
||||||
|
|
||||||
|
|
||||||
|
def text_of(msg: Entry) -> str:
|
||||||
|
"""All text in an entry, tool results included."""
|
||||||
|
content = _content(msg)
|
||||||
|
if isinstance(content, str):
|
||||||
|
return content
|
||||||
|
parts = []
|
||||||
|
for c in content:
|
||||||
|
if c.get("type") == "text":
|
||||||
|
parts.append(c.get("text", ""))
|
||||||
|
elif c.get("type") == "tool_result":
|
||||||
|
inner = c.get("content", "")
|
||||||
|
parts.append(
|
||||||
|
inner
|
||||||
|
if isinstance(inner, str)
|
||||||
|
else " ".join(x.get("text", "") for x in inner)
|
||||||
|
)
|
||||||
|
return "\n".join(parts)
|
||||||
|
|
||||||
|
|
||||||
|
def kuhy_text(msg: Entry) -> str:
|
||||||
|
"""Only what kuhy typed: no tool results, no skill body, no echoed hook reason."""
|
||||||
|
content = _content(msg)
|
||||||
|
texts = (
|
||||||
|
[content]
|
||||||
|
if isinstance(content, str)
|
||||||
|
else [c.get("text", "") for c in content if c.get("type") == "text"]
|
||||||
|
)
|
||||||
|
return "\n".join(t for t in texts if not t.startswith(NOT_KUHY))
|
||||||
|
|
||||||
|
|
||||||
|
def _signature_names(code: str) -> set[str]:
|
||||||
|
"""Function and parameter names of every signature in `code`."""
|
||||||
|
names: set[str] = set()
|
||||||
|
for sig in SIG_RE.finditer(code):
|
||||||
|
names.update(n for n in sig.groups()[:3] if n)
|
||||||
|
for raw in sig.group(4).split(","):
|
||||||
|
param = raw.strip().split(":")[0].split("=")[0].strip().lstrip("*")
|
||||||
|
if param:
|
||||||
|
names.add(param)
|
||||||
|
return names
|
||||||
|
|
||||||
|
|
||||||
|
def starter_identifiers(entries: list[Entry]) -> set[str]:
|
||||||
|
"""Function name + parameter names from the fetched '## Starter code' block."""
|
||||||
|
for e in entries:
|
||||||
|
if e.get("type") != "user":
|
||||||
|
continue
|
||||||
|
txt = text_of(e)
|
||||||
|
if "## Starter code" not in txt:
|
||||||
|
continue
|
||||||
|
block = txt.split("## Starter code", 1)[1]
|
||||||
|
m = re.search(r"```\n?(.*?)```", block, re.DOTALL)
|
||||||
|
code = m.group(1) if m else block
|
||||||
|
# LeetCode starters carry the test harness in a block comment; drop
|
||||||
|
# comments so its helpers (`log`, `...argsArr`) are not taken as symbols.
|
||||||
|
code = re.sub(r"/\*.*?\*/|//[^\n]*|#[^\n]*", "", code, flags=re.DOTALL)
|
||||||
|
names = _signature_names(code)
|
||||||
|
return {n for n in names if len(n) >= MIN_IDENT_LEN and n not in KEYWORDS}
|
||||||
|
return set()
|
||||||
|
|
||||||
|
|
||||||
|
def _squash(text: str) -> str:
|
||||||
|
"""Collapse every whitespace run to one space."""
|
||||||
|
return re.sub(r"\s+", " ", text).strip()
|
||||||
|
|
||||||
|
|
||||||
|
def allowed_text(entries: list[Entry]) -> str:
|
||||||
|
"""Everything the reply may quote: kuhy's messages and the fetched problem."""
|
||||||
|
parts = []
|
||||||
|
for e in entries:
|
||||||
|
if e.get("type") != "user":
|
||||||
|
continue
|
||||||
|
parts.append(kuhy_text(e))
|
||||||
|
content = _content(e)
|
||||||
|
if not isinstance(content, str):
|
||||||
|
parts.extend(
|
||||||
|
text_of({"message": {"content": [c]}})
|
||||||
|
for c in content
|
||||||
|
if c.get("type") == "tool_result"
|
||||||
|
)
|
||||||
|
return _squash("\n".join(parts))
|
||||||
|
|
||||||
|
|
||||||
|
def offending_snippets(reply: str, idents: set[str], allowed: str) -> list[str]:
|
||||||
|
"""Code in `reply` that uses a starter identifier and is not a quote."""
|
||||||
|
bad = []
|
||||||
|
for snippet in CODE_RE.findall(reply):
|
||||||
|
body = snippet.strip("`").strip()
|
||||||
|
if body.startswith(LANG_PREFIXES):
|
||||||
|
body = body.split("\n", 1)[1] if "\n" in body else ""
|
||||||
|
if not re.search(r"[(=]|\.\.\.", body):
|
||||||
|
continue
|
||||||
|
if not any(re.search(rf"\b{re.escape(i)}\b", body) for i in idents):
|
||||||
|
continue
|
||||||
|
probe = _squash(body).removesuffix("...").strip()
|
||||||
|
# Whole-token match: the harness has `...argsArr`; it must not admit `...args`.
|
||||||
|
if probe and re.search(rf"(?<!\w){re.escape(probe)}(?!\w)", allowed):
|
||||||
|
continue # quoting kuhy's code or the starter back at them
|
||||||
|
bad.append(_squash(body)[:80])
|
||||||
|
return bad
|
||||||
@ -0,0 +1,3 @@
|
|||||||
|
"""Shim: the /leetcode Stop gate lives in ~/src/leetcode-guard (leetcode_coach.stop_gate)."""
|
||||||
|
import os
|
||||||
|
if __name__ == "__main__": os.execv("/usr/bin/python3", ["/usr/bin/python3", "-P", "-m", "leetcode_coach.stop_gate"])
|
||||||
18
leetcode_coach/leetcode_coach/leetcode
Executable file
18
leetcode_coach/leetcode_coach/leetcode
Executable file
@ -0,0 +1,18 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
# ============================================================================
|
||||||
|
# leetcode — coaching session in print mode, on the cheapest model.
|
||||||
|
# Print mode matters: the no-solution Stop gate can only withhold a reply
|
||||||
|
# from a client that waits for hooks, and the interactive TUI streams first.
|
||||||
|
# leetcode <url-or-slug> [what you are stuck on...]
|
||||||
|
#
|
||||||
|
# Symlinked from ~/.local/bin/leetcode. The package lives in leetcode-guard;
|
||||||
|
# the repo root goes on PYTHONPATH so the launcher works from any checkout,
|
||||||
|
# installed or not, and -P keeps the caller's cwd off sys.path.
|
||||||
|
# ============================================================================
|
||||||
|
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
repo="$(dirname "$(dirname "$(readlink -f "$0")")")"
|
||||||
|
PYTHONPATH="${repo}${PYTHONPATH:+:${PYTHONPATH}}" \
|
||||||
|
exec /usr/bin/python3 -P -m leetcode_coach.leetcode_repl "$@"
|
||||||
146
leetcode_coach/leetcode_coach/leetcode_repl.py
Executable file
146
leetcode_coach/leetcode_coach/leetcode_repl.py
Executable file
@ -0,0 +1,146 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Coaching REPL for /leetcode that only ever shows the post-gate reply.
|
||||||
|
|
||||||
|
In the interactive TUI a Stop hook runs after the reply has streamed, so a
|
||||||
|
blocked reply (one that hands kuhy a pasteable line) is already on screen
|
||||||
|
before the rewrite appears. `claude -p` returns nothing until the hooks are
|
||||||
|
done, so this loop is the only client where the no-solution gate actually
|
||||||
|
withholds text from the human. Each turn is one `claude -p --resume` call.
|
||||||
|
|
||||||
|
When Claude answers with its usage-limit message the session continues on the
|
||||||
|
local model (`local_session.py`) with the conversation so far; `--local` starts
|
||||||
|
there. The signal is the synthetic reply "You've hit your session limit ·
|
||||||
|
resets 3pm (Europe/Warsaw)" (error rate_limit, HTTP 429), as recorded in real
|
||||||
|
transcripts on 2026-09-26.
|
||||||
|
|
||||||
|
Input: a block ends on an empty line (paste code, then press Enter twice).
|
||||||
|
`quit` or Ctrl-D leaves. `reveal` is passed through like any other message.
|
||||||
|
Started by the `leetcode` launcher as `python3 -m leetcode_coach.leetcode_repl`;
|
||||||
|
its prints ARE the session (T201 exemption in pyproject.toml).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
import re
|
||||||
|
import shutil
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from leetcode_coach import local_session
|
||||||
|
|
||||||
|
# Named, not __name__: under `-m` this module is `__main__`, which would escape
|
||||||
|
# the package logger's NullHandler and print the warning to the session.
|
||||||
|
_LOG = logging.getLogger("leetcode_coach.leetcode_repl")
|
||||||
|
|
||||||
|
# PATH lookup, not a fixed path: claude lives in ~/.local/bin here, and the
|
||||||
|
# old /usr/bin/claude default made every REPL turn crash (2026-10-20).
|
||||||
|
CLAUDE = os.environ.get("LEETCODE_CLAUDE_BIN") or shutil.which("claude") or "claude"
|
||||||
|
LIMIT_RE = re.compile(r"hit your (?:\w+ )?limit|usage limit reached", re.IGNORECASE)
|
||||||
|
RATE_LIMIT_STATUS = 429
|
||||||
|
|
||||||
|
|
||||||
|
def ask(args: list[str], prompt: str) -> dict[str, Any]:
|
||||||
|
"""One `claude -p` turn; the JSON result, or a stand-in carrying stderr."""
|
||||||
|
env = {**os.environ, "CLAUDE_TTY_FIX_OFF": "1"}
|
||||||
|
proc = subprocess.run(
|
||||||
|
[CLAUDE, "-p", *args, "--output-format", "json", prompt],
|
||||||
|
capture_output=True,
|
||||||
|
stdin=subprocess.DEVNULL, # claude -p would otherwise drain our REPL input
|
||||||
|
text=True,
|
||||||
|
env=env,
|
||||||
|
check=False,
|
||||||
|
cwd=Path.home(),
|
||||||
|
)
|
||||||
|
reply: object
|
||||||
|
try:
|
||||||
|
reply = json.loads(proc.stdout, strict=False) if proc.stdout.strip() else {}
|
||||||
|
except json.JSONDecodeError as exc:
|
||||||
|
_LOG.warning("claude -p printed something that is not JSON: %s", exc)
|
||||||
|
reply = {}
|
||||||
|
result: dict[str, Any] = reply if isinstance(reply, dict) else {}
|
||||||
|
result["_stderr"] = proc.stderr.strip()
|
||||||
|
if proc.returncode != 0 and "result" not in result:
|
||||||
|
result["result"] = f"[claude exited {proc.returncode}]\n{proc.stderr.strip()}"
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def is_limit_hit(reply: dict[str, Any]) -> bool:
|
||||||
|
"""Whether Claude refused this turn because the plan's usage limit is spent."""
|
||||||
|
if reply.get("api_error_status") == RATE_LIMIT_STATUS:
|
||||||
|
return True
|
||||||
|
text = f"{reply.get('result', '')}\n{reply.get('_stderr', '')}"
|
||||||
|
return bool(LIMIT_RE.search(text))
|
||||||
|
|
||||||
|
|
||||||
|
def read_block() -> str | None:
|
||||||
|
"""Lines until an empty one; None on EOF with nothing typed."""
|
||||||
|
print("\nyou> ", end="", flush=True)
|
||||||
|
lines: list[str] = []
|
||||||
|
while True:
|
||||||
|
try:
|
||||||
|
line = input()
|
||||||
|
except EOFError:
|
||||||
|
# Ctrl-D is how kuhy leaves; benign, logged only to satisfy the
|
||||||
|
# repo's no-silent-except rule (the package logger is silent).
|
||||||
|
_LOG.warning("stdin closed (Ctrl-D); ending the block")
|
||||||
|
return "\n".join(lines) if lines else None
|
||||||
|
if not line.strip():
|
||||||
|
if lines:
|
||||||
|
return "\n".join(lines)
|
||||||
|
continue
|
||||||
|
lines.append(line)
|
||||||
|
|
||||||
|
|
||||||
|
def run_claude(model: str, problem: list[str]) -> int:
|
||||||
|
"""The Claude session; hands over to the local model on a usage limit."""
|
||||||
|
opener = local_session.opening_message(problem[1:])
|
||||||
|
first = ask(["--model", model], "/leetcode " + " ".join(problem))
|
||||||
|
if is_limit_hit(first):
|
||||||
|
print(f"[{first.get('result', 'usage limit')} -- switching to local model]")
|
||||||
|
return local_session.run(problem[0], [], opener, read_block)
|
||||||
|
session = first.get("session_id")
|
||||||
|
print(first.get("result", ""))
|
||||||
|
if not session:
|
||||||
|
return 1
|
||||||
|
history = [
|
||||||
|
{"role": "user", "content": opener},
|
||||||
|
{"role": "assistant", "content": str(first.get("result", ""))},
|
||||||
|
]
|
||||||
|
while True:
|
||||||
|
block = read_block()
|
||||||
|
if block is None or local_session.is_quit(block):
|
||||||
|
return 0
|
||||||
|
print("\n...", flush=True)
|
||||||
|
reply = ask(["--model", model, "--resume", session], block)
|
||||||
|
if is_limit_hit(reply):
|
||||||
|
print(f"[{reply.get('result', 'usage limit')} -- continuing locally]")
|
||||||
|
return local_session.run(problem[0], history, block, read_block)
|
||||||
|
print(reply.get("result", ""))
|
||||||
|
history += [
|
||||||
|
{"role": "user", "content": block},
|
||||||
|
{"role": "assistant", "content": str(reply.get("result", ""))},
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv: list[str]) -> int:
|
||||||
|
"""Parse `[--local] <url-or-slug> [stuck on...]` and run a session."""
|
||||||
|
local = "--local" in argv
|
||||||
|
args = [a for a in argv if a != "--local"]
|
||||||
|
if not args or args[0] in {"-h", "--help"}:
|
||||||
|
print("Usage: leetcode [--local] <url-or-slug> [what you are stuck on...]")
|
||||||
|
print(" LEETCODE_MODEL=opus leetcode ... to use a stronger model")
|
||||||
|
print(" --local coach on the local Ollama model (free, offline)")
|
||||||
|
return 0
|
||||||
|
if local:
|
||||||
|
opener = local_session.opening_message(args[1:])
|
||||||
|
return local_session.run(args[0], [], opener, read_block)
|
||||||
|
return run_claude(os.environ.get("LEETCODE_MODEL", "sonnet"), args)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main(sys.argv[1:]))
|
||||||
151
leetcode_coach/leetcode_coach/local_backend.py
Normal file
151
leetcode_coach/leetcode_coach/local_backend.py
Normal file
@ -0,0 +1,151 @@
|
|||||||
|
"""Ollama transport for the offline /leetcode coach.
|
||||||
|
|
||||||
|
One non-streaming `/api/chat` call per reply, so the no-solution gate can judge
|
||||||
|
the whole reply before anything reaches the screen -- the same reason the
|
||||||
|
Claude path uses `claude -p`. Thinking is switched off (`think: false`) and any
|
||||||
|
`<think>` block that leaks anyway is stripped before the gate sees the text.
|
||||||
|
|
||||||
|
Ollama silently truncates a prompt longer than `num_ctx` from the front, which
|
||||||
|
would drop the system prompt (the coaching rules) without an error. So the
|
||||||
|
context is set explicitly and both the client-side estimate and the server's
|
||||||
|
`prompt_eval_count` are checked against it.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from typing import TYPE_CHECKING, Any
|
||||||
|
import urllib.request
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from collections.abc import Callable
|
||||||
|
|
||||||
|
OLLAMA_URL = "http://localhost:11434"
|
||||||
|
# The instruct build, not plain `qwen3:30b-a3b`: that tag is the 2507 Thinking
|
||||||
|
# build, which ignores `think: false` and `/no_think` and writes ~1k tokens of
|
||||||
|
# reasoning into `content` before every reply (measured 2026-10-03: 66 s turns).
|
||||||
|
MODEL = "qwen3:30b-a3b-instruct-2507-q4_K_M"
|
||||||
|
NUM_CTX = 16384
|
||||||
|
KEEP_ALIVE = "5m" # free the 3090 soon after a session ends
|
||||||
|
TEMPERATURE = 0.4
|
||||||
|
REQUEST_TIMEOUT_S = 300.0 # the first call also loads ~18 GB into VRAM
|
||||||
|
CHARS_PER_TOKEN = 3.0 # pessimistic, so the estimate trips before Ollama truncates
|
||||||
|
GPU_HOG_MIB = 2048
|
||||||
|
THINK_RE = re.compile(r"<think>.*?</think>\s*", re.DOTALL)
|
||||||
|
|
||||||
|
|
||||||
|
class ContextOverflowError(RuntimeError):
|
||||||
|
"""The conversation no longer fits in `NUM_CTX`; the rules would be cut."""
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class LocalReply:
|
||||||
|
"""One model reply, already stripped of thinking."""
|
||||||
|
|
||||||
|
text: str
|
||||||
|
seconds: float
|
||||||
|
prompt_tokens: int
|
||||||
|
|
||||||
|
|
||||||
|
def strip_think(text: str) -> str:
|
||||||
|
"""Drop `<think>...</think>` blocks, a dangling close and a dangling open."""
|
||||||
|
text = THINK_RE.sub("", text)
|
||||||
|
if "</think>" in text: # a thinking build opens the block in its template
|
||||||
|
text = text.rsplit("</think>", 1)[1]
|
||||||
|
if "<think>" in text:
|
||||||
|
text = text.split("<think>", 1)[0]
|
||||||
|
return text.strip()
|
||||||
|
|
||||||
|
|
||||||
|
def http_json(
|
||||||
|
url: str, payload: dict[str, Any] | None, timeout: float
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""POST `payload` as JSON (GET when None) and decode the JSON answer."""
|
||||||
|
if not url.startswith(OLLAMA_URL + "/"):
|
||||||
|
msg = f"refusing a non-Ollama URL: {url}"
|
||||||
|
raise ValueError(msg)
|
||||||
|
data = None if payload is None else json.dumps(payload).encode()
|
||||||
|
req = urllib.request.Request(
|
||||||
|
url, data=data, headers={"Content-Type": "application/json"}
|
||||||
|
)
|
||||||
|
with urllib.request.urlopen(req, timeout=timeout) as resp: # localhost only
|
||||||
|
answer: dict[str, Any] = json.loads(resp.read().decode())
|
||||||
|
return answer
|
||||||
|
|
||||||
|
|
||||||
|
def estimate_tokens(messages: list[dict[str, str]]) -> int:
|
||||||
|
"""Upper-ish bound on prompt tokens, from character count."""
|
||||||
|
return int(sum(len(m["content"]) for m in messages) / CHARS_PER_TOKEN)
|
||||||
|
|
||||||
|
|
||||||
|
def chat(
|
||||||
|
messages: list[dict[str, str]],
|
||||||
|
*,
|
||||||
|
post: Callable[[str, dict[str, Any] | None, float], dict[str, Any]] = http_json,
|
||||||
|
clock: Callable[[], float] = time.monotonic,
|
||||||
|
) -> LocalReply:
|
||||||
|
"""Ask the local model for the next assistant message.
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
ContextOverflowError: when the prompt would not fit in `NUM_CTX`.
|
||||||
|
"""
|
||||||
|
if estimate_tokens(messages) >= NUM_CTX:
|
||||||
|
msg = f"conversation too long for num_ctx={NUM_CTX}; start a new session"
|
||||||
|
raise ContextOverflowError(msg)
|
||||||
|
payload = {
|
||||||
|
"model": MODEL,
|
||||||
|
"messages": messages,
|
||||||
|
"stream": False,
|
||||||
|
"think": False,
|
||||||
|
"keep_alive": KEEP_ALIVE,
|
||||||
|
"options": {"num_ctx": NUM_CTX, "temperature": TEMPERATURE},
|
||||||
|
}
|
||||||
|
start = clock()
|
||||||
|
data = post(f"{OLLAMA_URL}/api/chat", payload, REQUEST_TIMEOUT_S)
|
||||||
|
seconds = clock() - start
|
||||||
|
prompt_tokens = int(data.get("prompt_eval_count") or 0)
|
||||||
|
if prompt_tokens >= NUM_CTX:
|
||||||
|
msg = f"Ollama evaluated {prompt_tokens} prompt tokens >= num_ctx={NUM_CTX}"
|
||||||
|
raise ContextOverflowError(msg)
|
||||||
|
text = strip_think(str(data.get("message", {}).get("content") or ""))
|
||||||
|
return LocalReply(text=text, seconds=seconds, prompt_tokens=prompt_tokens)
|
||||||
|
|
||||||
|
|
||||||
|
def model_loaded(
|
||||||
|
post: Callable[[str, dict[str, Any] | None, float], dict[str, Any]] = http_json,
|
||||||
|
) -> bool:
|
||||||
|
"""Whether `MODEL` is already resident (then loading costs nothing new)."""
|
||||||
|
data = post(f"{OLLAMA_URL}/api/ps", None, 5.0)
|
||||||
|
return any(m.get("name") == MODEL for m in data.get("models", []))
|
||||||
|
|
||||||
|
|
||||||
|
def gpu_hogs(
|
||||||
|
run: Callable[..., subprocess.CompletedProcess[str]] = subprocess.run,
|
||||||
|
) -> list[tuple[str, int]]:
|
||||||
|
"""Non-Ollama GPU processes holding more than `GPU_HOG_MIB` (likely a game).
|
||||||
|
|
||||||
|
Loading an 18 GB model next to one would either starve it or spill the
|
||||||
|
model onto the CPU, so the REPL asks first.
|
||||||
|
"""
|
||||||
|
proc = run(
|
||||||
|
[
|
||||||
|
"nvidia-smi",
|
||||||
|
"--query-compute-apps=process_name,used_memory",
|
||||||
|
"--format=csv,noheader,nounits",
|
||||||
|
],
|
||||||
|
capture_output=True,
|
||||||
|
text=True,
|
||||||
|
check=False,
|
||||||
|
)
|
||||||
|
hogs = []
|
||||||
|
for line in proc.stdout.splitlines():
|
||||||
|
name, _, mib = line.rpartition(",")
|
||||||
|
if not mib.strip().isdigit() or "ollama" in name.lower():
|
||||||
|
continue
|
||||||
|
if int(mib) > GPU_HOG_MIB:
|
||||||
|
hogs.append((name.strip(), int(mib)))
|
||||||
|
return hogs
|
||||||
53
leetcode_coach/leetcode_coach/local_cards.py
Normal file
53
leetcode_coach/leetcode_coach/local_cards.py
Normal file
@ -0,0 +1,53 @@
|
|||||||
|
"""Post-mortem cards from a local reply: extract the ```cards block, build the deck.
|
||||||
|
|
||||||
|
The local model has no tools, so instead of the Bash call SKILL.md describes it
|
||||||
|
emits the JSON in a fenced ```cards block; this module lifts it out of the
|
||||||
|
displayed text and pipes it to `make_cards`, whose validation is unchanged.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
from typing import TYPE_CHECKING, Any
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from collections.abc import Callable
|
||||||
|
|
||||||
|
_LOG = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
CARDS_RE = re.compile(r"```cards\s*\n(.*?)```", re.DOTALL)
|
||||||
|
|
||||||
|
|
||||||
|
def split_cards(reply: str) -> tuple[str, dict[str, Any] | None]:
|
||||||
|
"""Remove a ```cards block from `reply`; return (display text, parsed JSON)."""
|
||||||
|
match = CARDS_RE.search(reply)
|
||||||
|
if match is None:
|
||||||
|
return reply, None
|
||||||
|
display = (reply[: match.start()] + reply[match.end() :]).strip()
|
||||||
|
try:
|
||||||
|
payload = json.loads(match.group(1))
|
||||||
|
except json.JSONDecodeError as exc:
|
||||||
|
_LOG.warning("the cards block is not JSON: %s", exc)
|
||||||
|
return display, {"invalid": match.group(1)}
|
||||||
|
return display, payload if isinstance(payload, dict) else {"invalid": payload}
|
||||||
|
|
||||||
|
|
||||||
|
def deliver_cards(
|
||||||
|
payload: dict[str, Any],
|
||||||
|
run: Callable[..., subprocess.CompletedProcess[str]] = subprocess.run,
|
||||||
|
) -> tuple[bool, str]:
|
||||||
|
"""Pipe the cards to the card builder; (accepted, its stdout+stderr)."""
|
||||||
|
if "invalid" in payload:
|
||||||
|
return False, f"cards block is not a JSON object: {payload['invalid']!r}"
|
||||||
|
proc = run(
|
||||||
|
[sys.executable, "-m", "leetcode_coach.make_cards", "-"],
|
||||||
|
input=json.dumps(payload),
|
||||||
|
capture_output=True,
|
||||||
|
text=True,
|
||||||
|
check=False,
|
||||||
|
)
|
||||||
|
return proc.returncode == 0, (proc.stdout + proc.stderr).strip()
|
||||||
118
leetcode_coach/leetcode_coach/local_coach.py
Normal file
118
leetcode_coach/leetcode_coach/local_coach.py
Normal file
@ -0,0 +1,118 @@
|
|||||||
|
"""The offline /leetcode coach: SKILL.md rules, the no-solution gate, cards.
|
||||||
|
|
||||||
|
Claude Code enforces the gate as a Stop hook; there is no harness here, so the
|
||||||
|
same detector (`gate_detector`, which the Stop hook also uses) runs on every
|
||||||
|
local reply before it is shown. A blocked reply is sent back with the gate's
|
||||||
|
reason, like the hook's `{"decision": "block"}`, and after `MAX_TRIES` failed
|
||||||
|
drafts a fixed safe message is shown instead -- unlike the hook, which fails
|
||||||
|
open, because a small local model is likelier to keep leaking.
|
||||||
|
|
||||||
|
The model has no tools: the statement is prefetched into the system prompt,
|
||||||
|
and cards come back as a ```cards block (`local_cards.py`). "End with exactly
|
||||||
|
one task" is checked here too, as a soft gate: a reply that fails only that is
|
||||||
|
regenerated, and after `MAX_TRIES` the last code-free draft is shown anyway.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import TYPE_CHECKING
|
||||||
|
|
||||||
|
from leetcode_coach import gate_detector, local_backend, local_cards, local_prompt
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from collections.abc import Callable
|
||||||
|
|
||||||
|
SKILL_DIR = Path(__file__).resolve().parent
|
||||||
|
MAX_TRIES = 3
|
||||||
|
|
||||||
|
|
||||||
|
def gate_findings(reply: str, statement: str, user_texts: list[str]) -> list[str]:
|
||||||
|
"""Snippets in `reply` the hook would block, judged on synthetic entries."""
|
||||||
|
if any(t.strip() == "reveal" for t in user_texts):
|
||||||
|
return []
|
||||||
|
entries = [
|
||||||
|
{"type": "user", "message": {"content": text}}
|
||||||
|
for text in [statement, *user_texts]
|
||||||
|
]
|
||||||
|
idents = gate_detector.starter_identifiers(entries)
|
||||||
|
if not idents:
|
||||||
|
return []
|
||||||
|
allowed = gate_detector.allowed_text(entries)
|
||||||
|
return list(gate_detector.offending_snippets(reply, idents, allowed))
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class Turn:
|
||||||
|
"""What one user message produced."""
|
||||||
|
|
||||||
|
text: str
|
||||||
|
seconds: float
|
||||||
|
attempts: int
|
||||||
|
withheld: bool
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class LocalCoach:
|
||||||
|
"""A local coaching conversation, gate applied to every reply."""
|
||||||
|
|
||||||
|
statement: str
|
||||||
|
history: list[dict[str, str]]
|
||||||
|
chat: Callable[[list[dict[str, str]]], local_backend.LocalReply] = (
|
||||||
|
local_backend.chat
|
||||||
|
)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def start(
|
||||||
|
cls, statement: str, history: list[dict[str, str]] | None = None
|
||||||
|
) -> LocalCoach:
|
||||||
|
"""A coach seeded with SKILL.md and optionally a prior (Claude) history."""
|
||||||
|
skill = (SKILL_DIR / "SKILL.md").read_text(encoding="utf-8")
|
||||||
|
prompt = local_prompt.system_prompt(skill, statement)
|
||||||
|
system = {"role": "system", "content": prompt}
|
||||||
|
return cls(statement=statement, history=[system, *(history or [])])
|
||||||
|
|
||||||
|
def user_texts(self) -> list[str]:
|
||||||
|
"""Everything kuhy typed so far."""
|
||||||
|
return [m["content"] for m in self.history if m["role"] == "user"]
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _with_reminder(history: list[dict[str, str]]) -> list[dict[str, str]]:
|
||||||
|
"""`history` with `REMINDER` on its last message (a copy; history is kept)."""
|
||||||
|
last = history[-1]
|
||||||
|
reminded = {**last, "content": last["content"] + local_prompt.REMINDER}
|
||||||
|
return [*history[:-1], reminded]
|
||||||
|
|
||||||
|
def respond(self, text: str) -> Turn:
|
||||||
|
"""Append kuhy's message and return the first reply that passes the gate."""
|
||||||
|
self.history.append({"role": "user", "content": text})
|
||||||
|
scratch: list[dict[str, str]] = []
|
||||||
|
total = 0.0
|
||||||
|
last_clean: str | None = None
|
||||||
|
for attempt in range(1, MAX_TRIES + 1):
|
||||||
|
reply = self.chat(self._with_reminder(self.history) + scratch)
|
||||||
|
total += reply.seconds
|
||||||
|
display, payload = local_cards.split_cards(reply.text)
|
||||||
|
bad = gate_findings(display, self.statement, self.user_texts())
|
||||||
|
if not bad:
|
||||||
|
last_clean = reply.text
|
||||||
|
if payload is not None or local_prompt.ends_with_one_task(display):
|
||||||
|
return self._accept(reply.text, total, attempt)
|
||||||
|
reason = local_prompt.GATE_REASON_SHAPE
|
||||||
|
else:
|
||||||
|
snippets = "; ".join(f"`{b}`" for b in bad)
|
||||||
|
reason = local_prompt.GATE_REASON_CODE.format(snippets=snippets)
|
||||||
|
scratch += [
|
||||||
|
{"role": "assistant", "content": reply.text},
|
||||||
|
{"role": "user", "content": reason},
|
||||||
|
]
|
||||||
|
if last_clean is not None: # shape is a soft rule; code is the hard one
|
||||||
|
return self._accept(last_clean, total, MAX_TRIES)
|
||||||
|
self.history.append({"role": "assistant", "content": local_prompt.WITHHELD})
|
||||||
|
return Turn(local_prompt.WITHHELD, total, MAX_TRIES, withheld=True)
|
||||||
|
|
||||||
|
def _accept(self, text: str, seconds: float, attempts: int) -> Turn:
|
||||||
|
"""Record `text` as the assistant's reply and wrap it as a Turn."""
|
||||||
|
self.history.append({"role": "assistant", "content": text})
|
||||||
|
return Turn(text, seconds, attempts, withheld=False)
|
||||||
101
leetcode_coach/leetcode_coach/local_prompt.py
Normal file
101
leetcode_coach/leetcode_coach/local_prompt.py
Normal file
@ -0,0 +1,101 @@
|
|||||||
|
"""What the offline coach tells the local model, and the one-task shape check.
|
||||||
|
|
||||||
|
Split out of `local_coach.py` for the 250-line cap; texts are unchanged.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
|
||||||
|
WITHHELD = (
|
||||||
|
"(My draft gave away code, so I held it back.) Let's go one step smaller: "
|
||||||
|
"in your own words, what does your current approach do with each number "
|
||||||
|
"as it reads it?"
|
||||||
|
)
|
||||||
|
REMINDER = (
|
||||||
|
"\n\n[coach reminder, not from kuhy: one hint rung at most; never state "
|
||||||
|
"the key insight, the data structure or the fix; never answer your own "
|
||||||
|
"question; no code with the starter's names; end with exactly one "
|
||||||
|
"question or one instruction.]"
|
||||||
|
)
|
||||||
|
"""Appended to the newest user message for the model only -- never stored in
|
||||||
|
`history` and never seen by the gate. A 3B-active model follows the rules at
|
||||||
|
the top of a 3k-token system prompt far less reliably than a line next to the
|
||||||
|
question it is answering (measured on the eval: it named the fix in prose)."""
|
||||||
|
LOCAL_NOTES = """
|
||||||
|
## Local client notes (you are running offline, not in Claude Code)
|
||||||
|
|
||||||
|
- You have NO tools. The problem statement and starter code are already below;
|
||||||
|
never say you will fetch, run or test anything.
|
||||||
|
- Write plain markdown for a terminal. No emoji.
|
||||||
|
- Never write code that uses the starter's function or parameter names unless
|
||||||
|
kuhy has typed `reveal`. A client-side gate rejects such replies.
|
||||||
|
- Never name the data structure or algorithm that solves the problem before
|
||||||
|
rung H3 -- not even inside a calibration question. Ask about what kuhy
|
||||||
|
already said instead.
|
||||||
|
- The LAST line of every reply is exactly one question or one instruction for
|
||||||
|
kuhy. Nothing after it. ONE question mark in the whole closing paragraph:
|
||||||
|
never stack questions ("What happens? Why?").
|
||||||
|
- Never state the key insight, the fix, or what the code "must" do. On a buggy
|
||||||
|
paste, describe what the data does on one concrete input from the examples
|
||||||
|
("with [3,2,4] and target 6, at i=0 `seen` already holds 3 when the lookup
|
||||||
|
runs") and ask what that lookup returns. kuhy names the fix, not you.
|
||||||
|
- When kuhy gives up or begs for the answer, do not just point at `reveal`:
|
||||||
|
drop back one rung and end with a smaller question. You may mention once
|
||||||
|
that typing `reveal` ends the exercise.
|
||||||
|
- Never repeat a previous reply. Answer what kuhy just wrote.
|
||||||
|
- For the post-mortem cards (section 5b), do NOT print a Bash command. Emit one
|
||||||
|
fenced block tagged `cards` holding the JSON, e.g.
|
||||||
|
```cards
|
||||||
|
{"slug": "<slug>", "cards": [{"front": "JS: ...?", "back": "..."}]}
|
||||||
|
```
|
||||||
|
The client builds the deck and shows the result.
|
||||||
|
|
||||||
|
# The problem (already fetched)
|
||||||
|
|
||||||
|
"""
|
||||||
|
GATE_REASON_CODE = (
|
||||||
|
"leetcode gate: your reply hands kuhy code they can paste -- {snippets}. "
|
||||||
|
"Rewrite it without that snippet: describe the data or "
|
||||||
|
"name the construct, and keep the one question at the end."
|
||||||
|
)
|
||||||
|
GATE_REASON_SHAPE = (
|
||||||
|
"leetcode gate: end with exactly ONE question or ONE "
|
||||||
|
"instruction -- your closing paragraph asks more than one "
|
||||||
|
"thing or ends on neither. Rewrite it."
|
||||||
|
)
|
||||||
|
IMPERATIVE = re.compile(
|
||||||
|
r"^(try|write|predict|explain|tell|describe|trace|run|paste|list|sketch|"
|
||||||
|
r"walk|show|give|say|name|work|think|check|change|fix|look|answer|type|"
|
||||||
|
r"fill|pick|do)\b",
|
||||||
|
re.IGNORECASE,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def system_prompt(skill_md: str, statement: str) -> str:
|
||||||
|
"""SKILL.md minus frontmatter, fetch step and model gate, plus the problem."""
|
||||||
|
body = skill_md.split("\n---\n", 1)[1] if skill_md.startswith("---") else skill_md
|
||||||
|
body = re.sub(r"## 1\. Fetch the problem.*?(?=\n## 2\.)", "", body, flags=re.DOTALL)
|
||||||
|
body = body.split("\n## Model and client", 1)[0]
|
||||||
|
return body.strip() + "\n" + LOCAL_NOTES + statement.strip() + "\n"
|
||||||
|
|
||||||
|
|
||||||
|
def ends_with_one_task(text: str) -> bool:
|
||||||
|
"""Closing paragraph asks at most one question and ends on a task.
|
||||||
|
|
||||||
|
The task is the final sentence: a question, or an imperative ("Predict
|
||||||
|
before I tell you."). A question followed by the instruction to answer it
|
||||||
|
is still one task; two question marks are two. Code blocks are not prose,
|
||||||
|
and a quoted question ("has this been seen?") is not a second question.
|
||||||
|
"""
|
||||||
|
prose = re.sub(r"```.*?```", "\n\n", text, flags=re.DOTALL)
|
||||||
|
paras = [p.strip() for p in re.split(r"\n\s*\n", prose) if p.strip()]
|
||||||
|
if not paras:
|
||||||
|
return False
|
||||||
|
last = re.sub(r"\"[^\"]*\"|“[^”]*”", "it", paras[-1])
|
||||||
|
last = re.sub(r"[*_>`()]", "", last).strip()
|
||||||
|
if last.count("?") > 1:
|
||||||
|
return False
|
||||||
|
sentences = [x for x in re.split(r"(?<=[.?!])\s+", last) if x.strip()]
|
||||||
|
final = sentences[-1].strip()
|
||||||
|
return final.endswith("?") or bool(IMPERATIVE.match(final))
|
||||||
122
leetcode_coach/leetcode_coach/local_session.py
Normal file
122
leetcode_coach/leetcode_coach/local_session.py
Normal file
@ -0,0 +1,122 @@
|
|||||||
|
"""Run a /leetcode coaching loop on the local model.
|
||||||
|
|
||||||
|
Entered by `leetcode --local`, or by the REPL mid-session when Claude reports a
|
||||||
|
usage limit -- in which case `history` already holds the turns so far, so the
|
||||||
|
local model picks up where Claude stopped instead of starting over. Its prints
|
||||||
|
ARE the session (T201 exemption in pyproject.toml).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
from typing import TYPE_CHECKING
|
||||||
|
import urllib.error
|
||||||
|
|
||||||
|
from leetcode_coach import local_backend, local_cards, local_coach
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from collections.abc import Callable
|
||||||
|
|
||||||
|
_LOG = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
CARD_RETRIES = 2
|
||||||
|
FETCH_PREMIUM = 2
|
||||||
|
|
||||||
|
|
||||||
|
def opening_message(stuck: list[str]) -> str:
|
||||||
|
"""The first user message, standing in for `/leetcode <url> [stuck on]`."""
|
||||||
|
said = " ".join(stuck).strip()
|
||||||
|
return "Start the session. " + (
|
||||||
|
f"What I'm stuck on: {said}" if said else "I have not said what I'm stuck on."
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def is_quit(block: str) -> bool:
|
||||||
|
"""An explicit request to leave."""
|
||||||
|
return block.strip().lower() in {"quit", "exit"}
|
||||||
|
|
||||||
|
|
||||||
|
def fetch_statement(
|
||||||
|
problem: str,
|
||||||
|
read: Callable[[], str | None],
|
||||||
|
runner: Callable[..., subprocess.CompletedProcess[str]] = subprocess.run,
|
||||||
|
) -> str | None:
|
||||||
|
"""Statement + starter code, or whatever kuhy pastes when it cannot be fetched."""
|
||||||
|
proc = runner(
|
||||||
|
[sys.executable, str(local_coach.SKILL_DIR / "fetch_problem.py"), problem],
|
||||||
|
capture_output=True,
|
||||||
|
text=True,
|
||||||
|
check=False,
|
||||||
|
)
|
||||||
|
if proc.returncode == 0:
|
||||||
|
return proc.stdout
|
||||||
|
why = "premium or unknown problem" if proc.returncode == FETCH_PREMIUM else ""
|
||||||
|
print(f"[could not fetch the problem: {why or proc.stderr.strip()}]")
|
||||||
|
print("Paste the statement and starter code, then an empty line:")
|
||||||
|
return read()
|
||||||
|
|
||||||
|
|
||||||
|
def confirm_gpu(ask: Callable[[str], str] = input) -> bool:
|
||||||
|
"""Ask before loading ~18 GB of VRAM next to another GPU-heavy process."""
|
||||||
|
if local_backend.model_loaded():
|
||||||
|
return True
|
||||||
|
hogs = local_backend.gpu_hogs()
|
||||||
|
if not hogs:
|
||||||
|
return True
|
||||||
|
print("[the GPU is busy -- loading the local model would compete with:]")
|
||||||
|
for name, mib in hogs:
|
||||||
|
print(f" {name} {mib} MiB")
|
||||||
|
answer = ask(f"Load {local_backend.MODEL} anyway? [y/N] ")
|
||||||
|
return answer.strip().lower() in {"y", "yes"}
|
||||||
|
|
||||||
|
|
||||||
|
def show(turn: local_coach.Turn, coach: local_coach.LocalCoach) -> None:
|
||||||
|
"""Print one reply; build its cards deck, feeding rejections back."""
|
||||||
|
for _ in range(CARD_RETRIES + 1):
|
||||||
|
display, payload = local_cards.split_cards(turn.text)
|
||||||
|
print(display)
|
||||||
|
if payload is None:
|
||||||
|
return
|
||||||
|
ok, out = local_cards.deliver_cards(payload)
|
||||||
|
print(out)
|
||||||
|
if ok:
|
||||||
|
return
|
||||||
|
print("\n...", flush=True)
|
||||||
|
turn = coach.respond(
|
||||||
|
f"make_cards.py rejected the cards:\n{out}\n"
|
||||||
|
"Fix the named card and emit the whole ```cards block again."
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def run(
|
||||||
|
problem: str,
|
||||||
|
history: list[dict[str, str]],
|
||||||
|
first: str,
|
||||||
|
read: Callable[[], str | None],
|
||||||
|
) -> int:
|
||||||
|
"""Coach locally until kuhy quits. `first` is sent before reading input."""
|
||||||
|
statement = fetch_statement(problem, read)
|
||||||
|
if not statement:
|
||||||
|
return 1
|
||||||
|
try:
|
||||||
|
if not confirm_gpu():
|
||||||
|
print("[not loading the local model]")
|
||||||
|
return 1
|
||||||
|
coach = local_coach.LocalCoach.start(statement, history)
|
||||||
|
print(f"[local: {local_backend.MODEL} via Ollama]\n\n...", flush=True)
|
||||||
|
show(coach.respond(first), coach)
|
||||||
|
while (block := read()) is not None and not is_quit(block):
|
||||||
|
print("\n...", flush=True)
|
||||||
|
show(coach.respond(block), coach)
|
||||||
|
except (urllib.error.URLError, ConnectionError) as exc:
|
||||||
|
_LOG.warning("Ollama is not reachable: %s", exc)
|
||||||
|
print(f"[Ollama is not reachable at {local_backend.OLLAMA_URL}: {exc}]")
|
||||||
|
print("[start it with: systemctl start ollama]")
|
||||||
|
return 1
|
||||||
|
except local_backend.ContextOverflowError as exc:
|
||||||
|
_LOG.warning("local session ended: %s", exc)
|
||||||
|
print(f"[{exc}]")
|
||||||
|
return 1
|
||||||
|
return 0
|
||||||
170
leetcode_coach/leetcode_coach/make_cards.py
Executable file
170
leetcode_coach/leetcode_coach/make_cards.py
Executable file
@ -0,0 +1,170 @@
|
|||||||
|
#!/usr/bin/python3
|
||||||
|
"""Turn a /leetcode post-mortem into an Anki deck file (.apkg).
|
||||||
|
|
||||||
|
Usage: make_cards.py [--dry-run | --replace] <cards.json | -> (- = stdin)
|
||||||
|
{"slug": "timeout-cancellation",
|
||||||
|
"cards": [{"front": "JS: cancel a pending setTimeout?",
|
||||||
|
"back": "clearTimeout(id)"}]}
|
||||||
|
|
||||||
|
The shape rules (word caps, coin-flip and placeholder fronts) live in
|
||||||
|
`card_rules.py`; a violation fails the run (exit 1) and names the offender.
|
||||||
|
Deck and model ids are fixed and the note guid is derived from (slug, front),
|
||||||
|
so re-importing a regenerated file updates the same notes instead of
|
||||||
|
duplicating them. Fields are HTML, so text is escaped (`a<b` would vanish).
|
||||||
|
|
||||||
|
--dry-run validate and print the table; no log, no delivery
|
||||||
|
--replace make this the problem's whole card set: notes of the slug that
|
||||||
|
are not in the input get deleted. Needs Anki open (AnkiConnect);
|
||||||
|
an .apkg import cannot delete, so without it the run fails.
|
||||||
|
|
||||||
|
Delivery, in order of preference:
|
||||||
|
1. Anki desktop is open with AnkiConnect -> notes are upserted directly and
|
||||||
|
a sync to AnkiWeb is triggered, so AnkiDroid gets them at its next open.
|
||||||
|
2. Otherwise -> ~/data/learn/leetcode/anki/<date>-<slug>.apkg, imported by
|
||||||
|
`anki <file>` (which also starts Anki).
|
||||||
|
Either way the cards are appended to ~/data/learn/leetcode/cards.tsv.
|
||||||
|
|
||||||
|
Runs on the system python (genanki in its user site, installed by
|
||||||
|
install.sh); it used to be a `uv run --script` file. Its printed table and
|
||||||
|
delivery line ARE its interface (T201 exemption in pyproject.toml).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import datetime as dt
|
||||||
|
import html
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
import shlex
|
||||||
|
import sys
|
||||||
|
from typing import Protocol, cast
|
||||||
|
|
||||||
|
import genanki
|
||||||
|
|
||||||
|
from leetcode_coach import anki_connect, card_rules
|
||||||
|
|
||||||
|
DECK_ID = 1_695_042_117_001 # fixed: every session's file imports into the same deck
|
||||||
|
MODEL_ID = 1_695_042_117_002
|
||||||
|
DECK_NAME = "LeetCode"
|
||||||
|
OUT_DIR = Path.home() / "data" / "learn" / "leetcode" # ~/data: an allowed root bucket
|
||||||
|
FLAGS = ("--dry-run", "--replace")
|
||||||
|
|
||||||
|
MODEL_NAME = "LeetCode coach (basic)"
|
||||||
|
FIELDS = ["Front", "Back", "Problem"]
|
||||||
|
QFMT = "{{Front}}<br><br><span style='font-size:70%;color:#888'>{{Problem}}</span>"
|
||||||
|
AFMT = "{{FrontSide}}<hr id=answer>{{Back}}"
|
||||||
|
CSS = ".card { font-family: sans-serif; font-size: 22px; text-align: center; }"
|
||||||
|
|
||||||
|
MODEL = genanki.Model(
|
||||||
|
MODEL_ID,
|
||||||
|
MODEL_NAME,
|
||||||
|
fields=[{"name": f} for f in FIELDS],
|
||||||
|
templates=[{"name": "Card 1", "qfmt": QFMT, "afmt": AFMT}],
|
||||||
|
css=CSS,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class Note(Protocol):
|
||||||
|
"""The part of `genanki.Note` this module relies on (genanki is untyped)."""
|
||||||
|
|
||||||
|
@property
|
||||||
|
def guid(self) -> str:
|
||||||
|
"""Stable id: re-importing the same (slug, front) updates the note."""
|
||||||
|
|
||||||
|
@property
|
||||||
|
def fields(self) -> list[str]:
|
||||||
|
"""Front, Back, Problem -- HTML-escaped text."""
|
||||||
|
|
||||||
|
@property
|
||||||
|
def tags(self) -> list[str]:
|
||||||
|
"""Anki tags."""
|
||||||
|
|
||||||
|
|
||||||
|
def card_note(front: str, back: str, slug: str) -> Note:
|
||||||
|
"""A note whose guid ignores escaping, so a regenerated deck updates it."""
|
||||||
|
guid = genanki.guid_for(slug, html.unescape(front))
|
||||||
|
note = genanki.Note(
|
||||||
|
model=MODEL, fields=[front, back, slug], tags=["leetcode", slug], guid=guid
|
||||||
|
)
|
||||||
|
return cast("Note", note)
|
||||||
|
|
||||||
|
|
||||||
|
def _log(slug: str, cards: list[card_rules.Card], today: str) -> None:
|
||||||
|
"""Append the cards to the plain-text history."""
|
||||||
|
OUT_DIR.mkdir(parents=True, exist_ok=True)
|
||||||
|
with (OUT_DIR / "cards.tsv").open("a", encoding="utf-8") as fh:
|
||||||
|
for c in cards:
|
||||||
|
row = (today, slug, c["front"].strip(), c["back"].strip())
|
||||||
|
fh.write("\t".join(row) + "\n")
|
||||||
|
|
||||||
|
|
||||||
|
def push_direct(slug: str, cards: list[card_rules.Card], *, replace: bool) -> str:
|
||||||
|
"""AnkiConnect path: upsert into the open desktop collection, then sync."""
|
||||||
|
anki_connect.ensure_model(MODEL_NAME, FIELDS, QFMT, AFMT, CSS)
|
||||||
|
added, updated, deleted = anki_connect.sync_notes(
|
||||||
|
DECK_NAME, MODEL_NAME, slug, card_rules.escaped(cards), replace=replace
|
||||||
|
)
|
||||||
|
anki_connect.sync()
|
||||||
|
return (
|
||||||
|
f"{added} added, {updated} updated, {deleted} deleted in {DECK_NAME!r}; "
|
||||||
|
"AnkiWeb sync requested (lands on the phone at AnkiDroid's next sync)"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def build_apkg(slug: str, cards: list[card_rules.Card], today: str) -> Path:
|
||||||
|
"""Offline path: write a deck file that `anki <file>` imports."""
|
||||||
|
deck = genanki.Deck(DECK_ID, DECK_NAME)
|
||||||
|
for c in card_rules.escaped(cards):
|
||||||
|
deck.add_note(card_note(c["front"], c["back"], slug))
|
||||||
|
out_dir = OUT_DIR / "anki"
|
||||||
|
out_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
out = out_dir / f"{today}-{slug}.apkg"
|
||||||
|
genanki.Package(deck).write_to_file(str(out))
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def _usage() -> str:
|
||||||
|
"""The `Usage:` line of this module's docstring."""
|
||||||
|
return (__doc__ or "").strip().splitlines()[2]
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
"""Validate the cards on argv/stdin, then deliver them; 0 on success."""
|
||||||
|
args = sys.argv[1:]
|
||||||
|
flags = {a for a in args if a in FLAGS}
|
||||||
|
rest = [a for a in args if a not in FLAGS]
|
||||||
|
if len(rest) != 1 or len(flags) > 1:
|
||||||
|
print(_usage(), file=sys.stderr)
|
||||||
|
return 1
|
||||||
|
src = rest[0]
|
||||||
|
raw = sys.stdin.read() if src == "-" else Path(src).read_text(encoding="utf-8")
|
||||||
|
spec = json.loads(raw)
|
||||||
|
slug, cards = str(spec.get("slug", "")).strip(), list(spec.get("cards", []))
|
||||||
|
if errors := card_rules.validate(slug, cards):
|
||||||
|
print("cards rejected:\n " + "\n ".join(errors), file=sys.stderr)
|
||||||
|
return 1
|
||||||
|
if "--dry-run" in flags:
|
||||||
|
print(card_rules.table(cards))
|
||||||
|
return 0
|
||||||
|
replace = "--replace" in flags
|
||||||
|
online = anki_connect.available()
|
||||||
|
if replace and not online:
|
||||||
|
print(
|
||||||
|
"--replace needs Anki open (AnkiConnect); nothing changed", file=sys.stderr
|
||||||
|
)
|
||||||
|
return 1
|
||||||
|
today = dt.datetime.now(tz=dt.UTC).date().isoformat()
|
||||||
|
_log(slug, cards, today)
|
||||||
|
print(card_rules.table(cards))
|
||||||
|
if online:
|
||||||
|
done = push_direct(slug, cards, replace=replace)
|
||||||
|
print(f"{len(cards)} cards -> Anki desktop: {done}")
|
||||||
|
return 0
|
||||||
|
out = build_apkg(slug, cards, today)
|
||||||
|
print(f"{len(cards)} cards -> {out} (Anki desktop not open). Import with:")
|
||||||
|
print(f"anki {shlex.quote(str(out))}")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
104
leetcode_coach/leetcode_coach/stop_gate.py
Normal file
104
leetcode_coach/leetcode_coach/stop_gate.py
Normal file
@ -0,0 +1,104 @@
|
|||||||
|
"""Stop hook: in a /leetcode session, refuse a reply that hands kuhy a line of code.
|
||||||
|
|
||||||
|
The prose rule ("never write a line for their code") held in one haiku
|
||||||
|
role-play out of two, so it is adjudicated here instead. A reply is blocked
|
||||||
|
when any code span or fenced block in it uses an identifier from the
|
||||||
|
problem's starter signature (e.g. `fn`, `args`) together with a call/spread/
|
||||||
|
assignment -- unless that exact snippet already appears in one of kuhy's own
|
||||||
|
messages (quoting their code back is fine). `reveal` lifts the gate.
|
||||||
|
|
||||||
|
Blocking returns {"decision": "block"} so the model rewrites the reply; the
|
||||||
|
rewrite is judged again, at most MAX_BLOCKS times per prompt (counted in
|
||||||
|
~/.claude/state/leetcode/<session>.blocks), then the gate fails open.
|
||||||
|
|
||||||
|
Run as ``/usr/bin/python3 -P -m leetcode_coach.stop_gate``; the file in
|
||||||
|
``~/.claude/hooks`` is a shim that execs exactly that. The printed JSON line
|
||||||
|
IS the hook's interface (hence the T201 exemption in pyproject.toml).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
import sys
|
||||||
|
|
||||||
|
from leetcode_coach import gate_detector
|
||||||
|
|
||||||
|
SKILL_LINK = Path.home() / ".claude" / "skills" / "leetcode"
|
||||||
|
# A /leetcode slash command, or a model-invoked Skill(leetcode) load (which
|
||||||
|
# leetcode_url_userprompt.sh triggers when kuhy pastes a bare problem URL).
|
||||||
|
# The skill directory is a symlink to this package since 2026-10-23, and the
|
||||||
|
# harness may print either side of it as the base directory: both arm.
|
||||||
|
MARKERS = (
|
||||||
|
(
|
||||||
|
'"content":"<command-message>leetcode</command-message>'
|
||||||
|
"\\n<command-name>/leetcode</command-name>"
|
||||||
|
),
|
||||||
|
f'"text":"Base directory for this skill: {SKILL_LINK}',
|
||||||
|
f'"text":"Base directory for this skill: {Path(__file__).resolve().parent}',
|
||||||
|
)
|
||||||
|
STATE_DIR = Path.home() / ".claude" / "state" / "leetcode"
|
||||||
|
MAX_BLOCKS = 2 # rewrites per prompt before the gate lets the reply through
|
||||||
|
|
||||||
|
|
||||||
|
def verdict(raw: str, reply: str) -> str | None:
|
||||||
|
"""Reason to block `reply`, or None when it may stand."""
|
||||||
|
if not reply or not any(m in raw for m in MARKERS):
|
||||||
|
return None
|
||||||
|
entries = gate_detector.transcript_lines(raw)
|
||||||
|
users = [gate_detector.kuhy_text(e) for e in entries if e.get("type") == "user"]
|
||||||
|
if any(u.strip() == "reveal" for u in users):
|
||||||
|
return None
|
||||||
|
idents = gate_detector.starter_identifiers(entries)
|
||||||
|
if not idents:
|
||||||
|
return None
|
||||||
|
allowed = gate_detector.allowed_text(entries)
|
||||||
|
bad = gate_detector.offending_snippets(reply, idents, allowed)
|
||||||
|
if not bad:
|
||||||
|
return None
|
||||||
|
return (
|
||||||
|
"leetcode gate: your reply hands kuhy code they can paste \u2014 "
|
||||||
|
+ "; ".join(f"`{b}`" for b in bad)
|
||||||
|
+ ". Rewrite it without that snippet: describe the data or name the "
|
||||||
|
"construct, and keep the one question at the end."
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def blocks_so_far(session_id: str, prompt_id: str) -> int:
|
||||||
|
"""How many times this prompt's reply was already sent back.
|
||||||
|
|
||||||
|
Bounded so a model that cannot satisfy the gate still gets to answer
|
||||||
|
(fail-open at the cap).
|
||||||
|
"""
|
||||||
|
state = STATE_DIR / f"{session_id}.blocks"
|
||||||
|
entries = state.read_text(encoding="utf-8").split() if state.is_file() else []
|
||||||
|
n = entries.count(prompt_id)
|
||||||
|
STATE_DIR.mkdir(parents=True, exist_ok=True)
|
||||||
|
with state.open("a", encoding="utf-8") as fh:
|
||||||
|
fh.write(prompt_id + "\n")
|
||||||
|
return n
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
"""Judge the reply in the Stop event on stdin; print a block decision or nothing."""
|
||||||
|
data = json.load(sys.stdin)
|
||||||
|
path = Path(data.get("transcript_path", ""))
|
||||||
|
if not path.is_file():
|
||||||
|
return 0
|
||||||
|
# The transcript on disk still lacks the reply being judged; the harness
|
||||||
|
# hands it over separately (measured: `last_assistant_message`, 2.1.276).
|
||||||
|
reply = str(data.get("last_assistant_message") or "")
|
||||||
|
reason = verdict(path.read_text(encoding="utf-8", errors="replace"), reply)
|
||||||
|
if not reason:
|
||||||
|
return 0
|
||||||
|
if (
|
||||||
|
blocks_so_far(str(data.get("session_id")), str(data.get("prompt_id")))
|
||||||
|
>= MAX_BLOCKS
|
||||||
|
):
|
||||||
|
return 0
|
||||||
|
print(json.dumps({"decision": "block", "reason": reason}))
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
1
leetcode_coach/leetcode_coach/tests/__init__.py
Normal file
1
leetcode_coach/leetcode_coach/tests/__init__.py
Normal file
@ -0,0 +1 @@
|
|||||||
|
"""Tests for the /leetcode coach (moved from ~/.claude/skills/leetcode)."""
|
||||||
44
leetcode_coach/leetcode_coach/tests/_shared.py
Normal file
44
leetcode_coach/leetcode_coach/tests/_shared.py
Normal file
@ -0,0 +1,44 @@
|
|||||||
|
"""Fixtures shared by the coach tests: a two-sum statement and canned replies."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import subprocess
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from leetcode_coach import local_backend
|
||||||
|
|
||||||
|
STATEMENT = """# 1. Two Sum (Easy)
|
||||||
|
Return indices of two numbers adding up to target.
|
||||||
|
|
||||||
|
## Starter code (javascript)
|
||||||
|
```
|
||||||
|
var twoSum = function(nums, target) {
|
||||||
|
};
|
||||||
|
```
|
||||||
|
"""
|
||||||
|
LEAK = "Here:\n```js\nfor (const n of nums) seen.set(target - n, n)\n```\nTry it?"
|
||||||
|
SAFE = "What could you remember about each number you have already seen?"
|
||||||
|
|
||||||
|
|
||||||
|
def ollama_reply(text: str, tokens: int = 100) -> dict[str, Any]:
|
||||||
|
"""An Ollama /api/chat answer body."""
|
||||||
|
return {"message": {"content": text}, "prompt_eval_count": tokens}
|
||||||
|
|
||||||
|
|
||||||
|
def completed(
|
||||||
|
code: int, out: str = "", err: str = ""
|
||||||
|
) -> subprocess.CompletedProcess[str]:
|
||||||
|
"""A finished subprocess with the given exit code and output."""
|
||||||
|
return subprocess.CompletedProcess([], code, out, err)
|
||||||
|
|
||||||
|
|
||||||
|
class FakeChat:
|
||||||
|
"""Returns scripted replies and records every prompt it was sent."""
|
||||||
|
|
||||||
|
def __init__(self, *texts: str) -> None:
|
||||||
|
self.texts = list(texts)
|
||||||
|
self.prompts: list[list[dict[str, str]]] = []
|
||||||
|
|
||||||
|
def __call__(self, messages: list[dict[str, str]]) -> local_backend.LocalReply:
|
||||||
|
self.prompts.append(list(messages))
|
||||||
|
return local_backend.LocalReply(self.texts.pop(0), 1.0, 10)
|
||||||
152
leetcode_coach/leetcode_coach/tests/test_anki_connect.py
Normal file
152
leetcode_coach/leetcode_coach/tests/test_anki_connect.py
Normal file
@ -0,0 +1,152 @@
|
|||||||
|
"""Tests for the AnkiConnect side of the /leetcode card builder."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import io
|
||||||
|
import json
|
||||||
|
from typing import Any, Self
|
||||||
|
import urllib.error
|
||||||
|
import urllib.request
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from leetcode_coach import anki_connect, card_rules
|
||||||
|
|
||||||
|
|
||||||
|
def _card(front: str, back: str = "x") -> dict[str, str]:
|
||||||
|
return {"front": front, "back": back}
|
||||||
|
|
||||||
|
|
||||||
|
class FakeAnki:
|
||||||
|
"""Minimal in-memory AnkiConnect: notes keyed by id."""
|
||||||
|
|
||||||
|
def __init__(self, notes: dict[int, tuple[str, str]]) -> None:
|
||||||
|
self.notes = {i: {"Front": f, "Back": b} for i, (f, b) in notes.items()}
|
||||||
|
self.deleted: list[int] = []
|
||||||
|
self.next_id = 100
|
||||||
|
|
||||||
|
def __call__(self, action: str, **p: Any) -> Any:
|
||||||
|
if action == "findNotes":
|
||||||
|
return list(self.notes)
|
||||||
|
if action == "notesInfo":
|
||||||
|
return [
|
||||||
|
{
|
||||||
|
"noteId": i,
|
||||||
|
"fields": {k: {"value": v} for k, v in self.notes[i].items()},
|
||||||
|
}
|
||||||
|
for i in p["notes"]
|
||||||
|
]
|
||||||
|
if action == "updateNoteFields":
|
||||||
|
self.notes[p["note"]["id"]].update(p["note"]["fields"])
|
||||||
|
elif action == "addNote":
|
||||||
|
self.next_id += 1
|
||||||
|
self.notes[self.next_id] = dict(p["note"]["fields"])
|
||||||
|
return self.next_id
|
||||||
|
elif action == "deleteNotes":
|
||||||
|
self.deleted += p["notes"]
|
||||||
|
for i in p["notes"]:
|
||||||
|
del self.notes[i]
|
||||||
|
elif action == "findCards":
|
||||||
|
return [1]
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def test_sync_notes_replace_keeps_matching_front_and_deletes_the_rest(
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
) -> None:
|
||||||
|
fake = FakeAnki({1: ("n & m?", "old"), 2: ("dropped?", "gone")})
|
||||||
|
monkeypatch.setattr(anki_connect, "_call", fake)
|
||||||
|
cards = card_rules.escaped([_card("n & m?", "new"), _card("added?", "a")])
|
||||||
|
result = anki_connect.sync_notes("D", "M", "s", cards, replace=True)
|
||||||
|
assert result == (1, 1, 1)
|
||||||
|
assert fake.deleted == [2]
|
||||||
|
assert fake.notes[1] == {"Front": "n & m?", "Back": "new"}
|
||||||
|
|
||||||
|
|
||||||
|
def test_sync_notes_without_replace_deletes_nothing(
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
) -> None:
|
||||||
|
fake = FakeAnki({2: ("dropped?", "gone")})
|
||||||
|
monkeypatch.setattr(anki_connect, "_call", fake)
|
||||||
|
assert anki_connect.sync_notes("D", "M", "s", [_card("a?")], replace=False) == (
|
||||||
|
1,
|
||||||
|
0,
|
||||||
|
0,
|
||||||
|
)
|
||||||
|
assert fake.deleted == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_sync_notes_on_an_empty_problem_skips_notes_info(
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
) -> None:
|
||||||
|
fake = FakeAnki({})
|
||||||
|
monkeypatch.setattr(anki_connect, "_call", fake)
|
||||||
|
assert anki_connect.sync_notes("D", "M", "s", [_card("a?")], replace=True) == (
|
||||||
|
1,
|
||||||
|
0,
|
||||||
|
0,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class FakeResponse(io.BytesIO):
|
||||||
|
def __enter__(self) -> Self:
|
||||||
|
return self
|
||||||
|
|
||||||
|
def __exit__(self, *exc: object) -> None:
|
||||||
|
self.close()
|
||||||
|
|
||||||
|
|
||||||
|
def _serve(
|
||||||
|
monkeypatch: pytest.MonkeyPatch, replies: dict[str, Any]
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
sent: list[dict[str, Any]] = []
|
||||||
|
|
||||||
|
def urlopen(req: Any, timeout: int) -> FakeResponse:
|
||||||
|
body = json.loads(req.data)
|
||||||
|
sent.append(body)
|
||||||
|
reply = replies[body["action"]]
|
||||||
|
if isinstance(reply, Exception):
|
||||||
|
raise reply
|
||||||
|
return FakeResponse(json.dumps(reply).encode())
|
||||||
|
|
||||||
|
monkeypatch.setattr(urllib.request, "urlopen", urlopen)
|
||||||
|
return sent
|
||||||
|
|
||||||
|
|
||||||
|
def test_call_available_ensure_model_and_sync(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
sent = _serve(
|
||||||
|
monkeypatch,
|
||||||
|
{
|
||||||
|
"version": {"result": 6, "error": None},
|
||||||
|
"modelNames": {"result": ["Basic"], "error": None},
|
||||||
|
"createModel": {"result": {}, "error": None},
|
||||||
|
"sync": {"result": None, "error": None},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
assert anki_connect.available() is True
|
||||||
|
anki_connect.ensure_model("M", ["F"], "q", "a", "css")
|
||||||
|
anki_connect.sync()
|
||||||
|
assert [b["action"] for b in sent] == [
|
||||||
|
"version",
|
||||||
|
"modelNames",
|
||||||
|
"createModel",
|
||||||
|
"sync",
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def test_ensure_model_skips_existing_and_errors_surface(
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
) -> None:
|
||||||
|
sent = _serve(
|
||||||
|
monkeypatch,
|
||||||
|
{
|
||||||
|
"modelNames": {"result": ["M"], "error": None},
|
||||||
|
"deleteNotes": {"result": None, "error": "boom"},
|
||||||
|
"version": urllib.error.URLError("refused"),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
anki_connect.ensure_model("M", ["F"], "q", "a", "css")
|
||||||
|
assert [b["action"] for b in sent] == ["modelNames"]
|
||||||
|
with pytest.raises(anki_connect.AnkiConnectError, match="deleteNotes: boom"):
|
||||||
|
anki_connect._call("deleteNotes", notes=[1])
|
||||||
|
assert anki_connect.available() is False
|
||||||
66
leetcode_coach/leetcode_coach/tests/test_card_rules.py
Normal file
66
leetcode_coach/leetcode_coach/tests/test_card_rules.py
Normal file
@ -0,0 +1,66 @@
|
|||||||
|
"""Tests for the card shape rules the /leetcode builder enforces."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from leetcode_coach import card_rules
|
||||||
|
|
||||||
|
|
||||||
|
def _card(front: str, back: str = "x") -> dict[str, str]:
|
||||||
|
return {"front": front, "back": back}
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("front", "back", "error"),
|
||||||
|
[
|
||||||
|
("JS: nest or separate loops?", "Nest.", "either/or"),
|
||||||
|
("JS: setTimeout — built in, or do you write them?", "built in", "either/or"),
|
||||||
|
("JS: does clearTimeout stop a running fn?", "No", "yes/no"),
|
||||||
|
("JS: code after setTimeout — has cb run yet?", "No", "yes/no"),
|
||||||
|
("TS: Why does it break? What's the fix?", "closure", "two questions"),
|
||||||
|
("TS: return a function?", "return function(p) { ... }", "placeholder"),
|
||||||
|
("TS: return a function?", "return f …", "placeholder"),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_shape_rejects_coin_flips_and_placeholders(
|
||||||
|
front: str, back: str, error: str
|
||||||
|
) -> None:
|
||||||
|
assert any(error in e for e in card_rules.validate("s", [_card(front, back)]))
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("front", "back"),
|
||||||
|
[
|
||||||
|
("JS: What does XOR (^) return for different bits vs same?", "1 if different"),
|
||||||
|
("JS: write a regex matching every 'cat' or 'dog' in a string.", "/cat|dog/g"),
|
||||||
|
("TS: a loop runs while(a || b) — which nodes get skipped?", "leaves"),
|
||||||
|
("JS: call fn with the array as separate arguments?", "fn(...args)"),
|
||||||
|
("TS: one cur pointer walks one path. To visit all, add what?", "a stack"),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_shape_accepts_real_questions(front: str, back: str) -> None:
|
||||||
|
assert card_rules.validate("s", [_card(front, back)]) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_counts_lengths_and_slug() -> None:
|
||||||
|
errors = card_rules.validate(
|
||||||
|
"", [_card("q?", "one two three four five six seven eight nine")]
|
||||||
|
)
|
||||||
|
assert any("back is 9 words" in e for e in errors)
|
||||||
|
assert "missing slug" in errors
|
||||||
|
assert card_rules.validate("s", []) == ["0 cards; want 1-6"]
|
||||||
|
assert card_rules.validate("s", [_card("", "")]) == ["card 1: empty front or back"]
|
||||||
|
long_front = " ".join(["w"] * 21) + "?"
|
||||||
|
assert any(
|
||||||
|
"front is 21 words" in e for e in card_rules.validate("s", [_card(long_front)])
|
||||||
|
)
|
||||||
|
assert any("newlines" in e for e in card_rules.validate("s", [_card("a\nb?")]))
|
||||||
|
|
||||||
|
|
||||||
|
def test_table_escapes_pipes_and_fields_escape_html() -> None:
|
||||||
|
cards = [_card("JS: a<b and x|y?", "n & m")]
|
||||||
|
assert "| JS: a<b and x\\|y? | n & m |" in card_rules.table(cards)
|
||||||
|
assert card_rules.escaped(cards) == [
|
||||||
|
{"front": "JS: a<b and x|y?", "back": "n & m"}
|
||||||
|
]
|
||||||
70
leetcode_coach/leetcode_coach/tests/test_eval_local.py
Normal file
70
leetcode_coach/leetcode_coach/tests/test_eval_local.py
Normal file
@ -0,0 +1,70 @@
|
|||||||
|
"""Tests for the scripted acceptance run of the local coach (no Ollama needed)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
import subprocess
|
||||||
|
from typing import TYPE_CHECKING, Any
|
||||||
|
|
||||||
|
from leetcode_coach import eval_local, local_backend, local_coach
|
||||||
|
from leetcode_coach.tests._shared import LEAK, SAFE, STATEMENT, FakeChat, completed
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
|
||||||
|
def _wire(
|
||||||
|
monkeypatch: pytest.MonkeyPatch, tmp_path: Path, *texts: str
|
||||||
|
) -> list[list[str]]:
|
||||||
|
fetched: list[list[str]] = []
|
||||||
|
|
||||||
|
def run(cmd: list[str], **_k: Any) -> subprocess.CompletedProcess[str]:
|
||||||
|
fetched.append(cmd)
|
||||||
|
return completed(0, STATEMENT)
|
||||||
|
|
||||||
|
real_start = local_coach.LocalCoach.start
|
||||||
|
|
||||||
|
def start(text: str, history: Any = None) -> local_coach.LocalCoach:
|
||||||
|
coach = real_start(text, history)
|
||||||
|
coach.chat = FakeChat(*texts)
|
||||||
|
return coach
|
||||||
|
|
||||||
|
monkeypatch.setattr(subprocess, "run", run)
|
||||||
|
monkeypatch.setattr(local_backend, "chat", lambda _m: None)
|
||||||
|
monkeypatch.setattr(local_coach.LocalCoach, "start", start)
|
||||||
|
monkeypatch.setattr(eval_local, "OUT_DIR", tmp_path / "out")
|
||||||
|
return fetched
|
||||||
|
|
||||||
|
|
||||||
|
def test_six_clean_turns_pass(
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
capsys: pytest.CaptureFixture[str],
|
||||||
|
tmp_path: Path,
|
||||||
|
) -> None:
|
||||||
|
fetched = _wire(monkeypatch, tmp_path, *[SAFE] * len(eval_local.TURNS))
|
||||||
|
assert eval_local.main() == 0
|
||||||
|
assert fetched[0][1].endswith("leetcode_coach/fetch_problem.py")
|
||||||
|
assert fetched[0][2] == "two-sum"
|
||||||
|
lines = capsys.readouterr().out.splitlines()
|
||||||
|
assert lines[0].startswith("model load + warm-up: ")
|
||||||
|
assert lines[1] == "turn secs tries withheld no-code one-task fast"
|
||||||
|
assert lines[2] == " 1 1.0 1 False True True True"
|
||||||
|
assert lines[-1].startswith("PASS transcript: ")
|
||||||
|
transcript = Path(lines[-1].split("transcript: ", 1)[1])
|
||||||
|
assert transcript.parent == tmp_path / "out"
|
||||||
|
assert "**kuhy:** just give me the code" in transcript.read_text(encoding="utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_turn_without_a_task_fails(
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
capsys: pytest.CaptureFixture[str],
|
||||||
|
tmp_path: Path,
|
||||||
|
) -> None:
|
||||||
|
taskless = "Nice work."
|
||||||
|
texts = [taskless] * 3 + [LEAK] * 3 + [SAFE] * (len(eval_local.TURNS) - 2)
|
||||||
|
_wire(monkeypatch, tmp_path, *texts)
|
||||||
|
assert eval_local.main() == 1
|
||||||
|
lines = capsys.readouterr().out.splitlines()
|
||||||
|
assert lines[2] == " 1 3.0 3 False True False True"
|
||||||
|
assert lines[3] == " 2 3.0 3 True True True True"
|
||||||
|
assert lines[-1].startswith("FAIL transcript: ")
|
||||||
155
leetcode_coach/leetcode_coach/tests/test_fetch_problem.py
Normal file
155
leetcode_coach/leetcode_coach/tests/test_fetch_problem.py
Normal file
@ -0,0 +1,155 @@
|
|||||||
|
"""Tests for the LeetCode statement fetcher (public GraphQL, no login)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import io
|
||||||
|
import json
|
||||||
|
import runpy
|
||||||
|
from typing import Any, Self
|
||||||
|
import urllib.error
|
||||||
|
import urllib.request
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from leetcode_coach import fetch_problem
|
||||||
|
|
||||||
|
QUESTION = {
|
||||||
|
"questionFrontendId": "1",
|
||||||
|
"title": "Two Sum",
|
||||||
|
"difficulty": "Easy",
|
||||||
|
"content": (
|
||||||
|
"<p>Given <code>nums</code>, return <strong>indices</strong> "
|
||||||
|
"<em>exactly</em> once.</p><ul><li>one</li></ul><br>"
|
||||||
|
"<div><span>x</span></div><pre>Input: [2,7]\n Output: [0,1]</pre><b>b</b><i>i</i>"
|
||||||
|
),
|
||||||
|
"isPaidOnly": False,
|
||||||
|
"codeSnippets": [
|
||||||
|
{"langSlug": "python3", "code": "class Solution:\n pass\n"},
|
||||||
|
{"langSlug": "javascript", "code": "var twoSum = function(nums) {};"},
|
||||||
|
],
|
||||||
|
"topicTags": [{"name": "Array"}, {"name": "Hash Table"}],
|
||||||
|
"hints": ["Use a map."],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("arg", "slug"),
|
||||||
|
[
|
||||||
|
("https://leetcode.com/problems/two-sum/description/?x=1", "two-sum"),
|
||||||
|
("leetcode.cn/problems/add-two-numbers/submissions/1/", "add-two-numbers"),
|
||||||
|
("two-sum", "two-sum"),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_slug_from_urls_and_bare_slugs(arg: str, slug: str) -> None:
|
||||||
|
assert fetch_problem.slug_from(arg) == slug
|
||||||
|
|
||||||
|
|
||||||
|
def test_slug_from_rejects_anything_else() -> None:
|
||||||
|
with pytest.raises(ValueError, match="not a LeetCode problem URL or slug"):
|
||||||
|
fetch_problem.slug_from("Two Sum!")
|
||||||
|
|
||||||
|
|
||||||
|
def test_render_markdown_with_starter_and_hidden_hints() -> None:
|
||||||
|
out = fetch_problem.render(QUESTION, "javascript")
|
||||||
|
assert out.startswith("# 1. Two Sum (Easy)\nTags: Array, Hash Table\n")
|
||||||
|
assert "Languages available: python3, javascript" in out
|
||||||
|
assert "Given `nums`, return **indices** _exactly_ once." in out
|
||||||
|
assert "\n- one" in out
|
||||||
|
assert "```\nInput: [2,7]\n Output: [0,1]\n```" in out
|
||||||
|
assert "**b**_i_" in out
|
||||||
|
assert "## Starter code (javascript)\n```\nvar twoSum" in out
|
||||||
|
assert out.endswith("## Site hints (do NOT show to the user unasked)\n- Use a map.")
|
||||||
|
|
||||||
|
|
||||||
|
def test_render_falls_back_to_the_first_language_and_no_extras() -> None:
|
||||||
|
bare = {**QUESTION, "content": None, "topicTags": [], "hints": []}
|
||||||
|
out = fetch_problem.render(bare, "rust")
|
||||||
|
assert "Tags: none" in out
|
||||||
|
assert "## Starter code (python3)" in out
|
||||||
|
assert "Site hints" not in out
|
||||||
|
empty = {**bare, "codeSnippets": []}
|
||||||
|
out = fetch_problem.render(empty, "rust")
|
||||||
|
assert "Languages available: none" in out
|
||||||
|
assert "Starter code" not in out
|
||||||
|
|
||||||
|
|
||||||
|
class FakeResponse(io.BytesIO):
|
||||||
|
def __enter__(self) -> Self:
|
||||||
|
return self
|
||||||
|
|
||||||
|
def __exit__(self, *exc: object) -> None:
|
||||||
|
self.close()
|
||||||
|
|
||||||
|
|
||||||
|
def test_fetch_posts_the_query(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
seen: list[Any] = []
|
||||||
|
|
||||||
|
def urlopen(req: Any, timeout: int) -> FakeResponse:
|
||||||
|
seen.append((req.full_url, json.loads(req.data), req.get_header("Referer")))
|
||||||
|
return FakeResponse(json.dumps({"data": {"question": QUESTION}}).encode())
|
||||||
|
|
||||||
|
monkeypatch.setattr(urllib.request, "urlopen", urlopen)
|
||||||
|
assert fetch_problem.fetch("two-sum") == QUESTION
|
||||||
|
url, body, referer = seen[0]
|
||||||
|
assert url == "https://leetcode.com/graphql"
|
||||||
|
assert body["variables"] == {"s": "two-sum"}
|
||||||
|
assert referer == "https://leetcode.com/problems/two-sum/"
|
||||||
|
|
||||||
|
|
||||||
|
def _main(monkeypatch: pytest.MonkeyPatch, argv: list[str], fetched: Any) -> int:
|
||||||
|
def fetch(_slug: str) -> Any:
|
||||||
|
if isinstance(fetched, Exception):
|
||||||
|
raise fetched
|
||||||
|
return fetched
|
||||||
|
|
||||||
|
monkeypatch.setattr("sys.argv", ["fetch_problem.py", *argv])
|
||||||
|
monkeypatch.setattr(fetch_problem, "fetch", fetch)
|
||||||
|
return fetch_problem.main()
|
||||||
|
|
||||||
|
|
||||||
|
def test_main_prints_the_problem(
|
||||||
|
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
||||||
|
) -> None:
|
||||||
|
assert _main(monkeypatch, ["two-sum", "--lang", "python3"], QUESTION) == 0
|
||||||
|
out = capsys.readouterr().out
|
||||||
|
assert out.startswith("# 1. Two Sum (Easy)\n")
|
||||||
|
assert "## Starter code (python3)" in out
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("fetched", "code", "err"),
|
||||||
|
[
|
||||||
|
(urllib.error.URLError("offline"), 1, "fetch failed: <urlopen error offline>"),
|
||||||
|
(None, 2, "problem 'two-sum' is unknown or premium-only"),
|
||||||
|
({**QUESTION, "isPaidOnly": True}, 2, "premium-only"),
|
||||||
|
({**QUESTION, "content": ""}, 2, "premium-only"),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_main_failures_exit_with_a_reason(
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
capsys: pytest.CaptureFixture[str],
|
||||||
|
fetched: Any,
|
||||||
|
code: int,
|
||||||
|
err: str,
|
||||||
|
) -> None:
|
||||||
|
assert _main(monkeypatch, ["two-sum"], fetched) == code
|
||||||
|
captured = capsys.readouterr()
|
||||||
|
assert captured.out == ""
|
||||||
|
assert err in captured.err
|
||||||
|
|
||||||
|
|
||||||
|
def test_main_rejects_a_bad_argument(
|
||||||
|
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
||||||
|
) -> None:
|
||||||
|
assert _main(monkeypatch, ["Not A Slug"], QUESTION) == 1
|
||||||
|
assert capsys.readouterr().err.startswith("fetch failed: not a LeetCode problem")
|
||||||
|
|
||||||
|
|
||||||
|
def test_script_entry_point(
|
||||||
|
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
||||||
|
) -> None:
|
||||||
|
monkeypatch.setattr("sys.argv", ["fetch_problem.py", "Bad Slug"])
|
||||||
|
with pytest.raises(SystemExit) as exit_info:
|
||||||
|
runpy.run_path(fetch_problem.__file__, run_name="__main__")
|
||||||
|
assert exit_info.value.code == 1
|
||||||
|
assert "fetch failed" in capsys.readouterr().err
|
||||||
97
leetcode_coach/leetcode_coach/tests/test_gate_detector.py
Normal file
97
leetcode_coach/leetcode_coach/tests/test_gate_detector.py
Normal file
@ -0,0 +1,97 @@
|
|||||||
|
"""Tests for the transcript parsing and snippet detection behind the Stop gate."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from leetcode_coach import gate_detector as det
|
||||||
|
|
||||||
|
|
||||||
|
def _user(content: object) -> dict[str, Any]:
|
||||||
|
return {"type": "user", "message": {"content": content}}
|
||||||
|
|
||||||
|
|
||||||
|
def test_transcript_lines_skips_garbage_and_non_objects() -> None:
|
||||||
|
raw = '{"a": 1}\nnot json\n[1, 2]\n{"b": 2}\n'
|
||||||
|
assert det.transcript_lines(raw) == [{"a": 1}, {"b": 2}]
|
||||||
|
|
||||||
|
|
||||||
|
def test_text_of_joins_text_and_tool_results() -> None:
|
||||||
|
msg = _user(
|
||||||
|
[
|
||||||
|
{"type": "text", "text": "hi"},
|
||||||
|
{"type": "tool_result", "content": "plain"},
|
||||||
|
{"type": "tool_result", "content": [{"text": "a"}, {"text": "b"}]},
|
||||||
|
{"type": "image"},
|
||||||
|
]
|
||||||
|
)
|
||||||
|
assert det.text_of(msg) == "hi\nplain\na b"
|
||||||
|
assert det.text_of(_user("s")) == "s"
|
||||||
|
assert det.text_of({}) == ""
|
||||||
|
|
||||||
|
|
||||||
|
def test_kuhy_text_drops_skill_body_hook_echo_and_tool_results() -> None:
|
||||||
|
msg = _user(
|
||||||
|
[
|
||||||
|
{"type": "text", "text": "mine"},
|
||||||
|
{"type": "text", "text": "Base directory for this skill: /x"},
|
||||||
|
{"type": "text", "text": "Stop hook feedback: `fn(...args)`"},
|
||||||
|
{"type": "tool_result", "content": "fetched"},
|
||||||
|
]
|
||||||
|
)
|
||||||
|
assert det.kuhy_text(msg) == "mine"
|
||||||
|
assert det.kuhy_text(_user("typed")) == "typed"
|
||||||
|
|
||||||
|
|
||||||
|
def test_starter_identifiers_from_js_and_python_starters() -> None:
|
||||||
|
js = _user(
|
||||||
|
"## Starter code (javascript)\n```\n/**\n * @param {number[]} nums\n"
|
||||||
|
" * log(...argsArr)\n */\nvar twoSum = function(nums, target) {\n};\n```"
|
||||||
|
)
|
||||||
|
assert det.starter_identifiers([js]) == {"twoSum", "nums", "target"}
|
||||||
|
py = _user(
|
||||||
|
"## Starter code (python3)\n```\nclass Solution:\n"
|
||||||
|
" def maxArea(self, height: List[int], *rest, k=3) -> int:\n```"
|
||||||
|
)
|
||||||
|
assert det.starter_identifiers([py]) == {"maxArea", "height", "rest"}
|
||||||
|
unfenced = _user("## Starter code\nfunction sortPeople(names, heights) {")
|
||||||
|
assert det.starter_identifiers([unfenced]) == {"sortPeople", "names", "heights"}
|
||||||
|
no_params = _user("## Starter code\n```\nfunction makeCounter() {}\n```")
|
||||||
|
assert det.starter_identifiers([no_params]) == {"makeCounter"}
|
||||||
|
|
||||||
|
|
||||||
|
def test_starter_identifiers_ignores_other_entries() -> None:
|
||||||
|
assistant = {"type": "assistant", "message": {"content": "## Starter code\nf(ab)"}}
|
||||||
|
assert det.starter_identifiers([assistant, _user("no starter")]) == set()
|
||||||
|
assert det.starter_identifiers([_user("## Starter code\n```\nfn = (a)\n```")]) == {
|
||||||
|
"fn"
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_allowed_text_covers_kuhy_and_fetched_problem() -> None:
|
||||||
|
entries = [
|
||||||
|
_user([{"type": "text", "text": "my code\n x"}]),
|
||||||
|
_user([{"type": "tool_result", "content": "statement"}]),
|
||||||
|
{"type": "assistant", "message": {"content": "never"}},
|
||||||
|
_user("plain"),
|
||||||
|
]
|
||||||
|
assert det.allowed_text(entries) == "my code x statement plain"
|
||||||
|
|
||||||
|
|
||||||
|
def test_offending_snippets_rules() -> None:
|
||||||
|
idents = {"nums", "args"}
|
||||||
|
reply = (
|
||||||
|
"`nums.push(1)` and `nums` and `x = 1` and\n"
|
||||||
|
"```js\nconst s = new Set(nums)\n```\n"
|
||||||
|
"```javascript```\n"
|
||||||
|
"`fn(...args)` vs `fn(...argsArr)` and `" + "nums[0] = " + "1" * 90 + "`"
|
||||||
|
)
|
||||||
|
bad = det.offending_snippets(reply, idents, "fn(...argsArr)")
|
||||||
|
assert bad == [
|
||||||
|
"nums.push(1)",
|
||||||
|
"const s = new Set(nums)",
|
||||||
|
"fn(...args)",
|
||||||
|
("nums[0] = " + "1" * 90)[:80],
|
||||||
|
]
|
||||||
|
assert det.offending_snippets("`nums.push(1)`", idents, "x nums.push(1) y") == []
|
||||||
|
assert det.offending_snippets("`f(...)`", {"f"}, "") == ["f(...)"]
|
||||||
174
leetcode_coach/leetcode_coach/tests/test_leetcode_repl.py
Normal file
174
leetcode_coach/leetcode_coach/tests/test_leetcode_repl.py
Normal file
@ -0,0 +1,174 @@
|
|||||||
|
"""Tests for the print-mode /leetcode REPL and its hand-over to the local model."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
import subprocess
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from leetcode_coach import leetcode_repl, local_session
|
||||||
|
|
||||||
|
LAUNCHER = Path(leetcode_repl.__file__).with_name("leetcode")
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("reply", "hit"),
|
||||||
|
[
|
||||||
|
(
|
||||||
|
{"result": "You've hit your session limit · resets 3pm (Europe/Warsaw)"},
|
||||||
|
True,
|
||||||
|
),
|
||||||
|
({"result": "You\u2019ve hit your weekly limit · resets Mon"}, True),
|
||||||
|
({"result": "", "api_error_status": 429}, True),
|
||||||
|
({"result": "x", "_stderr": "Claude AI usage limit reached|1790427600"}, True),
|
||||||
|
({"result": "Does your loop hit every element?"}, False),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_is_limit_hit(reply: dict[str, Any], *, hit: bool) -> None:
|
||||||
|
assert leetcode_repl.is_limit_hit(reply) is hit
|
||||||
|
|
||||||
|
|
||||||
|
def _fake_run(handed: dict[str, Any]) -> Any:
|
||||||
|
def fake_run(
|
||||||
|
problem: str, history: list[dict[str, str]], first: str, _r: object
|
||||||
|
) -> int:
|
||||||
|
handed.update(problem=problem, history=history, first=first)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
return fake_run
|
||||||
|
|
||||||
|
|
||||||
|
def test_limit_mid_session_falls_back_with_history(
|
||||||
|
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
||||||
|
) -> None:
|
||||||
|
replies = iter(
|
||||||
|
[
|
||||||
|
{"result": "Restated. What feels unclear?", "session_id": "s1"},
|
||||||
|
{"result": "You've hit your session limit · resets 8pm (Europe/Warsaw)"},
|
||||||
|
]
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(leetcode_repl, "ask", lambda *_a: next(replies))
|
||||||
|
monkeypatch.setattr(leetcode_repl, "read_block", lambda: "the hash map part")
|
||||||
|
handed: dict[str, Any] = {}
|
||||||
|
monkeypatch.setattr(local_session, "run", _fake_run(handed))
|
||||||
|
assert leetcode_repl.main(["two-sum", "maps"]) == 0
|
||||||
|
assert handed["problem"] == "two-sum"
|
||||||
|
assert handed["first"] == "the hash map part"
|
||||||
|
assert handed["history"][1]["content"] == "Restated. What feels unclear?"
|
||||||
|
assert capsys.readouterr().out == (
|
||||||
|
"Restated. What feels unclear?\n\n...\n"
|
||||||
|
"[You've hit your session limit · resets 8pm (Europe/Warsaw) "
|
||||||
|
"-- continuing locally]\n"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_limit_on_the_first_turn_starts_locally(
|
||||||
|
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
||||||
|
) -> None:
|
||||||
|
monkeypatch.setattr(leetcode_repl, "ask", lambda *_a: {"api_error_status": 429})
|
||||||
|
handed: dict[str, Any] = {}
|
||||||
|
monkeypatch.setattr(local_session, "run", _fake_run(handed))
|
||||||
|
assert leetcode_repl.run_claude("sonnet", ["two-sum"]) == 0
|
||||||
|
assert handed["history"] == []
|
||||||
|
assert "not said" in handed["first"]
|
||||||
|
assert capsys.readouterr().out == "[usage limit -- switching to local model]\n"
|
||||||
|
|
||||||
|
|
||||||
|
def test_session_loop_until_quit_and_missing_session(
|
||||||
|
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
||||||
|
) -> None:
|
||||||
|
sent: list[tuple[list[str], str]] = []
|
||||||
|
replies = iter([{"result": "Open?", "session_id": "s1"}, {"result": "Next?"}])
|
||||||
|
|
||||||
|
def ask(args: list[str], prompt: str) -> dict[str, Any]:
|
||||||
|
sent.append((args, prompt))
|
||||||
|
return next(replies)
|
||||||
|
|
||||||
|
blocks = iter(["my try", "quit"])
|
||||||
|
monkeypatch.setattr(leetcode_repl, "ask", ask)
|
||||||
|
monkeypatch.setattr(leetcode_repl, "read_block", lambda: next(blocks))
|
||||||
|
assert leetcode_repl.run_claude("sonnet", ["two-sum", "maps"]) == 0
|
||||||
|
assert sent == [
|
||||||
|
(["--model", "sonnet"], "/leetcode two-sum maps"),
|
||||||
|
(["--model", "sonnet", "--resume", "s1"], "my try"),
|
||||||
|
]
|
||||||
|
assert capsys.readouterr().out == "Open?\n\n...\nNext?\n"
|
||||||
|
monkeypatch.setattr(leetcode_repl, "ask", lambda *_a: {"result": "no id"})
|
||||||
|
assert leetcode_repl.run_claude("sonnet", ["two-sum"]) == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_read_block_joins_lines_until_a_blank(
|
||||||
|
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
||||||
|
) -> None:
|
||||||
|
lines = iter(["", "a", "b", ""])
|
||||||
|
monkeypatch.setattr("builtins.input", lambda: next(lines))
|
||||||
|
assert leetcode_repl.read_block() == "a\nb"
|
||||||
|
assert capsys.readouterr().out == "\nyou> "
|
||||||
|
|
||||||
|
def eof() -> str:
|
||||||
|
raise EOFError
|
||||||
|
|
||||||
|
monkeypatch.setattr("builtins.input", eof)
|
||||||
|
assert leetcode_repl.read_block() is None
|
||||||
|
typed = iter(["x"])
|
||||||
|
|
||||||
|
def then_eof() -> str:
|
||||||
|
try:
|
||||||
|
return next(typed)
|
||||||
|
except StopIteration:
|
||||||
|
raise EOFError from None
|
||||||
|
|
||||||
|
monkeypatch.setattr("builtins.input", then_eof)
|
||||||
|
assert leetcode_repl.read_block() == "x"
|
||||||
|
|
||||||
|
|
||||||
|
def test_ask_parses_a_failing_claude_stub(
|
||||||
|
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
||||||
|
) -> None:
|
||||||
|
stub = tmp_path / "claude"
|
||||||
|
body = json.dumps({"is_error": True, "result": "You've hit your session limit"})
|
||||||
|
(tmp_path / "out.json").write_text(body)
|
||||||
|
stub.write_text(f"#!/bin/sh\ncat {tmp_path / 'out.json'}\necho boom >&2\nexit 1\n")
|
||||||
|
stub.chmod(0o755)
|
||||||
|
monkeypatch.setattr(leetcode_repl, "CLAUDE", str(stub))
|
||||||
|
reply = leetcode_repl.ask([], "hi")
|
||||||
|
assert leetcode_repl.is_limit_hit(reply)
|
||||||
|
assert reply["_stderr"] == "boom"
|
||||||
|
stub.write_text("#!/bin/sh\necho not-json\nexit 3\n")
|
||||||
|
assert leetcode_repl.ask([], "hi")["result"].startswith("[claude exited 3]")
|
||||||
|
stub.write_text("#!/bin/sh\necho '[1]'\n")
|
||||||
|
assert leetcode_repl.ask([], "hi") == {"_stderr": ""}
|
||||||
|
|
||||||
|
|
||||||
|
def test_repl_defaults_to_sonnet(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
seen: list[str] = []
|
||||||
|
monkeypatch.delenv("LEETCODE_MODEL", raising=False)
|
||||||
|
monkeypatch.setattr(leetcode_repl, "run_claude", lambda m, _a: seen.append(m))
|
||||||
|
leetcode_repl.main(["two-sum"])
|
||||||
|
assert seen == ["sonnet"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_local_flag_skips_claude(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
handed: dict[str, Any] = {}
|
||||||
|
monkeypatch.setattr(local_session, "run", _fake_run(handed))
|
||||||
|
assert leetcode_repl.main(["--local", "two-sum", "loops"]) == 0
|
||||||
|
assert handed["problem"] == "two-sum"
|
||||||
|
assert handed["first"].endswith("stuck on: loops")
|
||||||
|
|
||||||
|
|
||||||
|
def test_help_prints_usage(capsys: pytest.CaptureFixture[str]) -> None:
|
||||||
|
assert leetcode_repl.main([]) == 0
|
||||||
|
assert capsys.readouterr().out.startswith(
|
||||||
|
"Usage: leetcode [--local] <url-or-slug> [what you are stuck on...]\n"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_launcher_runs_the_package_repl() -> None:
|
||||||
|
proc = subprocess.run(
|
||||||
|
[str(LAUNCHER), "--help"], capture_output=True, text=True, check=False
|
||||||
|
)
|
||||||
|
assert proc.returncode == 0, proc.stderr
|
||||||
|
assert proc.stdout.startswith("Usage: leetcode [--local]")
|
||||||
90
leetcode_coach/leetcode_coach/tests/test_link_script.py
Normal file
90
leetcode_coach/leetcode_coach/tests/test_link_script.py
Normal file
@ -0,0 +1,90 @@
|
|||||||
|
"""Tests for scripts/link_leetcode_coach.sh, the install step for the coach.
|
||||||
|
|
||||||
|
Run against a throwaway HOME: the script must never be pointed at the real
|
||||||
|
~/.claude from a test.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
import subprocess
|
||||||
|
|
||||||
|
REPO = Path(__file__).resolve().parents[2]
|
||||||
|
SCRIPT = REPO / "scripts" / "link_leetcode_coach.sh"
|
||||||
|
COACH = REPO / "leetcode_coach"
|
||||||
|
|
||||||
|
|
||||||
|
def _run(home: Path) -> subprocess.CompletedProcess[str]:
|
||||||
|
return subprocess.run(
|
||||||
|
[str(SCRIPT)],
|
||||||
|
env={"HOME": str(home), "PATH": "/usr/bin:/bin"},
|
||||||
|
capture_output=True,
|
||||||
|
text=True,
|
||||||
|
check=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _old_skill(home: Path) -> Path:
|
||||||
|
old = home / ".claude" / "skills" / "leetcode"
|
||||||
|
old.mkdir(parents=True)
|
||||||
|
(old / "SKILL.md").write_text("---\nname: leetcode\n---\n", encoding="utf-8")
|
||||||
|
(old / "leetcode_repl.py").write_text("", encoding="utf-8")
|
||||||
|
return old
|
||||||
|
|
||||||
|
|
||||||
|
def test_fresh_home_gets_both_links(tmp_path: Path) -> None:
|
||||||
|
proc = _run(tmp_path)
|
||||||
|
assert proc.returncode == 0, proc.stderr
|
||||||
|
skill = tmp_path / ".claude" / "skills" / "leetcode"
|
||||||
|
assert skill.is_symlink()
|
||||||
|
assert skill.resolve() == COACH
|
||||||
|
assert (tmp_path / ".local" / "bin" / "leetcode").resolve() == COACH / "leetcode"
|
||||||
|
assert _run(tmp_path).returncode == 0 # idempotent
|
||||||
|
|
||||||
|
|
||||||
|
def test_old_skill_dir_is_backed_up_outside_skills(tmp_path: Path) -> None:
|
||||||
|
_old_skill(tmp_path)
|
||||||
|
bin_link = tmp_path / ".local" / "bin" / "leetcode"
|
||||||
|
bin_link.parent.mkdir(parents=True)
|
||||||
|
bin_link.symlink_to(tmp_path / ".claude" / "skills" / "leetcode" / "leetcode")
|
||||||
|
proc = _run(tmp_path)
|
||||||
|
assert proc.returncode == 0, proc.stderr
|
||||||
|
backups = list((tmp_path / ".claude" / "backups").glob("skill-leetcode-*"))
|
||||||
|
assert len(backups) == 1
|
||||||
|
assert (backups[0] / "leetcode_repl.py").is_file()
|
||||||
|
assert sorted(p.name for p in (tmp_path / ".claude" / "skills").iterdir()) == [
|
||||||
|
"leetcode"
|
||||||
|
]
|
||||||
|
assert (tmp_path / ".claude" / "skills" / "leetcode").resolve() == COACH
|
||||||
|
assert bin_link.resolve() == COACH / "leetcode"
|
||||||
|
|
||||||
|
|
||||||
|
def test_existing_symlink_is_repointed(tmp_path: Path) -> None:
|
||||||
|
skills = tmp_path / ".claude" / "skills"
|
||||||
|
skills.mkdir(parents=True)
|
||||||
|
elsewhere = tmp_path / "elsewhere"
|
||||||
|
elsewhere.mkdir()
|
||||||
|
(skills / "leetcode").symlink_to(elsewhere)
|
||||||
|
assert _run(tmp_path).returncode == 0
|
||||||
|
assert (skills / "leetcode").resolve() == COACH
|
||||||
|
assert list(elsewhere.iterdir()) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_unknown_directory_is_refused(tmp_path: Path) -> None:
|
||||||
|
other = tmp_path / ".claude" / "skills" / "leetcode"
|
||||||
|
other.mkdir(parents=True)
|
||||||
|
(other / "notes.txt").write_text("mine", encoding="utf-8")
|
||||||
|
proc = _run(tmp_path)
|
||||||
|
assert proc.returncode == 1
|
||||||
|
assert "not replacing it" in proc.stderr
|
||||||
|
assert (other / "notes.txt").is_file()
|
||||||
|
assert not other.is_symlink()
|
||||||
|
|
||||||
|
|
||||||
|
def test_regular_file_at_the_command_is_refused(tmp_path: Path) -> None:
|
||||||
|
bin_file = tmp_path / ".local" / "bin" / "leetcode"
|
||||||
|
bin_file.parent.mkdir(parents=True)
|
||||||
|
bin_file.write_text("#!/bin/sh\n", encoding="utf-8")
|
||||||
|
proc = _run(tmp_path)
|
||||||
|
assert proc.returncode == 1
|
||||||
|
assert not bin_file.is_symlink()
|
||||||
97
leetcode_coach/leetcode_coach/tests/test_local_backend.py
Normal file
97
leetcode_coach/leetcode_coach/tests/test_local_backend.py
Normal file
@ -0,0 +1,97 @@
|
|||||||
|
"""Tests for the Ollama transport of the offline /leetcode coach."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import io
|
||||||
|
import json
|
||||||
|
from typing import Any, Self
|
||||||
|
import urllib.request
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from leetcode_coach import local_backend
|
||||||
|
from leetcode_coach.tests._shared import completed, ollama_reply
|
||||||
|
|
||||||
|
|
||||||
|
def test_strip_think_removes_closed_and_dangling_blocks() -> None:
|
||||||
|
assert local_backend.strip_think("<think>x</think>\nHi") == "Hi"
|
||||||
|
assert local_backend.strip_think("Hi <think>never closed") == "Hi"
|
||||||
|
assert local_backend.strip_think("reasoning...\n</think>\n\nAnswer") == "Answer"
|
||||||
|
|
||||||
|
|
||||||
|
def test_chat_sends_explicit_ctx_no_think_and_keep_alive() -> None:
|
||||||
|
sent: list[dict[str, Any]] = []
|
||||||
|
|
||||||
|
def post(url: str, payload: dict[str, Any] | None, _t: float) -> dict[str, Any]:
|
||||||
|
assert url.endswith("/api/chat")
|
||||||
|
assert payload is not None
|
||||||
|
sent.append(payload)
|
||||||
|
return ollama_reply("<think>plan</think>Answer")
|
||||||
|
|
||||||
|
reply = local_backend.chat([{"role": "user", "content": "q"}], post=post)
|
||||||
|
assert reply.text == "Answer"
|
||||||
|
assert reply.prompt_tokens == 100
|
||||||
|
assert sent[0]["think"] is False
|
||||||
|
assert sent[0]["options"]["num_ctx"] == local_backend.NUM_CTX
|
||||||
|
assert sent[0]["keep_alive"] == local_backend.KEEP_ALIVE
|
||||||
|
assert sent[0]["stream"] is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_chat_tolerates_an_empty_answer() -> None:
|
||||||
|
reply = local_backend.chat(
|
||||||
|
[{"role": "user", "content": "q"}], post=lambda *_a: {}, clock=lambda: 1.0
|
||||||
|
)
|
||||||
|
assert (reply.text, reply.seconds, reply.prompt_tokens) == ("", 0.0, 0)
|
||||||
|
|
||||||
|
|
||||||
|
def test_chat_refuses_a_prompt_ollama_would_truncate() -> None:
|
||||||
|
def post(*_a: object) -> dict[str, Any]:
|
||||||
|
return ollama_reply("x", tokens=local_backend.NUM_CTX)
|
||||||
|
|
||||||
|
with pytest.raises(local_backend.ContextOverflowError):
|
||||||
|
local_backend.chat([{"role": "user", "content": "q"}], post=post)
|
||||||
|
huge = [{"role": "user", "content": "x" * local_backend.NUM_CTX * 4}]
|
||||||
|
with pytest.raises(local_backend.ContextOverflowError):
|
||||||
|
local_backend.chat(huge, post=post)
|
||||||
|
|
||||||
|
|
||||||
|
def test_gpu_hogs_ignores_ollama_and_small_processes() -> None:
|
||||||
|
out = "/usr/bin/ollama, 18000\n/game/bin/witcher3.exe, 6000\nXorg, 300\nbad\n"
|
||||||
|
hogs = local_backend.gpu_hogs(run=lambda *_a, **_k: completed(0, out))
|
||||||
|
assert hogs == [("/game/bin/witcher3.exe", 6000)]
|
||||||
|
|
||||||
|
|
||||||
|
def test_model_loaded_reads_api_ps() -> None:
|
||||||
|
def post(url: str, payload: object, _t: float) -> dict[str, Any]:
|
||||||
|
assert url.endswith("/api/ps")
|
||||||
|
assert payload is None
|
||||||
|
return {"models": [{"name": local_backend.MODEL}]}
|
||||||
|
|
||||||
|
assert local_backend.model_loaded(post)
|
||||||
|
assert not local_backend.model_loaded(lambda *_a: {})
|
||||||
|
|
||||||
|
|
||||||
|
class FakeResponse(io.BytesIO):
|
||||||
|
def __enter__(self) -> Self:
|
||||||
|
return self
|
||||||
|
|
||||||
|
def __exit__(self, *exc: object) -> None:
|
||||||
|
self.close()
|
||||||
|
|
||||||
|
|
||||||
|
def test_http_json_posts_and_gets_only_to_ollama(
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
) -> None:
|
||||||
|
seen: list[Any] = []
|
||||||
|
|
||||||
|
def urlopen(req: Any, timeout: float) -> FakeResponse:
|
||||||
|
seen.append((req.get_method(), req.data, timeout))
|
||||||
|
return FakeResponse(json.dumps({"ok": True}).encode())
|
||||||
|
|
||||||
|
monkeypatch.setattr(urllib.request, "urlopen", urlopen)
|
||||||
|
url = f"{local_backend.OLLAMA_URL}/api/chat"
|
||||||
|
assert local_backend.http_json(url, {"a": 1}, 2.0) == {"ok": True}
|
||||||
|
assert local_backend.http_json(url, None, 3.0) == {"ok": True}
|
||||||
|
assert seen == [("POST", b'{"a": 1}', 2.0), ("GET", None, 3.0)]
|
||||||
|
with pytest.raises(ValueError, match="non-Ollama URL"):
|
||||||
|
local_backend.http_json("http://example.com/api/chat", None, 1.0)
|
||||||
166
leetcode_coach/leetcode_coach/tests/test_local_coach.py
Normal file
166
leetcode_coach/leetcode_coach/tests/test_local_coach.py
Normal file
@ -0,0 +1,166 @@
|
|||||||
|
"""Tests for the offline /leetcode coach: prompt, no-solution gate, cards.
|
||||||
|
|
||||||
|
No network and no GPU: HTTP, nvidia-smi and subprocesses are stubbed.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
from typing import TYPE_CHECKING, Any
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from leetcode_coach import local_cards, local_coach, local_prompt, local_session
|
||||||
|
from leetcode_coach.tests._shared import LEAK, SAFE, STATEMENT, FakeChat, completed
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
import subprocess
|
||||||
|
|
||||||
|
|
||||||
|
def _coach(*texts: str) -> tuple[local_coach.LocalCoach, FakeChat]:
|
||||||
|
fake = FakeChat(*texts)
|
||||||
|
coach = local_coach.LocalCoach.start(STATEMENT)
|
||||||
|
coach.chat = fake
|
||||||
|
return coach, fake
|
||||||
|
|
||||||
|
|
||||||
|
def test_system_prompt_drops_fetch_step_and_model_gate() -> None:
|
||||||
|
skill = (local_coach.SKILL_DIR / "SKILL.md").read_text(encoding="utf-8")
|
||||||
|
prompt = local_prompt.system_prompt(skill, STATEMENT)
|
||||||
|
assert "fetch_problem.py" not in prompt
|
||||||
|
assert "/model sonnet" not in prompt
|
||||||
|
assert "## 3. The hint ladder" in prompt
|
||||||
|
assert prompt.rstrip().endswith("```")
|
||||||
|
assert "var twoSum" in prompt
|
||||||
|
|
||||||
|
|
||||||
|
def test_system_prompt_without_frontmatter_keeps_the_body() -> None:
|
||||||
|
prompt = local_prompt.system_prompt("# body only", "stmt")
|
||||||
|
assert prompt.startswith("# body only\n")
|
||||||
|
assert prompt.endswith("stmt\n")
|
||||||
|
|
||||||
|
|
||||||
|
def test_gate_blocks_starter_identifiers_but_allows_quoting_kuhy() -> None:
|
||||||
|
assert local_coach.gate_findings(LEAK, STATEMENT, ["hi"])
|
||||||
|
paste = "for (const n of nums) seen.set(target - n, n)"
|
||||||
|
assert not local_coach.gate_findings(LEAK, STATEMENT, [paste])
|
||||||
|
assert not local_coach.gate_findings(LEAK, STATEMENT, ["reveal"])
|
||||||
|
assert not local_coach.gate_findings(LEAK, "no starter here", ["hi"])
|
||||||
|
|
||||||
|
|
||||||
|
def test_leaky_reply_is_regenerated_with_the_gate_reason() -> None:
|
||||||
|
coach, fake = _coach(LEAK, SAFE)
|
||||||
|
turn = coach.respond("just give me the code")
|
||||||
|
assert turn.text == SAFE
|
||||||
|
assert turn.attempts == 2
|
||||||
|
assert "leetcode gate" in fake.prompts[1][-1]["content"]
|
||||||
|
assert coach.history[-1] == {"role": "assistant", "content": SAFE}
|
||||||
|
assert all("leetcode gate" not in m["content"] for m in coach.history)
|
||||||
|
|
||||||
|
|
||||||
|
def test_three_leaks_withhold_the_reply() -> None:
|
||||||
|
coach, _ = _coach(LEAK, LEAK, LEAK)
|
||||||
|
turn = coach.respond("code pls")
|
||||||
|
assert turn.withheld
|
||||||
|
assert turn.text == local_prompt.WITHHELD
|
||||||
|
assert "nums" not in turn.text
|
||||||
|
|
||||||
|
|
||||||
|
def test_prior_history_is_kept_for_a_mid_session_fallback() -> None:
|
||||||
|
prior = [{"role": "user", "content": "a"}, {"role": "assistant", "content": "b"}]
|
||||||
|
coach = local_coach.LocalCoach.start(STATEMENT, prior)
|
||||||
|
assert coach.history[1:] == prior
|
||||||
|
|
||||||
|
|
||||||
|
def test_split_cards_extracts_json_and_flags_garbage() -> None:
|
||||||
|
text = 'Done.\n```cards\n{"slug": "two-sum", "cards": []}\n```\n'
|
||||||
|
assert local_cards.split_cards(text) == ("Done.", {"slug": "two-sum", "cards": []})
|
||||||
|
assert local_cards.split_cards("plain") == ("plain", None)
|
||||||
|
_, bad = local_cards.split_cards("```cards\nnot json\n```")
|
||||||
|
assert bad is not None
|
||||||
|
assert "invalid" in bad
|
||||||
|
_, arr = local_cards.split_cards("```cards\n[1]\n```")
|
||||||
|
assert arr == {"invalid": [1]}
|
||||||
|
|
||||||
|
|
||||||
|
def test_deliver_cards_pipes_to_make_cards() -> None:
|
||||||
|
seen: dict[str, Any] = {}
|
||||||
|
|
||||||
|
def run(cmd: list[str], **kw: Any) -> subprocess.CompletedProcess[str]:
|
||||||
|
seen.update(cmd=cmd, input=kw["input"])
|
||||||
|
return completed(0, "| table |")
|
||||||
|
|
||||||
|
ok, out = local_cards.deliver_cards({"slug": "s", "cards": []}, run=run)
|
||||||
|
assert ok
|
||||||
|
assert out == "| table |"
|
||||||
|
assert seen["cmd"] == [sys.executable, "-m", "leetcode_coach.make_cards", "-"]
|
||||||
|
assert json.loads(seen["input"]) == {"slug": "s", "cards": []}
|
||||||
|
assert local_cards.deliver_cards({"invalid": "x"})[0] is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_rejected_cards_are_fed_back_once_fixed(
|
||||||
|
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
||||||
|
) -> None:
|
||||||
|
cards = '```cards\n{"slug": "s", "cards": []}\n```'
|
||||||
|
coach, _ = _coach("fixed\n" + cards)
|
||||||
|
results = iter([(False, "cards rejected: back too long"), (True, "| ok |")])
|
||||||
|
monkeypatch.setattr(local_cards, "deliver_cards", lambda _p: next(results))
|
||||||
|
local_session.show(local_coach.Turn("bye\n" + cards, 1.0, 1, withheld=False), coach)
|
||||||
|
out = capsys.readouterr().out
|
||||||
|
assert out == "bye\ncards rejected: back too long\n\n...\nfixed\n| ok |\n"
|
||||||
|
assert "rejected" in coach.user_texts()[-1]
|
||||||
|
|
||||||
|
|
||||||
|
def test_rejected_cards_give_up_after_the_retries(
|
||||||
|
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
||||||
|
) -> None:
|
||||||
|
cards = '```cards\n{"slug": "s", "cards": []}\n```'
|
||||||
|
# The last failed delivery still asks for a fix that is never shown: that
|
||||||
|
# is how the skill behaved before the move, kept as-is.
|
||||||
|
coach, fake = _coach("again\n" + cards, "still\n" + cards, "last\n" + cards)
|
||||||
|
monkeypatch.setattr(local_cards, "deliver_cards", lambda _p: (False, "no"))
|
||||||
|
local_session.show(local_coach.Turn("bye\n" + cards, 1.0, 1, withheld=False), coach)
|
||||||
|
out = capsys.readouterr().out
|
||||||
|
assert out.count("no\n") == local_session.CARD_RETRIES + 1
|
||||||
|
assert "last" not in out
|
||||||
|
assert fake.texts == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_stacked_questions_are_regenerated_then_shown_if_stuck() -> None:
|
||||||
|
coach, fake = _coach("What? Why?", SAFE)
|
||||||
|
turn = coach.respond("hint")
|
||||||
|
assert (turn.text, turn.attempts) == (SAFE, 2)
|
||||||
|
assert "exactly ONE question" in fake.prompts[1][-1]["content"]
|
||||||
|
coach, _ = _coach("What? Why?", "How? When?", "Who? Where?")
|
||||||
|
turn = coach.respond("hint")
|
||||||
|
assert not turn.withheld
|
||||||
|
assert turn.text == "Who? Where?"
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_cards_reply_needs_no_closing_question() -> None:
|
||||||
|
reply = 'Done.\n```cards\n{"slug": "s", "cards": []}\n```'
|
||||||
|
coach, _ = _coach(reply)
|
||||||
|
assert coach.respond("bye").text == reply
|
||||||
|
|
||||||
|
|
||||||
|
def test_reminder_reaches_the_model_but_not_history() -> None:
|
||||||
|
coach, fake = _coach(SAFE)
|
||||||
|
coach.respond("hint")
|
||||||
|
assert fake.prompts[0][-1]["content"].endswith(local_prompt.REMINDER)
|
||||||
|
assert coach.history[-2] == {"role": "user", "content": "hint"}
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("text", "ok"),
|
||||||
|
[
|
||||||
|
('What lets you check "has this been seen?" fast?', True),
|
||||||
|
("Try:\n```js\nx?.y\n```\nWhat prints? Predict before I tell you.", True),
|
||||||
|
("Answer in one word.", True),
|
||||||
|
("What happens? Why?", False),
|
||||||
|
("Nice work.", False),
|
||||||
|
("", False),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_eval_one_task_checker(text: str, *, ok: bool) -> None:
|
||||||
|
assert local_prompt.ends_with_one_task(text) is ok
|
||||||
142
leetcode_coach/leetcode_coach/tests/test_local_session.py
Normal file
142
leetcode_coach/leetcode_coach/tests/test_local_session.py
Normal file
@ -0,0 +1,142 @@
|
|||||||
|
"""Tests for the offline /leetcode session loop: fetch, GPU check, run."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import sys
|
||||||
|
from typing import Any
|
||||||
|
import urllib.error
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from leetcode_coach import local_backend, local_coach, local_session
|
||||||
|
from leetcode_coach.tests._shared import SAFE, STATEMENT, FakeChat, completed
|
||||||
|
|
||||||
|
|
||||||
|
def test_opening_message_and_quit() -> None:
|
||||||
|
assert local_session.opening_message(["loops"]) == (
|
||||||
|
"Start the session. What I'm stuck on: loops"
|
||||||
|
)
|
||||||
|
assert "not said" in local_session.opening_message([])
|
||||||
|
assert local_session.is_quit(" Quit ")
|
||||||
|
assert local_session.is_quit("exit")
|
||||||
|
assert not local_session.is_quit("quitting is hard")
|
||||||
|
|
||||||
|
|
||||||
|
def test_fetch_failure_asks_for_a_paste(capsys: pytest.CaptureFixture[str]) -> None:
|
||||||
|
got = local_session.fetch_statement(
|
||||||
|
"x", lambda: "pasted", runner=lambda *_a, **_k: completed(2)
|
||||||
|
)
|
||||||
|
assert got == "pasted"
|
||||||
|
assert capsys.readouterr().out == (
|
||||||
|
"[could not fetch the problem: premium or unknown problem]\n"
|
||||||
|
"Paste the statement and starter code, then an empty line:\n"
|
||||||
|
)
|
||||||
|
local_session.fetch_statement(
|
||||||
|
"x", lambda: None, runner=lambda *_a, **_k: completed(1, err=" offline \n")
|
||||||
|
)
|
||||||
|
assert capsys.readouterr().out.startswith("[could not fetch the problem: offline]")
|
||||||
|
ok = local_session.fetch_statement(
|
||||||
|
"x", lambda: None, runner=lambda *_a, **_k: completed(0, STATEMENT)
|
||||||
|
)
|
||||||
|
assert ok == STATEMENT
|
||||||
|
|
||||||
|
|
||||||
|
def test_fetch_runs_the_fetcher_by_path() -> None:
|
||||||
|
seen: list[list[str]] = []
|
||||||
|
|
||||||
|
def run(cmd: list[str], **_k: Any) -> Any:
|
||||||
|
seen.append(cmd)
|
||||||
|
return completed(0, "s")
|
||||||
|
|
||||||
|
local_session.fetch_statement("two-sum", lambda: None, runner=run)
|
||||||
|
assert seen[0][0] == sys.executable
|
||||||
|
assert seen[0][1].endswith("leetcode_coach/fetch_problem.py")
|
||||||
|
assert seen[0][2] == "two-sum"
|
||||||
|
|
||||||
|
|
||||||
|
def test_confirm_gpu_asks_only_when_something_else_holds_vram(
|
||||||
|
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
||||||
|
) -> None:
|
||||||
|
monkeypatch.setattr(local_backend, "model_loaded", lambda: False)
|
||||||
|
monkeypatch.setattr(local_backend, "gpu_hogs", lambda: [("game", 9000)])
|
||||||
|
assert local_session.confirm_gpu(lambda _q: "y")
|
||||||
|
assert capsys.readouterr().out == (
|
||||||
|
"[the GPU is busy -- loading the local model would compete with:]\n"
|
||||||
|
" game 9000 MiB\n"
|
||||||
|
)
|
||||||
|
assert not local_session.confirm_gpu(lambda _q: "")
|
||||||
|
monkeypatch.setattr(local_backend, "gpu_hogs", list)
|
||||||
|
assert local_session.confirm_gpu(lambda _q: pytest.fail("asked"))
|
||||||
|
monkeypatch.setattr(local_backend, "model_loaded", lambda: True)
|
||||||
|
assert local_session.confirm_gpu(lambda _q: pytest.fail("asked"))
|
||||||
|
|
||||||
|
|
||||||
|
def _session(
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
*,
|
||||||
|
statement: str | None = STATEMENT,
|
||||||
|
gpu_ok: bool = True,
|
||||||
|
chat: Any = None,
|
||||||
|
) -> None:
|
||||||
|
monkeypatch.setattr(local_session, "fetch_statement", lambda _p, _r: statement)
|
||||||
|
monkeypatch.setattr(local_session, "confirm_gpu", lambda: gpu_ok)
|
||||||
|
if chat is not None:
|
||||||
|
real_start = local_coach.LocalCoach.start
|
||||||
|
|
||||||
|
def start(
|
||||||
|
text: str, history: list[dict[str, str]] | None = None
|
||||||
|
) -> local_coach.LocalCoach:
|
||||||
|
coach = real_start(text, history)
|
||||||
|
coach.chat = chat
|
||||||
|
return coach
|
||||||
|
|
||||||
|
monkeypatch.setattr(local_coach.LocalCoach, "start", start)
|
||||||
|
|
||||||
|
|
||||||
|
def test_run_coaches_until_quit(
|
||||||
|
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
||||||
|
) -> None:
|
||||||
|
fake = FakeChat(SAFE, SAFE)
|
||||||
|
_session(monkeypatch, chat=fake)
|
||||||
|
blocks = iter(["hint", "quit"])
|
||||||
|
assert local_session.run("two-sum", [], "Start.", lambda: next(blocks)) == 0
|
||||||
|
out = capsys.readouterr().out
|
||||||
|
assert out.startswith(f"[local: {local_backend.MODEL} via Ollama]\n\n...\n")
|
||||||
|
assert out.count(SAFE) == 2
|
||||||
|
assert fake.texts == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_run_stops_without_a_statement_or_gpu(
|
||||||
|
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
||||||
|
) -> None:
|
||||||
|
_session(monkeypatch, statement=None)
|
||||||
|
assert local_session.run("x", [], "Start.", lambda: None) == 1
|
||||||
|
_session(monkeypatch, gpu_ok=False)
|
||||||
|
assert local_session.run("x", [], "Start.", lambda: None) == 1
|
||||||
|
assert capsys.readouterr().out == "[not loading the local model]\n"
|
||||||
|
|
||||||
|
|
||||||
|
def test_run_reports_an_unreachable_ollama(
|
||||||
|
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
||||||
|
) -> None:
|
||||||
|
def down(_m: object) -> object:
|
||||||
|
msg = "refused"
|
||||||
|
raise urllib.error.URLError(msg)
|
||||||
|
|
||||||
|
_session(monkeypatch, chat=down)
|
||||||
|
assert local_session.run("x", [], "Start.", lambda: None) == 1
|
||||||
|
out = capsys.readouterr().out
|
||||||
|
assert "[Ollama is not reachable at http://localhost:11434: " in out
|
||||||
|
assert out.endswith("[start it with: systemctl start ollama]\n")
|
||||||
|
|
||||||
|
|
||||||
|
def test_run_reports_a_context_overflow(
|
||||||
|
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
||||||
|
) -> None:
|
||||||
|
def full(_m: object) -> object:
|
||||||
|
msg = "too long"
|
||||||
|
raise local_backend.ContextOverflowError(msg)
|
||||||
|
|
||||||
|
_session(monkeypatch, chat=full)
|
||||||
|
assert local_session.run("x", [], "Start.", lambda: None) == 1
|
||||||
|
assert capsys.readouterr().out.endswith("[too long]\n")
|
||||||
126
leetcode_coach/leetcode_coach/tests/test_make_cards.py
Normal file
126
leetcode_coach/leetcode_coach/tests/test_make_cards.py
Normal file
@ -0,0 +1,126 @@
|
|||||||
|
"""Tests for the /leetcode card builder CLI: delivery paths and its output."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import io
|
||||||
|
import json
|
||||||
|
import runpy
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from leetcode_coach import anki_connect, make_cards
|
||||||
|
|
||||||
|
|
||||||
|
def _card(front: str, back: str = "x") -> dict[str, str]:
|
||||||
|
return {"front": front, "back": back}
|
||||||
|
|
||||||
|
|
||||||
|
def test_guid_ignores_escaping() -> None:
|
||||||
|
raw = make_cards.card_note("n & m?", "b", "slug")
|
||||||
|
esc = make_cards.card_note("n & m?", "b", "slug")
|
||||||
|
assert raw.guid == esc.guid
|
||||||
|
assert raw.fields == ["n & m?", "b", "slug"]
|
||||||
|
assert raw.tags == ["leetcode", "slug"]
|
||||||
|
|
||||||
|
|
||||||
|
def _run(monkeypatch: pytest.MonkeyPatch, argv: list[str], spec: dict[str, Any]) -> int:
|
||||||
|
monkeypatch.setattr("sys.argv", ["make_cards.py", *argv])
|
||||||
|
monkeypatch.setattr("sys.stdin", io.StringIO(json.dumps(spec)))
|
||||||
|
return make_cards.main()
|
||||||
|
|
||||||
|
|
||||||
|
GOOD = {
|
||||||
|
"slug": "s",
|
||||||
|
"cards": [_card("JS: cancel a pending setTimeout?", "clearTimeout(id)")],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_dry_run_prints_without_logging(
|
||||||
|
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str], tmp_path: Any
|
||||||
|
) -> None:
|
||||||
|
monkeypatch.setattr(make_cards, "OUT_DIR", tmp_path)
|
||||||
|
assert _run(monkeypatch, ["--dry-run", "-"], GOOD) == 0
|
||||||
|
assert capsys.readouterr().out == (
|
||||||
|
"| Front | Back |\n|---|---|\n"
|
||||||
|
"| JS: cancel a pending setTimeout? | clearTimeout(id) |\n"
|
||||||
|
)
|
||||||
|
assert not (tmp_path / "cards.tsv").exists()
|
||||||
|
|
||||||
|
|
||||||
|
def test_reads_a_file_argument(
|
||||||
|
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str], tmp_path: Any
|
||||||
|
) -> None:
|
||||||
|
spec = tmp_path / "cards.json"
|
||||||
|
spec.write_text(json.dumps(GOOD), encoding="utf-8")
|
||||||
|
assert _run(monkeypatch, ["--dry-run", str(spec)], {}) == 0
|
||||||
|
assert "clearTimeout(id)" in capsys.readouterr().out
|
||||||
|
|
||||||
|
|
||||||
|
def test_replace_offline_fails_closed(
|
||||||
|
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str], tmp_path: Any
|
||||||
|
) -> None:
|
||||||
|
monkeypatch.setattr(make_cards, "OUT_DIR", tmp_path)
|
||||||
|
monkeypatch.setattr(anki_connect, "available", lambda: False)
|
||||||
|
assert _run(monkeypatch, ["--replace", "-"], GOOD) == 1
|
||||||
|
assert "needs Anki open" in capsys.readouterr().err
|
||||||
|
assert not (tmp_path / "cards.tsv").exists()
|
||||||
|
|
||||||
|
|
||||||
|
def test_offline_writes_apkg_and_prints_bare_command(
|
||||||
|
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str], tmp_path: Any
|
||||||
|
) -> None:
|
||||||
|
monkeypatch.setattr(make_cards, "OUT_DIR", tmp_path)
|
||||||
|
monkeypatch.setattr(anki_connect, "available", lambda: False)
|
||||||
|
assert _run(monkeypatch, ["-"], GOOD) == 0
|
||||||
|
lines = capsys.readouterr().out.strip().splitlines()
|
||||||
|
assert lines[-1].startswith("anki ")
|
||||||
|
assert lines[-1].endswith(".apkg")
|
||||||
|
assert "(Anki desktop not open). Import with:" in lines[-2]
|
||||||
|
assert list((tmp_path / "anki").glob("*-s.apkg"))
|
||||||
|
tsv = (tmp_path / "cards.tsv").read_text(encoding="utf-8")
|
||||||
|
assert tsv.endswith("\ts\tJS: cancel a pending setTimeout?\tclearTimeout(id)\n")
|
||||||
|
|
||||||
|
|
||||||
|
def test_bad_arguments_and_rejected_cards(
|
||||||
|
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
||||||
|
) -> None:
|
||||||
|
assert _run(monkeypatch, ["--dry-run", "--replace", "-"], GOOD) == 1
|
||||||
|
usage = (
|
||||||
|
"Usage: make_cards.py [--dry-run | --replace] <cards.json | -> (- = stdin)"
|
||||||
|
)
|
||||||
|
assert capsys.readouterr().err.strip() == usage
|
||||||
|
assert _run(monkeypatch, [], GOOD) == 1
|
||||||
|
assert capsys.readouterr().err.strip() == usage
|
||||||
|
bad = {"slug": "s", "cards": [_card("JS: nest or separate?", "Nest.")]}
|
||||||
|
assert _run(monkeypatch, ["--dry-run", "-"], bad) == 1
|
||||||
|
err = capsys.readouterr().err
|
||||||
|
assert err.startswith("cards rejected:\n card 1: either/or front")
|
||||||
|
|
||||||
|
|
||||||
|
def test_online_path_pushes_and_reports(
|
||||||
|
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str], tmp_path: Any
|
||||||
|
) -> None:
|
||||||
|
calls: list[str] = []
|
||||||
|
monkeypatch.setattr(make_cards, "OUT_DIR", tmp_path)
|
||||||
|
monkeypatch.setattr(anki_connect, "available", lambda: True)
|
||||||
|
monkeypatch.setattr(anki_connect, "ensure_model", lambda *a: calls.append("model"))
|
||||||
|
monkeypatch.setattr(anki_connect, "sync", lambda: calls.append("sync"))
|
||||||
|
monkeypatch.setattr(
|
||||||
|
anki_connect, "sync_notes", lambda *a, replace: (1, 0, 2 if replace else 0)
|
||||||
|
)
|
||||||
|
assert _run(monkeypatch, ["--replace", "-"], GOOD) == 0
|
||||||
|
out = capsys.readouterr().out
|
||||||
|
assert "1 cards -> Anki desktop: 1 added, 0 updated, 2 deleted in 'LeetCode'" in out
|
||||||
|
assert "AnkiWeb sync requested" in out
|
||||||
|
assert calls == ["model", "sync"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_script_entry_point(
|
||||||
|
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
||||||
|
) -> None:
|
||||||
|
monkeypatch.setattr("sys.argv", ["make_cards.py"])
|
||||||
|
with pytest.raises(SystemExit) as exit_info:
|
||||||
|
runpy.run_path(make_cards.__file__, run_name="__main__")
|
||||||
|
assert exit_info.value.code == 1
|
||||||
|
assert capsys.readouterr().err.startswith("Usage: make_cards.py")
|
||||||
148
leetcode_coach/leetcode_coach/tests/test_stop_gate.py
Normal file
148
leetcode_coach/leetcode_coach/tests/test_stop_gate.py
Normal file
@ -0,0 +1,148 @@
|
|||||||
|
"""Tests for the /leetcode no-solution Stop hook (moved from ~/.claude/hooks)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import io
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from leetcode_coach import stop_gate
|
||||||
|
|
||||||
|
REPO = Path(stop_gate.__file__).resolve().parents[1]
|
||||||
|
SHIM = REPO / "leetcode_coach" / "hook_shims" / "leetcode_no_solution_stop.py"
|
||||||
|
SLASH_LOAD = {
|
||||||
|
"type": "user",
|
||||||
|
"message": {
|
||||||
|
"content": "<command-message>leetcode</command-message>\n"
|
||||||
|
"<command-name>/leetcode</command-name>"
|
||||||
|
},
|
||||||
|
}
|
||||||
|
STARTER = {
|
||||||
|
"type": "user",
|
||||||
|
"message": {"content": "## Starter code\nfunction sortPeople(names, heights) {"},
|
||||||
|
}
|
||||||
|
SOLUTION = (
|
||||||
|
"Fix: `return names.map((n, i) => [n, heights[i]]).sort((a, b) => b[1] - a[1]);`"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _skill_load(base: str) -> dict[str, Any]:
|
||||||
|
text = f"Base directory for this skill: {base}\n\n# /leetcode"
|
||||||
|
return {"type": "user", "message": {"content": [{"type": "text", "text": text}]}}
|
||||||
|
|
||||||
|
|
||||||
|
def _raw(*entries: dict[str, Any]) -> str:
|
||||||
|
return "".join(json.dumps(e, separators=(",", ":")) + "\n" for e in entries)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"load",
|
||||||
|
[
|
||||||
|
SLASH_LOAD,
|
||||||
|
_skill_load(str(Path.home() / ".claude" / "skills" / "leetcode")),
|
||||||
|
_skill_load(str(Path(stop_gate.__file__).resolve().parent)),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_gate_arms_on_every_load_kind(load: dict[str, Any]) -> None:
|
||||||
|
reason = stop_gate.verdict(_raw(load, STARTER), SOLUTION)
|
||||||
|
assert reason is not None
|
||||||
|
assert reason.startswith("leetcode gate: your reply hands kuhy code they can paste")
|
||||||
|
assert "`return names.map((n, i) => [n, heights[i]]).sort((a, b) =>" in reason
|
||||||
|
|
||||||
|
|
||||||
|
def test_gate_idle_without_load_reply_reveal_or_starter() -> None:
|
||||||
|
assert stop_gate.verdict(_raw(STARTER), SOLUTION) is None
|
||||||
|
assert stop_gate.verdict(_raw(SLASH_LOAD, STARTER), "") is None
|
||||||
|
reveal = {"type": "user", "message": {"content": "reveal"}}
|
||||||
|
assert stop_gate.verdict(_raw(SLASH_LOAD, STARTER, reveal), SOLUTION) is None
|
||||||
|
assert stop_gate.verdict(_raw(SLASH_LOAD), SOLUTION) is None
|
||||||
|
assert stop_gate.verdict(_raw(SLASH_LOAD, STARTER), "What is `names`?") is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_blocks_are_counted_per_prompt(
|
||||||
|
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
||||||
|
) -> None:
|
||||||
|
monkeypatch.setattr(stop_gate, "STATE_DIR", tmp_path / "state")
|
||||||
|
assert [stop_gate.blocks_so_far("s", "p1") for _ in range(3)] == [0, 1, 2]
|
||||||
|
assert stop_gate.blocks_so_far("s", "p2") == 0
|
||||||
|
assert (tmp_path / "state" / "s.blocks").read_text(encoding="utf-8") == (
|
||||||
|
"p1\np1\np1\np2\n"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _main(
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
capsys: pytest.CaptureFixture[str],
|
||||||
|
event: dict[str, Any],
|
||||||
|
) -> str:
|
||||||
|
monkeypatch.setattr("sys.stdin", io.StringIO(json.dumps(event)))
|
||||||
|
assert stop_gate.main() == 0
|
||||||
|
return capsys.readouterr().out
|
||||||
|
|
||||||
|
|
||||||
|
def test_main_blocks_then_fails_open_at_the_cap(
|
||||||
|
tmp_path: Path,
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
capsys: pytest.CaptureFixture[str],
|
||||||
|
) -> None:
|
||||||
|
monkeypatch.setattr(stop_gate, "STATE_DIR", tmp_path / "state")
|
||||||
|
transcript = tmp_path / "t.jsonl"
|
||||||
|
transcript.write_text(_raw(SLASH_LOAD, STARTER), encoding="utf-8")
|
||||||
|
event = {
|
||||||
|
"transcript_path": str(transcript),
|
||||||
|
"last_assistant_message": SOLUTION,
|
||||||
|
"session_id": "s",
|
||||||
|
"prompt_id": "p",
|
||||||
|
}
|
||||||
|
outs = [_main(monkeypatch, capsys, event) for _ in range(stop_gate.MAX_BLOCKS + 1)]
|
||||||
|
first = json.loads(outs[0])
|
||||||
|
assert first["decision"] == "block"
|
||||||
|
assert first["reason"] == stop_gate.verdict(transcript.read_text(), SOLUTION)
|
||||||
|
assert outs[1] == outs[0]
|
||||||
|
assert outs[-1] == ""
|
||||||
|
|
||||||
|
|
||||||
|
def test_main_passes_without_transcript_or_verdict(
|
||||||
|
tmp_path: Path,
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
capsys: pytest.CaptureFixture[str],
|
||||||
|
) -> None:
|
||||||
|
assert _main(monkeypatch, capsys, {"transcript_path": str(tmp_path / "no")}) == ""
|
||||||
|
assert _main(monkeypatch, capsys, {}) == ""
|
||||||
|
transcript = tmp_path / "t.jsonl"
|
||||||
|
transcript.write_text(_raw(STARTER), encoding="utf-8")
|
||||||
|
event = {"transcript_path": str(transcript), "last_assistant_message": SOLUTION}
|
||||||
|
assert _main(monkeypatch, capsys, event) == ""
|
||||||
|
|
||||||
|
|
||||||
|
def test_hook_shim_execs_the_package_gate(tmp_path: Path) -> None:
|
||||||
|
"""The ~/.claude/hooks shim hands stdin to `-m leetcode_coach.stop_gate`."""
|
||||||
|
transcript = tmp_path / "t.jsonl"
|
||||||
|
transcript.write_text(_raw(SLASH_LOAD, STARTER), encoding="utf-8")
|
||||||
|
event = {
|
||||||
|
"transcript_path": str(transcript),
|
||||||
|
"last_assistant_message": SOLUTION,
|
||||||
|
"session_id": f"shim-test-{tmp_path.name}",
|
||||||
|
"prompt_id": "p",
|
||||||
|
}
|
||||||
|
assert len(SHIM.read_text(encoding="utf-8").splitlines()) <= 3
|
||||||
|
proc = subprocess.run(
|
||||||
|
[sys.executable, str(SHIM)],
|
||||||
|
input=json.dumps(event),
|
||||||
|
capture_output=True,
|
||||||
|
text=True,
|
||||||
|
check=False,
|
||||||
|
cwd=tmp_path,
|
||||||
|
# -P in the shim keeps cwd off sys.path; the real hook finds the
|
||||||
|
# package through install.sh's editable install, this test through
|
||||||
|
# PYTHONPATH (which -P still honours). HOME keeps the block counter
|
||||||
|
# out of the real ~/.claude/state.
|
||||||
|
env={"HOME": str(tmp_path), "PATH": "/usr/bin:/bin", "PYTHONPATH": str(REPO)},
|
||||||
|
)
|
||||||
|
assert proc.returncode == 0, proc.stderr
|
||||||
|
assert json.loads(proc.stdout)["decision"] == "block"
|
||||||
Loading…
Reference in New Issue
Block a user