mirror of
https://github.com/kuhyx/todo-app.git
synced 2026-11-10 19:27:01 +01:00
Cut the llm-design-spec template from 12 sections to 7
Audited the template against 514 real Claude sessions (1503 top-level prompts) joined to the 49 notes in BACKLOG.md. Sections were judged on whether the agent actually read them, not on fill rate — four of them said "leave blank if none" in their own helper, so a low fill rate was the section working correctly. Measured over the 22 note/session pairs: estimate filled 22, referenced 1 (4%) done filled 22, quoted back 4 (18%) refs filled 14, URLs fetched 5 (35%) tech the agent opened pubspec.yaml/CLAUDE.md anyway in 54% ask 100% ask rate with it, 92% without — no measurable effect Kept: title, what, where, must, done. Dropped: tech (into `where`), ask, nice and never (into `must` as `optional:` / `must not:` lines), depends, estimate. Renamed `refs` -> `read first`. Added `verify`, the only addition: "test it yourself, on the phone" was the single most repeated correction in the corpus, and 3 of the 7 filled `never` sections were already smuggling it in. Fixes a latent data-loss bug found while testing this. parse() treats an unknown `## heading` as content on purpose (values may contain subheadings), so a legacy `## tech` block was folded into the preceding section's value AND the note still reported conforming — opening any old note in the stepper would have swallowed it on the next save. NoteTemplate.retiredLabels now makes those notes explicitly non-conforming, so they open raw and untouched. tool/migrate_backlog.dart converts an old export (export -> migrate -> import). It is idempotent and gated: it throws if a rewritten note does not conform or if any source line vanishes. All 49 notes migrated on the desktop and merged to the phone (0 new, 25 updated, 24 unchanged). Also formats the few files dart format had never been run over, and extracts a local in frame_stats_io.dart so the reflowed line stays under 80 chars. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
032376a4d7
commit
a874e09b65
47
CLAUDE.md
47
CLAUDE.md
@ -78,13 +78,26 @@ Three layers under `lib/`, each single-purpose:
|
||||
|
||||
### Note template (default content of every new note)
|
||||
|
||||
Every new note pre-fills this scaffold (matches the user's `<work_backlog>`
|
||||
format): a title line plus `what / where / must / nice / out / done / depends /
|
||||
estimate / refs` sections.
|
||||
`lib/data/note_template.dart` is the **single source of truth** — the stepper,
|
||||
the editor and the tests are all template-driven, so changing that list changes
|
||||
every surface. A new note pre-fills a title line plus
|
||||
`what / where / must / done / verify / read first`.
|
||||
|
||||
The set was cut from twelve sections to seven in 2026-07 after auditing 514 real
|
||||
sessions (`docs/llm-design-spec-audit.md`): `estimate`, `ask`, `nice`, `tech`,
|
||||
`never` and `depends` were measurably never read, and `verify` was added because
|
||||
"test it yourself, on the phone" was the single most repeated correction in the
|
||||
corpus. Do not re-add a section without evidence that it gets used.
|
||||
|
||||
`NoteTemplate.retiredLabels` lists the dropped headings. `parse()` reports any
|
||||
note containing one as **non-conforming** so it opens in the raw editor — without
|
||||
that, the stepper would fold a retired section into its neighbour on the next
|
||||
save. `tool/migrate_backlog.dart` converts an old export (Settings → Export →
|
||||
migrate → Settings → Import).
|
||||
|
||||
## Testing
|
||||
|
||||
- **The suite must stay fast (~5s) and fully green.** It is currently **101
|
||||
- **The suite must stay fast (~5s) and fully green.** It is currently **191
|
||||
tests at 100% line coverage**. Don't regress either.
|
||||
- Widget tests use `test/fake_note_repository.dart` — a `FakeNoteRepository`
|
||||
built on `StreamController`s, **not** a real DB. A real `sqlite_crdt` DB
|
||||
@ -477,15 +490,20 @@ linter:
|
||||
provide an intuitive and easy navigation bar or controls.
|
||||
* **Typography:** Stress and emphasize font sizes to ease understanding, e.g.,
|
||||
hero text, section headlines, list headlines, keywords in paragraphs.
|
||||
* **Background:** Apply subtle noise texture to the main background to add a
|
||||
premium, tactile feel.
|
||||
* **Shadows:** Multi-layered drop shadows create a strong sense of depth; cards
|
||||
have a soft, deep shadow to look "lifted."
|
||||
* **Background:** Keep backgrounds flat and simple — no noise texture or
|
||||
gradients. Follow the `unified-design-system` skill (safe-design-rules
|
||||
rule 17: simple-on-complex); let content and typography carry visual
|
||||
interest instead of background texture.
|
||||
* **Shadows:** Follow the `unified-design-system` skill's shadow policy: no
|
||||
shadows at all in dark UI (differentiate elevation with lighter surface
|
||||
fills, not shadows); in light UI, shadows are for floating/overlay
|
||||
elements only (menus, dialogs), never for inline cards, and never mixed
|
||||
with a border on the same surface.
|
||||
* **Icons:** Incorporate icons to enhance the user’s understanding and the
|
||||
logical navigation of the app.
|
||||
* **Interactive Elements:** Buttons, checkboxes, sliders, lists, charts, graphs,
|
||||
and other interactive elements have a shadow with elegant use of color to
|
||||
create a "glow" effect.
|
||||
* **Interactive Elements:** Buttons/controls signal state via color and fill
|
||||
changes, not a "glow" shadow — one depth technique per surface, per
|
||||
`unified-design-system`.
|
||||
|
||||
#### Theming
|
||||
* **Centralized Theme:** Define a centralized `ThemeData` object to ensure a
|
||||
@ -493,8 +511,11 @@ linter:
|
||||
* **Light and Dark Themes:** Implement support for both light and dark themes,
|
||||
ideal for a user-facing theme toggle (`ThemeMode.light`, `ThemeMode.dark`,
|
||||
`ThemeMode.system`).
|
||||
* **Color Scheme Generation:** Generate harmonious color palettes from a single
|
||||
color using `ColorScheme.fromSeed`.
|
||||
* **Color Scheme Generation:** Do NOT use `ColorScheme.fromSeed` for this
|
||||
app's palette — it can't reproduce hand-picked values exactly. Build
|
||||
`ThemeData` from explicit `ColorScheme.dark(...)`/`.light(...)` using the
|
||||
frozen, identical-everywhere tokens in the `unified-design-system` skill
|
||||
(`~/.claude/skills/unified-design-system/references/tokens.md`).
|
||||
|
||||
```dart
|
||||
final ThemeData lightTheme = ThemeData(
|
||||
|
||||
213
docs/llm-design-spec-audit.md
Normal file
213
docs/llm-design-spec-audit.md
Normal file
@ -0,0 +1,213 @@
|
||||
# Audit of the `llm-design-spec` note template against 514 real Claude sessions
|
||||
|
||||
## Context
|
||||
|
||||
`~/todo`'s default note template (`lib/data/note_template.dart`, `NoteTemplate.llmDesignSpec`)
|
||||
has 12 sections: title, what, where, tech, must, ask, nice, never, done, depends,
|
||||
estimate, refs. It was designed top-down from the `<work_backlog>` format in
|
||||
`~/.claude/CLAUDE.md`, never validated against what actually gets written or what
|
||||
Claude actually reads.
|
||||
|
||||
This audit measures it against the real corpus. Everything below is measured, not
|
||||
estimated. Fill rate alone was deliberately **not** used as the verdict — four
|
||||
sections say "Leave blank if none" in their own helper, so a low fill rate is the
|
||||
field working correctly. The discriminating test is outcome-linked: *was the section
|
||||
read, and did leaving it out cause a failure?*
|
||||
|
||||
## Evidence base
|
||||
|
||||
| Source | Size |
|
||||
| --- | --- |
|
||||
| `~/.claude/projects/**/*.jsonl` | 514 sessions, 495 with real user prompts, **1503 prompts** |
|
||||
| `~/todo/BACKLOG.md` | 49 notes — **25 structured, 24 freeform** |
|
||||
| note → session join (title match) | **22 note/session pairs**, 21 of 25 notes traced |
|
||||
| `~/.claude/memories/mistakes.md` + `projects/-home-kuhy/memory/feedback-*.md` | 13 dated mistakes, 33 feedback memories |
|
||||
|
||||
**How you actually prompt** (this frames everything else):
|
||||
- median prompt **324 chars**; 44% under 200 chars; 26% under 100
|
||||
- only **21 of 495 sessions (4%)** open with a structured template note
|
||||
- of the 24 freeform notes, **9 start with a bare URL and 8 are *only* a URL**
|
||||
|
||||
Half the backlog bypasses the template entirely. Any recommendation below is
|
||||
net-subtractive by default; every proposed addition has a named failure behind it.
|
||||
|
||||
## What Claude did not use (deterministic tests over the 22 pairs)
|
||||
|
||||
| Section | Filled | Provably used | Test |
|
||||
| --- | ---: | ---: | --- |
|
||||
| `estimate` | 22 | **1 (4%)** | word "estimate" ever appears in Claude's output |
|
||||
| `done` | 22 | **4 (18%)** | the done text is quoted back at verification time |
|
||||
| `refs` | 14 | **5 (35%)** | any listed URL/domain hit by WebFetch/servo-fetch/WebSearch/curl |
|
||||
| `tech` | 22 | 20 (90%) echoed — but Claude opened `pubspec.yaml`/`pyproject.toml`/`CLAUDE.md` **anyway in 54%** | token overlap + manifest reads |
|
||||
| `ask` | 9 | 100% asked — **but 92% asked without it too** (median 4 asks vs 5) | AskUserQuestion / ExitPlanMode calls |
|
||||
|
||||
The `ask` row is the sharpest result: filling it changes Claude's asking behaviour by
|
||||
roughly nothing, because `~/.claude/CLAUDE.md`'s "Ask first when unsure" + the
|
||||
spec-first gate already force it. The section is inert.
|
||||
|
||||
`done` at 18% is the most damaging: you write an observable success condition and in
|
||||
four cases out of five Claude verified against a criterion it invented itself.
|
||||
|
||||
## Where Claude went wrong, and what would have prevented it
|
||||
|
||||
Hand-read of the corrections inside note-driven sessions plus the sampled corrections
|
||||
across all 495 sessions. Three recurring classes, all traceable to a template gap:
|
||||
|
||||
1. **"Test it yourself, on the device"** — recurs verbatim across wake_alarm,
|
||||
diet_guard, screen-locker: *"as I said before, test it yourself pleeease"*,
|
||||
*"You did not install this new vesrion on phone, do not bother testing on desktop"*,
|
||||
*"isntead of making me test it test it yourself"*. Backed by five separate feedback
|
||||
memories (`feedback-device-verification-before-tests`,
|
||||
`feedback-verify-real-deployment-path`, `feedback-stop-when-device-disconnected`,
|
||||
`feedback-blind-adb-tap-unreliable`, `feedback-flutter-install-uninstalls`).
|
||||
**Gap: there is no `verify` section.** `done` conflates *what success looks like*
|
||||
with *how and where to check it*, and Claude drops the second half.
|
||||
Confirming evidence: 3 of the 7 filled `never` sections are smuggling this in —
|
||||
*"test ONLY on the phone, if the phone is NOT connected … STOP AND LET ME KNOW"*,
|
||||
*"do NOT work if the phone is NOT connected"*.
|
||||
|
||||
2. **Wrong *kind* of fix** — *"you are fixing the problem in a wrong way, instead of
|
||||
devising an algorithm to detect if a game is winnable 100% you are playing around
|
||||
[with the] number of pieces"*, *"I said make it possible to pass but you made it
|
||||
wayyy to easy"*. **Gap: `done` accepts a vague sentence.** This exact failure had
|
||||
to be patched globally afterwards ("Algorithmic guarantees over parameter tuning"
|
||||
in `workflow-rules.md`) — a note-level threshold would have caught it first.
|
||||
|
||||
3. **UI intent mismatch** — *"I was thinking more of an app icon like the one for
|
||||
network or nvidia"*, *"both sliders have bad ux"*, *"why is it upside down"*,
|
||||
*"this is ugly as it is not symmetrical"*. **Gap: nowhere to attach a visual
|
||||
reference.** `refs` is worded as reading material and is ignored 65% of the time.
|
||||
This is the same failure as the 2026-07-21 mistake entry (reference image read as
|
||||
content instead of format).
|
||||
|
||||
**Negative result worth stating plainly:** the other 10 of 13 entries in
|
||||
`mistakes.md` — git stash, feature-branch-vs-main, killing a running game session,
|
||||
mid-queue status reports, conftest-before-tests, heredoc escaping, over-claiming a
|
||||
rollout complete — map to **no template section at all**. They are global process
|
||||
failures. This is the argument against growing the template to chase them: it would
|
||||
add fields that cannot work.
|
||||
|
||||
## Per-section verdict
|
||||
|
||||
| Section | Verdict | Grounds |
|
||||
| --- | --- | --- |
|
||||
| `title` | **keep** | 25/25 |
|
||||
| `what` | **keep** | 25/25 — the load-bearing section |
|
||||
| `where` | **keep, widen** | 25/25. Widen to allow "the fix belongs in another repo" — two corrections were exactly that (*"our INSTALLER, installing files in a WRONG FOLDER"*, *"The real fix is in ~/screen-locker (a different repo)"*) |
|
||||
| `tech` | **merge into `where`** | Claude reads the manifest anyway in 54% of sessions. Load-bearing only for `where — new app: <name>`, which is where it should live |
|
||||
| `must` | **keep** | 25/25 |
|
||||
| `ask` | **remove** | Provably inert: 92% ask rate without it vs 100% with it. Global rules already own this |
|
||||
| `nice` | **remove** | 13/25, no evidence of use, and it invites the gold-plating that `never` then has to forbid |
|
||||
| `never` | **remove as a section** | 2 of 7 literally say "none"; 3 of the rest are verification constraints (→ `verify`); 1 is a `where` constraint; 1 is genuine scope (*"do not touch any other pieces of Todo app"*) → fold that shape into `must` as a "must not" line |
|
||||
| `done` | **keep, tighten** | 25/25 filled but quoted back only 18%. Helper must demand a threshold or a check command, not a sentence |
|
||||
| `depends` | **remove** | 13/25, zero evidence of use downstream |
|
||||
| `estimate` | **remove** | 25/25 filled, referenced once in 22 sessions. Pure write-only metadata |
|
||||
| `refs` | **keep, rename → `read first`** | Fetched in only 35% of cases. The name reads as "citations"; the helper already says "read first" — make the name match the instruction |
|
||||
| — | **add `verify`** | See failure class 1. Where and how to check: device, command, deploy path |
|
||||
| — | **add a visual-reference slot** | See failure class 3 — either a `looks like` section or image support in `read first` |
|
||||
|
||||
## What was implemented (2026-07-26)
|
||||
|
||||
**12 sections → 7:** `title, what, where, must, done, verify, read first` —
|
||||
`never` and `nice` became `must not:` / `optional:` line prefixes inside `must`
|
||||
rather than sections of their own, and `tech` was folded into `where`.
|
||||
|
||||
Landed in:
|
||||
- `lib/data/note_template.dart` — the new section list, plus
|
||||
`NoteTemplate.retiredLabels`. The audit turned up a latent bug while testing
|
||||
this: `parse()` treats an unknown `## heading` as content (deliberately — users
|
||||
write subheadings inside values), which meant a legacy `## tech` block was
|
||||
silently folded into the preceding section's value **and the note still
|
||||
reported as conforming**. Opening any old note in the stepper would have
|
||||
swallowed it on the next save. Retired labels are now explicitly
|
||||
non-conforming, so such notes open in the raw editor untouched.
|
||||
- `tool/migrate_backlog.dart` — export → transform → import migration, with two
|
||||
gates that throw rather than warn: every rewritten note must parse as
|
||||
conforming, and no source line may vanish except the deliberately dropped
|
||||
`ask`/`depends`/`estimate`.
|
||||
- `CLAUDE.md` (this repo, which was documenting a dead `out` section) and
|
||||
`~/.claude/CLAUDE.md`'s `<work_backlog>` format.
|
||||
|
||||
## Part 2 — what the opening prompt leaves out, and why more fields won't fix it
|
||||
|
||||
The first half of this audit asked "which template sections get read". The more
|
||||
useful question is "what does the opening prompt *miss*, and what would stop it
|
||||
being missed". Measured over 246 sessions with a real top-level prompt (sidechain
|
||||
and subagent turns excluded — they inflate every count roughly 2x).
|
||||
|
||||
**Three results, and the third overturns the premise.**
|
||||
|
||||
1. **Claude has to ask in 52% of sessions** (129/246), 716 questions total.
|
||||
2. **The first question arrives at assistant turn 24 (median); only 3% of asking
|
||||
sessions ask within the first three turns.** So Claude is almost never asking
|
||||
because the prompt was thin. It asks after ~24 turns of reading code and
|
||||
hitting a fork that *did not exist* at prompt time. These are discoveries, not
|
||||
omissions.
|
||||
3. **Opening-prompt length does not predict corrections.** Split 123 real
|
||||
multi-prompt sessions into thirds by opening length:
|
||||
|
||||
| opening length | n | median | corrections/session |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| shortest third | 41 | 102 ch | 0.29 |
|
||||
| middle third | 41 | 410 ch | 0.41 |
|
||||
| longest third | 41 | 2133 ch | 0.22 |
|
||||
|
||||
A 20x longer opening buys a 0.07 reduction, and the middle third is the worst.
|
||||
**Writing more upfront does not measurably help.** Any advice of the form
|
||||
"fill in more of the template before you start" is unsupported by the corpus.
|
||||
|
||||
### What genuinely does arrive late
|
||||
|
||||
62% of multi-prompt sessions get a constraint after prompt 1; 56% of opening
|
||||
prompts carry no constraint at all. Of 152 late-constraint messages:
|
||||
|
||||
| what the late constraint is about | n |
|
||||
| --- | ---: |
|
||||
| scope limit ("only this", "don't touch X") | 30 |
|
||||
| behaviour rule ("should always/never …") | 18 |
|
||||
| device / where to test | 17 |
|
||||
| visual-UX rule | 17 |
|
||||
| process/git | 10 |
|
||||
|
||||
Only the middle-of-the-road categories are knowable at prompt time. **"Device /
|
||||
where to test" is the one that is knowable, constant, and still repeated** — for
|
||||
`todo`, `diet_guard_app`, `workout_app` and `wake_alarm` the answer is always the
|
||||
phone, and it was still typed out 17 times, plus 5 feedback memories, plus 3 of
|
||||
the 7 old `never` sections.
|
||||
|
||||
### The uncomfortable conclusion
|
||||
|
||||
That constant already has a standing rule, a memory (`prefer-mobile-testing`) and
|
||||
a whole skill (`phone-deploy`) telling the agent mobile is primary — and it was
|
||||
still violated 17 times. **A constraint that is already written down three times
|
||||
and still missed is not a prompting defect.** No extra field, longer note, or
|
||||
stricter capture discipline can fix it, which is why this audit adds `verify` as
|
||||
the *only* new section and stops there.
|
||||
|
||||
The leverage is therefore not at capture time:
|
||||
- **Capture stays cheap.** The evidence supports fewer fields, not more. Median
|
||||
opening prompt is 363 chars and that is fine.
|
||||
- **Per-repo constants belong in that repo's `CLAUDE.md`, written once** — never
|
||||
re-typed per note. `verify` exists for the exception, not the rule.
|
||||
- **The real cost centre is the turn-24 fork**, which no prompt could have
|
||||
pre-empted. Improving that means better mid-session surfacing of decisions, not
|
||||
a better opening prompt.
|
||||
|
||||
## Re-running this audit
|
||||
|
||||
The measurements are all reproducible from `~/.claude/projects/**/*.jsonl` plus a
|
||||
fresh `BACKLOG.md` export. The two that matter for judging a section's worth:
|
||||
|
||||
- **is it read?** — join a note to the sessions that consumed it (match on the
|
||||
note's title), then test the section against the transcript: were the `refs`
|
||||
URLs actually fetched, was `done` quoted back at verification time, does
|
||||
`estimate` appear anywhere downstream.
|
||||
- **does it change behaviour?** — compare sessions where the section was filled
|
||||
against sessions where it was blank. This is what condemned `ask`: a 100% ask
|
||||
rate with it versus 92% without is not an effect.
|
||||
|
||||
Fill rate alone proves nothing — four of the removed sections said "Leave blank
|
||||
if none" in their own helper text, so a low fill rate was them working correctly.
|
||||
Do not re-add a section on a coverage argument; re-add it when a failure names
|
||||
it.
|
||||
|
||||
@ -18,9 +18,10 @@
|
||||
/// * `assemble(parse(text))` is idempotent for any text that *conforms* to a
|
||||
/// template, so opening and closing a structured note never mutates it.
|
||||
/// * Text that does **not** conform (freeform notes, the old `what —` format,
|
||||
/// hand-mangled raw edits) is reported as such so the UI can fall back to a
|
||||
/// raw editor and show it verbatim — we never force non-conforming text
|
||||
/// into the guided stepper and never drop a line we couldn't place.
|
||||
/// the pre-2026-07 twelve-section spec, hand-mangled raw edits) is reported
|
||||
/// as such so the UI can fall back to a raw editor and show it verbatim —
|
||||
/// we never force non-conforming text into the guided stepper and never
|
||||
/// drop a line we couldn't place.
|
||||
library;
|
||||
|
||||
/// One field of a structured template.
|
||||
@ -51,7 +52,7 @@ class TemplateSection {
|
||||
final String hint;
|
||||
|
||||
/// Whether the stepper renders a single-line input (title/what/…) versus a
|
||||
/// multi-line input for list-shaped sections (must/nice/out/refs). Purely a
|
||||
/// multi-line input for list-shaped sections (must/read first). Purely a
|
||||
/// UI hint; the stored format is the same for both.
|
||||
final bool inline;
|
||||
|
||||
@ -82,9 +83,17 @@ class NoteTemplate {
|
||||
/// Whether this template has no structure (just a plain-text body).
|
||||
bool get isFreeform => sections.isEmpty;
|
||||
|
||||
/// The LLM-oriented design-spec template (the default). Mirrors the user's
|
||||
/// backlog format; every section carries a self-documenting guidance line so
|
||||
/// the pasted note tells the LLM how to read it.
|
||||
/// The LLM-oriented design-spec template (the default). Every section carries
|
||||
/// a self-documenting guidance line so the pasted note tells the LLM how to
|
||||
/// read it.
|
||||
///
|
||||
/// The section set is deliberately small. It was cut from twelve to seven in
|
||||
/// 2026-07 after auditing 514 real sessions: `estimate` was referenced in 1
|
||||
/// of 22 note-driven sessions, `ask` changed the agent's asking behaviour not
|
||||
/// at all (92% ask rate without it vs 100% with), and `tech` lost to the
|
||||
/// agent just opening `pubspec.yaml`. `verify` is the one addition — the
|
||||
/// single most repeated correction in the corpus was "test it yourself, on
|
||||
/// the phone". See `docs/llm-design-spec-audit.md`.
|
||||
static const NoteTemplate llmDesignSpec = NoteTemplate(
|
||||
id: 'llm-design-spec',
|
||||
label: 'LLM design spec',
|
||||
@ -105,89 +114,72 @@ class NoteTemplate {
|
||||
TemplateSection(
|
||||
key: 'where',
|
||||
label: 'where',
|
||||
hint: 'repo + files/paths, or new app: <name>',
|
||||
hint: 'repo + files/paths, or new app: <name> + stack',
|
||||
helper:
|
||||
'Repo + target files/paths (not terminal dumps), or '
|
||||
"'new app: <name>', and platform(s).",
|
||||
),
|
||||
TemplateSection(
|
||||
key: 'tech',
|
||||
label: 'tech',
|
||||
hint: 'language/framework@version, key libraries',
|
||||
helper:
|
||||
'Tech stack with versions (e.g. Flutter 3.32, Dart 3.8, '
|
||||
'sqlite_crdt 0.5). Tells the agent the exact environment.',
|
||||
"'new app: <name>' with the stack and versions. Name the repo "
|
||||
'that actually owns the fix, even if it is not the obvious one.',
|
||||
),
|
||||
TemplateSection(
|
||||
key: 'must',
|
||||
label: 'must',
|
||||
inline: false,
|
||||
hint: '- required behaviour',
|
||||
hint: '- required behaviour / must not: hard stop',
|
||||
helper:
|
||||
'Required behaviours the agent should do without asking, '
|
||||
'one per line — hard requirements.',
|
||||
),
|
||||
TemplateSection(
|
||||
key: 'ask',
|
||||
label: 'ask',
|
||||
inline: false,
|
||||
hint: '- decision needing your approval before proceeding',
|
||||
helper:
|
||||
'High-impact decisions where the agent must stop and confirm '
|
||||
'with you first (e.g. schema changes, deleting data). '
|
||||
'Leave blank if none.',
|
||||
),
|
||||
TemplateSection(
|
||||
key: 'nice',
|
||||
label: 'nice',
|
||||
inline: false,
|
||||
hint: '- optional behaviour',
|
||||
helper:
|
||||
'Optional behaviours; skipping them is fine. '
|
||||
'Leave blank if none.',
|
||||
),
|
||||
TemplateSection(
|
||||
key: 'never',
|
||||
label: 'never',
|
||||
inline: false,
|
||||
hint: '- hard stop / explicitly out of scope',
|
||||
helper:
|
||||
'Hard stops and explicitly out-of-scope items — '
|
||||
'agent must not do these. Prevents gold-plating and '
|
||||
'dangerous actions. Leave blank if none.',
|
||||
'Required behaviours the agent does without asking, one per '
|
||||
"line. Prefix a hard stop with 'must not:' and an optional "
|
||||
"extra with 'optional:'.",
|
||||
),
|
||||
TemplateSection(
|
||||
key: 'done',
|
||||
label: 'done',
|
||||
hint: 'I can X and Y happens',
|
||||
hint: 'a threshold, a guarantee, or a check command',
|
||||
helper:
|
||||
"Observable success: 'I can X and Y happens' — used to verify "
|
||||
'completion.',
|
||||
'Observable success as a threshold, a guarantee, or a command '
|
||||
"whose output settles it — not a sentence. 'Feels better' is "
|
||||
"not done; '0 dropped frames at 4K' is.",
|
||||
),
|
||||
TemplateSection(
|
||||
key: 'depends',
|
||||
label: 'depends',
|
||||
hint: 'prerequisite tasks/notes',
|
||||
helper: 'Prerequisite tasks/notes. Leave blank if none.',
|
||||
),
|
||||
TemplateSection(
|
||||
key: 'estimate',
|
||||
label: 'estimate',
|
||||
hint: 'S | M | L',
|
||||
helper: 'Rough size: S / M / L.',
|
||||
key: 'verify',
|
||||
label: 'verify',
|
||||
hint: 'phone / desktop, and the exact command',
|
||||
helper:
|
||||
'Where and how this gets checked: which device, the exact '
|
||||
'command, the real deploy path. Blank means desktop, any way '
|
||||
'you like.',
|
||||
),
|
||||
TemplateSection(
|
||||
key: 'refs',
|
||||
label: 'refs',
|
||||
label: 'read first',
|
||||
inline: false,
|
||||
hint: 'links/docs the agent should read',
|
||||
hint: 'links/docs/code/screenshots to read before starting',
|
||||
helper:
|
||||
'Links/docs/code the agent should read first, one per line. '
|
||||
'Leave blank if none.',
|
||||
'Links, docs, code and reference screenshots the agent must '
|
||||
'read before starting, one per line. Say whether a screenshot '
|
||||
'is the format to copy or the content. Leave blank if none.',
|
||||
),
|
||||
],
|
||||
);
|
||||
|
||||
/// Section labels this template used to have and no longer does.
|
||||
///
|
||||
/// An unknown `## heading` is normally treated as *content* — the user may
|
||||
/// write subheadings inside a value. A retired label is different: it is a
|
||||
/// section the template once owned, so folding it into the preceding
|
||||
/// section's value and calling the note conforming would let the stepper
|
||||
/// silently swallow it on the next save. Seeing one means "this note was
|
||||
/// written against an older template" → non-conforming → raw editor.
|
||||
static const Set<String> retiredLabels = {
|
||||
'tech',
|
||||
'ask',
|
||||
'nice',
|
||||
'never',
|
||||
'out',
|
||||
'depends',
|
||||
'estimate',
|
||||
'refs',
|
||||
};
|
||||
|
||||
/// An empty, structure-free template.
|
||||
static const NoteTemplate blank = NoteTemplate(
|
||||
id: 'blank',
|
||||
@ -279,6 +271,11 @@ ParsedNote parse(NoteTemplate template, String text) {
|
||||
blocks[heading.key] = <String>[];
|
||||
continue;
|
||||
}
|
||||
// A heading the template retired means the note predates the current
|
||||
// section set. Keep the line as content (we never drop one) but refuse to
|
||||
// call the note conforming, so the UI shows it raw instead of letting the
|
||||
// stepper fold the retired section into its neighbour on the next save.
|
||||
if (_isRetiredHeading(line)) conforms = false;
|
||||
if (currentKey != null) {
|
||||
blocks[currentKey]!.add(line);
|
||||
continue;
|
||||
@ -327,6 +324,12 @@ TemplateSection? _matchHeading(String line, NoteTemplate template) {
|
||||
return null;
|
||||
}
|
||||
|
||||
/// Whether [line] is a `## heading` naming a section the template retired.
|
||||
bool _isRetiredHeading(String line) {
|
||||
if (!line.startsWith('## ')) return false;
|
||||
return NoteTemplate.retiredLabels.contains(line.substring(3).trim());
|
||||
}
|
||||
|
||||
/// Drops the leading blank lines and the single italic guidance line (if any)
|
||||
/// from a section's block, leaving just the user's value.
|
||||
String _stripGuidance(List<String> lines) {
|
||||
|
||||
@ -49,9 +49,9 @@ void _installFrameTimings() {
|
||||
final build = t.buildDuration.inMicroseconds;
|
||||
final raster = t.rasterDuration.inMicroseconds;
|
||||
if (build + raster > _slowFrameUs) {
|
||||
final total = ((build + raster) / 1000).toStringAsFixed(1);
|
||||
stdout.writeln(
|
||||
'[FRAME $_at s] total=${((build + raster) / 1000)
|
||||
.toStringAsFixed(1)}ms '
|
||||
'[FRAME $_at s] total=${total}ms '
|
||||
'(build=${(build / 1000).toStringAsFixed(1)} '
|
||||
'raster=${(raster / 1000).toStringAsFixed(1)})',
|
||||
);
|
||||
|
||||
@ -40,8 +40,8 @@ void installFrameStats() {
|
||||
|
||||
SchedulerBinding.instance.addTimingsCallback((timings) {
|
||||
for (final t in timings) {
|
||||
final total = t.buildDuration.inMilliseconds +
|
||||
t.rasterDuration.inMilliseconds;
|
||||
final total =
|
||||
t.buildDuration.inMilliseconds + t.rasterDuration.inMilliseconds;
|
||||
if (total > _slowFrameMs) {
|
||||
debugPrint(
|
||||
'[SLOW-FRAME $_at s] total=${total}ms '
|
||||
|
||||
@ -8,8 +8,8 @@ import 'package:share_plus/share_plus.dart';
|
||||
/// Overridable **so the test suite never writes to the real `~/todo`**. This is
|
||||
/// not hypothetical: before this seam existed, running `flutter test` exported
|
||||
/// a fake note over the user's canonical `~/todo/BACKLOG.md` every time.
|
||||
String Function() resolveExportHome =
|
||||
() => Platform.environment['HOME'] ?? Directory.current.path;
|
||||
String Function() resolveExportHome = () =>
|
||||
Platform.environment['HOME'] ?? Directory.current.path;
|
||||
|
||||
/// Writes the Markdown export somewhere the user can get at it, and returns a
|
||||
/// human-readable description of where it went.
|
||||
|
||||
@ -73,6 +73,34 @@ void main() {
|
||||
expect(find.byType(LinearProgressIndicator), findsOneWidget);
|
||||
});
|
||||
|
||||
testWidgets('guided: the stepper walks the seven audited sections', (
|
||||
tester,
|
||||
) async {
|
||||
// The user-visible contract of the 2026-07 template cut. Walks every step
|
||||
// and asserts the label and its guidance are the ones on screen, so a
|
||||
// section resurrected in note_template.dart cannot land unnoticed.
|
||||
await pumpEditor(tester, initialTemplate: spec);
|
||||
|
||||
for (var i = 0; i < spec.sections.length; i++) {
|
||||
final section = spec.sections[i];
|
||||
expect(find.text('${i + 1} / ${spec.sections.length}'), findsOneWidget);
|
||||
expect(
|
||||
find.text(section.isTitle ? 'title' : section.label),
|
||||
findsWidgets,
|
||||
reason: 'step ${i + 1} should show "${section.label}"',
|
||||
);
|
||||
expect(find.text(section.helper), findsOneWidget);
|
||||
if (i < spec.sections.length - 1) {
|
||||
await tester.tap(find.text('Next'));
|
||||
await tester.pump();
|
||||
}
|
||||
}
|
||||
|
||||
// The last step is `read first`; there is no Next past it.
|
||||
expect(spec.sections.last.label, 'read first');
|
||||
expect(find.text('Next'), findsNothing);
|
||||
});
|
||||
|
||||
testWidgets('switching to View renders the note as Markdown', (tester) async {
|
||||
await pumpEditor(
|
||||
tester,
|
||||
|
||||
@ -261,7 +261,8 @@ void main() {
|
||||
expect(
|
||||
field.$2.wallTimeMs,
|
||||
edited.millisecondsSinceEpoch,
|
||||
reason: 'clock must come from updatedAt so a restore cannot outrank '
|
||||
reason:
|
||||
'clock must come from updatedAt so a restore cannot outrank '
|
||||
'genuinely newer data on another device',
|
||||
);
|
||||
});
|
||||
|
||||
@ -22,11 +22,23 @@ void main() {
|
||||
expect(text, contains('A theme toggle.'));
|
||||
expect(text, contains('## must'));
|
||||
// Dropped empty sections leave no heading behind.
|
||||
expect(text, isNot(contains('## nice')));
|
||||
expect(text, isNot(contains('## refs')));
|
||||
expect(text, isNot(contains('## verify')));
|
||||
expect(text, isNot(contains('## read first')));
|
||||
},
|
||||
);
|
||||
|
||||
test('the multi-word "read first" label round-trips as a heading', () {
|
||||
final text = assemble(spec, {
|
||||
'title': 'T',
|
||||
'refs': '- https://example.com/doc',
|
||||
});
|
||||
expect(text, contains('## read first'));
|
||||
|
||||
final parsed = parse(spec, text);
|
||||
expect(parsed.conforms, isTrue);
|
||||
expect(parsed.values['refs'], '- https://example.com/doc');
|
||||
});
|
||||
|
||||
test('omits the title line when the title is blank', () {
|
||||
final text = assemble(spec, {'what': 'something'});
|
||||
// No `# ` title line; the body starts straight at the first section.
|
||||
@ -63,6 +75,32 @@ void main() {
|
||||
expect(parsed.conforms, isFalse);
|
||||
});
|
||||
|
||||
test('the pre-2026-07 twelve-section spec is non-conforming', () {
|
||||
// Notes written before `tech`/`ask`/`nice`/`never`/`depends`/`estimate`
|
||||
// were cut must fall back to the raw editor rather than be silently
|
||||
// stripped of the sections the template no longer knows about.
|
||||
const legacy =
|
||||
'# Old note\n\n## what\na thing\n\n## tech\nFlutter 3.32\n\n'
|
||||
'## must\n- do it\n\n## estimate\nM';
|
||||
final parsed = parse(spec, legacy);
|
||||
expect(parsed.conforms, isFalse);
|
||||
// The dropped sections are not resurrected as values either.
|
||||
expect(parsed.values.containsKey('tech'), isFalse);
|
||||
expect(parsed.values.containsKey('estimate'), isFalse);
|
||||
});
|
||||
|
||||
test('the section set is the audited seven', () {
|
||||
expect(spec.sections.map((s) => s.key), [
|
||||
'title',
|
||||
'what',
|
||||
'where',
|
||||
'must',
|
||||
'done',
|
||||
'verify',
|
||||
'refs',
|
||||
]);
|
||||
});
|
||||
|
||||
test('out-of-order / duplicate headings are non-conforming', () {
|
||||
const text = '# T\n\n## done\nx\n\n## what\ny';
|
||||
expect(parse(spec, text).conforms, isFalse);
|
||||
|
||||
@ -292,8 +292,8 @@ void main() {
|
||||
final home = Directory.systemTemp.createTempSync('todo_export_home');
|
||||
resolveExportHome = () => home.path;
|
||||
addTearDown(() {
|
||||
resolveExportHome =
|
||||
() => Platform.environment['HOME'] ?? Directory.current.path;
|
||||
resolveExportHome = () =>
|
||||
Platform.environment['HOME'] ?? Directory.current.path;
|
||||
home.deleteSync(recursive: true);
|
||||
});
|
||||
|
||||
|
||||
@ -135,8 +135,7 @@ void main() {
|
||||
});
|
||||
|
||||
test('contentTypeFor covers the asset kinds the build emits', () {
|
||||
String kind(String name) =>
|
||||
WrapperServer.contentTypeFor(name).mimeType;
|
||||
String kind(String name) => WrapperServer.contentTypeFor(name).mimeType;
|
||||
|
||||
expect(kind('a.html'), 'text/html');
|
||||
expect(kind('a.json'), 'application/json');
|
||||
|
||||
248
tool/migrate_backlog.dart
Normal file
248
tool/migrate_backlog.dart
Normal file
@ -0,0 +1,248 @@
|
||||
// One-shot migration of exported notes from the pre-2026-07 twelve-section
|
||||
// `llm-design-spec` template to the audited seven-section one.
|
||||
//
|
||||
// Why a file transform and not a database migration: the desktop app is the
|
||||
// web build, so the notes live in IndexedDB inside the fixed Chrome profile —
|
||||
// a CLI tool cannot reach them. The supported path is
|
||||
// Settings → Export → run this → Settings → Import, which routes through
|
||||
// [NoteRepository.importNotes] and merges by id.
|
||||
//
|
||||
// Folding rules (nothing is dropped silently — everything dropped is reported):
|
||||
// tech -> appended to `where`
|
||||
// never / out -> `must`, as `must not: …` lines (a literal "none" is skipped)
|
||||
// nice -> `must`, as `optional: …` lines
|
||||
// ask / depends / estimate -> dropped, listed per note in the report
|
||||
//
|
||||
// Usage:
|
||||
// dart run tool/migrate_backlog.dart BACKLOG.md # dry run
|
||||
// dart run tool/migrate_backlog.dart BACKLOG.md --apply -o out.md
|
||||
import 'dart:io';
|
||||
|
||||
import 'package:todo/data/note.dart';
|
||||
import 'package:todo/data/note_template.dart';
|
||||
import 'package:todo/sync/notes_markdown.dart';
|
||||
|
||||
/// Matches a `## label` heading, capturing the label.
|
||||
final _heading = RegExp(r'^## (.+)$');
|
||||
|
||||
/// Splits a legacy note body into `title` plus one entry per `## section`.
|
||||
///
|
||||
/// Deliberately independent of [parse]: that function is bound to the *current*
|
||||
/// template, and the twelve-section one it replaced no longer exists in `lib/`.
|
||||
Map<String, String> _parseLegacy(String text) {
|
||||
final values = <String, String>{};
|
||||
final titleLines = <String>[];
|
||||
String? current;
|
||||
final blocks = <String, List<String>>{};
|
||||
|
||||
for (final line in text.split('\n')) {
|
||||
final match = _heading.firstMatch(line);
|
||||
if (match != null) {
|
||||
current = match.group(1)!.trim();
|
||||
blocks.putIfAbsent(current, () => <String>[]);
|
||||
continue;
|
||||
}
|
||||
if (current != null) {
|
||||
blocks[current]!.add(line);
|
||||
continue;
|
||||
}
|
||||
final trimmed = line.trim();
|
||||
if (trimmed.startsWith('# ') && !trimmed.startsWith('## ')) {
|
||||
titleLines.add(trimmed.substring(2).trim());
|
||||
}
|
||||
}
|
||||
|
||||
values['title'] = titleLines.join('\n').trim();
|
||||
for (final entry in blocks.entries) {
|
||||
values[entry.key] = _stripGuidance(entry.value);
|
||||
}
|
||||
return values;
|
||||
}
|
||||
|
||||
/// Drops leading blanks and the italic guidance line from a section block.
|
||||
String _stripGuidance(List<String> lines) {
|
||||
final out = List<String>.from(lines);
|
||||
var i = 0;
|
||||
while (i < out.length && out[i].trim().isEmpty) {
|
||||
i++;
|
||||
}
|
||||
if (i < out.length) {
|
||||
final t = out[i].trim();
|
||||
if (t.length >= 2 && t.startsWith('_') && t.endsWith('_')) out.removeAt(i);
|
||||
}
|
||||
return out.join('\n').trim();
|
||||
}
|
||||
|
||||
/// Folds a retired section's [value] into `must` as `- <prefix>: …` lines.
|
||||
///
|
||||
/// Only a section whose every non-empty line is already a bullet becomes one
|
||||
/// bullet per line. Anything else is prose — several old `nice`/`never`
|
||||
/// sections hold pasted terminal output — and is kept as a single item with
|
||||
/// its block indented underneath, rather than being shredded into one bogus
|
||||
/// bullet per line.
|
||||
///
|
||||
/// A placeholder "none" yields nothing: five old notes wrote it instead of
|
||||
/// leaving the section blank.
|
||||
List<String> _foldInto(String prefix, String value) {
|
||||
final lines = value.split('\n');
|
||||
final nonEmpty = lines.where((l) => l.trim().isNotEmpty).toList();
|
||||
if (nonEmpty.isEmpty) return const [];
|
||||
if (nonEmpty.length == 1 && nonEmpty.first.trim().toLowerCase() == 'none') {
|
||||
return const [];
|
||||
}
|
||||
|
||||
final bulleted = RegExp(r'^\s*[-*]\s+');
|
||||
if (nonEmpty.every(bulleted.hasMatch)) {
|
||||
return nonEmpty
|
||||
.map((l) => '- $prefix: ${l.replaceFirst(bulleted, '').trim()}')
|
||||
.toList();
|
||||
}
|
||||
return ['- $prefix:', ...lines.map((l) => l.isEmpty ? l : ' $l')];
|
||||
}
|
||||
|
||||
/// What happened to one note, for the report.
|
||||
class _Outcome {
|
||||
_Outcome(this.note, {required this.folded, required this.dropped});
|
||||
|
||||
final Note note;
|
||||
final List<String> folded;
|
||||
final List<String> dropped;
|
||||
}
|
||||
|
||||
/// Rewrites [note] under the current template, or returns null if its body has
|
||||
/// no retired sections (freeform notes and already-migrated ones).
|
||||
_Outcome? _migrate(Note note, DateTime stamp) {
|
||||
final old = _parseLegacy(note.text);
|
||||
final retired = old.keys.where(NoteTemplate.retiredLabels.contains).toList();
|
||||
if (retired.isEmpty) return null;
|
||||
|
||||
final folded = <String>[];
|
||||
final dropped = <String>[];
|
||||
|
||||
final where = <String>[
|
||||
if ((old['where'] ?? '').isNotEmpty) old['where']!,
|
||||
];
|
||||
if ((old['tech'] ?? '').isNotEmpty) {
|
||||
where.add(old['tech']!);
|
||||
folded.add('tech -> where');
|
||||
}
|
||||
|
||||
final must = <String>[
|
||||
if ((old['must'] ?? '').trim().isNotEmpty) old['must']!.trim(),
|
||||
];
|
||||
for (final entry in [
|
||||
('never', 'must not'),
|
||||
('out', 'must not'),
|
||||
('nice', 'optional'),
|
||||
]) {
|
||||
final folded_ = _foldInto(entry.$2, old[entry.$1] ?? '');
|
||||
if (folded_.isEmpty) continue;
|
||||
must.addAll(folded_);
|
||||
folded.add('${entry.$1} -> must');
|
||||
}
|
||||
|
||||
for (final key in ['ask', 'depends', 'estimate']) {
|
||||
if ((old[key] ?? '').isNotEmpty) dropped.add(key);
|
||||
}
|
||||
|
||||
final text = assemble(NoteTemplate.llmDesignSpec, {
|
||||
'title': old['title'] ?? '',
|
||||
'what': old['what'] ?? '',
|
||||
'where': where.join('\n'),
|
||||
'must': must.join('\n'),
|
||||
'done': old['done'] ?? '',
|
||||
// `verify` is new: nothing in a legacy note maps to it.
|
||||
'verify': '',
|
||||
'refs': old['refs'] ?? '',
|
||||
});
|
||||
|
||||
// Gate, not a warning: a note that does not conform would open in the raw
|
||||
// editor, which is exactly what the migration exists to avoid.
|
||||
if (!parse(NoteTemplate.llmDesignSpec, text).conforms) {
|
||||
throw StateError('migrated note ${note.id} does not conform');
|
||||
}
|
||||
// Nothing may vanish except the sections we deliberately drop. Compare the
|
||||
// surviving source lines against the result.
|
||||
final kept = {...old.keys}..removeAll(['ask', 'depends', 'estimate']);
|
||||
for (final key in kept) {
|
||||
for (final line in (old[key] ?? '').split('\n')) {
|
||||
final needle = line.trim().replaceFirst(RegExp(r'^[-*]\s*'), '').trim();
|
||||
if (needle.isEmpty || needle.toLowerCase() == 'none') continue;
|
||||
if (!text.contains(needle)) {
|
||||
throw StateError('note ${note.id}: lost a line from "$key": $needle');
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return _Outcome(
|
||||
// The bumped timestamp is what makes the import win. `importNotes` stamps
|
||||
// the CRDT record at the note's own edit time precisely so a re-import does
|
||||
// not beat other devices just by being recent — so without this the phone
|
||||
// would silently keep the old text.
|
||||
note.copyWith(text: text, updatedAt: stamp),
|
||||
folded: folded,
|
||||
dropped: dropped,
|
||||
);
|
||||
}
|
||||
|
||||
void main(List<String> args) {
|
||||
final apply = args.contains('--apply');
|
||||
final positional = args.where((a) => !a.startsWith('-')).toList();
|
||||
if (positional.isEmpty) {
|
||||
stderr.writeln(
|
||||
'usage: dart run tool/migrate_backlog.dart <BACKLOG.md> '
|
||||
'[--apply] [-o <out.md>]',
|
||||
);
|
||||
exitCode = 2;
|
||||
return;
|
||||
}
|
||||
|
||||
final input = File(positional.first);
|
||||
if (!input.existsSync()) {
|
||||
stderr.writeln('error: ${input.path} does not exist');
|
||||
exitCode = 2;
|
||||
return;
|
||||
}
|
||||
|
||||
final oi = args.indexOf('-o');
|
||||
final outPath = oi >= 0 && oi + 1 < args.length
|
||||
? args[oi + 1]
|
||||
: '${input.path}.migrated.md';
|
||||
|
||||
final notes = NotesMarkdown.parse(input.readAsStringSync());
|
||||
final stamp = DateTime.now();
|
||||
|
||||
final result = <Note>[];
|
||||
final outcomes = <_Outcome>[];
|
||||
for (final note in notes) {
|
||||
final outcome = _migrate(note, stamp);
|
||||
if (outcome == null) {
|
||||
result.add(note); // untouched: freeform, or already on the new template
|
||||
continue;
|
||||
}
|
||||
result.add(outcome.note);
|
||||
outcomes.add(outcome);
|
||||
}
|
||||
|
||||
stdout
|
||||
..writeln('${notes.length} notes read from ${input.path}')
|
||||
..writeln('${outcomes.length} carry retired sections and were rewritten')
|
||||
..writeln('${notes.length - outcomes.length} left untouched\n');
|
||||
for (final o in outcomes) {
|
||||
final id = o.note.id.substring(0, 8);
|
||||
stdout.writeln(' $id ${noteTitle(o.note.text)}');
|
||||
if (o.folded.isNotEmpty) {
|
||||
stdout.writeln(' folded : ${o.folded.join(', ')}');
|
||||
}
|
||||
if (o.dropped.isNotEmpty) {
|
||||
stdout.writeln(' DROPPED: ${o.dropped.join(', ')}');
|
||||
}
|
||||
}
|
||||
|
||||
if (!apply) {
|
||||
stdout.writeln('\ndry run — nothing written. Re-run with --apply.');
|
||||
return;
|
||||
}
|
||||
File(outPath).writeAsStringSync(NotesMarkdown.export(result));
|
||||
stdout.writeln('\nwrote $outPath — import it via Settings -> Import notes.');
|
||||
}
|
||||
Loading…
Reference in New Issue
Block a user