utils/dep_freshness/quarantine.py
Krzysztof kuhy Rudnicki 1439c0f5c0 Make the allowlist inheritable, and cache the quarantine lookups
Two things that were fine in utils and would not have been in the next forty
repos.

**The allowlist is now fleet-wide plus per-repo.** `typescript-eslint` blocks
TS 7 in EVERY repo that pins typescript -- eight of them and counting -- and a
repo-only allowlist meant hand-copying one entry into eight files that then rot
independently. That is the four-way fork of the 250-line cap, reproduced inside
the mechanism built to avoid it. `~/utils/dependency-freshness.allowlist.yaml`
is inherited by every repo; a repo entry for the same `ecosystem:package`
overrides it.

Inherited entries are exempt from the no-longer-needed check everywhere except
the repo that owns the file. Without that carve-out a fleet-wide hold on
typescript exits 2 in every Flutter app -- it excuses nothing there, which is
the normal case, not rot -- and those repos become uncommittable. Confirmed
against punchme, which now exits 0 while utils still reports the entry as in
use.

**`quarantine.py` bypassed the cache entirely.** It ran for every npm finding
and fetched the FULL packument -- 7MB for react, 16MB for typescript -- with no
TTL and no reuse, inside a pre-commit hook. One finding in utils, but the JS
repos ahead have 16 to 37 each. Now cached under the normal 6h TTL, keyed
`<name>@<dist-tag>`: the tag has to be part of the key rather than clamped
afterwards, because clamping returns the ceiling and the ceiling is frequently
the very release the quarantine is excluding.

**Both cache and shared allowlist are now redirected by AUTOUSE fixtures.** A
quarantine test had already seeded a fake answer for package "x" into the real
~/.cache/dep-freshness/registry.json, and the next run read its own pollution
back and passed for the wrong reason. Removed the entry; the fixture makes it
unrepeatable.

292 tests, 100% branch coverage on every module.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CRykabMwPgDzFHcARoqpaK
2026-08-28 13:58:44 +02:00

102 lines
4.0 KiB
Python

"""What is the newest npm version the package manager will actually install?
pnpm 11 refuses any package published inside its `minimumReleaseAge` window --
a supply-chain measure that blunts the period during which a compromised
publish is live. Reporting a repo as "behind" a version pnpm would reject is a
finding nobody can act on, so the gate's notion of latest is narrowed to match.
Cost is the reason this is a separate pass rather than part of `npmjs.latest`.
Publish timestamps live only in the FULL registry document, which is 7MB for
react and 16MB for typescript, against 3MB and 9MB abbreviated. So the full
document is fetched only when a finding already exists -- in the steady state,
where everything is current, that is zero extra requests.
"""
from __future__ import annotations
from datetime import datetime, timedelta, timezone
from dep_freshness._tables import NPM_API, NPM_QUARANTINE_HOURS
from dep_freshness.cache import Cache
from dep_freshness.registries.http import Offline, get_json
from dep_freshness.versions import newest_stable, parse
# Its own cache namespace: the answer here is narrower than `npm/<name>` and
# the two must not overwrite each other.
CACHE_ECOSYSTEM = "npm-installable"
def cutoff(now: datetime | None = None) -> datetime:
"""Publishes at or after this instant are still quarantined."""
moment = now or datetime.now(timezone.utc)
return moment - timedelta(hours=NPM_QUARANTINE_HOURS)
def _parse(stamp: str) -> datetime | None:
try:
return datetime.fromisoformat(str(stamp).replace("Z", "+00:00"))
except (TypeError, ValueError):
return None
def installable_latest(
name: str,
ceiling: str,
now: datetime | None = None,
cache: Cache | None = None,
) -> str | None:
"""Newest stable release published before the quarantine cutoff.
`ceiling` is the registry's `dist-tags.latest`, and candidates above it are
ignored. npm lets a maintainer publish a version without tagging it
`latest` -- a maintenance-line release on an older major, say -- so the
highest version NUMBER is not always the current release. Falling back to
the numeric maximum here would recommend a version npm does not consider
current, and would disagree with `npmjs.latest`, which honours the tag.
Returns None when the answer cannot be determined (offline, or the
registry has no `time` map), which callers must treat as "no change" --
narrowing on a guess would hide a genuinely stale dependency.
"""
# Cached under the normal TTL. Without this every gate run in a JS-heavy
# repo re-downloads tens of full packuments, inside a pre-commit hook --
# the exact cost the 6h cache exists to prevent.
#
# The ceiling is part of the KEY, not applied afterwards: clamping a cached
# answer to the ceiling would return the ceiling itself, and the ceiling is
# frequently the very release the quarantine is excluding. `dist-tags.latest`
# is stable within a TTL, so hits still land across manifests.
store = cache if cache is not None else Cache()
key = f"{name}@{ceiling}"
entry = store.get(CACHE_ECOSYSTEM, key)
if entry is not None and entry.fresh(CACHE_ECOSYSTEM):
return entry.version
url = NPM_API.format(name=name.replace("/", "%2f"))
try:
payload = get_json(url)
except Offline:
return entry.version if entry is not None else None # stale beats none
if not payload:
return None
times = payload.get("time")
if not isinstance(times, dict):
return None
limit = cutoff(now)
top = parse(ceiling)
old_enough = []
for version, stamp in times.items():
if version in ("created", "modified"):
continue
when = _parse(stamp)
if when is None or when >= limit:
continue
here = parse(version)
if top is not None and (here is None or here > top):
continue
old_enough.append(version)
answer = newest_stable(old_enough)
store.put(CACHE_ECOSYSTEM, key, answer)
store.save()
return answer