signal-bot/tool/venv_bootstrap.py
Krzysztof kuhy Rudnicki d3d89c890e feat: python tool/tagger.py sets up .venv too, not just tag.sh
The shell entry points got this in 99aa194; running the .py directly still
died with ModuleNotFoundError, which is the command the file's own docstring
had been telling people to use for months.

tool/venv_bootstrap.py creates .venv, installs the project and re-executes
the same command line through it. It is stdlib-only by necessity: it runs on
an interpreter where this package does not import yet, so it cannot route
through signal_bot.external, the one module allowed to start a process. That
is why it gets the second scoped per-file-ignore rather than the rule being
bent -- kuhy's call. test_external.py now asserts a closed list of two, so a
third file still fails the gate.

Two things that are less obvious than they look:

It runs on import rather than exposing a function to call. A call would have
to happen before the caller's own third-party imports, and code before
imports is E402 -- so a call-based design needs a lint suppression in every
file it protects. Importing the module is the whole interface.

And isort would sort "venv_bootstrap" after "fastapi", silently breaking the
ordering the module depends on. It gets its own isort section ahead of the
standard library, so the formatter enforces the order instead of undoing it.

Caught while verifying: ensure() used one check to decide both whether to
re-exec and whether to install. The interpreter you typed never grows a
fastapi, so every launch reinstalled the project first. Splitting them
(ready() probes the venv, missing() probes the caller) takes a warm run from
a full pip install to 0.47s. Pinned by a test.

Verified on a clone with no .venv: first run installs and prints one line,
second and third print nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBrE1b9HvFw3xvG9bJiDFt
2026-08-31 13:34:27 +02:00

156 lines
6.1 KiB
Python
Executable File

#!/usr/bin/env python3
# Copyright (c) 2026 Krzysztof Rudnicki. MIT licence.
"""Make a script work with whatever interpreter you started it with.
``python tool/tagger.py`` failed with ``ModuleNotFoundError: fastapi`` because
the dependencies live in ``.venv`` and nothing said so. So instead of reporting
that, this fixes it: if the running interpreter cannot import what the script
needs, it creates ``.venv`` if there is none, installs the project into it, and
re-executes the same command line through it.
``tag.sh`` and ``run.sh`` get there through ``scripts/ensure_venv.sh``. This
module is the same job for the case where somebody runs the ``.py`` directly,
which is what the file's own docstring used to tell them to do.
**Stdlib only, and it must stay that way.** It runs *before* the environment
holding the dependencies exists, so importing anything third-party here would
be the same bug one level down. That is also why it cannot use
``signal_bot.external``, the one module allowed to start a subprocess: it lives
inside the package being installed. Hence the second, scoped entry in
``per-file-ignores`` -- added deliberately, on kuhy's call of 2026-08-31, and
the reason it is a whole file for two subprocess calls is so that entry can
name something this small.
The sentinel environment variable stops a loop: if the re-executed interpreter
still cannot import what it needs, the second pass gives up and lets the real
ImportError through, so the error names the missing module rather than this.
"""
from __future__ import annotations
import importlib.util
import os
import shutil
import subprocess
import sys
from pathlib import Path
from typing import Final
REPO_ROOT: Final = Path(__file__).resolve().parent.parent
VENV_DIR: Final = REPO_ROOT / ".venv"
VENV_PYTHON: Final = VENV_DIR / "bin" / "python"
SENTINEL: Final = "SIGNAL_BOT_VENV_BOOTSTRAPPED"
# Enough to tell "wrong interpreter" from "project not installed". Not the
# whole dependency list -- pip is the authority on that, not this file.
REQUIRED: Final = ("fastapi", "uvicorn", "httpx", "signal_bot")
def missing(packages: tuple[str, ...] = REQUIRED) -> list[str]:
"""Which of ``packages`` this interpreter cannot import.
``find_spec`` rather than ``import``: it answers the same question without
paying for the module or running its side effects.
"""
absent = []
for name in packages:
try:
found = importlib.util.find_spec(name) is not None
except ImportError, ValueError:
found = False
if not found:
absent.append(name)
return absent
def ensure(packages: tuple[str, ...] = REQUIRED) -> None:
"""Re-execute through ``.venv`` if this interpreter is missing anything.
Does nothing in the normal case: an interpreter that can already import
everything is the right one, whether or not it is the venv's.
Args:
packages: Module names the caller is about to import.
"""
absent = missing(packages)
if not absent or os.environ.get(SENTINEL):
return
# Only build if the venv cannot already do the job. Testing the *current*
# interpreter is what decides to re-exec; testing the *venv's* is what
# decides to install. Conflating them reinstalls on every single launch,
# because the interpreter you typed never grows a fastapi.
if not ready(packages):
sys.stderr.write(f"== setting up .venv (missing: {', '.join(absent)})\n")
provision()
_reexec()
def ready(packages: tuple[str, ...] = REQUIRED) -> bool:
"""Whether the venv's interpreter can already import ``packages``."""
if not VENV_PYTHON.exists():
return False
return _run([str(VENV_PYTHON), "-c", f"import {', '.join(packages)}"], quiet=True)
def provision() -> None:
"""Create the virtualenv if it is not there, then install the project."""
if not VENV_PYTHON.exists() and not _run([*_venv_creator(), str(VENV_DIR)]):
_die(f"could not create a virtualenv at {VENV_DIR}")
if not _install():
_die("could not install the project into .venv")
def _venv_creator() -> list[str]:
"""The command that makes a virtualenv, preferring uv when it is here."""
uv = shutil.which("uv")
return [uv, "venv"] if uv is not None else [sys.executable, "-m", "venv"]
def _install() -> bool:
"""Install the project editable, so a later git pull reaches the tagger."""
uv = shutil.which("uv")
target = f"{REPO_ROOT}[dev]"
if uv is not None:
return _run(
[uv, "pip", "install", "-e", target], {**os.environ, "VIRTUAL_ENV": str(VENV_DIR)}
)
return _run([str(VENV_PYTHON), "-m", "pip", "install", "-e", target])
def _reexec() -> None:
"""Hand the same command line to the interpreter that can run it."""
if not VENV_PYTHON.exists():
_die(f"no interpreter at {VENV_PYTHON} after install")
os.execve(str(VENV_PYTHON), [str(VENV_PYTHON), *sys.argv], {**os.environ, SENTINEL: "1"})
def _run(command: list[str], env: dict[str, str] | None = None, *, quiet: bool = False) -> bool:
"""Run one setup command and say whether it worked.
Setup output is shown, because an install that takes a minute should look
like it is doing something. The probe is quiet: it fails on every normal
first run and its traceback would be the first thing anybody saw.
"""
output = subprocess.DEVNULL if quiet else None
try:
completed = subprocess.run(command, check=False, env=env, stdout=output, stderr=output)
except OSError:
return False
return completed.returncode == 0
def _die(message: str) -> None:
"""Stop with the command to run, not just what broke."""
sys.stderr.write(f"Error: {message}\n")
sys.stderr.write(f" Try: cd {REPO_ROOT} && uv venv .venv && uv pip install -e '.[dev]'\n")
raise SystemExit(1)
# Run on import, not from a call site. A caller that had to remember to call
# ensure() would have to do it *before* its own third-party imports, and code
# before imports is E402 -- so the fix would need a lint suppression in every
# file it protects. Importing this module is the whole interface: take a
# constant from it and .venv is guaranteed by the time the next import runs.
ensure()