mirror of
https://github.com/kuhyx/testsAndMisc.git
synced 2026-11-10 21:07:03 +01:00
Krzysztof kuhy Rudnicki
c73cfd3573
The first run of the split workflow restored the pylint hook env from the old monolithic cache, yet still spent 30 s on apt for pygame's SDL headers, because the step was gated on an exact cache-key hit. That made the pylint group 71 s. Now the env build is attempted first (pre-commit installs the env, then has no files to lint) and the headers are installed and the build retried only when it actually fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KFwA9nymhUdTrVF65kigTV
224 lines
11 KiB
YAML
224 lines
11 KiB
YAML
name: Pre-commit checks
|
|
|
|
on:
|
|
push:
|
|
branches: [main]
|
|
pull_request:
|
|
branches: [main]
|
|
# Manual runs: this job's cost is dominated by whether the pre-commit hook
|
|
# environments are cached, and without a dispatch there is no way to
|
|
# re-run it on an unchanged tree to confirm a cache actually helps.
|
|
workflow_dispatch: {}
|
|
|
|
jobs:
|
|
# mypy (~24 s), shellcheck (~24 s) and pylint (~18 s) each walk the whole
|
|
# repo and made up most of a single ~82 s hook step; every other hook
|
|
# together is ~15 s. Each heavy hook gets its own runner and the rest share a
|
|
# fourth, so the workflow takes as long as the slowest group rather than the
|
|
# sum. The four groups together run exactly the hook set the single job ran.
|
|
pre-commit:
|
|
runs-on: ubuntu-latest
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
include:
|
|
- group: mypy
|
|
hooks: mypy
|
|
- group: pylint
|
|
hooks: pylint
|
|
- group: shellcheck
|
|
hooks: shellcheck
|
|
- group: rest
|
|
hooks: ""
|
|
name: pre-commit (${{ matrix.group }})
|
|
env:
|
|
# The md-naming / file-length hooks are thin shims that exec the shared
|
|
# gate out of the utils repo. A runner has no ~/src/utils, so without this
|
|
# checkout they fail with "shared gate not found" -- which is exactly
|
|
# how this workflow went red on 2026-08-24.
|
|
UTILS_ROOT: ${{ github.workspace }}/.utils
|
|
# SKIP=pytest-coverage: the Python tests workflow already runs the
|
|
# whole suite with the same 100% coverage gate, so running it again
|
|
# here doubled the cost of this job for no extra signal. It is the
|
|
# only hook skipped, and it is skipped because it is DUPLICATED, not
|
|
# because it is slow -- if that workflow is ever removed, drop this.
|
|
# `rest` also skips the three hooks that run in their own group above.
|
|
SKIP: ${{ matrix.group == 'rest' && 'pytest-coverage,mypy,pylint,shellcheck' || 'pytest-coverage' }}
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
- name: Check out the shared gates
|
|
uses: actions/checkout@v4
|
|
with:
|
|
repository: kuhyx/utils
|
|
path: .utils
|
|
|
|
# No `cache: pip` any more: it restored a 367 MB pip cache for the full
|
|
# requirements.txt, which this job no longer installs.
|
|
- uses: actions/setup-python@v5
|
|
id: setup-python
|
|
with:
|
|
python-version: "3.x"
|
|
- uses: astral-sh/setup-uv@v10.2.0
|
|
|
|
# Every hook brings its own environment except the `language: system`
|
|
# ones, and of those only dependency-freshness imports third-party code
|
|
# (pyyaml, packaging); check_requirements_sync.py, the evidence/contract
|
|
# validators and the session_autopsy mutation hook are stdlib-only. The
|
|
# full requirements.txt (~90 packages, a pygame source build included)
|
|
# used to be installed here for nothing: 49 s of every run.
|
|
- name: Install pre-commit and the system hooks' imports
|
|
run: uv pip install --system pre-commit pyyaml packaging
|
|
|
|
# pre-commit builds a virtualenv per hook repo on first use, which is
|
|
# slower than the pip install itself. Keyed on the hook definitions, so
|
|
# it invalidates exactly when a hook's repo or rev changes.
|
|
# The interpreter goes in the key, resolved rather than typed. A
|
|
# `language: python` hook env is built by whichever Python pre-commit ran
|
|
# under, and restoring one built by a different release is silent: the
|
|
# hooks keep working right up to the first syntax the old interpreter
|
|
# cannot parse. This key said `py3.12` as a literal while the job had
|
|
# moved to `3.x`, so 3.14-only source (PEP 758's `except A, B:`) reached
|
|
# pylint and the debug-statement hook running on a cached 3.12.14 and
|
|
# came back as "multiple exception types must be parenthesized" -- a
|
|
# failure with no local reproduction, since developer caches were built
|
|
# on 3.14. `steps.setup-python.outputs.python-version` cannot drift from
|
|
# what was actually installed.
|
|
#
|
|
# One cache per group: a shared key would hold whichever group saved
|
|
# first, and the others would rebuild their hook envs on every run. The
|
|
# restore-keys fall back to any env set built by the SAME interpreter
|
|
# (this group's older config first, then any group's), so a config
|
|
# change rebuilds only the hooks whose rev moved.
|
|
- uses: actions/cache@v4
|
|
id: cache
|
|
with:
|
|
path: ~/.cache/pre-commit
|
|
key: >-
|
|
pre-commit-${{ runner.os }}-py${{ steps.setup-python.outputs.python-version }}-${{ matrix.group }}-${{ hashFiles('.pre-commit-config.yaml') }}
|
|
restore-keys: |
|
|
pre-commit-${{ runner.os }}-py${{ steps.setup-python.outputs.python-version }}-${{ matrix.group }}-
|
|
pre-commit-${{ runner.os }}-py${{ steps.setup-python.outputs.python-version }}-
|
|
|
|
# zsh backs the zsh-syntax hook, which only the rest group runs. apt
|
|
# does not let a patch release be selected, so this is pinned only to
|
|
# what the runner image provides; the version is echoed so a jump shows
|
|
# up in the log rather than as a mystery parse difference. Local
|
|
# machines compare on major.minor only.
|
|
#
|
|
# The runner image ships apt lists, so the install is tried against
|
|
# them first; `apt-get update` (~14 s) runs only if that fails.
|
|
- name: Install system dependencies (zsh)
|
|
if: matrix.group == 'rest'
|
|
run: |
|
|
set -euo pipefail
|
|
pkgs=(zsh)
|
|
sudo apt-get install -y "${pkgs[@]}" ||
|
|
{ sudo apt-get update && sudo apt-get install -y "${pkgs[@]}"; }
|
|
zsh --version
|
|
|
|
# The SDL headers are for pygame, one of pylint's additional_dependencies.
|
|
# 2.6.1 is the newest release and its wheels stop at cp313, so on
|
|
# newest-stable Python pip falls back to a source build -- which dies on
|
|
# a missing `sdl-config` unless these are here. Only a pylint env BUILD
|
|
# needs them (30 s of apt), and a restored cache usually already holds
|
|
# the env, so the build is tried first and the headers are installed
|
|
# only if it fails. Run against a non-Python file: pre-commit installs
|
|
# the hook env before filtering files, then has nothing to lint, so the
|
|
# only way this exits non-zero is the env build itself failing.
|
|
- name: Build the pylint hook env (pygame headers only if needed)
|
|
if: matrix.group == 'pylint'
|
|
run: |
|
|
set -euo pipefail
|
|
if pre-commit run pylint --files .pre-commit-config.yaml; then
|
|
exit 0
|
|
fi
|
|
echo "pylint env build failed -- installing pygame's build headers and retrying"
|
|
pkgs=(libsdl2-dev libsdl2-image-dev libsdl2-mixer-dev libsdl2-ttf-dev
|
|
libfreetype6-dev libportmidi-dev)
|
|
sudo apt-get install -y "${pkgs[@]}" ||
|
|
{ sudo apt-get update && sudo apt-get install -y "${pkgs[@]}"; }
|
|
pre-commit run pylint --files .pre-commit-config.yaml
|
|
|
|
# Pin shellcheck instead of inheriting whatever ubuntu-latest ships.
|
|
#
|
|
# The runner image carried 0.9.0 while developer machines have 0.11.0
|
|
# from pacman, and the shellcheck hook is `language: system` -- so it
|
|
# used whichever binary happened to be on PATH. The two versions
|
|
# disagree (0.11.0 demoted SC2002 and several SC2015/SC2119 cases as
|
|
# noise), which meant a commit could pass every local check and still
|
|
# fail CI on findings the developer could not reproduce. That is exactly
|
|
# what kept this workflow red from b83ce5d onward.
|
|
#
|
|
# Keep SHELLCHECK_VERSION in step with the version developers install;
|
|
# the verification below fails loudly rather than silently drifting.
|
|
- name: Install shellcheck (pinned)
|
|
if: matrix.group == 'shellcheck'
|
|
env:
|
|
SHELLCHECK_VERSION: v0.11.0
|
|
SHELLCHECK_SHA256: 8c3be12b05d5c177a04c29e3c78ce89ac86f1595681cab149b65b97c4e227198
|
|
run: |
|
|
set -euo pipefail
|
|
tarball="shellcheck-${SHELLCHECK_VERSION}.linux.x86_64.tar.xz"
|
|
curl -fsSL -o "$tarball" \
|
|
"https://github.com/koalaman/shellcheck/releases/download/${SHELLCHECK_VERSION}/${tarball}"
|
|
echo "${SHELLCHECK_SHA256} ${tarball}" | sha256sum --check --strict
|
|
tar -xJf "$tarball"
|
|
sudo install -m 0755 "shellcheck-${SHELLCHECK_VERSION}/shellcheck" /usr/local/bin/shellcheck
|
|
# Fail here, not 40 hooks later, if PATH still resolves to another copy.
|
|
installed="$(shellcheck --version | awk '/^version:/ { print $2 }')"
|
|
expected="${SHELLCHECK_VERSION#v}"
|
|
if [ "$installed" != "$expected" ]; then
|
|
echo "::error::shellcheck on PATH is ${installed}, expected ${expected}" >&2
|
|
command -v -a shellcheck >&2
|
|
exit 1
|
|
fi
|
|
echo "shellcheck ${installed} pinned and on PATH"
|
|
|
|
# prettier and shfmt do NOT run in this job -- prettier is a pre-push
|
|
# hook and shfmt runs from the local pre-commit git hook. They are
|
|
# pinned here anyway so this file stays the single place that records
|
|
# the expected version of every `language: system` tool, and so a future
|
|
# workflow that does run them inherits the pin instead of re-deriving
|
|
# it. Their real divergence risk is between developers: both REWRITE
|
|
# files, so two versions reformat the repo back and forth. Installed in
|
|
# one group only; the other three would be paying for it twice over.
|
|
- name: Install prettier and shfmt (pinned)
|
|
if: matrix.group == 'rest'
|
|
env:
|
|
PRETTIER_VERSION: 3.9.5
|
|
SHFMT_VERSION: v3.13.1
|
|
SHFMT_SHA256: fb096c5d1ac6beabbdbaa2874d025badb03ee07929f0c9ff67563ce8c75398b1
|
|
run: |
|
|
set -euo pipefail
|
|
npm install --global --no-fund --no-audit "prettier@${PRETTIER_VERSION}"
|
|
|
|
curl -fsSL -o shfmt \
|
|
"https://github.com/mvdan/sh/releases/download/${SHFMT_VERSION}/shfmt_${SHFMT_VERSION}_linux_amd64"
|
|
echo "${SHFMT_SHA256} shfmt" | sha256sum --check --strict
|
|
sudo install -m 0755 shfmt /usr/local/bin/shfmt
|
|
|
|
# Assert what PATH actually resolves to, not what was just written:
|
|
# a copy earlier on PATH would otherwise win silently.
|
|
prettier_installed="$(prettier --version)"
|
|
shfmt_installed="$(shfmt --version)"
|
|
if [ "$prettier_installed" != "$PRETTIER_VERSION" ]; then
|
|
echo "::error::prettier on PATH is ${prettier_installed}, expected ${PRETTIER_VERSION}" >&2
|
|
exit 1
|
|
fi
|
|
if [ "$shfmt_installed" != "$SHFMT_VERSION" ]; then
|
|
echo "::error::shfmt on PATH is ${shfmt_installed}, expected ${SHFMT_VERSION}" >&2
|
|
exit 1
|
|
fi
|
|
echo "prettier ${prettier_installed} and shfmt ${shfmt_installed} pinned and on PATH"
|
|
|
|
- name: pre-commit run (${{ matrix.group }})
|
|
run: pre-commit run ${{ matrix.hooks }} --all-files --show-diff-on-failure
|
|
|
|
# A cache restored through restore-keys is saved again under this
|
|
# group's key, so without this the envs of every hook rev ever used
|
|
# would ride along forever. gc drops the ones the config no longer names.
|
|
- name: Prune stale hook envs before the cache is saved
|
|
if: steps.cache.outputs.cache-hit != 'true'
|
|
run: pre-commit gc
|