mirror of
https://github.com/kuhyx/utils.git
synced 2026-11-10 19:27:16 +01:00
Cache .mypy_cache so mypy stops re-analysing the stdlib every run
Profiled the 23s hook step per hook. It is not evenly spread: mypy ~10s and pylint ~9s are the whole of it, and the remaining 21 hooks share ~4s. Local timings with a warm cache put mypy at 0.25s against 5.7s cold, so nearly all of its CI cost is re-analysing the stdlib and third-party stubs from scratch into a 19MB cache dir that was then thrown away. Keyed on the config rather than on the sources. The mypy cache metadata already records the interpreter, the flag set and a hash per source file, so mypy re-checks whatever actually changed and a restored cache cannot let stale code pass. The config hash is in the key regardless, so bumping the mypy rev or editing its args busts it. Deliberately not keyed on github.sha, which is the other common pattern: that re-saves 19MB x3 on every commit and would evict the 68MB hook caches under the 10GB LRU budget for the repo. A fixed key keeps the dependency analysis, which is the part that costs the 10s. pylint needs no change -- it already runs --jobs=4. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019kMvSMtsVMB3d2uoZ56pbp
This commit is contained in:
parent
a473e91485
commit
8a556f8d7d
16
.github/workflows/crdt-sync-precommit.yml
vendored
16
.github/workflows/crdt-sync-precommit.yml
vendored
@ -48,6 +48,22 @@ jobs:
|
||||
# than a wrong hook, and cache still re-saves under the new
|
||||
# exact key after a prefix-only hit.
|
||||
restore-keys: pre-commit-crdt-sync-${{ env.pythonLocation }}-
|
||||
# mypy re-analyses the stdlib and every third-party stub from scratch
|
||||
# without this, which measured 10s of the hook step in CI against 0.25s
|
||||
# warm locally -- a 19MB cache dir does the whole difference. Keyed on
|
||||
# the config, not the sources: mypy's own cache metadata records the
|
||||
# interpreter, the flag set and a hash per source file, so it re-checks
|
||||
# what changed and cannot pass stale code. Deliberately NOT keyed on
|
||||
# github.sha -- that re-saves 19MB x3 every commit and would evict the
|
||||
# 68MB hook caches under the repo's 10GB LRU budget. A fixed key keeps
|
||||
# the expensive part, the dependency analysis, which is what costs 10s.
|
||||
- uses: actions/cache@v4
|
||||
with:
|
||||
path: .mypy_cache
|
||||
key: >-
|
||||
mypy-crdt-sync-${{ env.pythonLocation }}-${{
|
||||
hashFiles('crdt-sync/.pre-commit-config.yaml') }}
|
||||
restore-keys: mypy-crdt-sync-${{ env.pythonLocation }}-
|
||||
- name: Run the hooks
|
||||
run: >-
|
||||
pre-commit run --config crdt-sync/.pre-commit-config.yaml
|
||||
|
||||
16
.github/workflows/freedays-precommit.yml
vendored
16
.github/workflows/freedays-precommit.yml
vendored
@ -51,6 +51,22 @@ jobs:
|
||||
# than a wrong hook, and cache still re-saves under the new
|
||||
# exact key after a prefix-only hit.
|
||||
restore-keys: pre-commit-freedays-${{ env.pythonLocation }}-
|
||||
# mypy re-analyses the stdlib and every third-party stub from scratch
|
||||
# without this, which measured 10s of the hook step in CI against 0.25s
|
||||
# warm locally -- a 19MB cache dir does the whole difference. Keyed on
|
||||
# the config, not the sources: mypy's own cache metadata records the
|
||||
# interpreter, the flag set and a hash per source file, so it re-checks
|
||||
# what changed and cannot pass stale code. Deliberately NOT keyed on
|
||||
# github.sha -- that re-saves 19MB x3 every commit and would evict the
|
||||
# 68MB hook caches under the repo's 10GB LRU budget. A fixed key keeps
|
||||
# the expensive part, the dependency analysis, which is what costs 10s.
|
||||
- uses: actions/cache@v4
|
||||
with:
|
||||
path: .mypy_cache
|
||||
key: >-
|
||||
mypy-freedays-${{ env.pythonLocation }}-${{
|
||||
hashFiles('freedays/.pre-commit-config.yaml') }}
|
||||
restore-keys: mypy-freedays-${{ env.pythonLocation }}-
|
||||
- name: Run the hooks
|
||||
run: >-
|
||||
pre-commit run --config freedays/.pre-commit-config.yaml
|
||||
|
||||
16
.github/workflows/gatelock-precommit.yml
vendored
16
.github/workflows/gatelock-precommit.yml
vendored
@ -50,6 +50,22 @@ jobs:
|
||||
# than a wrong hook, and cache still re-saves under the new
|
||||
# exact key after a prefix-only hit.
|
||||
restore-keys: pre-commit-gatelock-${{ env.pythonLocation }}-
|
||||
# mypy re-analyses the stdlib and every third-party stub from scratch
|
||||
# without this, which measured 10s of the hook step in CI against 0.25s
|
||||
# warm locally -- a 19MB cache dir does the whole difference. Keyed on
|
||||
# the config, not the sources: mypy's own cache metadata records the
|
||||
# interpreter, the flag set and a hash per source file, so it re-checks
|
||||
# what changed and cannot pass stale code. Deliberately NOT keyed on
|
||||
# github.sha -- that re-saves 19MB x3 every commit and would evict the
|
||||
# 68MB hook caches under the repo's 10GB LRU budget. A fixed key keeps
|
||||
# the expensive part, the dependency analysis, which is what costs 10s.
|
||||
- uses: actions/cache@v4
|
||||
with:
|
||||
path: .mypy_cache
|
||||
key: >-
|
||||
mypy-gatelock-${{ env.pythonLocation }}-${{
|
||||
hashFiles('gatelock/.pre-commit-config.yaml') }}
|
||||
restore-keys: mypy-gatelock-${{ env.pythonLocation }}-
|
||||
- name: Run the hooks
|
||||
run: >-
|
||||
pre-commit run --config gatelock/.pre-commit-config.yaml
|
||||
|
||||
Loading…
Reference in New Issue
Block a user