mirror of
https://github.com/kuhyx/testsAndMisc.git
synced 2026-11-10 19:07:04 +01:00
Krzysztof kuhy Rudnicki
2a1bb60d9f
The push kept failing with "1798 passed" immediately followed by "Killed". The suite was never the problem: it peaks at ~0.29 GiB against its own 2G cap. ci_mirror.sh runs pre-commit --all-files and the pytest gate concurrently, and the pre-commit tree peaks at ~1.75 GiB (mypy and pylint repo-wide, invisible to /usr/bin/time because it measures only the parent). The pair wants ~2 GiB at once; the pytest scope has MemorySwapMax=0, so on a loaded machine it is the one that dies. Concurrency is now conditional on MemAvailable. Above 4 GiB the fast path is unchanged. The probe lives in ci_mirror_mem.sh, following the existing ci_mirror_range/ci_mirror_shell split, which also keeps ci_mirror.sh under the line cap. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PJrqKERAo5QUeEHWS5Asgp
48 lines
1.8 KiB
Bash
Executable File
48 lines
1.8 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# shellcheck shell=bash
|
|
|
|
# ============================================================================
|
|
# Memory probe for ci_mirror.sh.
|
|
#
|
|
# The gate runs `pre-commit run --all-files` and the pytest suite at the same
|
|
# time. Measured peaks: the pre-commit tree reaches ~1.75 GiB (mypy and pylint
|
|
# over the whole repo) and the pytest scope ~0.29 GiB, so the pair needs
|
|
# ~2 GiB simultaneously. The pytest scope runs under MemoryMax=2G with
|
|
# MemorySwapMax=0, which means that on a machine already under pressure it is
|
|
# the one the kernel kills - surfacing as "1798 passed" immediately followed
|
|
# by "Killed" and a failed push that has nothing to do with the code.
|
|
#
|
|
# Sourced by ci_mirror.sh; not meant to be executed directly.
|
|
# ============================================================================
|
|
|
|
# Below this much available memory the two heavy gates run one after the
|
|
# other instead of concurrently. 4 GiB is the ~2 GiB they need at the same
|
|
# instant plus the same again of headroom, because MemAvailable does not
|
|
# account for what they are about to allocate.
|
|
readonly SERIAL_BELOW_MIB=4096
|
|
|
|
# MemAvailable in MiB, read straight from /proc rather than parsing `free`.
|
|
# Returns a large number when /proc is unreadable so the fast path is kept:
|
|
# this is an optimisation guard, not a correctness gate.
|
|
available_mem_mib() {
|
|
local key value kib=""
|
|
while read -r key value _; do
|
|
if [[ "$key" == "MemAvailable:" ]]; then
|
|
kib="$value"
|
|
break
|
|
fi
|
|
done </proc/meminfo
|
|
if [[ -z "$kib" ]]; then
|
|
echo 999999
|
|
return
|
|
fi
|
|
echo $((kib / 1024))
|
|
}
|
|
|
|
# True when the machine is too tight to run the two heavy gates at once.
|
|
# Exposed as a predicate rather than a bare threshold so the constant has a
|
|
# use inside this file and the caller reads as intent, not arithmetic.
|
|
should_serialise_gates() {
|
|
(($(available_mem_mib) < SERIAL_BELOW_MIB))
|
|
}
|