mirror of
https://github.com/kuhyx/testsAndMisc.git
synced 2026-11-10 19:07:04 +01:00
Extract system-maintenance, the isolated subsystem
github.com/kuhyx/system-maintenance, cloned to ~/system-maintenance First of the four live subsystems, and the one with no cross-unit source edges, so it establishes the pattern for the harder three. Passed in vmbox first -- the installer ran from an extracted-layout copy in the guest and detected its distro without any monorepo path -- then on the host. Its 21 Python and shell test files lived in linux_configuration/tests/, reaching bin/ through a shared conftest. Twenty moved with the code and got their own conftest; the shared one stays, since it serves several script directories that are not going anywhere. test_shutdown_timer_monitor.sh stays here. It is a pair test: it checks that digital_wellbeing's installer template and the monitor it installs still agree, so it belongs to whichever half owns the installer. It now reads the monitor from ~/system-maintenance and SKIPs when that checkout is absent. An intermediate version of that test fell back to /usr/local/bin, which HUNG -- the installed copy is stale and differs from source, so the test was comparing an installer template against a binary it never generated. Falling back to it would have been worse than skipping, so the fallback is gone and the comment says why. Also removed test_hosts_file_monitor.sh from shell-tests.yml's explicit list: it left with the subsystem, and CI would have failed on the missing file rather than quietly skipping it. Verified: 136 tests pass in the new repo, 202 still pass here, the pair test passes against the checkout and skips without it, and both the shell coverage gate and shellcheck are clean. Depth 294 -> 267. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PJrqKERAo5QUeEHWS5Asgp
This commit is contained in:
parent
840df29818
commit
a1f8044a4a
2
.github/workflows/python-tests.yml
vendored
2
.github/workflows/python-tests.yml
vendored
@ -7,7 +7,6 @@ on:
|
||||
- "python_pkg/lichess_bot/**"
|
||||
- "python_pkg/**"
|
||||
- "tests/**"
|
||||
- "linux_configuration/scripts/periodic_background/system-maintenance/bin/**"
|
||||
- "linux_configuration/tests/**"
|
||||
# Both spellings: the root requirements.txt is a SYMLINK into meta/,
|
||||
# and GitHub matches path filters against the changed path as git
|
||||
@ -24,7 +23,6 @@ on:
|
||||
- "python_pkg/lichess_bot/**"
|
||||
- "python_pkg/**"
|
||||
- "tests/**"
|
||||
- "linux_configuration/scripts/periodic_background/system-maintenance/bin/**"
|
||||
- "linux_configuration/tests/**"
|
||||
# Both spellings: the root requirements.txt is a SYMLINK into meta/,
|
||||
# and GitHub matches path filters against the changed path as git
|
||||
|
||||
1
.github/workflows/shell-tests.yml
vendored
1
.github/workflows/shell-tests.yml
vendored
@ -139,7 +139,6 @@ jobs:
|
||||
run: |
|
||||
set -euo pipefail
|
||||
tests=(
|
||||
test_hosts_file_monitor.sh
|
||||
test_hosts_guard_pacman_integration.sh
|
||||
test_i3blocks_efficiency.sh
|
||||
test_makepkg_capped.sh
|
||||
|
||||
@ -28,7 +28,9 @@ Extracted to their own repos: [`steam-backlog-enforcer`](https://github.com/kuhy
|
||||
[`reverse-survivors`](https://github.com/kuhyx/reverse-survivors),
|
||||
[`kcd2-dice-solver`](https://github.com/kuhyx/kcd2-dice-solver) (lives at `~/kcd2-dice-solver`),
|
||||
[`focus-owner`](https://github.com/kuhyx/focus-owner),
|
||||
[`mtk-root`](https://github.com/kuhyx/mtk-root).
|
||||
[`mtk-root`](https://github.com/kuhyx/mtk-root),
|
||||
[`system-maintenance`](https://github.com/kuhyx/system-maintenance) (lives at
|
||||
`~/system-maintenance`).
|
||||
Archived work: [`testsAndMisc-archive`](https://github.com/kuhyx/testsAndMisc-archive).
|
||||
|
||||
## Adding a Flutter app
|
||||
|
||||
@ -0,0 +1,20 @@
|
||||
{
|
||||
"title": "Extract system-maintenance, the first of the four live subsystems",
|
||||
"objective": "Move the periodic maintenance subsystem to its own public repo. It is the isolated one -- no cross-unit source edges -- so it goes first and establishes the pattern for the three harder ones. Its Python tests live in linux_configuration/tests/ and reach it through a shared conftest, so the test half must move with it without breaking the suites that stay.",
|
||||
"acceptance_criteria": [
|
||||
"A vmbox pass first: the installer runs from an extracted-layout copy in the guest before anything is changed on the host, and the result is reported as sandbox-only.",
|
||||
"The new repo's Python tests pass from a clean clone with its own conftest; 136 tests.",
|
||||
"Both extracted shell suites pass after their hardcoded monorepo paths are fixed, and shellcheck exits 0 with no severity filter.",
|
||||
"The new repo declares BOTH CI halves, since python-tests.yml reached it by paths filter and shell-tests.yml by name list, and neither follows the code out.",
|
||||
"The suites remaining in the monorepo still pass: 202 Python tests.",
|
||||
"The cross-unit pair test test_shutdown_timer_monitor.sh keeps working, and does not silently degrade into asserting against a stale installed binary.",
|
||||
"No workflow, conftest, pyproject sys.path entry or coverage-allowlist line names a removed path.",
|
||||
"pre-commit passes without --no-verify."
|
||||
],
|
||||
"out_of_scope": [
|
||||
"Refreshing the stale /usr/local/bin/shutdown-timer-monitor.sh, which digital_wellbeing's installer owns.",
|
||||
"The other three subsystems.",
|
||||
"Moving the shared linux_configuration/tests/conftest.py, which serves several script directories that stay."
|
||||
],
|
||||
"verifier": "vm run in guest exercises the installer; python -m pytest tests -q reports 136 passed in the new repo; bash tests/*.sh and shellcheck both clean there; pytest linux_configuration/tests reports 202 passed here; bash linux_configuration/tests/test_shutdown_timer_monitor.sh passes with the checkout and SKIPs without it; check_shell_coverage.sh --all exits 0; check_directory_depth.sh --all drops 294 -> 267."
|
||||
}
|
||||
@ -0,0 +1,59 @@
|
||||
{
|
||||
"intent": "Extract the periodic system-maintenance subsystem into its own public repo with the 21 test files that lived in linux_configuration/tests/, and keep the one cross-unit pair test working here.",
|
||||
"scope": [
|
||||
"linux_configuration/scripts/periodic_background/system-maintenance/ (32 files) -> github.com/kuhyx/system-maintenance, cloned to ~/system-maintenance",
|
||||
"20 of its 21 test files in linux_configuration/tests/ moved with it",
|
||||
"test_shutdown_timer_monitor.sh KEPT here: it is the pair test against digital_wellbeing's installer",
|
||||
"conftest.py, meta/pyproject.toml, python-tests.yml, shell-tests.yml, shell-coverage-allowlist.txt cleaned",
|
||||
"Non-goal: the other three subsystems"
|
||||
],
|
||||
"changes": [
|
||||
"Extracted the subsystem and its tests in one filter-repo pass (28 commits), hoisting bin/, systemd/, logrotate/ to the repo root and linux_configuration/tests/ to tests/.",
|
||||
"Gave the new repo its own tests/conftest.py putting bin/ on sys.path, plus a pyproject.toml with testpaths and branch coverage. The monorepo's shared conftest served several script directories and stays.",
|
||||
"Fixed two extracted shell suites that hardcoded $REPO_DIR/scripts/periodic_background/system-maintenance/bin, now $REPO_DIR/bin.",
|
||||
"Re-declared both CI halves in the new repo: python-tests.yml reached these modules through a paths filter and shell-tests.yml through a name list, neither of which follows the code out.",
|
||||
"Repointed the kept pair test at ${SYSTEM_MAINTENANCE_DIR:-$HOME/system-maintenance}, skipping cleanly when absent.",
|
||||
"Removed test_hosts_file_monitor.sh from shell-tests.yml's explicit list, since it left with the subsystem and CI would otherwise fail on a missing file."
|
||||
],
|
||||
"verification": [
|
||||
{
|
||||
"command": "sandbox: vm new sm1 + run the installer from an extracted-layout copy",
|
||||
"result": "pass in vmbox, NOT host-verified",
|
||||
"evidence": "install_usage_monitoring.sh detected the Arch distro family and resolved its 35 packages from a copy rooted outside the monorepo. A grep for testsAndMisc/periodic_background/linux_configuration across the extracted tree found only one stale comment, confirming no path coupling."
|
||||
},
|
||||
{
|
||||
"command": "python -m pytest tests -q (extracted repo)",
|
||||
"result": "pass",
|
||||
"evidence": "136 passed."
|
||||
},
|
||||
{
|
||||
"command": "bash tests/*.sh (extracted repo) + shellcheck",
|
||||
"result": "pass",
|
||||
"evidence": "Both remaining shell suites pass after the path fix; shellcheck exits 0 over every script with .shellcheckrc carried across."
|
||||
},
|
||||
{
|
||||
"command": "pytest linux_configuration/tests (monorepo, after removal)",
|
||||
"result": "pass",
|
||||
"evidence": "202 passed."
|
||||
},
|
||||
{
|
||||
"command": "bash linux_configuration/tests/test_shutdown_timer_monitor.sh",
|
||||
"result": "pass",
|
||||
"evidence": "Passes against the ~/system-maintenance checkout, matching the baseline worktree's behaviour. An intermediate version fell back to /usr/local/bin and HUNG: the installed copy is stale and differs from source, so the fallback was removed rather than kept. SYSTEM_MAINTENANCE_DIR=/nonexistent prints SKIP and exits 0."
|
||||
},
|
||||
{
|
||||
"command": "bash meta/scripts/check_shell_coverage.sh --all",
|
||||
"result": "pass",
|
||||
"evidence": "Exit 0 with the five system-maintenance allowlist entries removed."
|
||||
}
|
||||
],
|
||||
"risks": [
|
||||
"/usr/local/bin/shutdown-timer-monitor.sh is stale relative to source. Left alone here; digital_wellbeing's installer owns reinstalling it, and that is the next extraction.",
|
||||
"The pair test now depends on a sibling checkout at ~/system-maintenance. It skips rather than fails when absent, so CI without the checkout reports a skip, not a false pass on assertions it never ran.",
|
||||
"/usr/local/bin/periodic-system-maintenance.sh still names digital_wellbeing and hosts paths inside this repo; those extractions must reinstall it."
|
||||
],
|
||||
"rollback": [
|
||||
"git revert to restore the subsystem and its 21 tests, then re-add the conftest entry, pyproject sys.path element and both workflow references.",
|
||||
"Re-run pytest linux_configuration/tests and check_shell_coverage.sh --all to confirm the restored state."
|
||||
]
|
||||
}
|
||||
@ -1 +0,0 @@
|
||||
"""Helpers for usage_report: parsing and type definitions."""
|
||||
@ -1,206 +0,0 @@
|
||||
"""Subprocess plumbing for usage_report.
|
||||
|
||||
Runs atop, streams its parseable output, and builds/caches the native
|
||||
`atop_agg` C helper.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime as _dt
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import subprocess
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from _usage_report_types import ProcAgg, _Window
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Iterator
|
||||
|
||||
from _usage_report_types import _Progress
|
||||
|
||||
_ATOP_AGG_CACHE_BIN = Path.home() / ".cache" / "usage_report" / "atop_agg"
|
||||
_ATOP_AGG_BIN_MODE = 0o755
|
||||
# Repo layout: linux_configuration/scripts/system-maintenance/bin/usage_report.py
|
||||
# -> parents[4] is the repo root which hosts the C/ source tree.
|
||||
_ATOP_AGG_SRC_DIR = Path(__file__).resolve().parents[4] / "C" / "atop_agg"
|
||||
_ATOP_AGG_BUILD_TIMEOUT_S = 60
|
||||
_NATIVE_TSV_NAME_LEN = 7
|
||||
_NATIVE_TSV_WIN_LEN = 5
|
||||
|
||||
|
||||
def _run(cmd: list[str]) -> str:
|
||||
"""Run *cmd* and return stdout (empty string on failure)."""
|
||||
try:
|
||||
proc = subprocess.run(
|
||||
cmd,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
timeout=60,
|
||||
)
|
||||
except (OSError, subprocess.TimeoutExpired):
|
||||
return ""
|
||||
return proc.stdout
|
||||
|
||||
|
||||
def _atop_read_cmd(
|
||||
log: Path,
|
||||
labels: str,
|
||||
begin: str | None,
|
||||
end: str | None,
|
||||
) -> list[str]:
|
||||
"""Build an `atop -r` command, optionally bounded by begin/end times.
|
||||
|
||||
*begin*/*end* are atop `-b`/`-e` arguments (`[YYYYMMDD]hhmm[ss]`) used to
|
||||
restrict replay to a sub-window of the day's log, so a "since last report"
|
||||
run does not double-count the part of the first day already reported.
|
||||
"""
|
||||
cmd = ["atop", "-r", str(log)]
|
||||
if begin is not None:
|
||||
cmd += ["-b", begin]
|
||||
if end is not None:
|
||||
cmd += ["-e", end]
|
||||
cmd += ["-P", labels]
|
||||
return cmd
|
||||
|
||||
|
||||
def _iter_atop_lines(
|
||||
log: Path,
|
||||
labels: str,
|
||||
begin: str | None = None,
|
||||
end: str | None = None,
|
||||
) -> Iterator[str]:
|
||||
"""Stream `atop -r LOG -P LABELS` stdout line-by-line.
|
||||
|
||||
Uses `Popen` so the report can show progress while atop is still
|
||||
decoding its binary log, rather than buffering the whole output.
|
||||
"""
|
||||
cmd = _atop_read_cmd(log, labels, begin, end)
|
||||
with subprocess.Popen(
|
||||
cmd,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.DEVNULL,
|
||||
text=True,
|
||||
) as proc:
|
||||
stdout = proc.stdout
|
||||
if stdout is None:
|
||||
return
|
||||
for raw in stdout:
|
||||
yield raw.rstrip("\n")
|
||||
|
||||
|
||||
def _atop_agg_binary() -> Path | None:
|
||||
"""Return a cached `atop_agg` binary path, auto-building if missing/stale.
|
||||
|
||||
Falls back to ``None`` when the C source tree or a system C compiler
|
||||
is unavailable, in which case callers use the pure-Python parser.
|
||||
"""
|
||||
src_c = _ATOP_AGG_SRC_DIR / "atop_agg.c"
|
||||
if not src_c.exists():
|
||||
# Source tree is gone (relocated/extracted): never trust an orphaned
|
||||
# cached binary whose provenance we can no longer verify against
|
||||
# source — a stale build can silently carry parsing bugs. Fall back to
|
||||
# the pure-Python parser instead.
|
||||
return None
|
||||
if (
|
||||
_ATOP_AGG_CACHE_BIN.exists()
|
||||
and src_c.stat().st_mtime <= _ATOP_AGG_CACHE_BIN.stat().st_mtime
|
||||
):
|
||||
return _ATOP_AGG_CACHE_BIN
|
||||
if shutil.which("cc") is None:
|
||||
return None
|
||||
_ATOP_AGG_CACHE_BIN.parent.mkdir(parents=True, exist_ok=True)
|
||||
make_cmd = ["make", "-s", "-C", str(_ATOP_AGG_SRC_DIR), "atop_agg"]
|
||||
try:
|
||||
subprocess.run(
|
||||
make_cmd,
|
||||
check=True,
|
||||
capture_output=True,
|
||||
timeout=_ATOP_AGG_BUILD_TIMEOUT_S,
|
||||
)
|
||||
except (OSError, subprocess.SubprocessError):
|
||||
return None
|
||||
built = _ATOP_AGG_SRC_DIR / "atop_agg"
|
||||
if not built.exists():
|
||||
return None
|
||||
shutil.copy2(built, _ATOP_AGG_CACHE_BIN)
|
||||
_ATOP_AGG_CACHE_BIN.chmod(_ATOP_AGG_BIN_MODE)
|
||||
return _ATOP_AGG_CACHE_BIN
|
||||
|
||||
|
||||
def _apply_native_name(parts: list[str], agg_map: dict[str, ProcAgg]) -> None:
|
||||
r"""Fold one `N\t<name>\t<cpu>\t<peak>\t<sum_avg>\t<ram_n>\t<pids>` row."""
|
||||
_, name, cpu_s, peak_s, sum_avg_s, rss_n_s, pids_s = parts
|
||||
entry = agg_map.setdefault(name, ProcAgg(name=name))
|
||||
entry.cpu_ticks = int(cpu_s)
|
||||
entry.peak_rss_kb = int(peak_s)
|
||||
entry.rss_kb_sum = int(sum_avg_s)
|
||||
entry.rss_samples = int(rss_n_s)
|
||||
# The C helper pre-aggregates by name; pid_set is unused in the native
|
||||
# path but `len(pid_set)` drives the "PIDs" column in the report.
|
||||
entry.pid_set = set(range(int(pids_s)))
|
||||
|
||||
|
||||
def _window_from_native(parts: list[str]) -> _Window:
|
||||
r"""Build a `_Window` from a `W\t<start>\t<end>\t<n>\t<interval>` row."""
|
||||
_, start_s, end_s, n_s, interval_s = parts
|
||||
n_epochs = int(n_s)
|
||||
if not n_epochs:
|
||||
return _Window()
|
||||
start_epoch = int(start_s)
|
||||
end_epoch = int(end_s)
|
||||
start_dt = _dt.datetime.fromtimestamp(start_epoch).astimezone()
|
||||
end_dt = _dt.datetime.fromtimestamp(end_epoch).astimezone()
|
||||
return _Window(
|
||||
start=start_dt.isoformat(timespec="seconds"),
|
||||
end=end_dt.isoformat(timespec="seconds"),
|
||||
distinct_samples=n_epochs,
|
||||
interval_s=int(interval_s),
|
||||
seconds=end_epoch - start_epoch,
|
||||
start_epoch=start_epoch,
|
||||
end_epoch=end_epoch,
|
||||
)
|
||||
|
||||
|
||||
def _aggregate_atop_native(
|
||||
log: Path,
|
||||
progress: _Progress,
|
||||
binary: Path,
|
||||
begin: str | None = None,
|
||||
end: str | None = None,
|
||||
) -> tuple[dict[str, ProcAgg], _Window]:
|
||||
"""Aggregate via `atop | atop_agg`; return `(by_name, window)`."""
|
||||
progress.start_stage("atop: parse PRC+PRM (native)")
|
||||
agg_map: dict[str, ProcAgg] = {}
|
||||
window = _Window()
|
||||
atop_cmd = _atop_read_cmd(log, "PRC,PRM", begin, end)
|
||||
agg_cmd = [str(binary)]
|
||||
with (
|
||||
subprocess.Popen(
|
||||
atop_cmd,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.DEVNULL,
|
||||
) as atop,
|
||||
subprocess.Popen(
|
||||
agg_cmd,
|
||||
stdin=atop.stdout,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.DEVNULL,
|
||||
text=True,
|
||||
) as agg,
|
||||
):
|
||||
if atop.stdout is not None:
|
||||
atop.stdout.close()
|
||||
stdout = agg.stdout
|
||||
if stdout is None:
|
||||
return agg_map, window
|
||||
for raw in stdout:
|
||||
parts = raw.rstrip("\n").split("\t")
|
||||
tag = parts[0]
|
||||
if tag == "N" and len(parts) == _NATIVE_TSV_NAME_LEN:
|
||||
_apply_native_name(parts, agg_map)
|
||||
elif tag == "W" and len(parts) == _NATIVE_TSV_WIN_LEN:
|
||||
window = _window_from_native(parts)
|
||||
progress.update(1.0)
|
||||
return agg_map, window
|
||||
@ -1,68 +0,0 @@
|
||||
"""Preflight checks and report output for usage_report.
|
||||
|
||||
Verifies atop and its log are present before a run, writes the finished report
|
||||
to stdout, and copies it to the system clipboard with whichever tool is
|
||||
installed.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
if TYPE_CHECKING:
|
||||
import argparse
|
||||
|
||||
|
||||
_INSTALL_SCRIPT = Path(__file__).with_name("install_usage_monitoring.sh")
|
||||
|
||||
|
||||
def _preflight(atop_log: Path) -> None:
|
||||
if not shutil.which("atop"):
|
||||
sys.exit(
|
||||
f"error: `atop` is not installed.\nrun: {_INSTALL_SCRIPT}",
|
||||
)
|
||||
if not atop_log.exists():
|
||||
sys.exit(
|
||||
f"error: atop log not found: {atop_log}\n"
|
||||
f"run: {_INSTALL_SCRIPT} (enables atop.service), "
|
||||
"then wait for the first sample.",
|
||||
)
|
||||
|
||||
|
||||
_CLIPBOARD_CANDIDATES: tuple[tuple[str, tuple[str, ...]], ...] = (
|
||||
("wl-copy", ("wl-copy",)),
|
||||
("xclip", ("xclip", "-selection", "clipboard")),
|
||||
("xsel", ("xsel", "--clipboard", "--input")),
|
||||
)
|
||||
|
||||
|
||||
def _copy_to_clipboard(text: str) -> None:
|
||||
"""Copy `text` to the system clipboard using the first available tool.
|
||||
|
||||
Prints a one-line status to stderr so the stdout report stays pristine
|
||||
for redirection.
|
||||
"""
|
||||
for name, cmd in _CLIPBOARD_CANDIDATES:
|
||||
if not shutil.which(name):
|
||||
continue
|
||||
try:
|
||||
subprocess.run(cmd, input=text, text=True, check=True)
|
||||
except (subprocess.CalledProcessError, OSError) as exc:
|
||||
sys.stderr.write(f"clipboard: {name} failed: {exc}\n")
|
||||
return
|
||||
sys.stderr.write(f"clipboard: copied {len(text)} chars via {name}\n")
|
||||
return
|
||||
sys.stderr.write(
|
||||
"clipboard: no wl-copy/xclip/xsel found; skipping copy\n",
|
||||
)
|
||||
|
||||
|
||||
def _emit(args: argparse.Namespace, report: str) -> None:
|
||||
"""Write the report to stdout and (unless suppressed) the clipboard."""
|
||||
sys.stdout.write(report)
|
||||
if not args.no_clipboard:
|
||||
_copy_to_clipboard(report)
|
||||
@ -1,24 +0,0 @@
|
||||
"""Markdown escaping and duration formatting shared by the report sections."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
_SEC_PER_HOUR = 3600
|
||||
_SEC_PER_MIN = 60
|
||||
|
||||
|
||||
def _md_escape(name: str) -> str:
|
||||
"""Escape characters that would break a Markdown table cell."""
|
||||
return name.replace("|", r"\|").replace("\n", " ")
|
||||
|
||||
|
||||
def _fmt_h(seconds: float) -> str:
|
||||
"""Human-friendly duration: `"1h 23m"` / `"4m 12s"` / `"8.3s"`."""
|
||||
if seconds >= _SEC_PER_HOUR:
|
||||
h = int(seconds // _SEC_PER_HOUR)
|
||||
m = int((seconds % _SEC_PER_HOUR) // _SEC_PER_MIN)
|
||||
return f"{h}h {m:02d}m"
|
||||
if seconds >= _SEC_PER_MIN:
|
||||
m = int(seconds // _SEC_PER_MIN)
|
||||
s = int(seconds % _SEC_PER_MIN)
|
||||
return f"{m}m {s:02d}s"
|
||||
return f"{seconds:.1f}s"
|
||||
@ -1,117 +0,0 @@
|
||||
"""Log discovery, report state and per-day segment planning for usage_report.
|
||||
|
||||
Resolves which daily atop/pmon logs a reporting window covers, persists the
|
||||
"last report generated" timestamp that drives the since-last-report mode, and
|
||||
describes the logs actually consumed for the report's methodology section.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
import datetime as _dt
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
from _usage_report_types import _PMON_INTERVAL_S
|
||||
|
||||
_ATOP_LOG_DIR = Path("/var/log/atop")
|
||||
_PMON_LOG_DIR = Path.home() / ".local/share/gpu-log"
|
||||
# Persisted marker of when the last report was generated. Lives under
|
||||
# ~/.local/share (durable app state), not ~/.cache, so clearing caches does not
|
||||
# silently reset the "since last report" window back to today-only.
|
||||
_STATE_DIR = Path.home() / ".local/share/usage_report"
|
||||
_STATE_FILE = _STATE_DIR / "last_report.json"
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Segment:
|
||||
"""One calendar day's resolved logs plus optional in-day start bounds.
|
||||
|
||||
*atop_begin* is an atop ``-b`` argument (``YYYYMMDDhhmmss``) and
|
||||
*pmon_begin_epoch* the matching local epoch; both are set only for the first
|
||||
day of a "since last report" window so re-runs do not double-count.
|
||||
"""
|
||||
|
||||
atop_log: Path
|
||||
pmon_log: Path
|
||||
atop_begin: str | None = None
|
||||
pmon_begin_epoch: float | None = None
|
||||
|
||||
|
||||
def _resolve_logs(date: str) -> tuple[Path, Path]:
|
||||
atop_log = _ATOP_LOG_DIR / f"atop_{date}"
|
||||
pmon_log = _PMON_LOG_DIR / f"pmon-{date}.log"
|
||||
return atop_log, pmon_log
|
||||
|
||||
|
||||
def _read_last_generated() -> _dt.datetime | None:
|
||||
"""Return the timestamp of the previous report run, or None if unknown."""
|
||||
try:
|
||||
raw = _STATE_FILE.read_text(encoding="utf-8")
|
||||
except OSError:
|
||||
return None
|
||||
try:
|
||||
stamp = json.loads(raw)["last_generated"]
|
||||
return _dt.datetime.fromisoformat(stamp).astimezone()
|
||||
except (ValueError, KeyError, TypeError):
|
||||
return None
|
||||
|
||||
|
||||
def _write_last_generated(when: _dt.datetime) -> None:
|
||||
"""Persist *when* as the last-report timestamp for the next run."""
|
||||
_STATE_DIR.mkdir(parents=True, exist_ok=True)
|
||||
payload = json.dumps({"last_generated": when.isoformat(timespec="seconds")})
|
||||
_STATE_FILE.write_text(payload + "\n", encoding="utf-8")
|
||||
|
||||
|
||||
def _has_time_of_day(when: _dt.datetime) -> bool:
|
||||
"""True when *when* is past local midnight, so a begin bound is needed."""
|
||||
return bool(when.hour or when.minute or when.second or when.microsecond)
|
||||
|
||||
|
||||
def _plan_segments(start: _dt.datetime, end: _dt.datetime) -> list[_Segment]:
|
||||
"""Resolve one `_Segment` per calendar day across ``[start, end]``.
|
||||
|
||||
The first day is bounded at *start*'s time-of-day so a same-day re-run only
|
||||
covers the slice since the previous report; later days are covered in full.
|
||||
Returns an empty list when *start* is after *end* (e.g. a future state file).
|
||||
"""
|
||||
segments: list[_Segment] = []
|
||||
day = start.date()
|
||||
while day <= end.date():
|
||||
atop_log, pmon_log = _resolve_logs(day.strftime("%Y%m%d"))
|
||||
if day == start.date() and _has_time_of_day(start):
|
||||
segments.append(
|
||||
_Segment(
|
||||
atop_log,
|
||||
pmon_log,
|
||||
start.strftime("%Y%m%d%H%M%S"),
|
||||
start.timestamp(),
|
||||
),
|
||||
)
|
||||
else:
|
||||
segments.append(_Segment(atop_log, pmon_log))
|
||||
day += _dt.timedelta(days=1)
|
||||
return segments
|
||||
|
||||
|
||||
def _describe_logs(paths: list[Path], how: str) -> str:
|
||||
"""One-line Markdown description of the log files actually consumed."""
|
||||
if not paths:
|
||||
return f"_none found_ (`{how}`)"
|
||||
if len(paths) == 1:
|
||||
return f"`{paths[0]}` (`{how}`)"
|
||||
return (
|
||||
f"{len(paths)} daily logs `{paths[0].name}` … `{paths[-1].name}` "
|
||||
f"in `{paths[0].parent}` (`{how}`)"
|
||||
)
|
||||
|
||||
|
||||
def _log_descriptions(segments: list[_Segment]) -> tuple[str, str]:
|
||||
"""Return ``(atop_desc, pmon_desc)`` for the logs present in *segments*."""
|
||||
atop_present = [seg.atop_log for seg in segments if seg.atop_log.exists()]
|
||||
pmon_present = [seg.pmon_log for seg in segments if seg.pmon_log.exists()]
|
||||
return (
|
||||
_describe_logs(atop_present, "atop -r"),
|
||||
_describe_logs(pmon_present, f"nvidia-smi pmon -d {_PMON_INTERVAL_S}"),
|
||||
)
|
||||
@ -1,243 +0,0 @@
|
||||
"""atop record parsing and per-day aggregate merging for usage_report."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import datetime as _dt
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from _usage_report_atop import (
|
||||
_aggregate_atop_native,
|
||||
_atop_agg_binary,
|
||||
_iter_atop_lines,
|
||||
)
|
||||
from _usage_report_types import (
|
||||
_MIN_SAMPLES_FOR_WINDOW,
|
||||
GpuAgg,
|
||||
ProcAgg,
|
||||
_PidCpu,
|
||||
_PidRam,
|
||||
_Window,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
from _usage_report_types import _Progress
|
||||
|
||||
# atop parseable output layout (atop 2.x, same on Arch/Debian/Ubuntu):
|
||||
# 0 label, 1 host, 2 epoch, 3 YYYY/MM/DD, 4 HH:MM:SS, 5 interval_s,
|
||||
# then per-process fields starting at index 6.
|
||||
# PRC per-proc: pid name(parens) state HZ utime_ticks stime_ticks ...
|
||||
# NOTE: atop inserts its clock-tick rate (HZ) between `state` and `utime`
|
||||
# (the PRC analogue of the pagesize field PRM inserts before its memory
|
||||
# columns); utime/stime therefore live two and three slots past `state`.
|
||||
_PRC_PID_IDX = 6
|
||||
_PRC_NAME_IDX = 7
|
||||
_PRC_MIN_LEN = 12
|
||||
# PRM per-proc: pid name state pagesz_b vsize_kb rsize_kb ...
|
||||
_PRM_PID_IDX = 6
|
||||
_PRM_NAME_IDX = 7
|
||||
_PRM_MIN_LEN = 12
|
||||
_CPU_RECORD_MIN_LEN = 5
|
||||
_PAREN_PAIR_MIN = 2
|
||||
|
||||
|
||||
def _parse_name(parts: list[str], name_idx: int) -> tuple[str, int]:
|
||||
"""Extract `(name, next_index)` from atop parseable output.
|
||||
|
||||
atop wraps process names in parentheses and the name itself may contain
|
||||
spaces, so we rejoin until we hit the closing `)`. Fast-paths the common
|
||||
case where the name is a single token (no embedded spaces).
|
||||
"""
|
||||
if name_idx >= len(parts):
|
||||
return "unknown", name_idx + 1
|
||||
token = parts[name_idx]
|
||||
# Fast path: `(name)` fully in one token.
|
||||
if len(token) >= _PAREN_PAIR_MIN and token[0] == "(" and token[-1] == ")":
|
||||
return token[1:-1] or "unknown", name_idx + 1
|
||||
if token.startswith("("):
|
||||
buf = [token]
|
||||
idx = name_idx
|
||||
while not buf[-1].endswith(")") and idx + 1 < len(parts):
|
||||
idx += 1
|
||||
buf.append(parts[idx])
|
||||
name = " ".join(buf)[1:-1] or "unknown"
|
||||
return name, idx + 1
|
||||
return token, name_idx + 1
|
||||
|
||||
|
||||
def _parse_prc(parts: list[str], pid_cpu: dict[int, _PidCpu]) -> None:
|
||||
"""Fold one PRC record into the per-PID CPU-ticks map."""
|
||||
try:
|
||||
pid = int(parts[_PRC_PID_IDX])
|
||||
except (ValueError, IndexError):
|
||||
return
|
||||
name, after = _parse_name(parts, _PRC_NAME_IDX)
|
||||
# After name comes: state HZ utime stime ... (HZ is atop's clock-tick
|
||||
# rate; skipping it is what keeps a constant 100 from being charged as
|
||||
# CPU to every record — the bug that made cpu-seconds collapse to PID
|
||||
# count for short-lived processes).
|
||||
try:
|
||||
utime = int(parts[after + 2])
|
||||
stime = int(parts[after + 3])
|
||||
except (ValueError, IndexError):
|
||||
return
|
||||
pid_cpu.setdefault(pid, _PidCpu()).observe(name, utime + stime)
|
||||
|
||||
|
||||
def _parse_prm(parts: list[str], pid_ram: dict[int, _PidRam]) -> None:
|
||||
"""Fold one PRM record into the per-PID RSS map."""
|
||||
try:
|
||||
pid = int(parts[_PRM_PID_IDX])
|
||||
except (ValueError, IndexError):
|
||||
return
|
||||
name, after = _parse_name(parts, _PRM_NAME_IDX)
|
||||
# After name: state pagesz_b vsize_kb rsize_kb ...
|
||||
try:
|
||||
rsize_kb = int(parts[after + 3])
|
||||
except (ValueError, IndexError):
|
||||
return
|
||||
pid_ram.setdefault(pid, _PidRam()).observe(name, rsize_kb)
|
||||
|
||||
|
||||
def _window_from_epochs(epochs: set[int]) -> _Window:
|
||||
"""Build a `_Window` from a set of sample epoch timestamps."""
|
||||
if not epochs:
|
||||
return _Window()
|
||||
ordered = sorted(epochs)
|
||||
start_dt = _dt.datetime.fromtimestamp(ordered[0]).astimezone()
|
||||
end_dt = _dt.datetime.fromtimestamp(ordered[-1]).astimezone()
|
||||
interval = 0
|
||||
if len(ordered) >= _MIN_SAMPLES_FOR_WINDOW:
|
||||
deltas = sorted(ordered[i + 1] - ordered[i] for i in range(len(ordered) - 1))
|
||||
interval = deltas[len(deltas) // 2]
|
||||
return _Window(
|
||||
start=start_dt.isoformat(timespec="seconds"),
|
||||
end=end_dt.isoformat(timespec="seconds"),
|
||||
distinct_samples=len(ordered),
|
||||
interval_s=interval,
|
||||
seconds=ordered[-1] - ordered[0],
|
||||
start_epoch=ordered[0],
|
||||
end_epoch=ordered[-1],
|
||||
)
|
||||
|
||||
|
||||
def aggregate_atop(
|
||||
log: Path,
|
||||
progress: _Progress,
|
||||
begin: str | None = None,
|
||||
end: str | None = None,
|
||||
) -> tuple[dict[str, ProcAgg], _Window]:
|
||||
"""Stream PRC+PRM records, fold them into `{name: ProcAgg}`, return window.
|
||||
|
||||
Prefers the native `atop_agg` C helper (auto-built into
|
||||
``~/.cache/usage_report/``) for ~7x speedup on full-day logs, falling
|
||||
back to an inline Python parser when the helper is unavailable.
|
||||
|
||||
*begin*/*end* are optional atop `-b`/`-e` arguments that bound replay to a
|
||||
sub-window of the day's log (used by the "since last report" mode).
|
||||
"""
|
||||
binary = _atop_agg_binary()
|
||||
if binary is not None:
|
||||
return _aggregate_atop_native(log, progress, binary, begin, end)
|
||||
progress.start_stage("atop: parse PRC+PRM")
|
||||
pid_cpu: dict[int, _PidCpu] = {}
|
||||
pid_ram: dict[int, _PidRam] = {}
|
||||
epochs: set[int] = set()
|
||||
log_size = max(log.stat().st_size, 1)
|
||||
bytes_seen = 0
|
||||
# Empirical: `atop -P PRC,PRM` stdout is ~11x the binary log size on a
|
||||
# 10-min-interval log. The fraction is only used for the progress bar,
|
||||
# so a rough calibration is fine; it caps at 99% if we underestimate.
|
||||
est_total_bytes = log_size * 11 or 1
|
||||
for raw in _iter_atop_lines(log, "PRC,PRM", begin, end):
|
||||
bytes_seen += len(raw) + 1
|
||||
if not raw or raw[0] == "#" or raw.startswith("RESET") or raw == "SEP":
|
||||
continue
|
||||
parts = raw.split()
|
||||
if not parts:
|
||||
continue
|
||||
label = parts[0]
|
||||
if label == "PRC" and len(parts) >= _PRC_MIN_LEN:
|
||||
with contextlib.suppress(ValueError):
|
||||
# atop always emits an integer epoch here; guard is defensive.
|
||||
epochs.add(int(parts[2]))
|
||||
progress.update(min(bytes_seen / est_total_bytes, 0.99))
|
||||
_parse_prc(parts, pid_cpu)
|
||||
elif label == "PRM" and len(parts) >= _PRM_MIN_LEN:
|
||||
_parse_prm(parts, pid_ram)
|
||||
progress.update(1.0)
|
||||
return _fold_pid_aggregates(pid_cpu, pid_ram), _window_from_epochs(epochs)
|
||||
|
||||
|
||||
def _fold_pid_aggregates(
|
||||
pid_cpu: dict[int, _PidCpu],
|
||||
pid_ram: dict[int, _PidRam],
|
||||
) -> dict[str, ProcAgg]:
|
||||
"""Collapse per-PID CPU/RAM trackers into per-program `ProcAgg` entries."""
|
||||
agg: dict[str, ProcAgg] = {}
|
||||
for pid, cpu in pid_cpu.items():
|
||||
entry = agg.setdefault(cpu.name, ProcAgg(name=cpu.name))
|
||||
entry.cpu_ticks += cpu.delta_ticks
|
||||
entry.pid_set.add(pid)
|
||||
for pid, ram in pid_ram.items():
|
||||
entry = agg.setdefault(ram.name, ProcAgg(name=ram.name))
|
||||
entry.peak_rss_kb = max(entry.peak_rss_kb, ram.peak_kb)
|
||||
entry.rss_kb_sum += int(ram.avg_kb)
|
||||
entry.rss_samples += 1
|
||||
entry.pid_set.add(pid)
|
||||
return agg
|
||||
|
||||
|
||||
def merge_proc_aggs(dst: dict[str, ProcAgg], src: dict[str, ProcAgg]) -> None:
|
||||
"""Fold one day's CPU/RAM aggregates (*src*) into the running *dst*.
|
||||
|
||||
CPU-seconds and RSS sample counts add across days; peak RSS is the max;
|
||||
PID counts add (each day contributes its own distinct PIDs).
|
||||
"""
|
||||
for name, item in src.items():
|
||||
entry = dst.setdefault(name, ProcAgg(name=name))
|
||||
entry.cpu_ticks += item.cpu_ticks
|
||||
entry.peak_rss_kb = max(entry.peak_rss_kb, item.peak_rss_kb)
|
||||
entry.rss_kb_sum += item.rss_kb_sum
|
||||
entry.rss_samples += item.rss_samples
|
||||
entry.extra_pids += item.pid_count
|
||||
|
||||
|
||||
def merge_gpu_aggs(dst: dict[str, GpuAgg], src: dict[str, GpuAgg]) -> None:
|
||||
"""Fold one day's GPU aggregates (*src*) into the running *dst*."""
|
||||
for name, item in src.items():
|
||||
entry = dst.setdefault(name, GpuAgg(name=name))
|
||||
entry.sm_pct_sum += item.sm_pct_sum
|
||||
entry.mem_pct_sum += item.mem_pct_sum
|
||||
entry.samples += item.samples
|
||||
entry.peak_sm_pct = max(entry.peak_sm_pct, item.peak_sm_pct)
|
||||
entry.peak_mem_pct = max(entry.peak_mem_pct, item.peak_mem_pct)
|
||||
entry.extra_pids += item.pid_count
|
||||
|
||||
|
||||
def merge_windows(windows: list[_Window]) -> _Window:
|
||||
"""Combine per-day coverage *windows* into one spanning window.
|
||||
|
||||
Start/end span the earliest and latest samples; ``seconds`` sums the
|
||||
per-day coverage (not wall-clock end-start) so the denominator for average
|
||||
CPU% reflects only the time actually monitored, excluding gap days.
|
||||
"""
|
||||
real = [w for w in windows if w.distinct_samples]
|
||||
if not real:
|
||||
return _Window()
|
||||
first = min(real, key=lambda w: w.start_epoch)
|
||||
last = max(real, key=lambda w: w.end_epoch)
|
||||
intervals = [w.interval_s for w in real if w.interval_s]
|
||||
# Representative interval = the most common per-day interval, if any.
|
||||
interval = max(set(intervals), key=intervals.count) if intervals else 0
|
||||
return _Window(
|
||||
start=first.start,
|
||||
end=last.end,
|
||||
distinct_samples=sum(w.distinct_samples for w in real),
|
||||
interval_s=interval,
|
||||
seconds=sum(w.seconds for w in real),
|
||||
start_epoch=first.start_epoch,
|
||||
end_epoch=last.end_epoch,
|
||||
)
|
||||
@ -1,128 +0,0 @@
|
||||
"""nvidia-smi pmon log parsing and aggregation helpers for usage_report."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime as _dt
|
||||
from pathlib import Path
|
||||
|
||||
from _usage_report_types import GpuAgg, _Progress
|
||||
|
||||
_PMON_MIN_FIELDS = 11
|
||||
|
||||
|
||||
def _pmon_fields(line: str) -> list[str] | None:
|
||||
"""Return stripped fields of a pmon data line, or None for headers/blanks."""
|
||||
s = line.strip()
|
||||
if not s or s.startswith("#"):
|
||||
return None
|
||||
return s.split()
|
||||
|
||||
|
||||
def _normalize_pmon_command(command_fields: list[str]) -> str:
|
||||
"""Normalize pmon command fields into a stable process-ish name.
|
||||
|
||||
`nvidia-smi pmon -o DT` emits fixed numeric columns followed by a command
|
||||
field that can include whitespace. We prefer the *first* non-option token
|
||||
(usually executable) and normalize it to a basename.
|
||||
"""
|
||||
tokens = [token.strip().strip("\"'") for token in command_fields if token.strip()]
|
||||
if not tokens:
|
||||
return "unknown"
|
||||
|
||||
selected = tokens[0]
|
||||
if selected.startswith("-"):
|
||||
for candidate in tokens[1:]:
|
||||
if not candidate.startswith("-"):
|
||||
selected = candidate
|
||||
break
|
||||
|
||||
name = Path(selected).name.strip(";,:")
|
||||
if not name:
|
||||
return "unknown"
|
||||
return name
|
||||
|
||||
|
||||
def _pid_comm_name(pid: int) -> str | None:
|
||||
"""Return `/proc/<pid>/comm` basename when available."""
|
||||
try:
|
||||
comm = Path(f"/proc/{pid}/comm").read_text(encoding="utf-8").strip()
|
||||
except OSError:
|
||||
return None
|
||||
return Path(comm).name if comm else None
|
||||
|
||||
|
||||
def _pmon_row_epoch(parts: list[str]) -> float | None:
|
||||
"""Local-time epoch of a pmon row from its `date`/`time` columns, or None.
|
||||
|
||||
pmon timestamps are naive local time (`YYYYMMDD HH:MM:SS`); `.astimezone()`
|
||||
attaches the local offset so the result is comparable to a `begin_epoch`
|
||||
derived the same way.
|
||||
"""
|
||||
try:
|
||||
stamp = _dt.datetime.strptime(
|
||||
f"{parts[0]} {parts[1]}",
|
||||
"%Y%m%d %H:%M:%S",
|
||||
).astimezone()
|
||||
except (ValueError, IndexError):
|
||||
return None
|
||||
return stamp.timestamp()
|
||||
|
||||
|
||||
def aggregate_pmon(
|
||||
log: Path,
|
||||
progress: _Progress,
|
||||
begin_epoch: float | None = None,
|
||||
) -> tuple[dict[str, GpuAgg], int]:
|
||||
"""Return `({program: GpuAgg}, sample_count)` from the pmon *log*.
|
||||
|
||||
When *begin_epoch* is set, rows timestamped before it are skipped so the
|
||||
first day of a "since last report" window starts at the previous run time.
|
||||
"""
|
||||
progress.start_stage("pmon log scan")
|
||||
agg: dict[str, GpuAgg] = {}
|
||||
samples = 0
|
||||
if not log.exists():
|
||||
progress.update(1.0)
|
||||
return agg, 0
|
||||
total_bytes = max(log.stat().st_size, 1)
|
||||
bytes_read = 0
|
||||
with log.open(encoding="utf-8") as fh:
|
||||
for line in fh:
|
||||
bytes_read += len(line)
|
||||
progress.update(min(bytes_read / total_bytes, 0.99))
|
||||
parts = _pmon_fields(line)
|
||||
if parts is None or len(parts) < _PMON_MIN_FIELDS:
|
||||
continue
|
||||
if begin_epoch is not None:
|
||||
row_epoch = _pmon_row_epoch(parts)
|
||||
if row_epoch is not None and row_epoch < begin_epoch:
|
||||
continue
|
||||
samples += _ingest_pmon_row(parts, agg)
|
||||
progress.update(1.0)
|
||||
return agg, samples
|
||||
|
||||
|
||||
def _ingest_pmon_row(parts: list[str], agg: dict[str, GpuAgg]) -> int:
|
||||
"""Fold a single pmon data row into *agg*; return 1 if consumed else 0."""
|
||||
# pmon -o DT fields:
|
||||
# date time gpu pid type sm mem enc dec jpg ofa command
|
||||
try:
|
||||
pid = int(parts[3])
|
||||
except ValueError:
|
||||
return 0
|
||||
sm_raw = parts[5]
|
||||
mem_raw = parts[6]
|
||||
command_fields = parts[11:]
|
||||
name = _normalize_pmon_command(command_fields)
|
||||
if name == "unknown":
|
||||
name = _pid_comm_name(pid) or "unknown"
|
||||
sm = float(sm_raw) if sm_raw != "-" else 0.0
|
||||
mem = float(mem_raw) if mem_raw != "-" else 0.0
|
||||
entry = agg.setdefault(name, GpuAgg(name=name))
|
||||
entry.sm_pct_sum += sm
|
||||
entry.mem_pct_sum += mem
|
||||
entry.samples += 1
|
||||
entry.pid_set.add(pid)
|
||||
entry.peak_sm_pct = max(entry.peak_sm_pct, sm)
|
||||
entry.peak_mem_pct = max(entry.peak_mem_pct, mem)
|
||||
return 1
|
||||
@ -1,181 +0,0 @@
|
||||
"""Markdown report assembly for usage_report (host, methodology, full report)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime as _dt
|
||||
import os
|
||||
from pathlib import Path
|
||||
import platform
|
||||
import re
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from _usage_report_atop import _run
|
||||
from _usage_report_format import _fmt_h
|
||||
from _usage_report_tables import _cpu_table, _gpu_table, _ram_table
|
||||
from _usage_report_types import _HZ, _PMON_INTERVAL_S, _Window
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from usage_report import _Aggregates
|
||||
|
||||
_PAGE_KB = os.sysconf("SC_PAGESIZE") // 1024 if hasattr(os, "sysconf") else 4
|
||||
|
||||
|
||||
def _host_profile() -> dict[str, str]:
|
||||
"""Collect a small bag of identifying facts about the host."""
|
||||
info: dict[str, str] = {
|
||||
"hostname": platform.node(),
|
||||
"kernel": platform.release(),
|
||||
"cpus_online": str(os.cpu_count() or 0),
|
||||
}
|
||||
try:
|
||||
with Path("/proc/cpuinfo").open(encoding="utf-8") as fh:
|
||||
for line in fh:
|
||||
if line.startswith("model name"):
|
||||
info["cpu_model"] = line.split(":", 1)[1].strip()
|
||||
break
|
||||
except OSError:
|
||||
pass
|
||||
try:
|
||||
with Path("/proc/meminfo").open(encoding="utf-8") as fh:
|
||||
for line in fh:
|
||||
if line.startswith("MemTotal:"):
|
||||
kb = int(re.findall(r"\d+", line)[0])
|
||||
info["memory_total_gib"] = f"{kb / 1024 / 1024:.1f}"
|
||||
break
|
||||
except (OSError, IndexError, ValueError):
|
||||
pass
|
||||
gpu = _run(
|
||||
[
|
||||
"nvidia-smi",
|
||||
"--query-gpu=name,memory.total",
|
||||
"--format=csv,noheader",
|
||||
],
|
||||
).strip()
|
||||
if gpu:
|
||||
info["gpu"] = gpu.replace("\n", "; ")
|
||||
return info
|
||||
|
||||
|
||||
def _fingerprint_section() -> list[str]:
|
||||
info = _host_profile()
|
||||
return [
|
||||
"## Host",
|
||||
"",
|
||||
*[f"- **{k}**: {v}" for k, v in info.items()],
|
||||
"",
|
||||
]
|
||||
|
||||
|
||||
def _methodology_section(
|
||||
atop_desc: str,
|
||||
pmon_desc: str,
|
||||
window: _Window,
|
||||
) -> list[str]:
|
||||
window_note = (
|
||||
f"- **Coverage window**: {_fmt_h(window.seconds)} "
|
||||
f"(sum of per-day atop coverage from first to last sample; excludes "
|
||||
f"any gap days where atop was not logging, and the final partial tick)."
|
||||
)
|
||||
interval_note = (
|
||||
f"- **atop sample interval (observed)**: {window.interval_s}s"
|
||||
if window.interval_s
|
||||
else "- **atop sample interval**: only one sample so far; interval unknown."
|
||||
)
|
||||
task_note = (
|
||||
"- atop's parseable output is **task-level** (threads get their own "
|
||||
"rows keyed by `/proc/<tid>/comm`); names like 'Main Thread' or "
|
||||
"'dxvk-frame' are usually Wine/game worker threads of one parent."
|
||||
)
|
||||
rss_note = (
|
||||
"- RSS is shared across threads of one process, so multiple rows "
|
||||
"with identical 'Peak RSS' almost certainly belong to a single "
|
||||
"parent. The RAM table dedupes by peak-RSS bucket and lists "
|
||||
"sibling thread names under `(+ siblings)`."
|
||||
)
|
||||
cpu_note = (
|
||||
"- **CPU-seconds** are computed per-PID as "
|
||||
"`last_cumulative_ticks - first_cumulative_ticks` (or the cumulative "
|
||||
"value itself for PIDs seen only once). They reflect CPU consumed "
|
||||
"during the coverage window only, not since process start."
|
||||
)
|
||||
gpu_note = (
|
||||
"- GPU SM-seconds = sum(sm% per sample) \u00d7 sample interval / 100; "
|
||||
"single-GPU equivalent."
|
||||
)
|
||||
prog_note = (
|
||||
"- 'Program' = executable/thread name; rows with the same name "
|
||||
"are summed across their distinct PIDs."
|
||||
)
|
||||
return [
|
||||
"## Methodology",
|
||||
"",
|
||||
f"- **atop log(s)**: {atop_desc}",
|
||||
f"- **pmon log(s)**: {pmon_desc}",
|
||||
f"- **HZ**: {_HZ} ticks/s; **page size**: {_PAGE_KB} KiB",
|
||||
window_note,
|
||||
interval_note,
|
||||
cpu_note,
|
||||
task_note,
|
||||
rss_note,
|
||||
gpu_note,
|
||||
prog_note,
|
||||
"",
|
||||
]
|
||||
|
||||
|
||||
_LLM_PROMPT = [
|
||||
"> Below is aggregated resource usage for my Linux workstation over the",
|
||||
"> reporting period shown above. Identify which programs are the biggest",
|
||||
"> hogs, flag anything that looks abnormal for a typical developer/gaming",
|
||||
"> setup, and suggest concrete optimisations (config tweaks, process limits,",
|
||||
"> alternative tools). Be specific.",
|
||||
]
|
||||
|
||||
|
||||
def _render_report(
|
||||
aggs: _Aggregates,
|
||||
*,
|
||||
top: int,
|
||||
atop_desc: str,
|
||||
pmon_desc: str,
|
||||
period_line: str,
|
||||
) -> str:
|
||||
"""Assemble the Markdown report from already-aggregated data."""
|
||||
window = aggs.window
|
||||
gpu_section = (
|
||||
_gpu_table(aggs.gpu, aggs.gpu_samples, top)
|
||||
if aggs.gpu
|
||||
else ["_No GPU pmon data found._"]
|
||||
)
|
||||
generated = _dt.datetime.now().astimezone().isoformat(timespec="seconds")
|
||||
interval = f"{window.interval_s}s" if window.interval_s else "n/a (single sample)"
|
||||
lines: list[str] = [
|
||||
"# System resource usage report",
|
||||
"",
|
||||
f"- **Generated**: {generated}",
|
||||
period_line,
|
||||
f"- **atop window**: {window.start} → {window.end}",
|
||||
f"- **atop samples**: {window.distinct_samples} distinct "
|
||||
f"timestamps (sample interval ≈ {interval})",
|
||||
f"- **GPU pmon samples**: {aggs.gpu_samples} (≈{_PMON_INTERVAL_S}s each)",
|
||||
"",
|
||||
*_fingerprint_section(),
|
||||
*_methodology_section(atop_desc, pmon_desc, window),
|
||||
"## Top CPU consumers",
|
||||
"",
|
||||
*_cpu_table(aggs.cpu.values(), window.seconds, top),
|
||||
"",
|
||||
"## Top RAM consumers (by peak RSS, deduped by shared-memory bucket)",
|
||||
"",
|
||||
*_ram_table(aggs.cpu.values(), top),
|
||||
"",
|
||||
"## Top GPU consumers",
|
||||
"",
|
||||
*gpu_section,
|
||||
"",
|
||||
"## Suggested LLM prompt",
|
||||
"",
|
||||
*_LLM_PROMPT,
|
||||
"",
|
||||
]
|
||||
return "\n".join(lines) + "\n"
|
||||
@ -1,182 +0,0 @@
|
||||
"""The two reporting runs usage_report dispatches between.
|
||||
|
||||
`_run_single_day` covers one calendar day ad hoc; `_run_since` spans every
|
||||
daily log from the last report to now and advances the saved timestamp.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
import datetime as _dt
|
||||
import shutil
|
||||
import sys
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from _usage_report_emit import _INSTALL_SCRIPT, _emit, _preflight
|
||||
from _usage_report_format import _fmt_h
|
||||
from _usage_report_logs import (
|
||||
_log_descriptions,
|
||||
_plan_segments,
|
||||
_read_last_generated,
|
||||
_resolve_logs,
|
||||
_Segment,
|
||||
_write_last_generated,
|
||||
)
|
||||
from _usage_report_parsing import (
|
||||
aggregate_atop,
|
||||
merge_gpu_aggs,
|
||||
merge_proc_aggs,
|
||||
merge_windows,
|
||||
)
|
||||
from _usage_report_pmon import aggregate_pmon
|
||||
from _usage_report_render import _render_report
|
||||
from _usage_report_types import GpuAgg, ProcAgg, _Progress, _Window
|
||||
|
||||
_SEC_PER_DAY = 86_400
|
||||
|
||||
if TYPE_CHECKING:
|
||||
import argparse
|
||||
|
||||
|
||||
_REPORT_STAGES = 2
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Aggregates:
|
||||
"""Merged CPU/GPU aggregates and coverage window for a reporting window.
|
||||
|
||||
*days_with_data* is the number of daily logs that actually yielded atop
|
||||
samples (gap days where the machine was off contribute nothing).
|
||||
"""
|
||||
|
||||
cpu: dict[str, ProcAgg]
|
||||
gpu: dict[str, GpuAgg]
|
||||
window: _Window
|
||||
gpu_samples: int
|
||||
days_with_data: int
|
||||
|
||||
|
||||
def _aggregate_segments(
|
||||
segments: list[_Segment],
|
||||
progress: _Progress,
|
||||
) -> _Aggregates:
|
||||
"""Aggregate and merge every existing daily log in *segments*.
|
||||
|
||||
Missing daily logs (gap days) are skipped silently.
|
||||
"""
|
||||
cpu_total: dict[str, ProcAgg] = {}
|
||||
gpu_total: dict[str, GpuAgg] = {}
|
||||
windows: list[_Window] = []
|
||||
gpu_samples = 0
|
||||
days_with_data = 0
|
||||
for seg in segments:
|
||||
if seg.atop_log.exists():
|
||||
cpu, window = aggregate_atop(seg.atop_log, progress, seg.atop_begin)
|
||||
merge_proc_aggs(cpu_total, cpu)
|
||||
if window.distinct_samples:
|
||||
windows.append(window)
|
||||
days_with_data += 1
|
||||
gpu, samples = aggregate_pmon(seg.pmon_log, progress, seg.pmon_begin_epoch)
|
||||
merge_gpu_aggs(gpu_total, gpu)
|
||||
gpu_samples += samples
|
||||
return _Aggregates(
|
||||
cpu_total,
|
||||
gpu_total,
|
||||
merge_windows(windows),
|
||||
gpu_samples,
|
||||
days_with_data,
|
||||
)
|
||||
|
||||
|
||||
def _period_line(start: _dt.datetime, end: _dt.datetime) -> str:
|
||||
"""Markdown bullet describing the requested reporting period."""
|
||||
span = _fmt_h(max((end - start).total_seconds(), 0.0))
|
||||
return (
|
||||
f"- **Reporting period**: {start.isoformat(timespec='seconds')} → "
|
||||
f"{end.isoformat(timespec='seconds')} ({span})"
|
||||
)
|
||||
|
||||
|
||||
def _should_advance_state(args: argparse.Namespace) -> bool:
|
||||
"""Advance the saved timestamp only for genuine since-last-report runs.
|
||||
|
||||
An explicit ``--since`` is treated as a read-only ad-hoc query (like
|
||||
``--date``) so "let me look from date X" never silently re-baselines the
|
||||
saved tracking point.
|
||||
"""
|
||||
return args.since is None and not args.no_update_state
|
||||
|
||||
|
||||
def _run_single_day(args: argparse.Namespace, now: _dt.datetime) -> int:
|
||||
"""Report on one specific day (legacy behaviour); never touches state."""
|
||||
date = args.date or now.strftime("%Y%m%d")
|
||||
atop_default, pmon_default = _resolve_logs(date)
|
||||
atop_log = args.atop_log or atop_default
|
||||
pmon_log = args.pmon_log or pmon_default
|
||||
_preflight(atop_log)
|
||||
segment = _Segment(atop_log, pmon_log)
|
||||
progress = _Progress(enabled=not args.quiet, total_stages=_REPORT_STAGES)
|
||||
aggs = _aggregate_segments([segment], progress)
|
||||
progress.finish()
|
||||
if not aggs.window.seconds:
|
||||
aggs.window.seconds = _SEC_PER_DAY
|
||||
atop_desc, pmon_desc = _log_descriptions([segment])
|
||||
_emit(
|
||||
args,
|
||||
_render_report(
|
||||
aggs,
|
||||
top=args.top,
|
||||
atop_desc=atop_desc,
|
||||
pmon_desc=pmon_desc,
|
||||
period_line=f"- **Reporting period**: {date} (single day)",
|
||||
),
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
def _resolve_start(args: argparse.Namespace, now: _dt.datetime) -> _dt.datetime:
|
||||
"""Pick the window start: --since, else last report, else today midnight."""
|
||||
if args.since is not None:
|
||||
return _dt.datetime.strptime(args.since, "%Y%m%d").astimezone()
|
||||
last = _read_last_generated()
|
||||
if last is not None:
|
||||
return last
|
||||
return now.replace(hour=0, minute=0, second=0, microsecond=0)
|
||||
|
||||
|
||||
def _run_since(args: argparse.Namespace, now: _dt.datetime) -> int:
|
||||
"""Report on everything since the last run, spanning multiple daily logs."""
|
||||
if not shutil.which("atop"):
|
||||
sys.exit(f"error: `atop` is not installed.\nrun: {_INSTALL_SCRIPT}")
|
||||
start = _resolve_start(args, now)
|
||||
segments = _plan_segments(start, now)
|
||||
progress = _Progress(
|
||||
enabled=not args.quiet,
|
||||
total_stages=max(2 * len(segments), 1),
|
||||
)
|
||||
aggs = _aggregate_segments(segments, progress)
|
||||
progress.finish()
|
||||
if aggs.days_with_data == 0:
|
||||
sys.stderr.write(
|
||||
f"no atop logs with data for {start.date()} … {now.date()}; "
|
||||
"nothing to report.\n",
|
||||
)
|
||||
if _should_advance_state(args):
|
||||
_write_last_generated(now)
|
||||
return 0
|
||||
if not aggs.window.seconds:
|
||||
aggs.window.seconds = _SEC_PER_DAY
|
||||
atop_desc, pmon_desc = _log_descriptions(segments)
|
||||
_emit(
|
||||
args,
|
||||
_render_report(
|
||||
aggs,
|
||||
top=args.top,
|
||||
atop_desc=atop_desc,
|
||||
pmon_desc=pmon_desc,
|
||||
period_line=_period_line(start, now),
|
||||
),
|
||||
)
|
||||
if _should_advance_state(args):
|
||||
_write_last_generated(now)
|
||||
return 0
|
||||
@ -1,123 +0,0 @@
|
||||
"""Markdown table builders for usage_report (CPU, RAM and GPU sections)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections import defaultdict
|
||||
import os
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from _usage_report_format import _fmt_h, _md_escape
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Iterable
|
||||
|
||||
from _usage_report_types import GpuAgg, ProcAgg
|
||||
|
||||
_RAM_BUCKET_MIB = 1 # dedupe rows whose peak RSS rounds to the same MiB
|
||||
_MAX_SIBLINGS_SHOWN = 6
|
||||
|
||||
|
||||
def _cpu_table(aggs: Iterable[ProcAgg], window_s: int, top: int) -> list[str]:
|
||||
ncpu = os.cpu_count() or 1
|
||||
header = (
|
||||
"| # | Program | CPU-seconds | Avg CPU% (of 1 core) |"
|
||||
" Avg CPU% (of box) | Peak RSS | PIDs |"
|
||||
)
|
||||
sep = (
|
||||
"|---|---------|------------:|---------------------:|"
|
||||
"------------------:|---------:|-----:|"
|
||||
)
|
||||
rows: list[str] = [header, sep]
|
||||
top_items = sorted(aggs, key=lambda a: a.cpu_ticks, reverse=True)[:top]
|
||||
for idx, item in enumerate(top_items, start=1):
|
||||
single = (item.cpu_seconds / window_s * 100) if window_s else 0.0
|
||||
box = single / ncpu
|
||||
rows.append(
|
||||
"| "
|
||||
f"{idx} | {_md_escape(item.name)} | "
|
||||
f"{item.cpu_seconds:,.0f}s ({_fmt_h(item.cpu_seconds)}) | "
|
||||
f"{single:.1f}% | {box:.1f}% | "
|
||||
f"{item.peak_rss_mb:,.0f} MiB | {item.pid_count} |",
|
||||
)
|
||||
return rows
|
||||
|
||||
|
||||
def _dedupe_ram(aggs: Iterable[ProcAgg]) -> list[tuple[ProcAgg, list[str]]]:
|
||||
"""Group rows by peak-RSS bucket; keep the top-CPU row per bucket.
|
||||
|
||||
Returns a list of `(representative, sibling_names)` ordered by peak RSS
|
||||
descending. Siblings are the other names that shared the same RSS bucket
|
||||
(likely threads of the same parent process).
|
||||
"""
|
||||
buckets: dict[int, list[ProcAgg]] = defaultdict(list)
|
||||
for item in aggs:
|
||||
if item.peak_rss_kb <= 0:
|
||||
continue
|
||||
key = round(item.peak_rss_kb / 1024 / _RAM_BUCKET_MIB)
|
||||
buckets[key].append(item)
|
||||
result: list[tuple[ProcAgg, list[str]]] = []
|
||||
for bucket in buckets.values():
|
||||
bucket.sort(key=lambda a: (a.cpu_ticks, a.pid_count), reverse=True)
|
||||
rep = bucket[0]
|
||||
siblings = [b.name for b in bucket[1:]]
|
||||
result.append((rep, siblings))
|
||||
result.sort(key=lambda t: t[0].peak_rss_kb, reverse=True)
|
||||
return result
|
||||
|
||||
|
||||
def _ram_table(aggs: Iterable[ProcAgg], top: int) -> list[str]:
|
||||
header = (
|
||||
"| # | Program | Peak RSS | Avg RSS | CPU-seconds | PIDs |"
|
||||
" Sibling names (shared RSS) |"
|
||||
)
|
||||
sep = (
|
||||
"|---|---------|---------:|--------:|------------:|-----:|"
|
||||
"----------------------------|"
|
||||
)
|
||||
rows: list[str] = [header, sep]
|
||||
for idx, (item, siblings) in enumerate(_dedupe_ram(aggs)[:top], start=1):
|
||||
if not siblings:
|
||||
sib = "—"
|
||||
else:
|
||||
shown = ", ".join(_md_escape(s) for s in siblings[:_MAX_SIBLINGS_SHOWN])
|
||||
extra = (
|
||||
f" (+{len(siblings) - _MAX_SIBLINGS_SHOWN} more)"
|
||||
if len(siblings) > _MAX_SIBLINGS_SHOWN
|
||||
else ""
|
||||
)
|
||||
sib = f"{shown}{extra}"
|
||||
rows.append(
|
||||
"| "
|
||||
f"{idx} | {_md_escape(item.name)} | "
|
||||
f"{item.peak_rss_mb:,.0f} MiB | "
|
||||
f"{item.avg_rss_mb:,.0f} MiB | "
|
||||
f"{item.cpu_seconds:,.0f}s | "
|
||||
f"{item.pid_count} | {sib} |",
|
||||
)
|
||||
return rows
|
||||
|
||||
|
||||
def _gpu_table(aggs: dict[str, GpuAgg], total_samples: int, top: int) -> list[str]:
|
||||
header = (
|
||||
"| # | Program | GPU SM-seconds | Avg SM% (when present) |"
|
||||
" Peak SM% | Peak MEM% | Samples | PIDs |"
|
||||
)
|
||||
sep = (
|
||||
"|---|---------|---------------:|-----------------------:|"
|
||||
"---------:|----------:|--------:|-----:|"
|
||||
)
|
||||
rows: list[str] = [header, sep]
|
||||
top_items = sorted(aggs.values(), key=lambda a: a.gpu_seconds, reverse=True)[:top]
|
||||
for idx, item in enumerate(top_items, start=1):
|
||||
presence = (item.samples / total_samples * 100) if total_samples else 0.0
|
||||
rows.append(
|
||||
"| "
|
||||
f"{idx} | {_md_escape(item.name)} | "
|
||||
f"{item.gpu_seconds:,.0f}s ({_fmt_h(item.gpu_seconds)}) | "
|
||||
f"{item.avg_sm_pct:.1f}% | "
|
||||
f"{item.peak_sm_pct:.0f}% | "
|
||||
f"{item.peak_mem_pct:.0f}% | "
|
||||
f"{item.samples} ({presence:.0f}%) | "
|
||||
f"{item.pid_count} |",
|
||||
)
|
||||
return rows
|
||||
@ -1,212 +0,0 @@
|
||||
"""Shared data-class types and progress reporter for usage_report."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
import os
|
||||
import sys
|
||||
import time as _time
|
||||
|
||||
_HZ = os.sysconf("SC_CLK_TCK") if hasattr(os, "sysconf") else 100
|
||||
_MIN_SAMPLES_FOR_WINDOW = 2
|
||||
# Default pmon interval is 10 s (matches the systemd service we set up).
|
||||
_PMON_INTERVAL_S = 10
|
||||
_PROGRESS_MIN_UPDATE_S = 0.1
|
||||
_ETA_MIN_FRACTION = 0.01
|
||||
|
||||
|
||||
@dataclass
|
||||
class _PidCpu:
|
||||
"""Per-PID cumulative-ticks tracker across atop samples."""
|
||||
|
||||
name: str = ""
|
||||
first_ticks: int = -1
|
||||
last_ticks: int = 0
|
||||
samples: int = 0
|
||||
|
||||
def observe(self, name: str, ticks: int) -> None:
|
||||
"""Record one observation for this PID."""
|
||||
self.name = name # last-seen name wins (stable for one PID)
|
||||
if self.first_ticks < 0:
|
||||
self.first_ticks = ticks
|
||||
self.last_ticks = ticks
|
||||
self.samples += 1
|
||||
|
||||
@property
|
||||
def delta_ticks(self) -> int:
|
||||
"""CPU ticks consumed during the observation window.
|
||||
|
||||
For PIDs seen in >=2 samples the value is `last - first`, which is the
|
||||
actual CPU consumed between the first and last atop tick. For PIDs seen
|
||||
only once (short-lived processes that existed during exactly one tick)
|
||||
the cumulative value itself is used — this is close to the true
|
||||
lifetime cost for a short-lived process.
|
||||
"""
|
||||
if self.samples >= _MIN_SAMPLES_FOR_WINDOW:
|
||||
return max(self.last_ticks - self.first_ticks, 0)
|
||||
return self.last_ticks
|
||||
|
||||
|
||||
@dataclass
|
||||
class _PidRam:
|
||||
"""Per-PID peak/avg RSS tracker across atop samples."""
|
||||
|
||||
name: str = ""
|
||||
peak_kb: int = 0
|
||||
sum_kb: int = 0
|
||||
samples: int = 0
|
||||
|
||||
def observe(self, name: str, rss_kb: int) -> None:
|
||||
"""Record one RSS observation for this PID."""
|
||||
self.name = name
|
||||
self.peak_kb = max(self.peak_kb, rss_kb)
|
||||
self.sum_kb += rss_kb
|
||||
self.samples += 1
|
||||
|
||||
@property
|
||||
def avg_kb(self) -> float:
|
||||
"""Mean RSS across the samples where this PID appeared."""
|
||||
return self.sum_kb / self.samples if self.samples else 0.0
|
||||
|
||||
|
||||
@dataclass
|
||||
class ProcAgg:
|
||||
"""Aggregated metrics for one program name across all atop samples."""
|
||||
|
||||
name: str
|
||||
cpu_ticks: int = 0
|
||||
peak_rss_kb: int = 0
|
||||
rss_kb_sum: int = 0
|
||||
rss_samples: int = 0
|
||||
pid_set: set[int] = field(default_factory=set)
|
||||
# PID counts folded in when merging per-day aggregates. Tracked as a plain
|
||||
# integer (not by extending `pid_set`) because the native parser stores a
|
||||
# synthetic `range(n)` set whose union across days would collapse counts.
|
||||
extra_pids: int = 0
|
||||
|
||||
@property
|
||||
def pid_count(self) -> int:
|
||||
"""Distinct PIDs seen, including those merged from other day windows."""
|
||||
return len(self.pid_set) + self.extra_pids
|
||||
|
||||
@property
|
||||
def cpu_seconds(self) -> float:
|
||||
"""CPU-seconds consumed (sum of user + system time)."""
|
||||
return self.cpu_ticks / _HZ
|
||||
|
||||
@property
|
||||
def peak_rss_mb(self) -> float:
|
||||
"""Peak resident memory observed across the window, in MiB."""
|
||||
return self.peak_rss_kb / 1024
|
||||
|
||||
@property
|
||||
def avg_rss_mb(self) -> float:
|
||||
"""Average resident memory across samples where the program appeared."""
|
||||
if not self.rss_samples:
|
||||
return 0.0
|
||||
return (self.rss_kb_sum / self.rss_samples) / 1024
|
||||
|
||||
|
||||
@dataclass
|
||||
class GpuAgg:
|
||||
"""Aggregated GPU metrics for one program name from pmon logs."""
|
||||
|
||||
name: str
|
||||
sm_pct_sum: float = 0.0
|
||||
mem_pct_sum: float = 0.0
|
||||
samples: int = 0
|
||||
peak_sm_pct: float = 0.0
|
||||
peak_mem_pct: float = 0.0
|
||||
pid_set: set[int] = field(default_factory=set)
|
||||
# PID counts folded in when merging per-day aggregates (see ProcAgg).
|
||||
extra_pids: int = 0
|
||||
|
||||
@property
|
||||
def pid_count(self) -> int:
|
||||
"""Distinct PIDs seen, including those merged from other day windows."""
|
||||
return len(self.pid_set) + self.extra_pids
|
||||
|
||||
@property
|
||||
def gpu_seconds(self) -> float:
|
||||
"""SM-seconds (single-GPU equivalent); sm% * seconds_per_sample / 100."""
|
||||
return self.sm_pct_sum * _PMON_INTERVAL_S / 100.0
|
||||
|
||||
@property
|
||||
def avg_sm_pct(self) -> float:
|
||||
"""Mean SM utilization across samples where the process was present."""
|
||||
if not self.samples:
|
||||
return 0.0
|
||||
return self.sm_pct_sum / self.samples
|
||||
|
||||
|
||||
class _Progress:
|
||||
"""Minimal stage+percent+ETA reporter on stderr.
|
||||
|
||||
Disabled automatically when stderr is not a TTY or when the caller
|
||||
constructs with `enabled=False`, so redirected output stays clean.
|
||||
"""
|
||||
|
||||
def __init__(self, *, enabled: bool, total_stages: int) -> None:
|
||||
self._enabled = enabled and sys.stderr.isatty()
|
||||
self._total_stages = total_stages
|
||||
self._stage_idx = 0
|
||||
self._stage_label = ""
|
||||
self._stage_start = 0.0
|
||||
self._t0 = _time.monotonic()
|
||||
self._last_draw = 0.0
|
||||
self._max_width = 0
|
||||
|
||||
def start_stage(self, label: str) -> None:
|
||||
"""Begin a new stage with its human label."""
|
||||
self._stage_idx += 1
|
||||
self._stage_label = label
|
||||
self._stage_start = _time.monotonic()
|
||||
self.update(0.0)
|
||||
|
||||
def update(self, fraction: float) -> None:
|
||||
"""Redraw the progress line for the current stage (0.0..1.0)."""
|
||||
if not self._enabled:
|
||||
return
|
||||
now = _time.monotonic()
|
||||
if now - self._last_draw < _PROGRESS_MIN_UPDATE_S and fraction < 1.0:
|
||||
return
|
||||
self._last_draw = now
|
||||
elapsed = now - self._stage_start
|
||||
pct = max(0.0, min(fraction, 1.0))
|
||||
if pct > _ETA_MIN_FRACTION:
|
||||
eta = elapsed * (1 - pct) / pct
|
||||
eta_str = f"~{eta:4.1f}s left"
|
||||
else:
|
||||
eta_str = "estimating…"
|
||||
msg = (
|
||||
f"[{self._stage_idx}/{self._total_stages}] "
|
||||
f"{self._stage_label:<22} {pct * 100:5.1f}% "
|
||||
f"{elapsed:5.1f}s elapsed, {eta_str}"
|
||||
)
|
||||
self._max_width = max(self._max_width, len(msg))
|
||||
sys.stderr.write("\r" + msg.ljust(self._max_width))
|
||||
sys.stderr.flush()
|
||||
|
||||
def finish(self) -> None:
|
||||
"""Clear the progress line and print total elapsed time."""
|
||||
if not self._enabled:
|
||||
return
|
||||
total = _time.monotonic() - self._t0
|
||||
sys.stderr.write("\r" + " " * self._max_width + "\r")
|
||||
sys.stderr.write(f"done in {total:.1f}s\n")
|
||||
sys.stderr.flush()
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Window:
|
||||
"""Observed atop coverage window."""
|
||||
|
||||
start: str = "n/a"
|
||||
end: str = "n/a"
|
||||
distinct_samples: int = 0
|
||||
interval_s: int = 0
|
||||
seconds: int = 0
|
||||
# Raw epoch bounds, kept so multiple per-day windows can be merged by
|
||||
# min(start)/max(end) without re-parsing the ISO strings above.
|
||||
start_epoch: int = 0
|
||||
end_epoch: int = 0
|
||||
@ -1,51 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Automatic system update script for Arch Linux
|
||||
# Runs pacman -Syuu and yay -Sua non-interactively
|
||||
# This file is installed by setup_periodic_system.sh
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
readonly LOG_FILE="/var/log/auto-system-update.log"
|
||||
readonly LOCK_FILE="/var/lock/auto-system-update.lock"
|
||||
readonly ACTUAL_USER="__ACTUAL_USER__"
|
||||
|
||||
log_msg() {
|
||||
echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE" >&2
|
||||
}
|
||||
|
||||
cleanup() {
|
||||
rm -f "$LOCK_FILE"
|
||||
}
|
||||
|
||||
trap cleanup EXIT
|
||||
|
||||
# Prevent concurrent runs
|
||||
if ! (set -o noclobber && echo $$ > "$LOCK_FILE") 2>/dev/null; then
|
||||
log_msg "Another update is already running (lock: $LOCK_FILE). Exiting."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
log_msg "=== Automatic System Update Started ==="
|
||||
|
||||
# --- Official repository update (pacman) ---
|
||||
log_msg "Running pacman -Syuu --noconfirm ..."
|
||||
if /usr/bin/pacman -Syuu --noconfirm >> "$LOG_FILE" 2>&1; then
|
||||
log_msg "pacman update completed successfully"
|
||||
else
|
||||
log_msg "pacman update failed (exit $?)"
|
||||
fi
|
||||
|
||||
# --- AUR update (yay) ---
|
||||
# yay must not run as root; run as the actual user
|
||||
if command -v /usr/bin/yay > /dev/null 2>&1; then
|
||||
log_msg "Running yay -Sua --noconfirm as $ACTUAL_USER ..."
|
||||
if sudo -u "$ACTUAL_USER" /usr/bin/yay -Sua --noconfirm 2>&1 | tee -a "$LOG_FILE" > /dev/null; then
|
||||
log_msg "yay AUR update completed successfully"
|
||||
else
|
||||
log_msg "yay AUR update failed (exit $?)"
|
||||
fi
|
||||
else
|
||||
log_msg "yay not found, skipping AUR updates"
|
||||
fi
|
||||
|
||||
log_msg "=== Automatic System Update Completed ==="
|
||||
@ -1,191 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Generic pre-exec wrapper for browsers: ensures /etc/hosts is (re)installed
|
||||
# before launching the actual browser binary, and — for Chromium-family
|
||||
# browsers — captures a crash log for every launch.
|
||||
#
|
||||
# Why the crash logging exists (diagnosed 2026-08-09):
|
||||
# Thorium was "closing and dying randomly" with NO evidence anywhere:
|
||||
# Crashpad was never enabled (no crash_reporting consent key, empty crash DB),
|
||||
# and stderr from a dmenu/.desktop launch goes to no terminal and no journal.
|
||||
# The only Thorium lines ever seen in the journal came from a systemd-launched
|
||||
# instance. So the browser could die repeatedly and leave nothing behind.
|
||||
#
|
||||
# Each launch now gets its own log holding: a VRAM/RAM snapshot before start,
|
||||
# the browser's own stderr (--enable-logging=stderr --v=1), and — on exit —
|
||||
# the exit code, decoded signal, and a SECOND VRAM snapshot. That last pair is
|
||||
# the discriminator between the two live hypotheses: a GPU/VRAM death shows
|
||||
# NVRM/NV_ERR_NO_MEMORY or near-full VRAM at exit, whereas a stale-binary
|
||||
# death shows SIGSEGV/SIGILL with the GPU idle.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
HOSTS_INSTALL_SCRIPT="__HOSTS_INSTALL_SCRIPT__"
|
||||
|
||||
# Where per-launch logs go, and how many to keep.
|
||||
readonly LOG_DIR="${XDG_STATE_HOME:-$HOME/.local/state}/browser-crash-logs"
|
||||
readonly LOG_KEEP=20
|
||||
|
||||
# Per-log byte cap. Measured 2026-08-09: Thorium writes ~2 MB in the first 45s
|
||||
# (65% of it VERBOSE1 net/TLS chatter) and ~0 while idle, so cost scales with
|
||||
# navigation, not wall-clock. Capping COUNT alone is therefore not enough — a
|
||||
# heavy browsing day could fill the disk, and a full disk causes exactly the
|
||||
# kind of random death this logging exists to diagnose.
|
||||
#
|
||||
# Enforced by the watchdog below, which keeps the NEWEST bytes: the lines
|
||||
# immediately before death are the diagnostic ones, so trimming must drop the
|
||||
# head, never the tail.
|
||||
readonly LOG_MAX_BYTES=$((32 * 1024 * 1024))
|
||||
|
||||
prog_name="$(basename "$0")"
|
||||
real_bin="/usr/bin/${prog_name}"
|
||||
|
||||
# If run directly (not via a browser symlink) or if the target binary doesn't exist,
|
||||
# allow passing the real browser command as the first argument for testing:
|
||||
if [[ ! -x $real_bin || $prog_name == "browser-preexec-wrapper.sh" ]]; then
|
||||
if [[ $# -ge 1 ]]; then
|
||||
real_bin="$1"
|
||||
shift
|
||||
else
|
||||
echo "Error: could not resolve real browser binary for '$prog_name'." >&2
|
||||
echo "Usage (testing): $0 <real-browser-command> [args...]" >&2
|
||||
echo "Typical install: symlink this script as /usr/local/bin/<browser> so it wraps /usr/bin/<browser>." >&2
|
||||
exit 127
|
||||
fi
|
||||
fi
|
||||
|
||||
# Best-effort: install hosts file quietly; don't block browser startup
|
||||
if command -v sudo >/dev/null 2>&1; then
|
||||
sudo -n "$HOSTS_INSTALL_SCRIPT" >/dev/null 2>&1 || true
|
||||
else
|
||||
"$HOSTS_INSTALL_SCRIPT" >/dev/null 2>&1 || true
|
||||
fi
|
||||
|
||||
# Only Chromium-family browsers understand --enable-logging/--v; everything else
|
||||
# (firefox, librewolf) is exec'd unchanged so this wrapper stays generic.
|
||||
is_chromium_family() {
|
||||
case "$prog_name" in
|
||||
thorium-browser | google-chrome | google-chrome-stable | chromium | brave | brave-browser | vivaldi-stable)
|
||||
return 0
|
||||
;;
|
||||
*) return 1 ;;
|
||||
esac
|
||||
}
|
||||
|
||||
# GPU + RAM state. Written before launch and again at exit so the two can be
|
||||
# compared; nvidia-smi may be absent (or the GPU wedged), so never fail here.
|
||||
emit_resource_snapshot() {
|
||||
local label="$1"
|
||||
echo "--- ${label} $(date '+%Y-%m-%d %H:%M:%S') ---"
|
||||
if command -v nvidia-smi >/dev/null 2>&1; then
|
||||
echo -n "VRAM used/total: "
|
||||
nvidia-smi --query-gpu=memory.used,memory.total --format=csv,noheader 2>&1 || echo "nvidia-smi failed"
|
||||
else
|
||||
echo "VRAM: nvidia-smi unavailable"
|
||||
fi
|
||||
echo -n "RAM: "
|
||||
free -m | awk '/^Mem:/ {printf "%s MiB used / %s MiB total\n", $3, $2}'
|
||||
echo "kernel: $(uname -r)"
|
||||
# Match the dotted version anywhere on the line rather than by field index:
|
||||
# the surrounding wording differs between open/proprietary driver builds.
|
||||
# Worth logging because a module/userspace version skew (nvidia-utils upgraded
|
||||
# without a reboot) is a known cause of GL crashes in Chromium.
|
||||
if [[ -r /proc/driver/nvidia/version ]]; then
|
||||
echo -n "nvidia module: "
|
||||
grep -oE '[0-9]+\.[0-9]+\.[0-9]+' /proc/driver/nvidia/version 2>/dev/null | head -1 || echo "unreadable"
|
||||
fi
|
||||
}
|
||||
|
||||
# Keep only the newest $keep logs. Uses -printf/sort -z so filenames containing
|
||||
# spaces or newlines can't split a record. Called with LOG_KEEP-1 before a new
|
||||
# log is created, so the directory settles at exactly LOG_KEEP afterwards.
|
||||
rotate_logs() {
|
||||
local keep="$1"
|
||||
local old
|
||||
while IFS= read -r -d '' old; do
|
||||
rm -f -- "$old"
|
||||
done < <(
|
||||
find "$LOG_DIR" -maxdepth 1 -type f -name '*.log' -printf '%T@\t%p\0' 2>/dev/null |
|
||||
sort -zrn |
|
||||
tail -zn "+$((keep + 1))" |
|
||||
cut -zf2-
|
||||
)
|
||||
}
|
||||
|
||||
if ! is_chromium_family; then
|
||||
exec "$real_bin" "$@"
|
||||
fi
|
||||
|
||||
mkdir -p "$LOG_DIR"
|
||||
rotate_logs "$((LOG_KEEP - 1))"
|
||||
log_file="${LOG_DIR}/${prog_name}-$(date +%Y%m%d-%H%M%S)-$$.log"
|
||||
|
||||
{
|
||||
echo "=== launch $(date '+%Y-%m-%d %H:%M:%S') ==="
|
||||
echo "binary: $real_bin"
|
||||
echo "args: $*"
|
||||
emit_resource_snapshot "PRE-LAUNCH"
|
||||
echo "=== browser stderr follows ==="
|
||||
} >"$log_file"
|
||||
|
||||
# Written from a trap rather than inline: if the wrapper itself is killed (the
|
||||
# session ends, the process group is torn down, the OOM killer picks it), an
|
||||
# inline block after `wait` would never run and the log would end mid-stream
|
||||
# with no verdict — indistinguishable from a browser still running.
|
||||
record_exit() {
|
||||
local code="$1"
|
||||
# Fire once; EXIT still runs after a signal trap has already reported.
|
||||
[[ -n ${exit_recorded:-} ]] && return 0
|
||||
exit_recorded=1
|
||||
{
|
||||
echo "=== exit ==="
|
||||
echo "exit code: $code"
|
||||
if ((code > 128)); then
|
||||
local signal_num=$((code - 128))
|
||||
echo "died on signal: ${signal_num} ($(kill -l "$signal_num" 2>/dev/null || echo unknown))"
|
||||
fi
|
||||
emit_resource_snapshot "POST-EXIT"
|
||||
} >>"$log_file"
|
||||
}
|
||||
|
||||
# Run in the background rather than exec'ing, so this shell survives to record
|
||||
# how the browser died. `wait` reports 128+N for a signal death, which is the
|
||||
# single most useful fact the old setup threw away.
|
||||
#
|
||||
# Deliberately a direct redirect, NOT a pipe into a size-limiting filter: in a
|
||||
# pipeline `wait` reports the LAST element's status, which would report the
|
||||
# filter's exit and discard the browser's signal — the one fact this whole
|
||||
# wrapper exists to capture. Size is bounded by the watchdog below instead.
|
||||
"$real_bin" --enable-logging=stderr --v=1 "$@" >>"$log_file" 2>&1 &
|
||||
browser_pid=$!
|
||||
|
||||
# Watchdog: if a browsing session makes the log exceed the cap, keep the most
|
||||
# recent half and continue appending. Checks once a minute — the measured burst
|
||||
# rate (~2 MB/45s of navigation, ~0 idle) can't overshoot 32 MB meaningfully in
|
||||
# that window, and a 1/min poll costs nothing next to the browser itself.
|
||||
(
|
||||
while kill -0 "$browser_pid" 2>/dev/null; do
|
||||
sleep 60
|
||||
size="$(stat -c %s "$log_file" 2>/dev/null || echo 0)"
|
||||
if ((size > LOG_MAX_BYTES)); then
|
||||
tail -c "$((LOG_MAX_BYTES / 2))" "$log_file" >"${log_file}.trim" 2>/dev/null &&
|
||||
mv "${log_file}.trim" "$log_file" &&
|
||||
echo "=== log trimmed to newest $((LOG_MAX_BYTES / 2)) bytes at $(date '+%H:%M:%S') ===" >>"$log_file"
|
||||
fi
|
||||
done
|
||||
) &
|
||||
watchdog_pid=$!
|
||||
|
||||
# A signal aimed at the wrapper is recorded as the wrapper's own death, and the
|
||||
# browser is taken down with it so no orphan outlives the log describing it.
|
||||
# Written inline (not via a helper) so shellcheck can see the calls; each then
|
||||
# re-raises with the default handler so the exit status stays truthful.
|
||||
trap 'record_exit 129; kill "$browser_pid" "$watchdog_pid" 2>/dev/null || true; trap - HUP; kill -s HUP $$' HUP
|
||||
trap 'record_exit 130; kill "$browser_pid" "$watchdog_pid" 2>/dev/null || true; trap - INT; kill -s INT $$' INT
|
||||
trap 'record_exit 143; kill "$browser_pid" "$watchdog_pid" 2>/dev/null || true; trap - TERM; kill -s TERM $$' TERM
|
||||
|
||||
exit_code=0
|
||||
wait "$browser_pid" || exit_code=$?
|
||||
|
||||
kill "$watchdog_pid" 2>/dev/null || true
|
||||
record_exit "$exit_code"
|
||||
exit "$exit_code"
|
||||
@ -1,170 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Hosts file monitor script
|
||||
# Watches /etc/hosts for changes and restores it if needed
|
||||
# This file is installed by setup_periodic_system.sh
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
LOG_FILE="/var/log/hosts-file-monitor.log"
|
||||
HOSTS_FILE="/etc/hosts"
|
||||
HOSTS_INSTALL_SCRIPT="__HOSTS_INSTALL_SCRIPT__"
|
||||
readonly MIN_HOSTS_LINES=1000
|
||||
readonly EVENT_COOLDOWN_S=5
|
||||
|
||||
current_epoch() {
|
||||
local out_var="${1:-}"
|
||||
if [[ -n $out_var ]]; then
|
||||
printf -v "$out_var" '%(%s)T' -1
|
||||
else
|
||||
printf '%(%s)T\n' -1
|
||||
fi
|
||||
}
|
||||
|
||||
wait_seconds() {
|
||||
local timeout_s=$1
|
||||
local start_ts end_ts elapsed_s remaining_s
|
||||
|
||||
printf -v start_ts '%(%s)T' -1
|
||||
IFS= read -r -t "$timeout_s" || true
|
||||
printf -v end_ts '%(%s)T' -1
|
||||
|
||||
elapsed_s=$((end_ts - start_ts))
|
||||
if (( elapsed_s < timeout_s )); then
|
||||
remaining_s=$((timeout_s - elapsed_s))
|
||||
sleep "$remaining_s"
|
||||
fi
|
||||
}
|
||||
|
||||
# Log with timestamp (hosts-file-monitor specific)
|
||||
log_message() {
|
||||
local _ts
|
||||
local msg
|
||||
printf -v _ts '%(%Y-%m-%d %H:%M:%S)T' -1
|
||||
printf -v msg '%s [hosts-monitor] %s' "$_ts" "$1"
|
||||
printf '%s\n' "$msg" >&2
|
||||
printf '%s\n' "$msg" >> "$LOG_FILE" 2>/dev/null || true
|
||||
}
|
||||
|
||||
# Function to check if hosts file needs restoration
|
||||
needs_restoration() {
|
||||
# Check if file exists
|
||||
if [[ ! -f $HOSTS_FILE ]]; then
|
||||
return 0 # File missing, needs restoration
|
||||
fi
|
||||
|
||||
# Check if file is empty or too small (less than MIN_HOSTS_LINES indicates tampering)
|
||||
local line_count=0
|
||||
local has_custom_entries=0
|
||||
local has_stevenblack_entries=0
|
||||
local line=""
|
||||
|
||||
while IFS= read -r line || [[ -n $line ]]; do
|
||||
line_count=$((line_count + 1))
|
||||
|
||||
if [[ $has_custom_entries -eq 0 && $line == *"Custom blocking entries"* ]]; then
|
||||
has_custom_entries=1
|
||||
fi
|
||||
|
||||
if [[ $has_stevenblack_entries -eq 0 && $line == *"StevenBlack"* ]]; then
|
||||
has_stevenblack_entries=1
|
||||
fi
|
||||
|
||||
if (( line_count >= MIN_HOSTS_LINES && has_custom_entries == 1 && has_stevenblack_entries == 1 )); then
|
||||
return 1 # File seems intact
|
||||
fi
|
||||
done < "$HOSTS_FILE"
|
||||
|
||||
if [[ $line_count -lt $MIN_HOSTS_LINES ]]; then
|
||||
return 0 # File too small, likely tampered with
|
||||
fi
|
||||
|
||||
# Check if our custom entries are missing
|
||||
if [[ $has_custom_entries -eq 0 ]]; then
|
||||
return 0 # Our custom entries missing, needs restoration
|
||||
fi
|
||||
|
||||
# Check if StevenBlack entries are missing
|
||||
if [[ $has_stevenblack_entries -eq 0 ]]; then
|
||||
return 0 # StevenBlack entries missing, needs restoration
|
||||
fi
|
||||
|
||||
return 1 # File seems intact
|
||||
}
|
||||
|
||||
# Function to restore hosts file
|
||||
restore_hosts_file() {
|
||||
log_message "Hosts file modification detected - initiating restoration"
|
||||
|
||||
if [[ -f $HOSTS_INSTALL_SCRIPT ]]; then
|
||||
log_message "Running hosts installation script: $HOSTS_INSTALL_SCRIPT"
|
||||
|
||||
if bash "$HOSTS_INSTALL_SCRIPT" >> "$LOG_FILE" 2>&1; then
|
||||
log_message "Hosts file restoration completed successfully"
|
||||
else
|
||||
log_message "Hosts file restoration failed with exit code $?"
|
||||
fi
|
||||
else
|
||||
log_message "ERROR: Hosts install script not found at $HOSTS_INSTALL_SCRIPT"
|
||||
fi
|
||||
}
|
||||
|
||||
# Function to monitor with inotifywait
|
||||
monitor_with_inotify() {
|
||||
log_message "Starting hosts file monitoring with inotify"
|
||||
local last_check_ts=0
|
||||
|
||||
# Monitor the hosts file and its directory for various events
|
||||
inotifywait -m -e delete,move,modify,attrib,create --format '%w%f %e %T' --timefmt '%Y-%m-%d %H:%M:%S' "$HOSTS_FILE" /etc/ 2> /dev/null |
|
||||
while read -r file event time; do
|
||||
# Check if the event is related to our hosts file
|
||||
if [[ $file == "$HOSTS_FILE" ]] || [[ $file == "/etc/hosts" ]]; then
|
||||
local now_ts
|
||||
current_epoch now_ts
|
||||
if (( now_ts - last_check_ts < EVENT_COOLDOWN_S )); then
|
||||
continue
|
||||
fi
|
||||
last_check_ts=$now_ts
|
||||
|
||||
log_message "Event detected: $event on $file at $time"
|
||||
|
||||
# Check if restoration is needed
|
||||
if needs_restoration; then
|
||||
restore_hosts_file
|
||||
else
|
||||
log_message "Hosts file check passed - no restoration needed"
|
||||
fi
|
||||
fi
|
||||
done
|
||||
}
|
||||
|
||||
# Function to monitor with polling (fallback)
|
||||
monitor_with_polling() {
|
||||
log_message "Starting hosts file monitoring with polling (fallback method)"
|
||||
|
||||
while true; do
|
||||
if needs_restoration; then
|
||||
restore_hosts_file
|
||||
fi
|
||||
|
||||
# Check every 30 seconds
|
||||
wait_seconds 30
|
||||
done
|
||||
}
|
||||
|
||||
start_monitoring() {
|
||||
log_message "=== Hosts File Monitor Started ==="
|
||||
|
||||
if command -v inotifywait > /dev/null 2>&1; then
|
||||
log_message "Using inotify for file monitoring"
|
||||
monitor_with_inotify
|
||||
else
|
||||
log_message "inotify-tools not available, using polling method"
|
||||
log_message "Consider installing inotify-tools for better performance: pacman -S inotify-tools"
|
||||
monitor_with_polling
|
||||
fi
|
||||
}
|
||||
|
||||
# Main execution
|
||||
if [[ ${HOSTS_FILE_MONITOR_SKIP_MAIN:-0} -ne 1 ]]; then
|
||||
start_monitoring
|
||||
fi
|
||||
@ -1,53 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Install and enable the resource-monitoring stack used by usage_report.py:
|
||||
# atop -- daily CPU/RAM/disk history (systemd service + rotation)
|
||||
# nvtop -- live GPU top (optional, NVIDIA/AMD/Intel)
|
||||
# netdata -- live dashboard on http://localhost:19999 (optional)
|
||||
# a clipboard tool (wl-clipboard or xclip) so usage_report.py can paste
|
||||
#
|
||||
# Plus an `nvidia-pmon` user service that logs per-process GPU samples to
|
||||
# ~/.local/share/gpu-log/pmon-YYYYMMDD.log (only if nvidia-smi is present).
|
||||
#
|
||||
# Works on Arch, Debian/Ubuntu (and derivatives), Fedora/RHEL, openSUSE.
|
||||
# Re-run safely; everything is idempotent.
|
||||
#
|
||||
# The steps live in lib/; this file owns the shared helpers, the globals that
|
||||
# cross between steps (FAMILY, pkgs, clip, unit_dir, REPO_DIR) and the order.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
log() { printf '[install-usage] %s\n' "$*" >&2; }
|
||||
die() {
|
||||
printf '[install-usage] ERROR: %s\n' "$*" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
# readlink -f so a symlinked entry point still finds lib/.
|
||||
SCRIPT_DIR="$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")"
|
||||
# shellcheck source=lib/distro_detect.sh
|
||||
. "$SCRIPT_DIR/lib/distro_detect.sh"
|
||||
# shellcheck source=lib/packages.sh
|
||||
. "$SCRIPT_DIR/lib/packages.sh"
|
||||
# shellcheck source=lib/system_services.sh
|
||||
. "$SCRIPT_DIR/lib/system_services.sh"
|
||||
# shellcheck source=lib/nvidia_pmon.sh
|
||||
. "$SCRIPT_DIR/lib/nvidia_pmon.sh"
|
||||
# shellcheck source=lib/catchup_timer.sh
|
||||
. "$SCRIPT_DIR/lib/catchup_timer.sh"
|
||||
|
||||
[[ $EUID -eq 0 ]] && die "run as your normal user; sudo is invoked where needed"
|
||||
command -v sudo >/dev/null 2>&1 || die "sudo is required"
|
||||
|
||||
detect_family
|
||||
resolve_and_install_packages
|
||||
enable_system_services
|
||||
setup_nvidia_pmon
|
||||
|
||||
# Resolved here, once, from the entry script's own location: lib/ sits one
|
||||
# level deeper, so a lib recomputing this would point at the wrong repo.
|
||||
REPO_DIR="$SCRIPT_DIR/../../../../.."
|
||||
REPO_DIR="$(readlink -f "$REPO_DIR")"
|
||||
setup_catchup_timer
|
||||
|
||||
log "done. Wait for the first atop sample (default 10 min), then run:"
|
||||
log " python $SCRIPT_DIR/usage_report.py"
|
||||
@ -1,82 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Installs the hourly usage-report catch-up user timer.
|
||||
# Sourced by install_usage_monitoring.sh; inherits the caller's strict mode.
|
||||
#
|
||||
# REPO_DIR is computed ONCE by the entry script and read here as a global. It
|
||||
# is interpolated into the generated catch-up script at write time, so
|
||||
# recomputing it from this file's own depth would silently point the generated
|
||||
# script at the wrong repo while still exiting 0.
|
||||
|
||||
setup_catchup_timer() {
|
||||
unit_dir="$HOME/.config/systemd/user"
|
||||
mkdir -p "$unit_dir" "$HOME/.local/bin" "$HOME/.local/share/usage-reports"
|
||||
|
||||
# Unquoted delimiter: $HOME and $REPO_DIR interpolate at write time, while
|
||||
# \$REPO and \$OUT_DIR stay literal. The escaping is load-bearing.
|
||||
cat >"$HOME/.local/bin/usage-report-catchup.sh" <<SCRIPT
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
|
||||
REPO="$REPO_DIR"
|
||||
RUN_SCRIPT="\$REPO/run.sh"
|
||||
OUT_DIR="\$HOME/.local/share/usage-reports"
|
||||
ATOP_DIR="/var/log/atop"
|
||||
|
||||
mkdir -p "\$OUT_DIR"
|
||||
|
||||
if [[ ! -x "\$RUN_SCRIPT" ]]; then
|
||||
echo "usage-report-catchup: missing executable \$RUN_SCRIPT" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
shopt -s nullglob
|
||||
TODAY="\$(date +%Y%m%d)"
|
||||
for atop_file in "\$ATOP_DIR"/atop_*; do
|
||||
date_part="\${atop_file##*_}"
|
||||
if [[ ! "\$date_part" =~ ^[0-9]{8}\$ ]]; then
|
||||
continue
|
||||
fi
|
||||
|
||||
out_file="\$OUT_DIR/usage-report-\${date_part}.md"
|
||||
tmp_file="\$out_file.tmp"
|
||||
|
||||
if [[ "\$date_part" == "\$TODAY" || ! -s "\$out_file" ]]; then
|
||||
if "\$RUN_SCRIPT" --date "\$date_part" > "\$tmp_file"; then
|
||||
mv -f "\$tmp_file" "\$out_file"
|
||||
else
|
||||
rm -f "\$tmp_file"
|
||||
fi
|
||||
fi
|
||||
done
|
||||
SCRIPT
|
||||
chmod +x "$HOME/.local/bin/usage-report-catchup.sh"
|
||||
|
||||
cat >"$unit_dir/usage-report-catchup.service" <<'UNIT'
|
||||
[Unit]
|
||||
Description=Generate usage reports for available atop days
|
||||
After=default.target
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStart=%h/.local/bin/usage-report-catchup.sh
|
||||
UNIT
|
||||
|
||||
cat >"$unit_dir/usage-report-catchup.timer" <<'UNIT'
|
||||
[Unit]
|
||||
Description=Run usage report catch-up hourly
|
||||
Requires=usage-report-catchup.service
|
||||
|
||||
[Timer]
|
||||
OnBootSec=2min
|
||||
OnCalendar=hourly
|
||||
RandomizedDelaySec=2min
|
||||
Persistent=true
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
UNIT
|
||||
|
||||
systemctl --user daemon-reload
|
||||
systemctl --user enable --now usage-report-catchup.timer || log "warn: usage-report-catchup timer failed"
|
||||
log "usage reports will be generated hourly in $HOME/.local/share/usage-reports/"
|
||||
}
|
||||
@ -1,36 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Maps /etc/os-release onto the FAMILY value the package layer switches on.
|
||||
# Sourced by install_usage_monitoring.sh; inherits the caller's strict mode.
|
||||
|
||||
# Sets the global FAMILY. Uses log()/die() from the entry script.
|
||||
detect_family() {
|
||||
. /etc/os-release 2>/dev/null || die "cannot read /etc/os-release"
|
||||
|
||||
FAMILY=""
|
||||
for id in ${ID:-} ${ID_LIKE:-}; do
|
||||
case "$id" in
|
||||
arch | manjaro | endeavouros)
|
||||
FAMILY="arch"
|
||||
break
|
||||
;;
|
||||
debian | ubuntu | linuxmint | pop)
|
||||
FAMILY="debian"
|
||||
break
|
||||
;;
|
||||
elementary)
|
||||
FAMILY="debian"
|
||||
break
|
||||
;;
|
||||
fedora | rhel | centos)
|
||||
FAMILY="fedora"
|
||||
break
|
||||
;;
|
||||
opensuse* | suse | sles)
|
||||
FAMILY="suse"
|
||||
break
|
||||
;;
|
||||
esac
|
||||
done
|
||||
[[ -n $FAMILY ]] || die "unsupported distro: ID=${ID:-?} ID_LIKE=${ID_LIKE:-?}"
|
||||
log "detected distro family: $FAMILY (${PRETTY_NAME:-unknown})"
|
||||
}
|
||||
@ -1,89 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Installs the optional per-process NVIDIA GPU logger (user service).
|
||||
# Sourced by install_usage_monitoring.sh; inherits the caller's strict mode.
|
||||
#
|
||||
# The nvidia-pmon-logger.sh heredoc below is asserted against by
|
||||
# tests/test_usage_monitoring_installer_efficiency.sh, which greps THIS file's
|
||||
# source text for the `cat > ... << 'SCRIPT'` anchor line. That match is
|
||||
# whitespace-insensitive, so shfmt's `cat >"..." <<'SCRIPT'` form is fine, but
|
||||
# the heredoc BODY is compared property-by-property -- keep it verbatim.
|
||||
|
||||
setup_nvidia_pmon() {
|
||||
if command -v nvidia-smi >/dev/null 2>&1; then
|
||||
log "setting up nvidia-pmon user service"
|
||||
mkdir -p "$HOME/.local/share/gpu-log"
|
||||
mkdir -p "$HOME/.local/bin"
|
||||
unit_dir="$HOME/.config/systemd/user"
|
||||
mkdir -p "$unit_dir"
|
||||
|
||||
# Install the day-rolling wrapper script.
|
||||
cat >"$HOME/.local/bin/nvidia-pmon-logger.sh" <<'SCRIPT'
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
|
||||
LOG_DIR="$HOME/.local/share/gpu-log"
|
||||
ERR_LOG="$LOG_DIR/pmon-errors.log"
|
||||
mkdir -p "$LOG_DIR"
|
||||
|
||||
if ! command -v nvidia-smi >/dev/null 2>&1; then
|
||||
echo "nvidia-pmon-logger: nvidia-smi not found" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
current_day() {
|
||||
printf '%(%Y%m%d)T' -1
|
||||
}
|
||||
|
||||
seconds_until_next_day() {
|
||||
local hour minute second
|
||||
printf -v hour '%(%H)T' -1
|
||||
printf -v minute '%(%M)T' -1
|
||||
printf -v second '%(%S)T' -1
|
||||
printf '%s\n' $(((23 - 10#$hour) * 3600 + (59 - 10#$minute) * 60 + (60 - 10#$second)))
|
||||
}
|
||||
|
||||
while true; do
|
||||
day="$(current_day)"
|
||||
out_file="$LOG_DIR/pmon-${day}.log"
|
||||
rollover_pid=''
|
||||
|
||||
nvidia-smi pmon -d 10 -o DT >> "$out_file" 2>> "$ERR_LOG" &
|
||||
pmon_pid=$!
|
||||
|
||||
(
|
||||
sleep "$(seconds_until_next_day)"
|
||||
kill "$pmon_pid" >/dev/null 2>&1 || true
|
||||
) &
|
||||
rollover_pid=$!
|
||||
|
||||
wait "$pmon_pid" || true
|
||||
|
||||
if [[ -n $rollover_pid ]]; then
|
||||
kill "$rollover_pid" >/dev/null 2>&1 || true
|
||||
wait "$rollover_pid" 2>/dev/null || true
|
||||
fi
|
||||
|
||||
done
|
||||
SCRIPT
|
||||
chmod +x "$HOME/.local/bin/nvidia-pmon-logger.sh"
|
||||
|
||||
cat >"$unit_dir/nvidia-pmon.service" <<'UNIT'
|
||||
[Unit]
|
||||
Description=Per-day NVIDIA pmon logger
|
||||
After=default.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
ExecStart=%h/.local/bin/nvidia-pmon-logger.sh
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
UNIT
|
||||
systemctl --user daemon-reload
|
||||
systemctl --user enable --now nvidia-pmon.service || log "warn: nvidia-pmon user service failed"
|
||||
else
|
||||
log "no nvidia-smi found; skipping GPU per-process logger"
|
||||
fi
|
||||
}
|
||||
@ -1,73 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Per-family package name tables and the installer front-end.
|
||||
# Sourced by install_usage_monitoring.sh; inherits the caller's strict mode.
|
||||
|
||||
# Format: "<generic>=<package>"; empty package = skip on this distro.
|
||||
# These stay at top level on purpose: `declare -A` inside a function is local,
|
||||
# which would make every pkg_name lookup return empty.
|
||||
declare -A PKG_ARCH=(
|
||||
[atop]=atop [nvtop]=nvtop [netdata]=netdata
|
||||
[wl_clipboard]=wl-clipboard [xclip]=xclip
|
||||
)
|
||||
declare -A PKG_DEBIAN=(
|
||||
[atop]=atop [nvtop]=nvtop [netdata]=netdata
|
||||
[wl_clipboard]=wl-clipboard [xclip]=xclip
|
||||
)
|
||||
declare -A PKG_FEDORA=(
|
||||
[atop]=atop [nvtop]=nvtop [netdata]=netdata
|
||||
[wl_clipboard]=wl-clipboard [xclip]=xclip
|
||||
)
|
||||
declare -A PKG_SUSE=(
|
||||
[atop]=atop [nvtop]=nvtop [netdata]=netdata
|
||||
[wl_clipboard]=wl-clipboard [xclip]=xclip
|
||||
)
|
||||
|
||||
pkg_name() {
|
||||
local key=$1
|
||||
case "$FAMILY" in
|
||||
arch) printf '%s' "${PKG_ARCH[$key]-}" ;;
|
||||
debian) printf '%s' "${PKG_DEBIAN[$key]-}" ;;
|
||||
fedora) printf '%s' "${PKG_FEDORA[$key]-}" ;;
|
||||
suse) printf '%s' "${PKG_SUSE[$key]-}" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
install_packages() {
|
||||
local -a pkgs=("$@")
|
||||
[[ ${#pkgs[@]} -eq 0 ]] && return 0
|
||||
log "installing: ${pkgs[*]}"
|
||||
case "$FAMILY" in
|
||||
arch) sudo pacman -S --needed --noconfirm "${pkgs[@]}" ;;
|
||||
debian)
|
||||
sudo apt-get update -qq
|
||||
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y "${pkgs[@]}"
|
||||
;;
|
||||
fedora) sudo dnf install -y "${pkgs[@]}" ;;
|
||||
suse) sudo zypper --non-interactive install "${pkgs[@]}" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
# Pick a clipboard tool matching the session type.
|
||||
clipboard_pkg() {
|
||||
if [[ ${XDG_SESSION_TYPE:-} == "wayland" ]]; then
|
||||
pkg_name wl_clipboard
|
||||
else
|
||||
pkg_name xclip
|
||||
fi
|
||||
}
|
||||
|
||||
# Resolves the global `pkgs` array (and `clip`) from FAMILY, then installs.
|
||||
# The trailing `|| true` guards the bare conditional from ending the function
|
||||
# with a false return value under `set -e`.
|
||||
resolve_and_install_packages() {
|
||||
want_keys=(atop nvtop netdata)
|
||||
pkgs=()
|
||||
for key in "${want_keys[@]}"; do
|
||||
p=$(pkg_name "$key")
|
||||
[[ -n $p ]] && pkgs+=("$p")
|
||||
done
|
||||
clip=$(clipboard_pkg)
|
||||
[[ -n $clip ]] && pkgs+=("$clip") || true
|
||||
|
||||
install_packages "${pkgs[@]}"
|
||||
}
|
||||
@ -1,20 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Enables the system-level units the monitoring stack needs.
|
||||
# Sourced by install_usage_monitoring.sh; inherits the caller's strict mode.
|
||||
|
||||
enable_unit() {
|
||||
local unit=$1
|
||||
if systemctl list-unit-files "$unit" >/dev/null 2>&1; then
|
||||
log "enabling $unit"
|
||||
sudo systemctl enable --now "$unit" || log "warn: failed to enable $unit"
|
||||
else
|
||||
log "skip $unit (not present on this system)"
|
||||
fi
|
||||
}
|
||||
|
||||
enable_system_services() {
|
||||
enable_unit atop.service
|
||||
# atop-rotate exists on Arch; Debian/Ubuntu rotate via cron instead.
|
||||
enable_unit atop-rotate.timer
|
||||
enable_unit netdata.service
|
||||
}
|
||||
@ -1,113 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Periodic system maintenance script
|
||||
# Installs pacman/makepkg wrappers and updates hosts file
|
||||
# This file is installed by setup_periodic_system.sh
|
||||
#
|
||||
# FAILURES ARE LOUD, ON PURPOSE.
|
||||
# This script used to swallow every failure: a missing repair script was one
|
||||
# line appended to a log nobody reads, execute_with_log returned 0, and the
|
||||
# systemd unit therefore reported SUCCESS. That is exactly how a stale path
|
||||
# (from the 2026-05-15 repo reorg) silently disabled the pacman-wrapper and
|
||||
# hosts self-repair for five months without a single visible symptom.
|
||||
#
|
||||
# Any failure now surfaces four ways:
|
||||
# 1. journal at ERROR priority -> journalctl -p err -t periodic-system-maintenance
|
||||
# 2. the systemd unit FAILS -> systemctl --failed / red timer (exit 1 below)
|
||||
# 3. a critical desktop notification that does not auto-dismiss
|
||||
# 4. a breadcrumb file -> /var/lib/periodic-system-maintenance/LAST_RUN_FAILED
|
||||
|
||||
set -e
|
||||
|
||||
LOG_FILE="/var/log/periodic-system-maintenance.log"
|
||||
STATE_DIR="/var/lib/periodic-system-maintenance"
|
||||
FAILURE_FLAG="$STATE_DIR/LAST_RUN_FAILED"
|
||||
|
||||
# Path placeholders replaced at install time
|
||||
PACMAN_WRAPPER_INSTALL="__PACMAN_WRAPPER_INSTALL__"
|
||||
MAKEPKG_WRAPPER_INSTALL="__MAKEPKG_WRAPPER_INSTALL__"
|
||||
HOSTS_INSTALL_SCRIPT="__HOSTS_INSTALL_SCRIPT__"
|
||||
|
||||
declare -a FAILURES=()
|
||||
|
||||
# Function to log with timestamp
|
||||
log_message() {
|
||||
echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" >> "$LOG_FILE"
|
||||
}
|
||||
|
||||
# Record a failure loudly: journal at ERROR priority + collect for the summary.
|
||||
fail_loudly() {
|
||||
local msg="$1"
|
||||
log_message "FAILURE: $msg"
|
||||
echo "✗ $msg" >&2
|
||||
logger -t periodic-system-maintenance -p user.err "FAILURE: $msg"
|
||||
FAILURES+=("$msg")
|
||||
}
|
||||
|
||||
# Desktop notification for the graphical user (this runs as root under systemd,
|
||||
# so we have to reach the user's session bus explicitly).
|
||||
notify_user() {
|
||||
local title="$1" body="$2"
|
||||
command -v notify-send >/dev/null 2>&1 || return 0
|
||||
local target_user target_uid
|
||||
target_user="$(loginctl list-sessions --no-legend 2>/dev/null | awk 'NF {print $3; exit}')"
|
||||
[[ -n $target_user ]] || return 0
|
||||
target_uid="$(id -u "$target_user" 2>/dev/null)" || return 0
|
||||
# -t 0 => never auto-dismiss; the user must acknowledge it.
|
||||
sudo -u "$target_user" \
|
||||
DBUS_SESSION_BUS_ADDRESS="unix:path=/run/user/$target_uid/bus" \
|
||||
notify-send -u critical -t 0 "$title" "$body" 2>/dev/null || true
|
||||
}
|
||||
|
||||
# Function to execute with logging
|
||||
execute_with_log() {
|
||||
local script_path="$1"
|
||||
local script_name="$2"
|
||||
local rc=0
|
||||
|
||||
log_message "Starting $script_name"
|
||||
echo "Executing $script_name..." >&2
|
||||
|
||||
if [[ ! -f $script_path ]]; then
|
||||
fail_loudly "$script_name: script NOT FOUND at $script_path"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# `|| rc=$?` keeps set -e from aborting so every step still runs and we can
|
||||
# report ALL failures, not just the first.
|
||||
bash "$script_path" >> "$LOG_FILE" 2>&1 || rc=$?
|
||||
if ((rc == 0)); then
|
||||
log_message "$script_name completed successfully"
|
||||
echo "✓ $script_name completed successfully" >&2
|
||||
else
|
||||
fail_loudly "$script_name: FAILED (exit $rc) — see $LOG_FILE"
|
||||
fi
|
||||
}
|
||||
|
||||
# Start maintenance
|
||||
log_message "=== Periodic System Maintenance Started ==="
|
||||
|
||||
# Install pacman/makepkg wrappers
|
||||
execute_with_log "$PACMAN_WRAPPER_INSTALL" "Pacman Wrapper Installation"
|
||||
execute_with_log "$MAKEPKG_WRAPPER_INSTALL" "Makepkg Wrapper Installation"
|
||||
|
||||
# Update hosts file
|
||||
execute_with_log "$HOSTS_INSTALL_SCRIPT" "Hosts File Update"
|
||||
|
||||
if ((${#FAILURES[@]} > 0)); then
|
||||
mkdir -p "$STATE_DIR"
|
||||
printf '%s\n' "${FAILURES[@]}" > "$FAILURE_FLAG"
|
||||
log_message "=== Periodic System Maintenance FAILED (${#FAILURES[@]} failure(s)) ==="
|
||||
echo "✗ Periodic system maintenance FAILED with ${#FAILURES[@]} failure(s):" >&2
|
||||
printf ' - %s\n' "${FAILURES[@]}" >&2
|
||||
notify_user "⚠ System maintenance FAILED" \
|
||||
"$(printf '%s\n' "${FAILURES[@]}")
|
||||
|
||||
journalctl -p err -t periodic-system-maintenance"
|
||||
# Non-zero => systemd marks the unit failed, so `systemctl --failed` and the
|
||||
# timer's status show it instead of a green "success".
|
||||
exit 1
|
||||
fi
|
||||
|
||||
rm -f "$FAILURE_FLAG" 2>/dev/null || true
|
||||
log_message "=== Periodic System Maintenance Completed ==="
|
||||
echo "Periodic system maintenance completed. Check $LOG_FILE for details." >&2
|
||||
@ -1,177 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Shutdown timer monitor script
|
||||
# Watches the day-specific-shutdown timer and re-enables it if disabled
|
||||
# This file is installed by setup_midnight_shutdown.sh
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
LOG_FILE="/var/log/shutdown-timer-monitor.log"
|
||||
TIMER_NAME="day-specific-shutdown.timer"
|
||||
SERVICE_NAME="day-specific-shutdown.service"
|
||||
CHECK_INTERVAL=30
|
||||
|
||||
current_epoch() {
|
||||
local out_var="${1:-}"
|
||||
if [[ -n $out_var ]]; then
|
||||
printf -v "$out_var" '%(%s)T' -1
|
||||
else
|
||||
printf '%(%s)T\n' -1
|
||||
fi
|
||||
}
|
||||
|
||||
wait_seconds() {
|
||||
local timeout_s=$1
|
||||
local start_ts end_ts elapsed_s remaining_s
|
||||
|
||||
printf -v start_ts '%(%s)T' -1
|
||||
IFS= read -r -t "$timeout_s" || true
|
||||
printf -v end_ts '%(%s)T' -1
|
||||
|
||||
elapsed_s=$((end_ts - start_ts))
|
||||
if (( elapsed_s < timeout_s )); then
|
||||
remaining_s=$((timeout_s - elapsed_s))
|
||||
sleep "$remaining_s"
|
||||
fi
|
||||
}
|
||||
|
||||
# Log with timestamp (shutdown-timer-monitor specific)
|
||||
log_message() {
|
||||
local _ts
|
||||
local msg
|
||||
printf -v _ts '%(%Y-%m-%d %H:%M:%S)T' -1
|
||||
printf -v msg '%s [shutdown-monitor] %s' "$_ts" "$1"
|
||||
printf '%s\n' "$msg" >&2
|
||||
printf '%s\n' "$msg" >> "$LOG_FILE" 2>/dev/null || true
|
||||
}
|
||||
|
||||
# Function to check if timer needs to be re-enabled
|
||||
timer_needs_restoration() {
|
||||
# Check if timer is enabled
|
||||
if ! systemctl is-enabled "$TIMER_NAME" &>/dev/null; then
|
||||
log_message "Timer $TIMER_NAME is not enabled"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# Check if timer is active
|
||||
if ! systemctl is-active "$TIMER_NAME" &>/dev/null; then
|
||||
log_message "Timer $TIMER_NAME is not active"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# Check if timer unit file exists
|
||||
if [[ ! -f "/etc/systemd/system/$TIMER_NAME" ]]; then
|
||||
log_message "Timer unit file missing: /etc/systemd/system/$TIMER_NAME"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# Check if service unit file exists
|
||||
if [[ ! -f "/etc/systemd/system/$SERVICE_NAME" ]]; then
|
||||
log_message "Service unit file missing: /etc/systemd/system/$SERVICE_NAME"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# Check if check script exists
|
||||
if [[ ! -f "/usr/local/bin/day-specific-shutdown-check.sh" ]]; then
|
||||
log_message "Check script missing: /usr/local/bin/day-specific-shutdown-check.sh"
|
||||
return 0
|
||||
fi
|
||||
|
||||
return 1 # Timer is properly configured
|
||||
}
|
||||
|
||||
# Function to restore timer
|
||||
restore_timer() {
|
||||
log_message "Shutdown timer tampering detected - initiating restoration"
|
||||
|
||||
# Reload systemd daemon in case unit files were modified
|
||||
systemctl daemon-reload
|
||||
|
||||
# Re-enable timer if disabled
|
||||
if ! systemctl is-enabled "$TIMER_NAME" &>/dev/null; then
|
||||
log_message "Re-enabling $TIMER_NAME"
|
||||
systemctl enable "$TIMER_NAME" 2>/dev/null || true
|
||||
fi
|
||||
|
||||
# Re-start timer if not active
|
||||
if ! systemctl is-active "$TIMER_NAME" &>/dev/null; then
|
||||
log_message "Re-starting $TIMER_NAME"
|
||||
systemctl start "$TIMER_NAME" 2>/dev/null || true
|
||||
fi
|
||||
|
||||
# Verify restoration
|
||||
if systemctl is-active "$TIMER_NAME" &>/dev/null; then
|
||||
log_message "Timer restoration completed successfully"
|
||||
else
|
||||
log_message "WARNING: Timer restoration may have failed"
|
||||
fi
|
||||
}
|
||||
|
||||
# Function to monitor timer with systemd events
|
||||
monitor_with_dbus() {
|
||||
log_message "Starting shutdown timer monitoring with D-Bus events"
|
||||
local last_check_ts=0
|
||||
|
||||
# Use busctl to monitor systemd unit changes
|
||||
# Fall back to polling if this fails.
|
||||
if command -v busctl &>/dev/null; then
|
||||
# Monitor for unit state changes
|
||||
busctl monitor --system org.freedesktop.systemd1 2>/dev/null |
|
||||
while read -r line; do
|
||||
# Check if the line mentions our timer
|
||||
if [[ $line == *"$TIMER_NAME"* || $line == *"$SERVICE_NAME"* ]]; then
|
||||
local now_ts
|
||||
current_epoch now_ts
|
||||
if (( now_ts - last_check_ts < CHECK_INTERVAL )); then
|
||||
continue
|
||||
fi
|
||||
last_check_ts=$now_ts
|
||||
log_message "Systemd event detected for shutdown timer"
|
||||
if timer_needs_restoration; then
|
||||
restore_timer
|
||||
fi
|
||||
fi
|
||||
done
|
||||
else
|
||||
log_message "busctl not available, falling back to polling"
|
||||
monitor_with_polling
|
||||
fi
|
||||
}
|
||||
|
||||
# Function to monitor with polling (primary method for reliability)
|
||||
monitor_with_polling() {
|
||||
log_message "Starting shutdown timer monitoring with polling (interval: ${CHECK_INTERVAL}s)"
|
||||
|
||||
while true; do
|
||||
if timer_needs_restoration; then
|
||||
restore_timer
|
||||
fi
|
||||
wait_seconds "$CHECK_INTERVAL"
|
||||
done
|
||||
}
|
||||
|
||||
start_monitoring() {
|
||||
log_message "=== Shutdown Timer Monitor Started ==="
|
||||
log_message "Monitoring timer: $TIMER_NAME"
|
||||
log_message "Monitoring service: $SERVICE_NAME"
|
||||
|
||||
# Initial check
|
||||
if timer_needs_restoration; then
|
||||
log_message "Initial check: Timer needs restoration"
|
||||
restore_timer
|
||||
else
|
||||
log_message "Initial check: Timer is properly configured"
|
||||
fi
|
||||
|
||||
# Prefer D-Bus monitoring, with polling as the fallback path.
|
||||
if command -v busctl &>/dev/null; then
|
||||
monitor_with_dbus
|
||||
else
|
||||
log_message "busctl not available, falling back to polling"
|
||||
monitor_with_polling
|
||||
fi
|
||||
}
|
||||
|
||||
# Main execution
|
||||
if [[ ${SHUTDOWN_TIMER_MONITOR_SKIP_MAIN:-0} -ne 1 ]]; then
|
||||
start_monitoring
|
||||
fi
|
||||
@ -1,102 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Resource usage report from atop + nvidia-smi pmon logs.
|
||||
|
||||
Parses one or more daily `atop` binary logs via `atop -P PRC,PRM -r` and the
|
||||
per-process nvidia-smi pmon logs, aggregates CPU seconds, peak/average RSS, and
|
||||
GPU SM-% seconds per program, and prints a compact Markdown report intended to
|
||||
be pasted into an LLM (Claude / Copilot) for further analysis.
|
||||
|
||||
Run with no arguments to report on **everything since the last report**: the
|
||||
previous run's timestamp is persisted, and each run covers the whole window
|
||||
from then until now, spanning as many daily logs as needed (so skipped days are
|
||||
never lost). After a successful report the timestamp is advanced to "now".
|
||||
|
||||
usage_report.py # since the last report (multi-day)
|
||||
usage_report.py --since 20260419 # ad hoc: from a date to now, no state
|
||||
usage_report.py --date 20260419 # one specific day (ad hoc, no state)
|
||||
usage_report.py --top 20 # keep 20 rows per table
|
||||
usage_report.py --no-update-state # don't advance the saved timestamp
|
||||
usage_report.py > report.md # redirect to a file
|
||||
|
||||
The output intentionally front-loads metadata (hostname, period, window, sample
|
||||
count, HZ, machine specs) so the LLM never has to guess context.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import datetime as _dt
|
||||
from pathlib import Path
|
||||
|
||||
from _usage_report_run import _run_since, _run_single_day
|
||||
|
||||
_DEFAULT_TOP = 15
|
||||
|
||||
|
||||
def _is_single_day_mode(args: argparse.Namespace) -> bool:
|
||||
"""True when the user pinned an exact day or explicit log paths."""
|
||||
return (
|
||||
args.date is not None or args.atop_log is not None or args.pmon_log is not None
|
||||
)
|
||||
|
||||
|
||||
def _build_parser() -> argparse.ArgumentParser:
|
||||
"""Construct the command-line argument parser."""
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument(
|
||||
"--date",
|
||||
default=None,
|
||||
help="report on one specific day (YYYYMMDD); ad hoc, ignores state",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--since",
|
||||
default=None,
|
||||
help="ad-hoc: report from this date (YYYYMMDD) to now; leaves state",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--top",
|
||||
type=int,
|
||||
default=_DEFAULT_TOP,
|
||||
help=f"rows per table (default: {_DEFAULT_TOP})",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--atop-log",
|
||||
type=Path,
|
||||
default=None,
|
||||
help="override atop log path (implies single-day mode)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--pmon-log",
|
||||
type=Path,
|
||||
default=None,
|
||||
help="override pmon log path (implies single-day mode)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--no-clipboard",
|
||||
action="store_true",
|
||||
help="skip copying the report to the X clipboard",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--no-update-state",
|
||||
action="store_true",
|
||||
help="do not advance the saved last-report timestamp",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--quiet",
|
||||
action="store_true",
|
||||
help="suppress the progress line on stderr",
|
||||
)
|
||||
return parser
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
"""Entry point; see module docstring for CLI."""
|
||||
args = _build_parser().parse_args(argv)
|
||||
now = _dt.datetime.now().astimezone()
|
||||
if _is_single_day_mode(args):
|
||||
return _run_single_day(args, now)
|
||||
return _run_since(args, now)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@ -1,38 +0,0 @@
|
||||
/var/log/periodic-system-maintenance.log {
|
||||
weekly
|
||||
rotate 4
|
||||
compress
|
||||
delaycompress
|
||||
missingok
|
||||
notifempty
|
||||
create 644 root root
|
||||
postrotate
|
||||
systemctl reload-or-restart rsyslog > /dev/null 2>&1 || true
|
||||
endscript
|
||||
}
|
||||
|
||||
/var/log/hosts-file-monitor.log {
|
||||
weekly
|
||||
rotate 4
|
||||
compress
|
||||
delaycompress
|
||||
missingok
|
||||
notifempty
|
||||
create 644 root root
|
||||
postrotate
|
||||
systemctl reload-or-restart rsyslog > /dev/null 2>&1 || true
|
||||
endscript
|
||||
}
|
||||
|
||||
/var/log/auto-system-update.log {
|
||||
weekly
|
||||
rotate 4
|
||||
compress
|
||||
delaycompress
|
||||
missingok
|
||||
notifempty
|
||||
create 644 root root
|
||||
postrotate
|
||||
systemctl reload-or-restart rsyslog > /dev/null 2>&1 || true
|
||||
endscript
|
||||
}
|
||||
@ -1,20 +0,0 @@
|
||||
[Unit]
|
||||
Description=Automatic System Update (pacman + yay AUR)
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
User=root
|
||||
ExecStart=/usr/local/bin/auto-system-update.sh
|
||||
StandardOutput=journal
|
||||
StandardError=journal
|
||||
|
||||
# System updates can take a while on slow connections
|
||||
TimeoutStartSec=1800
|
||||
TimeoutStopSec=30
|
||||
|
||||
Restart=no
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@ -1,14 +0,0 @@
|
||||
[Unit]
|
||||
Description=Run Automatic System Update daily
|
||||
Requires=auto-system-update.service
|
||||
|
||||
[Timer]
|
||||
# Run once per day at 04:00
|
||||
OnCalendar=*-*-* 04:00:00
|
||||
# Randomize by up to 30 minutes to avoid mirror congestion
|
||||
RandomizedDelaySec=1800
|
||||
# If a run was missed (e.g. machine was off), run on next boot
|
||||
Persistent=true
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
@ -1,27 +0,0 @@
|
||||
[Unit]
|
||||
Description=Hosts File Monitor and Auto-Restore Service
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=root
|
||||
ExecStart=/usr/local/bin/hosts-file-monitor.sh
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
StandardOutput=journal
|
||||
StandardError=journal
|
||||
|
||||
# Environment
|
||||
Environment=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
|
||||
|
||||
# Security settings
|
||||
NoNewPrivileges=false
|
||||
PrivateTmp=true
|
||||
|
||||
# Resource limits
|
||||
MemoryMax=50M
|
||||
CPUQuota=10%
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@ -1,20 +0,0 @@
|
||||
[Unit]
|
||||
Description=Resource-capped slice for user monitoring / status-bar scripts
|
||||
Documentation=https://www.freedesktop.org/software/systemd/man/systemd.slice.html
|
||||
|
||||
[Slice]
|
||||
# Cap the entire slice at 50% of one CPU and 512 MiB of RAM. If i3blocks or
|
||||
# any other status-bar tooling enters a fork-storm regime (as was observed
|
||||
# when polling scripts forked awk/tr/grep/bc every tick), the kernel will
|
||||
# throttle the slice rather than let it eat the box.
|
||||
CPUQuota=50%
|
||||
# MemorySwapMax=0 is required on systems with zram: without it, a cgroup
|
||||
# hitting MemoryMax thrashes zram instead of being OOM-killed, freezing
|
||||
# the machine. See .github/skills/oom-prevention/SKILL.md.
|
||||
MemoryMax=512M
|
||||
MemorySwapMax=0
|
||||
TasksMax=256
|
||||
|
||||
# Make sure killing the slice reaps every descendant.
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@ -1,21 +0,0 @@
|
||||
[Unit]
|
||||
Description=Periodic System Maintenance (Pacman Wrapper & Hosts File)
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
User=root
|
||||
ExecStart=/usr/local/bin/periodic-system-maintenance.sh
|
||||
StandardOutput=journal
|
||||
StandardError=journal
|
||||
|
||||
# Timeout settings
|
||||
TimeoutStartSec=300
|
||||
TimeoutStopSec=30
|
||||
|
||||
# Restart settings
|
||||
Restart=no
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@ -1,16 +0,0 @@
|
||||
[Unit]
|
||||
Description=Run Periodic System Maintenance every hour
|
||||
Requires=periodic-system-maintenance.service
|
||||
|
||||
[Timer]
|
||||
# Run every hour
|
||||
OnCalendar=hourly
|
||||
# Run on system startup (5 minutes after boot)
|
||||
OnBootSec=5min
|
||||
# Add randomization to prevent all systems from running simultaneously
|
||||
RandomizedDelaySec=300
|
||||
# Ensure timer persists across reboots
|
||||
Persistent=true
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
@ -1,19 +0,0 @@
|
||||
[Unit]
|
||||
Description=System Maintenance on Startup (Pacman Wrapper & Hosts File)
|
||||
After=network-online.target systemd-resolved.service
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
User=root
|
||||
ExecStart=/usr/local/bin/periodic-system-maintenance.sh
|
||||
StandardOutput=journal
|
||||
StandardError=journal
|
||||
RemainAfterExit=yes
|
||||
|
||||
# Timeout settings
|
||||
TimeoutStartSec=300
|
||||
TimeoutStopSec=30
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@ -1,27 +0,0 @@
|
||||
[Unit]
|
||||
Description=Shutdown Timer Monitor and Auto-Restore Service
|
||||
After=network-online.target day-specific-shutdown.timer
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=root
|
||||
ExecStart=/usr/local/bin/shutdown-timer-monitor.sh
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
StandardOutput=journal
|
||||
StandardError=journal
|
||||
|
||||
# Environment
|
||||
Environment=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
|
||||
|
||||
# Security settings
|
||||
NoNewPrivileges=false
|
||||
PrivateTmp=true
|
||||
|
||||
# Resource limits
|
||||
MemoryMax=50M
|
||||
CPUQuota=10%
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@ -22,11 +22,6 @@ _REPO_ROOT = Path(__file__).resolve().parents[2]
|
||||
|
||||
# Each standalone script directory whose Python modules these tests import.
|
||||
_SCRIPT_DIRS = (
|
||||
Path(__file__).resolve().parents[1]
|
||||
/ "scripts"
|
||||
/ "periodic_background"
|
||||
/ "system-maintenance"
|
||||
/ "bin", # usage_report modules
|
||||
_REPO_ROOT / "meta" / "scripts", # validate_evidence, validate_contract
|
||||
_REPO_ROOT / "phone_focus_mode" / "lib", # monitor_report
|
||||
_REPO_ROOT / "phone_focus_mode", # strip_workout_hosts
|
||||
|
||||
@ -1,139 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Regression tests for hosts-file-monitor.sh behavior.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)
|
||||
REPO_DIR=$(cd -- "$SCRIPT_DIR/.." && pwd)
|
||||
TARGET_SCRIPT="$REPO_DIR/scripts/periodic_background/system-maintenance/bin/hosts-file-monitor.sh"
|
||||
|
||||
fail() {
|
||||
printf 'FAIL: %s\n' "$1" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
assert_equals() {
|
||||
local expected="$1"
|
||||
local actual="$2"
|
||||
local context="$3"
|
||||
if [[ "$expected" != "$actual" ]]; then
|
||||
fail "$context (expected: '$expected', actual: '$actual')"
|
||||
fi
|
||||
}
|
||||
|
||||
TMP_DIR=$(mktemp -d)
|
||||
cleanup() {
|
||||
rm -rf "$TMP_DIR"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
WORKTREE="$TMP_DIR/worktree"
|
||||
BIN_DIR="$TMP_DIR/bin"
|
||||
mkdir -p "$WORKTREE/scripts/system-maintenance/bin" "$BIN_DIR"
|
||||
cp "$TARGET_SCRIPT" "$WORKTREE/scripts/system-maintenance/bin/hosts-file-monitor.sh"
|
||||
|
||||
cat >"$BIN_DIR/tee" <<'EOF'
|
||||
#!/bin/bash
|
||||
while IFS= read -r _; do
|
||||
:
|
||||
done
|
||||
EOF
|
||||
chmod +x "$BIN_DIR/tee"
|
||||
|
||||
cat >"$BIN_DIR/sleep" <<'EOF'
|
||||
#!/bin/bash
|
||||
printf '%s\n' "$1" >> "${SLEEP_LOG:?}"
|
||||
exit 99
|
||||
EOF
|
||||
chmod +x "$BIN_DIR/sleep"
|
||||
|
||||
cat >"$BIN_DIR/inotifywait" <<'EOF'
|
||||
#!/bin/bash
|
||||
while IFS= read -r line; do
|
||||
printf '%s\n' "$line"
|
||||
done <<< "${MOCK_INOTIFY_OUTPUT:-}"
|
||||
EOF
|
||||
chmod +x "$BIN_DIR/inotifywait"
|
||||
|
||||
run_shell() {
|
||||
env -i PATH="$BIN_DIR" HOSTS_FILE_MONITOR_SKIP_MAIN=1 /bin/bash -c "$1"
|
||||
}
|
||||
|
||||
make_hosts_file() {
|
||||
local file_path="$1"
|
||||
local include_custom="$2"
|
||||
local include_stevenblack="$3"
|
||||
local i
|
||||
|
||||
: >"$file_path"
|
||||
for ((i = 1; i <= 1005; i++)); do
|
||||
printf '127.0.0.1 example-%d.test\n' "$i" >>"$file_path"
|
||||
done
|
||||
|
||||
if [[ $include_custom -eq 1 ]]; then
|
||||
printf '# Custom blocking entries\n' >>"$file_path"
|
||||
fi
|
||||
|
||||
if [[ $include_stevenblack -eq 1 ]]; then
|
||||
printf '# StevenBlack hosts\n' >>"$file_path"
|
||||
fi
|
||||
}
|
||||
|
||||
printf 'Checking intact hosts files are accepted...\n'
|
||||
hosts_ok="$TMP_DIR/hosts-ok"
|
||||
make_hosts_file "$hosts_ok" 1 1
|
||||
ok_result=$(run_shell "source '$WORKTREE/scripts/system-maintenance/bin/hosts-file-monitor.sh'; HOSTS_FILE='$hosts_ok'; if needs_restoration; then printf restore; else printf ok; fi")
|
||||
assert_equals 'ok' "$ok_result" 'needs_restoration should accept intact hosts files'
|
||||
|
||||
printf 'Checking missing markers trigger restoration...\n'
|
||||
hosts_missing="$TMP_DIR/hosts-missing"
|
||||
make_hosts_file "$hosts_missing" 1 0
|
||||
missing_result=$(run_shell "source '$WORKTREE/scripts/system-maintenance/bin/hosts-file-monitor.sh'; HOSTS_FILE='$hosts_missing'; if needs_restoration; then printf restore; else printf ok; fi")
|
||||
assert_equals 'restore' "$missing_result" 'needs_restoration should reject files missing required markers'
|
||||
|
||||
printf 'Checking inotify path is preferred when available...\n'
|
||||
inotify_mode=$(run_shell "source '$WORKTREE/scripts/system-maintenance/bin/hosts-file-monitor.sh'; monitor_with_inotify() { printf inotify; }; monitor_with_polling() { printf polling; }; start_monitoring")
|
||||
assert_equals 'inotify' "$inotify_mode" 'start_monitoring should prefer inotifywait when present'
|
||||
|
||||
printf 'Checking polling fallback is used without inotifywait...\n'
|
||||
mv "$BIN_DIR/inotifywait" "$BIN_DIR/inotifywait.off"
|
||||
poll_mode=$(run_shell "source '$WORKTREE/scripts/system-maintenance/bin/hosts-file-monitor.sh'; monitor_with_inotify() { printf inotify; }; monitor_with_polling() { printf polling; }; start_monitoring")
|
||||
mv "$BIN_DIR/inotifywait.off" "$BIN_DIR/inotifywait"
|
||||
assert_equals 'polling' "$poll_mode" 'start_monitoring should fall back to polling when inotifywait is absent'
|
||||
|
||||
printf 'Checking inotify event path avoids per-event sleep and debounces bursts...\n'
|
||||
sleep_log="$TMP_DIR/sleep.log"
|
||||
: >"$sleep_log"
|
||||
counter_file="$TMP_DIR/debounce-count.log"
|
||||
: >"$counter_file"
|
||||
debounce_calls=$(env -i PATH="$BIN_DIR" HOSTS_FILE_MONITOR_SKIP_MAIN=1 SLEEP_LOG="$sleep_log" COUNTER_FILE="$counter_file" MOCK_INOTIFY_OUTPUT=$'/etc/hosts MODIFY 2026-01-01 00:00:00\n/etc/hosts ATTRIB 2026-01-01 00:00:01\n/etc/hosts MODIFY 2026-01-01 00:00:02' /bin/bash -c \
|
||||
"source '$WORKTREE/scripts/system-maintenance/bin/hosts-file-monitor.sh'; \
|
||||
needs_restoration() { printf 'x\n' >> \"\$COUNTER_FILE\"; return 1; }; \
|
||||
idx=0; \
|
||||
current_epoch() { \
|
||||
local out_var=\"\${1:-}\"; \
|
||||
local ts; \
|
||||
case \$idx in 0) ts='100';; 1) ts='101';; 2) ts='106';; *) ts='999';; esac; \
|
||||
idx=\$((idx + 1)); \
|
||||
if [[ -n \$out_var ]]; then printf -v \"\$out_var\" '%s' \"\$ts\"; else printf '%s\\n' \"\$ts\"; fi; \
|
||||
}; \
|
||||
monitor_with_inotify >/dev/null 2>&1 || true; \
|
||||
total=0; \
|
||||
while IFS= read -r _; do total=\$((total + 1)); done < \"\$COUNTER_FILE\"; \
|
||||
printf '%s' \"\$total\"")
|
||||
assert_equals '2' "$debounce_calls" 'monitor_with_inotify should debounce rapid successive events'
|
||||
|
||||
if [[ -s $sleep_log ]]; then
|
||||
fail 'monitor_with_inotify should not call sleep in the event path'
|
||||
fi
|
||||
|
||||
printf 'Checking polling wait helper enforces delay on /dev/null stdin...\n'
|
||||
wait_elapsed=$(env -i PATH="/usr/bin:/bin" HOSTS_FILE_MONITOR_SKIP_MAIN=1 /bin/bash -c \
|
||||
"source '$WORKTREE/scripts/system-maintenance/bin/hosts-file-monitor.sh'; \
|
||||
start=\$(printf '%(%s)T' -1); \
|
||||
wait_seconds 1; \
|
||||
end=\$(printf '%(%s)T' -1); \
|
||||
printf '%s' \$((end-start))" </dev/null)
|
||||
assert_equals '1' "$wait_elapsed" 'wait_seconds should not return immediately on /dev/null stdin'
|
||||
|
||||
printf 'hosts-file-monitor.sh regression checks passed.\n'
|
||||
@ -5,7 +5,20 @@ set -euo pipefail
|
||||
|
||||
SCRIPT_DIR=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)
|
||||
REPO_DIR=$(cd -- "$SCRIPT_DIR/.." && pwd)
|
||||
TARGET_SCRIPT="$REPO_DIR/scripts/periodic_background/system-maintenance/bin/shutdown-timer-monitor.sh"
|
||||
# shutdown-timer-monitor.sh moved to github.com/kuhyx/system-maintenance with
|
||||
# the rest of that subsystem, but this suite is the pair test: it checks that
|
||||
# digital_wellbeing's installer and the monitor it installs still agree. Prefer
|
||||
# a sibling checkout, fall back to the installed copy, and skip rather than
|
||||
# fail when neither is present -- a missing checkout is not a regression.
|
||||
TARGET_SCRIPT="${SYSTEM_MAINTENANCE_DIR:-$HOME/system-maintenance}/bin/shutdown-timer-monitor.sh"
|
||||
# Deliberately NOT falling back to /usr/local/bin: the installed copy can be
|
||||
# older than the source (it is, on this machine), and testing the installer
|
||||
# template against a stale binary compares two things that were never meant to
|
||||
# match. Skip instead -- a missing checkout is not a regression.
|
||||
if [[ ! -f $TARGET_SCRIPT ]]; then
|
||||
printf 'SKIP: shutdown-timer-monitor.sh not found (set SYSTEM_MAINTENANCE_DIR)\n'
|
||||
exit 0
|
||||
fi
|
||||
SETUP_SCRIPT="$REPO_DIR/scripts/periodic_background/digital_wellbeing/setup_midnight_shutdown.sh"
|
||||
# The installer was split into lib/ms_*.sh under the 250-line cap, so these
|
||||
# template checks search the entry script AND its libs: the monitor template
|
||||
|
||||
@ -1,58 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Regression tests for nvidia-pmon logger installer template efficiency.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)
|
||||
REPO_DIR=$(cd -- "$SCRIPT_DIR/.." && pwd)
|
||||
INSTALLER="$REPO_DIR/scripts/periodic_background/system-maintenance/bin/lib/nvidia_pmon.sh"
|
||||
|
||||
fail() {
|
||||
printf 'FAIL: %s\n' "$1" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
logger_template=$(
|
||||
awk '
|
||||
/cat >[[:space:]]*"\$HOME\/.local\/bin\/nvidia-pmon-logger\.sh"[[:space:]]*<<[[:space:]]*'\''SCRIPT'\''/ {capture=1; next}
|
||||
capture && /^SCRIPT$/ {capture=0; exit}
|
||||
capture {print}
|
||||
' "$INSTALLER"
|
||||
)
|
||||
|
||||
[[ -n $logger_template ]] || fail 'could not extract nvidia-pmon-logger template from installer'
|
||||
|
||||
printf 'Checking pmon logger template avoids read -t busy-loop pattern...\n'
|
||||
! grep -q 'read -r -t' <<<"$logger_template" ||
|
||||
fail 'logger template must not use read -t as sleep surrogate'
|
||||
|
||||
# The template used to poll with `sleep 60` until the day changed. It now
|
||||
# sleeps exactly once, until the next midnight, and blocks on `wait` for the
|
||||
# pmon child — strictly fewer wakeups. Assert that property rather than the
|
||||
# old literal, so an accidental return to polling still fails the test.
|
||||
printf 'Checking pmon logger template sleeps until rollover rather than polling...\n'
|
||||
# Literal search pattern: the $ must reach grep as text, not be expanded
|
||||
# by this shell.
|
||||
# shellcheck disable=SC2016
|
||||
grep -q 'sleep "\$(seconds_until_next_day)"' <<<"$logger_template" ||
|
||||
fail 'logger template must sleep until the next day boundary'
|
||||
|
||||
grep -q 'seconds_until_next_day()' <<<"$logger_template" ||
|
||||
fail 'logger template must define seconds_until_next_day'
|
||||
|
||||
printf 'Checking pmon logger template blocks on wait instead of spinning...\n'
|
||||
# Literal search pattern: the $ must reach grep as text, not be expanded
|
||||
# by this shell.
|
||||
# shellcheck disable=SC2016
|
||||
grep -qE '^\s*wait "\$pmon_pid"' <<<"$logger_template" ||
|
||||
fail 'logger template must block on wait for the pmon child'
|
||||
|
||||
printf 'Checking pmon logger template uses fork-free date builtin...\n'
|
||||
grep -q "printf '%(%Y%m%d)T' -1" <<<"$logger_template" ||
|
||||
fail 'logger template must use bash printf time builtin for current day'
|
||||
|
||||
printf 'Checking pmon logger template avoids external date command...\n'
|
||||
! grep -q 'date +%Y%m%d' <<<"$logger_template" ||
|
||||
fail 'logger template must not call external date command in hot path'
|
||||
|
||||
printf 'Usage monitoring installer efficiency tests passed.\n'
|
||||
@ -1,111 +0,0 @@
|
||||
"""Tests for aggregate_atop, usage_report's pure-Python fallback parser used
|
||||
when the native atop_agg helper is unavailable.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
import _usage_report_parsing as parsing
|
||||
from _usage_report_types import _Progress
|
||||
|
||||
if TYPE_CHECKING:
|
||||
import pytest
|
||||
|
||||
|
||||
def _progress() -> _Progress:
|
||||
"""A progress reporter that never draws, so stderr stays clean."""
|
||||
return _Progress(enabled=False, total_stages=1)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# aggregate_atop (pure-Python fallback)
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_aggregate_atop_parses_prc_and_prm(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
"""With no native helper, the Python parser folds PRC and PRM records."""
|
||||
log = tmp_path / "atop.log"
|
||||
log.write_bytes(b"x" * 100)
|
||||
lines = [
|
||||
"# a comment",
|
||||
"RESET",
|
||||
"SEP",
|
||||
"",
|
||||
"PRC host 1000 d t 600 42 (python) S 100 500 20 a b c",
|
||||
"PRC host 1600 d t 600 42 (python) S 100 900 40 a b c",
|
||||
"PRM host 1000 d t 600 42 (python) S 4096 9999 2048 a b c",
|
||||
"PRC short row",
|
||||
]
|
||||
monkeypatch.setattr(parsing, "_atop_agg_binary", lambda: None)
|
||||
monkeypatch.setattr(parsing, "_iter_atop_lines", lambda *_a, **_k: iter(lines))
|
||||
|
||||
agg, window = parsing.aggregate_atop(log, _progress())
|
||||
|
||||
# utime+stime deltas: (900+40) - (500+20) = 420.
|
||||
assert agg["python"].cpu_ticks == 420
|
||||
assert agg["python"].peak_rss_kb == 2048
|
||||
assert window.distinct_samples == 2
|
||||
assert window.interval_s == 600
|
||||
|
||||
|
||||
def test_aggregate_atop_ignores_unparseable_epoch(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
"""A PRC row with a non-numeric epoch still parses, minus the timestamp."""
|
||||
log = tmp_path / "atop.log"
|
||||
log.write_bytes(b"x" * 10)
|
||||
lines = ["PRC host notanepoch d t 600 42 (python) S 100 500 20 a b c"]
|
||||
monkeypatch.setattr(parsing, "_atop_agg_binary", lambda: None)
|
||||
monkeypatch.setattr(parsing, "_iter_atop_lines", lambda *_a, **_k: iter(lines))
|
||||
|
||||
agg, window = parsing.aggregate_atop(log, _progress())
|
||||
|
||||
assert agg["python"].pid_count == 1
|
||||
assert window.distinct_samples == 0
|
||||
|
||||
|
||||
def test_aggregate_atop_delegates_to_native_when_available(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
"""When the C helper builds, the native path is used instead."""
|
||||
log = tmp_path / "atop.log"
|
||||
log.write_bytes(b"x")
|
||||
monkeypatch.setattr(parsing, "_atop_agg_binary", lambda: Path("/bin/atop_agg"))
|
||||
called: list[Path] = []
|
||||
|
||||
def fake_native(
|
||||
log_arg: Path,
|
||||
_progress_arg: _Progress,
|
||||
binary: Path,
|
||||
_begin: str | None = None,
|
||||
_end: str | None = None,
|
||||
) -> tuple[dict[str, object], object]:
|
||||
called.append(binary)
|
||||
return {}, parsing._Window()
|
||||
|
||||
monkeypatch.setattr(parsing, "_aggregate_atop_native", fake_native)
|
||||
|
||||
parsing.aggregate_atop(log, _progress())
|
||||
|
||||
assert called == [Path("/bin/atop_agg")]
|
||||
|
||||
|
||||
def test_aggregate_atop_skips_whitespace_only_rows(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
"""A row of only whitespace splits to nothing and is skipped."""
|
||||
log = tmp_path / "atop.log"
|
||||
log.write_bytes(b"x")
|
||||
monkeypatch.setattr(parsing, "_atop_agg_binary", lambda: None)
|
||||
monkeypatch.setattr(parsing, "_iter_atop_lines", lambda *_a, **_k: iter([" "]))
|
||||
|
||||
agg, window = parsing.aggregate_atop(log, _progress())
|
||||
|
||||
assert agg == {}
|
||||
assert window.distinct_samples == 0
|
||||
@ -1,134 +0,0 @@
|
||||
"""Tests for usage_report's native atop_agg helper: when the cached binary is
|
||||
reused, when it is rebuilt, and every way the build can fail into the
|
||||
pure-Python fallback.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
import _usage_report_atop as parsing
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
def test_atop_agg_binary_uses_fresh_cache(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
"""A cached binary newer than the source is returned without rebuilding."""
|
||||
src_dir = tmp_path / "src"
|
||||
src_dir.mkdir()
|
||||
src_c = src_dir / "atop_agg.c"
|
||||
src_c.write_text("int main(void){return 0;}", encoding="utf-8")
|
||||
cache = tmp_path / "cached"
|
||||
cache.write_bytes(b"binary")
|
||||
monkeypatch.setattr(parsing, "_ATOP_AGG_SRC_DIR", src_dir)
|
||||
monkeypatch.setattr(parsing, "_ATOP_AGG_CACHE_BIN", cache)
|
||||
|
||||
def fail(*_a: object, **_k: object) -> None:
|
||||
msg = "should not rebuild when the cache is fresh"
|
||||
raise AssertionError(msg)
|
||||
|
||||
monkeypatch.setattr(parsing.subprocess, "run", fail)
|
||||
|
||||
assert parsing._atop_agg_binary() == cache
|
||||
|
||||
|
||||
def test_atop_agg_binary_none_without_compiler(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
"""With a stale cache and no `cc`, the caller falls back to Python."""
|
||||
src_dir = tmp_path / "src"
|
||||
src_dir.mkdir()
|
||||
(src_dir / "atop_agg.c").write_text("x", encoding="utf-8")
|
||||
monkeypatch.setattr(parsing, "_ATOP_AGG_SRC_DIR", src_dir)
|
||||
monkeypatch.setattr(parsing, "_ATOP_AGG_CACHE_BIN", tmp_path / "absent")
|
||||
monkeypatch.setattr(parsing.shutil, "which", lambda _name: None)
|
||||
|
||||
assert parsing._atop_agg_binary() is None
|
||||
|
||||
|
||||
def test_atop_agg_binary_none_when_build_fails(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
"""A failing `make` falls back rather than propagating the error."""
|
||||
src_dir = tmp_path / "src"
|
||||
src_dir.mkdir()
|
||||
(src_dir / "atop_agg.c").write_text("x", encoding="utf-8")
|
||||
monkeypatch.setattr(parsing, "_ATOP_AGG_SRC_DIR", src_dir)
|
||||
monkeypatch.setattr(parsing, "_ATOP_AGG_CACHE_BIN", tmp_path / "cache" / "bin")
|
||||
monkeypatch.setattr(parsing.shutil, "which", lambda _name: "/usr/bin/cc")
|
||||
|
||||
def boom(*_a: object, **_k: object) -> None:
|
||||
raise subprocess.CalledProcessError(1, "make")
|
||||
|
||||
monkeypatch.setattr(parsing.subprocess, "run", boom)
|
||||
|
||||
assert parsing._atop_agg_binary() is None
|
||||
|
||||
|
||||
def test_atop_agg_binary_none_when_make_produces_nothing(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
"""A `make` that exits 0 without writing the binary still falls back."""
|
||||
src_dir = tmp_path / "src"
|
||||
src_dir.mkdir()
|
||||
(src_dir / "atop_agg.c").write_text("x", encoding="utf-8")
|
||||
monkeypatch.setattr(parsing, "_ATOP_AGG_SRC_DIR", src_dir)
|
||||
monkeypatch.setattr(parsing, "_ATOP_AGG_CACHE_BIN", tmp_path / "cache" / "bin")
|
||||
monkeypatch.setattr(parsing.shutil, "which", lambda _name: "/usr/bin/cc")
|
||||
monkeypatch.setattr(
|
||||
parsing.subprocess,
|
||||
"run",
|
||||
lambda *_a, **_k: subprocess.CompletedProcess([], 0),
|
||||
)
|
||||
|
||||
assert parsing._atop_agg_binary() is None
|
||||
|
||||
|
||||
def test_atop_agg_binary_builds_and_caches(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
"""A successful build is copied into the cache and made executable."""
|
||||
src_dir = tmp_path / "src"
|
||||
src_dir.mkdir()
|
||||
(src_dir / "atop_agg.c").write_text("x", encoding="utf-8")
|
||||
cache = tmp_path / "cache" / "atop_agg"
|
||||
monkeypatch.setattr(parsing, "_ATOP_AGG_SRC_DIR", src_dir)
|
||||
monkeypatch.setattr(parsing, "_ATOP_AGG_CACHE_BIN", cache)
|
||||
monkeypatch.setattr(parsing.shutil, "which", lambda _name: "/usr/bin/cc")
|
||||
|
||||
def fake_make(*_a: object, **_k: object) -> subprocess.CompletedProcess[bytes]:
|
||||
(src_dir / "atop_agg").write_bytes(b"ELF")
|
||||
return subprocess.CompletedProcess([], 0)
|
||||
|
||||
monkeypatch.setattr(parsing.subprocess, "run", fake_make)
|
||||
|
||||
result = parsing._atop_agg_binary()
|
||||
|
||||
assert result == cache
|
||||
assert cache.read_bytes() == b"ELF"
|
||||
assert cache.stat().st_mode & 0o777 == parsing._ATOP_AGG_BIN_MODE
|
||||
|
||||
|
||||
def test_atop_agg_binary_missing_source_falls_back(
|
||||
tmp_path: Path,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""A deleted C source tree yields None (Python fallback) even when a cached
|
||||
binary exists - never trust an orphaned, unverifiable build."""
|
||||
monkeypatch.setattr(parsing, "_ATOP_AGG_SRC_DIR", tmp_path / "gone")
|
||||
cache = tmp_path / "atop_agg"
|
||||
cache.write_text("stale binary", encoding="utf-8")
|
||||
monkeypatch.setattr(parsing, "_ATOP_AGG_CACHE_BIN", cache)
|
||||
|
||||
assert parsing._atop_agg_binary() is None
|
||||
@ -1,142 +0,0 @@
|
||||
"""Tests for usage_report's atop subprocess plumbing and the native atop_agg
|
||||
helper: command construction, line streaming, binary build/cache decisions and
|
||||
the TSV rows the C helper emits.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
import _usage_report_atop as parsing
|
||||
from _usage_report_types import ProcAgg, _Progress
|
||||
from typing_extensions import Self
|
||||
|
||||
if TYPE_CHECKING:
|
||||
import pytest
|
||||
|
||||
|
||||
def _progress() -> _Progress:
|
||||
"""A progress reporter that never draws, so stderr stays clean."""
|
||||
return _Progress(enabled=False, total_stages=1)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# _run
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_run_returns_stdout(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""A successful command's stdout is returned verbatim."""
|
||||
monkeypatch.setattr(
|
||||
parsing.subprocess,
|
||||
"run",
|
||||
lambda *_a, **_k: subprocess.CompletedProcess([], 0, stdout="out", stderr=""),
|
||||
)
|
||||
|
||||
assert parsing._run(["true"]) == "out"
|
||||
|
||||
|
||||
def test_run_swallows_os_error(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""A missing binary yields an empty string rather than raising."""
|
||||
|
||||
def boom(*_a: object, **_k: object) -> None:
|
||||
msg = "no such binary"
|
||||
raise OSError(msg)
|
||||
|
||||
monkeypatch.setattr(parsing.subprocess, "run", boom)
|
||||
|
||||
assert parsing._run(["nope"]) == ""
|
||||
|
||||
|
||||
def test_run_swallows_timeout(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""A command that overruns its timeout also yields an empty string."""
|
||||
|
||||
def boom(*_a: object, **_k: object) -> None:
|
||||
raise subprocess.TimeoutExpired(cmd="slow", timeout=60)
|
||||
|
||||
monkeypatch.setattr(parsing.subprocess, "run", boom)
|
||||
|
||||
assert parsing._run(["slow"]) == ""
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# _iter_atop_lines
|
||||
# --------------------------------------------------------------------------- #
|
||||
class _FakePopen:
|
||||
"""Minimal Popen stand-in yielding canned stdout lines."""
|
||||
|
||||
def __init__(self, lines: list[str] | None) -> None:
|
||||
self.stdout = lines
|
||||
self.stdin = None
|
||||
|
||||
def __enter__(self) -> Self:
|
||||
return self
|
||||
|
||||
def __exit__(self, *_exc: object) -> None:
|
||||
"""Never suppresses an exception."""
|
||||
|
||||
|
||||
def test_iter_atop_lines_strips_newlines(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""Each streamed line comes back without its trailing newline."""
|
||||
monkeypatch.setattr(
|
||||
parsing.subprocess,
|
||||
"Popen",
|
||||
lambda *_a, **_k: _FakePopen(["a\n", "b\n"]),
|
||||
)
|
||||
|
||||
assert list(parsing._iter_atop_lines(Path("log"), "PRC")) == ["a", "b"]
|
||||
|
||||
|
||||
def test_iter_atop_lines_handles_no_stdout(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""A process with no stdout pipe yields nothing instead of raising."""
|
||||
monkeypatch.setattr(
|
||||
parsing.subprocess,
|
||||
"Popen",
|
||||
lambda *_a, **_k: _FakePopen(None),
|
||||
)
|
||||
|
||||
assert list(parsing._iter_atop_lines(Path("log"), "PRC")) == []
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# _atop_agg_binary
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# _apply_native_name / _window_from_native
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_apply_native_name_fills_every_field() -> None:
|
||||
"""An N row populates the ProcAgg, with pid_set sized to the PID count."""
|
||||
agg: dict[str, ProcAgg] = {}
|
||||
|
||||
parsing._apply_native_name(
|
||||
["N", "firefox", "1200", "8192", "4096", "10", "3"],
|
||||
agg,
|
||||
)
|
||||
|
||||
entry = agg["firefox"]
|
||||
assert entry.cpu_ticks == 1200
|
||||
assert entry.peak_rss_kb == 8192
|
||||
assert entry.rss_kb_sum == 4096
|
||||
assert entry.rss_samples == 10
|
||||
assert entry.pid_count == 3
|
||||
|
||||
|
||||
def test_window_from_native_zero_samples_is_empty() -> None:
|
||||
"""A W row reporting no epochs yields the default empty window."""
|
||||
window = parsing._window_from_native(["W", "0", "0", "0", "0"])
|
||||
|
||||
assert window.distinct_samples == 0
|
||||
assert window.start == "n/a"
|
||||
|
||||
|
||||
def test_window_from_native_spans_the_reported_epochs() -> None:
|
||||
"""A populated W row carries its bounds, count and interval through."""
|
||||
window = parsing._window_from_native(["W", "1000", "4600", "7", "600"])
|
||||
|
||||
assert window.distinct_samples == 7
|
||||
assert window.interval_s == 600
|
||||
assert window.seconds == 3600
|
||||
assert window.start_epoch == 1000
|
||||
assert window.end_epoch == 4600
|
||||
@ -1,135 +0,0 @@
|
||||
"""Tests for the `atop | atop_agg` pipeline: how usage_report reads the native
|
||||
helper's TSV rows and copes with a producer or consumer that has no stdout.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
import _usage_report_atop as parsing
|
||||
from _usage_report_types import _Progress
|
||||
from typing_extensions import Self
|
||||
|
||||
if TYPE_CHECKING:
|
||||
import pytest
|
||||
|
||||
|
||||
def _progress() -> _Progress:
|
||||
"""A progress reporter that never draws, so stderr stays clean."""
|
||||
return _Progress(enabled=False, total_stages=1)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# _aggregate_atop_native
|
||||
# --------------------------------------------------------------------------- #
|
||||
class _ClosableLines(list[str]):
|
||||
"""A line list that also answers .close(), as a real pipe would."""
|
||||
|
||||
def close(self) -> None:
|
||||
"""Called on the producer's stdout by the pipeline code."""
|
||||
|
||||
|
||||
class _FakePipeline:
|
||||
"""Two-process context manager mimicking the atop | atop_agg pipeline."""
|
||||
|
||||
def __init__(self, lines: list[str] | None) -> None:
|
||||
self.stdout = None if lines is None else _ClosableLines(lines)
|
||||
|
||||
def __enter__(self) -> Self:
|
||||
return self
|
||||
|
||||
def __exit__(self, *_exc: object) -> None:
|
||||
"""Never suppresses an exception."""
|
||||
|
||||
|
||||
def _fake_popen_factory(
|
||||
rows: list[str] | None,
|
||||
) -> object:
|
||||
"""Build a Popen replacement: first call is atop, second is atop_agg."""
|
||||
calls: list[int] = []
|
||||
|
||||
def fake_popen(*_a: object, **_k: object) -> _FakePipeline:
|
||||
calls.append(1)
|
||||
# The first Popen is atop (its stdout is only closed by the caller);
|
||||
# the second is atop_agg, whose stdout the loop reads.
|
||||
return _FakePipeline(["x"] if len(calls) == 1 else rows)
|
||||
|
||||
return fake_popen
|
||||
|
||||
|
||||
def test_aggregate_atop_native_reads_n_and_w_rows(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""N rows become aggregates and a W row becomes the window."""
|
||||
rows = [
|
||||
"N\tfirefox\t900\t4096\t2048\t5\t2\n",
|
||||
"W\t100\t3700\t6\t600\n",
|
||||
]
|
||||
monkeypatch.setattr(parsing.subprocess, "Popen", _fake_popen_factory(rows))
|
||||
|
||||
agg, window = parsing._aggregate_atop_native(
|
||||
Path("log"),
|
||||
_progress(),
|
||||
Path("/bin/atop_agg"),
|
||||
)
|
||||
|
||||
assert agg["firefox"].cpu_ticks == 900
|
||||
assert window.distinct_samples == 6
|
||||
assert window.interval_s == 600
|
||||
|
||||
|
||||
def test_aggregate_atop_native_ignores_malformed_rows(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Rows with the wrong field count are skipped, not fatal."""
|
||||
rows = ["N\ttoo\tshort\n", "W\tbad\n", "Z\tunknown\ttag\n"]
|
||||
monkeypatch.setattr(parsing.subprocess, "Popen", _fake_popen_factory(rows))
|
||||
|
||||
agg, window = parsing._aggregate_atop_native(
|
||||
Path("log"),
|
||||
_progress(),
|
||||
Path("/bin/atop_agg"),
|
||||
)
|
||||
|
||||
assert agg == {}
|
||||
assert window.distinct_samples == 0
|
||||
|
||||
|
||||
def test_aggregate_atop_native_handles_no_stdout(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""A helper with no stdout pipe returns empty results rather than raising."""
|
||||
monkeypatch.setattr(parsing.subprocess, "Popen", _fake_popen_factory(None))
|
||||
|
||||
agg, window = parsing._aggregate_atop_native(
|
||||
Path("log"),
|
||||
_progress(),
|
||||
Path("/bin/atop_agg"),
|
||||
)
|
||||
|
||||
assert agg == {}
|
||||
assert window.distinct_samples == 0
|
||||
|
||||
|
||||
def test_aggregate_atop_native_tolerates_producer_without_stdout(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""When atop exposes no stdout there is nothing to close; the read proceeds."""
|
||||
calls: list[int] = []
|
||||
|
||||
def fake_popen(*_a: object, **_k: object) -> _FakePipeline:
|
||||
calls.append(1)
|
||||
# First Popen is atop: give it no stdout, so the close() is skipped.
|
||||
return _FakePipeline(None if len(calls) == 1 else ["W\t0\t0\t0\t0\n"])
|
||||
|
||||
monkeypatch.setattr(parsing.subprocess, "Popen", fake_popen)
|
||||
|
||||
agg, window = parsing._aggregate_atop_native(
|
||||
Path("log"),
|
||||
_progress(),
|
||||
Path("/bin/atop_agg"),
|
||||
)
|
||||
|
||||
assert agg == {}
|
||||
assert window.distinct_samples == 0
|
||||
@ -1,179 +0,0 @@
|
||||
"""Tests for usage_report's command-line surface: argument parsing, the
|
||||
single-day and since-last-report entry points, and main's dispatch between
|
||||
them.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime as _dt
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
import _usage_report_run as run
|
||||
from _usage_report_types import _Progress, _Window
|
||||
import usage_report
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Callable
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
def _progress() -> _Progress:
|
||||
"""A progress reporter that never draws, so stderr stays clean."""
|
||||
return _Progress(enabled=False, total_stages=1)
|
||||
|
||||
|
||||
def _args(**overrides: object) -> object:
|
||||
"""Build a parsed-args namespace with the parser's own defaults."""
|
||||
args = usage_report._build_parser().parse_args([])
|
||||
for key, value in overrides.items():
|
||||
setattr(args, key, value)
|
||||
return args
|
||||
|
||||
|
||||
def _aggregates(*, days_with_data: int = 1, seconds: int = 600) -> object:
|
||||
"""Build an _Aggregates with a window that looks like real coverage."""
|
||||
return run._Aggregates(
|
||||
cpu={},
|
||||
gpu={},
|
||||
window=_Window(distinct_samples=2, interval_s=600, seconds=seconds),
|
||||
gpu_samples=0,
|
||||
days_with_data=days_with_data,
|
||||
)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# _build_parser
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_parser_defaults() -> None:
|
||||
"""With no flags every option takes its documented default."""
|
||||
args = usage_report._build_parser().parse_args([])
|
||||
|
||||
assert args.date is None
|
||||
assert args.since is None
|
||||
assert args.top == usage_report._DEFAULT_TOP
|
||||
assert args.atop_log is None
|
||||
assert args.pmon_log is None
|
||||
assert args.no_clipboard is False
|
||||
assert args.no_update_state is False
|
||||
assert args.quiet is False
|
||||
|
||||
|
||||
def test_parser_reads_every_flag() -> None:
|
||||
"""Each flag lands on the namespace with the right type."""
|
||||
args = usage_report._build_parser().parse_args(
|
||||
[
|
||||
"--date",
|
||||
"20260817",
|
||||
"--since",
|
||||
"20260801",
|
||||
"--top",
|
||||
"20",
|
||||
"--atop-log",
|
||||
"/nonexistent/a",
|
||||
"--pmon-log",
|
||||
"/nonexistent/p",
|
||||
"--no-clipboard",
|
||||
"--no-update-state",
|
||||
"--quiet",
|
||||
],
|
||||
)
|
||||
|
||||
assert args.date == "20260817"
|
||||
assert args.since == "20260801"
|
||||
assert args.top == 20
|
||||
assert args.atop_log == Path("/nonexistent/a")
|
||||
assert args.pmon_log == Path("/nonexistent/p")
|
||||
assert args.no_clipboard is True
|
||||
assert args.no_update_state is True
|
||||
assert args.quiet is True
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# main dispatch
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_main_routes_single_day(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""A --date run goes to the single-day path."""
|
||||
seen: list[str] = []
|
||||
|
||||
def record(label: str) -> Callable[..., int]:
|
||||
def run(*_a: object) -> int:
|
||||
seen.append(label)
|
||||
return 0
|
||||
|
||||
return run
|
||||
|
||||
monkeypatch.setattr(usage_report, "_run_single_day", record("single"))
|
||||
monkeypatch.setattr(usage_report, "_run_since", record("since"))
|
||||
|
||||
assert usage_report.main(["--date", "20260817"]) == 0
|
||||
assert seen == ["single"]
|
||||
|
||||
|
||||
def test_main_routes_since_by_default(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""With no day-selecting flag the since-last-report path runs."""
|
||||
seen: list[str] = []
|
||||
|
||||
def record(label: str) -> Callable[..., int]:
|
||||
def run(*_a: object) -> int:
|
||||
seen.append(label)
|
||||
return 0
|
||||
|
||||
return run
|
||||
|
||||
monkeypatch.setattr(usage_report, "_run_single_day", record("single"))
|
||||
monkeypatch.setattr(usage_report, "_run_since", record("since"))
|
||||
|
||||
assert usage_report.main([]) == 0
|
||||
assert seen == ["since"]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# _run_single_day
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_run_single_day_emits_a_report(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""The single-day path renders once and writes the result out."""
|
||||
monkeypatch.setattr(run, "_preflight", lambda _log: None)
|
||||
monkeypatch.setattr(
|
||||
run,
|
||||
"_resolve_logs",
|
||||
lambda _date: (Path("/nonexistent/atop"), Path("/nonexistent/pmon")),
|
||||
)
|
||||
monkeypatch.setattr(run, "_aggregate_segments", lambda *_a: _aggregates())
|
||||
monkeypatch.setattr(run, "_log_descriptions", lambda _s: ("a", "p"))
|
||||
monkeypatch.setattr(run, "_render_report", lambda *_a, **k: k["period_line"])
|
||||
emitted: list[str] = []
|
||||
monkeypatch.setattr(run, "_emit", lambda _args, r: emitted.append(r))
|
||||
|
||||
rc = run._run_single_day(
|
||||
_args(date="20260817", quiet=True),
|
||||
_dt.datetime.now().astimezone(),
|
||||
)
|
||||
|
||||
assert rc == 0
|
||||
assert emitted == ["- **Reporting period**: 20260817 (single day)"]
|
||||
|
||||
|
||||
def test_run_single_day_defaults_an_empty_window_to_a_day(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""A window with no measured seconds is treated as a full day."""
|
||||
aggs = _aggregates(seconds=0)
|
||||
monkeypatch.setattr(run, "_preflight", lambda _log: None)
|
||||
monkeypatch.setattr(
|
||||
run,
|
||||
"_resolve_logs",
|
||||
lambda _date: (Path("/nonexistent/atop"), Path("/nonexistent/pmon")),
|
||||
)
|
||||
monkeypatch.setattr(run, "_aggregate_segments", lambda *_a: aggs)
|
||||
monkeypatch.setattr(run, "_log_descriptions", lambda _s: ("a", "p"))
|
||||
monkeypatch.setattr(run, "_render_report", lambda *_a, **_k: "report")
|
||||
monkeypatch.setattr(run, "_emit", lambda *_a: None)
|
||||
|
||||
run._run_single_day(
|
||||
_args(date="20260817", quiet=True),
|
||||
_dt.datetime.now().astimezone(),
|
||||
)
|
||||
|
||||
assert aggs.window.seconds == run._SEC_PER_DAY
|
||||
@ -1,159 +0,0 @@
|
||||
"""Tests for usage_report's output side and its preflight checks: clipboard
|
||||
tool selection, stdout emission, the atop/log existence guards, and the
|
||||
since-last-report run including its no-data path.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
|
||||
import _usage_report_emit as emit
|
||||
import _usage_report_run as run
|
||||
from _usage_report_types import _Window
|
||||
import pytest
|
||||
import usage_report
|
||||
|
||||
|
||||
def _args(**overrides: object) -> object:
|
||||
"""Build a parsed-args namespace with the parser's own defaults."""
|
||||
args = usage_report._build_parser().parse_args([])
|
||||
for key, value in overrides.items():
|
||||
setattr(args, key, value)
|
||||
return args
|
||||
|
||||
|
||||
def _aggregates(*, days_with_data: int = 1, seconds: int = 600) -> object:
|
||||
"""Build an _Aggregates with a window that looks like real coverage."""
|
||||
return run._Aggregates(
|
||||
cpu={},
|
||||
gpu={},
|
||||
window=_Window(distinct_samples=2, interval_s=600, seconds=seconds),
|
||||
gpu_samples=0,
|
||||
days_with_data=days_with_data,
|
||||
)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# _preflight
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_preflight_exits_without_atop(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""A missing atop binary aborts with an install hint."""
|
||||
monkeypatch.setattr(emit.shutil, "which", lambda _n: None)
|
||||
|
||||
with pytest.raises(SystemExit) as excinfo:
|
||||
emit._preflight(Path("/nonexistent/whatever"))
|
||||
|
||||
assert "not installed" in str(excinfo.value)
|
||||
|
||||
|
||||
def test_preflight_exits_without_log(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
"""A missing log file aborts and names the path."""
|
||||
monkeypatch.setattr(emit.shutil, "which", lambda _n: "/usr/bin/atop")
|
||||
missing = tmp_path / "absent"
|
||||
|
||||
with pytest.raises(SystemExit) as excinfo:
|
||||
emit._preflight(missing)
|
||||
|
||||
assert str(missing) in str(excinfo.value)
|
||||
|
||||
|
||||
def test_preflight_passes_when_both_present(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
"""With atop installed and the log present, preflight is silent."""
|
||||
monkeypatch.setattr(emit.shutil, "which", lambda _n: "/usr/bin/atop")
|
||||
log = tmp_path / "atop.log"
|
||||
log.write_text("x", encoding="utf-8")
|
||||
|
||||
assert emit._preflight(log) is None
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# _copy_to_clipboard
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_copy_to_clipboard_uses_first_available_tool(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
capsys: pytest.CaptureFixture[str],
|
||||
) -> None:
|
||||
"""The first candidate present on PATH is the one invoked."""
|
||||
monkeypatch.setattr(
|
||||
emit.shutil,
|
||||
"which",
|
||||
lambda name: "/usr/bin/xclip" if name == "xclip" else None,
|
||||
)
|
||||
calls: list[tuple[str, ...]] = []
|
||||
monkeypatch.setattr(
|
||||
emit.subprocess,
|
||||
"run",
|
||||
lambda cmd, **_k: calls.append(tuple(cmd)),
|
||||
)
|
||||
|
||||
emit._copy_to_clipboard("hello")
|
||||
|
||||
assert calls == [("xclip", "-selection", "clipboard")]
|
||||
assert "copied 5 chars via xclip" in capsys.readouterr().err
|
||||
|
||||
|
||||
def test_copy_to_clipboard_reports_a_failing_tool(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
capsys: pytest.CaptureFixture[str],
|
||||
) -> None:
|
||||
"""A tool that errors is reported on stderr and not retried elsewhere."""
|
||||
monkeypatch.setattr(emit.shutil, "which", lambda _n: "/usr/bin/wl-copy")
|
||||
|
||||
def boom(*_a: object, **_k: object) -> None:
|
||||
raise subprocess.CalledProcessError(1, "wl-copy")
|
||||
|
||||
monkeypatch.setattr(emit.subprocess, "run", boom)
|
||||
|
||||
emit._copy_to_clipboard("hello")
|
||||
|
||||
assert "wl-copy failed" in capsys.readouterr().err
|
||||
|
||||
|
||||
def test_copy_to_clipboard_without_any_tool(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
capsys: pytest.CaptureFixture[str],
|
||||
) -> None:
|
||||
"""With no clipboard tool installed the copy is skipped, not fatal."""
|
||||
monkeypatch.setattr(emit.shutil, "which", lambda _n: None)
|
||||
|
||||
emit._copy_to_clipboard("hello")
|
||||
|
||||
assert "skipping copy" in capsys.readouterr().err
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# _emit
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_emit_writes_stdout_and_copies(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
capsys: pytest.CaptureFixture[str],
|
||||
) -> None:
|
||||
"""By default the report goes to stdout and the clipboard."""
|
||||
copied: list[str] = []
|
||||
monkeypatch.setattr(emit, "_copy_to_clipboard", copied.append)
|
||||
|
||||
emit._emit(_args(no_clipboard=False), "REPORT")
|
||||
|
||||
assert capsys.readouterr().out == "REPORT"
|
||||
assert copied == ["REPORT"]
|
||||
|
||||
|
||||
def test_emit_skips_clipboard_when_suppressed(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
capsys: pytest.CaptureFixture[str],
|
||||
) -> None:
|
||||
"""--no-clipboard leaves stdout untouched but skips the copy."""
|
||||
copied: list[str] = []
|
||||
monkeypatch.setattr(emit, "_copy_to_clipboard", copied.append)
|
||||
|
||||
emit._emit(_args(no_clipboard=True), "REPORT")
|
||||
|
||||
assert capsys.readouterr().out == "REPORT"
|
||||
assert copied == []
|
||||
@ -1,40 +0,0 @@
|
||||
"""Tests for usage_report's Markdown escaping and duration formatting."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import _usage_report_format as fmt
|
||||
import pytest
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# _fmt_h
|
||||
# --------------------------------------------------------------------------- #
|
||||
@pytest.mark.parametrize(
|
||||
("seconds", "expected"),
|
||||
[
|
||||
(0.0, "0.0s"),
|
||||
(8.34, "8.3s"),
|
||||
(59.9, "59.9s"),
|
||||
(60.0, "1m 00s"),
|
||||
(252.0, "4m 12s"),
|
||||
(3599.0, "59m 59s"),
|
||||
(3600.0, "1h 00m"),
|
||||
(5000.0, "1h 23m"),
|
||||
],
|
||||
)
|
||||
def test_fmt_h_picks_unit_by_magnitude(seconds: float, expected: str) -> None:
|
||||
"""Durations switch h/m/s at exactly 3600 and 60 seconds."""
|
||||
assert fmt._fmt_h(seconds) == expected
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# _md_escape
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_md_escape_escapes_pipe_and_flattens_newline() -> None:
|
||||
"""Pipes are backslash-escaped and newlines become spaces."""
|
||||
assert fmt._md_escape("a|b\nc") == r"a\|b c"
|
||||
|
||||
|
||||
def test_md_escape_leaves_plain_names_untouched() -> None:
|
||||
"""A name with no table-breaking characters passes through unchanged."""
|
||||
assert fmt._md_escape("firefox") == "firefox"
|
||||
@ -1,56 +0,0 @@
|
||||
"""Tests for usage_report's GPU Markdown table."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import _usage_report_tables as tables
|
||||
from _usage_report_types import GpuAgg
|
||||
|
||||
|
||||
def _cells(row: str) -> list[str]:
|
||||
"""Split a Markdown table row into stripped cell values."""
|
||||
return [c.strip() for c in row.strip().strip("|").split("|")]
|
||||
|
||||
|
||||
def test_gpu_table_sorts_by_gpu_seconds_and_honours_top() -> None:
|
||||
"""Rows are ordered by SM-seconds descending and truncated to *top*."""
|
||||
aggs = {
|
||||
"low": GpuAgg("low", sm_pct_sum=10.0, samples=1),
|
||||
"high": GpuAgg("high", sm_pct_sum=900.0, samples=1),
|
||||
"mid": GpuAgg("mid", sm_pct_sum=100.0, samples=1),
|
||||
}
|
||||
|
||||
rows = tables._gpu_table(aggs, total_samples=10, top=2)
|
||||
|
||||
assert len(rows) == 4
|
||||
assert _cells(rows[2])[1] == "high"
|
||||
assert _cells(rows[3])[1] == "mid"
|
||||
|
||||
|
||||
def test_gpu_table_reports_sample_presence_percentage() -> None:
|
||||
"""The Samples column shows the share of total samples the process was in."""
|
||||
aggs = {"g": GpuAgg("g", sm_pct_sum=50.0, samples=5)}
|
||||
|
||||
cells = _cells(tables._gpu_table(aggs, total_samples=20, top=1)[2])
|
||||
|
||||
assert cells[6] == "5 (25%)"
|
||||
|
||||
|
||||
def test_gpu_table_zero_total_samples_yields_zero_presence() -> None:
|
||||
"""A zero sample total cannot be divided by, so presence renders as 0%."""
|
||||
aggs = {"g": GpuAgg("g", sm_pct_sum=50.0, samples=5)}
|
||||
|
||||
cells = _cells(tables._gpu_table(aggs, total_samples=0, top=1)[2])
|
||||
|
||||
assert cells[6] == "5 (0%)"
|
||||
|
||||
|
||||
def test_gpu_table_escapes_program_names() -> None:
|
||||
"""A pipe in a GPU process name cannot break out of its cell."""
|
||||
aggs = {"a|b": GpuAgg("a|b", sm_pct_sum=1.0, samples=1)}
|
||||
|
||||
assert r"a\|b" in tables._gpu_table(aggs, total_samples=1, top=1)[2]
|
||||
|
||||
|
||||
def test_gpu_table_empty_input_is_header_only() -> None:
|
||||
"""No GPU aggregates means header and separator only."""
|
||||
assert len(tables._gpu_table({}, total_samples=0, top=5)) == 2
|
||||
@ -1,150 +0,0 @@
|
||||
"""Tests for usage_report's host fingerprint: the /proc and nvidia-smi probes
|
||||
behind _host_profile, and the Markdown bullets _fingerprint_section builds.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import io
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
import _usage_report_render as render
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
def _fake_proc_files(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
contents: dict[str, str | OSError],
|
||||
) -> None:
|
||||
"""Route Path.open for /proc/cpuinfo and /proc/meminfo to canned text.
|
||||
|
||||
_host_profile only ever opens those two paths, so every caller supplies
|
||||
both and there is no real-filesystem fallback to keep here.
|
||||
"""
|
||||
|
||||
def fake_open(
|
||||
self: Path,
|
||||
*_args: object,
|
||||
**_kwargs: object,
|
||||
) -> io.StringIO:
|
||||
value = contents[str(self)]
|
||||
if isinstance(value, OSError):
|
||||
raise value
|
||||
return io.StringIO(value)
|
||||
|
||||
monkeypatch.setattr(render.Path, "open", fake_open)
|
||||
|
||||
|
||||
def _stub_platform(monkeypatch: pytest.MonkeyPatch, *, cpus: int | None) -> None:
|
||||
"""Pin hostname, kernel and CPU count so assertions are machine-agnostic."""
|
||||
monkeypatch.setattr(render.platform, "node", lambda: "box")
|
||||
monkeypatch.setattr(render.platform, "release", lambda: "6.1.0")
|
||||
monkeypatch.setattr(render.os, "cpu_count", lambda: cpus)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# _host_profile
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_host_profile_reads_cpu_model_and_memory(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""CPU model and total memory are lifted out of the /proc files."""
|
||||
_stub_platform(monkeypatch, cpus=8)
|
||||
monkeypatch.setattr(render, "_run", lambda _cmd: "")
|
||||
_fake_proc_files(
|
||||
monkeypatch,
|
||||
{
|
||||
"/proc/cpuinfo": "flags\t: x\nmodel name\t: Ryzen 9\nmodel name\t: other\n",
|
||||
"/proc/meminfo": "SwapTotal: 1 kB\nMemTotal: 32768000 kB\n",
|
||||
},
|
||||
)
|
||||
|
||||
info = render._host_profile()
|
||||
|
||||
assert info["hostname"] == "box"
|
||||
assert info["kernel"] == "6.1.0"
|
||||
assert info["cpus_online"] == "8"
|
||||
assert info["cpu_model"] == "Ryzen 9"
|
||||
assert info["memory_total_gib"] == "31.2"
|
||||
assert "gpu" not in info
|
||||
|
||||
|
||||
def test_host_profile_includes_gpu_when_nvidia_smi_answers(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Multi-line nvidia-smi output is flattened onto one line."""
|
||||
_stub_platform(monkeypatch, cpus=1)
|
||||
monkeypatch.setattr(render, "_run", lambda _cmd: "GPU0, 8 GiB\nGPU1, 8 GiB\n")
|
||||
_fake_proc_files(monkeypatch, {"/proc/cpuinfo": "", "/proc/meminfo": ""})
|
||||
|
||||
assert render._host_profile()["gpu"] == "GPU0, 8 GiB; GPU1, 8 GiB"
|
||||
|
||||
|
||||
def test_host_profile_survives_unreadable_proc_files(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""An OSError on either /proc file leaves that key absent, not raising."""
|
||||
_stub_platform(monkeypatch, cpus=1)
|
||||
monkeypatch.setattr(render, "_run", lambda _cmd: "")
|
||||
_fake_proc_files(
|
||||
monkeypatch,
|
||||
{
|
||||
"/proc/cpuinfo": OSError("nope"),
|
||||
"/proc/meminfo": OSError("nope"),
|
||||
},
|
||||
)
|
||||
|
||||
info = render._host_profile()
|
||||
|
||||
assert "cpu_model" not in info
|
||||
assert "memory_total_gib" not in info
|
||||
|
||||
|
||||
def test_host_profile_survives_meminfo_without_digits(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""A MemTotal line with no parseable number is skipped, not fatal."""
|
||||
_stub_platform(monkeypatch, cpus=1)
|
||||
monkeypatch.setattr(render, "_run", lambda _cmd: "")
|
||||
_fake_proc_files(
|
||||
monkeypatch,
|
||||
{"/proc/cpuinfo": "", "/proc/meminfo": "MemTotal: kB\n"},
|
||||
)
|
||||
|
||||
assert "memory_total_gib" not in render._host_profile()
|
||||
|
||||
|
||||
def test_host_profile_defaults_cpu_count_to_zero(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""os.cpu_count() returning None renders as '0' rather than 'None'."""
|
||||
_stub_platform(monkeypatch, cpus=None)
|
||||
monkeypatch.setattr(render, "_run", lambda _cmd: "")
|
||||
_fake_proc_files(monkeypatch, {"/proc/cpuinfo": "", "/proc/meminfo": ""})
|
||||
|
||||
assert render._host_profile()["cpus_online"] == "0"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# _fingerprint_section
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_fingerprint_section_renders_each_fact_as_a_bullet(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Every host-profile key becomes one bold Markdown bullet."""
|
||||
monkeypatch.setattr(
|
||||
render,
|
||||
"_host_profile",
|
||||
lambda: {"hostname": "box", "kernel": "6"},
|
||||
)
|
||||
|
||||
assert render._fingerprint_section() == [
|
||||
"## Host",
|
||||
"",
|
||||
"- **hostname**: box",
|
||||
"- **kernel**: 6",
|
||||
"",
|
||||
]
|
||||
@ -1,146 +0,0 @@
|
||||
"""Tests for merging usage_report daily aggregates: process and GPU aggregate
|
||||
merging, window merging, and PID-count carry-over across days.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import _usage_report_parsing as parsing
|
||||
from _usage_report_types import GpuAgg, ProcAgg, _Window
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# PID-count carry-over (types)
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_proc_pid_count_combines_set_and_extra() -> None:
|
||||
"""`pid_count` adds the live set length and merged-in extras."""
|
||||
agg = ProcAgg("x", pid_set={1, 2, 3}, extra_pids=2)
|
||||
|
||||
assert agg.pid_count == 5
|
||||
|
||||
|
||||
def test_gpu_pid_count_combines_set_and_extra() -> None:
|
||||
"""GpuAgg exposes the same combined PID count."""
|
||||
agg = GpuAgg("x", pid_set={9}, extra_pids=4)
|
||||
|
||||
assert agg.pid_count == 5
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Aggregate merging (parsing)
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_merge_proc_aggs_sums_and_takes_peak() -> None:
|
||||
"""CPU/RSS sums accumulate, peak RSS is the max, PID counts add."""
|
||||
dst: dict[str, ProcAgg] = {}
|
||||
parsing.merge_proc_aggs(
|
||||
dst,
|
||||
{
|
||||
"a": ProcAgg(
|
||||
"a",
|
||||
cpu_ticks=100,
|
||||
peak_rss_kb=200,
|
||||
rss_kb_sum=50,
|
||||
rss_samples=2,
|
||||
pid_set={1, 2},
|
||||
)
|
||||
},
|
||||
)
|
||||
parsing.merge_proc_aggs(
|
||||
dst,
|
||||
{
|
||||
"a": ProcAgg(
|
||||
"a",
|
||||
cpu_ticks=10,
|
||||
peak_rss_kb=500,
|
||||
rss_kb_sum=5,
|
||||
rss_samples=1,
|
||||
pid_set={3},
|
||||
)
|
||||
},
|
||||
)
|
||||
|
||||
entry = dst["a"]
|
||||
assert entry.cpu_ticks == 110
|
||||
assert entry.peak_rss_kb == 500
|
||||
assert entry.rss_kb_sum == 55
|
||||
assert entry.rss_samples == 3
|
||||
assert entry.pid_count == 3
|
||||
|
||||
|
||||
def test_merge_gpu_aggs_sums_and_takes_peak() -> None:
|
||||
"""GPU sample sums accumulate and peaks take the max across days."""
|
||||
dst: dict[str, GpuAgg] = {}
|
||||
parsing.merge_gpu_aggs(
|
||||
dst,
|
||||
{
|
||||
"g": GpuAgg(
|
||||
"g",
|
||||
sm_pct_sum=30.0,
|
||||
mem_pct_sum=10.0,
|
||||
samples=3,
|
||||
peak_sm_pct=40.0,
|
||||
peak_mem_pct=20.0,
|
||||
pid_set={1},
|
||||
)
|
||||
},
|
||||
)
|
||||
parsing.merge_gpu_aggs(
|
||||
dst,
|
||||
{
|
||||
"g": GpuAgg(
|
||||
"g",
|
||||
sm_pct_sum=5.0,
|
||||
mem_pct_sum=2.0,
|
||||
samples=1,
|
||||
peak_sm_pct=80.0,
|
||||
peak_mem_pct=15.0,
|
||||
pid_set={2, 3},
|
||||
)
|
||||
},
|
||||
)
|
||||
|
||||
entry = dst["g"]
|
||||
assert entry.sm_pct_sum == 35.0
|
||||
assert entry.samples == 4
|
||||
assert entry.peak_sm_pct == 80.0
|
||||
assert entry.peak_mem_pct == 20.0
|
||||
assert entry.pid_count == 3
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Window merging (parsing)
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_merge_windows_empty_returns_default() -> None:
|
||||
"""Merging no real windows yields the empty default window."""
|
||||
assert parsing.merge_windows([]).distinct_samples == 0
|
||||
assert parsing.merge_windows([_Window()]).distinct_samples == 0
|
||||
|
||||
|
||||
def test_merge_windows_spans_and_sums() -> None:
|
||||
"""Span uses min start / max end; samples and seconds sum; interval is modal."""
|
||||
w_empty = _Window() # distinct_samples == 0, must be ignored
|
||||
w1 = _Window(
|
||||
start="s1",
|
||||
end="e1",
|
||||
distinct_samples=5,
|
||||
interval_s=600,
|
||||
seconds=100,
|
||||
start_epoch=1000,
|
||||
end_epoch=2000,
|
||||
)
|
||||
w2 = _Window(
|
||||
start="s2",
|
||||
end="e2",
|
||||
distinct_samples=3,
|
||||
interval_s=600,
|
||||
seconds=50,
|
||||
start_epoch=500,
|
||||
end_epoch=3000,
|
||||
)
|
||||
|
||||
merged = parsing.merge_windows([w_empty, w1, w2])
|
||||
|
||||
assert merged.start == "s2" # earliest start_epoch (500)
|
||||
assert merged.end == "e2" # latest end_epoch (3000)
|
||||
assert merged.distinct_samples == 8
|
||||
assert merged.seconds == 150
|
||||
assert merged.interval_s == 600
|
||||
@ -1,169 +0,0 @@
|
||||
"""Tests for usage_report run modes and field parsing: start resolution,
|
||||
single-day vs multi-day dispatch, the report period fragments, the PRC
|
||||
HZ-field regression, and native-helper selection.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import datetime as _dt
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
import _usage_report_logs as logs
|
||||
import _usage_report_parsing as parsing
|
||||
import _usage_report_run as run
|
||||
import usage_report
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from _usage_report_types import _PidCpu
|
||||
import pytest
|
||||
|
||||
|
||||
# Aware timezone matching how the parser localizes naive timestamps, so epochs
|
||||
# computed here line up with `_pmon_row_epoch`'s `.astimezone()` conversion.
|
||||
_LOCAL_TZ = _dt.datetime.now().astimezone().tzinfo
|
||||
|
||||
|
||||
def _at(
|
||||
year: int, month: int, day: int, hour: int = 0, minute: int = 0
|
||||
) -> _dt.datetime:
|
||||
"""Build an aware local datetime for tests."""
|
||||
return _dt.datetime(year, month, day, hour, minute, tzinfo=_LOCAL_TZ)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Start resolution and mode dispatch (usage_report)
|
||||
# --------------------------------------------------------------------------- #
|
||||
def _args(**overrides: object) -> argparse.Namespace:
|
||||
"""Build a Namespace with the usage_report CLI defaults."""
|
||||
base: dict[str, object] = {
|
||||
"date": None,
|
||||
"since": None,
|
||||
"atop_log": None,
|
||||
"pmon_log": None,
|
||||
}
|
||||
base.update(overrides)
|
||||
return argparse.Namespace(**base)
|
||||
|
||||
|
||||
def test_resolve_start_prefers_since(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""--since wins over any saved state and starts at local midnight."""
|
||||
monkeypatch.setattr(run, "_read_last_generated", lambda: _at(2026, 1, 1))
|
||||
start = run._resolve_start(_args(since="20260604"), _at(2026, 6, 4, 12))
|
||||
|
||||
assert start.date() == _dt.date(2026, 6, 4)
|
||||
assert (start.hour, start.minute) == (0, 0)
|
||||
|
||||
|
||||
def test_resolve_start_uses_last_report(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""Without --since, the saved last-report timestamp is the start."""
|
||||
last = _at(2026, 6, 2, 9, 0)
|
||||
monkeypatch.setattr(run, "_read_last_generated", lambda: last)
|
||||
|
||||
assert run._resolve_start(_args(), _at(2026, 6, 4, 12)) == last
|
||||
|
||||
|
||||
def test_resolve_start_first_run_is_today_midnight(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""First-ever run (no state) covers today from local midnight."""
|
||||
monkeypatch.setattr(run, "_read_last_generated", lambda: None)
|
||||
now = _at(2026, 6, 4, 12, 30)
|
||||
|
||||
assert run._resolve_start(_args(), now) == _at(2026, 6, 4, 0, 0)
|
||||
|
||||
|
||||
def test_is_single_day_mode() -> None:
|
||||
"""Pinning a date or explicit log path selects single-day mode."""
|
||||
assert usage_report._is_single_day_mode(_args(date="20260604")) is True
|
||||
assert usage_report._is_single_day_mode(_args(atop_log=Path("/x"))) is True
|
||||
assert usage_report._is_single_day_mode(_args(pmon_log=Path("/x"))) is True
|
||||
assert usage_report._is_single_day_mode(_args()) is False
|
||||
|
||||
|
||||
def test_should_advance_state_only_for_default_run() -> None:
|
||||
"""Only a plain since-last-report run re-baselines the saved timestamp."""
|
||||
assert run._should_advance_state(_args(no_update_state=False)) is True
|
||||
assert run._should_advance_state(_args(no_update_state=True)) is False
|
||||
# --since is an ad-hoc query and must never advance state.
|
||||
assert (
|
||||
run._should_advance_state(
|
||||
_args(since="20260510", no_update_state=False),
|
||||
)
|
||||
is False
|
||||
)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Report fragments (usage_report)
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_period_line_contains_both_bounds() -> None:
|
||||
"""The period bullet shows start, end, and the span."""
|
||||
line = run._period_line(_at(2026, 6, 2, 9), _at(2026, 6, 4, 9))
|
||||
|
||||
assert "2026-06-02T09:00:00" in line
|
||||
assert "2026-06-04T09:00:00" in line
|
||||
assert "→" in line
|
||||
|
||||
|
||||
def test_describe_logs_counts() -> None:
|
||||
"""Log description switches between none / single / multiple wording."""
|
||||
assert "none found" in logs._describe_logs([], "atop -r")
|
||||
assert logs._describe_logs(
|
||||
[Path("/var/log/atop/atop_20260604")], "atop -r"
|
||||
).startswith(
|
||||
"`/var/log/atop/atop_20260604`",
|
||||
)
|
||||
many = logs._describe_logs(
|
||||
[Path("/v/atop_20260601"), Path("/v/atop_20260604")],
|
||||
"atop -r",
|
||||
)
|
||||
assert "2 daily logs" in many
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# PRC field parsing — HZ-field regression (parsing)
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_parse_prc_does_not_charge_hz_as_cpu() -> None:
|
||||
"""atop emits `... pid (name) state HZ utime stime`; the HZ column must be
|
||||
skipped, never summed as CPU.
|
||||
|
||||
Regression for the off-by-one that read HZ (100) as utime, which inflated
|
||||
every process's CPU-seconds to its record/PID count (xset showing 67h).
|
||||
"""
|
||||
pid_cpu: dict[int, _PidCpu] = {}
|
||||
# 6 generic fields, pid, (name), state, HZ=100, utime=7, stime=3, + tail.
|
||||
line = "PRC host 1000 2026/06/04 12:00:00 600 4242 (xset) E 100 7 3 0 0 0"
|
||||
|
||||
parsing._parse_prc(line.split(), pid_cpu)
|
||||
|
||||
entry = pid_cpu[4242]
|
||||
assert entry.name == "xset"
|
||||
assert entry.delta_ticks == 10 # utime+stime, never the HZ constant (100)
|
||||
|
||||
|
||||
def test_parse_prc_skips_hz_with_multiword_name() -> None:
|
||||
"""The HZ skip stays aligned when the name spans several tokens."""
|
||||
pid_cpu: dict[int, _PidCpu] = {}
|
||||
line = "PRC h 1000 d t 600 99 (Web Content) S 100 40 2 0 0"
|
||||
|
||||
parsing._parse_prc(line.split(), pid_cpu)
|
||||
|
||||
assert pid_cpu[99].name == "Web Content"
|
||||
assert pid_cpu[99].delta_ticks == 42 # 40+2, HZ(100) skipped
|
||||
|
||||
|
||||
def test_parse_prc_too_short_is_ignored() -> None:
|
||||
"""A truncated PRC record (missing stime) is skipped, not a crash."""
|
||||
pid_cpu: dict[int, _PidCpu] = {}
|
||||
# Tokens run out at utime — no stime at after+3, so the record is dropped.
|
||||
line = "PRC h 1000 d t 600 7 (x) S 100 5"
|
||||
|
||||
parsing._parse_prc(line.split(), pid_cpu)
|
||||
|
||||
assert pid_cpu == {}
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Native helper selection (parsing)
|
||||
@ -1,174 +0,0 @@
|
||||
"""Tests for usage_report's pure-Python atop record parsing: process-name
|
||||
extraction, PRM folding, epoch windows, per-PID collapsing, and the
|
||||
aggregate_atop fallback used when the native helper is unavailable.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import _usage_report_parsing as parsing
|
||||
from _usage_report_types import _PidCpu, _PidRam, _Progress
|
||||
|
||||
|
||||
def _progress() -> _Progress:
|
||||
"""A progress reporter that never draws, so stderr stays clean."""
|
||||
return _Progress(enabled=False, total_stages=1)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# _parse_name
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_parse_name_reads_single_token() -> None:
|
||||
"""The common `(name)` case returns the bare name and the next index."""
|
||||
assert parsing._parse_name(["x", "(firefox)"], 1) == ("firefox", 2)
|
||||
|
||||
|
||||
def test_parse_name_rejoins_a_name_with_spaces() -> None:
|
||||
"""A parenthesised name split across tokens is rejoined."""
|
||||
parts = ["x", "(Draw", "GameThread)", "S"]
|
||||
|
||||
assert parsing._parse_name(parts, 1) == ("Draw GameThread", 3)
|
||||
|
||||
|
||||
def test_parse_name_past_end_is_unknown() -> None:
|
||||
"""An index beyond the record yields the unknown placeholder."""
|
||||
assert parsing._parse_name(["x"], 5) == ("unknown", 6)
|
||||
|
||||
|
||||
def test_parse_name_empty_parens_is_unknown() -> None:
|
||||
"""`()` carries no name, so the placeholder is used."""
|
||||
assert parsing._parse_name(["x", "()"], 1) == ("unknown", 2)
|
||||
|
||||
|
||||
def test_parse_name_unparenthesised_token_passes_through() -> None:
|
||||
"""A token with no leading paren is taken as the name verbatim."""
|
||||
assert parsing._parse_name(["x", "bash"], 1) == ("bash", 2)
|
||||
|
||||
|
||||
def test_parse_name_unterminated_parens_consumes_the_rest() -> None:
|
||||
"""A name whose closing paren never arrives stops at the record end."""
|
||||
name, nxt = parsing._parse_name(["x", "(never", "closed"], 1)
|
||||
|
||||
assert name == "never close"
|
||||
assert nxt == 3
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# _parse_prm
|
||||
# --------------------------------------------------------------------------- #
|
||||
def _prm(pid: str, name: str, rsize: str) -> list[str]:
|
||||
"""Build a PRM record: 6 header fields, then pid name state pagesz vsize rsize."""
|
||||
return [
|
||||
"PRM",
|
||||
"host",
|
||||
"1000",
|
||||
"d",
|
||||
"t",
|
||||
"600",
|
||||
pid,
|
||||
name,
|
||||
"S",
|
||||
"4096",
|
||||
"9999",
|
||||
rsize,
|
||||
]
|
||||
|
||||
|
||||
def test_parse_prm_records_rss() -> None:
|
||||
"""A well-formed PRM row lands in the per-PID RSS map."""
|
||||
ram: dict[int, _PidRam] = {}
|
||||
|
||||
parsing._parse_prm(_prm("42", "(firefox)", "8192"), ram)
|
||||
|
||||
assert ram[42].name == "firefox"
|
||||
assert ram[42].peak_kb == 8192
|
||||
|
||||
|
||||
def test_parse_prm_non_numeric_pid_is_ignored() -> None:
|
||||
"""A row whose PID will not parse is dropped rather than raising."""
|
||||
ram: dict[int, _PidRam] = {}
|
||||
|
||||
parsing._parse_prm(_prm("notapid", "(x)", "1"), ram)
|
||||
|
||||
assert ram == {}
|
||||
|
||||
|
||||
def test_parse_prm_truncated_row_is_ignored() -> None:
|
||||
"""A row that ends before the RSS column is dropped."""
|
||||
ram: dict[int, _PidRam] = {}
|
||||
|
||||
parsing._parse_prm(["PRM", "host", "1000", "d", "t", "600", "42", "(x)"], ram)
|
||||
|
||||
assert ram == {}
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# _window_from_epochs
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_window_from_epochs_empty_is_default() -> None:
|
||||
"""No samples yields the placeholder window."""
|
||||
window = parsing._window_from_epochs(set())
|
||||
|
||||
assert window.distinct_samples == 0
|
||||
assert window.start == "n/a"
|
||||
|
||||
|
||||
def test_window_from_epochs_single_sample_has_no_interval() -> None:
|
||||
"""One sample cannot imply an interval, so it stays zero."""
|
||||
window = parsing._window_from_epochs({1000})
|
||||
|
||||
assert window.distinct_samples == 1
|
||||
assert window.interval_s == 0
|
||||
assert window.seconds == 0
|
||||
|
||||
|
||||
def test_window_from_epochs_uses_median_interval() -> None:
|
||||
"""The interval is the median gap, so one long gap does not skew it."""
|
||||
window = parsing._window_from_epochs({0, 600, 1200, 6000})
|
||||
|
||||
assert window.distinct_samples == 4
|
||||
assert window.interval_s == 600
|
||||
assert window.seconds == 6000
|
||||
assert window.start_epoch == 0
|
||||
assert window.end_epoch == 6000
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# _fold_pid_aggregates
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_fold_pid_aggregates_sums_cpu_across_pids() -> None:
|
||||
"""Two PIDs of one program contribute their deltas to one entry."""
|
||||
cpu = {1: _PidCpu(), 2: _PidCpu()}
|
||||
cpu[1].observe("python", 100)
|
||||
cpu[1].observe("python", 400)
|
||||
cpu[2].observe("python", 50)
|
||||
|
||||
agg = parsing._fold_pid_aggregates(cpu, {})
|
||||
|
||||
# PID 1 seen twice contributes last-first = 300; PID 2 seen once gives 50.
|
||||
assert agg["python"].cpu_ticks == 350
|
||||
assert agg["python"].pid_count == 2
|
||||
|
||||
|
||||
def test_fold_pid_aggregates_takes_peak_and_counts_rss_samples() -> None:
|
||||
"""Peak RSS is the max across PIDs and each PID adds one RSS sample."""
|
||||
ram = {1: _PidRam(), 2: _PidRam()}
|
||||
ram[1].observe("firefox", 4096)
|
||||
ram[2].observe("firefox", 8192)
|
||||
|
||||
agg = parsing._fold_pid_aggregates({}, ram)
|
||||
|
||||
assert agg["firefox"].peak_rss_kb == 8192
|
||||
assert agg["firefox"].rss_samples == 2
|
||||
assert agg["firefox"].pid_count == 2
|
||||
|
||||
|
||||
def test_parse_prc_non_numeric_pid_is_ignored() -> None:
|
||||
"""A PRC row whose PID will not parse is dropped rather than raising."""
|
||||
cpu: dict[int, _PidCpu] = {}
|
||||
|
||||
parsing._parse_prc(
|
||||
["PRC", "host", "1000", "d", "t", "600", "notapid", "(x)", "S", "1", "2", "3"],
|
||||
cpu,
|
||||
)
|
||||
|
||||
assert cpu == {}
|
||||
@ -1,76 +0,0 @@
|
||||
"""Regression tests for pmon process-name normalization in usage_report."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
import _usage_report_pmon as pmon
|
||||
|
||||
if TYPE_CHECKING:
|
||||
import pytest
|
||||
|
||||
|
||||
def test_normalize_pmon_command_prefers_first_executable_token() -> None:
|
||||
"""The parser should keep executable-like token, not trailing args."""
|
||||
tokens = ["code-insiders", "--type=", "gpu-process", "Not"]
|
||||
|
||||
assert pmon._normalize_pmon_command(tokens) == "code-insiders"
|
||||
|
||||
|
||||
def test_normalize_pmon_command_skips_leading_option_tokens() -> None:
|
||||
"""If the first token is an option, use the next non-option token."""
|
||||
tokens = ["--type=", "code-insiders", "--flag"]
|
||||
|
||||
assert pmon._normalize_pmon_command(tokens) == "code-insiders"
|
||||
|
||||
|
||||
def test_ingest_pmon_row_uses_command_field_start_not_last_token() -> None:
|
||||
"""Rows with command args should aggregate under process name, not args."""
|
||||
row = [
|
||||
"20260507",
|
||||
"12:00:00",
|
||||
"0",
|
||||
"123",
|
||||
"C",
|
||||
"10",
|
||||
"5",
|
||||
"0",
|
||||
"0",
|
||||
"0",
|
||||
"0",
|
||||
"code-insiders",
|
||||
"--type=",
|
||||
"gpu-process",
|
||||
]
|
||||
agg: dict[str, object] = {}
|
||||
|
||||
consumed = pmon._ingest_pmon_row(row, agg)
|
||||
|
||||
assert consumed == 1
|
||||
assert "code-insiders" in agg
|
||||
|
||||
|
||||
def test_ingest_pmon_row_falls_back_to_proc_comm_on_unknown(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""When command field is empty, parser can recover with /proc/<pid>/comm."""
|
||||
row = [
|
||||
"20260507",
|
||||
"12:00:00",
|
||||
"0",
|
||||
"999",
|
||||
"C",
|
||||
"30",
|
||||
"10",
|
||||
"0",
|
||||
"0",
|
||||
"0",
|
||||
"0",
|
||||
]
|
||||
agg: dict[str, object] = {}
|
||||
|
||||
monkeypatch.setattr(pmon, "_pid_comm_name", lambda _pid: "python")
|
||||
consumed = pmon._ingest_pmon_row(row, agg)
|
||||
|
||||
assert consumed == 1
|
||||
assert "python" in agg
|
||||
@ -1,146 +0,0 @@
|
||||
"""Tests for usage_report report assembly: the methodology section and the
|
||||
full Markdown report that stitches every section together.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
import _usage_report_render as render
|
||||
from _usage_report_run import _Aggregates
|
||||
from _usage_report_types import GpuAgg, ProcAgg, _Window
|
||||
|
||||
if TYPE_CHECKING:
|
||||
import pytest
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# _methodology_section
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_methodology_section_reports_observed_interval() -> None:
|
||||
"""A known sample interval is stated as an observed value."""
|
||||
window = _Window(interval_s=10, seconds=3600)
|
||||
|
||||
lines = render._methodology_section("atop.log", "pmon.log", window)
|
||||
|
||||
assert "- **atop sample interval (observed)**: 10s" in lines
|
||||
assert "- **atop log(s)**: atop.log" in lines
|
||||
assert "- **pmon log(s)**: pmon.log" in lines
|
||||
|
||||
|
||||
def test_methodology_section_flags_unknown_interval() -> None:
|
||||
"""A zero interval means only one sample, and says so instead of '0s'."""
|
||||
lines = render._methodology_section("a", "p", _Window(interval_s=0))
|
||||
|
||||
assert any("interval unknown" in line for line in lines)
|
||||
assert not any("(observed)" in line for line in lines)
|
||||
|
||||
|
||||
def test_methodology_section_states_coverage_window_in_human_units() -> None:
|
||||
"""The coverage window is formatted with _fmt_h, not raw seconds."""
|
||||
lines = render._methodology_section("a", "p", _Window(interval_s=10, seconds=5000))
|
||||
|
||||
assert any("1h 23m" in line for line in lines)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# _render_report
|
||||
# --------------------------------------------------------------------------- #
|
||||
def _stub_sections(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""Replace the host section so reports do not depend on the real machine."""
|
||||
monkeypatch.setattr(render, "_host_profile", lambda: {"hostname": "box"})
|
||||
monkeypatch.setattr(render.os, "cpu_count", lambda: 4)
|
||||
|
||||
|
||||
def _aggregates(
|
||||
*,
|
||||
cpu: dict[str, ProcAgg] | None = None,
|
||||
gpu: dict[str, GpuAgg] | None = None,
|
||||
window: _Window | None = None,
|
||||
gpu_samples: int = 0,
|
||||
) -> _Aggregates:
|
||||
"""Build an _Aggregates with only the fields a given test cares about."""
|
||||
return _Aggregates(
|
||||
cpu={} if cpu is None else cpu,
|
||||
gpu={} if gpu is None else gpu,
|
||||
window=_Window() if window is None else window,
|
||||
gpu_samples=gpu_samples,
|
||||
days_with_data=1,
|
||||
)
|
||||
|
||||
|
||||
def test_render_report_includes_every_section(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""The assembled report carries all five headings and the LLM prompt."""
|
||||
_stub_sections(monkeypatch)
|
||||
aggs = _aggregates(
|
||||
cpu={"firefox": ProcAgg("firefox", cpu_ticks=1000, peak_rss_kb=4096)},
|
||||
gpu={"game": GpuAgg("game", sm_pct_sum=50.0, samples=5)},
|
||||
window=_Window(
|
||||
start="A",
|
||||
end="B",
|
||||
distinct_samples=7,
|
||||
interval_s=10,
|
||||
seconds=60,
|
||||
),
|
||||
gpu_samples=5,
|
||||
)
|
||||
|
||||
out = render._render_report(
|
||||
aggs,
|
||||
top=5,
|
||||
atop_desc="atop.log",
|
||||
pmon_desc="pmon.log",
|
||||
period_line="- **Period**: today",
|
||||
)
|
||||
|
||||
assert out.startswith("# System resource usage report\n")
|
||||
for heading in (
|
||||
"## Host",
|
||||
"## Methodology",
|
||||
"## Top CPU consumers",
|
||||
"## Top RAM consumers",
|
||||
"## Top GPU consumers",
|
||||
"## Suggested LLM prompt",
|
||||
):
|
||||
assert heading in out
|
||||
assert "- **Period**: today" in out
|
||||
assert "- **atop window**: A → B" in out
|
||||
assert "7 distinct" in out
|
||||
assert "firefox" in out
|
||||
assert "game" in out
|
||||
assert out.endswith("\n")
|
||||
|
||||
|
||||
def test_render_report_notes_absent_gpu_data(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""With no GPU aggregates the GPU section is a placeholder, not a table."""
|
||||
_stub_sections(monkeypatch)
|
||||
|
||||
out = render._render_report(
|
||||
_aggregates(window=_Window(interval_s=10, seconds=60)),
|
||||
top=5,
|
||||
atop_desc="a",
|
||||
pmon_desc="p",
|
||||
period_line="- **Period**: x",
|
||||
)
|
||||
|
||||
assert "_No GPU pmon data found._" in out
|
||||
# The phrase "GPU SM-seconds" also appears in the methodology prose, so
|
||||
# assert on the table header cell instead to prove no table was emitted.
|
||||
assert "| GPU SM-seconds |" not in out
|
||||
|
||||
|
||||
def test_render_report_marks_single_sample_interval(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""A zero interval renders as 'n/a (single sample)' in the header block."""
|
||||
_stub_sections(monkeypatch)
|
||||
|
||||
out = render._render_report(
|
||||
_aggregates(window=_Window(interval_s=0)),
|
||||
top=5,
|
||||
atop_desc="a",
|
||||
pmon_desc="p",
|
||||
period_line="- **Period**: x",
|
||||
)
|
||||
|
||||
assert "sample interval ≈ n/a (single sample)" in out
|
||||
@ -1,155 +0,0 @@
|
||||
"""Tests for usage_report's since-last-report run: the atop guard, the normal
|
||||
report path, the no-data path, and when the saved timestamp advances.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime as _dt
|
||||
|
||||
import _usage_report_run as run
|
||||
from _usage_report_types import _Window
|
||||
import pytest
|
||||
import usage_report
|
||||
|
||||
|
||||
def _args(**overrides: object) -> object:
|
||||
"""Build a parsed-args namespace with the parser's own defaults."""
|
||||
args = usage_report._build_parser().parse_args([])
|
||||
for key, value in overrides.items():
|
||||
setattr(args, key, value)
|
||||
return args
|
||||
|
||||
|
||||
def _aggregates(*, days_with_data: int = 1, seconds: int = 600) -> object:
|
||||
"""Build an _Aggregates with a window that looks like real coverage."""
|
||||
return run._Aggregates(
|
||||
cpu={},
|
||||
gpu={},
|
||||
window=_Window(distinct_samples=2, interval_s=600, seconds=seconds),
|
||||
gpu_samples=0,
|
||||
days_with_data=days_with_data,
|
||||
)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# _run_since
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_run_since_exits_without_atop(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""The multi-day path also refuses to run without atop installed."""
|
||||
monkeypatch.setattr(run.shutil, "which", lambda _n: None)
|
||||
|
||||
with pytest.raises(SystemExit):
|
||||
run._run_since(_args(), _dt.datetime.now().astimezone())
|
||||
|
||||
|
||||
def test_run_since_reports_and_advances_state(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""A normal run emits a report and moves the saved timestamp forward."""
|
||||
now = _dt.datetime.now().astimezone()
|
||||
monkeypatch.setattr(run.shutil, "which", lambda _n: "/usr/bin/atop")
|
||||
monkeypatch.setattr(run, "_resolve_start", lambda *_a: now)
|
||||
monkeypatch.setattr(run, "_plan_segments", lambda *_a: [])
|
||||
monkeypatch.setattr(run, "_aggregate_segments", lambda *_a: _aggregates())
|
||||
monkeypatch.setattr(run, "_log_descriptions", lambda _s: ("a", "p"))
|
||||
monkeypatch.setattr(run, "_render_report", lambda *_a, **_k: "report")
|
||||
emitted: list[str] = []
|
||||
monkeypatch.setattr(run, "_emit", lambda _a, r: emitted.append(r))
|
||||
advanced: list[object] = []
|
||||
monkeypatch.setattr(run, "_write_last_generated", advanced.append)
|
||||
|
||||
rc = run._run_since(_args(quiet=True), now)
|
||||
|
||||
assert rc == 0
|
||||
assert emitted == ["report"]
|
||||
assert advanced == [now]
|
||||
|
||||
|
||||
def test_run_since_with_no_data_still_advances_state(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
capsys: pytest.CaptureFixture[str],
|
||||
) -> None:
|
||||
"""Gap days produce a stderr note, no report, but the state still moves."""
|
||||
now = _dt.datetime.now().astimezone()
|
||||
monkeypatch.setattr(run.shutil, "which", lambda _n: "/usr/bin/atop")
|
||||
monkeypatch.setattr(run, "_resolve_start", lambda *_a: now)
|
||||
monkeypatch.setattr(run, "_plan_segments", lambda *_a: [])
|
||||
monkeypatch.setattr(
|
||||
run,
|
||||
"_aggregate_segments",
|
||||
lambda *_a: _aggregates(days_with_data=0),
|
||||
)
|
||||
emitted: list[str] = []
|
||||
monkeypatch.setattr(run, "_emit", lambda _a, r: emitted.append(r))
|
||||
advanced: list[object] = []
|
||||
monkeypatch.setattr(run, "_write_last_generated", advanced.append)
|
||||
|
||||
rc = run._run_since(_args(quiet=True), now)
|
||||
|
||||
assert rc == 0
|
||||
assert emitted == []
|
||||
assert advanced == [now]
|
||||
assert "nothing to report" in capsys.readouterr().err
|
||||
|
||||
|
||||
def test_run_since_with_no_data_leaves_state_when_suppressed(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""--no-update-state keeps the saved timestamp even on the no-data path."""
|
||||
now = _dt.datetime.now().astimezone()
|
||||
monkeypatch.setattr(run.shutil, "which", lambda _n: "/usr/bin/atop")
|
||||
monkeypatch.setattr(run, "_resolve_start", lambda *_a: now)
|
||||
monkeypatch.setattr(run, "_plan_segments", lambda *_a: [])
|
||||
monkeypatch.setattr(
|
||||
run,
|
||||
"_aggregate_segments",
|
||||
lambda *_a: _aggregates(days_with_data=0),
|
||||
)
|
||||
advanced: list[object] = []
|
||||
monkeypatch.setattr(run, "_write_last_generated", advanced.append)
|
||||
|
||||
run._run_since(_args(quiet=True, no_update_state=True), now)
|
||||
|
||||
assert advanced == []
|
||||
|
||||
|
||||
def test_run_since_defaults_an_empty_window_to_a_day(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""A window with no measured seconds is treated as a full day."""
|
||||
now = _dt.datetime.now().astimezone()
|
||||
aggs = _aggregates(seconds=0)
|
||||
monkeypatch.setattr(run.shutil, "which", lambda _n: "/usr/bin/atop")
|
||||
monkeypatch.setattr(run, "_resolve_start", lambda *_a: now)
|
||||
monkeypatch.setattr(run, "_plan_segments", lambda *_a: [])
|
||||
monkeypatch.setattr(run, "_aggregate_segments", lambda *_a: aggs)
|
||||
monkeypatch.setattr(run, "_log_descriptions", lambda _s: ("a", "p"))
|
||||
monkeypatch.setattr(run, "_render_report", lambda *_a, **_k: "report")
|
||||
monkeypatch.setattr(run, "_emit", lambda *_a: None)
|
||||
monkeypatch.setattr(run, "_write_last_generated", lambda *_a: None)
|
||||
|
||||
run._run_since(_args(quiet=True), now)
|
||||
|
||||
assert aggs.window.seconds == run._SEC_PER_DAY
|
||||
|
||||
|
||||
def test_run_since_leaves_state_when_suppressed_after_a_report(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""--no-update-state keeps the timestamp even when a report was emitted."""
|
||||
now = _dt.datetime.now().astimezone()
|
||||
monkeypatch.setattr(run.shutil, "which", lambda _n: "/usr/bin/atop")
|
||||
monkeypatch.setattr(run, "_resolve_start", lambda *_a: now)
|
||||
monkeypatch.setattr(run, "_plan_segments", lambda *_a: [])
|
||||
monkeypatch.setattr(run, "_aggregate_segments", lambda *_a: _aggregates())
|
||||
monkeypatch.setattr(run, "_log_descriptions", lambda _s: ("a", "p"))
|
||||
monkeypatch.setattr(run, "_render_report", lambda *_a, **_k: "report")
|
||||
emitted: list[str] = []
|
||||
monkeypatch.setattr(run, "_emit", lambda _a, r: emitted.append(r))
|
||||
advanced: list[object] = []
|
||||
monkeypatch.setattr(run, "_write_last_generated", advanced.append)
|
||||
|
||||
run._run_since(_args(quiet=True, no_update_state=True), now)
|
||||
|
||||
assert emitted == ["report"]
|
||||
assert advanced == []
|
||||
@ -1,84 +0,0 @@
|
||||
"""Tests for usage_report's per-day segment aggregation and the
|
||||
log-description strings.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
import _usage_report_run as run
|
||||
from _usage_report_types import _Progress, _Window
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
def _progress() -> _Progress:
|
||||
"""A progress reporter that never draws, so stderr stays clean."""
|
||||
return _Progress(enabled=False, total_stages=1)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# _aggregate_segments / _log_descriptions
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_aggregate_segments_skips_missing_atop_logs(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
"""A gap day contributes no atop data but its pmon log is still read."""
|
||||
present = tmp_path / "atop-present"
|
||||
present.write_text("x", encoding="utf-8")
|
||||
segments = [
|
||||
run._Segment(present, tmp_path / "pmon-a"),
|
||||
run._Segment(tmp_path / "atop-absent", tmp_path / "pmon-b"),
|
||||
]
|
||||
monkeypatch.setattr(
|
||||
run,
|
||||
"aggregate_atop",
|
||||
lambda *_a: ({}, _Window(distinct_samples=2, seconds=600)),
|
||||
)
|
||||
pmon_calls: list[Path] = []
|
||||
|
||||
def fake_pmon(log: Path, *_a: object) -> tuple[dict[str, object], int]:
|
||||
pmon_calls.append(log)
|
||||
return {}, 5
|
||||
|
||||
monkeypatch.setattr(run, "aggregate_pmon", fake_pmon)
|
||||
|
||||
aggs = run._aggregate_segments(segments, _progress())
|
||||
|
||||
assert aggs.days_with_data == 1
|
||||
assert aggs.gpu_samples == 10
|
||||
assert len(pmon_calls) == 2
|
||||
|
||||
|
||||
def test_log_descriptions_names_only_existing_logs(tmp_path: Path) -> None:
|
||||
"""Descriptions cover the logs that are actually on disk."""
|
||||
atop = tmp_path / "atop"
|
||||
atop.write_text("x", encoding="utf-8")
|
||||
segments = [run._Segment(atop, tmp_path / "pmon-absent")]
|
||||
|
||||
atop_desc, pmon_desc = run._log_descriptions(segments)
|
||||
|
||||
assert str(atop) in atop_desc
|
||||
assert "none" in pmon_desc.lower() or pmon_desc
|
||||
|
||||
|
||||
def test_aggregate_segments_ignores_a_log_with_no_samples(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
"""A present atop log that yields no samples does not count as a data day."""
|
||||
present = tmp_path / "atop-empty"
|
||||
present.write_text("x", encoding="utf-8")
|
||||
monkeypatch.setattr(run, "aggregate_atop", lambda *_a: ({}, _Window()))
|
||||
monkeypatch.setattr(run, "aggregate_pmon", lambda *_a: ({}, 0))
|
||||
|
||||
aggs = run._aggregate_segments(
|
||||
[run._Segment(present, tmp_path / "pmon")],
|
||||
_progress(),
|
||||
)
|
||||
|
||||
assert aggs.days_with_data == 0
|
||||
@ -1,189 +0,0 @@
|
||||
"""Tests for spanning several days in usage_report: pmon timestamp filtering,
|
||||
atop command bounding, the persisted last-report state, and day-segment
|
||||
planning.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime as _dt
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
import _usage_report_atop as atop_io
|
||||
import _usage_report_logs as logs
|
||||
import _usage_report_pmon as pmon
|
||||
from _usage_report_types import _Progress
|
||||
|
||||
if TYPE_CHECKING:
|
||||
import pytest
|
||||
|
||||
|
||||
# Aware timezone matching how the parser localizes naive timestamps, so
|
||||
# epochs computed here line up with `_pmon_row_epoch`'s `.astimezone()`.
|
||||
_LOCAL_TZ = _dt.datetime.now().astimezone().tzinfo
|
||||
|
||||
|
||||
def _at(
|
||||
year: int, month: int, day: int, hour: int = 0, minute: int = 0
|
||||
) -> _dt.datetime:
|
||||
"""Build an aware local datetime for tests."""
|
||||
return _dt.datetime(year, month, day, hour, minute, tzinfo=_LOCAL_TZ)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# pmon timestamp helpers (parsing)
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_pmon_row_epoch_parses_valid_row() -> None:
|
||||
"""A well-formed pmon row yields the matching local epoch."""
|
||||
row = ["20260604", "10:30:00", "0", "100", "G", "5", "1"]
|
||||
|
||||
assert pmon._pmon_row_epoch(row) == _at(2026, 6, 4, 10, 30).timestamp()
|
||||
|
||||
|
||||
def test_pmon_row_epoch_returns_none_on_bad_input() -> None:
|
||||
"""Malformed or short rows return None rather than raising."""
|
||||
assert pmon._pmon_row_epoch([]) is None
|
||||
assert pmon._pmon_row_epoch(["nope", "alsonope"]) is None
|
||||
|
||||
|
||||
def _write_pmon(path: Path) -> None:
|
||||
"""Write a tiny pmon log with two rows ten minutes apart."""
|
||||
path.write_text(
|
||||
"#Date Time gpu pid type sm mem enc dec jpg ofa command\n"
|
||||
" 20260604 10:00:00 0 100 G 5 1 - - - - Xorg\n"
|
||||
" 20260604 11:00:00 0 101 G 7 2 - - - - thorium\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
|
||||
def test_aggregate_pmon_without_bound_keeps_all_rows(tmp_path: Path) -> None:
|
||||
"""No begin_epoch means every data row counts."""
|
||||
log = tmp_path / "pmon.log"
|
||||
_write_pmon(log)
|
||||
|
||||
_, samples = pmon.aggregate_pmon(log, _Progress(enabled=False, total_stages=1))
|
||||
|
||||
assert samples == 2
|
||||
|
||||
|
||||
def test_aggregate_pmon_filters_rows_before_begin(tmp_path: Path) -> None:
|
||||
"""Rows timestamped before begin_epoch are skipped."""
|
||||
log = tmp_path / "pmon.log"
|
||||
_write_pmon(log)
|
||||
cutoff = _at(2026, 6, 4, 10, 30).timestamp()
|
||||
|
||||
agg, samples = pmon.aggregate_pmon(
|
||||
log,
|
||||
_Progress(enabled=False, total_stages=1),
|
||||
begin_epoch=cutoff,
|
||||
)
|
||||
|
||||
assert samples == 1
|
||||
assert "thorium" in agg
|
||||
assert "Xorg" not in agg
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# atop command bounding (parsing)
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_atop_read_cmd_unbounded() -> None:
|
||||
"""Without bounds the command is a plain replay."""
|
||||
cmd = atop_io._atop_read_cmd(
|
||||
Path("/var/log/atop/atop_20260604"), "PRC,PRM", None, None
|
||||
)
|
||||
|
||||
assert cmd == ["atop", "-r", "/var/log/atop/atop_20260604", "-P", "PRC,PRM"]
|
||||
|
||||
|
||||
def test_atop_read_cmd_with_begin_and_end() -> None:
|
||||
"""Begin/end inject -b/-e before the -P selector."""
|
||||
cmd = atop_io._atop_read_cmd(Path("/x"), "PRC", "202606041400", "202606042000")
|
||||
|
||||
assert cmd == [
|
||||
"atop",
|
||||
"-r",
|
||||
"/x",
|
||||
"-b",
|
||||
"202606041400",
|
||||
"-e",
|
||||
"202606042000",
|
||||
"-P",
|
||||
"PRC",
|
||||
]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Persisted last-report state (usage_report)
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_state_round_trip(
|
||||
tmp_path: Path,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""A written timestamp reads back as an equal aware datetime."""
|
||||
state = tmp_path / "state" / "last_report.json"
|
||||
monkeypatch.setattr(logs, "_STATE_DIR", state.parent)
|
||||
monkeypatch.setattr(logs, "_STATE_FILE", state)
|
||||
when = _at(2026, 6, 2, 9, 0)
|
||||
|
||||
logs._write_last_generated(when)
|
||||
|
||||
assert logs._read_last_generated() == when
|
||||
|
||||
|
||||
def test_state_missing_file_returns_none(
|
||||
tmp_path: Path,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""No state file yet means "unknown", so the caller falls back to today."""
|
||||
monkeypatch.setattr(logs, "_STATE_FILE", tmp_path / "absent.json")
|
||||
|
||||
assert logs._read_last_generated() is None
|
||||
|
||||
|
||||
def test_state_corrupt_file_returns_none(
|
||||
tmp_path: Path,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Corrupt or partial JSON is treated as unknown, not a crash."""
|
||||
bad = tmp_path / "bad.json"
|
||||
bad.write_text("{ not json", encoding="utf-8")
|
||||
monkeypatch.setattr(logs, "_STATE_FILE", bad)
|
||||
assert logs._read_last_generated() is None
|
||||
|
||||
bad.write_text("{}", encoding="utf-8") # valid JSON, missing key
|
||||
assert logs._read_last_generated() is None
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Day-segment planning (usage_report)
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_has_time_of_day() -> None:
|
||||
"""Midnight needs no begin bound; any later time does."""
|
||||
assert logs._has_time_of_day(_at(2026, 6, 4, 14, 30)) is True
|
||||
assert logs._has_time_of_day(_at(2026, 6, 4, 0, 0)) is False
|
||||
|
||||
|
||||
def test_plan_segments_single_day_midnight_unbounded() -> None:
|
||||
"""A start at local midnight covers the whole first day (no -b bound)."""
|
||||
segments = logs._plan_segments(_at(2026, 6, 4), _at(2026, 6, 4, 12))
|
||||
|
||||
assert len(segments) == 1
|
||||
assert segments[0].atop_begin is None
|
||||
assert segments[0].pmon_begin_epoch is None
|
||||
|
||||
|
||||
def test_plan_segments_bounds_only_first_day() -> None:
|
||||
"""A mid-day start bounds the first day only; later days are full."""
|
||||
start = _at(2026, 6, 2, 14, 0)
|
||||
segments = logs._plan_segments(start, _at(2026, 6, 4, 10, 0))
|
||||
|
||||
assert len(segments) == 3
|
||||
assert segments[0].atop_begin == "20260602140000"
|
||||
assert segments[0].pmon_begin_epoch == start.timestamp()
|
||||
assert all(seg.atop_begin is None for seg in segments[1:])
|
||||
assert segments[-1].atop_log.name == "atop_20260604"
|
||||
|
||||
|
||||
def test_plan_segments_start_after_end_is_empty() -> None:
|
||||
"""A future state file (start past end) yields no segments."""
|
||||
assert logs._plan_segments(_at(2026, 6, 5), _at(2026, 6, 4)) == []
|
||||
@ -1,216 +0,0 @@
|
||||
"""Tests for usage_report's CPU and RAM Markdown tables, including the
|
||||
peak-RSS bucket dedupe that collapses thread rows onto their parent.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
import _usage_report_tables as tables
|
||||
from _usage_report_types import _HZ, ProcAgg
|
||||
|
||||
if TYPE_CHECKING:
|
||||
import pytest
|
||||
|
||||
|
||||
def _proc(
|
||||
name: str,
|
||||
*,
|
||||
cpu_ticks: int = 0,
|
||||
peak_rss_kb: int = 0,
|
||||
pid_set: set[int] | None = None,
|
||||
) -> ProcAgg:
|
||||
"""Build a ProcAgg with only the fields these table tests care about."""
|
||||
return ProcAgg(
|
||||
name,
|
||||
cpu_ticks=cpu_ticks,
|
||||
peak_rss_kb=peak_rss_kb,
|
||||
pid_set=set() if pid_set is None else pid_set,
|
||||
)
|
||||
|
||||
|
||||
def _cells(row: str) -> list[str]:
|
||||
"""Split a Markdown table row into stripped cell values."""
|
||||
return [c.strip() for c in row.strip().strip("|").split("|")]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# _cpu_table
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_cpu_table_sorts_by_ticks_and_honours_top(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Rows are ordered by CPU ticks descending and truncated to *top*."""
|
||||
monkeypatch.setattr(tables.os, "cpu_count", lambda: 4)
|
||||
aggs = [
|
||||
_proc("low", cpu_ticks=100),
|
||||
_proc("high", cpu_ticks=900),
|
||||
_proc("mid", cpu_ticks=500),
|
||||
]
|
||||
|
||||
rows = tables._cpu_table(aggs, window_s=60, top=2)
|
||||
|
||||
assert len(rows) == 4 # header + separator + 2 data rows
|
||||
assert _cells(rows[2])[1] == "high"
|
||||
assert _cells(rows[3])[1] == "mid"
|
||||
|
||||
|
||||
def test_cpu_table_divides_box_percentage_by_cpu_count(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""'Avg CPU% (of box)' is the single-core percentage divided by ncpu."""
|
||||
monkeypatch.setattr(tables.os, "cpu_count", lambda: 4)
|
||||
# ProcAgg.cpu_seconds divides ticks by the real clock tick rate, so derive
|
||||
# the tick count from it: one full core busy for the whole 60 s window.
|
||||
aggs = [_proc("busy", cpu_ticks=60 * _HZ)]
|
||||
|
||||
rows = tables._cpu_table(aggs, window_s=60, top=1)
|
||||
|
||||
cells = _cells(rows[2])
|
||||
assert cells[3] == "100.0%"
|
||||
assert cells[4] == "25.0%"
|
||||
|
||||
|
||||
def test_cpu_table_zero_window_yields_zero_percent(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""A zero-length window cannot be divided by, so percentages are 0."""
|
||||
monkeypatch.setattr(tables.os, "cpu_count", lambda: 2)
|
||||
rows = tables._cpu_table([_proc("x", cpu_ticks=500)], window_s=0, top=1)
|
||||
|
||||
cells = _cells(rows[2])
|
||||
assert cells[3] == "0.0%"
|
||||
assert cells[4] == "0.0%"
|
||||
|
||||
|
||||
def test_cpu_table_falls_back_to_one_cpu(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""When os.cpu_count() returns None the box percentage assumes one core."""
|
||||
monkeypatch.setattr(tables.os, "cpu_count", lambda: None)
|
||||
rows = tables._cpu_table([_proc("x", cpu_ticks=100)], window_s=60, top=1)
|
||||
|
||||
cells = _cells(rows[2])
|
||||
assert cells[3] == cells[4]
|
||||
|
||||
|
||||
def test_cpu_table_escapes_program_names(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""A pipe in a program name cannot break out of its table cell."""
|
||||
monkeypatch.setattr(tables.os, "cpu_count", lambda: 1)
|
||||
rows = tables._cpu_table([_proc("a|b", cpu_ticks=1)], window_s=60, top=1)
|
||||
|
||||
assert r"a\|b" in rows[2]
|
||||
|
||||
|
||||
def test_cpu_table_empty_input_is_header_only(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""No aggregates means header and separator with no data rows."""
|
||||
monkeypatch.setattr(tables.os, "cpu_count", lambda: 1)
|
||||
assert len(tables._cpu_table([], window_s=60, top=5)) == 2
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# _dedupe_ram
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_dedupe_ram_groups_by_bucket_and_keeps_top_cpu() -> None:
|
||||
"""Same-RSS rows collapse to the highest-CPU representative."""
|
||||
same_rss = 4096
|
||||
rows = tables._dedupe_ram(
|
||||
[
|
||||
_proc("worker", peak_rss_kb=same_rss, cpu_ticks=10),
|
||||
_proc("parent", peak_rss_kb=same_rss, cpu_ticks=99),
|
||||
_proc("other", peak_rss_kb=same_rss, cpu_ticks=5),
|
||||
],
|
||||
)
|
||||
|
||||
assert len(rows) == 1
|
||||
rep, siblings = rows[0]
|
||||
assert rep.name == "parent"
|
||||
assert sorted(siblings) == ["other", "worker"]
|
||||
|
||||
|
||||
def test_dedupe_ram_skips_non_positive_rss() -> None:
|
||||
"""Rows with no measured RSS are dropped entirely."""
|
||||
assert tables._dedupe_ram([_proc("ghost", peak_rss_kb=0)]) == []
|
||||
|
||||
|
||||
def test_dedupe_ram_orders_buckets_by_rss_descending() -> None:
|
||||
"""Distinct RSS buckets come back largest-first."""
|
||||
rows = tables._dedupe_ram(
|
||||
[
|
||||
_proc("small", peak_rss_kb=1024),
|
||||
_proc("large", peak_rss_kb=10 * 1024),
|
||||
],
|
||||
)
|
||||
|
||||
assert [rep.name for rep, _ in rows] == ["large", "small"]
|
||||
|
||||
|
||||
def test_dedupe_ram_breaks_cpu_ties_on_pid_count() -> None:
|
||||
"""Equal CPU ticks fall back to the higher PID count for the rep."""
|
||||
rows = tables._dedupe_ram(
|
||||
[
|
||||
_proc("one_pid", peak_rss_kb=2048, cpu_ticks=7, pid_set={1}),
|
||||
_proc("many_pids", peak_rss_kb=2048, cpu_ticks=7, pid_set={1, 2, 3}),
|
||||
],
|
||||
)
|
||||
|
||||
assert rows[0][0].name == "many_pids"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# _ram_table
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_ram_table_shows_dash_when_no_siblings() -> None:
|
||||
"""A lone row in its bucket renders an em dash in the sibling column."""
|
||||
rows = tables._ram_table([_proc("solo", peak_rss_kb=2048)], top=5)
|
||||
|
||||
assert _cells(rows[2])[-1] == "—"
|
||||
|
||||
|
||||
def test_ram_table_lists_siblings_without_overflow_note() -> None:
|
||||
"""Up to the display cap, every sibling name is listed and no '+N more'."""
|
||||
aggs = [_proc("rep", peak_rss_kb=4096, cpu_ticks=100)]
|
||||
aggs += [_proc(f"s{i}", peak_rss_kb=4096, cpu_ticks=1) for i in range(3)]
|
||||
|
||||
sib = _cells(tables._ram_table(aggs, top=5)[2])[-1]
|
||||
|
||||
assert "more" not in sib
|
||||
assert sorted(sib.split(", ")) == ["s0", "s1", "s2"]
|
||||
|
||||
|
||||
def test_ram_table_truncates_sibling_list_with_count() -> None:
|
||||
"""Beyond the cap, extra siblings collapse into a '(+N more)' suffix."""
|
||||
over = tables._MAX_SIBLINGS_SHOWN + 2
|
||||
aggs = [_proc("rep", peak_rss_kb=4096, cpu_ticks=100)]
|
||||
aggs += [_proc(f"s{i}", peak_rss_kb=4096, cpu_ticks=1) for i in range(over)]
|
||||
|
||||
sib = _cells(tables._ram_table(aggs, top=5)[2])[-1]
|
||||
|
||||
assert sib.endswith(f"(+{over - tables._MAX_SIBLINGS_SHOWN} more)")
|
||||
# Exactly _MAX_SIBLINGS_SHOWN names are listed, joined by ", ".
|
||||
assert sib.count(",") == tables._MAX_SIBLINGS_SHOWN - 1
|
||||
|
||||
|
||||
def test_ram_table_escapes_sibling_and_rep_names() -> None:
|
||||
"""Pipes are escaped in both the representative and sibling columns."""
|
||||
aggs = [
|
||||
_proc("re|p", peak_rss_kb=4096, cpu_ticks=100),
|
||||
_proc("si|b", peak_rss_kb=4096, cpu_ticks=1),
|
||||
]
|
||||
|
||||
row = tables._ram_table(aggs, top=5)[2]
|
||||
|
||||
assert r"re\|p" in row
|
||||
assert r"si\|b" in row
|
||||
|
||||
|
||||
def test_ram_table_honours_top_limit() -> None:
|
||||
"""Only *top* buckets are rendered."""
|
||||
aggs = [_proc(f"p{i}", peak_rss_kb=(i + 1) * 4096) for i in range(5)]
|
||||
|
||||
assert len(tables._ram_table(aggs, top=2)) == 4
|
||||
|
||||
|
||||
def test_ram_table_empty_input_is_header_only() -> None:
|
||||
"""No aggregates means header and separator only."""
|
||||
assert len(tables._ram_table([], top=5)) == 2
|
||||
@ -170,7 +170,7 @@ disallow_untyped_decorators = false
|
||||
# bare name (the same dirs linux_configuration/tests/conftest.py adds to
|
||||
# sys.path at runtime) resolve under static analysis instead of raising E0401.
|
||||
# Paths are relative to the repo root, which is pre-commit's working directory.
|
||||
init-hook = "import sys; sys.path[:0] = ['meta/scripts', 'python_pkg', 'phone_focus_mode', 'phone_focus_mode/lib', 'linux_configuration/scripts/single_use/utils', 'linux_configuration/scripts/periodic_background/system-maintenance/bin', 'linux_configuration/scripts/periodic_background/digital_wellbeing', 'linux_configuration/scripts/single_use/misc/testsAndMisc-bash/tools']"
|
||||
init-hook = "import sys; sys.path[:0] = ['meta/scripts', 'python_pkg', 'phone_focus_mode', 'phone_focus_mode/lib', 'linux_configuration/scripts/single_use/utils', 'linux_configuration/scripts/periodic_background/digital_wellbeing', 'linux_configuration/scripts/single_use/misc/testsAndMisc-bash/tools']"
|
||||
# Analyse import fallback blocks
|
||||
analyse-fallback-blocks = true
|
||||
# Pickle collected data for later comparisons
|
||||
|
||||
@ -20,11 +20,6 @@ linux_configuration/scripts/periodic_background/digital_wellbeing/pacman/lib/int
|
||||
linux_configuration/scripts/periodic_background/digital_wellbeing/pacman/lib/managed_copy.sh
|
||||
linux_configuration/scripts/periodic_background/digital_wellbeing/pacman/lib/protect_and_extras.sh
|
||||
linux_configuration/scripts/periodic_background/digital_wellbeing/virtualbox/lib/vbox_disk_inject.sh
|
||||
linux_configuration/scripts/periodic_background/system-maintenance/bin/lib/catchup_timer.sh
|
||||
linux_configuration/scripts/periodic_background/system-maintenance/bin/lib/distro_detect.sh
|
||||
linux_configuration/scripts/periodic_background/system-maintenance/bin/lib/nvidia_pmon.sh
|
||||
linux_configuration/scripts/periodic_background/system-maintenance/bin/lib/packages.sh
|
||||
linux_configuration/scripts/periodic_background/system-maintenance/bin/lib/system_services.sh
|
||||
linux_configuration/scripts/single_use/features/lib/aw_autostart.sh
|
||||
linux_configuration/scripts/single_use/features/lib/aw_install.sh
|
||||
linux_configuration/scripts/single_use/features/lib/aw_service.sh
|
||||
|
||||
Loading…
Reference in New Issue
Block a user