mirror of
https://github.com/kuhyx/testsAndMisc.git
synced 2026-11-10 20:47:03 +01:00
shell: split clean_audio, enforce_vbox_hosts and migrate_hosts_guard
clean_audio.sh 419 -> 230 + 204
enforce_vbox_hosts.sh 443 -> 220 + 236
migrate_hosts_guard.sh 444 -> 167 + 162 + 141
migrate_hosts_guard needed two libraries: with all ten work functions moved
the single lib was 295 lines, so migrate and rollback were separated.
The extractor's brace matcher required index > start and so silently skipped
one-line `warn() { ...; }` definitions. Fixed, and regression-checked against
the committed fresh-install/main.sh split byte-for-byte.
The harness earned its keep on a fourth file: adding `export` to libre_translate
to satisfy SC2034 changed two function bodies, and verify_shell_split.sh caught
what lint had accepted. Those exports were reverted, and libre_translate.sh is
parked - what remains after its clean seams both assigns and reads the same
globals, so getting it under the cap needs parse_args restructured rather than
relocated, which is a refactor rather than a verbatim move.
That cost is now written down: a lib must contain the readers of every global
it assigns. The precedent at install_pacman_wrapper.sh:29 is about
definitions-only libs, not about assignments as such, and the pre-commit hook
runs shellcheck with no -x, so a source= directive will not make it follow the
link. This is the shell analogue of the monkeypatch-target trap.
Verified: all 8, 13 and 19 functions verbatim; all four libraries source with
no side effects; shellcheck passes standalone on every file with no
suppression; bash -n passes on all seven. None of the three was executed -
they drive ffmpeg over real media, mutate VirtualBox guest disks, and rewrite
systemd units.
Over-cap files 29 -> 26.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011w2Pj4Ek3ZNMQVYcFHgXDY
This commit is contained in:
parent
efd7f022fd
commit
76e73830f6
@ -134,3 +134,33 @@ Run before designing any seam, per the section above. `deploy.sh` copies a
|
||||
`pacman_wrapper.sh`, `generate_study_materials.sh`,
|
||||
`check_and_enable_services.sh`, `setup_midnight_shutdown.sh`) — no deployed
|
||||
copy found, so an entry+`lib/` shape is safe.
|
||||
|
||||
### A lib must contain the readers of every global it assigns
|
||||
|
||||
`install_pacman_wrapper.sh:29` records the constraint: a **definitions-only**
|
||||
lib "assigns without referencing, which is SC2034, and the repo forbids
|
||||
suppressions." The rule that follows from it is not "assignments stay in the
|
||||
entry script" — it is that a file must not assign a variable it never reads.
|
||||
A lib holding both the writer and its readers is clean.
|
||||
|
||||
Getting this inverted is easy and costs several revert cycles.
|
||||
`libre_translate.sh` was split with `parse_args` (the writer) left in the entry
|
||||
and `health_check` / `sample_request` / `start_container_ephemeral` (the
|
||||
readers) moved to the lib — the worst of both, and SC2034 fired on nine
|
||||
globals. The fix is to colocate: move the writer **and** its consumers into the
|
||||
same file.
|
||||
|
||||
The pre-commit hook is `shellcheck` with **no `-x`**, so each file is analysed
|
||||
standalone and a `# shellcheck source=` directive will not make the checker
|
||||
follow the link. Verify by running `shellcheck <lib>` on its own before
|
||||
committing; the hook will not be any more forgiving.
|
||||
|
||||
This is the shell analogue of the `monkeypatch.setattr` target trap: the
|
||||
question is always which file's namespace owns the name.
|
||||
|
||||
**`libre_translate.sh` (488) is parked.** What remains after its clean seams
|
||||
are taken is `parse_args` (111 lines), `write_env_file`, `detect_container_user`
|
||||
and `main`, which between them assign and read the same configuration globals.
|
||||
Getting the entry under the cap needs `parse_args` restructured rather than
|
||||
relocated — a refactor, not a verbatim move, so it is out of scope for this
|
||||
method. Attempt it with a test harness, or accept an entry that stays over.
|
||||
|
||||
@ -0,0 +1,17 @@
|
||||
{
|
||||
"title": "Split three more over-cap shell scripts",
|
||||
"objective": "clean_audio.sh, enforce_vbox_hosts.sh and migrate_hosts_guard_to_guard_lib.sh are over the 250-line cap with no deployed copy, so the entry+lib shape is safe. Move each script's work functions into libraries beside it, verbatim, leaving the entry owning the CLI, the ordering and the globals it assigns.",
|
||||
"acceptance_criteria": [
|
||||
"All seven resulting files are under 250 lines; the over-cap total falls 29 -> 26",
|
||||
"Every top-level function is verbatim, proved by meta/scripts/verify_shell_split.sh",
|
||||
"Each library sources with no side effects and defines every function moved into it",
|
||||
"shellcheck passes on every file standalone, with no suppression added",
|
||||
"bash -n passes on all seven"
|
||||
],
|
||||
"out_of_scope": [
|
||||
"libre_translate.sh, whose remaining seam needs parse_args restructured rather than relocated - a refactor, not a verbatim move",
|
||||
"Running any of the three scripts: they drive ffmpeg over real media, mutate VirtualBox guest disks, and rewrite systemd units",
|
||||
"The phone_focus_mode scripts and the pacman-wrapper pair"
|
||||
],
|
||||
"verifier": "bash meta/scripts/verify_shell_split.sh for the verbatim claim; sourcing each library in a subshell; shellcheck standalone on each new file; bash -n on all seven; bash ~/utils/scripts/check_file_length.sh --all."
|
||||
}
|
||||
62
docs/superpowers/evidence/shell-splits-batch-2026-08-17.json
Normal file
62
docs/superpowers/evidence/shell-splits-batch-2026-08-17.json
Normal file
@ -0,0 +1,62 @@
|
||||
{
|
||||
"intent": "Split three more over-cap shell scripts - clean_audio.sh, enforce_vbox_hosts.sh and migrate_hosts_guard_to_guard_lib.sh - into entry scripts plus sourced libraries, and record the SC2034 writer/reader rule that decides where a seam can fall.",
|
||||
"scope": [
|
||||
"clean_audio.sh 419 -> 230, new lib/clean_audio_filters.sh 204",
|
||||
"enforce_vbox_hosts.sh 443 -> 220, new lib/vbox_disk_inject.sh 236",
|
||||
"migrate_hosts_guard_to_guard_lib.sh 444 -> 167, new lib/hosts_guard_migrate.sh 162 and lib/hosts_guard_rollback.sh 141",
|
||||
"extract_shell_functions.py: handle one-line `name() { ...; }` definitions",
|
||||
"docs/shell-split-verification.md: the writer/reader rule and the parked libre_translate.sh",
|
||||
"Non-goal: libre_translate.sh, parked with its reasoning recorded"
|
||||
],
|
||||
"changes": [
|
||||
"Each entry script keeps its CLI, its ordering and the globals it assigns; the libraries take the work functions.",
|
||||
"migrate_hosts_guard needed two libraries rather than one: with all ten work functions moved, the single library was 295 lines, so migrate and rollback were separated.",
|
||||
"The extractor's brace matcher required index > start, which silently skipped one-line functions like `warn() { ...; }`. Fixed and regression-checked against the already-verified fresh-install/main.sh split.",
|
||||
"Documented that a lib must contain the readers of every global it assigns - the SC2034 constraint from install_pacman_wrapper.sh:29, which is about definitions-only libs rather than about assignments per se. The pre-commit hook runs shellcheck without -x, so a source= directive does not make it follow the link."
|
||||
],
|
||||
"verification": [
|
||||
{
|
||||
"command": "bash meta/scripts/verify_shell_split.sh HEAD <entry> <libs> <entry>, for each of the three",
|
||||
"result": "pass",
|
||||
"evidence": "clean_audio: all 8 functions verbatim. enforce_vbox_hosts: all 13. migrate_hosts_guard: all 19. Re-run after the shellcheck fixes, and still identical."
|
||||
},
|
||||
{
|
||||
"command": "The same harness on an earlier libre_translate attempt",
|
||||
"result": "pass",
|
||||
"evidence": "DIFFERENCE FOUND, naming parse_args and write_env_file: adding `export` to satisfy SC2034 had changed two function bodies, so it was no longer a verbatim move. The harness caught a change that lint had accepted, and the exports were reverted."
|
||||
},
|
||||
{
|
||||
"command": "Sourcing each of the four new libraries in a subshell",
|
||||
"result": "pass",
|
||||
"evidence": "All four source cleanly with no side effects and define every function moved into them."
|
||||
},
|
||||
{
|
||||
"command": "shellcheck via pre-commit on all seven changed files",
|
||||
"result": "pass",
|
||||
"evidence": "Passed. Reaching that took four attempts on libre_translate: a comment beginning 'shellcheck can see...' was parsed as a malformed directive (SC1073), a bare `declare` did not satisfy SC2034, and `export` satisfied it only by changing behaviour. No suppression was added; the one added mid-attempt was removed."
|
||||
},
|
||||
{
|
||||
"command": "bash -n on all seven files",
|
||||
"result": "pass",
|
||||
"evidence": "All parse. None of the three scripts was executed: clean_audio drives ffmpeg over real media, enforce_vbox_hosts mutates VirtualBox guest disks, and migrate_hosts_guard rewrites systemd units for the hosts guard."
|
||||
},
|
||||
{
|
||||
"command": "Regression check of the extractor after the one-line-function fix",
|
||||
"result": "pass",
|
||||
"evidence": "Re-derived the committed fresh-install/main.sh split in a worktree at HEAD~3; both files byte-identical."
|
||||
},
|
||||
{
|
||||
"command": "bash ~/utils/scripts/check_file_length.sh --all",
|
||||
"result": "pass",
|
||||
"evidence": "26 files over cap, down from 29."
|
||||
}
|
||||
],
|
||||
"risks": [
|
||||
"enforce_vbox_hosts.sh's is_virtualbox is defined inside a heredoc that generates a guest-side script. The brace matcher counts it as part of the enclosing function, so it correctly stayed in the entry script - but this is the heredoc limitation recorded when the harness was built, and it happened to be safe here rather than by design. The verifier reporting 13 functions where grep counts 14 is how it surfaced.",
|
||||
"None of the three scripts has a test, so the evidence is static: verbatim proof, side-effect-free sourcing, shellcheck and bash -n."
|
||||
],
|
||||
"rollback": [
|
||||
"git revert the commit; the three entry scripts return to their original sizes and the four libraries disappear.",
|
||||
"After rollback, confirm bash -n passes on all three and that each contains its full set of function definitions again."
|
||||
]
|
||||
}
|
||||
@ -5,6 +5,9 @@
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# shellcheck source=lib/vbox_disk_inject.sh
|
||||
. "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/lib/vbox_disk_inject.sh"
|
||||
|
||||
# Colors
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
@ -166,237 +169,11 @@ EOF
|
||||
echo -e "${CYAN}Copy this script to your VMs and add it to their startup (e.g., /etc/rc.local or systemd)${NC}"
|
||||
}
|
||||
|
||||
# Get the disk image path for a VM (first SATA/IDE .vdi/.vmdk/.vhd)
|
||||
get_vm_disk_path() {
|
||||
local vm_name="$1"
|
||||
vboxmanage_as_user showvminfo "$vm_name" --machinereadable 2>/dev/null |
|
||||
grep -E '^"(SATA|IDE|SCSI|NVMe)-[0-9]+-[0-9]+"=' |
|
||||
grep -vE '="none"$' |
|
||||
grep -vE '\.iso"$' |
|
||||
head -1 |
|
||||
sed 's/^[^=]*="//; s/"$//'
|
||||
}
|
||||
|
||||
# Inject host's /etc/hosts directly into a VM disk image using qemu-nbd.
|
||||
# This is the only reliable way to enforce blocking, because NAT DNS proxy
|
||||
# alone does not work when the guest browser uses DNS-over-HTTPS (DoH).
|
||||
inject_hosts_into_vm_disk() {
|
||||
local vm_name="$1"
|
||||
local disk_path
|
||||
disk_path="$(get_vm_disk_path "$vm_name")"
|
||||
|
||||
if [[ -z $disk_path || ! -f $disk_path ]]; then
|
||||
echo -e "${YELLOW}Could not find disk image for VM '${vm_name}', skipping hosts injection${NC}"
|
||||
return 1
|
||||
fi
|
||||
|
||||
# Ensure VM is not running
|
||||
if vboxmanage_as_user list runningvms 2>/dev/null | grep -q "\"${vm_name}\""; then
|
||||
echo -e "${YELLOW}VM '${vm_name}' is running, cannot inject hosts file. Stop it first.${NC}"
|
||||
return 1
|
||||
fi
|
||||
|
||||
# Check for qemu-nbd
|
||||
if ! command -v qemu-nbd >/dev/null 2>&1; then
|
||||
echo -e "${YELLOW}qemu-nbd not found. Install qemu-base to enable hosts file injection.${NC}"
|
||||
return 1
|
||||
fi
|
||||
|
||||
echo -e "${BLUE}Injecting /etc/hosts into disk image for VM: ${vm_name}${NC}"
|
||||
|
||||
# Load nbd module if needed
|
||||
if [[ ! -e /dev/nbd0 ]]; then
|
||||
modprobe nbd max_part=8 2>/dev/null || {
|
||||
echo -e "${YELLOW}Could not load nbd kernel module${NC}"
|
||||
return 1
|
||||
}
|
||||
fi
|
||||
|
||||
# Find a free nbd device
|
||||
local nbd_dev=""
|
||||
for dev in /dev/nbd{0..15}; do
|
||||
if [[ -e $dev ]] && ! lsblk "$dev" >/dev/null 2>&1; then
|
||||
nbd_dev="$dev"
|
||||
break
|
||||
fi
|
||||
done
|
||||
# Fallback: try /dev/nbd0 if no device was found via lsblk check
|
||||
if [[ -z $nbd_dev ]]; then
|
||||
nbd_dev="/dev/nbd0"
|
||||
fi
|
||||
|
||||
local mount_point="/tmp/vbox_hosts_inject_$$"
|
||||
|
||||
# Connect disk image
|
||||
qemu-nbd --connect="$nbd_dev" "$disk_path" 2>/dev/null || {
|
||||
echo -e "${YELLOW}Could not connect disk image via qemu-nbd${NC}"
|
||||
return 1
|
||||
}
|
||||
|
||||
# Wait for partitions to appear
|
||||
sleep 1
|
||||
partprobe "$nbd_dev" 2>/dev/null || true
|
||||
|
||||
# Find the root partition (first Linux partition)
|
||||
local part=""
|
||||
for p in "${nbd_dev}p1" "${nbd_dev}p2" "${nbd_dev}p3"; do
|
||||
if [[ -b $p ]]; then
|
||||
part="$p"
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
if [[ -z $part ]]; then
|
||||
echo -e "${YELLOW}No partitions found on disk image${NC}"
|
||||
qemu-nbd --disconnect "$nbd_dev" 2>/dev/null || true
|
||||
return 1
|
||||
fi
|
||||
|
||||
# Mount the partition
|
||||
mkdir -p "$mount_point"
|
||||
if ! mount "$part" "$mount_point" 2>/dev/null; then
|
||||
# Journal may need recovery — run e2fsck then retry
|
||||
e2fsck -y "$part" >/dev/null 2>&1 || true
|
||||
if ! mount "$part" "$mount_point" 2>/dev/null; then
|
||||
echo -e "${YELLOW}Could not mount partition $part${NC}"
|
||||
qemu-nbd --disconnect "$nbd_dev" 2>/dev/null || true
|
||||
rmdir "$mount_point" 2>/dev/null || true
|
||||
return 1
|
||||
fi
|
||||
fi
|
||||
|
||||
# Check if this partition has /etc/hosts (i.e., it's the root fs)
|
||||
if [[ ! -f "$mount_point/etc/hosts" ]]; then
|
||||
echo -e "${YELLOW}Partition does not appear to be root filesystem (no /etc/hosts)${NC}"
|
||||
umount "$mount_point" 2>/dev/null || true
|
||||
qemu-nbd --disconnect "$nbd_dev" 2>/dev/null || true
|
||||
rmdir "$mount_point" 2>/dev/null || true
|
||||
return 1
|
||||
fi
|
||||
|
||||
# Backup original if not already backed up
|
||||
if [[ ! -f "$mount_point/etc/hosts.original" ]]; then
|
||||
cp "$mount_point/etc/hosts" "$mount_point/etc/hosts.original"
|
||||
echo -e "${CYAN}Backed up original hosts file${NC}"
|
||||
fi
|
||||
|
||||
# Copy host's /etc/hosts into VM
|
||||
cp /etc/hosts "$mount_point/etc/hosts"
|
||||
chmod 444 "$mount_point/etc/hosts"
|
||||
|
||||
local blocked_count
|
||||
blocked_count="$(grep -c '0.0.0.0' "$mount_point/etc/hosts")"
|
||||
|
||||
# Cleanup: unmount and disconnect
|
||||
umount "$mount_point" 2>/dev/null || true
|
||||
qemu-nbd --disconnect "$nbd_dev" 2>/dev/null || true
|
||||
rmdir "$mount_point" 2>/dev/null || true
|
||||
|
||||
echo -e "${GREEN}Hosts file injected into VM '${vm_name}' (${blocked_count} domains blocked)${NC}"
|
||||
return 0
|
||||
}
|
||||
|
||||
# Apply enforcement to all VMs
|
||||
enforce_all_vms() {
|
||||
local -a vms
|
||||
mapfile -t vms < <(get_all_vms)
|
||||
|
||||
if [[ ${#vms[@]} -eq 0 ]]; then
|
||||
echo -e "${YELLOW}No VirtualBox VMs found.${NC}"
|
||||
return 0
|
||||
fi
|
||||
|
||||
echo -e "${CYAN}Found ${#vms[@]} VM(s). Applying /etc/hosts enforcement...${NC}"
|
||||
|
||||
local success=0
|
||||
local failed=0
|
||||
|
||||
for vm in "${vms[@]}"; do
|
||||
echo -e "\n${BLUE}Processing VM: ${vm}${NC}"
|
||||
|
||||
# Configure DNS settings (works even when VM is running)
|
||||
configure_vm_dns "$vm"
|
||||
|
||||
# Try to configure shared folder (only works when VM is stopped)
|
||||
if configure_hosts_shared_folder "$vm"; then
|
||||
((++success))
|
||||
else
|
||||
((++failed))
|
||||
echo -e "${YELLOW}Note: Stop the VM and run this script again to add shared folder${NC}"
|
||||
fi
|
||||
|
||||
# Inject hosts file directly into VM disk (the actual enforcement)
|
||||
inject_hosts_into_vm_disk "$vm" || true
|
||||
done
|
||||
|
||||
echo -e "\n${GREEN}Enforcement complete!${NC}"
|
||||
echo -e "Successfully configured: ${success} VM(s)"
|
||||
[[ $failed -gt 0 ]] && echo -e "${YELLOW}Needs VM shutdown for full config: ${failed} VM(s)${NC}"
|
||||
|
||||
# Mark that enforcement has been applied
|
||||
touch "$HOSTS_ENFORCEMENT_MARKER"
|
||||
}
|
||||
|
||||
# Check if a single VM has the shared folder configured
|
||||
vm_has_shared_folder() {
|
||||
local vm_name="$1"
|
||||
vboxmanage_as_user showvminfo "$vm_name" --machinereadable 2>/dev/null |
|
||||
grep -q "SharedFolderNameMachineMapping.*=\"${VBOX_SHARED_FOLDER_NAME}\""
|
||||
}
|
||||
|
||||
# Check if enforcement is applied to ALL registered VMs
|
||||
check_enforcement_status() {
|
||||
local -a vms
|
||||
mapfile -t vms < <(get_all_vms)
|
||||
|
||||
if [[ ${#vms[@]} -eq 0 ]]; then
|
||||
echo -e "${GREEN}No VMs to enforce.${NC}"
|
||||
return 0
|
||||
fi
|
||||
|
||||
for vm in "${vms[@]}"; do
|
||||
if ! vm_has_shared_folder "$vm"; then
|
||||
echo -e "${YELLOW}VM '${vm}' is missing hosts enforcement.${NC}"
|
||||
return 1
|
||||
fi
|
||||
done
|
||||
|
||||
echo -e "${GREEN}All ${#vms[@]} VM(s) have hosts enforcement applied.${NC}"
|
||||
return 0
|
||||
}
|
||||
|
||||
# Show status
|
||||
show_status() {
|
||||
echo -e "${CYAN}VirtualBox Hosts Enforcement Status${NC}"
|
||||
echo -e "${CYAN}====================================${NC}\n"
|
||||
|
||||
local -a all_vms running_vms
|
||||
mapfile -t all_vms < <(get_all_vms)
|
||||
mapfile -t running_vms < <(get_running_vms)
|
||||
|
||||
echo -e "Total VMs: ${#all_vms[@]}"
|
||||
echo -e "Running VMs: ${#running_vms[@]}"
|
||||
|
||||
if check_enforcement_status >/dev/null 2>&1; then
|
||||
echo -e "Enforcement status: ${GREEN}Applied to all VMs${NC}"
|
||||
else
|
||||
echo -e "Enforcement status: ${RED}Not fully applied${NC}"
|
||||
fi
|
||||
|
||||
echo -e "\n${CYAN}VMs:${NC}"
|
||||
for vm in "${all_vms[@]}"; do
|
||||
local flags=""
|
||||
if printf '%s\n' "${running_vms[@]}" | grep -qx "$vm"; then
|
||||
flags+=" ${GREEN}[RUNNING]${NC}"
|
||||
fi
|
||||
if vm_has_shared_folder "$vm"; then
|
||||
flags+=" ${GREEN}[ENFORCED]${NC}"
|
||||
else
|
||||
flags+=" ${RED}[NOT ENFORCED]${NC}"
|
||||
fi
|
||||
echo -e " - ${vm}${flags}"
|
||||
done
|
||||
}
|
||||
|
||||
# Main function
|
||||
main() {
|
||||
|
||||
@ -0,0 +1,236 @@
|
||||
#!/usr/bin/env bash
|
||||
# Guest-disk hosts injection and enforcement status for enforce_vbox_hosts.sh.
|
||||
#
|
||||
# Sourced by the entry script, which owns the CLI and the VM enumeration.
|
||||
|
||||
# Get the disk image path for a VM (first SATA/IDE .vdi/.vmdk/.vhd)
|
||||
get_vm_disk_path() {
|
||||
local vm_name="$1"
|
||||
vboxmanage_as_user showvminfo "$vm_name" --machinereadable 2>/dev/null |
|
||||
grep -E '^"(SATA|IDE|SCSI|NVMe)-[0-9]+-[0-9]+"=' |
|
||||
grep -vE '="none"$' |
|
||||
grep -vE '\.iso"$' |
|
||||
head -1 |
|
||||
sed 's/^[^=]*="//; s/"$//'
|
||||
}
|
||||
|
||||
# Inject host's /etc/hosts directly into a VM disk image using qemu-nbd.
|
||||
# This is the only reliable way to enforce blocking, because NAT DNS proxy
|
||||
# alone does not work when the guest browser uses DNS-over-HTTPS (DoH).
|
||||
inject_hosts_into_vm_disk() {
|
||||
local vm_name="$1"
|
||||
local disk_path
|
||||
disk_path="$(get_vm_disk_path "$vm_name")"
|
||||
|
||||
if [[ -z $disk_path || ! -f $disk_path ]]; then
|
||||
echo -e "${YELLOW}Could not find disk image for VM '${vm_name}', skipping hosts injection${NC}"
|
||||
return 1
|
||||
fi
|
||||
|
||||
# Ensure VM is not running
|
||||
if vboxmanage_as_user list runningvms 2>/dev/null | grep -q "\"${vm_name}\""; then
|
||||
echo -e "${YELLOW}VM '${vm_name}' is running, cannot inject hosts file. Stop it first.${NC}"
|
||||
return 1
|
||||
fi
|
||||
|
||||
# Check for qemu-nbd
|
||||
if ! command -v qemu-nbd >/dev/null 2>&1; then
|
||||
echo -e "${YELLOW}qemu-nbd not found. Install qemu-base to enable hosts file injection.${NC}"
|
||||
return 1
|
||||
fi
|
||||
|
||||
echo -e "${BLUE}Injecting /etc/hosts into disk image for VM: ${vm_name}${NC}"
|
||||
|
||||
# Load nbd module if needed
|
||||
if [[ ! -e /dev/nbd0 ]]; then
|
||||
modprobe nbd max_part=8 2>/dev/null || {
|
||||
echo -e "${YELLOW}Could not load nbd kernel module${NC}"
|
||||
return 1
|
||||
}
|
||||
fi
|
||||
|
||||
# Find a free nbd device
|
||||
local nbd_dev=""
|
||||
for dev in /dev/nbd{0..15}; do
|
||||
if [[ -e $dev ]] && ! lsblk "$dev" >/dev/null 2>&1; then
|
||||
nbd_dev="$dev"
|
||||
break
|
||||
fi
|
||||
done
|
||||
# Fallback: try /dev/nbd0 if no device was found via lsblk check
|
||||
if [[ -z $nbd_dev ]]; then
|
||||
nbd_dev="/dev/nbd0"
|
||||
fi
|
||||
|
||||
local mount_point="/tmp/vbox_hosts_inject_$$"
|
||||
|
||||
# Connect disk image
|
||||
qemu-nbd --connect="$nbd_dev" "$disk_path" 2>/dev/null || {
|
||||
echo -e "${YELLOW}Could not connect disk image via qemu-nbd${NC}"
|
||||
return 1
|
||||
}
|
||||
|
||||
# Wait for partitions to appear
|
||||
sleep 1
|
||||
partprobe "$nbd_dev" 2>/dev/null || true
|
||||
|
||||
# Find the root partition (first Linux partition)
|
||||
local part=""
|
||||
for p in "${nbd_dev}p1" "${nbd_dev}p2" "${nbd_dev}p3"; do
|
||||
if [[ -b $p ]]; then
|
||||
part="$p"
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
if [[ -z $part ]]; then
|
||||
echo -e "${YELLOW}No partitions found on disk image${NC}"
|
||||
qemu-nbd --disconnect "$nbd_dev" 2>/dev/null || true
|
||||
return 1
|
||||
fi
|
||||
|
||||
# Mount the partition
|
||||
mkdir -p "$mount_point"
|
||||
if ! mount "$part" "$mount_point" 2>/dev/null; then
|
||||
# Journal may need recovery — run e2fsck then retry
|
||||
e2fsck -y "$part" >/dev/null 2>&1 || true
|
||||
if ! mount "$part" "$mount_point" 2>/dev/null; then
|
||||
echo -e "${YELLOW}Could not mount partition $part${NC}"
|
||||
qemu-nbd --disconnect "$nbd_dev" 2>/dev/null || true
|
||||
rmdir "$mount_point" 2>/dev/null || true
|
||||
return 1
|
||||
fi
|
||||
fi
|
||||
|
||||
# Check if this partition has /etc/hosts (i.e., it's the root fs)
|
||||
if [[ ! -f "$mount_point/etc/hosts" ]]; then
|
||||
echo -e "${YELLOW}Partition does not appear to be root filesystem (no /etc/hosts)${NC}"
|
||||
umount "$mount_point" 2>/dev/null || true
|
||||
qemu-nbd --disconnect "$nbd_dev" 2>/dev/null || true
|
||||
rmdir "$mount_point" 2>/dev/null || true
|
||||
return 1
|
||||
fi
|
||||
|
||||
# Backup original if not already backed up
|
||||
if [[ ! -f "$mount_point/etc/hosts.original" ]]; then
|
||||
cp "$mount_point/etc/hosts" "$mount_point/etc/hosts.original"
|
||||
echo -e "${CYAN}Backed up original hosts file${NC}"
|
||||
fi
|
||||
|
||||
# Copy host's /etc/hosts into VM
|
||||
cp /etc/hosts "$mount_point/etc/hosts"
|
||||
chmod 444 "$mount_point/etc/hosts"
|
||||
|
||||
local blocked_count
|
||||
blocked_count="$(grep -c '0.0.0.0' "$mount_point/etc/hosts")"
|
||||
|
||||
# Cleanup: unmount and disconnect
|
||||
umount "$mount_point" 2>/dev/null || true
|
||||
qemu-nbd --disconnect "$nbd_dev" 2>/dev/null || true
|
||||
rmdir "$mount_point" 2>/dev/null || true
|
||||
|
||||
echo -e "${GREEN}Hosts file injected into VM '${vm_name}' (${blocked_count} domains blocked)${NC}"
|
||||
return 0
|
||||
}
|
||||
|
||||
# Apply enforcement to all VMs
|
||||
enforce_all_vms() {
|
||||
local -a vms
|
||||
mapfile -t vms < <(get_all_vms)
|
||||
|
||||
if [[ ${#vms[@]} -eq 0 ]]; then
|
||||
echo -e "${YELLOW}No VirtualBox VMs found.${NC}"
|
||||
return 0
|
||||
fi
|
||||
|
||||
echo -e "${CYAN}Found ${#vms[@]} VM(s). Applying /etc/hosts enforcement...${NC}"
|
||||
|
||||
local success=0
|
||||
local failed=0
|
||||
|
||||
for vm in "${vms[@]}"; do
|
||||
echo -e "\n${BLUE}Processing VM: ${vm}${NC}"
|
||||
|
||||
# Configure DNS settings (works even when VM is running)
|
||||
configure_vm_dns "$vm"
|
||||
|
||||
# Try to configure shared folder (only works when VM is stopped)
|
||||
if configure_hosts_shared_folder "$vm"; then
|
||||
((++success))
|
||||
else
|
||||
((++failed))
|
||||
echo -e "${YELLOW}Note: Stop the VM and run this script again to add shared folder${NC}"
|
||||
fi
|
||||
|
||||
# Inject hosts file directly into VM disk (the actual enforcement)
|
||||
inject_hosts_into_vm_disk "$vm" || true
|
||||
done
|
||||
|
||||
echo -e "\n${GREEN}Enforcement complete!${NC}"
|
||||
echo -e "Successfully configured: ${success} VM(s)"
|
||||
[[ $failed -gt 0 ]] && echo -e "${YELLOW}Needs VM shutdown for full config: ${failed} VM(s)${NC}"
|
||||
|
||||
# Mark that enforcement has been applied
|
||||
touch "$HOSTS_ENFORCEMENT_MARKER"
|
||||
}
|
||||
|
||||
# Check if a single VM has the shared folder configured
|
||||
vm_has_shared_folder() {
|
||||
local vm_name="$1"
|
||||
vboxmanage_as_user showvminfo "$vm_name" --machinereadable 2>/dev/null |
|
||||
grep -q "SharedFolderNameMachineMapping.*=\"${VBOX_SHARED_FOLDER_NAME}\""
|
||||
}
|
||||
|
||||
# Check if enforcement is applied to ALL registered VMs
|
||||
check_enforcement_status() {
|
||||
local -a vms
|
||||
mapfile -t vms < <(get_all_vms)
|
||||
|
||||
if [[ ${#vms[@]} -eq 0 ]]; then
|
||||
echo -e "${GREEN}No VMs to enforce.${NC}"
|
||||
return 0
|
||||
fi
|
||||
|
||||
for vm in "${vms[@]}"; do
|
||||
if ! vm_has_shared_folder "$vm"; then
|
||||
echo -e "${YELLOW}VM '${vm}' is missing hosts enforcement.${NC}"
|
||||
return 1
|
||||
fi
|
||||
done
|
||||
|
||||
echo -e "${GREEN}All ${#vms[@]} VM(s) have hosts enforcement applied.${NC}"
|
||||
return 0
|
||||
}
|
||||
|
||||
# Show status
|
||||
show_status() {
|
||||
echo -e "${CYAN}VirtualBox Hosts Enforcement Status${NC}"
|
||||
echo -e "${CYAN}====================================${NC}\n"
|
||||
|
||||
local -a all_vms running_vms
|
||||
mapfile -t all_vms < <(get_all_vms)
|
||||
mapfile -t running_vms < <(get_running_vms)
|
||||
|
||||
echo -e "Total VMs: ${#all_vms[@]}"
|
||||
echo -e "Running VMs: ${#running_vms[@]}"
|
||||
|
||||
if check_enforcement_status >/dev/null 2>&1; then
|
||||
echo -e "Enforcement status: ${GREEN}Applied to all VMs${NC}"
|
||||
else
|
||||
echo -e "Enforcement status: ${RED}Not fully applied${NC}"
|
||||
fi
|
||||
|
||||
echo -e "\n${CYAN}VMs:${NC}"
|
||||
for vm in "${all_vms[@]}"; do
|
||||
local flags=""
|
||||
if printf '%s\n' "${running_vms[@]}" | grep -qx "$vm"; then
|
||||
flags+=" ${GREEN}[RUNNING]${NC}"
|
||||
fi
|
||||
if vm_has_shared_folder "$vm"; then
|
||||
flags+=" ${GREEN}[ENFORCED]${NC}"
|
||||
else
|
||||
flags+=" ${RED}[NOT ENFORCED]${NC}"
|
||||
fi
|
||||
echo -e " - ${vm}${flags}"
|
||||
done
|
||||
}
|
||||
161
linux_configuration/scripts/single_use/fixes/lib/hosts_guard_migrate.sh
Executable file
161
linux_configuration/scripts/single_use/fixes/lib/hosts_guard_migrate.sh
Executable file
@ -0,0 +1,161 @@
|
||||
#!/usr/bin/env bash
|
||||
# The migration and rollback steps for hosts-guard, plus their validation and
|
||||
# status reporting.
|
||||
#
|
||||
# Sourced by the entry script, which owns the CLI and the instance registry.
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# Preconditions
|
||||
# ----------------------------------------------------------------------------
|
||||
validate_requirements() {
|
||||
[[ -x $GUARDCTL ]] || {
|
||||
err "guardctl not found at $GUARDCTL - run guard-lib's install.sh first"
|
||||
exit 1
|
||||
}
|
||||
|
||||
local unit
|
||||
for unit in guard-file@.path guard-file@.service guard-bind-mount@.service; do
|
||||
[[ -f "/etc/systemd/system/$unit" ]] || {
|
||||
err "missing systemd template /etc/systemd/system/$unit - run guard-lib's install.sh first"
|
||||
exit 1
|
||||
}
|
||||
done
|
||||
|
||||
[[ -d $PLUGIN_SRC_DIR ]] || {
|
||||
err "plugin sources not found at $PLUGIN_SRC_DIR"
|
||||
exit 1
|
||||
}
|
||||
|
||||
# A live transaction would race every chattr and umount below.
|
||||
if [[ -e /var/lib/pacman/db.lck ]]; then
|
||||
err "/var/lib/pacman/db.lck exists - a pacman transaction is in flight"
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
|
||||
|
||||
install_plugins() {
|
||||
run mkdir -p "$PLUGIN_INSTALL_DIR"
|
||||
local src
|
||||
for src in "$PLUGIN_SRC_DIR"/*.sh; do
|
||||
[[ -e $src ]] || continue
|
||||
run install -m 755 "$src" "$PLUGIN_INSTALL_DIR/$(basename "$src")"
|
||||
done
|
||||
msg "plugins installed to $PLUGIN_INSTALL_DIR"
|
||||
}
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# Migration
|
||||
# ----------------------------------------------------------------------------
|
||||
stop_legacy_units_for() { # <instance name>
|
||||
local prefix="$1"
|
||||
local unit
|
||||
for unit in "${LEGACY_UNITS[@]}"; do
|
||||
[[ $unit == "$prefix"-* ]] || continue
|
||||
systemctl list-unit-files "$unit" &>/dev/null || continue
|
||||
run systemctl disable --now "$unit" 2>/dev/null || true
|
||||
done
|
||||
}
|
||||
|
||||
# Unmount every stacked bind layer so chattr can reach the real inode. Mirrors
|
||||
# the legacy pacman-pre-unlock-hosts.sh, which had to do exactly this.
|
||||
# Every branch is a full `if` and the function ends in an explicit `return 0`.
|
||||
# The obvious `((i > 20)) && break` form leaves the loop body's exit status at 1
|
||||
# whenever the arithmetic is false, and under `set -e` that aborted the whole
|
||||
# rollback silently — but ONLY when the loop actually ran, so --dry-run and an
|
||||
# already-unmounted target both looked fine. It cost a live /etc/hosts left with
|
||||
# neither the immutable flag nor its read-only mount to notice.
|
||||
collapse_mounts() { # <path>
|
||||
local target="$1" i=0
|
||||
while mountpoint -q "$target"; do
|
||||
if ! run umount -l "$target" 2>/dev/null; then
|
||||
break
|
||||
fi
|
||||
i=$((i + 1))
|
||||
if ((i > 20)); then
|
||||
break
|
||||
fi
|
||||
# Nothing is really unmounted under --dry-run, so `mountpoint` stays
|
||||
# true forever; report the intent once instead of 20 identical lines.
|
||||
if ((DRY_RUN == 1)); then
|
||||
break
|
||||
fi
|
||||
done
|
||||
return 0
|
||||
}
|
||||
|
||||
migrate_instance() { # <name>
|
||||
local name="$1"
|
||||
local spec target bind plugin also_watch
|
||||
spec="$(instance_spec "$name")"
|
||||
IFS='|' read -r target bind plugin also_watch <<<"$spec"
|
||||
|
||||
if instance_registered "$name"; then
|
||||
msg "$name already registered - skipping"
|
||||
return 0
|
||||
fi
|
||||
|
||||
if [[ ! -e $target ]]; then
|
||||
warn "$name: target $target does not exist - skipping"
|
||||
return 0
|
||||
fi
|
||||
|
||||
note "migrating $name ($target)"
|
||||
|
||||
stop_legacy_units_for "$name"
|
||||
collapse_mounts "$target"
|
||||
run chattr -i "$target" 2>/dev/null || true
|
||||
|
||||
local -a args=(file-guard install "$name" --target "$target")
|
||||
[[ $bind == "yes" ]] && args+=(--bind-mount)
|
||||
[[ -n $plugin ]] && args+=(--plugin "$PLUGIN_INSTALL_DIR/$plugin")
|
||||
[[ -n $also_watch ]] && args+=(--also-watch "$also_watch")
|
||||
|
||||
run "$GUARDCTL" "${args[@]}"
|
||||
msg "$name migrated"
|
||||
}
|
||||
|
||||
retire_legacy_hooks() {
|
||||
local hook removed=0
|
||||
for hook in "${LEGACY_HOOKS[@]}"; do
|
||||
if [[ -f "$HOOKS_DIR/$hook" ]]; then
|
||||
run rm -f "$HOOKS_DIR/$hook"
|
||||
msg "retired $hook"
|
||||
removed=1
|
||||
fi
|
||||
done
|
||||
((removed == 0)) && msg "legacy pacman hooks already retired"
|
||||
return 0
|
||||
}
|
||||
|
||||
do_migrate() {
|
||||
validate_requirements
|
||||
save_rollback_state
|
||||
install_plugins
|
||||
|
||||
local name
|
||||
for name in "${INSTANCES[@]}"; do
|
||||
migrate_instance "$name"
|
||||
done
|
||||
|
||||
# Only retire the legacy hooks once guard-lib actually owns something -
|
||||
# otherwise a failed migration would leave the files with NO pacman
|
||||
# unlock hook at all, and the next transaction would fight chattr +i.
|
||||
local registered=0
|
||||
for name in "${INSTANCES[@]}"; do
|
||||
instance_registered "$name" && registered=$((registered + 1))
|
||||
done
|
||||
|
||||
if ((registered == 0)) && ((DRY_RUN == 0)); then
|
||||
err "no guard-lib instance registered - refusing to retire the legacy hooks"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
retire_legacy_hooks
|
||||
|
||||
printf "\n"
|
||||
msg "migration complete ($registered instance(s) registered)"
|
||||
note "verify with: sudo $SCRIPT_NAME --status"
|
||||
note "undo with: sudo $SCRIPT_NAME --rollback"
|
||||
}
|
||||
141
linux_configuration/scripts/single_use/fixes/lib/hosts_guard_rollback.sh
Executable file
141
linux_configuration/scripts/single_use/fixes/lib/hosts_guard_rollback.sh
Executable file
@ -0,0 +1,141 @@
|
||||
#!/usr/bin/env bash
|
||||
# Rollback state capture, rollback itself, and status reporting for the
|
||||
# hosts-guard migration.
|
||||
#
|
||||
# Sourced by the entry script alongside hosts_guard_migrate.sh.
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# Status
|
||||
# ----------------------------------------------------------------------------
|
||||
show_status() {
|
||||
printf "\n%s\n" "=== guard-lib instances ==="
|
||||
local name
|
||||
for name in "${INSTANCES[@]}"; do
|
||||
if instance_registered "$name"; then
|
||||
msg "$name registered"
|
||||
"$GUARDCTL" file-guard status "$name" 2>&1 | sed 's/^/ /'
|
||||
else
|
||||
warn "$name NOT registered"
|
||||
fi
|
||||
done
|
||||
|
||||
printf "\n%s\n" "=== legacy pacman hooks ==="
|
||||
local hook
|
||||
for hook in "${LEGACY_HOOKS[@]}"; do
|
||||
if [[ -f "$HOOKS_DIR/$hook" ]]; then
|
||||
warn "$hook still present"
|
||||
else
|
||||
msg "$hook retired"
|
||||
fi
|
||||
done
|
||||
|
||||
printf "\n%s\n" "=== legacy systemd units ==="
|
||||
local unit state
|
||||
for unit in "${LEGACY_UNITS[@]}"; do
|
||||
# `is-enabled` PRINTS the state but EXITS 1 for anything not enabled, so
|
||||
# `|| echo absent` would append a second line to a perfectly good answer.
|
||||
state="$(systemctl is-enabled "$unit" 2>/dev/null)" || true
|
||||
[[ -n $state ]] || state="absent"
|
||||
if [[ $state == "enabled" ]]; then
|
||||
warn "$unit is $state"
|
||||
else
|
||||
msg "$unit is $state"
|
||||
fi
|
||||
done
|
||||
}
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# Rollback state
|
||||
# ----------------------------------------------------------------------------
|
||||
save_rollback_state() {
|
||||
run mkdir -p "$STATE_DIR/hooks"
|
||||
|
||||
local hook
|
||||
for hook in "${LEGACY_HOOKS[@]}"; do
|
||||
if [[ -f "$HOOKS_DIR/$hook" && ! -f "$STATE_DIR/hooks/$hook" ]]; then
|
||||
run cp -a "$HOOKS_DIR/$hook" "$STATE_DIR/hooks/$hook"
|
||||
note "backed up $hook"
|
||||
fi
|
||||
done
|
||||
|
||||
# Record enablement so rollback restores the exact prior state rather than
|
||||
# guessing (resolved-guard.* were already disabled here, for instance).
|
||||
if [[ ! -f "$STATE_DIR/units.state" ]] && ((DRY_RUN == 0)); then
|
||||
local unit ustate
|
||||
for unit in "${LEGACY_UNITS[@]}"; do
|
||||
ustate="$(systemctl is-enabled "$unit" 2>/dev/null)" || true
|
||||
[[ -n $ustate ]] || ustate="absent"
|
||||
printf '%s=%s\n' "$unit" "$ustate"
|
||||
done >"$STATE_DIR/units.state"
|
||||
note "recorded legacy unit states in $STATE_DIR/units.state"
|
||||
fi
|
||||
}
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# Rollback
|
||||
# ----------------------------------------------------------------------------
|
||||
do_rollback() {
|
||||
[[ -d $STATE_DIR ]] || {
|
||||
err "no rollback state at $STATE_DIR - nothing to roll back to"
|
||||
exit 1
|
||||
}
|
||||
|
||||
local name
|
||||
for name in "${INSTANCES[@]}"; do
|
||||
if instance_registered "$name"; then
|
||||
# Keep the canonical: it is the known-good copy of the file, and
|
||||
# the legacy layer re-snapshots from the live file, which may have
|
||||
# drifted. Losing it would turn a rollback into data loss.
|
||||
run "$GUARDCTL" file-guard uninstall "$name" --keep-canonical || true
|
||||
msg "uninstalled guard-lib instance $name"
|
||||
fi
|
||||
done
|
||||
|
||||
local hook
|
||||
for hook in "${LEGACY_HOOKS[@]}"; do
|
||||
if [[ -f "$STATE_DIR/hooks/$hook" ]]; then
|
||||
run cp -a "$STATE_DIR/hooks/$hook" "$HOOKS_DIR/$hook"
|
||||
msg "restored $hook"
|
||||
fi
|
||||
done
|
||||
|
||||
if [[ -f "$STATE_DIR/units.state" ]]; then
|
||||
local unit state
|
||||
while IFS='=' read -r unit state; do
|
||||
[[ $state == "enabled" ]] || continue
|
||||
run systemctl enable --now "$unit" 2>/dev/null || warn "could not re-enable $unit"
|
||||
msg "re-enabled $unit"
|
||||
done <"$STATE_DIR/units.state"
|
||||
else
|
||||
warn "no units.state recorded - legacy units left as-is"
|
||||
fi
|
||||
|
||||
# Re-assert the legacy protections NOW rather than waiting for a watcher.
|
||||
# `guardctl file-guard uninstall` chattr -i's the target on its way out, and
|
||||
# the legacy *-guard.path units only fire on PathModified - so without this
|
||||
# a rolled-back /etc/hosts sits with NO immutable flag until something
|
||||
# happens to write to it. Caught by actually running --rollback and looking
|
||||
# at lsattr, which is the only reason it is handled at all.
|
||||
#
|
||||
# Order matters for hosts specifically: it is bind-mounted read-only, and
|
||||
# chattr cannot write through a ro mount. enforce-hosts.sh does a bare
|
||||
# `chattr +i` with no collapse (it relies on running before the mount at
|
||||
# boot), so it fails silently here unless the mount is dropped first. So:
|
||||
# collapse, enforce, then let the bind-mount unit rebuild the ro layer.
|
||||
collapse_mounts /etc/hosts
|
||||
|
||||
local svc
|
||||
for svc in hosts-guard.service nsswitch-guard.service resolved-guard.service; do
|
||||
systemctl cat "$svc" &>/dev/null || continue
|
||||
run systemctl start "$svc" 2>/dev/null || warn "could not run $svc"
|
||||
done
|
||||
|
||||
if systemctl cat hosts-bind-mount.service &>/dev/null; then
|
||||
run systemctl restart hosts-bind-mount.service 2>/dev/null || warn "could not restart hosts-bind-mount.service"
|
||||
fi
|
||||
msg "re-ran legacy enforcement (restores chattr +i and the ro bind mount)"
|
||||
|
||||
printf "\n"
|
||||
msg "rollback complete"
|
||||
note "verify with: sudo $SCRIPT_NAME --status"
|
||||
}
|
||||
@ -29,6 +29,11 @@
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# shellcheck source=lib/hosts_guard_migrate.sh
|
||||
. "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/lib/hosts_guard_migrate.sh"
|
||||
# shellcheck source=lib/hosts_guard_rollback.sh
|
||||
. "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/lib/hosts_guard_rollback.sh"
|
||||
|
||||
SCRIPT_NAME="$(basename "$0")"
|
||||
readonly SCRIPT_NAME
|
||||
readonly GUARDCTL="/usr/local/bin/guardctl"
|
||||
@ -120,295 +125,15 @@ readonly INSTANCES=(hosts nsswitch resolved)
|
||||
|
||||
instance_registered() { [[ -f "$TARGETS_DIR/$1.conf" ]]; }
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# Preconditions
|
||||
# ----------------------------------------------------------------------------
|
||||
validate_requirements() {
|
||||
[[ -x $GUARDCTL ]] || {
|
||||
err "guardctl not found at $GUARDCTL - run guard-lib's install.sh first"
|
||||
exit 1
|
||||
}
|
||||
|
||||
local unit
|
||||
for unit in guard-file@.path guard-file@.service guard-bind-mount@.service; do
|
||||
[[ -f "/etc/systemd/system/$unit" ]] || {
|
||||
err "missing systemd template /etc/systemd/system/$unit - run guard-lib's install.sh first"
|
||||
exit 1
|
||||
}
|
||||
done
|
||||
|
||||
[[ -d $PLUGIN_SRC_DIR ]] || {
|
||||
err "plugin sources not found at $PLUGIN_SRC_DIR"
|
||||
exit 1
|
||||
}
|
||||
|
||||
# A live transaction would race every chattr and umount below.
|
||||
if [[ -e /var/lib/pacman/db.lck ]]; then
|
||||
err "/var/lib/pacman/db.lck exists - a pacman transaction is in flight"
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# Status
|
||||
# ----------------------------------------------------------------------------
|
||||
show_status() {
|
||||
printf "\n%s\n" "=== guard-lib instances ==="
|
||||
local name
|
||||
for name in "${INSTANCES[@]}"; do
|
||||
if instance_registered "$name"; then
|
||||
msg "$name registered"
|
||||
"$GUARDCTL" file-guard status "$name" 2>&1 | sed 's/^/ /'
|
||||
else
|
||||
warn "$name NOT registered"
|
||||
fi
|
||||
done
|
||||
|
||||
printf "\n%s\n" "=== legacy pacman hooks ==="
|
||||
local hook
|
||||
for hook in "${LEGACY_HOOKS[@]}"; do
|
||||
if [[ -f "$HOOKS_DIR/$hook" ]]; then
|
||||
warn "$hook still present"
|
||||
else
|
||||
msg "$hook retired"
|
||||
fi
|
||||
done
|
||||
|
||||
printf "\n%s\n" "=== legacy systemd units ==="
|
||||
local unit state
|
||||
for unit in "${LEGACY_UNITS[@]}"; do
|
||||
# `is-enabled` PRINTS the state but EXITS 1 for anything not enabled, so
|
||||
# `|| echo absent` would append a second line to a perfectly good answer.
|
||||
state="$(systemctl is-enabled "$unit" 2>/dev/null)" || true
|
||||
[[ -n $state ]] || state="absent"
|
||||
if [[ $state == "enabled" ]]; then
|
||||
warn "$unit is $state"
|
||||
else
|
||||
msg "$unit is $state"
|
||||
fi
|
||||
done
|
||||
}
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# Rollback state
|
||||
# ----------------------------------------------------------------------------
|
||||
save_rollback_state() {
|
||||
run mkdir -p "$STATE_DIR/hooks"
|
||||
|
||||
local hook
|
||||
for hook in "${LEGACY_HOOKS[@]}"; do
|
||||
if [[ -f "$HOOKS_DIR/$hook" && ! -f "$STATE_DIR/hooks/$hook" ]]; then
|
||||
run cp -a "$HOOKS_DIR/$hook" "$STATE_DIR/hooks/$hook"
|
||||
note "backed up $hook"
|
||||
fi
|
||||
done
|
||||
|
||||
# Record enablement so rollback restores the exact prior state rather than
|
||||
# guessing (resolved-guard.* were already disabled here, for instance).
|
||||
if [[ ! -f "$STATE_DIR/units.state" ]] && ((DRY_RUN == 0)); then
|
||||
local unit ustate
|
||||
for unit in "${LEGACY_UNITS[@]}"; do
|
||||
ustate="$(systemctl is-enabled "$unit" 2>/dev/null)" || true
|
||||
[[ -n $ustate ]] || ustate="absent"
|
||||
printf '%s=%s\n' "$unit" "$ustate"
|
||||
done >"$STATE_DIR/units.state"
|
||||
note "recorded legacy unit states in $STATE_DIR/units.state"
|
||||
fi
|
||||
}
|
||||
|
||||
install_plugins() {
|
||||
run mkdir -p "$PLUGIN_INSTALL_DIR"
|
||||
local src
|
||||
for src in "$PLUGIN_SRC_DIR"/*.sh; do
|
||||
[[ -e $src ]] || continue
|
||||
run install -m 755 "$src" "$PLUGIN_INSTALL_DIR/$(basename "$src")"
|
||||
done
|
||||
msg "plugins installed to $PLUGIN_INSTALL_DIR"
|
||||
}
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# Migration
|
||||
# ----------------------------------------------------------------------------
|
||||
stop_legacy_units_for() { # <instance name>
|
||||
local prefix="$1"
|
||||
local unit
|
||||
for unit in "${LEGACY_UNITS[@]}"; do
|
||||
[[ $unit == "$prefix"-* ]] || continue
|
||||
systemctl list-unit-files "$unit" &>/dev/null || continue
|
||||
run systemctl disable --now "$unit" 2>/dev/null || true
|
||||
done
|
||||
}
|
||||
|
||||
# Unmount every stacked bind layer so chattr can reach the real inode. Mirrors
|
||||
# the legacy pacman-pre-unlock-hosts.sh, which had to do exactly this.
|
||||
# Every branch is a full `if` and the function ends in an explicit `return 0`.
|
||||
# The obvious `((i > 20)) && break` form leaves the loop body's exit status at 1
|
||||
# whenever the arithmetic is false, and under `set -e` that aborted the whole
|
||||
# rollback silently — but ONLY when the loop actually ran, so --dry-run and an
|
||||
# already-unmounted target both looked fine. It cost a live /etc/hosts left with
|
||||
# neither the immutable flag nor its read-only mount to notice.
|
||||
collapse_mounts() { # <path>
|
||||
local target="$1" i=0
|
||||
while mountpoint -q "$target"; do
|
||||
if ! run umount -l "$target" 2>/dev/null; then
|
||||
break
|
||||
fi
|
||||
i=$((i + 1))
|
||||
if ((i > 20)); then
|
||||
break
|
||||
fi
|
||||
# Nothing is really unmounted under --dry-run, so `mountpoint` stays
|
||||
# true forever; report the intent once instead of 20 identical lines.
|
||||
if ((DRY_RUN == 1)); then
|
||||
break
|
||||
fi
|
||||
done
|
||||
return 0
|
||||
}
|
||||
|
||||
migrate_instance() { # <name>
|
||||
local name="$1"
|
||||
local spec target bind plugin also_watch
|
||||
spec="$(instance_spec "$name")"
|
||||
IFS='|' read -r target bind plugin also_watch <<<"$spec"
|
||||
|
||||
if instance_registered "$name"; then
|
||||
msg "$name already registered - skipping"
|
||||
return 0
|
||||
fi
|
||||
|
||||
if [[ ! -e $target ]]; then
|
||||
warn "$name: target $target does not exist - skipping"
|
||||
return 0
|
||||
fi
|
||||
|
||||
note "migrating $name ($target)"
|
||||
|
||||
stop_legacy_units_for "$name"
|
||||
collapse_mounts "$target"
|
||||
run chattr -i "$target" 2>/dev/null || true
|
||||
|
||||
local -a args=(file-guard install "$name" --target "$target")
|
||||
[[ $bind == "yes" ]] && args+=(--bind-mount)
|
||||
[[ -n $plugin ]] && args+=(--plugin "$PLUGIN_INSTALL_DIR/$plugin")
|
||||
[[ -n $also_watch ]] && args+=(--also-watch "$also_watch")
|
||||
|
||||
run "$GUARDCTL" "${args[@]}"
|
||||
msg "$name migrated"
|
||||
}
|
||||
|
||||
retire_legacy_hooks() {
|
||||
local hook removed=0
|
||||
for hook in "${LEGACY_HOOKS[@]}"; do
|
||||
if [[ -f "$HOOKS_DIR/$hook" ]]; then
|
||||
run rm -f "$HOOKS_DIR/$hook"
|
||||
msg "retired $hook"
|
||||
removed=1
|
||||
fi
|
||||
done
|
||||
((removed == 0)) && msg "legacy pacman hooks already retired"
|
||||
return 0
|
||||
}
|
||||
|
||||
do_migrate() {
|
||||
validate_requirements
|
||||
save_rollback_state
|
||||
install_plugins
|
||||
|
||||
local name
|
||||
for name in "${INSTANCES[@]}"; do
|
||||
migrate_instance "$name"
|
||||
done
|
||||
|
||||
# Only retire the legacy hooks once guard-lib actually owns something -
|
||||
# otherwise a failed migration would leave the files with NO pacman
|
||||
# unlock hook at all, and the next transaction would fight chattr +i.
|
||||
local registered=0
|
||||
for name in "${INSTANCES[@]}"; do
|
||||
instance_registered "$name" && registered=$((registered + 1))
|
||||
done
|
||||
|
||||
if ((registered == 0)) && ((DRY_RUN == 0)); then
|
||||
err "no guard-lib instance registered - refusing to retire the legacy hooks"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
retire_legacy_hooks
|
||||
|
||||
printf "\n"
|
||||
msg "migration complete ($registered instance(s) registered)"
|
||||
note "verify with: sudo $SCRIPT_NAME --status"
|
||||
note "undo with: sudo $SCRIPT_NAME --rollback"
|
||||
}
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# Rollback
|
||||
# ----------------------------------------------------------------------------
|
||||
do_rollback() {
|
||||
[[ -d $STATE_DIR ]] || {
|
||||
err "no rollback state at $STATE_DIR - nothing to roll back to"
|
||||
exit 1
|
||||
}
|
||||
|
||||
local name
|
||||
for name in "${INSTANCES[@]}"; do
|
||||
if instance_registered "$name"; then
|
||||
# Keep the canonical: it is the known-good copy of the file, and
|
||||
# the legacy layer re-snapshots from the live file, which may have
|
||||
# drifted. Losing it would turn a rollback into data loss.
|
||||
run "$GUARDCTL" file-guard uninstall "$name" --keep-canonical || true
|
||||
msg "uninstalled guard-lib instance $name"
|
||||
fi
|
||||
done
|
||||
|
||||
local hook
|
||||
for hook in "${LEGACY_HOOKS[@]}"; do
|
||||
if [[ -f "$STATE_DIR/hooks/$hook" ]]; then
|
||||
run cp -a "$STATE_DIR/hooks/$hook" "$HOOKS_DIR/$hook"
|
||||
msg "restored $hook"
|
||||
fi
|
||||
done
|
||||
|
||||
if [[ -f "$STATE_DIR/units.state" ]]; then
|
||||
local unit state
|
||||
while IFS='=' read -r unit state; do
|
||||
[[ $state == "enabled" ]] || continue
|
||||
run systemctl enable --now "$unit" 2>/dev/null || warn "could not re-enable $unit"
|
||||
msg "re-enabled $unit"
|
||||
done <"$STATE_DIR/units.state"
|
||||
else
|
||||
warn "no units.state recorded - legacy units left as-is"
|
||||
fi
|
||||
|
||||
# Re-assert the legacy protections NOW rather than waiting for a watcher.
|
||||
# `guardctl file-guard uninstall` chattr -i's the target on its way out, and
|
||||
# the legacy *-guard.path units only fire on PathModified - so without this
|
||||
# a rolled-back /etc/hosts sits with NO immutable flag until something
|
||||
# happens to write to it. Caught by actually running --rollback and looking
|
||||
# at lsattr, which is the only reason it is handled at all.
|
||||
#
|
||||
# Order matters for hosts specifically: it is bind-mounted read-only, and
|
||||
# chattr cannot write through a ro mount. enforce-hosts.sh does a bare
|
||||
# `chattr +i` with no collapse (it relies on running before the mount at
|
||||
# boot), so it fails silently here unless the mount is dropped first. So:
|
||||
# collapse, enforce, then let the bind-mount unit rebuild the ro layer.
|
||||
collapse_mounts /etc/hosts
|
||||
|
||||
local svc
|
||||
for svc in hosts-guard.service nsswitch-guard.service resolved-guard.service; do
|
||||
systemctl cat "$svc" &>/dev/null || continue
|
||||
run systemctl start "$svc" 2>/dev/null || warn "could not run $svc"
|
||||
done
|
||||
|
||||
if systemctl cat hosts-bind-mount.service &>/dev/null; then
|
||||
run systemctl restart hosts-bind-mount.service 2>/dev/null || warn "could not restart hosts-bind-mount.service"
|
||||
fi
|
||||
msg "re-ran legacy enforcement (restores chattr +i and the ro bind mount)"
|
||||
|
||||
printf "\n"
|
||||
msg "rollback complete"
|
||||
note "verify with: sudo $SCRIPT_NAME --status"
|
||||
}
|
||||
|
||||
# Parse arguments
|
||||
require_root "$@"
|
||||
|
||||
@ -2,6 +2,9 @@
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# shellcheck source=lib/clean_audio_filters.sh
|
||||
. "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/lib/clean_audio_filters.sh"
|
||||
|
||||
# clean_audio.sh — Fully automatic audio cleaner for speech (ASR-friendly)
|
||||
#
|
||||
# - Default preset is tuned for ASR (faster-whisper):
|
||||
@ -24,36 +27,7 @@ set -euo pipefail
|
||||
|
||||
SCRIPT_DIR=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &> /dev/null && pwd)
|
||||
|
||||
print_usage() {
|
||||
cat << EOF
|
||||
Usage: $0 <input-file|input-dir> [options]
|
||||
|
||||
Options:
|
||||
-O, --out-dir DIR Output directory (default: alongside input file).
|
||||
-e, --ext EXT Output extension/container: wav|flac (default: wav).
|
||||
-m, --model PATH RNNoise model file for arnndn; required by default unless --allow-fallback.
|
||||
--no-ml Do not use arnndn even if model is provided (requires --allow-fallback).
|
||||
--preset NAME asr (default) | podcast | aggressive
|
||||
-j, --jobs N Parallel jobs for directory mode (default: 1).
|
||||
-f, --force Overwrite outputs if they exist (ffmpeg -y).
|
||||
-q, --quiet Reduce ffmpeg logging noise.
|
||||
--lowpass FREQ Optional low-pass cutoff (e.g., 8000). Disabled by default.
|
||||
--suffix SUF Suffix for output basename (default: _clean).
|
||||
-h, --help Show this help.
|
||||
|
||||
Notes:
|
||||
- Default sample rate is 16 kHz mono PCM 16-bit (good for most speech ASR models).
|
||||
- If arnndn (RNNoise) is used, it usually outperforms afftdn for speech denoise.
|
||||
- The 'podcast' preset adds gentle dynamics and loudness normalization (single-pass).
|
||||
EOF
|
||||
}
|
||||
|
||||
require_cmd() {
|
||||
command -v "$1" > /dev/null 2>&1 || {
|
||||
echo "Error: Required command '$1' not found in PATH" >&2
|
||||
exit 1
|
||||
}
|
||||
}
|
||||
|
||||
# Defaults
|
||||
OUT_DIR=""
|
||||
@ -193,39 +167,6 @@ if "$FFMPEG_BIN" -hide_banner -h filter=afftdn 2> /dev/null | grep -q " md="; th
|
||||
afftdn_supports_md=true
|
||||
fi
|
||||
|
||||
# Try to auto-discover an RNNoise model if none provided
|
||||
find_default_rn_model() {
|
||||
# local candidate reserved for future selection logic
|
||||
# Allow env variable override
|
||||
if [[ -n ${RNNOISE_MODEL:-} && -f ${RNNOISE_MODEL} ]]; then
|
||||
echo "${RNNOISE_MODEL}"
|
||||
return 0
|
||||
fi
|
||||
local dirs=(
|
||||
"$SCRIPT_DIR/models"
|
||||
"$SCRIPT_DIR/../models"
|
||||
"/usr/share/rnnoise"
|
||||
"/usr/local/share/rnnoise"
|
||||
"/usr/share/ffmpeg/models"
|
||||
"$HOME/.local/share/rnnoise"
|
||||
)
|
||||
# Prefer '.rnnn' models (rnnoise-nu style) over legacy '.nn'
|
||||
local exts=("rnnn" "nn" "model")
|
||||
for d in "${dirs[@]}"; do
|
||||
if [[ -d $d ]]; then
|
||||
for ext in "${exts[@]}"; do
|
||||
# Pick the first matching model file
|
||||
for f in "$d"/*."$ext"; do
|
||||
if [[ -f $f ]]; then
|
||||
echo "$f"
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
done
|
||||
fi
|
||||
done
|
||||
return 1
|
||||
}
|
||||
|
||||
use_arnndn=false
|
||||
if [[ $NO_ML == false ]]; then
|
||||
@ -260,107 +201,8 @@ if [[ $NO_ML == false ]]; then
|
||||
fi
|
||||
fi
|
||||
|
||||
build_filters() {
|
||||
local filters=()
|
||||
# Remove low-frequency rumble typical for handheld/room noise
|
||||
filters+=("highpass=f=${HIGHPASS}")
|
||||
|
||||
# Denoise
|
||||
if $use_arnndn; then
|
||||
# arnndn with full mix keeps the model output; if no external model, rely on built-in
|
||||
filters+=("aresample=48000")
|
||||
filters+=("arnndn=m=${RN_MODEL}:mix=1.0")
|
||||
else
|
||||
# afftdn: FFT-based denoise, tune nf (noise floor) as needed
|
||||
if $REQUIRE_ML; then
|
||||
echo "Error: RNNoise required but not in use; aborting rather than falling back to afftdn. Use --allow-fallback to permit." >&2
|
||||
exit 11
|
||||
fi
|
||||
if $NO_ADVANCED; then
|
||||
filters+=("afftdn=nf=${AFFTDN_NF}")
|
||||
else
|
||||
if $afftdn_supports_md; then
|
||||
filters+=("afftdn=nf=${AFFTDN_NF}:md=${AFFTDN_MD}")
|
||||
else
|
||||
echo "Error: Your ffmpeg's afftdn filter does not support 'md='." >&2
|
||||
echo "Hint: Install/upgrade ffmpeg to a build that supports afftdn md or rerun with --no-advanced." >&2
|
||||
echo " On Debian/Ubuntu you may need a newer ffmpeg from a PPA or build from source." >&2
|
||||
exit 8
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
# Optional low-pass to shave hiss; keep disabled unless requested
|
||||
if [[ -n $LOWPASS ]]; then
|
||||
filters+=("lowpass=f=${LOWPASS}")
|
||||
fi
|
||||
|
||||
case "$PRESET" in
|
||||
asr)
|
||||
# ASR-friendly: avoid heavy gating/leveling, just prevent clipping
|
||||
filters+=("alimiter=limit=0.94")
|
||||
;;
|
||||
podcast)
|
||||
# Gentle dynamic normalization and broadcast-ish loudness (single-pass)
|
||||
# Note: single-pass loudnorm is approximate but OK for quick workflows
|
||||
filters+=("dynaudnorm=f=500:g=5:p=0.1")
|
||||
filters+=("loudnorm=i=-18:lra=9:tp=-2.0")
|
||||
;;
|
||||
aggressive)
|
||||
# Heavier clean-up; may harm ASR slightly but suppress background more
|
||||
filters+=("agate=threshold=0.012:ratio=2.5:release=200")
|
||||
filters+=("dynaudnorm=f=400:g=7:p=0.1")
|
||||
filters+=("loudnorm=i=-18:lra=9:tp=-2.0")
|
||||
;;
|
||||
*) ;;
|
||||
esac
|
||||
|
||||
# Resample and format at the end for ASR
|
||||
filters+=("aresample=16000")
|
||||
filters+=("aformat=channel_layouts=mono:sample_fmts=s16")
|
||||
|
||||
local IFS=","
|
||||
echo "${filters[*]}"
|
||||
}
|
||||
|
||||
make_out_path_for_file() {
|
||||
local in_file="$1"
|
||||
local base
|
||||
base=$(basename -- "$in_file")
|
||||
base="${base%.*}"
|
||||
local out_base="${base}${SUFFIX}.${OUT_EXT}"
|
||||
if [[ -n $OUT_DIR ]]; then
|
||||
mkdir -p -- "$OUT_DIR"
|
||||
echo "$OUT_DIR/$out_base"
|
||||
else
|
||||
local dir
|
||||
dir=$(dirname -- "$in_file")
|
||||
echo "$dir/$out_base"
|
||||
fi
|
||||
}
|
||||
|
||||
process_one() {
|
||||
local in_file="$1"
|
||||
local out_file
|
||||
out_file=$(make_out_path_for_file "$in_file")
|
||||
|
||||
# Choose codec based on extension
|
||||
local codec=(-c:a pcm_s16le)
|
||||
if [[ $OUT_EXT == "flac" ]]; then
|
||||
codec=(-c:a flac)
|
||||
fi
|
||||
|
||||
local af
|
||||
af=$(build_filters)
|
||||
|
||||
if [[ -f $out_file && $FORCE == false ]]; then
|
||||
echo "Skip (exists): $out_file"
|
||||
return 0
|
||||
fi
|
||||
|
||||
echo "Cleaning: $in_file -> $out_file"
|
||||
"$FFMPEG_BIN" "${FFMPEG_LOG[@]}" "${FFMPEG_OVERWRITE[@]}" -i "$in_file" -af "$af" "${codec[@]}" "$out_file"
|
||||
}
|
||||
|
||||
# Concurrency helpers (bash >= 5 supports wait -n; fallback to sequential if not)
|
||||
supports_wait_n=false
|
||||
@ -368,37 +210,6 @@ if [[ -n ${BASH_VERSINFO:-} && ${BASH_VERSINFO[0]} -ge 5 ]]; then
|
||||
supports_wait_n=true
|
||||
fi
|
||||
|
||||
run_dir() {
|
||||
local dir="$1"
|
||||
# Common audio extensions (case-insensitive)
|
||||
mapfile -d '' files < <(find "$dir" -type f \
|
||||
\( -iname "*.wav" -o -iname "*.mp3" -o -iname "*.m4a" -o -iname "*.aac" -o -iname "*.flac" \
|
||||
-o -iname "*.ogg" -o -iname "*.opus" -o -iname "*.wma" -o -iname "*.webm" \) -print0)
|
||||
|
||||
if [[ ${#files[@]} -eq 0 ]]; then
|
||||
echo "No audio files found in: $dir"
|
||||
return 0
|
||||
fi
|
||||
|
||||
local running=0
|
||||
for f in "${files[@]}"; do
|
||||
if [[ $JOBS -le 1 || $supports_wait_n == false ]]; then
|
||||
process_one "$f"
|
||||
else
|
||||
process_one "$f" &
|
||||
((running++))
|
||||
if ((running >= JOBS)); then
|
||||
wait -n || true
|
||||
((running--))
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
# Wait for any remaining background jobs
|
||||
if ((JOBS > 1)) && $supports_wait_n; then
|
||||
wait || true
|
||||
fi
|
||||
}
|
||||
|
||||
main() {
|
||||
# Sanity checks and notices
|
||||
|
||||
@ -0,0 +1,209 @@
|
||||
#!/usr/bin/env bash
|
||||
# Usage text, dependency checks, ffmpeg filter construction and per-file
|
||||
# processing for clean_audio.sh.
|
||||
#
|
||||
# Sourced by the entry script, which owns argument parsing and the run flow.
|
||||
#
|
||||
# The entry script probes ffmpeg once and sets these before calling
|
||||
# build_filters. Declaring them here makes that contract explicit and keeps
|
||||
# the static checker able to see it across the file boundary.
|
||||
declare use_arnndn afftdn_supports_md supports_wait_n
|
||||
|
||||
print_usage() {
|
||||
cat << EOF
|
||||
Usage: $0 <input-file|input-dir> [options]
|
||||
|
||||
Options:
|
||||
-O, --out-dir DIR Output directory (default: alongside input file).
|
||||
-e, --ext EXT Output extension/container: wav|flac (default: wav).
|
||||
-m, --model PATH RNNoise model file for arnndn; required by default unless --allow-fallback.
|
||||
--no-ml Do not use arnndn even if model is provided (requires --allow-fallback).
|
||||
--preset NAME asr (default) | podcast | aggressive
|
||||
-j, --jobs N Parallel jobs for directory mode (default: 1).
|
||||
-f, --force Overwrite outputs if they exist (ffmpeg -y).
|
||||
-q, --quiet Reduce ffmpeg logging noise.
|
||||
--lowpass FREQ Optional low-pass cutoff (e.g., 8000). Disabled by default.
|
||||
--suffix SUF Suffix for output basename (default: _clean).
|
||||
-h, --help Show this help.
|
||||
|
||||
Notes:
|
||||
- Default sample rate is 16 kHz mono PCM 16-bit (good for most speech ASR models).
|
||||
- If arnndn (RNNoise) is used, it usually outperforms afftdn for speech denoise.
|
||||
- The 'podcast' preset adds gentle dynamics and loudness normalization (single-pass).
|
||||
EOF
|
||||
}
|
||||
|
||||
require_cmd() {
|
||||
command -v "$1" > /dev/null 2>&1 || {
|
||||
echo "Error: Required command '$1' not found in PATH" >&2
|
||||
exit 1
|
||||
}
|
||||
}
|
||||
|
||||
# Try to auto-discover an RNNoise model if none provided
|
||||
find_default_rn_model() {
|
||||
# local candidate reserved for future selection logic
|
||||
# Allow env variable override
|
||||
if [[ -n ${RNNOISE_MODEL:-} && -f ${RNNOISE_MODEL} ]]; then
|
||||
echo "${RNNOISE_MODEL}"
|
||||
return 0
|
||||
fi
|
||||
local dirs=(
|
||||
"$SCRIPT_DIR/models"
|
||||
"$SCRIPT_DIR/../models"
|
||||
"/usr/share/rnnoise"
|
||||
"/usr/local/share/rnnoise"
|
||||
"/usr/share/ffmpeg/models"
|
||||
"$HOME/.local/share/rnnoise"
|
||||
)
|
||||
# Prefer '.rnnn' models (rnnoise-nu style) over legacy '.nn'
|
||||
local exts=("rnnn" "nn" "model")
|
||||
for d in "${dirs[@]}"; do
|
||||
if [[ -d $d ]]; then
|
||||
for ext in "${exts[@]}"; do
|
||||
# Pick the first matching model file
|
||||
for f in "$d"/*."$ext"; do
|
||||
if [[ -f $f ]]; then
|
||||
echo "$f"
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
done
|
||||
fi
|
||||
done
|
||||
return 1
|
||||
}
|
||||
|
||||
build_filters() {
|
||||
local filters=()
|
||||
# Remove low-frequency rumble typical for handheld/room noise
|
||||
filters+=("highpass=f=${HIGHPASS}")
|
||||
|
||||
# Denoise
|
||||
if $use_arnndn; then
|
||||
# arnndn with full mix keeps the model output; if no external model, rely on built-in
|
||||
filters+=("aresample=48000")
|
||||
filters+=("arnndn=m=${RN_MODEL}:mix=1.0")
|
||||
else
|
||||
# afftdn: FFT-based denoise, tune nf (noise floor) as needed
|
||||
if $REQUIRE_ML; then
|
||||
echo "Error: RNNoise required but not in use; aborting rather than falling back to afftdn. Use --allow-fallback to permit." >&2
|
||||
exit 11
|
||||
fi
|
||||
if $NO_ADVANCED; then
|
||||
filters+=("afftdn=nf=${AFFTDN_NF}")
|
||||
else
|
||||
if $afftdn_supports_md; then
|
||||
filters+=("afftdn=nf=${AFFTDN_NF}:md=${AFFTDN_MD}")
|
||||
else
|
||||
echo "Error: Your ffmpeg's afftdn filter does not support 'md='." >&2
|
||||
echo "Hint: Install/upgrade ffmpeg to a build that supports afftdn md or rerun with --no-advanced." >&2
|
||||
echo " On Debian/Ubuntu you may need a newer ffmpeg from a PPA or build from source." >&2
|
||||
exit 8
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
# Optional low-pass to shave hiss; keep disabled unless requested
|
||||
if [[ -n $LOWPASS ]]; then
|
||||
filters+=("lowpass=f=${LOWPASS}")
|
||||
fi
|
||||
|
||||
case "$PRESET" in
|
||||
asr)
|
||||
# ASR-friendly: avoid heavy gating/leveling, just prevent clipping
|
||||
filters+=("alimiter=limit=0.94")
|
||||
;;
|
||||
podcast)
|
||||
# Gentle dynamic normalization and broadcast-ish loudness (single-pass)
|
||||
# Note: single-pass loudnorm is approximate but OK for quick workflows
|
||||
filters+=("dynaudnorm=f=500:g=5:p=0.1")
|
||||
filters+=("loudnorm=i=-18:lra=9:tp=-2.0")
|
||||
;;
|
||||
aggressive)
|
||||
# Heavier clean-up; may harm ASR slightly but suppress background more
|
||||
filters+=("agate=threshold=0.012:ratio=2.5:release=200")
|
||||
filters+=("dynaudnorm=f=400:g=7:p=0.1")
|
||||
filters+=("loudnorm=i=-18:lra=9:tp=-2.0")
|
||||
;;
|
||||
*) ;;
|
||||
esac
|
||||
|
||||
# Resample and format at the end for ASR
|
||||
filters+=("aresample=16000")
|
||||
filters+=("aformat=channel_layouts=mono:sample_fmts=s16")
|
||||
|
||||
local IFS=","
|
||||
echo "${filters[*]}"
|
||||
}
|
||||
|
||||
make_out_path_for_file() {
|
||||
local in_file="$1"
|
||||
local base
|
||||
base=$(basename -- "$in_file")
|
||||
base="${base%.*}"
|
||||
local out_base="${base}${SUFFIX}.${OUT_EXT}"
|
||||
if [[ -n $OUT_DIR ]]; then
|
||||
mkdir -p -- "$OUT_DIR"
|
||||
echo "$OUT_DIR/$out_base"
|
||||
else
|
||||
local dir
|
||||
dir=$(dirname -- "$in_file")
|
||||
echo "$dir/$out_base"
|
||||
fi
|
||||
}
|
||||
|
||||
process_one() {
|
||||
local in_file="$1"
|
||||
local out_file
|
||||
out_file=$(make_out_path_for_file "$in_file")
|
||||
|
||||
# Choose codec based on extension
|
||||
local codec=(-c:a pcm_s16le)
|
||||
if [[ $OUT_EXT == "flac" ]]; then
|
||||
codec=(-c:a flac)
|
||||
fi
|
||||
|
||||
local af
|
||||
af=$(build_filters)
|
||||
|
||||
if [[ -f $out_file && $FORCE == false ]]; then
|
||||
echo "Skip (exists): $out_file"
|
||||
return 0
|
||||
fi
|
||||
|
||||
echo "Cleaning: $in_file -> $out_file"
|
||||
"$FFMPEG_BIN" "${FFMPEG_LOG[@]}" "${FFMPEG_OVERWRITE[@]}" -i "$in_file" -af "$af" "${codec[@]}" "$out_file"
|
||||
}
|
||||
|
||||
run_dir() {
|
||||
local dir="$1"
|
||||
# Common audio extensions (case-insensitive)
|
||||
mapfile -d '' files < <(find "$dir" -type f \
|
||||
\( -iname "*.wav" -o -iname "*.mp3" -o -iname "*.m4a" -o -iname "*.aac" -o -iname "*.flac" \
|
||||
-o -iname "*.ogg" -o -iname "*.opus" -o -iname "*.wma" -o -iname "*.webm" \) -print0)
|
||||
|
||||
if [[ ${#files[@]} -eq 0 ]]; then
|
||||
echo "No audio files found in: $dir"
|
||||
return 0
|
||||
fi
|
||||
|
||||
local running=0
|
||||
for f in "${files[@]}"; do
|
||||
if [[ $JOBS -le 1 || $supports_wait_n == false ]]; then
|
||||
process_one "$f"
|
||||
else
|
||||
process_one "$f" &
|
||||
((running++))
|
||||
if ((running >= JOBS)); then
|
||||
wait -n || true
|
||||
((running--))
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
# Wait for any remaining background jobs
|
||||
if ((JOBS > 1)) && $supports_wait_n; then
|
||||
wait || true
|
||||
fi
|
||||
}
|
||||
@ -30,7 +30,9 @@ def find_function_blocks(lines: list[str]) -> list[tuple[int, int, str]]:
|
||||
start, depth = index, 0
|
||||
while index < len(lines):
|
||||
depth += lines[index].count("{") - lines[index].count("}")
|
||||
if depth <= 0 and index > start:
|
||||
# A one-line `name() { ...; }` closes on its opening line, so
|
||||
# the depth check must accept index == start.
|
||||
if depth <= 0:
|
||||
break
|
||||
index += 1
|
||||
blocks.append((start, index, lines[start].split("(")[0]))
|
||||
|
||||
Loading…
Reference in New Issue
Block a user