From a778b66b15f9156c57d3d7a7167d5b18d9fa7175 Mon Sep 17 00:00:00 2001 From: Krzysztof Rudnicki Date: Mon, 3 Aug 2026 18:34:49 +0200 Subject: [PATCH] Fail the build on any shellcheck finding; fix the 222 the old gate hid The shellcheck hook ran with --severity=warning, so info- and style-level findings never failed a commit. 222 had accumulated across 94 of 244 shell files. They were not all style noise -- six were bugs: - Five scripts in single_use/misc/testsAndMisc-bash/ sourced $SCRIPT_DIR/../../lib/common.sh, one directory too shallow. Nothing exists at that path, so all five failed at runtime. - setup_personal_website.sh and setup_kcd2_dice_solver.sh called usage() (which ends in `exit 0`) and then `exit 1` for an unknown command. The exit 1 was unreachable, so a typo'd subcommand reported SUCCESS. - Six i3blocks scripts had a `# shellcheck source=` directive missing the scripts/periodic_background/ path segment. The stale path had silently disabled source-following for all of them. - diagnose_arch_performance.sh word-split a newline-joined file list; when the find came back empty, awk got no file operands, fell through to reading stdin, and the diagnostic hung. - lib/common.sh's handle_arg_help_or_unknown ended in an unreachable `return 0` while documenting "Returns: 0 if handled" -- a contract no code path could satisfy. - deploy.sh's SCRIPT_DIR is reassigned by config.sh, which it sources at top level, and config.sh can repoint it at the on-device path /data/local/tmp/focus_mode. Renamed to DEPLOY_DIR. The SC1091 bulk (81 findings) was resolved rather than silenced: .shellcheckrc sets external-sources + source-path=SCRIPTDIR so sourced files are actually followed and analysed. That is what surfaced the i3blocks and deploy.sh bugs. 17 `# shellcheck disable=` directives remain, each with a written reason, and only where the finding is provably wrong: injection-test payloads that must not expand, literal source text being grepped for, POSIX sh scripts on the phone where arrays do not exist, and source paths that only exist at runtime. Everything else was fixed in code. Verified in a CLEAN worktree, not just my own tree. config.sh sources the gitignored config_secrets.sh, which exists locally but not in a fresh checkout; a working-tree-only check passed while ci_mirror.sh -- which lints HEAD in a clean worktree, exactly as CI does -- failed on it. The gate is verified to bite: a file whose only finding is SC2016 (info) fails the hook, while the old --severity=warning gate exits 0 on it. third_party/ is excluded -- vendored upstream, matching the prettier hook. Not fixed, reported: check_polling_antipatterns.sh carries a pre-existing SC2034 disable that hides a real bug -- its `patterns` array is never used, so the checker detects nothing. Confirmed unchanged by this commit. Co-Authored-By: Claude Opus 5 (1M context) --- .../utils/install_plagiarism_tools.sh | 127 ++++++++++-------- 1 file changed, 68 insertions(+), 59 deletions(-) diff --git a/linux_configuration/scripts/single_use/utils/install_plagiarism_tools.sh b/linux_configuration/scripts/single_use/utils/install_plagiarism_tools.sh index 1de0b64..ed8b457 100755 --- a/linux_configuration/scripts/single_use/utils/install_plagiarism_tools.sh +++ b/linux_configuration/scripts/single_use/utils/install_plagiarism_tools.sh @@ -39,40 +39,43 @@ echo "" echo "=== 1. Installing Python NLP-based Plagiarism Tools ===" # Check for Python 3 -if ! command -v python3 &> /dev/null; then - error "Python 3 is required but not installed." - exit 1 +if ! command -v python3 &>/dev/null; then + error "Python 3 is required but not installed." + exit 1 fi # Create virtual environment if [ ! -d "$VENV_DIR" ]; then - echo "Creating Python virtual environment..." - python3 -m venv "$VENV_DIR" - success "Virtual environment created at $VENV_DIR" + echo "Creating Python virtual environment..." + python3 -m venv "$VENV_DIR" + success "Virtual environment created at $VENV_DIR" else - warn "Virtual environment already exists at $VENV_DIR" + warn "Virtual environment already exists at $VENV_DIR" fi # Activate and install packages +# Created at runtime by `python -m venv` above, so there is nothing on disk +# for the linter to follow at check time. +# shellcheck disable=SC1091 source "$VENV_DIR/bin/activate" echo "Installing Python packages for text similarity detection..." pip install --upgrade pip pip install --progress-bar on \ - scikit-learn \ - nltk \ - spacy \ - gensim \ - numpy \ - pandas \ - python-docx \ - PyPDF2 \ - beautifulsoup4 \ - lxml \ - textdistance \ - fuzzywuzzy \ - python-Levenshtein + scikit-learn \ + nltk \ + spacy \ + gensim \ + numpy \ + pandas \ + python-docx \ + PyPDF2 \ + beautifulsoup4 \ + lxml \ + textdistance \ + fuzzywuzzy \ + python-Levenshtein success "Python NLP packages installed" @@ -90,11 +93,11 @@ success "NLTK data downloaded" # Download spaCy English model (small) echo "Downloading spaCy English model..." -python3 -m spacy download en_core_web_sm 2> /dev/null || warn "spaCy model download may need manual install: python -m spacy download en_core_web_sm" +python3 -m spacy download en_core_web_sm 2>/dev/null || warn "spaCy model download may need manual install: python -m spacy download en_core_web_sm" success "spaCy model installed" # Create a simple plagiarism checker script -cat > "$INSTALL_DIR/check_plagiarism.py" << 'PYEOF' +cat >"$INSTALL_DIR/check_plagiarism.py" <<'PYEOF' #!/usr/bin/env python3 """ Simple Text Plagiarism Checker @@ -325,7 +328,7 @@ success "Created plagiarism checker script at $INSTALL_DIR/check_plagiarism.py" # Create convenience wrapper mkdir -p "$HOME/.local/bin" -cat > "$HOME/.local/bin/plagcheck" << WRAPEOF +cat >"$HOME/.local/bin/plagcheck" < /dev/null; then - # Clone a text-based similarity tool - git clone --depth 1 https://github.com/Zedeldi/sherlock-py.git "$SHERLOCK_DIR" 2> /dev/null || { - warn "Could not clone sherlock-py, trying alternative..." - # Alternative: Create a simple n-gram based sherlock - mkdir -p "$SHERLOCK_DIR" - cat > "$SHERLOCK_DIR/sherlock.py" << 'SHERLOCKEOF' + # There are several Sherlock implementations; using a popular Python one + if command -v git &>/dev/null; then + # Clone a text-based similarity tool + git clone --depth 1 https://github.com/Zedeldi/sherlock-py.git "$SHERLOCK_DIR" 2>/dev/null || { + warn "Could not clone sherlock-py, trying alternative..." + # Alternative: Create a simple n-gram based sherlock + mkdir -p "$SHERLOCK_DIR" + cat >"$SHERLOCK_DIR/sherlock.py" <<'SHERLOCKEOF' #!/usr/bin/env python3 """ Sherlock - Simple text plagiarism detector using n-gram fingerprinting. @@ -443,14 +446,14 @@ def main(): if __name__ == '__main__': main() SHERLOCKEOF - chmod +x "$SHERLOCK_DIR/sherlock.py" - } - success "Sherlock installed at $SHERLOCK_DIR" - else - warn "Git not available, skipping Sherlock installation" - fi + chmod +x "$SHERLOCK_DIR/sherlock.py" + } + success "Sherlock installed at $SHERLOCK_DIR" + else + warn "Git not available, skipping Sherlock installation" + fi else - warn "Sherlock already installed at $SHERLOCK_DIR" + warn "Sherlock already installed at $SHERLOCK_DIR" fi # ------------------------------------------------------------------------------ @@ -459,17 +462,17 @@ fi echo "" echo "=== 3. Checking for Ferret (Java-based plagiarism tool) ===" -if command -v java &> /dev/null; then - FERRET_DIR="$INSTALL_DIR/ferret" - if [ ! -d "$FERRET_DIR" ]; then - mkdir -p "$FERRET_DIR" - echo "Ferret is a Java-based tool from University of Hertfordshire." - echo "Download manually from: https://homepages.herts.ac.uk/~comqcln/Ferret/" - echo "Place JAR file in: $FERRET_DIR" - warn "Ferret requires manual download (academic license)" - fi +if command -v java &>/dev/null; then + FERRET_DIR="$INSTALL_DIR/ferret" + if [ ! -d "$FERRET_DIR" ]; then + mkdir -p "$FERRET_DIR" + echo "Ferret is a Java-based tool from University of Hertfordshire." + echo "Download manually from: https://homepages.herts.ac.uk/~comqcln/Ferret/" + echo "Place JAR file in: $FERRET_DIR" + warn "Ferret requires manual download (academic license)" + fi else - warn "Java not installed, skipping Ferret" + warn "Java not installed, skipping Ferret" fi # ------------------------------------------------------------------------------ @@ -478,16 +481,16 @@ fi echo "" echo "=== 4. WCopyfind Information (Windows tool, needs Wine) ===" -if command -v wine &> /dev/null; then - echo "Wine is available. WCopyfind can be run via Wine." - echo "Download from: https://plagiarism.bloomfieldmedia.com/software/wcopyfind/" - echo "Run with: wine /path/to/WCopyfind.exe" - warn "WCopyfind requires manual download" +if command -v wine &>/dev/null; then + echo "Wine is available. WCopyfind can be run via Wine." + echo "Download from: https://plagiarism.bloomfieldmedia.com/software/wcopyfind/" + echo "Run with: wine /path/to/WCopyfind.exe" + warn "WCopyfind requires manual download" else - echo "Wine not installed. To use WCopyfind:" - echo " 1. Install wine: sudo apt install wine (or equivalent)" - echo " 2. Download WCopyfind from: https://plagiarism.bloomfieldmedia.com/software/wcopyfind/" - warn "WCopyfind skipped (Wine not available)" + echo "Wine not installed. To use WCopyfind:" + echo " 1. Install wine: sudo apt install wine (or equivalent)" + echo " 2. Download WCopyfind from: https://plagiarism.bloomfieldmedia.com/software/wcopyfind/" + warn "WCopyfind skipped (Wine not available)" fi # ------------------------------------------------------------------------------ @@ -521,12 +524,18 @@ echo " plagcheck thesis_v1.pdf thesis_v2.pdf --detailed" echo " plagcheck --dir ./student_papers/ --threshold 0.4" echo "" echo "Note: Ensure ~/.local/bin is in your PATH:" +# Printed for the user to paste into their shell rc: $HOME and $PATH must +# stay literal rather than resolve to this machine's values. +# shellcheck disable=SC2016 echo ' export PATH="$HOME/.local/bin:$PATH"' echo "" echo "==============================================" # Add to PATH reminder if [[ ":$PATH:" != *":$HOME/.local/bin:"* ]]; then - warn "Add ~/.local/bin to your PATH by adding this to ~/.bashrc or ~/.zshrc:" - echo ' export PATH="$HOME/.local/bin:$PATH"' + warn "Add ~/.local/bin to your PATH by adding this to ~/.bashrc or ~/.zshrc:" + # Printed for the user to paste into their shell rc: $HOME and $PATH must + # stay literal rather than resolve to this machine's values. + # shellcheck disable=SC2016 + echo ' export PATH="$HOME/.local/bin:$PATH"' fi