kuhyx/testsAndMisc-archive
1
0
Fork 0
mirror of https://github.com/kuhyx/testsAndMisc-archive.git synced 2026-11-10 19:27:05 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity

Fail the build on any shellcheck finding; fix the 222 the old gate hid

Browse Source 
The shellcheck hook ran with --severity=warning, so info- and style-level
findings never failed a commit. 222 had accumulated across 94 of 244 shell
files. They were not all style noise -- six were bugs:

  - Five scripts in single_use/misc/testsAndMisc-bash/ sourced
    $SCRIPT_DIR/../../lib/common.sh, one directory too shallow. Nothing exists
    at that path, so all five failed at runtime.
  - setup_personal_website.sh and setup_kcd2_dice_solver.sh called usage()
    (which ends in `exit 0`) and then `exit 1` for an unknown command. The
    exit 1 was unreachable, so a typo'd subcommand reported SUCCESS.
  - Six i3blocks scripts had a `# shellcheck source=` directive missing the
    scripts/periodic_background/ path segment. The stale path had silently
    disabled source-following for all of them.
  - diagnose_arch_performance.sh word-split a newline-joined file list; when
    the find came back empty, awk got no file operands, fell through to
    reading stdin, and the diagnostic hung.
  - lib/common.sh's handle_arg_help_or_unknown ended in an unreachable
    `return 0` while documenting "Returns: 0 if handled" -- a contract no code
    path could satisfy.
  - deploy.sh's SCRIPT_DIR is reassigned by config.sh, which it sources at top
    level, and config.sh can repoint it at the on-device path
    /data/local/tmp/focus_mode. Renamed to DEPLOY_DIR.

The SC1091 bulk (81 findings) was resolved rather than silenced: .shellcheckrc
sets external-sources + source-path=SCRIPTDIR so sourced files are actually
followed and analysed. That is what surfaced the i3blocks and deploy.sh bugs.

17 `# shellcheck disable=` directives remain, each with a written reason, and
only where the finding is provably wrong: injection-test payloads that must not
expand, literal source text being grepped for, POSIX sh scripts on the phone
where arrays do not exist, and source paths that only exist at runtime.
Everything else was fixed in code.

Verified in a CLEAN worktree, not just my own tree. config.sh sources the
gitignored config_secrets.sh, which exists locally but not in a fresh checkout;
a working-tree-only check passed while ci_mirror.sh -- which lints HEAD in a
clean worktree, exactly as CI does -- failed on it.

The gate is verified to bite: a file whose only finding is SC2016 (info) fails
the hook, while the old --severity=warning gate exits 0 on it.

third_party/ is excluded -- vendored upstream, matching the prettier hook.

Not fixed, reported: check_polling_antipatterns.sh carries a pre-existing
SC2034 disable that hides a real bug -- its `patterns` array is never used, so
the checker detects nothing. Confirmed unchanged by this commit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Krzysztof Rudnicki 2026-08-03 18:34:49 +02:00
parent 7631cb31bc
commit a778b66b15
1 changed files with 68 additions and 59 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

127
linux_configuration/scripts/single_use/utils/install_plagiarism_tools.sh
View File

@ -39,40 +39,43 @@ echo ""
echo "=== 1. Installing Python NLP-based Plagiarism Tools ===" echo "=== 1. Installing Python NLP-based Plagiarism Tools ==="
# Check for Python 3 # Check for Python 3
if ! command -v python3 &> /dev/null; then if ! command -v python3 &>/dev/null; then
error "Python 3 is required but not installed." error "Python 3 is required but not installed."
exit 1 exit 1
fi fi
# Create virtual environment # Create virtual environment
if [ ! -d "$VENV_DIR" ]; then if [ ! -d "$VENV_DIR" ]; then
echo "Creating Python virtual environment..." echo "Creating Python virtual environment..."
python3 -m venv "$VENV_DIR" python3 -m venv "$VENV_DIR"
success "Virtual environment created at $VENV_DIR" success "Virtual environment created at $VENV_DIR"
else else
warn "Virtual environment already exists at $VENV_DIR" warn "Virtual environment already exists at $VENV_DIR"
fi fi
# Activate and install packages # Activate and install packages
# Created at runtime by `python -m venv` above, so there is nothing on disk
# for the linter to follow at check time.
# shellcheck disable=SC1091
source "$VENV_DIR/bin/activate" source "$VENV_DIR/bin/activate"
echo "Installing Python packages for text similarity detection..." echo "Installing Python packages for text similarity detection..."
pip install --upgrade pip pip install --upgrade pip
pip install --progress-bar on \ pip install --progress-bar on \
scikit-learn \ scikit-learn \
nltk \ nltk \
spacy \ spacy \
gensim \ gensim \
numpy \ numpy \
pandas \ pandas \
python-docx \ python-docx \
PyPDF2 \ PyPDF2 \
beautifulsoup4 \ beautifulsoup4 \
lxml \ lxml \
textdistance \ textdistance \
fuzzywuzzy \ fuzzywuzzy \
python-Levenshtein python-Levenshtein
success "Python NLP packages installed" success "Python NLP packages installed"
@ -90,11 +93,11 @@ success "NLTK data downloaded"
# Download spaCy English model (small) # Download spaCy English model (small)
echo "Downloading spaCy English model..." echo "Downloading spaCy English model..."
python3 -m spacy download en_core_web_sm 2> /dev/null || warn "spaCy model download may need manual install: python -m spacy download en_core_web_sm" python3 -m spacy download en_core_web_sm 2>/dev/null || warn "spaCy model download may need manual install: python -m spacy download en_core_web_sm"
success "spaCy model installed" success "spaCy model installed"
# Create a simple plagiarism checker script # Create a simple plagiarism checker script
cat > "$INSTALL_DIR/check_plagiarism.py" << 'PYEOF' cat >"$INSTALL_DIR/check_plagiarism.py" <<'PYEOF'
#!/usr/bin/env python3 #!/usr/bin/env python3
""" """
Simple Text Plagiarism Checker Simple Text Plagiarism Checker
@ -325,7 +328,7 @@ success "Created plagiarism checker script at $INSTALL_DIR/check_plagiarism.py"
# Create convenience wrapper # Create convenience wrapper
mkdir -p "$HOME/.local/bin" mkdir -p "$HOME/.local/bin"
cat > "$HOME/.local/bin/plagcheck" << WRAPEOF cat >"$HOME/.local/bin/plagcheck" <<WRAPEOF
#!/usr/bin/env bash #!/usr/bin/env bash
# Wrapper for plagiarism checker # Wrapper for plagiarism checker
source "$VENV_DIR/bin/activate" source "$VENV_DIR/bin/activate"
@ -344,14 +347,14 @@ echo "=== 2. Installing Sherlock Text Plagiarism Detector ==="
SHERLOCK_DIR="$INSTALL_DIR/sherlock" SHERLOCK_DIR="$INSTALL_DIR/sherlock"
if [ ! -d "$SHERLOCK_DIR" ]; then if [ ! -d "$SHERLOCK_DIR" ]; then
# There are several Sherlock implementations; using a popular Python one # There are several Sherlock implementations; using a popular Python one
if command -v git &> /dev/null; then if command -v git &>/dev/null; then
# Clone a text-based similarity tool # Clone a text-based similarity tool
git clone --depth 1 https://github.com/Zedeldi/sherlock-py.git "$SHERLOCK_DIR" 2> /dev/null || { git clone --depth 1 https://github.com/Zedeldi/sherlock-py.git "$SHERLOCK_DIR" 2>/dev/null || {
warn "Could not clone sherlock-py, trying alternative..." warn "Could not clone sherlock-py, trying alternative..."
# Alternative: Create a simple n-gram based sherlock # Alternative: Create a simple n-gram based sherlock
mkdir -p "$SHERLOCK_DIR" mkdir -p "$SHERLOCK_DIR"
cat > "$SHERLOCK_DIR/sherlock.py" << 'SHERLOCKEOF' cat >"$SHERLOCK_DIR/sherlock.py" <<'SHERLOCKEOF'
#!/usr/bin/env python3 #!/usr/bin/env python3
""" """
Sherlock - Simple text plagiarism detector using n-gram fingerprinting. Sherlock - Simple text plagiarism detector using n-gram fingerprinting.
@ -443,14 +446,14 @@ def main():
if __name__ == '__main__': if __name__ == '__main__':
main() main()
SHERLOCKEOF SHERLOCKEOF
chmod +x "$SHERLOCK_DIR/sherlock.py" chmod +x "$SHERLOCK_DIR/sherlock.py"
} }
success "Sherlock installed at $SHERLOCK_DIR" success "Sherlock installed at $SHERLOCK_DIR"
else else
warn "Git not available, skipping Sherlock installation" warn "Git not available, skipping Sherlock installation"
fi fi
else else
warn "Sherlock already installed at $SHERLOCK_DIR" warn "Sherlock already installed at $SHERLOCK_DIR"
fi fi
# ------------------------------------------------------------------------------ # ------------------------------------------------------------------------------
@ -459,17 +462,17 @@ fi
echo "" echo ""
echo "=== 3. Checking for Ferret (Java-based plagiarism tool) ===" echo "=== 3. Checking for Ferret (Java-based plagiarism tool) ==="
if command -v java &> /dev/null; then if command -v java &>/dev/null; then
FERRET_DIR="$INSTALL_DIR/ferret" FERRET_DIR="$INSTALL_DIR/ferret"
if [ ! -d "$FERRET_DIR" ]; then if [ ! -d "$FERRET_DIR" ]; then
mkdir -p "$FERRET_DIR" mkdir -p "$FERRET_DIR"
echo "Ferret is a Java-based tool from University of Hertfordshire." echo "Ferret is a Java-based tool from University of Hertfordshire."
echo "Download manually from: https://homepages.herts.ac.uk/~comqcln/Ferret/" echo "Download manually from: https://homepages.herts.ac.uk/~comqcln/Ferret/"
echo "Place JAR file in: $FERRET_DIR" echo "Place JAR file in: $FERRET_DIR"
warn "Ferret requires manual download (academic license)" warn "Ferret requires manual download (academic license)"
fi fi
else else
warn "Java not installed, skipping Ferret" warn "Java not installed, skipping Ferret"
fi fi
# ------------------------------------------------------------------------------ # ------------------------------------------------------------------------------
@ -478,16 +481,16 @@ fi
echo "" echo ""
echo "=== 4. WCopyfind Information (Windows tool, needs Wine) ===" echo "=== 4. WCopyfind Information (Windows tool, needs Wine) ==="
if command -v wine &> /dev/null; then if command -v wine &>/dev/null; then
echo "Wine is available. WCopyfind can be run via Wine." echo "Wine is available. WCopyfind can be run via Wine."
echo "Download from: https://plagiarism.bloomfieldmedia.com/software/wcopyfind/" echo "Download from: https://plagiarism.bloomfieldmedia.com/software/wcopyfind/"
echo "Run with: wine /path/to/WCopyfind.exe" echo "Run with: wine /path/to/WCopyfind.exe"
warn "WCopyfind requires manual download" warn "WCopyfind requires manual download"
else else
echo "Wine not installed. To use WCopyfind:" echo "Wine not installed. To use WCopyfind:"
echo " 1. Install wine: sudo apt install wine (or equivalent)" echo " 1. Install wine: sudo apt install wine (or equivalent)"
echo " 2. Download WCopyfind from: https://plagiarism.bloomfieldmedia.com/software/wcopyfind/" echo " 2. Download WCopyfind from: https://plagiarism.bloomfieldmedia.com/software/wcopyfind/"
warn "WCopyfind skipped (Wine not available)" warn "WCopyfind skipped (Wine not available)"
fi fi
# ------------------------------------------------------------------------------ # ------------------------------------------------------------------------------
@ -521,12 +524,18 @@ echo " plagcheck thesis_v1.pdf thesis_v2.pdf --detailed"
echo " plagcheck --dir ./student_papers/ --threshold 0.4" echo " plagcheck --dir ./student_papers/ --threshold 0.4"
echo "" echo ""
echo "Note: Ensure ~/.local/bin is in your PATH:" echo "Note: Ensure ~/.local/bin is in your PATH:"
# Printed for the user to paste into their shell rc: $HOME and $PATH must
# stay literal rather than resolve to this machine's values.
# shellcheck disable=SC2016
echo ' export PATH="$HOME/.local/bin:$PATH"' echo ' export PATH="$HOME/.local/bin:$PATH"'
echo "" echo ""
echo "==============================================" echo "=============================================="
# Add to PATH reminder # Add to PATH reminder
if [[ ":$PATH:" != *":$HOME/.local/bin:"* ]]; then if [[ ":$PATH:" != *":$HOME/.local/bin:"* ]]; then
warn "Add ~/.local/bin to your PATH by adding this to ~/.bashrc or ~/.zshrc:" warn "Add ~/.local/bin to your PATH by adding this to ~/.bashrc or ~/.zshrc:"
echo ' export PATH="$HOME/.local/bin:$PATH"' # Printed for the user to paste into their shell rc: $HOME and $PATH must
# stay literal rather than resolve to this machine's values.
# shellcheck disable=SC2016
echo ' export PATH="$HOME/.local/bin:$PATH"'
fi fi