From dadd33f82d0d8d78493b9b3e33f8f57d968679f8 Mon Sep 17 00:00:00 2001 From: Krzysztof kuhy Rudnicki Date: Sun, 16 Aug 2026 18:33:59 +0200 Subject: [PATCH] Split install_joplin.sh and establish the shell-split pattern 299 -> 103, with the --with-server half in lib/joplin_server.sh. First of the 84 linux_configuration files, done slowly to settle the pattern. Three things bash -n and shellcheck do not catch, all found here: - SCRIPT_DIR from "$0" breaks when the entry script is itself sourced. Use "${BASH_SOURCE[0]}"; keep readlink -f for the one symlinked script. - A lib needs a shebang AND +x: without the shebang shellcheck raises SC2148, with it the shebang-executable hook fails. Match common.sh. - Only a stubbed run proves the source line resolves. Copy entry+lib to a mktemp -d, sed `main` into a declare -F probe, run that. No installer step executes. Also enumerates which scripts escape the repo: three copied out by install_pacman_hooks.sh, one symlink, everything else generated. All recorded in docs/shell-split-recipes.md. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_011w2Pj4Ek3ZNMQVYcFHgXDY --- docs/shell-split-recipes.md | 65 +++++- .../linux-config-joplin-split-2026-08-16.json | 50 +++++ .../scripts/single_use/install_joplin.sh | 202 +---------------- .../scripts/single_use/lib/joplin_server.sh | 209 ++++++++++++++++++ 4 files changed, 323 insertions(+), 203 deletions(-) create mode 100644 docs/superpowers/evidence/linux-config-joplin-split-2026-08-16.json create mode 100755 linux_configuration/scripts/single_use/lib/joplin_server.sh diff --git a/docs/shell-split-recipes.md b/docs/shell-split-recipes.md index 38c685ff..8ac0f58e 100644 --- a/docs/shell-split-recipes.md +++ b/docs/shell-split-recipes.md @@ -9,10 +9,40 @@ coverage gate**, so none of the above transfers. Decision 6 governs: enforcement script that mutates the live system or the phone.** - After each split, `bash -n` and `shellcheck` the entry script _and_ every new - lib. Neither catches a lib that is never sourced. -- So also: for each new `lib/*.sh`, grep the entry script for the `source`/`.` - line naming it, and confirm the path resolves from where the script actually - runs (`${BASH_SOURCE[0]%/*}` or absolute — never a bare relative `./lib/`). + lib. **Neither catches a `source` line that resolves to the wrong path** — + both passed clean on a split whose lib was unreachable at runtime. +- So the real check is a **stubbed run**: copy the entry script and its libs to + a `mktemp -d`, `sed` the final `main` invocation into a `declare -F` probe, + and run it. That executes the `source` lines and nothing else. This caught + the bug below; no static check did. + +### Use `${BASH_SOURCE[0]}`, not `$0` + +The repo's existing convention is +`SCRIPT_DIR="$(dirname "$(readlink -f "$0")")"`, which works only while the +script is executed directly. `$0` is the **sourcing** script, so the moment an +entry script is itself sourced, `SCRIPT_DIR` points somewhere else and the lib +is not found. Write new ones as: + +```bash +SCRIPT_DIR="$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")" +``` + +`readlink -f` is still required — it is what makes the one symlinked script +(`/usr/local/bin/start-player2`) resolve back into the repo. + +### Shape of a new lib (two hooks disagree unless you get this right) + +Follow `scripts/lib/common.sh`: **shebang present, file executable.** The two +obvious-looking alternatives each fail a hook: + +- No shebang → `shellcheck` SC2148 ("target shell unknown"). +- Shebang but not executable → `check-shebang-scripts-are-executable`. + +So: `#!/bin/bash`, `chmod +x`, and **no `set -euo pipefail`** — a sourced lib +inherits the caller's strict mode, and redeclaring it is duplicated +boilerplate that feeds jscpd. Say so in the lib's header comment. + - **Two live units point at repo paths** and break if a file moves or is renamed. `systemctl cat` them after touching either: - `/etc/systemd/system/dns-blocklist-refresh.service` → @@ -23,6 +53,33 @@ enforcement script that mutates the live system or the phone.** Every other in-repo unit references an installed copy under `/usr/local/`, so those are only at risk if the _installer_ is what you split. +## Which scripts escape the repo (enumerated 2026-08-16) + +A split is only safe if the file still resolves its libs from wherever it +actually runs. Two ways a script leaves the repo, both silent when broken: + +- **Copied out** — only three, all under + `scripts/periodic_background/hosts/guard/pacman-hooks/`, installed to + `/usr/local/share/hosts-guard/` by `hosts/guard/install_pacman_hooks.sh`. + Splitting any of those three means the installer must copy the new libs too, + **in the same commit** — the same trap as `phone_focus_mode/deploy.sh`. +- **Symlinked in** — exactly one: `/usr/local/bin/start-player2` → + `scripts/gaming/start-player2.sh`. `${BASH_SOURCE[0]%/*}` resolves to + `/usr/local/bin` there, so a lib path must go through + `readlink -f "${BASH_SOURCE[0]}"` first. + +Everything else under `/usr/local/` is **generated** by an installer (heredoc +or `cat >`), not copied, so splitting the generator is safe as long as the +generated content is unchanged. Re-run both checks if that assumption is ever +in doubt: + +```bash +find /usr/local/bin /usr/local/sbin -maxdepth 1 \( -type l -o -name '*.sh' \) \ + -exec readlink -f {} \; | grep testsAndMisc +grep -rn -E '^\s*(sudo )?(install|cp)\s.*(SCRIPT_DIR|REPO).*/usr/local/' \ + linux_configuration --include='*.sh' +``` + - Prefer files that **nothing** references first — pure lint risk. - Watch **jscpd** on the first multi-lib commit: repeated `set -euo pipefail` + source-guard headers across several new libs is exactly what pushes diff --git a/docs/superpowers/evidence/linux-config-joplin-split-2026-08-16.json b/docs/superpowers/evidence/linux-config-joplin-split-2026-08-16.json new file mode 100644 index 00000000..86cd93c8 --- /dev/null +++ b/docs/superpowers/evidence/linux-config-joplin-split-2026-08-16.json @@ -0,0 +1,50 @@ +{ + "intent": "Bring linux_configuration/scripts/single_use/install_joplin.sh under the 250-line cap, and establish the verified shell-split pattern for the remaining 83 files in that directory.", + "scope": [ + "linux_configuration/scripts/single_use/install_joplin.sh (299 -> 103)", + "linux_configuration/scripts/single_use/lib/joplin_server.sh (new)", + "docs/shell-split-recipes.md (the pattern this split established)", + "Non-goal: no behaviour change; no installer step was executed" + ], + "changes": [ + "Moved setup_duckdns and setup_joplin_server, the whole --with-server half, into lib/joplin_server.sh, sourced by the entry script before main runs.", + "The lib carries a shebang and is executable, matching scripts/lib/common.sh, and deliberately omits set -euo pipefail: it is sourced, so it inherits the caller's strict mode, and repeating the boilerplate across many new libs is what feeds jscpd's duplication threshold.", + "SCRIPT_DIR uses ${BASH_SOURCE[0]} rather than the repo's usual $0, because $0 is the sourcing script and breaks the moment an entry script is itself sourced. readlink -f is kept, since that is what makes the one symlinked script in the repo resolve correctly.", + "Recorded in docs/shell-split-recipes.md: the enumeration of which scripts escape the repo (three copied to /usr/local/share/hosts-guard, one symlinked), the stubbed-run verification, and the lib shape that satisfies both shellcheck and the shebang-executable hook." + ], + "verification": [ + { + "command": "bash -n on the entry script and the lib; shellcheck on both", + "result": "pass", + "evidence": "Both clean. Recorded because these are NOT sufficient: they passed on the first version of this split, whose lib was unreachable at runtime." + }, + { + "command": "stubbed run: copy entry + lib to a mktemp -d, sed the final `main` invocation into a `declare -F` probe, run it", + "result": "fail then pass", + "evidence": "First run failed with 'lib/joplin_server.sh: No such file or directory' - SCRIPT_DIR was computed from $0. Fixed to ${BASH_SOURCE[0]}; the probe then reported 'all functions defined; lib sourced correctly'. This executes the source lines and nothing else, so no installer step ran." + }, + { + "command": "grep for the source line and resolve its path", + "result": "pass", + "evidence": "install_joplin.sh:74 sources $SCRIPT_DIR/lib/joplin_server.sh, which resolves to the file that exists." + }, + { + "command": "pre-commit run --files ", + "result": "fail then pass", + "evidence": "check-shebang-scripts-are-executable rejected the lib, then shellcheck SC2148 rejected removing the shebang. Resolved by matching scripts/lib/common.sh (shebang + chmod +x), which is now recorded in the recipes doc so the next 83 splits do not rediscover it." + }, + { + "command": "wc -l", + "result": "pass", + "evidence": "103 and 209 lines, both under the cap." + } + ], + "risks": [ + "install_joplin.sh is an installer that runs pacman/docker and writes to $HOME. It was never executed, per decision 6; the stubbed run only proves the functions are defined and the source line resolves. Whether the installer still works end-to-end is unverified and unverifiable without running it.", + "The lib assumes the caller has defined info/warn/error and the JOPLIN_*/DUCKDNS_* variables. That holds because it is sourced after them, but it is an ordering dependency the file cannot enforce." + ], + "rollback": [ + "git revert the commit; install_joplin.sh returns to 299 lines and the lib disappears.", + "After rollback, re-run bash -n and shellcheck on the restored script." + ] +} diff --git a/linux_configuration/scripts/single_use/install_joplin.sh b/linux_configuration/scripts/single_use/install_joplin.sh index 73db5650..d01fe463 100755 --- a/linux_configuration/scripts/single_use/install_joplin.sh +++ b/linux_configuration/scripts/single_use/install_joplin.sh @@ -69,205 +69,9 @@ install_joplin_desktop() { info "Launch with: ~/.joplin/Joplin.AppImage (or 'joplin-desktop' from menu)" } -# ── Set up DuckDNS for stable URL ─────────────────────────────────── -setup_duckdns() { - info "Setting up DuckDNS for stable server URL..." - - if [[ -z "$DUCKDNS_DOMAIN" ]]; then - echo "" - info "Your public IP may change. DuckDNS provides a free stable hostname." - info "1. Go to https://www.duckdns.org/ and sign in (Google/GitHub/etc.)" - info "2. Create a subdomain (e.g. 'myjoplin' for myjoplin.duckdns.org)" - info "3. Copy your token from the DuckDNS dashboard" - echo "" - read -r -p "Enter your DuckDNS subdomain (without .duckdns.org): " DUCKDNS_DOMAIN - read -r -p "Enter your DuckDNS token: " DUCKDNS_TOKEN - fi - - if [[ -z "$DUCKDNS_DOMAIN" ]] || [[ -z "$DUCKDNS_TOKEN" ]]; then - warn "DuckDNS not configured. Falling back to raw public IP (may change!)." - return 1 - fi - - local full_domain="${DUCKDNS_DOMAIN}.duckdns.org" - - # Update DuckDNS now - info "Updating DuckDNS record for ${full_domain}..." - local result - result=$(curl -s "https://www.duckdns.org/update?domains=${DUCKDNS_DOMAIN}&token=${DUCKDNS_TOKEN}&ip=") - if [[ "$result" == "OK" ]]; then - info "DuckDNS updated successfully: ${full_domain}" - else - warn "DuckDNS update returned: $result" - fi - - # Set up cron job to keep IP updated every 5 minutes - local duckdns_script="$JOPLIN_DATA_DIR/duckdns-update.sh" - cat >"$duckdns_script" <> "$JOPLIN_DATA_DIR/duckdns.log" -DUCKEOF - chmod +x "$duckdns_script" - - # Add cron job (remove old one if exists, add new) - ( - crontab -l 2>/dev/null | grep -v "duckdns-update.sh" - echo "*/5 * * * * $duckdns_script" - ) | crontab - - info "DuckDNS cron job installed (updates every 5 minutes)" - - # Save config for future runs - local config_file="$JOPLIN_DATA_DIR/.duckdns.conf" - cat >"$config_file" </dev/null 2>&1; then - info "Installing Docker..." - sudo pacman -S --needed --noconfirm docker docker-compose - fi - - if ! systemctl is-active --quiet docker; then - info "Starting Docker service..." - sudo systemctl enable --now docker - fi - - # Add user to docker group if not already a member - if ! groups | grep -q '\bdocker\b'; then - warn "Adding $USER to docker group (re-login required for group to take effect)." - sudo usermod -aG docker "$USER" - fi - - # Create data directory - mkdir -p "$JOPLIN_DATA_DIR" - - local compose_file="$JOPLIN_DATA_DIR/docker-compose.yml" - - # Load saved DuckDNS config if it exists - if [[ -f "$JOPLIN_DATA_DIR/.duckdns.conf" ]]; then - # shellcheck source=/dev/null - source "$JOPLIN_DATA_DIR/.duckdns.conf" - fi - - # Set up DuckDNS for a stable hostname - local server_url - if setup_duckdns; then - server_url="http://${DUCKDNS_DOMAIN}.duckdns.org:${JOPLIN_SERVER_PORT}" - info "Using stable DuckDNS URL: $server_url" - else - # Fallback to public IP - local host_ip - host_ip="$(curl -s --max-time 5 ifconfig.me 2>/dev/null)" - if [[ -z "$host_ip" ]]; then - host_ip="$(ip -4 route get 1.1.1.1 2>/dev/null | awk '{print $7; exit}')" - fi - if [[ -z "$host_ip" ]]; then - host_ip="$(hostname -I 2>/dev/null | awk '{print $1}')" - fi - if [[ -z "$host_ip" ]]; then - warn "Could not detect external IP. Falling back to 0.0.0.0" - host_ip="0.0.0.0" - fi - server_url="http://${host_ip}:${JOPLIN_SERVER_PORT}" - warn "Using raw IP URL (may change!): $server_url" - fi - - cat >"$compose_file" </dev/null 2>&1; then - sudo ufw allow "${JOPLIN_SERVER_PORT}/tcp" || warn "Could not configure ufw" - elif command -v firewall-cmd >/dev/null 2>&1; then - if sudo firewall-cmd --permanent --add-port="${JOPLIN_SERVER_PORT}/tcp" && - sudo firewall-cmd --reload; then - : - else - warn "Could not configure firewalld" - fi - elif command -v iptables >/dev/null 2>&1; then - sudo iptables -A INPUT -p tcp --dport "${JOPLIN_SERVER_PORT}" -j ACCEPT || warn "Could not configure iptables" - else - warn "No firewall tool found. Ensure port ${JOPLIN_SERVER_PORT}/tcp is open manually." - fi - echo "" - echo " To connect Joplin desktop/Android to this server:" - echo " 1. Open Joplin → Tools → Options → Synchronisation" - echo " 2. Set target to 'Joplin Server'" - echo " 3. Enter URL: ${server_url}" - echo " 4. Enter your Joplin Server email and password" - echo "" - echo " Server management:" - echo " Start: docker compose -f $compose_file up -d" - echo " Stop: docker compose -f $compose_file down" - echo " Logs: docker compose -f $compose_file logs -f" - echo " Update: docker compose -f $compose_file pull && docker compose -f $compose_file up -d" -} +SCRIPT_DIR="$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")" +# shellcheck source=lib/joplin_server.sh +source "$SCRIPT_DIR/lib/joplin_server.sh" # ── Main ───────────────────────────────────────────────────────────── main() { diff --git a/linux_configuration/scripts/single_use/lib/joplin_server.sh b/linux_configuration/scripts/single_use/lib/joplin_server.sh new file mode 100755 index 00000000..7e14e482 --- /dev/null +++ b/linux_configuration/scripts/single_use/lib/joplin_server.sh @@ -0,0 +1,209 @@ +#!/bin/bash +# Joplin Server setup: DuckDNS dynamic DNS plus the Docker compose stack. +# +# Sourced by install_joplin.sh when it is run with --with-server; split out to +# keep that script under the 250-line cap. Expects the caller to have defined +# info/warn/error and the JOPLIN_* and DUCKDNS_* variables. Sourced, not run, +# so it inherits the caller's strict mode; the shebang is for shellcheck and +# matches scripts/lib/common.sh. + + +# ── Set up DuckDNS for stable URL ─────────────────────────────────── +setup_duckdns() { + info "Setting up DuckDNS for stable server URL..." + + if [[ -z "$DUCKDNS_DOMAIN" ]]; then + echo "" + info "Your public IP may change. DuckDNS provides a free stable hostname." + info "1. Go to https://www.duckdns.org/ and sign in (Google/GitHub/etc.)" + info "2. Create a subdomain (e.g. 'myjoplin' for myjoplin.duckdns.org)" + info "3. Copy your token from the DuckDNS dashboard" + echo "" + read -r -p "Enter your DuckDNS subdomain (without .duckdns.org): " DUCKDNS_DOMAIN + read -r -p "Enter your DuckDNS token: " DUCKDNS_TOKEN + fi + + if [[ -z "$DUCKDNS_DOMAIN" ]] || [[ -z "$DUCKDNS_TOKEN" ]]; then + warn "DuckDNS not configured. Falling back to raw public IP (may change!)." + return 1 + fi + + local full_domain="${DUCKDNS_DOMAIN}.duckdns.org" + + # Update DuckDNS now + info "Updating DuckDNS record for ${full_domain}..." + local result + result=$(curl -s "https://www.duckdns.org/update?domains=${DUCKDNS_DOMAIN}&token=${DUCKDNS_TOKEN}&ip=") + if [[ "$result" == "OK" ]]; then + info "DuckDNS updated successfully: ${full_domain}" + else + warn "DuckDNS update returned: $result" + fi + + # Set up cron job to keep IP updated every 5 minutes + local duckdns_script="$JOPLIN_DATA_DIR/duckdns-update.sh" + cat >"$duckdns_script" <> "$JOPLIN_DATA_DIR/duckdns.log" +DUCKEOF + chmod +x "$duckdns_script" + + # Add cron job (remove old one if exists, add new) + ( + crontab -l 2>/dev/null | grep -v "duckdns-update.sh" + echo "*/5 * * * * $duckdns_script" + ) | crontab - + info "DuckDNS cron job installed (updates every 5 minutes)" + + # Save config for future runs + local config_file="$JOPLIN_DATA_DIR/.duckdns.conf" + cat >"$config_file" </dev/null 2>&1; then + info "Installing Docker..." + sudo pacman -S --needed --noconfirm docker docker-compose + fi + + if ! systemctl is-active --quiet docker; then + info "Starting Docker service..." + sudo systemctl enable --now docker + fi + + # Add user to docker group if not already a member + if ! groups | grep -q '\bdocker\b'; then + warn "Adding $USER to docker group (re-login required for group to take effect)." + sudo usermod -aG docker "$USER" + fi + + # Create data directory + mkdir -p "$JOPLIN_DATA_DIR" + + local compose_file="$JOPLIN_DATA_DIR/docker-compose.yml" + + # Load saved DuckDNS config if it exists + if [[ -f "$JOPLIN_DATA_DIR/.duckdns.conf" ]]; then + # shellcheck source=/dev/null + source "$JOPLIN_DATA_DIR/.duckdns.conf" + fi + + # Set up DuckDNS for a stable hostname + local server_url + if setup_duckdns; then + server_url="http://${DUCKDNS_DOMAIN}.duckdns.org:${JOPLIN_SERVER_PORT}" + info "Using stable DuckDNS URL: $server_url" + else + # Fallback to public IP + local host_ip + host_ip="$(curl -s --max-time 5 ifconfig.me 2>/dev/null)" + if [[ -z "$host_ip" ]]; then + host_ip="$(ip -4 route get 1.1.1.1 2>/dev/null | awk '{print $7; exit}')" + fi + if [[ -z "$host_ip" ]]; then + host_ip="$(hostname -I 2>/dev/null | awk '{print $1}')" + fi + if [[ -z "$host_ip" ]]; then + warn "Could not detect external IP. Falling back to 0.0.0.0" + host_ip="0.0.0.0" + fi + server_url="http://${host_ip}:${JOPLIN_SERVER_PORT}" + warn "Using raw IP URL (may change!): $server_url" + fi + + cat >"$compose_file" </dev/null 2>&1; then + sudo ufw allow "${JOPLIN_SERVER_PORT}/tcp" || warn "Could not configure ufw" + elif command -v firewall-cmd >/dev/null 2>&1; then + if sudo firewall-cmd --permanent --add-port="${JOPLIN_SERVER_PORT}/tcp" && + sudo firewall-cmd --reload; then + : + else + warn "Could not configure firewalld" + fi + elif command -v iptables >/dev/null 2>&1; then + sudo iptables -A INPUT -p tcp --dport "${JOPLIN_SERVER_PORT}" -j ACCEPT || warn "Could not configure iptables" + else + warn "No firewall tool found. Ensure port ${JOPLIN_SERVER_PORT}/tcp is open manually." + fi + echo "" + echo " To connect Joplin desktop/Android to this server:" + echo " 1. Open Joplin → Tools → Options → Synchronisation" + echo " 2. Set target to 'Joplin Server'" + echo " 3. Enter URL: ${server_url}" + echo " 4. Enter your Joplin Server email and password" + echo "" + echo " Server management:" + echo " Start: docker compose -f $compose_file up -d" + echo " Stop: docker compose -f $compose_file down" + echo " Logs: docker compose -f $compose_file logs -f" + echo " Update: docker compose -f $compose_file pull && docker compose -f $compose_file up -d" +}