Split install_joplin.sh and establish the shell-split pattern
Some checks are pending
Pre-commit checks / pre-commit (push) Waiting to run
Python tests / test (push) Waiting to run
Shell tests / shell-tests (push) Waiting to run
Shell tests / arch-tests (push) Waiting to run

299 -> 103, with the --with-server half in lib/joplin_server.sh. First
of the 84 linux_configuration files, done slowly to settle the pattern.

Three things bash -n and shellcheck do not catch, all found here:

- SCRIPT_DIR from "$0" breaks when the entry script is itself sourced.
  Use "${BASH_SOURCE[0]}"; keep readlink -f for the one symlinked script.
- A lib needs a shebang AND +x: without the shebang shellcheck raises
  SC2148, with it the shebang-executable hook fails. Match common.sh.
- Only a stubbed run proves the source line resolves. Copy entry+lib to
  a mktemp -d, sed `main` into a declare -F probe, run that. No
  installer step executes.

Also enumerates which scripts escape the repo: three copied out by
install_pacman_hooks.sh, one symlink, everything else generated. All
recorded in docs/shell-split-recipes.md.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011w2Pj4Ek3ZNMQVYcFHgXDY
This commit is contained in:
Krzysztof Rudnicki 2026-08-16 18:33:59 +02:00
parent de930f3277
commit dadd33f82d
4 changed files with 323 additions and 203 deletions

View File

@ -9,10 +9,40 @@ coverage gate**, so none of the above transfers. Decision 6 governs:
enforcement script that mutates the live system or the phone.**
- After each split, `bash -n` and `shellcheck` the entry script _and_ every new
lib. Neither catches a lib that is never sourced.
- So also: for each new `lib/*.sh`, grep the entry script for the `source`/`.`
line naming it, and confirm the path resolves from where the script actually
runs (`${BASH_SOURCE[0]%/*}` or absolute — never a bare relative `./lib/`).
lib. **Neither catches a `source` line that resolves to the wrong path** —
both passed clean on a split whose lib was unreachable at runtime.
- So the real check is a **stubbed run**: copy the entry script and its libs to
a `mktemp -d`, `sed` the final `main` invocation into a `declare -F` probe,
and run it. That executes the `source` lines and nothing else. This caught
the bug below; no static check did.
### Use `${BASH_SOURCE[0]}`, not `$0`
The repo's existing convention is
`SCRIPT_DIR="$(dirname "$(readlink -f "$0")")"`, which works only while the
script is executed directly. `$0` is the **sourcing** script, so the moment an
entry script is itself sourced, `SCRIPT_DIR` points somewhere else and the lib
is not found. Write new ones as:
```bash
SCRIPT_DIR="$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")"
```
`readlink -f` is still required — it is what makes the one symlinked script
(`/usr/local/bin/start-player2`) resolve back into the repo.
### Shape of a new lib (two hooks disagree unless you get this right)
Follow `scripts/lib/common.sh`: **shebang present, file executable.** The two
obvious-looking alternatives each fail a hook:
- No shebang → `shellcheck` SC2148 ("target shell unknown").
- Shebang but not executable → `check-shebang-scripts-are-executable`.
So: `#!/bin/bash`, `chmod +x`, and **no `set -euo pipefail`** — a sourced lib
inherits the caller's strict mode, and redeclaring it is duplicated
boilerplate that feeds jscpd. Say so in the lib's header comment.
- **Two live units point at repo paths** and break if a file moves or is
renamed. `systemctl cat` them after touching either:
- `/etc/systemd/system/dns-blocklist-refresh.service` →
@ -23,6 +53,33 @@ enforcement script that mutates the live system or the phone.**
Every other in-repo unit references an installed copy under `/usr/local/`,
so those are only at risk if the _installer_ is what you split.
## Which scripts escape the repo (enumerated 2026-08-16)
A split is only safe if the file still resolves its libs from wherever it
actually runs. Two ways a script leaves the repo, both silent when broken:
- **Copied out** — only three, all under
`scripts/periodic_background/hosts/guard/pacman-hooks/`, installed to
`/usr/local/share/hosts-guard/` by `hosts/guard/install_pacman_hooks.sh`.
Splitting any of those three means the installer must copy the new libs too,
**in the same commit** — the same trap as `phone_focus_mode/deploy.sh`.
- **Symlinked in** — exactly one: `/usr/local/bin/start-player2` →
`scripts/gaming/start-player2.sh`. `${BASH_SOURCE[0]%/*}` resolves to
`/usr/local/bin` there, so a lib path must go through
`readlink -f "${BASH_SOURCE[0]}"` first.
Everything else under `/usr/local/` is **generated** by an installer (heredoc
or `cat >`), not copied, so splitting the generator is safe as long as the
generated content is unchanged. Re-run both checks if that assumption is ever
in doubt:
```bash
find /usr/local/bin /usr/local/sbin -maxdepth 1 \( -type l -o -name '*.sh' \) \
-exec readlink -f {} \; | grep testsAndMisc
grep -rn -E '^\s*(sudo )?(install|cp)\s.*(SCRIPT_DIR|REPO).*/usr/local/' \
linux_configuration --include='*.sh'
```
- Prefer files that **nothing** references first — pure lint risk.
- Watch **jscpd** on the first multi-lib commit: repeated `set -euo pipefail` +
source-guard headers across several new libs is exactly what pushes

View File

@ -0,0 +1,50 @@
{
"intent": "Bring linux_configuration/scripts/single_use/install_joplin.sh under the 250-line cap, and establish the verified shell-split pattern for the remaining 83 files in that directory.",
"scope": [
"linux_configuration/scripts/single_use/install_joplin.sh (299 -> 103)",
"linux_configuration/scripts/single_use/lib/joplin_server.sh (new)",
"docs/shell-split-recipes.md (the pattern this split established)",
"Non-goal: no behaviour change; no installer step was executed"
],
"changes": [
"Moved setup_duckdns and setup_joplin_server, the whole --with-server half, into lib/joplin_server.sh, sourced by the entry script before main runs.",
"The lib carries a shebang and is executable, matching scripts/lib/common.sh, and deliberately omits set -euo pipefail: it is sourced, so it inherits the caller's strict mode, and repeating the boilerplate across many new libs is what feeds jscpd's duplication threshold.",
"SCRIPT_DIR uses ${BASH_SOURCE[0]} rather than the repo's usual $0, because $0 is the sourcing script and breaks the moment an entry script is itself sourced. readlink -f is kept, since that is what makes the one symlinked script in the repo resolve correctly.",
"Recorded in docs/shell-split-recipes.md: the enumeration of which scripts escape the repo (three copied to /usr/local/share/hosts-guard, one symlinked), the stubbed-run verification, and the lib shape that satisfies both shellcheck and the shebang-executable hook."
],
"verification": [
{
"command": "bash -n on the entry script and the lib; shellcheck on both",
"result": "pass",
"evidence": "Both clean. Recorded because these are NOT sufficient: they passed on the first version of this split, whose lib was unreachable at runtime."
},
{
"command": "stubbed run: copy entry + lib to a mktemp -d, sed the final `main` invocation into a `declare -F` probe, run it",
"result": "fail then pass",
"evidence": "First run failed with 'lib/joplin_server.sh: No such file or directory' - SCRIPT_DIR was computed from $0. Fixed to ${BASH_SOURCE[0]}; the probe then reported 'all functions defined; lib sourced correctly'. This executes the source lines and nothing else, so no installer step ran."
},
{
"command": "grep for the source line and resolve its path",
"result": "pass",
"evidence": "install_joplin.sh:74 sources $SCRIPT_DIR/lib/joplin_server.sh, which resolves to the file that exists."
},
{
"command": "pre-commit run --files <changed>",
"result": "fail then pass",
"evidence": "check-shebang-scripts-are-executable rejected the lib, then shellcheck SC2148 rejected removing the shebang. Resolved by matching scripts/lib/common.sh (shebang + chmod +x), which is now recorded in the recipes doc so the next 83 splits do not rediscover it."
},
{
"command": "wc -l",
"result": "pass",
"evidence": "103 and 209 lines, both under the cap."
}
],
"risks": [
"install_joplin.sh is an installer that runs pacman/docker and writes to $HOME. It was never executed, per decision 6; the stubbed run only proves the functions are defined and the source line resolves. Whether the installer still works end-to-end is unverified and unverifiable without running it.",
"The lib assumes the caller has defined info/warn/error and the JOPLIN_*/DUCKDNS_* variables. That holds because it is sourced after them, but it is an ordering dependency the file cannot enforce."
],
"rollback": [
"git revert the commit; install_joplin.sh returns to 299 lines and the lib disappears.",
"After rollback, re-run bash -n and shellcheck on the restored script."
]
}

View File

@ -69,205 +69,9 @@ install_joplin_desktop() {
info "Launch with: ~/.joplin/Joplin.AppImage (or 'joplin-desktop' from menu)"
}
# ── Set up DuckDNS for stable URL ───────────────────────────────────
setup_duckdns() {
info "Setting up DuckDNS for stable server URL..."
if [[ -z "$DUCKDNS_DOMAIN" ]]; then
echo ""
info "Your public IP may change. DuckDNS provides a free stable hostname."
info "1. Go to https://www.duckdns.org/ and sign in (Google/GitHub/etc.)"
info "2. Create a subdomain (e.g. 'myjoplin' for myjoplin.duckdns.org)"
info "3. Copy your token from the DuckDNS dashboard"
echo ""
read -r -p "Enter your DuckDNS subdomain (without .duckdns.org): " DUCKDNS_DOMAIN
read -r -p "Enter your DuckDNS token: " DUCKDNS_TOKEN
fi
if [[ -z "$DUCKDNS_DOMAIN" ]] || [[ -z "$DUCKDNS_TOKEN" ]]; then
warn "DuckDNS not configured. Falling back to raw public IP (may change!)."
return 1
fi
local full_domain="${DUCKDNS_DOMAIN}.duckdns.org"
# Update DuckDNS now
info "Updating DuckDNS record for ${full_domain}..."
local result
result=$(curl -s "https://www.duckdns.org/update?domains=${DUCKDNS_DOMAIN}&token=${DUCKDNS_TOKEN}&ip=")
if [[ "$result" == "OK" ]]; then
info "DuckDNS updated successfully: ${full_domain}"
else
warn "DuckDNS update returned: $result"
fi
# Set up cron job to keep IP updated every 5 minutes
local duckdns_script="$JOPLIN_DATA_DIR/duckdns-update.sh"
cat >"$duckdns_script" <<DUCKEOF
#!/usr/bin/env bash
result=\$(curl -s "https://www.duckdns.org/update?domains=${DUCKDNS_DOMAIN}&token=${DUCKDNS_TOKEN}&ip=")
echo "\$(date): \$result" >> "$JOPLIN_DATA_DIR/duckdns.log"
DUCKEOF
chmod +x "$duckdns_script"
# Add cron job (remove old one if exists, add new)
(
crontab -l 2>/dev/null | grep -v "duckdns-update.sh"
echo "*/5 * * * * $duckdns_script"
) | crontab -
info "DuckDNS cron job installed (updates every 5 minutes)"
# Save config for future runs
local config_file="$JOPLIN_DATA_DIR/.duckdns.conf"
cat >"$config_file" <<CONFEOF
DUCKDNS_DOMAIN="${DUCKDNS_DOMAIN}"
DUCKDNS_TOKEN="${DUCKDNS_TOKEN}"
CONFEOF
chmod 600 "$config_file"
return 0
}
# ── Set up Joplin Server (Docker) ───────────────────────────────────
setup_joplin_server() {
info "Setting up Joplin Server via Docker..."
# Ensure Docker is installed and running
if ! command -v docker >/dev/null 2>&1; then
info "Installing Docker..."
sudo pacman -S --needed --noconfirm docker docker-compose
fi
if ! systemctl is-active --quiet docker; then
info "Starting Docker service..."
sudo systemctl enable --now docker
fi
# Add user to docker group if not already a member
if ! groups | grep -q '\bdocker\b'; then
warn "Adding $USER to docker group (re-login required for group to take effect)."
sudo usermod -aG docker "$USER"
fi
# Create data directory
mkdir -p "$JOPLIN_DATA_DIR"
local compose_file="$JOPLIN_DATA_DIR/docker-compose.yml"
# Load saved DuckDNS config if it exists
if [[ -f "$JOPLIN_DATA_DIR/.duckdns.conf" ]]; then
# shellcheck source=/dev/null
source "$JOPLIN_DATA_DIR/.duckdns.conf"
fi
# Set up DuckDNS for a stable hostname
local server_url
if setup_duckdns; then
server_url="http://${DUCKDNS_DOMAIN}.duckdns.org:${JOPLIN_SERVER_PORT}"
info "Using stable DuckDNS URL: $server_url"
else
# Fallback to public IP
local host_ip
host_ip="$(curl -s --max-time 5 ifconfig.me 2>/dev/null)"
if [[ -z "$host_ip" ]]; then
host_ip="$(ip -4 route get 1.1.1.1 2>/dev/null | awk '{print $7; exit}')"
fi
if [[ -z "$host_ip" ]]; then
host_ip="$(hostname -I 2>/dev/null | awk '{print $1}')"
fi
if [[ -z "$host_ip" ]]; then
warn "Could not detect external IP. Falling back to 0.0.0.0"
host_ip="0.0.0.0"
fi
server_url="http://${host_ip}:${JOPLIN_SERVER_PORT}"
warn "Using raw IP URL (may change!): $server_url"
fi
cat >"$compose_file" <<EOF
version: "3"
services:
joplin-db:
image: postgres:16
container_name: joplin-db
restart: unless-stopped
volumes:
- joplin-db-data:/var/lib/postgresql/data
environment:
POSTGRES_DB: joplin
POSTGRES_USER: joplin
POSTGRES_PASSWORD: changeme
networks:
- joplin-net
joplin-server:
image: joplin/server:latest
container_name: joplin-server
restart: unless-stopped
depends_on:
- joplin-db
ports:
- "0.0.0.0:${JOPLIN_SERVER_PORT}:22300"
environment:
APP_PORT: 22300
APP_BASE_URL: "${server_url}"
DB_CLIENT: pg
POSTGRES_HOST: joplin-db
POSTGRES_PORT: 5432
POSTGRES_DATABASE: joplin
POSTGRES_USER: joplin
POSTGRES_PASSWORD: changeme
networks:
- joplin-net
volumes:
joplin-db-data:
networks:
joplin-net:
EOF
info "Starting Joplin Server..."
docker compose -f "$compose_file" up -d
echo ""
info "Joplin Server is running at: ${server_url}"
echo ""
echo " Default admin credentials:"
echo " Email: admin@localhost"
echo " Password: admin"
echo ""
warn "IMPORTANT: Change the default admin password and the database"
warn "password (POSTGRES_PASSWORD in $compose_file) immediately!"
echo ""
info "Firewall: opening port ${JOPLIN_SERVER_PORT}/tcp for external access..."
if command -v ufw >/dev/null 2>&1; then
sudo ufw allow "${JOPLIN_SERVER_PORT}/tcp" || warn "Could not configure ufw"
elif command -v firewall-cmd >/dev/null 2>&1; then
if sudo firewall-cmd --permanent --add-port="${JOPLIN_SERVER_PORT}/tcp" &&
sudo firewall-cmd --reload; then
:
else
warn "Could not configure firewalld"
fi
elif command -v iptables >/dev/null 2>&1; then
sudo iptables -A INPUT -p tcp --dport "${JOPLIN_SERVER_PORT}" -j ACCEPT || warn "Could not configure iptables"
else
warn "No firewall tool found. Ensure port ${JOPLIN_SERVER_PORT}/tcp is open manually."
fi
echo ""
echo " To connect Joplin desktop/Android to this server:"
echo " 1. Open Joplin → Tools → Options → Synchronisation"
echo " 2. Set target to 'Joplin Server'"
echo " 3. Enter URL: ${server_url}"
echo " 4. Enter your Joplin Server email and password"
echo ""
echo " Server management:"
echo " Start: docker compose -f $compose_file up -d"
echo " Stop: docker compose -f $compose_file down"
echo " Logs: docker compose -f $compose_file logs -f"
echo " Update: docker compose -f $compose_file pull && docker compose -f $compose_file up -d"
}
SCRIPT_DIR="$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")"
# shellcheck source=lib/joplin_server.sh
source "$SCRIPT_DIR/lib/joplin_server.sh"
# ── Main ─────────────────────────────────────────────────────────────
main() {

View File

@ -0,0 +1,209 @@
#!/bin/bash
# Joplin Server setup: DuckDNS dynamic DNS plus the Docker compose stack.
#
# Sourced by install_joplin.sh when it is run with --with-server; split out to
# keep that script under the 250-line cap. Expects the caller to have defined
# info/warn/error and the JOPLIN_* and DUCKDNS_* variables. Sourced, not run,
# so it inherits the caller's strict mode; the shebang is for shellcheck and
# matches scripts/lib/common.sh.
# ── Set up DuckDNS for stable URL ───────────────────────────────────
setup_duckdns() {
info "Setting up DuckDNS for stable server URL..."
if [[ -z "$DUCKDNS_DOMAIN" ]]; then
echo ""
info "Your public IP may change. DuckDNS provides a free stable hostname."
info "1. Go to https://www.duckdns.org/ and sign in (Google/GitHub/etc.)"
info "2. Create a subdomain (e.g. 'myjoplin' for myjoplin.duckdns.org)"
info "3. Copy your token from the DuckDNS dashboard"
echo ""
read -r -p "Enter your DuckDNS subdomain (without .duckdns.org): " DUCKDNS_DOMAIN
read -r -p "Enter your DuckDNS token: " DUCKDNS_TOKEN
fi
if [[ -z "$DUCKDNS_DOMAIN" ]] || [[ -z "$DUCKDNS_TOKEN" ]]; then
warn "DuckDNS not configured. Falling back to raw public IP (may change!)."
return 1
fi
local full_domain="${DUCKDNS_DOMAIN}.duckdns.org"
# Update DuckDNS now
info "Updating DuckDNS record for ${full_domain}..."
local result
result=$(curl -s "https://www.duckdns.org/update?domains=${DUCKDNS_DOMAIN}&token=${DUCKDNS_TOKEN}&ip=")
if [[ "$result" == "OK" ]]; then
info "DuckDNS updated successfully: ${full_domain}"
else
warn "DuckDNS update returned: $result"
fi
# Set up cron job to keep IP updated every 5 minutes
local duckdns_script="$JOPLIN_DATA_DIR/duckdns-update.sh"
cat >"$duckdns_script" <<DUCKEOF
#!/usr/bin/env bash
result=\$(curl -s "https://www.duckdns.org/update?domains=${DUCKDNS_DOMAIN}&token=${DUCKDNS_TOKEN}&ip=")
echo "\$(date): \$result" >> "$JOPLIN_DATA_DIR/duckdns.log"
DUCKEOF
chmod +x "$duckdns_script"
# Add cron job (remove old one if exists, add new)
(
crontab -l 2>/dev/null | grep -v "duckdns-update.sh"
echo "*/5 * * * * $duckdns_script"
) | crontab -
info "DuckDNS cron job installed (updates every 5 minutes)"
# Save config for future runs
local config_file="$JOPLIN_DATA_DIR/.duckdns.conf"
cat >"$config_file" <<CONFEOF
DUCKDNS_DOMAIN="${DUCKDNS_DOMAIN}"
DUCKDNS_TOKEN="${DUCKDNS_TOKEN}"
CONFEOF
chmod 600 "$config_file"
return 0
}
# ── Set up Joplin Server (Docker) ───────────────────────────────────
setup_joplin_server() {
info "Setting up Joplin Server via Docker..."
# Ensure Docker is installed and running
if ! command -v docker >/dev/null 2>&1; then
info "Installing Docker..."
sudo pacman -S --needed --noconfirm docker docker-compose
fi
if ! systemctl is-active --quiet docker; then
info "Starting Docker service..."
sudo systemctl enable --now docker
fi
# Add user to docker group if not already a member
if ! groups | grep -q '\bdocker\b'; then
warn "Adding $USER to docker group (re-login required for group to take effect)."
sudo usermod -aG docker "$USER"
fi
# Create data directory
mkdir -p "$JOPLIN_DATA_DIR"
local compose_file="$JOPLIN_DATA_DIR/docker-compose.yml"
# Load saved DuckDNS config if it exists
if [[ -f "$JOPLIN_DATA_DIR/.duckdns.conf" ]]; then
# shellcheck source=/dev/null
source "$JOPLIN_DATA_DIR/.duckdns.conf"
fi
# Set up DuckDNS for a stable hostname
local server_url
if setup_duckdns; then
server_url="http://${DUCKDNS_DOMAIN}.duckdns.org:${JOPLIN_SERVER_PORT}"
info "Using stable DuckDNS URL: $server_url"
else
# Fallback to public IP
local host_ip
host_ip="$(curl -s --max-time 5 ifconfig.me 2>/dev/null)"
if [[ -z "$host_ip" ]]; then
host_ip="$(ip -4 route get 1.1.1.1 2>/dev/null | awk '{print $7; exit}')"
fi
if [[ -z "$host_ip" ]]; then
host_ip="$(hostname -I 2>/dev/null | awk '{print $1}')"
fi
if [[ -z "$host_ip" ]]; then
warn "Could not detect external IP. Falling back to 0.0.0.0"
host_ip="0.0.0.0"
fi
server_url="http://${host_ip}:${JOPLIN_SERVER_PORT}"
warn "Using raw IP URL (may change!): $server_url"
fi
cat >"$compose_file" <<EOF
version: "3"
services:
joplin-db:
image: postgres:16
container_name: joplin-db
restart: unless-stopped
volumes:
- joplin-db-data:/var/lib/postgresql/data
environment:
POSTGRES_DB: joplin
POSTGRES_USER: joplin
POSTGRES_PASSWORD: changeme
networks:
- joplin-net
joplin-server:
image: joplin/server:latest
container_name: joplin-server
restart: unless-stopped
depends_on:
- joplin-db
ports:
- "0.0.0.0:${JOPLIN_SERVER_PORT}:22300"
environment:
APP_PORT: 22300
APP_BASE_URL: "${server_url}"
DB_CLIENT: pg
POSTGRES_HOST: joplin-db
POSTGRES_PORT: 5432
POSTGRES_DATABASE: joplin
POSTGRES_USER: joplin
POSTGRES_PASSWORD: changeme
networks:
- joplin-net
volumes:
joplin-db-data:
networks:
joplin-net:
EOF
info "Starting Joplin Server..."
docker compose -f "$compose_file" up -d
echo ""
info "Joplin Server is running at: ${server_url}"
echo ""
echo " Default admin credentials:"
echo " Email: admin@localhost"
echo " Password: admin"
echo ""
warn "IMPORTANT: Change the default admin password and the database"
warn "password (POSTGRES_PASSWORD in $compose_file) immediately!"
echo ""
info "Firewall: opening port ${JOPLIN_SERVER_PORT}/tcp for external access..."
if command -v ufw >/dev/null 2>&1; then
sudo ufw allow "${JOPLIN_SERVER_PORT}/tcp" || warn "Could not configure ufw"
elif command -v firewall-cmd >/dev/null 2>&1; then
if sudo firewall-cmd --permanent --add-port="${JOPLIN_SERVER_PORT}/tcp" &&
sudo firewall-cmd --reload; then
:
else
warn "Could not configure firewalld"
fi
elif command -v iptables >/dev/null 2>&1; then
sudo iptables -A INPUT -p tcp --dport "${JOPLIN_SERVER_PORT}" -j ACCEPT || warn "Could not configure iptables"
else
warn "No firewall tool found. Ensure port ${JOPLIN_SERVER_PORT}/tcp is open manually."
fi
echo ""
echo " To connect Joplin desktop/Android to this server:"
echo " 1. Open Joplin → Tools → Options → Synchronisation"
echo " 2. Set target to 'Joplin Server'"
echo " 3. Enter URL: ${server_url}"
echo " 4. Enter your Joplin Server email and password"
echo ""
echo " Server management:"
echo " Start: docker compose -f $compose_file up -d"
echo " Stop: docker compose -f $compose_file down"
echo " Logs: docker compose -f $compose_file logs -f"
echo " Update: docker compose -f $compose_file pull && docker compose -f $compose_file up -d"
}