diff --git a/docs/superpowers/evidence/setup-searxng-2026-08-09.json b/docs/superpowers/evidence/setup-searxng-2026-08-09.json new file mode 100644 index 00000000..fa057a22 --- /dev/null +++ b/docs/superpowers/evidence/setup-searxng-2026-08-09.json @@ -0,0 +1,110 @@ +{ + "intent": "Self-host a SearXNG metasearch instance at searx.kuhy.duckdns.org behind the existing shared Caddy edge, configured to earn an A+ Mozilla Observatory grade while remaining 'Vanilla' (settings-only changes, no template/static edits). Publicly reachable but deliberately NOT submitted to the searx.space directory. JSON API enabled for local programmatic/MCP use.", + "scope": [ + "NEW: linux_configuration/scripts/single_use/features/setup_searxng.sh (idempotent setup|status|help)", + "Generated at runtime (outside repo): ~/searxng/docker-compose.yml, ~/searxng/core-config/{settings.yml,limiter.toml}, ~/gitea/sites/searx.caddy", + "Non-goal: searx.space listing (user opted out), so 'Vanilla' is a build constraint rather than a graded outcome", + "Non-goal: sub-200ms search latency -- physically unachievable for a metasearch (see verification)", + "Does NOT add a second Caddy, a second DuckDNS updater, or hand-edit /etc/nftables.conf" + ], + "changes": [ + "Both containers run network_mode: host. Docker BRIDGE networking has zero egress on this host (nftables 'chain forward { policy drop; }' is terminal at NF_INET_FORWARD regardless of Docker's legacy-iptables ACCEPTs), so a stock bridge compose cannot reach a single search engine.", + "GRANIAN_HOST=127.0.0.1 / GRANIAN_PORT=8090 pin the bind. VERIFIED EMPIRICALLY that granian binds :: (all interfaces) by default in this image despite its own docs claiming 127.0.0.1 -- with host networking that would have LAN-exposed it. server.bind_address in settings.yml is inert (read only under __main__/Flask dev server).", + "Caddy snippet sets 5 security headers using the BARE header name ('set' semantics). The '>' prefix is NOT a header setter -- it adapts to {\"replace\":{\"\":[{\"search_regexp\":...}]}} and silently sets nothing; '+' appends and would duplicate. SearXNG sends X-Content-Type-Options and Referrer-Policy itself (its Referrer-Policy is 'no-referrer'), so 'set' is required to override rather than duplicate.", + "limiter.toml written explicitly: without it SearXNG logs 'missing config file' and runs with no trusted_proxies, so botdetection 429s every request including the homepage.", + "server.public_instance deliberately NOT set: it force-enables link_token (a browser-proof challenge no scripted client can satisfy), which makes the requested JSON API unusable. limiter and image_proxy are kept via their own keys.", + "botdetection pass_ip scoped to loopback ONLY, so local tools can use format=json while public traffic (real client IP via X-Forwarded-For) stays rate-limited.", + "start_stack does an explicit docker restart: 'compose up -d' is a no-op when only a bind-mounted config file changed, so without it every verification silently tests the previous config.", + "reclaim_config_ownership: the entrypoint chowns core-config to uid 977, which would make the second setup run fail with 'Permission denied'. settings.yml is 640 not 600 (600 broke the worker's read).", + "No cap_drop on the searxng container -- it needs DAC_OVERRIDE/FOWNER to chown and read config before dropping to uid 977. valkey keeps cap_drop: [ALL] (verified clean).", + "All curl-based checks send browser-shaped headers (UA + gzip Accept-Encoding + Sec-Fetch-*). Bare curl gets 429 from botdetection, so the status check previously reported a perfectly healthy instance as broken.", + "Engine pruning is measured, not guessed: brave/duckduckgo/startpage disabled (CAPTCHA on 100% of queries), flickr (1.0s median), pinterest (blackholed to 0.0.0.0 by this host's own /etc/hosts -- a LOCAL failure that would otherwise be misdiagnosed as a dead upstream)." + ], + "verification": [ + { + "command": "shellcheck + shfmt -d setup_searxng.sh", + "result": "pass", + "evidence": "Both clean, no findings. bash -n syntax OK." + }, + { + "command": "setup_searxng.sh setup (run end to end, repeatedly)", + "result": "pass", + "evidence": "12 [ OK ] lines. Idempotent across 6 consecutive runs; secret_key preserved (verified c72ff7f5e187... survived re-runs)." + }, + { + "command": "setup_searxng.sh status", + "result": "pass", + "evidence": "18/18 checks OK: both containers running + loopback-only, valkey PONG, snippet present, DNS resolves, external HTTPS 200, all 5 headers exactly 1, live search 29 results." + }, + { + "command": "curl -sI https://searx.kuhy.duckdns.org/ | grep -ci '^
:' (x5)", + "result": "pass", + "evidence": "content-security-policy=1 strict-transport-security=1 x-content-type-options=1 referrer-policy=1 permissions-policy=1. referrer-policy value is 'strict-origin-when-cross-origin' (ours) overriding SearXNG's 'no-referrer', proving 'set' semantics work." + }, + { + "command": "loopback binding assertion on both containers", + "result": "pass", + "evidence": "searxng 127.0.0.1:8090, searxng-valkey 127.0.0.1:6379. Guard rejects *, 0.0.0.0 AND [::], and demands positive evidence first (fails closed)." + }, + { + "command": "real search through public HTTPS with browser headers", + "result": "pass", + "evidence": "HTTP 200, 28 results in 0.90s for 'arch linux'. Post-prune: 20 results/query with unresponsive_engines=[] across 3 queries." + }, + { + "command": "JSON API scoping (the decisive check)", + "result": "pass", + "evidence": "Loopback client -> 200 with real JSON results. Simulated public scripted client (X-Forwarded-For set, no Sec-Fetch-*) -> 429. Exactly the intended split." + }, + { + "command": "CSP compatibility audit of actual page assets", + "result": "pass", + "evidence": "All CSS/JS/img/manifest same-origin (/static/...). ZERO inline scripts (the single