signal-bot/DOCS-latency.md

Latency: where the 3.7 seconds actually go

Measured on the live bot, 2026-08-29, for one real !cat in the production group. Every number here came from journalctl --user -u signal-bot.service, not from a model of the system.

22:59:37.984  running command cat
22:59:38.370  GET api.thecatapi.com/…/search   200    386 ms
22:59:39.054  GET s3…/MjA2MDgxMA.jpg           200    684 ms
22:59:41.702  POST localhost:9922/v2/send      201   2648 ms

Fetching the picture — the JSON lookup plus the download — is 1.07 s of the 3.7 s. POST /v2/send on its own is 2.65 s, and that is the signal-cli-rest-api container uploading the attachment to Signal. It is 72% of the wall clock and none of it is this repo's code.

The consequence for the latency release

The plan proposed an image prefetch pool as "the single biggest win": keep N pre-fetched, pre-base64'd images per command in memory so a command costs one send instead of fetch JSON → download → encode. The measurement says that buys 3.7 s → about 2.7 s. Worth having, but it is not the biggest win and it does not get anywhere near a sub-second reply, because the send is untouched.

Do not design the latency release on the plan's assumption. Before building anything, measure these two separately:

1. A text-only reply, end to end. That isolates POST /v2/send without an attachment and says how much of the 2.65 s is upload versus fixed overhead.
2. Whether signal-cli can reuse an already-uploaded attachment id, so a repeated image costs a reference rather than a fresh upload. If it can, that is the real win and it makes the prefetch pool far more valuable, because the pool could hold ids instead of bytes.

Head-of-line blocking is a separate, confirmed problem and does not need re-measuring: bot.py awaits each frame to completion, so one slow upstream fetch delays every other message behind it. Dispatching each frame as its own task under a bounded semaphore fixes that regardless of what the send costs.

Rule

An upstream latency claim gets measured before it gets designed around. The prefetch-pool assumption survived a whole planning pass and was wrong by more than a factor of two in the direction that mattered.