libsql-server: live checks for the namespace store

[tszymczyszyn-shopify]

What

Scripts under libsql-server/scripts/namespace-live-checks that run a built sqld locally and exercise the namespace store against a real server:

• load.py: keep-alive clients driving POST /v2/pipeline, tallying status codes per phase, with an optional SIGTERM or shell action at a given time.
• scenario.sh <sqld> evict|sigterm: cache capacity far below the hot set (entries evicted under requests all the time), or SIGTERM under load.
• regress.sh <sqld> [label] [other sqld]: a 65-check matrix over the admin and user HTTP APIs: lifecycle and real 404s, checkpoint, fork (hot, cold, unknown, existing destination), create from a dump, shared schema, delete under load, persistence across SIGTERM with and without --snapshot-at-shutdown, SIGTERM with namespaces disabled, a primary plus replica (catch-up, proxied writes, replica and primary SIGTERM, recovery, destroy on the primary), and mixed versions when a second binary is given.
• compare.py: two results.tsv files side by side, differences marked.
• replica-sigterm.sh <sqld> [label]: eight embedded replicas (the libsql crate, through the new libsql/examples/replica_sync_loop.rs) syncing in a tight loop while their primary is stopped and started again. Prints every distinct sync() error with the time it was first seen, and the replicator's own retry log lines. This is what a client built on the crate sees across a primary restart.
• window.sh <sqld> <namespaces> [label]: loads N namespaces, each with a WAL, and measures the shutdown window: how long clients get 503 before the port closes. The window grows with the number of loaded namespaces; it is what client retry budgets have to cover.

Everything listens on 127.0.0.1. Works with bash 3.2; needs curl, python3, lsof. The replica check needs cargo build -p libsql --example replica_sync_loop.

Why

The failures in this area are races: an entry emptied under a request, a shutdown with requests in flight, a destroy racing a lookup. Unit tests pin each one deterministically. These scripts show the same races on a running server and let two binaries be compared before and after a change. They were written to verify #56, where they reproduced the false 404s on the base binary (5.9% of answers under eviction churn; 13 to 41 per shutdown) and showed none with the fix. The matrix also found two pre-existing problems unrelated to #56, listed in that PR's follow-ups.

The replica check showed the client-side difference directly. Against the base binary, sync() failed at the signal with Requested namespace doesn't exist, which the client does not retry. Against #56 it never did; the client's own retry line (error connecting to primary. retrying. error: status: Unavailable, message: "Namespace store has shutdown") appeared instead, and every replica synced again once the server was back. The window measurement gave about 0.2 to 0.3 ms per loaded namespace in a debug build (0.21 s for 1,000, 0.48 s for 2,000, 1.7 s for 5,000).

Not a CI job

Debug builds, several seconds per scenario, and real SIGTERMs. Meant to be run by hand before and after changes to namespace/store.rs, the eviction listener, or the gRPC status mapping. The README says what a healthy run looks like and how to read the output.