Skip to content

[fence 8/9] libsql-server: replica servers honour the replicated fence - #48

Closed
tszymczyszyn-shopify wants to merge 3 commits into
namespace-fence/6-typed-outcomesfrom
namespace-fence/7-replicas
Closed

tszymczyszyn-shopify wants to merge 3 commits into
namespace-fence/6-typed-outcomesfrom
namespace-fence/7-replicas

Conversation

@tszymczyszyn-shopify

@tszymczyszyn-shopify tszymczyszyn-shopify commented Oct 5, 2026 •

Copy link
Copy Markdown

Parked (2026-10-14). We are not pursuing this stack for now, so please do not review or merge it. The existing per-namespace block_writes and block_reads settings, together with operator checks, cover what we need today. The branches stay up for reference.

If we need more server support later, the likely follow-ups are small, separate changes: refusing replication and /dump while reads are blocked, a conditional POST /config, and block flags at namespace creation.


Replica servers publish the primary's replicated fence as a local read denial and end replication with a typed status; fence refusals, including those delivered as a stream's terminal status, advance the reconnect backoff. A replica no longer lazily creates a namespace that the primary's fence refused.

Review focus

Replica-side behaviour and backoff.

Since the first version, from review: a replica server's namespaces never use the SQL read fast path from #44, so observe_primary can cancel every read it must stop, and the fast-path flag follows the primary's denial.

Commits

  • libsql-server: honour the replicated fence on replica servers
  • libsql-server: do not create a replica namespace the primary's fence refuses
  • libsql-server: keep replica reads off the SQL read fast path (review)

Stack

Part 8 of 9, based on namespace-fence/6-typed-outcomes. The stack sits on #53 (tszymczyszyn/atomic-namespace-create), which sits on #51: review and land #51 → #53 → #41 → … → #49 bottom-up, restacking after each squash or rebase merge.

The first version of these PRs carried #35's fence diff unchanged on v0.9.30-shopify-patches. On 2026-10-13 the stack was rebased onto #53 (how the conflicts were resolved is summarised in a comment on #41) and gained the commits marked above.

shopify-river and others added 3 commits October 13, 2026 22:49
A replica server whose primary refuses replication of a namespace with a
fence code (a refused `hello`, `log_entries` or `snapshot`, or a stream
the primary ended with the typed status) now refuses local reads and
streams of its copy with the same code, and asks the reads it had already
admitted to stop. The denial is published on the namespace's fence
controller on the replica, so every user protocol reports it as it does
on the primary; writes keep going to the primary, which refuses them
itself. A `hello` the primary answers lifts it.

The refusal is no longer retried every second by the handshake loop: the
replica's replication loop waits 1 s, doubling to 15 s, until the primary
answers again, and counts each refused call in
`libsql_server_replica_fence_refusals_total{code}`.

Co-authored-by: Tomasz Szymczyszyn <tomasz.szymczyszyn@shopify.com>
…refuses

A replica server creating a namespace lazily for a name the primary's fence
refuses (a quarantined or aborted target, a read-fenced source, a fence state
the primary cannot establish) now fails the request at once with the primary's
code (Error::NamespaceFence: 423 and the stable code) instead of retrying the
handshake, and leaves no local namespace behind: the namespace directory the
setup created is removed (a directory that already existed is kept), and
NamespaceStore::with forgets the metastore entry handle() added and the
controller the attempt created, when neither holds anything durable or is in
use by another attempt. A later request, once the primary admits the name,
creates it normally.

Co-authored-by: Tomasz Szymczyszyn <tomasz.szymczyszyn@shopify.com>
A replica server learns of the primary's read fence through
observe_primary, which denies local reads and cancels every read lease
held, so that work admitted before it learned does not outlive it. A
read counted on the fast path cannot be cancelled. Mark a replica
namespace before its first connection so that all its reads take the
registered lease, and refresh the fast-path flag when the primary's
denial changes.
@tszymczyszyn-shopify
tszymczyszyn-shopify force-pushed the namespace-fence/6-typed-outcomes branch from cdd8e0b to 6caa89f Compare October 13, 2026 20:51
@tszymczyszyn-shopify

Copy link
Copy Markdown
Author

Closing. This stack is parked; the note at the top of the description explains why. The branch is kept at 7291705b5d, so this PR can be reopened if an automated mover or concurrent moves make an operation-owned fence necessary.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants