Skip to content

[fence 1/9] libsql-server: document the namespace fence contract - #41

Closed
tszymczyszyn-shopify wants to merge 3 commits into
tszymczyszyn/atomic-namespace-createfrom
namespace-fence/0-contract
Closed

tszymczyszyn-shopify wants to merge 3 commits into
tszymczyszyn/atomic-namespace-createfrom
namespace-fence/0-contract

Conversation

@tszymczyszyn-shopify

@tszymczyszyn-shopify tszymczyszyn-shopify commented Oct 5, 2026 •

Copy link
Copy Markdown

Parked (2026-10-14). We are not pursuing this stack for now, so please do not review or merge it. The existing per-namespace block_writes and block_reads settings, together with operator checks, cover what we need today. The branches stay up for reference.

If we need more server support later, the likely follow-ups are small, separate changes: refusing replication and /dump while reads are blocked, a conditional POST /config, and block flags at namespace creation.


Adds docs/NAMESPACE_FENCE.md, the contract for the operation-owned namespace fence: state model and permission matrix, admin API, durable state, outcome codes and their mapping per protocol, write admission and positive drain, source read fence, target lifecycle, restart and failure semantics, deployment and compatibility, and observability.

It started as the final version of the document from the source PR, landed first so the design can be reviewed before any code; two follow-up commits bring it up to date with the rebase onto #53 and with review. The code pieces that follow implement it; none of them edits the document.

Review focus

The state model (section 3), the outcome codes (section 6), and the deployment/compatibility rules (section 13). Agreeing on these first makes the rest of the stack a check against the contract.

Since the first version: the import API names the shared dump importers (rebase onto #53), and from review the section 3.3 matrix now matches section 3.2 and transition.rs, section 13.5 states the cross-namespace lock order for fork, section 9.1 describes the unfenced SQL read fast path, and section 17 no longer claims every test is in one change.

Commits

Stack

Part 1 of 9, based on tszymczyszyn/atomic-namespace-create. The stack sits on #53 (tszymczyszyn/atomic-namespace-create), which sits on #51: review and land #51 → #53 → #41 → … → #49 bottom-up, restacking after each squash or rebase merge.

The first version of these PRs carried #35's fence diff unchanged on v0.9.30-shopify-patches. On 2026-10-13 the stack was rebased onto #53 (how the conflicts were resolved is summarised in a comment on #41) and gained the commits marked above.

shopify-river and others added 2 commits October 13, 2026 20:58
Add docs/NAMESPACE_FENCE.md, the contract and design for a durable,
operation-owned namespace fence: source and target state machines and
permission matrix, admin API, stable outcome codes and their mapping to
HTTP, Hrana, gRPC, the write proxy and replication, metastore schema and
CAS semantics, write admission generations and positive drain, read
fence, quarantined target lifecycle and the reusable import capability,
restart and eviction rules, fail-closed metastore recovery, deployment
flag and legacy-binary protection, code-path coverage, test strategy and
the acceptance-test map.

Co-authored-by: Tomasz Szymczyszyn <tomasz.szymczyszyn@shopify.com>
The import capability now loads dumps through namespace::dump_import (the
buffered importer by default, the memory-bounded streaming importer on
request) rather than a separate copy of the loader.
@tszymczyszyn-shopify
tszymczyszyn-shopify changed the base branch from v0.9.30-shopify-patches to tszymczyszyn/atomic-namespace-create October 13, 2026 19:41
@tszymczyszyn-shopify

Copy link
Copy Markdown
Author

Rebased the fence stack (#41–#49) onto #53 (tszymczyszyn/atomic-namespace-create at bb446259f6), so the namespace fence and the streaming dump importer / atomic namespace creation can land together. This PR's base is now that branch. Intended merge order: #51 → #53 → #41 → … → #49.

How the conflicts were resolved:

  • Namespace creation. Generic create and fork keep libsql-server: make namespace creation all-or-nothing (stacked on #51) #53's Reservation; creating a quarantined migration target keeps its single metastore transaction. The fence's checks (check_available, check_lifecycle) now run inside Reservation::acquire, under the name's entry lock, so they cannot go stale; fork still holds the source's transition lock.
  • One read-only lookup. The fence's MetaStore::lookup() replaces libsql-server: make namespace creation all-or-nothing (stacked on #51) #53's existing_handle(); MetaStore::handle() is fallible everywhere.
  • One dump loader. ImportSession loads through namespace::dump_import (buffered by default). New ImportSession::load_dump_with(kind, cfg, stream) can select the memory-bounded streaming importer and returns its statistics.
  • Both clean-ups stay. Discard (undo a creation that failed) and forget_unstored (forget a lazy creation the fence refused) do different things and are kept separate.

New commits (everything else is the original commits replayed):

Local verification (macOS): cargo fmt --check clean; cargo check --all-targets --all-features for libsql-server/libsql_replication with no new warnings; lib tests at every PR head; the nextest integration suite gives the same result as the stack before the rebase.

The section 3.3 matrix omitted operations section 3.2 and transition.rs
allow: abort from TARGET_IMPORT_DRAINING, TARGET_VALIDATING and
TARGET_WRITE_FENCED, status and the seal drain in TARGET_IMPORT_DRAINING,
recording validation and publishing in TARGET_VALIDATING, and adoption.
Section 13.5 stated the lock order per namespace only; add the rule for
fork, the one operation that holds two namespaces' locks, and why
reversed forks cannot wait for each other. Add section 9.1, the fast
path that counts unfenced SQL reads instead of registering a lease. Say
that the acceptance tests are spread across the stack.
@tszymczyszyn-shopify

Copy link
Copy Markdown
Author

Closing. This stack is parked; the note at the top of the description explains why. The branch is kept at 761a904e39, so this PR can be reopened if an automated mover or concurrent moves make an operation-owned fence necessary.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants