Implementation status
offshoot's original design describes a v1 scope larger than what's shipped so far. This page is the honest accounting: what's tested in anger, what's built but unverified in some dimension, and what's still just a plan. If a guarantee you're relying on isn't marked shipped-and-tested below, verify it yourself before depending on it in production.
Status legend:
- shipped-and-tested — implemented, and exercised by an automated test (unit, integration, property, or torture) that would fail if it broke.
- shipped — implemented and in the code path you'll actually hit, but without the same weight of testing (e.g. verified against one provider, not the class of providers it claims to support).
- not yet implemented — spec'd or planned, no code behind it yet. Each row links to the roadmap milestone tracking it.
- deliberately deferred — considered for this milestone and explicitly declined, with a stated reason, rather than simply not yet started; see the row's Notes for why.
Not yet done
Out-of-band items that no further engineering closes, listed here so they don't vanish into individual rows:
- OpenSSF Best Practices badge — most of the passing-level questionnaire is already true (SECURITY.md with private reporting, CONTRIBUTING.md, GOVERNANCE.md, CI on every PR, CodeQL, signed and attested releases, a changelog); the registration itself is not done.
- PyPI / npm publication — deferred. The
offshoot-dbpackage and the@offshoot-dbnpm scope are not published; install both SDKs from git (installation). The publish pipeline exists and dry-runs (make dry-run-sdks). - MCP registry listing —
server.jsonis drafted in-repo; not submitted. - LangGraph community listing — the listing entry is drafted; it waits on the PyPI publication above, since the listed install command needs to resolve.
- Trademark / name-collision check on "offshoot" in developer tooling — not started.
- Domain — none; the site is served from GitHub Pages at
https://sricola.github.io/offshoot/. The Homebrew formula
(
Formula/offshoot.rb) and theghcr.io/sricola/offshootimage namespace are done.
Core model
| Guarantee / feature | Status | Notes |
|---|---|---|
| Branch = named ref (lineage, epoch, head txid, checkpoints, TTL, protected flag) | shipped-and-tested | internal/ops/ops.go, internal/store |
Protected flag settable per branch (offshoot protect/unprotect <db>[@branch]) |
shipped-and-tested | internal/ops/protect_test.go's TestSetProtectedFlipsTheFlagAndGatesDestroy; CLI-only (no daemon op, no MCP tool) — an agent sees the flag via offshoot_list but can't flip it; doesn't touch the branch's TouchedAt/TTL clock; refuses on a branch already claimed by a reaper or destroy; v0.2.11 |
Checkpoint pruning (offshoot prune <db>[@branch] (<cp>... | --keep N) [--force], daemon op prune, MCP offshoot_prune_checkpoints, SDK prune): metadata-only, all-or-nothing on an unknown name, protected branches need force, safe beside a live session; gc reclaims what only the pruned checkpoints kept |
shipped-and-tested | internal/ops/prune_test.go (by name, keep order and ties, unknown name, protected gate, claimed branch, CAS-loss recompute, prune then GC keeps the head readable), internal/ops/prune_session_test.go's TestPruneBesideALiveSessionStaysPruned (prune lands inside a flush; mutation-verified against a flush retry that kept its stale checkpoint map), internal/daemon/prune_test.go, internal/mcp/prune_test.go, cmd/offshoot/prune_test.go, both SDK suites (Python keep=0 mutation-verified). The ref format is unchanged; a separately stored checkpoint index stays a ROADMAP item |
| One writer per lineage, enforced by lease + epoch fencing | shipped-and-tested | internal/ops/fencing_test.go; every object write lands under the epoch current at write time |
| CAS on every ref mutation | shipped-and-tested | Local backend: per-ref flock(2) lock file (TestLocalLockNeverAdmitsTwoHolders). S3-compatible: conditional PutIf, gated by an attach-time capability probe |
| Copy-on-write fork (shared by default via a durable base pointer; materialized fallback at the fork-time snapshot floor) | shipped-and-tested | v0.2.0 (internal/ops/fork_cow_test.go, cow_divergence_test.go): the common-case fork writes data/{lineage}/base.json and zero data objects; store.Chain resolves base pointers transitively under a strict never-merge-across-lineages rule; two automatic snapshot floors (fork-time ops.ForkShareMaxDepth/-snapshot-every, plus the divergence floor) keep reads bounded; status/branches report storage=shared|materialized per branch. The materialize fallback (and compact, and promote/rollback when they copy — see the shared rollback and promote row) keeps the CopyObject fast path for single-snapshot chains (local: reflink; S3: server-side copy — single-request ≤5 GiB, multipart UploadPartCopy up to 5 TiB since v0.2.4), falling back to materialize-and-re-encode otherwise — see Fork performance below. Unreleased: a fork whose create-only ref write lands but comes back as an error (the S3 SDK's retry answering 412 to its own first attempt, or a timeout that loses the response) no longer deletes the base pointer or copied snapshot it just wrote; settleRepoint re-reads the ref, and only a match on the lineage fork minted turns the write into a reported success — anything else (the branch already existing, another writer's lineage, or no ref at all) leaves the object for offshoot gc to reclaim and returns the same "already exists (offshoot status lists branches)" or "fork db@branch: err" text as before; a re-read that itself fails also keeps the object, and returns an error saying the write may have landed, naming the kept lineage and telling the user to run 'offshoot status' before retrying. A fork that materializes now reads the destination ref before copying a snapshot (one GET of the ref key, no LIST), so forking onto a taken name on that path writes nothing there at all; the shared path keeps no such pre-check and still relies on the ref write's compare-and-swap to settle the race. internal/ops/repoint_landed_test.go: TestForkLandedWriteKeepsBaseAndIsReadable, TestForkLostRaceLeavesOrphanForGC, TestForkLandedThenDestroyedKeepsObjects, TestForkMaterializeOntoTakenNameWritesNothing, TestForkMaterializeOntoFreeNameReadsRefOnce, TestRepointSettleUnknownKeepsObjects |
offshoot compact (make a shared fork self-contained; ancestor storage becomes reclaimable) |
shipped-and-tested | v0.2.0 (internal/ops/compact_test.go, compact_cow_test.go); no-op on an already self-contained branch; daemon op + SDK compact() both languages; the daemon refuses while the branch has an open session (v0.2.1) |
Compact preserves every checkpoint (rollback-style: each is copied into the new self-contained lineage and rewritten to epoch 1, plus an added compact checkpoint at head) |
shipped-and-tested | internal/ops/compact_test.go's TestCompactPreservesCheckpoints; earlier versions dropped every other checkpoint and left only a single compact one — this row now describes the current, preserving behavior. Cost: one snapshot copy per distinct checkpoint txid kept, on top of the head copy compact already paid; v0.2.11. Unreleased: a compact whose ref write lands but comes back as an error used to treat its own successful write as a lost race and delete the copied snapshots, leaving the branch that now named the new lineage unreadable; settleRepoint now re-reads the ref against the lineage compact minted and the lineage it read before the write, deletes the copies only when the re-read proves the write did not land (the ref still names the lineage read before the write) and returns "compact lost a race (retry)", keeps them for offshoot gc when a third lineage or a missing ref makes the outcome unknown, and names the kept lineage with advice to run offshoot status when the re-read itself fails. internal/ops/repoint_landed_test.go's TestCompactLandedWriteKeepsCopies and TestRepointSettleUnknownKeepsObjects; compact_test.go's TestCompactCASRaceCleansOrphan still pins the one case (a rival write that keeps the branch's lineage) that deletes immediately |
| Checkpoint (named state within a branch, not inherited by children) | shipped-and-tested | |
| Rollback (repoint at new lineage seeded from a checkpoint) | shipped-and-tested | Since v0.2.12 the shared case copies nothing: the new lineage is a base pointer at the kept checkpoint (see the shared rollback and promote row), so the checkpoints kept at or below it resolve through that pointer rather than being copied forward. --materialize (daemon/SDK materialize), and the fork-time depth floor, still copy the kept checkpoints' snapshots into the new lineage as before |
Rollback safety fork (<branch>-pre-rollback: the branch's previous head kept as a shared, TTL'd fork before the repoint; rolling one-per-branch; marker-guarded so a user's branch is never replaced; --no-backup/--backup-ttl, daemon no_backup/backup_ttl, SDK backup/backup_ttl; always on for MCP, which names it as backup in the result) |
shipped-and-tested | internal/ops/rollback_backup_test.go's TestRollbackKeepsSafetyFork; mirrors promote's safety fork below — same safetyFork implementation, default TTL 24h; v0.2.11 |
| Promote (repoint target at source's head via fork machinery) | shipped-and-tested | Protected targets require --force |
Promote safety fork (<target>-pre-promote: the target's previous head kept as a shared, TTL'd fork before the repoint; rolling one-per-target; marker-guarded so a user's branch is never replaced; --no-backup/--backup-ttl, daemon no_backup/backup_ttl, SDK backup/noBackup; always on for MCP, which names it in the result) |
shipped-and-tested | internal/ops/promote_backup_test.go, daemon TestPromoteKeepsSafetyForkThroughDaemon, MCP TestPromoteKeepsSafetyForkAndSaysSo, both SDK suites. Prompted by discussion #40. The fork's base pointer pins the target's old lineage until the TTL reaps it. Since v0.2.12 promote itself shares too (see the shared rollback and promote row) |
Clone-based checkouts from a content-addressed by-chain cache (checkouts-ro/<db>/~by-chain/<chainID>.db; checkout, internal checkout refreshes, session open and checkout --at clone from an immutable entry built first — from a cached chain prefix plus ltxio.ApplySegments, or from the store; plain materialize where the filesystem cannot clone) |
shipped-and-tested | internal/ops/clone_test.go: TestForkCheckoutsCloneFromByChainCache, TestByChainEntriesAreEvictedAndRecreated, TestClonePathDegradesWhenUnsupported (always runs: forces reflink.ErrUnsupported), TestByChainEntryWithoutUsableSidecarIsAMiss, TestDivergedChildClonesPrefixAndAppliesSegments, TestCheckoutAtSharesTheByChainCache, TestMaterializeMissBuildsImmutableEntryFirst, TestByChainAreaIsBoundedByDefault, TestByChainAndShadowModesArePrivate; internal/ltxio's TestApplySegmentsMatchesMaterializeChain; internal/reflink's TestCloneProducesIdenticalFile; internal/session's TestOpenOnClonedCheckoutSkipsSettlingFlush. By-chain entries are capped at 64 per database by default (ops.DefaultByChainMaxEntries, LRU-pruned after each populate, no daemon needed; destroy does not remove them), count toward -ro-cache-budget at logical size (a --at miss counts twice) and are evicted as branch ~by-chain; entries are 0400 in 0700 directories; checkouts-ro stays safe to rm -rf. The clone tests skip on a filesystem that cannot clone — see the CI row under Platform; v0.2.12 |
O(1) clean check for an unchanged checkout (.sum sidecar fingerprint: size, mtime_ns, SQLite change_counter, stamped_ns; clean only when identity and fingerprint match and the mtime is more than 1 s older than the stamp; the change counter is not evidence in WAL mode; any mismatch hashes and re-stamps) |
shipped-and-tested | internal/ops/sidecar_test.go: TestCleanSkipDoesNotHashWhenFingerprintMatches, TestModifiedCheckoutStillDetectedRollbackJournalAndWAL, TestModifiedCheckoutWALForcedMatchingMtimeStillDetectedViaRacilyCleanGuard, TestRacilyCleanGuardHashesWithinMarginThenFastPathOnceSettled, TestTouchedButUnchangedCheckoutHashesAndStaysClean, TestOldSidecarWithoutFingerprintHashes, TestShortHeaderStampSumSucceedsAndCheckoutStateHashes, TestNoCacheFallbackWriteDuringHashRecordsNoFingerprint, TestSandwichedStampAnchorsStampedNSBeforeTheHash. Every stamp whose hash is computed here sandwiches it between two fingerprints (a write during the hash records no fingerprint) and anchors stamped_ns at the first. Filesystem-independent. Known blind spot, shared with git status: an edit that restores both size and mtime and (outside WAL mode) the change counter is invisible until the next hash. offshoot status's dirty check still always hashes; v0.2.12. Unreleased: an at-rest checkpoint's stamp takes no fingerprint shortcut and so no margin: its one read of the checkout computes both the sidecar's SHA-256 and the LTX checksum (ltxio.StreamChecksum), and it is trusted only when that checksum equals the head's and the WAL beside the file is empty — internal/ops/sidecar_test.go: TestStampCheckpointDistrustsSameTickWrite, TestStampCheckpointTrustsUnchangedContentInOnePass, TestStampCheckpointDistrustsNonEmptyWAL, TestStampCheckpointGarbageCheckoutHashesLikeFileSum; internal/ltxio/stream_test.go |
At-rest segment checkpoints (offshoot checkpoint diffs the checkout against a reflinked .shadow and writes an LTX segment of the changed pages when the shadow verifies, identity matches, the chain is below the snapshot bound and under half the pages changed; else a snapshot; --snapshot forces one; ref checkpoint entries record kind; MCP result carries kind) |
shipped-and-tested | internal/ops/checkpoint_delta_test.go: TestSecondCheckpointWritesASegment, TestDeltaCheckpointsRoundTripUnderRandomMutations (30 rounds of random insert/update/delete, VACUUM and incremental vacuum; every checkpoint materialized and compared byte for byte), TestShadowIdentityMismatchFallsBackToSnapshot, TestDivergedShadowFallsBackToSnapshot, TestEveryBoundthCheckpointIsASnapshot, TestLargeChangeFractionForcesSnapshot, TestSnapshotOptionForces, TestNoShadowWhenReflinkUnsupported, TestForkFromSegmentHeadMaterializes, TestDestroyRemovesShadowAndItsTemp; internal/ops/checkpoint_lease_test.go's TestStragglerUnderOldEpochCannotAnchorHead (a snapshot or segment a failed attempt left at the next txid sits under an older epoch, so the checkpoint writes the segment it planned, the straggler never anchors the head, and GC reclaims it); internal/session's TestAtRestCheckpointAfterCloseWritesASegment; internal/store's TestCheckpointKindRoundTripsAndDefaultsEmpty; cmd/offshoot's TestCheckpointOutputNamesItsKind. The diff still reads the whole checkout and shadow (O(size) local I/O); changed pages are buffered in memory until upload, up to half the database. On a filesystem that cannot clone there is no shadow and every at-rest checkpoint is a snapshot; v0.2.12. v0.2.14: checkpoint (like rollback, promote --onto and compact) refuses a branch with a live lease unless --force, refuses one mid-destroy or mid-reap outright, and refuses a detached checkout (sidecar lineage differs from the ref's) unless --force; checkout warns before replacing a stale checkout that holds un-checkpointed edits — internal/ops/guards_test.go: TestAtRestVerbsRefuseLiveLease, TestCheckpointRefusesDetachedCheckout, TestCheckoutWarnsBeforeDiscardingStaleEdits. v0.2.17: checkpoint refuses a live lease with or without --force (from v0.2.14 through v0.2.16, --force wrote the head under the holder's own epoch; the checkpoint now takes the lease itself, see the concurrent-checkpoint row), and the refusal names offshoot lease release <db>@<branch> --holder <that holder> for a holder that is gone — internal/ops/guards_test.go: TestCheckpointRefusesLiveLeaseEvenWithForce, TestLeaseRefusalsNameAnInProgressCheckpoint; cmd/offshoot's TestCheckpointForceDoesNotTakeOverALease |
Shared rollback and promote (a base pointer at the kept checkpoint / the source's head via fork's newLineageAt, not a snapshot copy; --materialize / daemon materialize / SDK materialize forces the copy; the fork-time depth floor still copies; Store.CollapseBase keeps repeated rollbacks' base spine one hop deep; results carry shared; offshoot_rollback_total{mode} / offshoot_promote_total{mode}) |
shipped-and-tested | internal/ops/repoint_shared_test.go: TestRollbackToKeptCheckpointIsShared, TestPromoteIsSharedAndSourceStaysResolvable, TestGCAfterSharedRollbackReclaimsTheAbandonedFuture, TestCompactAfterSharedRollbackDetaches, TestRepeatedRollbacksHitTheDepthFloor, TestRepeatedRollbacksToOneCheckpointKeepTheSpineFlat, TestMaterializeOptionForcesCopy; internal/store's TestCollapseBase; cmd/offshoot's TestRollbackAndPromoteOutputNameTheirMode; internal/daemon's TestMetricsSmokeOpenWriteFlushFork; both SDK suites assert the materialize wire field. Filesystem-independent. Behaviour change: offshoot rollback prints a second line after the checkout path. Compact still always copies; v0.2.12. Unreleased: a rollback or promote whose ref write lands but comes back as an error (the S3 SDK's retry answering 412 to its own first attempt, or a timeout that loses the response) used to treat its own successful write as a lost race and delete the new lineage's base pointer or copied snapshot, leaving the branch that now named it unreadable until something rewrote it; settleRepoint now re-reads the ref and compares it against the lineage ID the verb minted and the lineage it read before the write: a match on the minted lineage reports success and the verb falls through to its usual checkout refresh; the ref still naming the lineage read before the write proves the write did not land, and only then does the existing cleanup delete the new lineage's objects and return "rollback/promote lost a race (retry)"; a third lineage or a missing ref leaves the objects for offshoot gc to reclaim, and a re-read that itself fails returns an error naming the kept lineage and pointing at offshoot status, with the same "lost a race (retry)" text otherwise unchanged. internal/ops/repoint_landed_test.go: TestRollbackLandedWriteKeepsObjects, TestPromoteLandedWriteKeepsObjects, TestRollbackLandedThenRepointedAwayKeepsObjects, TestRollbackLostRaceLeavesObjectsForGC, TestRollbackLostToRefWriteRemovesObjects, TestRollbackSettleUnknownKeepsObjects, TestPromoteLostToRefWriteRemovesObjects, TestPromoteSettleUnknownKeepsObjects |
Two concurrent at-rest offshoot checkpoint calls on one branch (refused, lease-fenced) |
shipped-and-tested | An at-rest checkpoint takes the branch lease (holder checkpoint:<host>/<pid>/<8 hex>, a per-call nonce so two calls in one process never share an epoch) before any object write, running its refusals (a live lease, a taken name, a missing or detached checkout) on the ref the acquire reads, before the acquire writes (store.AcquireLeaseRefIf), so a refused checkpoint writes nothing and a writer that moves the ref in between fails the acquire's compare-and-swap; it renews it every LeaseTTL/3 while it quiesces, encodes and uploads, and releases it in the ref write that advances the head. The acquire bumps the epoch, so the object key is private to the call: the object goes up create-only, the probes for another checkpoint's snapshot at its txid are gone (two fewer LISTs per segment checkpoint), and a straggler under an older epoch loses chain resolution (keepHighestEpoch) and is reclaimed by GC. An acquire whose write landed but reported failure (an S3 SDK retry answering 412 to its own first attempt) adopts the lease it wrote from the re-read ref, with no second write that could fail the same way; one that lost only to a metadata write (touch, protect) is retried once; one that finds another writer came and went in between is refused like a live lease; and a refusal of a ref carrying its own lease releases that lease. While it runs, a second checkpoint, a session open, and an unforced rollback/promote --onto/compact/destroy are refused (errors.Is(err, store.ErrLeaseHeld)); checkpoint --force never takes over a live lease. A lease that ends while it runs (a forced rollback, promote --onto or compact, lease release, a reclaim after expiry, destroy --force), whether a renewal or the head write's re-check finds it, fails the checkpoint without committing, and it deletes its own object; the lease is checked once more just before the upload, so a loss the renewer has already seen uploads nothing. The head write re-checks holder, epoch, lineage and head and will not write over a destroy or reap claim (the release then leaves the claim alone too, so the destroy's conditional delete goes through), makes up to 3 attempts when a compare-and-swap is lost to a metadata write, recognises its own write when only the response was lost, and deletes its own object only when the ref proves nothing names it and no head write it sent can still land. It keeps the object after three lost compare-and-swaps (on S3 a 409 conflict can be the SDK's retry meeting its own first attempt still in flight), and keeps it and reports that it may have committed after a head write that got no verdict from the store (a timeout), or one the ref can no longer account for (a repoint replaced the lineage, the ref cannot be read, or a destroy or reap claim landed after it, which an abandoned destroy or reap undoes to the very bytes the write was sent against). Before any head write is sent, a head that an older binary's checkpoint --force moved to its txid under its own epoch keeps the object too, since that head may name it. Renewals, the checkpoint's and a session's, write over a destroy claim and leave it set, as on a branch without one, so a claim a killed destroy stranded never lapses a live lease; destroy --force's local conditional delete re-reads the ref and deletes again (up to 4 attempts) while the ref is its claim with only the lease expiry moved, so a renewal landing between its claim and its delete does not fail it, and the holder's next renewal finds the branch gone. The acquire costs one more durable ref write per checkpoint: on a local macOS store BranchBench's simulation checkpoint p50 rose about 3 ms run one at a time and about 11 ms eight at a time; on S3 a snapshot checkpoint makes one more ref GET and PUT and a segment checkpoint the same number of requests as before; and on a local store each checkpoint adds an epoch directory, so chain resolution on one lineage slows as checkpoints accumulate until a compact (BenchmarkChainAfterCheckpoints; benchmarks) — the row below documents the recorded-chain cache, which removes this cost from a segment checkpoint's own resolve specifically (not from checkout/fork-below-head/materialize, which are unaffected). The post-commit HEAD/trailer check and the stamp's one-pass content check (the read that hashes the checkout for the sidecar also computes its LTX checksum, which must equal the head's) remain, for overwrites a lease cannot fence and writes to the checkout. internal/ops/checkpoint_lease_test.go: TestConcurrentAtRestCheckpointsAreSerialized (local and fake S3; while the first checkpoint runs it also asserts the branch reads active through BranchState, offshoot status's Status and offshoot lease list's Leases, naming the checkpoint holder), TestCheckpointHoldsTheLeaseThroughItsUpload, TestTwoCheckpointsInOneProcessGetDistinctEpochs, TestCheckpointReleasesLeaseInHeadWrite, TestCheckpointBuildsOnACommitBeforeItsAcquire, TestCheckpointFailureReleasesLease, TestCrashAfterAcquireLeavesReclaimableLease, TestCheckpointReclaimsAnExpiredLease, TestAcquireThatLandedIsAdopted, TestAcquireThatAlwaysReportsFailureIsAdopted, TestRefusedAdoptionReleasesItsLease, TestLateAcquireRefusedOnRetryReleasesItsLease, TestAcquireLostToATouchIsRetried, TestAcquireLostToACompletedCheckpointIsRefused, TestStragglerUnderOldEpochCannotAnchorHead, TestCheckpointRenewsWhileUploading, TestRenewLeaseLostAbortsBeforeRefWrite, TestLeaseLostBeforeUploadSkipsTheUpload, TestRepointVerbsDuringCheckpoint (rollback, promote onto and compact, unforced and forced), TestLeaseReleaseDuringCheckpointFailsIt, TestDestroyForceDuringCheckpointAbortsIt, TestDestroyClaimBeforeHeadWriteFailsTheCheckpoint, TestHeadWriteSentBeforeAClaimKeepsItsObject (a destroy or reap claim after a held head write, undone, after which the write lands on an object that is still there), TestDestroyForceAbortsACheckpointWhoseRenewalLandsInItsWindow (local and fake S3), TestOlderBinaryCheckpointUnderOurLeaseKeepsTheHeadsObject, TestCheckpointChecksTheRefItsAcquireReads, TestFinalCASRetriesAfterConcurrentTouch, TestFinalCASGivesUpAfterThreeLosses, TestHeadWriteHiddenBehindAConflictKeepsItsObject, TestHeadWriteThatLandedIsNotUndone, TestHeadWriteThatFailedThenLostTheLeaseDeletesItsObject, TestHeadWriteWithoutAVerdictKeepsItsObject (a timed-out head write the store applies after the checkpoint gives up, as its last attempt or its first), TestHeadWriteThatLandedThenMovedKeepsItsObject (a fork and a rollback, or an unreadable ref, after a head write that landed), TestRetriedObjectPutOfOurOwnBytesIsAccepted, TestForeignObjectAtPrivateKeyFailsTheCheckpoint, TestObjectPutConflictWithNothingToReadIsNotCorruption, TestReleaseFailureLogsTheLeasesRealExpiry, TestCheckpointHolderIsPerCall, TestRenewErrorClassification and the TestCheckpointRenewer* tests; internal/ops/checkpoint_lease_session_test.go (a real daemon session): TestSessionOpenDuringAtRestCheckpointIsRefused, TestForceCannotTakeOverLiveLease, TestCheckpointPlansFromTheAcquiredRef, TestAtRestCheckpointIsRefusedWhileASessionCloses (a close held past its lease's expiry keeps renewing, so a checkpoint, forced or not, is refused throughout and goes through once the close releases); internal/ops/guards_test.go: TestCheckpointRefusesLiveLeaseEvenWithForce, TestLeaseRefusalsNameAnInProgressCheckpoint, TestLeaseRefusalsWithoutForceAdvice; internal/ops/rollback_backup_test.go's TestSafetyForkUnderACheckpointLeaseSaysRetry; internal/ops/checkpoint_epoch_bench_test.go's BenchmarkChainAfterCheckpoints; internal/ops/rpc_count_test.go's TestAtRestCheckpointStoreRequests; internal/ops/ops_test.go's TestConcurrentCheckpointsOnlyOneWins and TestConcurrentCheckpointsOnlyOneWinsOnS3; internal/ops/checkpoint_overwrite_test.go (the post-commit checks, the foreign object now put at the private key by hand): TestOverwriteWithIdenticalBytesStaysTrusted and TestOlderEncodeOverwriteReadsModified (local and fake S3), TestWriteBetweenEncodeAndStampDistrusts, TestUnverifiableOverwriteIsLoggedAndDistrusted, TestIdenticalOverwriteIsNotFlagged, TestOverwriteDetectionSkipsWhenHeadUnsupported, TestReformattedEtagSkipsTheGet; internal/store/storetest's conformance subtests HeadReturnsEtagAndSize, HeadMissingIsErrNotFound, HeadEtagChangesWhenContentChanges; internal/store's TestAcquireLeaseRefReturnsTheWrittenRef, TestAcquireLeaseRefRefusesALiveLease, TestAcquireLeaseRefIfChecksTheRefItWrites, TestRenewLeaseLeavesAClaimSet, TestStoreHeadFallsBackToGet; internal/daemon's wiring pins TestBranchesReportsAnAtRestCheckpointAsActive and TestOpenRefusedByCheckpointHolderSaysInProgress (a hand-acquired checkpoint: lease: they pin that the branches and open ops carry the holder through, not that a checkpoint takes it) and TestCheckpointOverwriteCounterIsWired, and TestRollbackAndCompactRefusalsOfferNoForce; internal/mcp's TestCheckpointSurfacesLeaseRefusalUnchanged, TestRefusalsUnderACheckpointLeaseSayRetry, TestLeaseRefusalsNeverOfferAForceTheToolLacks, TestCheckpointDetachedRefusalNamesTheToolCalls; cmd/offshoot's TestCheckpointForceDoesNotTakeOverALease (limitations) |
At-rest checkpoint, fork-at-head, promote and rollback-to-head reuse the .sum sidecar's recorded chain instead of re-listing the lineage (removes the epoch-count slowdown on a local store; removes one LIST per segment checkpoint on S3) (chain, additive sidecar field) |
shipped-and-tested | internal/ops/chain_cache_test.go: TestSecondCheckpointUsesTheRecordedChain, TestCachedChainMissesAfterRepoint, TestCachedChainSurvivesGC, TestOldSidecarWithoutChainResolves, TestCorruptRecordedChainResolves, TestForkAtHeadUsesTheRecordedChain, TestCheckpointStampRecordsChainOnlyWhenTrusted, and for a shared child lineage TestSharedChildSecondCheckpointUsesTheRecordedChain, TestSharedChildAfterOwnSnapshotUsesOwnChain, TestPassThroughSpineUsesTheRecordedChain, TestTwoHopSpineUsesTheRecordedChain, TestSharedChildCacheSurvivesBaseDestroyAndGC, TestSharedChildCorruptSeamResolves, TestSharedChildSeamMustMatchRefBase, TestForeignKeysWithoutBaseResolve, TestForkOfSharedChildAtHeadUsesTheRecordedChain, TestForkAfterOwnSnapshotFirstResolvesThenHits, TestPromoteAtHeadUsesTheRecordedChain, TestRollbackToHeadUsesTheRecordedChain, TestSharedChildPrefixMustEndOnBaseLineage, TestPromoteMaterializeFromRecordedChain, TestRollbackMaterializeAtHeadFromRecordedChain; internal/ops/checkpoint_epoch_bench_test.go's BenchmarkSharedChildCheckpointAfterParentCheckpoints; internal/ops/chain_cache_session_test.go's TestCachedChainMissesAfterSessionFlush (package ops_test, since the identity-mismatch-after-flush case needs a real daemon session, which package ops can't import without a cycle); internal/ops/rpc_count_test.go's TestAtRestCheckpointStoreRequests pins the S3 LIST reduction. The shortcut applies only when the sidecar's identity matches the ref's head and the recorded chain is snapshot-anchored, contiguous, on the ref's lineage and ends at the head under that same epoch; any mismatch (a repoint, a session flush, an old or corrupt sidecar) falls back to Store.Chain and re-records the chain. A shared child lineage (resolving through a base pointer) is served too, with no store read: the keys on other lineages must end exactly at ref.Base.TXID, the last of them on ref.Base.Lineage, and the lineage's own must be segments starting at ref.Base.TXID + 1, so only its first checkpoint after a checkout lists, where before this it listed on every checkpoint until it wrote its own snapshot. checkout, a fork below head and materialize still call Store.Chain directly and are unaffected (benchmarks) |
LTX decoder hardening (internal/ltxio: frameGuard on every decode path; checkGrowth/CheckGrowth in applySegments and session.recordApply) |
shipped-and-tested | Found by the Tier 2 fuzz targets (make fuzz, nightly fuzz job). The pinned decoder, github.com/superfly/ltx, accepts objects that make it allocate up to 4 GiB from an unchecked size field, or that grow the database past the pages they carry. offshoot now refuses, before decoding them: a header page size that is not a power of two in [512, 65536]; a block-format page frame whose compressed size exceeds the LZ4 worst-case bound for one page; an LZ4-frame-format page in any shape other than the one ltx v0.5.1 wrote (FLG 0x64, BD 0x40, exactly one block that decompresses to exactly one page, then the end mark and content checksum), including legacy and skippable LZ4 magics, which pierrec/lz4 would accept; and a segment whose commit grows the database without carrying every new page (except the lock page), which ltx would apply by zero-filling (ErrUncarriedGrowth; a session whose capture would produce one fails rather than writing it). Every object this project ever wrote is unaffected: ltx v0.5.1 (frame format, 2026-07-30 to 2026-09-26, verified at every page size 512 to 65536) and v0.5.3 (block format) are the only versions shipped. internal/ltxio/guard_test.go: TestDecodeRejectsOversizedPageFrame, TestDecodeRejectsIllegalPageSize, TestFrameGuardPinsTheV051FrameShape (each refusal allocates under 8 MiB), TestApplyRejectsUncarriedGrowth, TestCheckGrowth, TestDecodesLTXv051FrameFormat (committed v0.5.1 fixtures materialize byte-identical); internal/ltxio/fuzz_test.go's FuzzDecodeSnapshot, FuzzApplySegments; internal/session/growth_test.go's TestUncarriedGrowthFailsTheSession. Not yet covered by a fixture: a 64 KiB-page or incompressible-page v0.5.1 object (ROADMAP Tier 2). Unreleased: the SQLite database header is validated too, by the shared parser every reader goes through (readDBHeader: the 16-byte magic, a page size that is a power of two in [512, 65536]), so ChecksumDatabase (which a daemon session runs over its replica) and the segment planner's compare of the checkout's header with the shadow's error on a file whose header decodes to a page size of 0 instead of dividing by zero in the lock-page computation, and EncodeSnapshot, StreamChecksum and ApplySegments return the same specific error; internal/ltxio/header_test.go: TestReadDBHeaderValidatesMagicAndPageSize, TestChecksumDatabaseRejectsBadHeader, TestEncodeSnapshotRejectsBadHeader, TestApplySegmentsRejectsBadStartHeader |
| Object-granular reachability GC (mark every live branch's resolved chain at head + every checkpoint, transitively through base pointers; two-phase tombstone → grace → re-mark → sweep per object) | shipped-and-tested | Rewritten for copy-on-write in v0.2.0 (internal/ops/gc_test.go, gc_chain_test.go, gc_batch_test.go); counts objects, not lineages; sweep batch-deletes via store.BatchDeleter on S3 (v0.2.2); the never-delete-above-a-live-head compensating rule is epoch-aware since v0.2.4 (a provably fenced orphan is swept once its tombstone clears grace); GC fails closed, with offshoot_gc_errors_total as the alertable signal (v0.2.1) |
| TTL reaping (from the later of last-touch and lease expiry) | shipped-and-tested | internal/ops/reap_test.go, reap_cas_test.go |
Layout versioning (offshoot.json, refuse-newer-than-known) |
shipped-and-tested |
Storage backends
| Guarantee / feature | Status | Notes |
|---|---|---|
| Local directory backend | shipped-and-tested | Quickstart's default; no bucket required |
Local backend removes an epoch directory, then its lineage directory, when a delete empties it; writers retry their create (bounded, 32 attempts) while the directory keeps vanishing underneath them; List skips a directory a concurrent delete removes mid-walk |
shipped-and-tested | internal/store/local_dirs_test.go: TestDeleteRemovesEmptiedEpochAndLineageDirectories, TestDeleteLeavesNonEmptyDirectories, TestDeleteKeepsRefsAndTopLevelDirectories, TestDeleteIfRemovesEmptiedDirectories, TestDeleteObjectsRemovesEmptiedDirectories, TestDeleteKeepsDirectoryHoldingALiveLock, TestPutIfRacingDirectoryRemovalSucceeds, TestPutRetriesWhenItsDirectoryVanishes, TestListAfterDirectoryRemovalIsCorrect, TestWriteSurvivesEpochAndLineageVanishingRepeatedly, TestLockOutlastsTheRetryCapUntilItsDeadline, TestListSkipsADirectoryRemovedMidWalk. Bounds the epoch directories a GC sweep or a failed checkpoint attempt leaves behind; a directory still holding a kept checkpoint is untouched (ROADMAP) |
| RustFS | shipped-and-tested | rustfs/rustfs:1.0.0 (digest-pinned in ci.yml), make test-s3 passes the conformance suite for real on every PR (see README's provider table) — probe, full conformance, and the real multipart-upload subtest whose CompleteMultipartUpload precondition/checksum handling is the one thing the in-process fake can never substitute for. CI's local provider since v0.2.10 |
| MinIO | shipped | Verified through v0.2.9 (minio/minio:latest, last green 2026-09-10); MinIO withdrew its community images (Docker Hub, quay.io) and binaries (dl.min.io) in 2026, so nothing re-verifies it. Same code path as RustFS/AWS — downgraded from shipped-and-tested for honesty, not because anything broke |
| AWS S3 | shipped-and-tested | TestS3RealProvider (probe + full conformance + the multipart subtest) passes against a real AWS S3 bucket (us-east-1; nightly in CI since 2026-09-25) — including the CompleteMultipartUpload precondition/checksum semantics the in-process fake cannot exercise |
| Google Cloud Storage | not supported | GCS's S3-interop API has no conditional writes; the attach probe refuses it outright rather than degrading |
Daemon and durability
| Guarantee / feature | Status | Notes |
|---|---|---|
| Live WAL capture (continuous, foreign-connection-safe) | shipped-and-tested | internal/capture/torture_test.go — kill -9 mid-write, repeated, verifies checksum + state equivalence after every bounce |
| Incremental LTX segments (flush ships only changed pages) | shipped-and-tested | |
| Bounded replay (snapshot every 16th flush; a read applies ≤1 snapshot + 15 segments) | shipped-and-tested | Cadence is Options.SnapshotEvery in the embeddable session library; offshoot serve -snapshot-every N exposes it to the daemon (Milestone 4 Task 6a) — see the Resource-behavior section below. The bound survives copy-on-write sharing via the two snapshot floors (v0.2.0): a shared child's counter seeds from its durable divergence, and a fork already at the depth bound materializes instead of sharing; the daemon's fork floor tracks its configured -snapshot-every (v0.2.1), while an at-rest CLI fork always uses the library default of 16 (documented in v0.2.4) |
Explicit flush durability (session status reports durable-through txid) |
shipped-and-tested | Durability advances on an explicit flush/checkpoint and on the daemon's -flush-every timer (default 30 s; 0 disables); nothing else ships to the store between them |
Background flush interval (serve -flush-every, on by default) |
shipped-and-tested | internal/session/flush_test.go (TestAutoFlushShipsWritesWithoutManualFlush, TestIdleAutoFlushWritesNothing, TestAutoFlushFailureSurfacesAndRecovers), internal/daemon/lifecycle_test.go (TestServeFlushesAutomaticallyWithoutAnExplicitFlushOp). Default 30s, 0 disables; bounds data loss on daemon crash to at most one -flush-every interval — see README's What a flush costs. One daemon-wide cadence applied to every session it opens — see "Per-session FlushEvery override" below for what this is not |
| Settling-flush checksum-compare suppression (skip the mandatory first-open full snapshot when content is provably unchanged since the last one) | shipped-and-tested | rebaseline's very first call (internal/session/session.go) skips the settling flush only when ALL of: the checkout Open received was already proven byte-identical to the branch's head AT OPEN TIME (ops.CheckoutProven's .sum sidecar clean-and-current fast path, cleanAtOpen) AND the checksum RECORDED IN THAT SAME SIDECAR (ops.CheckoutResult.PostApplyChecksum — the LTX postApplyChecksum the checkout embodied when the sidecar was last stamped, by Checkout/Checkpoint/Rollback/Promote, or by a session's own clean Close, see the row below) exactly equals what rebaseline freshly computes from the replica once the engine's REAL startup rebase actually runs. The second condition is load-bearing, not redundant: Open can return to its caller before that real rebase finishes (see Session.rebaseline's doc comment), so a write landing in that window is folded in by the rebase's own checkpoint without ever passing through Apply; the checksum comparison is what catches exactly that case and forces a real settle instead of silently losing the write. Reading the checksum out of the LOCAL sidecar — rather than fetching it fresh from the store, which an earlier, since-reverted version of this fix did — costs Open NO store read at all beyond the two tiny ref-metadata Gets it already needed (TestReadOnlySessionWithCleanCheckoutMakesNoStoreWrites asserts exactly 2, never a Get on the head object itself, which the earlier version paid unconditionally — a full download whenever the head was a snapshot, permanently for a read-only branch that never flushes). Trust-boundary note: this reads the sidecar's recorded checksum as trusted input the same way every other same-host decision in this codebase already does (the checkout directory, .sum sidecar included, is fully trusted under the same-host/same-user unix-socket threat model — see docs/reference.md's daemon-ops threat-model note) — but it widens the CONSEQUENCE of a forged or corrupted local .sum sidecar, not the trust boundary itself: before this suppression existed, a bad sidecar could only make a checkout serve stale local bytes (a read-side effect, confined to this host); now, since a suppressed settle skips the real re-verification snapshot and lets subsequent ordinary flushes encode deltas straight off whatever the replica already held, a forged sidecar can seed what becomes the branch's durable baseline in the store, not just what's served locally. A sidecar that never recorded a checksum (older format) reads back as absent (0, fail-toward-settling) rather than being trusted — TestSettleStillHappensWithOldFormatSidecarLackingChecksum. A first-ever open, or one against a dirty/stale checkout, still settles exactly as before. internal/session/flush_test.go: TestReadOnlySessionWithCleanCheckoutMakesNoStoreWrites, TestSettleStillHappensWhenCheckoutWasStaleAtOpen, TestSettleStillHappensWithOldFormatSidecarLackingChecksum, TestSettlingSuppressionCatchesRaceWindowFold (white-box pins the race-window case directly; fails against the pre-fix cleanAtOpen-only gate), TestCleanAtOpenSessionsFirstRealFlushIsASegment (the suppressed session's first REAL flush, once one happens, is a segment continuing directly from the pre-session head — a previously-unreachable shape); M2's TestFlushLoopFlushesRebaseFoldedContentWhenOtherwiseIdle and TestFlushLoopRetriesAfterRebaseDuringUpload stay green unmodified — a mid-session rebase-on-divergence is untouched by this change. BenchmarkSessionOpen (internal/ops/fork_bench_test.go), re-run: 64MB db ~30.0ms/op, 512MB db ~199.5ms/op, B/op/allocs/op flat across that 8x size range — see docs/benchmarks.md |
Sidecar refresh on clean Close (rewrite the checkout's .sum sidecar to the branch's current head txid — and its LTX checksum, see the row above — when a session closes cleanly, not just on Checkout/Checkpoint/Rollback/Promote) |
shipped-and-tested | Session.Close now re-stamps the sidecar (ops.StampSum, including PostApplyChecksum = s.flushChecksum — the row above is what actually reads it back on the next Open) when the close is provably clean: no session error, nothing left unflushed (autoFlushPending()), at least one flush actually succeeded, the branch head hasn't moved past what this session flushed, the replica was never rebuilt by anything beyond its own mandatory startup rebase (Session.singleStartupRebase), AND — the stamped hash itself — the capture engine's OWN post-shutdown fingerprint (capture.State.Clean/MainHash, persisted only when the engine's shutdown fully verified drain+checkpoint(RESTART)+no-WAL-race+checkpoint(TRUNCATE)) is reused directly rather than independently re-derived; any of shutdown's four early-return paths (a foreign write racing the final checkpoint, most directly) leaves Clean=false and the sidecar is left unstamped. Split around the capture engine's own shutdown join: the pending decision needs the engine alive (a final DrainNow), the physical stamp needs the engine's shutdown to have already run (successfully or not — its verdict is exactly what gates this). internal/session/flush_test.go: TestCloseRefreshesSidecarSoReopenCleanSkips, TestCloseAfterFailedFlushDoesNotStampSidecar, TestSidecarNotStampedAfterMidSessionRebase, TestCloseDoesNotStampSidecarAfterUnverifiedShutdown (forces a real shutdown-verification failure via capture.ShutdownRaceHook; fails against the pre-fix version that independently re-quiesced+re-hashed instead of consulting capture.State); internal/capture/engine_test.go: TestEngineShutdownLeavesUnverifiedStateAfterRacedRestart |
| Clean-and-current checkout served without chain validation (an intentional, accepted tradeoff of the clean-skip fast path above, not a defect) | shipped-and-tested (documented tradeoff) | Once a checkout's .sum sidecar is clean-and-current — whether stamped by Checkout/Checkpoint/Rollback/Promote (Milestone 2 Task 1) or, as of the row above, by a session's own clean Close — the NEXT Checkout call for it is served straight from disk without ever touching the object store's chain, including when that chain has since been corrupted or had a member deleted out from under it. This widens (in reach, not in kind) with the sidecar-refresh row above: it now also applies after an ordinary session's clean close, not only after the four ops entry points. internal/session/flush_test.go's TestCleanCheckoutServedWithoutChainValidationAcrossClose pins this directly (positive test: a corrupted chain does NOT fail a clean-skip Checkout); TestMissingSegmentIsLoud (internal/session/segment_stress_test.go) was updated to remove the checkout+sidecar before asserting a missing chain member fails loudly, since it otherwise now legitimately clean-skips past exactly the corruption it means to detect — controller sign-off granted on this modification |
A branch's open serialized against its own close in the daemon |
shipped-and-tested | opClose keeps the session's slot as a closing marker until Session.Close returns (internal/daemon/server.go's closeSlot). An open of the branch waits up to 15 s (closeWaitBudget), one deadline across every wait, then takes a fresh epoch, or fails with is still closing; retry. A duplicate close waits for the first and returns its result. open returns a session_id (128 random bits, newSessionID), and a close that sends it back (the SDKs' Session.close() do; offshoot session close does not) acts only on that session, open or closing, so a close retried after its session has closed fails with session <id> on <db>@<branch> is not open instead of closing a session opened on the branch since. The lease epoch could not serve: rollback, promote, compact and a re-created branch all restart it, so a later session can land on an epoch an earlier one already had. Flush, fork, promote, rollback, compact, checkout and destroy refuse a closing branch with a retryable error. branches reports closing (it outranks error), status lists the session with state: "closing", /healthz and the session gauges count open sessions only, and session_closed fires only after the slot is gone, carrying holder and epoch, published in the same critical section that frees the slot, so it always precedes a reopen's session_opened; an open that Shutdown interrupts publishes it for the session it self-closes. Shutdown waits for every close, including ones opClose started, and never closes a session twice; a second Shutdown waits for the first; serve exits after the shutdown op only once Shutdown has returned (Server.WaitShutdown), with an error if any close's release failed or if a janitor pass was still running after half the shutdown budget (sessions are closed regardless; since 2026-10-10 every close runs concurrently). Shutdown removes the socket only by closing its listener, before any session closes, so a serve started on the same path while the old daemon is still closing sessions keeps its socket. Session.Close renews the lease until just before it releases it, and retries a failed release up to three times; a session that fails stops renewing at once. MCP's offshoot_checkout/offshoot_checkpoint/rollback/promote/destroy refuse a closing session with "closing; retry". Each daemon session now holds a lease holder of its own (session:<host>/<pid>/<8 hex>, a fresh nonce minted per session.Open — see the per-session lease holders row below); a reopen after a close whose release failed is refused, not granted under the closed session's holder, until that orphaned lease lapses on its own or is freed by its exact holder with offshoot lease release --holder, so its session_opened never repeats a pair a session_closed already carried; a retried close carrying the closed session's session_id still leaves the reopened session open. Tests: internal/daemon/closing_test.go's TestReopenDuringCloseWaitsAndGetsAFreshEpoch (the review's reproduction, deterministic through session.CloseReleaseHook; before the fix the reopen returned under the closing session's epoch and the ref's holder was then cleared), TestReopenDuringCloseTimesOut, TestConcurrentReopensDuringCloseOpenOnce, TestCloseOnClosingSlotWaits, TestStaleCloseLeavesTheReopenedSessionOpen (before the fix, a close retried after a reopen closed the reopened session; matching by lease epoch instead still did after a rollback or a failed release, which reopen at the closed session's epoch), TestFlushOnClosingSlotIsRefused, TestForkFromClosingSourceIsRefused, TestRollbackDuringCloseIsRefused, TestSessionClosedEventFollowsSlotRelease, TestSessionClosedPrecedesTheReopen (holds the event bus's lock while the close finishes: once closeSlot is blocked publishing, the slot must still be closing), TestShutdownDuringClose, TestShutdownOpWaitsForSessions, TestSecondShutdownWaitsForTheFirst, TestShutdownReturnsAFailedRelease, TestShutdownLeavesTheNextDaemonsSocketAlone, TestReopenWaitingWhenShutdownBeginsIsRefused, TestBranchesReportsClosing, TestStatusReportsClosing, TestSessionCountAndGaugesExcludeClosing; internal/daemon/server_test.go's TestShutdownDuringInFlightOpenLeavesNoLease (one session_closed, with the session_opened holder and epoch, from the self-close); internal/session/growth_test.go's TestUncarriedGrowthFailsTheSession (renewal stops once the capture engine fails the session); internal/session/close_test.go's TestLeaseStaysLiveThroughClose, TestReleaseLeaseRetryPolicy, TestCloseRetriesRelease, TestClosedAndFencedTransitionsCarryHolderAndEpoch; internal/mcp/daemon_test.go's TestHealthySessionExcludesClosing, TestCheckoutRefusesClosingSession, TestCheckpointRefusesClosingSession, TestRollbackRefusesClosingSession; cmd/offshoot's TestServeWaitsForShutdownOpToReleaseLeases, TestSessionStatusLineShowsState; the SDKs' status state tests (Python test_status_passes_session_state_through and test_rollback_promote_status, TypeScript status: session state is passed through, and absent from an older daemon and rollback, promote, status); the SDKs' close-session-id tests (Python TestSessionCloseSessionID and test_stale_session_close_leaves_a_reopened_session_open, TypeScript session close sends back the session id its open returned, and none to an older daemon and a stale session close leaves another client's reopened session open, each with a rollback case). make test-torture and make test-sdks green |
| Per-session lease holders (each daemon session's own holder, never shared with another session on the same host/pid) | shipped-and-tested | Each session.Open mints session:<host>/<pid>/<8 hex> (ops.NewSessionHolder), a fresh nonce per call, so no two sessions — even a reopen of the same branch by the same daemon — ever share a holder or, through AcquireLease's same-holder self-renew rule, an epoch. offshoot lease list, offshoot status and the daemon's branches op show it; offshoot lease acquire is unaffected (still the bare <host>/<pid>), as is an at-rest checkpoint's checkpoint:<host>/<pid>/<8 hex> (unchanged, built on the same generator). A refusal under a live lease names the holder kind: (an open daemon session) for a session: holder (advice: close the session, or, if the daemon that held it has exited, lease release --holder <its exact holder>, with no "never while that daemon still runs" caveat, since the nonce means that release can only free that one session), vs. (an open daemon session, or 'offshoot lease acquire') for a bare one (same caveat as before --holder existed). A reopen after a close whose lease release failed is refused — not granted under the orphaned lease in place — for up to the lease TTL (30 s default); the close's own error already names that holder, when its lease lapses (rounded up to the second), and offshoot lease release <db>@<branch> --holder <holder>. Two daemons in look-alike containers (same hostname, offshoot at PID 1) used to renew each other's live leases and share an epoch, a split brain, and a daemon restarted under its dead predecessor's hostname and PID 1 picked that lease (and its epoch) up in place; neither can happen now. The daemon mints each session's holder itself and records, per branch, the exact holder of any session of its own whose lease release failed (a close's, or an open that failed after its acquire); only a lease held by exactly that holder gets "held by an earlier session of this daemon whose lease release failed ... free it now with 'offshoot lease release session: holder, including a twin daemon's under the same session:<host>/<pid>/ prefix or a restarted daemon's predecessor, gets the conditional advice shared with the checkpoint refusal (ops.LeaseHeldAdvice): close that session, or, if its daemon has exited, lease release --holder it. The refusal is read while opOpen's reservation of the branch still stands. An acquire whose write landed but reported failure (an S3 SDK retry answering 412, or a lost response) is adopted from the re-read ref — safe only because the holder is unique to that one Open call; a re-read that itself fails instead names the latest possible expiry (now + the requested TTL) and the same --holder advice, except when the re-read fails with "not found," which returns the original error unchanged. offshoot lease release --holder H is a compare-and-swap on the exact holder: a mismatch (including an unleased ref, which never equals a non-empty H) refuses with lease release: <db>@<branch> is held by <current>, not H; nothing released (or ... is not leased; nothing released) and writes nothing, and an empty H is a usage error; lease release, with or without --holder, now prints released <db>@<branch> held by <holder>. internal/ops/lease_test.go: TestNewSessionHolderFormatAndUniqueness, TestRefuseIfHeldNamesTheHolderKind, TestLeaseHeldAdviceIsTheCheckpointRefusalsAdvice, TestLapseTimeRoundsUp; internal/session/session_test.go: TestOpenUsesAUniqueSessionHolder, TestSecondOpenWhileFirstIsLiveIsRefusedAndLeavesTheLeaseAlone, TestOpenAdoptsAnAcquireThatLandedButReportedFailure, TestOpenSettleReadFailureNamesTheExpiry, TestFailedCheckoutReleasesWithRetry, TestFailedRebaselineReleasesWithRetry, TestReopenRefusedByCheckpointHolderSaysInProgress; internal/session/close_test.go: TestCloseRetriesRelease; internal/daemon/branchstate_test.go: TestOpenRefusedByOwnOrphanedLeaseNamesTheHolder, TestTwinDaemonPrefixGetsGenericAdvice, TestOrphanedHolderRefusalNeverNamesALiveSession, TestStatusHolderIsTheSessionsOwn; internal/store/lease_test.go: TestReleaseLeaseByHolderMatches, TestReleaseLeaseByHolderMismatchWritesNothing, TestReleaseLeaseByHolderOnUnleasedRefRefuses, TestReleaseLeaseByHolderRejectsAnEmptyHolder; cmd/offshoot/main_test.go: TestLeaseReleaseWithHolderMatchingFreesIt, TestLeaseReleaseWithHolderMismatchExitsNonZero, TestLeaseReleaseHolderFlagParsing |
A branch's open serialized against the daemon's at-rest refreshes |
not implemented — known race | The daemon's at-rest checkout, rollback, promote and compact handlers check for a claim on the branch (refuseIfClaimed) without reserving it. An open landing while one runs starts a capture engine while that op is still re-materializing the same checkout; a re-materialize can rename a new file over the checkout the new engine opened, and that engine then fails at once with dbfile.ErrReplaced (its SQLite locks are not dropped: the shadow clone goes through dbfile). Pre-existing, found in the FD-budget review; the close half of the same finding is fixed (row above). The fix is to reserve the slot for the at-rest ops; until then, wait for the at-rest op to return before opening the branch |
Per-session FlushEvery override (a distinct cadence per session open call, over the wire) |
deliberately deferred (YAGNI) | -flush-every is one setting for the whole daemon (internal/daemon/server.go's SetFlushEvery), applied to every session it opens; the daemon protocol's open op has no field for a caller to request a different cadence for just one session. Not scoped in Milestone 2 or planned elsewhere yet — revisit if a real caller needs mixed cadences on one daemon |
| Connection contract enforcement (live WAL-mode probe, rollback-journal detection) | not implemented — documented assumption | No live polling or journal watcher exists; the contract's WAL-only and shared-kernel clauses are trusted, and a mid-session violation is caught only by the restart-time divergence check (WAL-emptiness + main-file hash continuity in internal/capture's resume gate), which marks the checkout dirty rather than pretending continuity. Live enforcement is future work |
Reflink/clonefile fork, server-side S3 copy fork (design spec's ~40ms figure was the target this work aimed at, not a number reproduced below — see notes). Since v0.2.0 this fast path serves the materialize cases only — the floor-tripped fork, rollback or promote, --materialize, and compact — because the default fork shares instead of copying |
shipped-and-tested (single-snapshot-chain window) | Tasks 6a+6b of Milestone 2 — ops.Workspace.Fork's copySnapshotToNewLineage copies the source snapshot object directly (local: internal/reflink, clonefile(2) on darwin / FICLONE on Linux, silent plain-copy fallback otherwise; S3: store.S3.CopyObject, a real server-side CopyObject API call, no download or re-upload through this process) instead of materializing and re-encoding, when the source checkpoint's chain resolves to exactly one snapshot. Falls back to the pre-6a path once a daemon session has flushed segments past the last snapshot. S3's former ≤5GB CopyObject gate is gone: since v0.2.4, sources over 5 GiB take a multipart server-side copy (UploadPartCopy) up to S3's 5 TiB per-object ceiling, and store.ErrCopyUnsupported fires only beyond that. Measured (MinIO-local, not an AWS claim) 512MB fork ~198ms on APFS (was 2.87s pre-6a) and ~1.03s over S3 (was 4.57s pre-6b); ~9.3ms for the local copy alone in isolation from an unrelated pre-existing O(size) check, the closest this suite gets to the design spec's ~40ms figure directly — see docs/benchmarks.md |
Async fork-point upload (pending marker + fork pin on GC) |
deliberately deferred — and largely superseded by copy-on-write | Scoped in Milestone 2's original fork-performance bullet but not built: making the upload async introduces a new pending chain state that readers, GC, and fencing would all have to understand mid-upload. Since v0.2.0 the common-case fork shares and uploads nothing at all — there is no fork-point upload to make async and no pending window to caveat (noted in the v0.2.1 changelog). Only the materialize fallback (fork-floor trip) still pays a synchronous snapshot copy, made cheap by the reflink/CopyObject fast path above |
TTL, GC, and janitor
| Guarantee / feature | Status | Notes |
|---|---|---|
offshoot serve -reap-every / -gc-grace (janitor loop) |
shipped-and-tested | -reap-every 0 disables the janitor; offshoot gc remains available on demand |
| Protected branches never reaped | shipped-and-tested | internal/mcp/tools_test.go's TestReapOnceNeverReapsProtected; internal/ops/security_qa_test.go pins the ops-level reap |
| Live lease always defers reaping | shipped-and-tested | |
CAS-conditional ref delete (close the Destroy GetRef→lease-check→delete TOCTOU) |
shipped-and-tested | Milestone 4 Task 6b — ops.Workspace.Destroy (internal/ops/gc.go) CAS-writes a Deleting claim on the ref (store.Ref.Deleting/DeletingAt) before it does anything irreversible; store.AcquireLease refuses outright once it sees the claim (store.ErrDeleting), closing the window in which a lease acquired between Destroy's initial GetRef and its actual delete could have its branch deleted out from under it — the exact race an earlier design review documented for the M2 Destroy path. Scoped per the task's own timebox as a sibling claim field, not a unification with Reap's existing Reaping claim: Reaping's CAS mechanics are torture/race-tested and deliberately untouched here (internal/ops/reap.go/reap_test.go/reap_cas_test.go all pass unmodified). --force bypasses the protected/live-lease pre-checks only, never the claim itself — a lease that wins the underlying CAS race still survives a concurrent forced destroy. Backend split: Local gets a TRUE conditional delete (Local.DeleteIf, the same per-key lock file PutIf already uses) on top of the claim, exposed via Store.DeleteRefIf and the new store.ConditionalDeleter optional-capability interface; S3's DeleteObject has no compare-and-delete precondition at all, so DeleteRefIf falls back to an unconditional delete there and the CAS-written claim marker is the entire safety mechanism on that backend, documented as such rather than faked. A crashed Destroy (claimed, never deleted) self-heals via ops.Workspace.ClearStaleDeleteClaims (age-based, 30s — no TTL/deadline concept exists for a delete claim the way Reap's ReapDeadline gives it one), wired into the daemon janitor's janitorTick and the CLI offshoot gc on the same "report and press on" convention as a reap failure. Tests: internal/ops/destroy_claim_test.go's TestConcurrentDestroyAndAcquireLeaseHaveExactlyOneWinner (20-iteration race loop, mirroring TestConcurrentTouchAndReapHaveExactlyOneWinner: exactly one of Destroy/AcquireLease wins, the branch is never left half-deleted with a live lease on it), TestForceDestroyStillClaimGuards (same race with force=true), TestDestroySelfHealsStaleDeletingClaim (stale claim cleared, branch untouched, AcquireLease succeeds again; a fresh claim survives a pass), TestRenewingHolderSurvivesAStrandedDestroyClaim (a holder renewing every TTL/3 under a claim a killed destroy left: every renewal writes over the claim and leaves it set, so the lease stays live through the 30 s the claim stands — the branch reads active, an unforced destroy is refused, and after the clear another acquirer is refused and the holder renews); internal/ops/destroy_renewal_test.go's TestStrandedClaimCostsNoRenewalSlack (the same with one renewal failing and the next late, or two failing: the slack is the two missed renewals a lease has without a claim), TestDestroyDeletesPastARenewalOverItsClaim (a renewal between the claim and the delete: a forced destroy deletes on Local, re-reading and deleting again up to 4 attempts, and on S3; an unforced one is refused as a live lease on Local; a flush, touch, release, another destroy's claim, or a renewal before every attempt fails it as a lost race and unwinds its own claim, leaving another destroy's), TestDestroyForceAbortsACheckpointWhoseRenewalLandsInItsWindow; destroy_claim_test.go's TestFailedDestroyLeavesAnotherDestroysClaim (local and fake S3: a destroy that fails after another destroy's claim lands over its own leaves that claim, so an acquire is still refused), TestDestroyRefusesABranchAnotherDestroyHasClaimed (local and fake S3: a second destroy --force while the first sits between its claim and its delete is refused with ErrDeleting and writes nothing; the first deletes), TestDestroyTakesOverAClaimNoDestroyIsUsing (a stale, unreadable, or future-dated claim does not refuse a destroy), TestDeleteClaimsStampedAheadOfTheClock (the janitor clears a claim stamped more than a minute ahead of its clock; one within the minute is live: a destroy is refused under it and the janitor leaves it), TestFailedDestroyUnwindsPastARenewal (a renewal that wins the unwind's first compare-and-swap does not strand the claim; the retried destroy goes through), TestDestroyGoesOnWhenItsClaimLandedButReportedFailure (fake S3, plus a local-unsettled variant that runs the same path over Local's conditional delete through a wrapper hiding its store.SettledWriter, since a real local claim write that reports failure never lands: a claim write that landed in time but reported a lost race or a timeout, with or without a renewal over it before the destroy hears back, is the destroy's own; it deletes the branch and the holder's next renewal finds it gone), TestDestroyUnwindsALandedClaimItCannotCarryThrough (fake S3 and the same local-unsettled wrapper: a landed claim followed by a touch, or, unforced, by a renewal that makes the lease live again: the destroy fails as a lost race or a live lease, clears its own claim, and the retry goes through), TestDestroyWaitsForAClaimWriteStillInFlight (fake S3: a claim write that reported a 409 or a timeout and lands after the destroy's first re-read is the destroy's own and it deletes; one that never lands fails it saying the claim may still land; a 412 after another write is reported at the first re-read), TestDestroyReportsAFailedClaimWriteAtOnceOnALocalStore (a local store settles its writes, store.SettledWriter: a failed claim write, even one that leaves the ref as it was, is reported at once with no re-read and no warning that it may still land; the retry goes through; internal/store's TestOnlyLocalSettlesItsWrites), TestDestroyDeletesOnlyWhileItsClaimStands (under synctest, over Local behind a wrapper that, as S3 does, deletes unconditionally and does not settle its writes: a claim write held up until the claim is stale, or within 10 s of it, whether it succeeds or reports a timeout and lands during the settle reads, sends no delete, clears its claim and fails retryable, so the janitor cannot clear the claim and a new acquire lose the branch to that delete; the retry, and a destroy that reaches its delete in time, delete the branch with acquires refused throughout), TestDestroyChecksItsClaimsAgeAfterTheQuiesce (under synctest: a checkout quiesce held up until the claim is within 10 s of stale sends no delete, clears the claim and fails retryable; one that ends in time deletes), TestDestroyDeletesPastAReapingClaimClearedUnderIt (a reaper's unwind of its own reaping claim between the destroy's claim and its delete does not fail it); reap_test.go's TestReapLeavesABranchADestroyHasClaimed and TestReapLeavesABranchADestroyClaimsMidReap (an expired branch under a live destroy claim is left to that destroy, with no reap error and no write to its ref, even when the pass had claimed it for reaping first; local and fake S3, a real destroy, forced or not, then deletes it), TestReapWritesNothingUnderADestroyClaimOverAStaleReapingClaim (local and fake S3: a stale reaping claim, deferred or with no deadline, under a destroy's live claim is not cleared until that destroy ends; the ref's etag is unchanged and the destroy deletes), TestReapReapsABranchOnceAMidReapDestroyUnwinds (the reaping claim left under that destroy's claim does not strand the branch); internal/ops/destroy_session_renewal_test.go's TestForceDestroyEndsASessionThatRenewsOverItsClaim (a real session's renewal over the claim: the destroy deletes and the session ends); internal/session/renew_test.go's TestRenewalKeepsTheLeaseUnderAStrandedDestroyClaim (a real session renewing on every tick under the claim, never below half its TTL, which then flushes); internal/store/local_test.go's TestLocalDeleteIfConditionalDelete/TestStoreDeleteRefIfUsesConditionalDeleteOnLocal (true CAS delete exercised directly); internal/daemon/destroy_claim_test.go's TestJanitorTickClearsStaleDeleteClaim (through a real janitor pass, not the ops call in isolation). go test ./internal/ops ./internal/store ./internal/daemon ./cmd/offshoot -count=1 -race clean; no internal/session/internal/capture changes, so no torture run was required |
Integration surface
| Guarantee / feature | Status | Notes |
|---|---|---|
| CLI (create/checkout/checkpoint/fork/rollback/promote/compact/destroy/touch/protect/unprotect/export/diff/gc/status/path/lease/serve/session/mcp) | shipped-and-tested | See docs/reference.md for every command |
| MCP server, 10 tools, at rest | shipped-and-tested | offshoot mcp; protected-branch rules enforced same as CLI; no offshoot_protect/offshoot_unprotect tool (CLI-only by design, see the Core model section above) |
MCP force gate (offshoot mcp -allow-force, off by default): an agent's force:true against a protected target/branch is refused before any mutation unless the server was started with the flag; offshoot_destroy's live-lease bypass is disabled the same way; the gate fails closed on a transient GetRef error. The gate also reaches a protected branch's own rollback/promote safety fork (<target>-pre-rollback/<target>-pre-promote), which is never itself protected: without -allow-force, offshoot_destroy, TTL-changing offshoot_touch, and offshoot_promote (when the fork is the target) calls on it are refused (guardProtectedSafetyFork) — promoting onto it would repoint (destroy) the undo point just as surely as an actual destroy — and a second offshoot_rollback of the protected branch is refused outright while its safety fork from an earlier rollback still exists, rather than silently replacing it; a plain offshoot_touch that only extends the fork's life is never blocked. This guard fails CLOSED on a transient error reading the target's own ref, since (unlike refuseForceOnProtected) there is no downstream backstop — the fork itself is never protected, so an unguarded ops call would sail through |
shipped-and-tested | internal/mcp/tools_test.go's TestMCPRefusesForceOnProtectedByDefault, TestMCPForceFailsClosedOnTransientGetRefError, TestMCPDestroyNeedsAllowForceForLiveLeaseToo, TestGuardProtectedSafetyFork (covers destroy/touch/second-rollback/promote-onto on a protected branch's fork, all proceeding once -allow-force is set, and the converse on an unprotected branch's fork), TestGuardProtectedSafetyForkFailsClosedOnTransientReadError (a one-shot injected GetRef error on the protected target's ref, or on the rollback safety-fork name itself, refuses rather than proceeding). Behavior change: earlier versions of offshoot mcp honored an agent-supplied force:true unconditionally, same as the CLI's --force; see docs/demo/mcp-walkthrough.md for the refusal firing for real, followed by offshoot_diff and a human-run CLI promote; v0.2.11 |
| MCP forks carry a TTL | shipped-and-tested | offshoot_fork takes ttl (explicit always wins) and falls back to offshoot mcp -default-ttl (default 24h; 0/none disables) when omitted; ttl:"none" overrides even a configured default. The response echoes the applied TTL and computed expiry. Reaping is offshoot mcp -reap-every (see the row below), a reachable offshoot serve daemon's janitor, or offshoot gc by hand |
offshoot mcp -reap-every DURATION|none (default 60s): daemonless TTL reaping |
shipped-and-tested | internal/mcp/tools_test.go's TestReapOnceReapsExpiredForksWhenNoDaemon, TestStartReaperTicksAndStopsOnCancel. Reaps expired forks and self-heals stranded delete claims on this cadence for as long as the MCP process is up — a bare offshoot mcp used to leave TTL expiry entirely to offshoot gc run by hand; this row is the daemonless fallback that changes that. Defers entirely (skips its own pass, logs once) when a real offshoot serve daemon is reachable at the same socket, so it's never a second writer against one store. Runs no GC either way — reclaiming a reaped branch's storage still needs offshoot gc or a running daemon; v0.2.11 |
| MCP rides the daemon (live capture, session-aware checkpoint) | shipped-and-tested | internal/mcp/daemon_test.go; offshoot_checkpoint/offshoot_fork/offshoot_checkout each probe the daemon per call and take its live path when a session is already open (and healthy) on the branch (opened by a harness — no MCP tool opens one itself, i.e. the good path requires a harness-opened session); a fenced session falls back to at-rest with a warning naming its error; no session, or no daemon, and the tool runs exactly at rest as before. offshoot_rollback/offshoot_promote (target)/offshoot_destroy refuse — even with force, which has no effect on this refusal — rather than proceed at rest when the daemon has any session (healthy or fenced) on the affected branch; offshoot_promote's source is not guarded the same way (TestPromoteFromOpenSourceProceedsAtRest) — an open session there doesn't block the promote, but the promoted state is the source's last-flushed head, not any unflushed write in that session. offshoot mcp -socket PATH names the daemon to ride |
MCP registry submission (server.json) |
drafted, not yet submitted | server.json (repo root) is a draft manifest; the registry's current schema will be validated against at submission time |
| LangGraph community-integration listing | drafted, not submitted | a community-integration PR for offshoot.langgraph.ThreadForks is drafted; blocked on PyPI publication (the install command it quotes needs to work) — see the SDK publish-pipeline row above |
| Python SDK (stdlib-only, over the unix-socket lifecycle API) | shipped-and-tested | sdk/python; publish pipeline prepared, actual PyPI publication user-gated (see the publish-pipeline row below and Milestone 3) |
| TypeScript SDK (zero runtime deps) | shipped-and-tested | sdk/typescript; publish pipeline prepared, actual npm publication user-gated (see the publish-pipeline row below and Milestone 3) |
| SDK typed surfaces (typing polish, Milestone 4 Task 7) | shipped-and-tested | TypeScript: _call's internal wire responses are typed (RawResponse/RawBranchInfo/RawCheckpointInfo/RawSessionInfo/RawAck/RawEvent mirroring internal/daemon/protocol.go's Response), not any; _call itself is @internal + stripInternal-stripped from the published .d.ts (still a real, fully-typed method at runtime and for this package's own tests, which compile against source).Verified additive: a dist/client.d.ts before/after diff shows exactly one removal (the now-internal _call line) and no other symbol changed; dist/testkit.d.ts byte-identical.Python: offshoot/py.typed (PEP 561) ships in the wheel ([tool.setuptools.package-data], confirmed via the dry-run wheel's contents); mypy --strict offshoot is clean across all four modules, fixed with annotations only (no behavior changes) — typing.cast() on JSON-derived returns, Mapping[str, str] narrowing, dict/Popen generic args, a module-private _TTL/_Seed type alias each. make test-sdks and make test-pytest-plugin green (as of 2026-10-02 the suites hold 47 Python def test_ methods across tests/test_client.py and tests/test_langgraph.py, 54 TypeScript test(/it( calls across test/client.test.ts and test/testkit.test.ts, and 61 top-level def test_ functions in tests/test_pytest_plugin.py, counted by grep); tsc --noEmit clean; no Go changes |
SDK publish pipeline (.github/workflows/publish.yml: PyPI Trusted Publishing/OIDC, npm --provenance) |
shipped | Triggered by sdk-v* tags or workflow_dispatch; gated on the PUBLISH_ENABLED repository variable, default off — both jobs build the real sdist/wheel/npm-tarball, run twine check, and install-test them regardless of the gate, only skipping the upload step when it's off. That dry-run tier (make dry-run-sdks) runs on every PR via ci.yml's sdks job (verified locally too: python3 -m build + twine check + venv install/import for the wheel, npm pack + tarball install + import() test for the npm package, all green — see the Task 7 report). The upload path itself (pypa/gh-action-pypi-publish, npm publish --provenance) cannot be exercised without live PyPI/npm credentials that don't exist yet — actual publication is user-gated: claiming the offshoot-db PyPI name and @offshoot-db npm scope, configuring Trusted Publishing/NPM_TOKEN, and flipping PUBLISH_ENABLED are all manual, one-time, out-of-band actions (see CONTRIBUTING.md's Release process, ROADMAP.md). Manifests filled out for real publication (sdk/python/pyproject.toml: readme, urls, classifiers, authors, SPDX license; sdk/typescript/package.json: repository/bugs/homepage, files whitelist, prepublishOnly build). SDK version is a single source of truth (sdk/VERSION) checked against both manifests by scripts/check_sdk_versions.py |
| LangGraph integrations | shipped-and-tested | sdk/python-langgraph's OffshootSaver is a real BaseCheckpointSaver companion around the stock SqliteSaver, exercised by make test-python-langgraph against a compiled StateGraph (9 tests, wired into CI through its explicit [test] extra); the core SDK's offshoot.langgraph.ThreadForks keeps an existing checkpointer and maps threads to branches of the agent's separate application database, exercised end to end by examples/langgraph-rewind/ |
List databases (daemon dbs op, CLI offshoot session dbs, SDK dbs()) |
shipped-and-tested | Milestone 3 Task 1 — store.Store.ListRefs's keys, sorted; internal/daemon/metadata_test.go (TestOpDbsListsSortedDatabases, TestOpDbsOnEmptyStoreIsEmptyNotError), sdk/python/tests/test_client.py::test_dbs_lists_every_database_sorted, sdk/typescript/test/client.test.ts ("dbs lists every database sorted") |
Branch/checkpoint metadata and timestamps (Ref.Meta, per-checkpoint CreatedAt/Meta, fork/checkpoint/flush meta params, BranchInfo.touched_at/checkpoints_v2) |
shipped-and-tested | Milestone 3 Task 1 — capped at the ops layer (ops.ValidateMeta: ≤32 keys, keys ≤64 bytes, values ≤512 bytes; internal/ops/ops_test.go's TestValidateMetaCaps and the *RejectsMetaOverCap tests). Fork's meta sets the child branch's Ref.Meta (branch-level lineage — no row-level provenance); Checkpoint's meta sets that one checkpoint's own Meta. Every checkpoint-creating call site (Create, Checkpoint, Fork, Promote, and a daemon session's named flush) stamps CreatedAt (RFC3339 UTC) — internal/ops/ops_test.go's Test*StampsCreatedAt* tests, internal/session/flush_test.go's TestNamedFlushStampsCheckpointMetaAndCreatedAt. No store schema bump: new fields are omitempty, round-trip-and-old-shape-decodes-clean tested exactly like the TTL fields (internal/store/store_test.go's TestRefMetaAndCheckpointFieldsRoundTripAndOldRefsDecode, including a hand-written-JSON old-shaped-ref decode, not just a Go zero-value round trip). Wire compat: BranchInfo's pre-existing checkpoints (names-only) field is untouched; checkpoints_v2 ({name, txid, created_at}) rides alongside it; an old-client-shaped request (no meta field at all, constructed as literal JSON bytes, not a Go zero value) still works against every touched op — internal/daemon/metadata_test.go's TestWireCompatOldShaped*RequestWorks. CLI: fork/checkpoint gain repeatable --meta k=v; SDK parity both languages. Bug fixed in the same pass: Rollback's kept-checkpoint epoch relocation used to silently drop CreatedAt/Meta — internal/ops/ops_test.go's TestRollbackPreservesCheckpointCreatedAtAndMeta |
MCP tool metadata exposure (offshoot_fork/offshoot_checkpoint taking a caller-supplied meta map) |
shipped-and-tested | offshoot_fork/offshoot_checkpoint take meta (TestForkAndCheckpointCarryMeta); v0.2.11 |
| MCP tool annotations (explicit read-only/destructive/idempotent hints, openWorld false) | shipped-and-tested | TestToolAnnotationsClassifyEveryTool; v0.2.11 |
| MCP structuredContent on every success | shipped-and-tested | TestStructuredContentAccompaniesProse; v0.2.11 |
offshoot_touch |
shipped-and-tested | TestTouchExtendsALeasedAttempt; v0.2.11 |
Claude Code plugin + marketplace (plugin/, .claude-plugin/marketplace.json) |
shipped | make check-plugin in CI; hooks advisory only; v0.2.11 |
create --from reach: daemon protocol op, SDK create(..., from_path=...)/{fromPath}, .db-file seeds in both fixtures |
shipped-and-tested | The daemon's create op now accepts an optional path request field (a server-side, absolute path to an existing SQLite file to import via ops.CreateFrom instead of minting an empty database); a relative path is refused with an error containing "absolute", and create with path set is refused over HTTP with daemon: create with path is not available over HTTP; use the local socket — the same same-host path-trust story export already uses (the daemon reads a path the caller names, trusted because same-host/same-user unix-socket access already implies that trust), not the upload-channel alternative once considered; plain create (no path) keeps working over HTTP unchanged. internal/daemon/lifecycle_test.go's TestCreateFromPathImportsAnExistingFile, TestCreateWithRelativePathRefused, TestCreateFromPathToNonSQLiteFileErrors; internal/daemon/http_test.go's TestHTTPCreateWithPathRejectedPlainCreateStillWorks. Python Client.create(db, from_path=None) and TypeScript create(db, { fromPath }) resolve the path to absolute client-side before sending it as path (sdk/python/tests/test_client.py, sdk/typescript/test/client.test.ts). Both fixtures ride the same reach: offshoot.pytest_plugin's offshoot_seed/_SeedFactory and the TypeScript testkit's seedOnce accept a path to an existing SQLite .db file (detected by its SQLite format 3\0 header, never by extension alone), import it via create(..., from_path=...), and fork from the imported init checkpoint with fingerprint db:<sha256 of file bytes> — sdk/python/tests/test_pytest_plugin.py::test_db_file_seed_imports_and_forks_from_init and the TypeScript twin in sdk/typescript/test/testkit.test.ts ("seedOnce: a path to an existing SQLite db file imports it and forks from its init checkpoint"). MCP deliberately does not get this — excluded by design, not deferred: an MCP tool must never import an arbitrary host file named by an agent, so no offshoot_create-style tool exists and no from/path argument was added to any MCP tool. CLI offshoot create <db> --from file remains available too, unchanged; v0.2.11. Unreleased: create and create --from (both through createFromQuiesced) settle their create-only write of db@main as fork does: a write that lands but comes back as an error (the S3 SDK's retry answering 412 to its own first attempt, or a timeout that loses the response) used to delete the snapshot it had just written and report "already exists", leaving db@main naming a lineage with no snapshot; settleRepoint now re-reads the ref, and a match on the minted lineage reports success. Anything else (an existing database, no ref) keeps the snapshot for offshoot gc to reclaim and returns the same database "db" already exists (offshoot status lists databases) or "create db: err" text as before; a re-read that itself fails returns an error saying the write may have landed, naming the kept lineage and pointing at offshoot status. internal/ops/repoint_landed_test.go: TestCreateLandedWriteKeepsSnapshotAndIsReadable, TestCreateOntoTakenNameLeavesSnapshotForGC |
MCP session open/close (an offshoot_open/offshoot_close tool pair) |
deliberately deferred | Milestone 2's "MCP rides the daemon" bullet named this gap and pointed at Milestone 3's fixture plugin as the eventual lifecycle owner; that fixture (offshoot.pytest_plugin, the row above) and its TypeScript counterpart (testkit, the row below) both now EXIST and ship in this milestone — but neither is an MCP tool, and no MCP open/close tool pair was built alongside them. The reasoning stands exactly as amended: an MCP-opened session has no natural owner responsible for closing it — a bare tool call has no equivalent of a fixture's guaranteed teardown or a with block's guaranteed exit, so an agent that opens and then gets interrupted, crashes, or simply forgets to call the matching close tool leaks a lease and a branch, precisely the class of leak this whole milestone's TTL/background-flush work exists to prevent. The good path for MCP daemon-mode capture remains what docs/reference.md's offshoot mcp entry and the README's MCP section already document: a harness (the SDKs, offshoot session open, or a custom loop) opens the session and stays responsible for closing it; MCP's checkpoint/fork/checkout tools ride that session without ever opening or closing one themselves. See docs/recipes/claude-agent-sdk.md for the concrete "open before, close after" wiring this implies for an agent harness |
Export (ops.Workspace.Export, CLI offshoot export, daemon export op, SDK export() both languages) |
shipped-and-tested | Milestone 3 Task 2 — materializes any checkpoint (or head) to a plain SQLite file anywhere, with zero ongoing relationship to the store: no .sum sidecar, no lease. Refuses to overwrite an existing destination unless force; the write is atomic (temp file in the destination's OWN directory, renamed into place only once every chain member is fetched and verified — reuses materializeChainAt/ltxio.MaterializeChain's existing atomicity, so a failed export never leaves a partial file, pinned by internal/ops/export_ops_test.go's TestExportAtomicOnMidWriteFailureLeavesNoTempOrPartialFile, which corrupts a segment's checksum mid-chain and asserts the destination directory ends up completely empty). Reads the branch's last DURABLE state from the store, never a live session's checkout — an open daemon session's unflushed writes are NOT in the export, asserted directly over the wire by internal/daemon/export_test.go's TestOpExportMissesUnflushedSessionWrites (write through a live session's checkout without flushing, export, assert the row count matches the pre-write durable state; then flush and export again, assert it now includes the write). Daemon export op requires an ABSOLUTE destination path (same-host/same-user unix-socket trust model — see docs/reference.md's daemon-ops section) and is deliberately NOT guarded by the open-session refusal checkout/rollback/promote use, since it never touches the live checkout. CLI parses the db[@branch[@checkpoint]] triple-@ target form (ops.ParseExportTarget) |
Read-only historical checkout (ops.Workspace.CheckoutAt, CLI offshoot checkout --at --read-only, daemon checkout-at op, SDK checkout_at()/checkoutAt()) |
shipped-and-tested | Milestone 3 Task 2 — materializes a NAMED checkpoint (no head alias) into a dedicated cache path, <store-root>/checkouts-ro/<db>/<branch>@<checkpoint>.db, chmod 0444, distinct from and never touching the writable checkouts/<db>/<branch>.db path or its .sum sidecar/lease — safe to call alongside a live daemon session on the SAME branch (internal/daemon/export_test.go's TestOpCheckoutAtSafeAlongsideOpenSessionOnSameBranch), unlike checkout/rollback/promote's refuseIfClaimed guard. Nothing in this codebase opens a checkouts-ro file through a live capture engine, so internal/dbfile's stray-close lock hazard doesn't apply to it either. A repeat call with force=false is a pure cache hit — no store access at all, not even a GetRef — because a checkpoint's content is immutable once created; destroy removes the branch's checkouts-ro/<db>/<branch>@*.db entries with the ref, so a branch destroyed and re-forked under the same name (every fork's auto checkpoint is named fork) is materialized at its new fork point, not served the dead branch's state (internal/ops/export_ops_test.go's TestCheckoutAtWithoutForceIsACacheHitUntilTheBranchIsDestroyed and TestCheckoutAtAfterDestroyAndReforkServesTheNewBranch; until 2026-10-10 the cache survived destroy and the old test asserted it); force=true re-reads the store and can fail once the checkpoint is gone (TestCheckoutAtForceRematerializesAndCanFail) — see README's Resource behavior |
offshoot.pytest_plugin fixture plugin (offshoot-db[pytest], sdk/python/offshoot/pytest_plugin.py, pytest11 entry point) |
shipped-and-tested | Milestone 3 Task 4 — the seed-once-fork-many paved road: offshoot_daemon (session-scoped: locates the binary via OFFSHOOT_BIN env or PATH, pytest.skips with install instructions when neither has one; temp store + socket; terminates at session end), offshoot_db (session-scoped NAMED-SEED FACTORY — offshoot_db(name="default", seed=None), memoized per name; creates eval-{name}, runs seed — a callable(path) or a SQL string — checkpoints it seed; seed=None falls back to the offshoot_seed ini option, a path to a .sql file, the zero-code default-seed case), offshoot_fork (function-scoped — offshoot_fork(seed_handle=None), forks a fresh worker-safe-named branch (t-{worker}-{testname-hash}-{n}, sanitized via offshoot.langgraph._sanitize) from the seed checkpoint with a TTL (default 1h, offshoot_ttl ini-overridable), opens a session, returns an object with .path/.client/.db/.branch; teardown closes the session then destroys the branch — a destroy failure is a UserWarning, never a test failure). offshoot_dump(path) -> str is the golden-file helper (sqlite3 .dump text) — deliberately: SQLite's on-disk bytes are not deterministic for identical logical content, so golden assertions compare offshoot_dump output, NEVER raw bytes (sdk/python/tests/test_pytest_plugin.py::test_offshoot_dump_is_the_right_comparison_not_bytes proves a vacuum changes the bytes but not the dump). Package extra offshoot-db[pytest] (pytest>=7); the pytest11 entry point is registered unconditionally in pyproject.toml but the module import-guards pytest, so the base offshoot-db install stays stdlib-only (verified: make test-sdks — the plain-unittest suites — passes with no pytest installed at all; make dry-run-python-sdk's built wheel import offshoot-tests clean in a pytest-free venv too).xdist stance (a locked design decision): one offshoot daemon + temp store per xdist worker (pytest has no cross-worker fixture-sharing mechanism), so seed cost is paid once per worker, not once total — measured on this repo's own smoke test (test_xdist_two_worker_run_passes_and_measures_seed_cost, a CREATE TABLE + 200-row seed, macOS arm64): ~80-90ms per worker, ~170ms of total seed work under -n2 against ~85-90ms of wall-clock time (workers seed concurrently). Full rationale and the numbers live in pytest_plugin.py's module docstring and sdk/python/README.md's pytest-fixture-plugin section.Found+fixed in passing: _run_seed's SQL-string path now wraps an unwrapped multi-statement seed in one transaction — sqlite3.Connection.executescript otherwise runs every statement as its own autocommit transaction, measured ~1.6s for 200 unwrapped INSERTs vs. ~17ms wrapped (~100x) — pinned by test_seed_factory_wraps_multi_statement_seed_in_one_transaction/test_seed_factory_wraps_unterminated_single_statement_seed/test_seed_factory_respects_seed_with_its_own_begin_commit.Testing: 50 direct-daemon logic tests (naming, TTL, teardown ordering close-then-destroy, factory memoization, skip-when-no-binary, destroy-failure-warns, seed-transaction-wrapping, dump-shaped/comment-prefixed seeds, dead-daemon-connection wrapping, seed-mismatch detection) plus 8 pytester-driven scenarios (entry-point autoload, fork-per-test isolation, the xdist 2-worker run that also measures the seed-cost numbers above, OFFSHOOT_BIN misconfigured fails loud, offshoot_require_binary ini fails loud, offshoot_require_binary ini defaults to a skip, offshoot_seed ini resolves relative to rootdir not cwd, teardown warnings don't escalate under -W error) — 58 tests total (re-counted directly for this row: grep -c '^def test_' sdk/python/tests/test_pytest_plugin.py and a live make test-pytest-plugin run both pass (61 top-level def test_ functions as of 2026-10-02, counted by grep); the pytester tier grew from the plan's originally-scoped 2-3 scenarios to 8 during Task 4's review round) — make test-pytest-plugin, wired into ci.yml's sdks job (installs offshoot-db[pytest] + pytest-xdist AFTER make test-sdks already proved pytest-free, so the ordering itself is part of the proof) |
TypeScript testkit (sdk/typescript/src/testkit.ts, @offshoot-db/client/testkit) |
shipped-and-tested | Milestone 3 Task 5 — the vitest/jest counterpart of offshoot.pytest_plugin: framework-agnostic FUNCTIONS, not fixtures (no vitest/jest runtime dependency — this SDK stays zero-runtime-deps — and nothing registers itself automatically; the caller wires these into their own beforeAll/afterEach). startDaemon(opts?) -> Promise<DaemonHandle> locates the binary via OFFSHOOT_BIN env or PATH (a clear error naming both when neither has one — no skip tier, since this module has no test-framework integration to skip through, unlike the pytest plugin), starts it on a fresh temp store + socket, returns { sock, store, proc, stderrTail(), stop() }.seedOnce(daemon, {name?, seed}) -> Promise<SeedHandle> is a NAMED-SEED cache memoized per (daemon, name); seed is a SQL string, a path to a .sql file (detected: no newline, ends .sql, file exists — there's no ini file here to hold that decision separately the way pytest's offshoot_seed option does), or an async (dbPath) => void callback; a later call for the same name with a different seed (fingerprinted: SQL/path text by content hash, callables by identity) throws rather than silently keeping the first one, mirroring offshoot_db's mismatch semantics exactly. Ports _skip_leading_noise/_seed_opens_own_transaction verbatim, so a sqlite3 .dump's text (PRAGMA before BEGIN TRANSACTION) works as a seed unmodified and a plain multi-statement seed is wrapped in one transaction rather than paying one autocommit transaction per statement.forkPerTest(daemon, seedHandleOrName, opts?) -> Promise<ForkedSession> forks a fresh, worker-safe-named branch (t-{worker}-{sanitized-hint}-{n}; worker id from VITEST_POOL_ID/JEST_WORKER_ID when present, else "local" — sanitize is offshoot.langgraph._sanitize ported verbatim, since there's no TypeScript langgraph module to import it from) from the seed checkpoint with a TTL (default "1h", opts.ttl overrides), opens a session, returns { path, db, branch, client, flush(name?), close() }; close() closes the session and destroys the branch, and either failing is a console.warn, never a throw. Each ForkedSession owns its own connection and teardown independently of every other one, so — unlike the pytest plugin's shared-per-test _ForkFactory.teardown() loop — one fork's cleanup trouble can never block another's, by construction rather than an explicit try/except around each step. dump(path) -> Promise<string> is sqlite3 <path> .dump's text output, the same golden-comparison method as offshoot_dump: SQLite's on-disk bytes are not deterministic across writes with identical logical content, so golden assertions must compare dump output, never raw bytes (test/testkit.test.ts's "is the right comparison, not raw bytes" test proves a VACUUM changes the bytes but not the dump, mirroring the Python plugin's own such test). Since this SDK ships no SQLite driver, seeding and dumping both shell out to the sqlite3 CLI, same as test/client.test.ts already does.Exported as a new package.json exports subpath, @offshoot-db/client/testkit (the root export gained an explicit exports entry alongside it; main/types kept for back-compat with tools that don't read exports).Testing: sdk/typescript/test/testkit.test.ts, run via node:test against a real daemon — mirrors the Python plugin's direct-daemon tier (binary resolution naming both OFFSHOOT_BIN and PATH, naming, TTL default+override, seedOnce memoization + fingerprint mismatch, a .dump-shaped seed, a path-to-.sql seed, an async-callback seed, fork-per-test isolation, the string-name shorthand, teardown-warn-not-throw against a killed daemon, a golden-file-not-bytes scenario) plus one integration test wiring startDaemon/seedOnce/forkPerTest into node:test's own before/after/beforeEach/afterEach the way a real suite would — 22 tests in testkit.test.ts itself (re-counted directly: node --test test-dist/test/testkit.test.js in isolation reports tests 22/pass 22), wired into the existing make test-ts-sdk target alongside client.test.ts's own suite (no new Makefile target needed: it already globs test-dist/test/*.test.js; the CI-verified combined TS count lives in the typed-surfaces row above). make dry-run-ts-sdk's tarball exact-match assertion updated to expect dist/testkit.js/dist/testkit.d.ts alongside dist/client.*, plus a second install-then-import() check for the @offshoot-db/client/testkit subpath specifically (not just the package root) |
Branch diff, content-aware and everywhere (offshoot diff, ops.Workspace.MaterializeForDiff, ops.DiffSummary/ops.FormatDiffSummary, daemon diff op, SDK diff(), MCP offshoot_diff) |
shipped-and-tested | Milestone 3 Task 6 — offshoot diff <db>[@branch[@checkpoint]] <db>[@branch[@checkpoint]] [--summary] [--table T] materializes both sides READ-ONLY (never a live checkout, never a lease) and either streams sqldiff's output over them (default) or prints a stdlib-only, sqldiff-free content-aware per-table summary (--summary): for a table present on both sides with matching columns, rows added/removed/changed by the table's declared primary key when it's actually unique on both sides, else its internal rowid (TestDiffSummaryIsContentAware, TestDiffSummaryNonUniqueDeclaredPKFallsBackToRowid), with schema changes flagged (a SchemaChanged table's STATUS cell gets " (schema)"); a table whose column list differs between sides is reported — with row counts — but not compared. --table restricts either mode to one table. This is no longer CLI-only: the daemon protocol's diff op (left, right, table, full, max_bytes request fields; a diff response object with tables/totals and, when full was set, capped sqldiff SQL) returns the same summary as JSON over the wire — allowed over the HTTP surface (unlike export) since its response is always JSON built from store data, never a filesystem path — pinned by TestDiffOpReturnsContentAwareSummaryOverTheWire. Both SDKs expose diff(left, right, *, table=None, full=False, max_bytes=None) (Python) / diff(left, right, opts) with DiffOptions{table?, full?, maxBytes?} (TypeScript), tested by sdk/python/tests/test_client.py::test_diff_is_content_aware_and_needs_no_sqldiff and sdk/typescript/test/client.test.ts's "diff: content-aware summary over the wire, no sqldiff". MCP's ninth tool, offshoot_diff (database, left, right, table?, full?, max_bytes?, read-only annotations, full capped at 32768 bytes by default and 262144 at most), is tested by TestDiffToolComparesAttemptsContentAware. Materialization split (the staleness decision, locked and documented on MaterializeForDiff's own doc comment): a named checkpoint uses the existing CheckoutAt read-only cache (immutable content, legitimately reusable across diffs); a bare head target CANNOT be cached that way — head moves — so it is always freshly Exported to a private temp file removed after the diff, proven directly by internal/ops/diff_test.go's TestMaterializeForDiffHeadSideAlwaysReflectsANewWrite and the CLI-level TestDiffCLIHeadSideReflectsNewWriteNotStaleCache (a write + checkpoint between two --summary calls against the same head target changes the reported delta, not a stale cache). Both sides are opened file:<abs>?mode=ro&immutable=1 — verified empirically against a real chmod 0444 file (internal/ops/diff_test.go's TestTableRowCountsOpensReadOnlyEvenOnA0444File) that SQLite accepts a mode=ro URI parameter alongside the driver's always-requested SQLITE_OPEN_READWRITE|SQLITE_OPEN_CREATE flags (SQLite only rejects a mode LESS restrictive than the flags argument; ro is strictly more restrictive, so this is not a documented-but-untested claim); the right side is ATTACHed to the same connection so both sides are compared in one query, SetMaxOpenConns(1) pinning every query in a call to that attachment. sqldiff's absence (a SEPARATE binary from sqlite3 on most platforms) produces a clear, per-OS-hinted error rather than a bare "executable file not found" — the macOS hint (brew install sqldiff) was corrected mid-task after installing Homebrew's general sqlite formula locally and confirming it does NOT ship sqldiff (only the dedicated sqldiff formula does); the Debian/Ubuntu hint (sqlite3-tools) was verified against a real ubuntu:24.04 container's apt-cache show sqlite3-tools output listing /usr/bin/sqldiff, and .github/workflows/ci.yml's Linux job now installs that package (alongside the pre-existing plain sqlite3) so the CLI tests that exercise a REAL sqldiff invocation (TestDiffCLISqldiffPresentStreamsOutput, TestDiffCLIIdenticalSidesProduceNoSqldiffOutput) run in CI rather than permanently skipping (they gate on exec.LookPath("sqldiff") the same way requireSQLite3 gates on sqlite3 itself, so a local dev machine without sqldiff still skips cleanly); the daemon's full mode fails loud instead when sqldiff isn't on the daemon host's PATH. Cross-db diff (two entirely different db names) is explicitly supported and tested (TestDiffSummaryCLIWorksAcrossTwoDifferentDatabases) — a legitimate shape for eval comparisons, everywhere except MCP (one database; left/right are branch[@checkpoint] there, not db@branch[@checkpoint]). docs/diff.md documents the command, the raw by-hand export-twice-then-sqldiff recipe, and links the FAQ's no-merge stance; v0.2.11 |
BranchBench topologies harness (cmd/branchbench, make bench-branchbench; five macrobenchmark workflows from BranchBench, arXiv:2604.17180, each run against its own fresh local store) |
shipped-and-tested | go test ./cmd/branchbench's TestEveryWorkflowRunsAtQuickScale runs all five topologies at -quick scale in the normal go test ./... run, so this stays exercised on every change, not just when someone remembers to run the full benchmark by hand. The full-scale table — every workflow completing every step, 2,310/2,310 worker-steps, 0 CAS retries, nothing aborted or timed out — is pasted verbatim in docs/benchmarks.md's "BranchBench topologies (v0.2.12)" section. The parameter tuples are BranchBench's own; the numbers are ours, re-derived against our own generator/schema/SQL (the upstream harness at github.com/ElaineAng/db-fork carries no license file, so none of it is vendored here). The paper's hosted-Postgres-family numbers (Dolt 170/1000 worker-steps on MCTS before a 2h timeout; Neon capped by its 20-live-branch ceiling) are quoted for context in that section, never re-run by us; v0.2.11 |
BranchBench A/B tool (cmd/bench-ab, make bench-ab) |
shipped-and-tested | cmd/bench-ab builds cmd/branchbench once from git archive <before> and once from the working tree, discards one warm-up run of the after build per workflow, then alternates the two for 4 rounds per workflow (odd rounds before→after, even rounds after→before) on the default workflows failure_repro:1,simulation:8. -before is required: the tool has no default ref and, without one, prints a line and exits 0; make bench-ab and the nightly supply the latest release tag (git describe --tags --abbrev=0 --match 'v[0-9]*', so a stray non-release tag can't become the baseline). Only a gating metric's depth-1 p50 — fork, checkout, checkpoint — can fail the run: it regressed when the after median exceeds the before median by more than 15% (-threshold) AND every after sample exceeds every before sample; wall and eval are informational only and never set the exit code. The built binary exits 0 (no -before ref, or nothing regressed), 1 (a gating metric regressed), 2 (bad flags, unknown ref, a build, run, or parse failure); go run folds any non-zero status into its own 1 and make into its own 2, so make bench-ab (bin/bench-ab) and the nightly ($RUNNER_TEMP/bench-ab) build the binary first, and under make a regression shows as make's exit 2 — the report tells the two apart. Each branchbench run gets -timeout set to -run-timeout (default 15m; make bench-ab and the nightly pass 10m); one minute after that it is sent SIGTERM, and 30s later it is killed. Each raw per-run file starts with a # bench-ab side=<side> ref=<short rev> round=<n> line written by bench-ab. 19 tests in cmd/bench-ab: TestParseRowDepthOne, TestParseRowDeeperTreeReadsTheDepthOneP50, TestParseRowWallInMinutes, TestParseRowWallInMilliseconds, TestParseRowRejectsGarbage, TestParseRowRejectsAMalformedNumber, TestParseRowRejectsATimedOutRun, TestFindRowPicksTheWorkflow, TestMedian, TestVerdictRequiresEveryAfterAboveEveryBefore, TestVerdictRejectsTooFewRounds, TestImprovedMirrorsTheRegressionRule, TestExtractTarKeepsModesAndRefusesEscapes, TestRunOrderAlternates, TestTailLines, TestRawHeaderNamesSideAndRefAndKeepsTheRowFindable, TestOnlyGatingMetricsDecide, TestGatingRegressionStillFailsTheRun, TestParseSpecs |
Nightly A/B of BranchBench against the latest release tag (perf-ab job in .github/workflows/nightly.yml) |
shipped | The job runs the tool above against the latest release tag on days main moves, on the Sunday leg and on manual dispatch, and fails on a gating regression (exit 1) or a tool error (exit 2). It catches a shift like the v0.2.16 per-checkpoint regression — found three releases late by building each tag by hand and alternating runs — when the rule's conditions hold (a gating p50 more than 15% up and no before/after overlap). The 15% threshold was tuned on the maintainer's own machine, where the v0.2.15 → v0.2.16 shift measured +23% on checkpoint p50 with non-overlapping before/after samples and within-series noise under 5%. First run on the shared runner (manual dispatch 2026-10-09, run 37931344620, v0.2.19 against main at 58ece9b, simulation then at eight workers): the job took 29 min 21 s and exited 0; failure_repro's checkpoint p50 cleared the rule as "improved" (204.0 → 165.2 ms), simulation's fork p50 likewise (17.1 → 14.4 ms), every other gating row "ok" (the only runtime change between v0.2.19 and that commit is the pierrec/lz4 4.1.33 and aws-sdk-go-v2/service/s3 bump, so the improvements are that bump or runner drift, not separated by this run), and a bimodal fork sample set read "ok" because one before run overlapped the after runs. Eight workers drove the four-vCPU runner's load average to 9 with runs of about three minutes each, so the job now runs simulation at four workers; the timing at four is not yet measured (benchmarks.md) |
Observability and security
| Guarantee / feature | Status | Notes |
|---|---|---|
offshoot status (branch state, checkpoints, TTL remaining) |
shipped-and-tested | |
session status (durable-through txid, epoch, holder, errors) |
shipped-and-tested | |
| Structured branch-state-transition logging | shipped-and-tested | internal/session/session_test.go (TestSessionTransitionLogsOpenedFlushedClosed), internal/session/flush_test.go (TestAutoFlushTransitionLogsRecordKindAutoAndFailure) — every session state transition (opened; flushed, tagged kind=manual/kind=auto with its txid; flush-failed with the error, ErrClosed races excluded; fenced with the terminal cause; closed) writes one offshoot: session: db@branch: event key=value ... line to stderr, matching the daemon janitor's existing offshoot: janitor: ... line family |
Metrics registry + instrumentation (internal/metrics, offshoot_* names locked) |
shipped-and-tested | Milestone 4 Task 2 — hand-rolled, zero-dependency, concurrent-safe Prometheus text-exposition registry (Counter/Gauge/Histogram with fixed buckets, CounterVec/GaugeVec for bounded label sets, Registry.WritePrometheus); zero-dep rationale is one sentence on the package doc comment. Every metric on the plan's LOCKED list is registered by internal/daemon's newMetrics — offshoot_build_info{version}, offshoot_sessions_open, offshoot_capture_lag_bytes{db,branch}/offshoot_durable_age_seconds{db,branch} (open sessions only, computed at SCRAPE TIME from the sessions map via a Registry.Collect callback, not continuously), offshoot_flush_total{result,kind}/offshoot_flush_duration_seconds, offshoot_fork_total{path}/offshoot_fork_duration_seconds, offshoot_checkpoint_duration_seconds, offshoot_reap_total, offshoot_gc_tombstoned_total/offshoot_gc_deleted_total/offshoot_gc_backlog, offshoot_ro_cache_bytes/offshoot_ro_cache_evictions_total (driven by the janitor's ro-cache pass as of Task 5 — see the Resource-behavior section below), offshoot_janitor_runs_total{result}.Two families were added after the lock, in v0.2.1 (additions are fine; renames are not): offshoot_fork_mode_total{mode="shared"|"materialized"} (the copy-on-write storage mode at fork time, orthogonal to offshoot_fork_total{path}) and offshoot_gc_errors_total (janitor GC passes that returned an error — the alertable signal for a fails-closed GC that would otherwise stall silently), bringing the family count to eighteen; offshoot_rollback_total{mode}/offshoot_promote_total{mode} (v0.2.12) and offshoot_checkpoint_overwrite_detected_total (Tier 2) bring it to twenty-one — see docs/operations.md. Four more families were added with the FD budget: offshoot_dbfile_descriptors, offshoot_dbfile_pins, offshoot_dbfile_stranded_pinned (gauges, read at scrape time) and offshoot_dbfile_evicted_total{reason="stranded"|"budget"} (TestJanitorEvictsUnderFDBudget and TestJanitorBudgetZeroIsUnlimited check each value against dbfile.ReadStats), bringing it to twenty-five.Instrumentation: internal/ops's ObserveFork/ObserveCheckpoint are package-level, nil-checked injected hooks (ops must not import internal/metrics) the daemon assigns once at NewServer construction; internal/session's OnTransition hook rides the SAME transition-log call site Milestone 2 added (logTransition), not a new one, feeding flush counters/durations from the existing "flushed"/"flush-failed" events. WritePrometheus output is golden-file-tested (internal/metrics/metrics_test.go) and validated against promtool check metrics (TestPromtoolCheckMetrics/TestPromtoolCheckRealMetrics, loud-skip-if-absent locally, installed explicitly in ci.yml's new metrics-lint job). Exposed over HTTP as of Task 3 — see the next row.Concurrent-scrape safety (T2 review carried item, closed in Task 3): Registry.WritePrometheus now serializes end to end via a scrapeMu mutex — a multi-step Collect callback (the session-gauge collector below Resets then repopulates two GaugeVecs) is otherwise not atomic across two concurrent scrapes, which could tear the output (e.g. offshoot_sessions_open reporting N while offshoot_capture_lag_bytes carries fewer than N samples in the SAME response). internal/metrics/metrics_test.go's TestWritePrometheusSerializesConcurrentCollectorRuns and internal/daemon/http_test.go's TestConcurrentMetricsScrapesWithSessionChurn (real HTTP, real session open/close churn) both pin the invariant and are confirmed to fail reliably without the mutex |
Prometheus /metrics HTTP endpoint |
shipped-and-tested | Milestone 4 Task 3 — GET /metrics (internal/daemon/http.go's handleMetrics) calls the registry above directly; token-gated like everything but /healthz. See the row above for the concurrent-scrape fix and its tests |
| HTTP binding + single-token auth | shipped-and-tested | Milestone 4 Task 3 — internal/daemon/http.go. Off by default; serve -http 127.0.0.1:PORT binds loopback with no further ack needed; a non-loopback bind additionally requires -http-allow-non-loopback AND an explicit -token/OFFSHOOT_TOKEN (two DISTINCT startup errors if either is missing — ValidateHTTPBind, TestValidateHTTPBindErrorsAreDistinct). Token: -token/OFFSHOOT_TOKEN, else auto-generated (GenerateToken, 256-bit) and printed once to stderr (loopback only). Every request but GET /healthz requires Authorization: Bearer <token>, checked with crypto/subtle.ConstantTimeCompare (checkAuth); a token is never logged in full again after that one startup line — only its 8-character TokenFingerprint appears in ongoing output (TestHTTPTokenRedaction, TestHTTPAutoGeneratedTokenPrintedOnceThenOnlyFingerprint). Routes: POST /rpc (the same Request/Response JSON and Server.dispatch the unix socket uses — TestHTTPRPCParityWithSocket; 1MiB body cap via http.MaxBytesReader, 413 on overflow, connection still usable after — TestHTTPRPCBodySizeLimit; Content-Type: application/json required), GET /debug/pprof/* (net/http/pprof, same Bearer auth — TestHTTPPprofBehindAuth), GET /healthz (unauthenticated, {"ok":true,"sessions":N}). http.Server timeouts are explicit and justified in http.go's doc comment (ReadHeaderTimeout 5s, ReadTimeout 30s, WriteTimeout 90s — sized to leave headroom for /debug/pprof/profile's default 30s capture window, IdleTimeout 2m). Shutdown ordering: the HTTP listener is closed (http.Server.Close, immediate — matching the unix socket's own force-close-live-connections philosophy) alongside the unix listener in Server.Shutdown; an HTTP op=shutdown request gets the same respond-then-shutdown-trigger ordering fix the unix socket got in Milestone 3 (TestHTTPShutdownRespondsBeforeClosingRequestingConn), and a generic hammer test (TestHTTPShutdownWhileRequestsInFlightIsSafe) proves no panic/hang under -race while Shutdown races in-flight requests. Threat model: single-tenant, same-host-or-trusted-network auth — the token is a shared secret, not a multi-tenant isolation boundary; see docs/reference.md's -http ADDR section and docs/operations.md for the full operator-facing writeup (Milestone 4 Task 8) |
Branch state taxonomy (active / pending / closing / error / dirty / detached / idle) |
shipped-and-tested | Milestone 4 Task 1 — ops.BranchStateAt (internal/ops/status.go) computes active/dirty/detached/idle from ref+lease+checkout-sidecar truth alone (no daemon dependency); a daemon layers pending/closing/error on top from its own in-memory session map (internal/daemon/server.go's Server.branchState) — see docs/reference.md's Branch states table for the full precedence (closing > error > pending > active > dirty > detached > idle) and why idle is a deliberate addition the design spec's original taxonomy lacked (it assumed a daemon was always present). Computed only, nothing persisted. Surfaced in offshoot status (CLI/at-rest), the daemon branches op's BranchInfo.state (always present, no omitempty — wire-additive so pre-Milestone-4 SDKs against a new daemon are unaffected), and both SDKs' Branch.state (defaults to "" against a pre-Milestone-4 daemon that never sends the field). internal/ops/states_test.go constructs each ops-level state directly (including detached via a Promote whose best-effort checkout refresh is forced to fail, per that op's own doc comment) plus an exactly-one-state invariant test across a matrix workspace; internal/daemon/branchstate_test.go covers pending (the openDelay test hook holding an open in flight) and error (lease-theft fencing, mirroring internal/session/renew_test.go's own technique) including the precedence case where a fenced session's ref shows an active lease held by the thief but the daemon still correctly reports error; internal/daemon/closing_test.go's TestBranchesReportsClosing covers closing, including that it outranks error |
Protected-branch flag (default on for main) |
shipped-and-tested | Enforced by CLI, daemon, and MCP alike: internal/ops/protect_test.go's TestSetProtectedFlipsTheFlagAndGatesDestroy, cmd/offshoot/protect_test.go, internal/daemon/lifecycle_test.go's TestDestroyForceOnProtectedBranch, internal/mcp/tools_test.go's TestMCPRefusesForceOnProtectedByDefault; unforced promote onto a protected target over the daemon and CLI in security_qa_test.go |
Operator documentation (docs/operations.md) |
shipped | Milestone 4 Task 8 — the metrics reference table, branch-states table, event schema, budget mechanics, and HTTP/auth threat model in one operator-focused page, cross-linked from docs/reference.md's wire-level sections; its own metrics table is verified against a real scrape (go build + serve -http + curl /metrics, # TYPE line count matched against the table row count). docs/recipes/kubernetes.md ships alongside it: a real sidecar manifest (docs/recipes/k8s/offshoot-sidecar.yaml), schema-validated with both kubectl apply --dry-run=client and --dry-run=server against a disposable local rancher/k3s control plane (no cluster was otherwise available in this environment). "shipped" not "shipped-and-tested" per this page's own legend — no automated test enforces documentation accuracy going forward |
| TLS on the HTTP listener | deliberately deferred | Milestone 4's self-review scoped -http to loopback-by-default plus a shared-secret token, explicitly not TLS — a non-loopback bind is documented as needing a trusted network boundary (VPN, private subnet) of its own instead. Revisit with real non-loopback demand; see docs/operations.md |
| Per-branch at-rest metrics by default | deliberately deferred | A gauge per branch that has never been opened this process would scale label cardinality with total branch count rather than open-session count — offshoot_capture_lag_bytes/offshoot_durable_age_seconds stay open-sessions-only by design (see docs/operations.md). A dbs-scoped scrape option for at-rest branches was named as a future addition in the Milestone 4 plan, not built this pass |
| Metrics push/remote-write | deliberately deferred | GET /metrics is pull-only, matching every other route on the opt-in HTTP listener; no push-gateway or remote-write integration was scoped for Milestone 4 |
Eventing: event bus, socket subscribe op, GET /events SSE |
shipped-and-tested | Milestone 4 Task 4a — internal/daemon/events.go's in-daemon bus fans one versioned event schema out to subscribers over two transports that share the SAME encoder (encodeEvent): the unix socket's subscribe op and HTTP's GET /events (SSE). type is session_opened/flushed/flush_failed/fenced/session_closed (the first four fed from the SAME internal/session transition-callback call site; session_closed is published by the daemon once the closed session's slot is gone — see the row above on open serialized against its own close. That call site is the one Task 2 already hooked — session.OnTransition is a single package-level func var, so wireEvents composes onto whatever Task 2's wireHooks already assigned rather than clobbering it; internal/session itself is unmodified), reaped (fed from the janitor's Reap pass in janitorTick), and evicted (as of Milestone 4 Task 5, fed from the janitor's ro-cache LRU eviction pass in that same janitorTick — see the Resource-behavior section below). Never blocks the daemon (Global Constraint): eventBus.publish is entirely non-blocking — a subscriber whose bounded buffer (eventSubscriberBuffer, 64) is full is immediately removed, sent one terminal {type:"dropped_slow_consumer"} event, and has its channel closed, all without the publisher (a session transition or the janitor) ever waiting — TestSlowSubscriberDroppedSessionKeepsFlushing proves a subscriber that never reads its channel is dropped while a concurrently write-heavy session keeps flushing successfully throughout. Socket subscribe acks THEN the connection permanently leaves request/response mode and streams line-per-event JSON until disconnect (handle() special-cases it exactly like it already special-cases shutdown, subscribing to the bus before sending the ack to close an ack-vs-transition race); SDKs must use a fresh, dedicated connection — it is unix-socket-only, refused with a pointer to GET /events if sent over HTTP POST /rpc (TestSubscribeOverHTTPRPCRefused). GET /events requires the same Bearer auth as everything but /healthz, emits a periodic : ping keepalive comment and — the loud warning Task 3's http.go left for this task — bounds its writes so Task 3's 90s http.Server.WriteTimeout can never hard-cut a long-lived stream: rather than clearing the write deadline once, permanently (a review round caught that this let a subscriber that stayed CONNECTED but simply stopped reading pin the handler goroutine and its connection's file descriptor open forever, since nothing ever re-armed a bound on it again), handleEvents re-arms a fresh eventWriteDeadline-out deadline (default 45s) immediately before every write — the header write, each event, each ping — via http.ResponseController; a live stream re-arms this forward on every successful write (and at least every keepalive tick) so it's never affected, while a genuinely stalled reader's write now times out and the handler gives up. streamEvents (the unix socket side) gets the identical per-write re-arm before every c.Write. TestSSEStreamSurvivesPastWriteTimeout proves the live-stream side structurally and cheaply (a test-only httpWriteTimeout var shrunk to 150ms, stream proven alive well past it via a real event delivered afterward — no 90-second sleep); TestStalledSocketSubscriberConnectionIsClosedWithinWriteDeadline/TestStalledSSESubscriberConnectionIsClosedWithinWriteDeadline prove the stalled-reader side through the REAL write path (an oversized event forces an actual blocking write, not a simulated one), both confirmed to fail reliably against the pre-fix code. TestSSEParityWithSocketSubscribe subscribes both transports before a real session lifecycle and asserts identical observed event sequences. No internal/session changes, so no torture run was required for this task (internal/daemon/events_test.go, full suite green under -race). Task 4b — SDK stream helpers: both SDKs ship a thin events() helper over the subscribe op — Python Client.events() (sdk/python/offshoot/client.py) is a generator yielding Event dataclass instances, TypeScript Client.events() (sdk/typescript/src/client.ts) is an AsyncGenerator<OffshootEvent> — each opening its OWN fresh, dedicated socket connection (never the caller's own Client connection, per Task 4a's dedicated-connection requirement above). dropped_slow_consumer is yielded like any other event and then the stream simply ends (not raised/thrown) — an explicit, documented decision; a caller checks the last event's type if it cares. Stopping iteration early (Python break/.close(); TS break/.return()) closes the dedicated socket with no leaked fd, proved against a real daemon via lsof -p <pid> filtered to TYPE unix (a raw total-fd count is confounded by internal/dbfile's cached checkout descriptors, so the tests isolate socket fds specifically) returning to its pre-subscribe baseline after a helper connection closes mid-stream. Real end-to-end lifecycle tests drive session_opened/flushed/session_closed on a separate connection and assert the helper observes them in order; the slow-consumer-drop path is hard to force deterministically from the SDK side (it needs the real bus's buffer-overflow timing, already covered server-side by TestEventBusDropsSlowSubscriberWithTerminalEvent), so both SDKs instead unit-test the helper's decode/contract path end to end against a small scripted fake daemon. Both SDK suites green under make test-sdks; both confirmed stdlib-only/zero-dep |
OpenSSF Scorecard workflow (.github/workflows/scorecard.yml) + README badge |
shipped | Runs on push to main, a weekly cron (Mondays 06:17 UTC), branch_protection_rule changes, and workflow_dispatch; ossf/scorecard-action with publish_results: true (the badge and the scorecard.dev viewer page) and its SARIF uploaded to code scanning via github/codeql-action/upload-sarif. shipped; first run on main pending (badge shows no score until then). "shipped" not "shipped-and-tested" per this page's legend: no automated test enforces the workflow stays wired going forward; v0.2.11 |
Scorecard remediation: hash-pinned CI installs, digest-pinned base images, CodeQL SAST (.github/workflows/codeql.yml) |
shipped | 2026-10-01. Every third-party Python package CI installs comes from a hash-pinned lock (requirements/*.txt, generated from the .in beside it — see CONTRIBUTING.md's "Refreshing the hash-pinned CI requirements"), with the repo's own SDKs installed --no-deps -e; publish.yml uses npm ci; the Dockerfile's base images are image:tag@sha256: pinned and bumped by Dependabot's docker ecosystem; SECURITY.md carries absolute reporting/verification URLs; CodeQL analyses Go on every PR, push to main and weekly. Local scorecard --local: Pinned-Dependencies 6 → 9, Security-Policy 4 → 10. The last Pinned-Dependencies finding (code-scanning alert 29, publish.yml's packed-tarball import test doing npm install of the local tarball) was closed on 2026-10-01 by unpacking the tarball straight into a throwaway node_modules with tar instead: the package has no runtime dependencies, so resolving @offshoot-db/client from there exercises the same files list and exports map with no package manager involved. SAST rises as merged PRs accumulate CodeQL check-runs. Fuzzing is tracked separately; the OpenSSF Best Practices badge is under Not yet done |
CodeQL code scanning: name-derived paths contained under Workspace.Root; client-named paths dismissed as by design |
shipped-and-tested | First run 2026-10-01 (PR #62): 14 high-severity go/path-injection alerts (15-28), every one traced from a daemon request field (the HTTP /rpc body). 7 fixed in code (15, 16, 24-28: os.Stat/os.Open/os.ReadFile/os.WriteFile on a checkout, its .sum sidecar, and dbfile's reads of it, reached from db/branch names): ops.Workspace.underRoot is the one containment check every name-derived local path goes through (CheckoutPath, CheckoutAtPath, byChainPath; sidecar and shadow paths are suffixes of those). It joins under Root, filepath.Cleans and requires the Root+separator prefix (filepath.IsLocal for a . root). Every input already passed store.ValidateName, so the check cannot fail. On a validation bypass it panics, naming the invariant. internal/ops/underroot_test.go's TestUnderRootAcceptsEveryValidNameShape (outputs identical to the old filepath.Join across absolute, relative, ., "" and / roots) and TestUnderRootRejectsEscapingInput. 7 dismissed "won't fix" (17-23: export's destination at ops/export.go and through ltxio.MaterializeChain's temp file, rename and -wal/-shm removal; create's import source at copyFile): client-named file paths by design. The CLI path is the invoking user's own, and the daemon's Request.Path is a trusted absolute server-side path (internal/daemon/protocol.go; SECURITY.md). The HTTP source CodeQL traced cannot reach either sink: export is in httpForbiddenOps, and create with path is refused before dispatch. No query was disabled and no path filtered. Open CodeQL alerts on the PR after the re-run: 0 |
| Signed and attested releases (keyless cosign signatures on every tarball, SLSA build provenance, an SPDX SBOM with its own attestation, a cosign-signed and attested GHCR image) | shipped-and-tested | release.yml's collect job signs every tarball with keyless cosign (<asset>.sigstore.json), emits an SPDX SBOM (offshoot_<tag>.spdx.json) with a matching attestation, and produces SLSA build provenance (the GitHub attestation store plus an offshoot_<tag>_provenance.intoto.jsonl release asset); the docker job cosign-signs the GHCR image digest and pushes its own provenance attestation. Verification recipe: docs/installation.md's "Verify what you downloaded", linked from SECURITY.md and docs/testing.md. Validated end to end by a workflow_dispatch dev build (run 36251100872, tag dev-0935a75, since deleted): gh attestation verify confirmed both the SLSA v1 provenance and the SPDX 2.3 SBOM predicates, cosign verify-blob reported Verified OK against the release workflow's certificate identity, the plain sha256 checked out, and both cosign verify and gh attestation verify oci:// passed against the image. Releases before v0.2.11 carry checksums only; v0.2.11 |
Resource behavior
| Guarantee / feature | Status | Notes |
|---|---|---|
| Checkout-cache disk budget + LRU eviction | shipped-and-tested | Milestone 4 Task 5 — offshoot serve -ro-cache-budget <bytes|0> (default 0 = unlimited) bounds checkouts-ro (the read-only CheckoutAt cache, Milestone 3 Task 2), never checkouts/ (writable, leased — see below). Server.janitorTick (internal/daemon/server.go) runs a new ro-cache pass on the same -reap-every cadence as reap/GC: always computes and republishes current usage into offshoot_ro_cache_bytes (the Task-2-registered gauge, previously always 0 — now updated once per janitor pass, not continuously; see docs/reference.md's -ro-cache-budget section for the resulting between-passes staleness), and once usage exceeds the budget, LRU-evicts (ops.Workspace.EvictROCache, internal/ops/rocache.go) oldest-first until back under it. LRU clock: a .last-used touch-on-HIT marker file, not the .db file's own mtime — CheckoutAt's force=false cache-hit path now calls touchLastUsed, Chtimes-ing a <cachefile>.last-used sidecar to now on every hit; necessary because materializeAt's rename-into-place is the LAST thing that ever touches the .db file's mtime (a hit is a pure read, never touching it again), so without a separate marker "least recently used" would collapse to "least recently created" — exactly backwards for a cache whose point is that a repeatedly-hit checkpoint stays hot. lruClock falls back to the .db file's own mtime as the floor for an entry materialized but never since hit. TestEvictROCacheLastUsedTouchBeatsCreationOrder (internal/ops) and TestJanitorTickEvictsLRUUnderBudgetAndFiresEvictedEvent (internal/daemon) both prove an OLDER-by-creation entry that gets hit survives while a NEWER-by-creation, never-hit entry is evicted instead. checkouts/ is never evicted by construction, not a runtime check: EvictROCache only ever walks/removes paths under the separate checkouts-ro tree, never joined with or reachable from checkouts/'s own path shape — TestEvictROCacheNeverTouchesWritableCheckout and TestJanitorROCacheEvictionNeverTouchesLeasedWritableCheckout (a real open, leased session survives an aggressive budget=1 pass and can still flush afterward) confirm this end to end. Eviction is LOUD: one offshoot: janitor: ro-cache: evicted <db>@<branch>@<checkpoint> (<bytes> bytes) stderr line per eviction, offshoot_ro_cache_evictions_total incremented (previously always 0), and Task 4a's reserved evicted event type gets its emitter here ({type:"evicted", db, branch, detail:{checkpoint, bytes}}, published non-blockingly to the T4a bus at the eviction call site). offshoot status gains an ro-cache usage summary line (ops.Workspace.ROCacheUsage) plus a display-only -ro-cache-budget flag of its own (the budget itself is never persisted, matching every other serve tuning flag, so this at-rest command has no other way to show usage against it without a live daemon). Each eviction removes both the .db and its .last-used marker together; a mid-pass os.Remove failure follows Reap/GC's own partial-progress-plus-first-error convention. go test ./internal/ops ./internal/daemon -race clean; no internal/session/internal/capture changes, so no torture run was required |
| FD budget with idle-checkout eviction | shipped-and-tested | Closes Milestone 4's consciously narrowed bullet. internal/dbfile keeps a pin registry keyed by inode: every raw read (dbfile.Reader, closed by its caller) and every in-process SQLite open of a checkout (dbfile.Hold before sql.Open, released after the connection closes, dbfile.Verify after the first connection, opened with dbfile.NoCreateDSN so a checkout removed after the hold fails to open instead of coming back empty) pins the file, and only an unpinned descriptor is ever closed. A Hold also covers its path: database/sql connects lazily and the connect itself takes SHARED, so a re-materialize landing between the Hold and the connect would leave the connection locking an inode the pin does not cover, and SQLite would hand a lock dropped there to the next connection on that inode; while a Hold is active, nothing cached under its path is closed, whatever inode it names. A close is decided under the registry lock, which drops the entry and marks its path and inode closing, and runs after the lock is released, so freeing a large unlinked file in close(2) stalls no other Reader or Hold; a Hold waits for any close in flight on its path or inode. refreshShadow clones the checkout through the pinned descriptor (Section.CloneTo, reflink.CloneFrom), so on Linux it no longer opens and closes the checkout to FICLONE it, which dropped a concurrently opened session's locks. A descriptor whose path is renamed over or removed becomes an orphan the registry keeps referenced, instead of one Go's *os.File finalizer closes at some later GC whatever its inode's locks. offshoot serve -fd-budget N (default 64, 0 unlimited): each janitor tick runs dbfile.EvictStranded then dbfile.Evict(N). Stranded descriptors (renamed-over or deleted checkouts, by-chain build stages, pruned by-chain entries) are also reclaimed right after each materialization, by-chain prune and destroy (dbfile.EvictStrandedAt, scoped to the paths that operation touched), so a process with no janitor reclaims the strands it makes itself. A checkout another process removes or replaces is found only by the full dbfile.EvictStranded: the janitor runs it, offshoot mcp runs it on every -reap-every tick (daemon or not), and serve -reap-every 0 keeps such descriptors until restart. The janitor's log line for stranded descriptors something still pins (their inode, or their path through an active Hold) names their paths (up to three). Metrics offshoot_dbfile_descriptors, offshoot_dbfile_pins, offshoot_dbfile_stranded_pinned, offshoot_dbfile_evicted_total{reason}; status op dbfile_descriptors, SDK daemon_status()/daemonStatus(). Tests: internal/dbfile's TestEvictNeverClosesAHeldInode (a helper subprocess reads SQLite's SHARED byte range with F_GETLK: held across Evict(0) and EvictStranded, including as an orphan, and free after an os.Open+Close control; no dependence on the sqlite3 CLI's build), TestEvictSkipsInFlightReader, TestEvictStrandedAfterRename, TestEvictStrandedAfterDelete, TestEvictStrandedAt, TestPinnedOrphanIsReportedNotClosed, TestOrphanKeptByAHeldPathIsReported (an orphan an active Hold on its path keeps open, on an inode nothing pins, counts in StrandedPinned), TestEvictHonorsBudgetLRU, TestEvictStopsAtPinnedEntries, TestHoldSandwichDetectsRename, TestHoldReleaseIsIdempotent, TestSectionCloseIsIdempotent, TestNoCreateDSN, TestEvictRacesHandle (under -race), TestEveryCloseIsMarkedAndUnlocked (every close goes through one function, and at each the registry lock is free and the entry's path and inode are marked closing), TestCloseRunsOutsideTheRegistryLock (a blocked close stalls no Reader, Hold or ReadStats on another path, and a Hold on its own path waits for it), TestHoldProtectsEveryInodeItsPathNames (a re-materialize between a Hold and its connect: the connection's SHARED lock on the new inode survives Evict(0) and EvictStranded, live or orphaned, and a second pinned open joining that inode holds a real lock after the first closes), TestPinProtectsEveryPathToTheInode, TestEveryInProcessSQLiteOpenIsClassified and TestGuardrailFlagsUnclassifiedOpens (the go/ast guardrail: a new sql.Open/OpenDB or go-sqlite3 driver use fails until classified pinned or not-checkout, and a pinned one fails unless it defers its Hold's release before the open, so the release runs after the connection's Close, and calls it nowhere else); internal/capture's TestEngineReleasesItsHold (normal shutdown and every early return, including a checkout replaced or removed between Hold and Conn); internal/ops's TestQuiesceHoldsTheCheckout and TestCreateFromHoldsItsImportSource (with the connection open, the pin is in place and an eviction closes nothing on the inode; at release, the connection is already closed), TestQuiesceRefusesAReplacedCheckout, TestQuiesceDoesNotCreateAMissingCheckout (including its "removed between Hold and Conn" subtest), TestQuiesceRaceErrorsSayRetry (a lost open race says "(retry)" only from checkout, checkpoint and destroy, where nothing has committed; the rollback, promote and compact refreshes say the operation stands and name offshoot checkout, since retrying a promote or rollback would replace its safety fork with the repointed head; none says "in use"), TestRefreshShadowKeepsInProcessLocks (a pinned WAL connection's SHARED lock survives refreshShadow, then an os.Open+Close control frees it; fails on Linux with the old path clone), TestCheckoutReclaimsTheDescriptorItStrands, TestRollbackRefreshReclaimsTheDescriptorItStrands, TestDestroyReclaimsTheCheckoutDescriptor, TestPruneByChainReclaimsTheEntriesItRemoves; internal/daemon's TestJanitorEvictsUnderFDBudget (a budget-2 pass keeps the pinned session and the most recently used checkout, the session still flushes, and each offshoot_dbfile_* sample equals dbfile's own count), TestJanitorBudgetZeroIsUnlimited (no budget pass, stranded reclaim still runs, and a checkout removed while pinned is kept open and reported as offshoot_dbfile_stranded_pinned), TestStatusReportsDescriptors, TestPinnedStrandsMessageCapsThePaths, TestPinnedStrandsMessageNamesHeldPaths; internal/mcp's TestReapOnceReclaimsDescriptorsStrandedElsewhere and TestSkippedReapOnceStillReclaimsDescriptors (a checkout removed behind the process's back is reclaimed by the next reaper tick, with or without a daemon); internal/reflink's TestCloneFromLeavesSourceOpen; cmd/offshoot's TestServeBadFDBudgetIsRejected, TestFDBudgetFlag (a given 0 stays unlimited) and TestServeFDBudgetReachesTheJanitor (serve -fd-budget 1 evicts this process's cached checkouts down to one); make test-torture and make test-sdks green |
SnapshotEvery tuning exposed via the daemon |
shipped-and-tested | Milestone 4 Task 6a — offshoot serve -snapshot-every N (default 16, unchanged if omitted; rejects < 1) plumbs into session.Options.SnapshotEvery at opOpen the same way -flush-every plumbs FlushEvery (Server.SetSnapshotEvery, single-writer-before-Serve contract). Unlike -flush-every 0, there is no "disabled" sentinel — every flush must eventually snapshot, so SetSnapshotEvery's n <= 0 means "unset, session.Open applies its own default" rather than "off," and the CLI only calls it when the flag was actually given. internal/daemon/snapshot_every_test.go's TestServeSnapshotEveryWiringDrivesCadence opens a session over the socket with SetSnapshotEvery(4), drives 9 flushes, and asserts both a bounded store.Chain (≤4 members, mirroring ops_test.TestReplayStaysBoundedAcrossManyFlushes) and more than one snapshot object across the lineage listing — proof the cadence actually recurred under daemon wiring, not just the mandatory first-ever snapshot. cmd/offshoot/main_test.go's TestServeNonPositiveSnapshotEveryIsRejected pins 0/negative/non-integer values as usage errors. See README's What a flush costs for the cadence's bandwidth-vs-replay-latency trade-off |
| Current per-session disk/FD costs documented | shipped | Documented in README's Resource behavior: each open session pins one checkout descriptor; a re-materialized or deleted checkout's descriptor, and the disk behind it, is reclaimed once unpinned, and -fd-budget bounds the rest (see the FD budget row); Checkout's clean-skip fast path still avoids most re-materializations in the first place. "shipped" rather than "shipped-and-tested" per this page's own legend — no automated test enforces documentation accuracy |
Platform
| Guarantee / feature | Status | Notes |
|---|---|---|
| Linux + macOS | shipped-and-tested | |
| Copy-on-write fast paths tested on every push (clone checkouts, checkpoint shadows, segment checkpoints) | shipped-and-tested | The push/PR test job runs on ubuntu-latest, whose ext4 cannot clone, so every clone/shadow/segment fast-path test skips there and only the fallback is exercised; the full macOS go test runs weekly (nightly.yml). The cow-paths job in .github/workflows/ci.yml closes that gap: go test ./internal/ops ./internal/ltxio ./internal/reflink plus internal/session's two clone tests (TestOpenOnClonedCheckoutSkipsSettlingFlush, TestAtRestCheckpointAfterCloseWritesASegment) on macos-latest (APFS) on every push and PR, with a 20-minute timeout, after a probe step that fails the job if TestCloneProducesIdenticalFile skips — so a runner that stops cloning cannot turn the job green on skips. Scoped that way to bound the 10x macOS bill and to keep the load-sensitive session-flush heartbeat test off every push; Linux reflink (FICLONE) is exercised by nightly.yml's linux-reflink job on a loopback XFS filesystem with reflink=1 (same probe gate; added 2026-10-02, first run green); v0.2.12 |
| Windows | not supported (non-goal) | Capture path and lock/SHM probing are POSIX-dependent; see ROADMAP non-goals |
| Multi-node orchestration / clustering | not supported (non-goal) | Shared-bucket safety is guaranteed by fencing; placement/failover/routing are explicitly out of scope — see ROADMAP non-goals |
| Merge (three-way) | not supported (non-goal) | Forks are pick-a-winner via promote; the escape hatch is sqldiff over two checkouts — see docs/faq.md |