Implementation status
offshoot's original design describes a v1 scope larger than what's shipped so far. This page is the honest accounting: what's tested in anger, what's built but unverified in some dimension, and what's still just a plan. If a guarantee you're relying on isn't marked shipped-and-tested below, verify it yourself before depending on it in production.
Status legend:
- shipped-and-tested — implemented, and exercised by an automated test (unit, integration, property, or torture) that would fail if it broke.
- shipped — implemented and in the code path you'll actually hit, but without the same weight of testing (e.g. verified against one provider, not the class of providers it claims to support).
- not yet implemented — spec'd or planned, no code behind it yet. Each row links to the roadmap milestone tracking it.
- deliberately deferred — considered for this milestone and explicitly declined, with a stated reason, rather than simply not yet started; see the row's Notes for why.
Standing nag: user-gated launch items
Everything on this page above and below is code and docs a controller/agent loop can finish on its own. These cannot be — they're one-time, out-of-band actions only a human with the right credentials/accounts can take, and no amount of further engineering substitutes for them. This section exists so the list doesn't quietly disappear into individual rows scattered across the doc set — every item below is genuinely ready to go, blocked only on the button-press:
| Item | Ready since | Blocked on |
|---|---|---|
Claim offshoot-db on PyPI and the @offshoot-db npm scope |
Milestone 3 Task 7 — manifests filled out, publish pipeline built and dry-run-verified | This page's own SDK publish-pipeline row: real sdist/wheel/npm-tarball builds pass twine check/npm pack today; only the actual name claim + Trusted Publishing/NPM_TOKEN configuration + flipping the PUBLISH_ENABLED repository variable are outstanding (see CONTRIBUTING.md's Release process) |
| Submit the MCP registry listing | Milestone 3 — server.json drafted in-repo |
the current registry schema will be validated at submission time |
| Submit the LangGraph community-integration PR | Milestone 3 — PR title/description/listing-table entry drafted | blocked on the PyPI publication above, since the listed install command needs to resolve |
Claim a domain, the Homebrew formula name, a docker image namespace |
Milestone 1's "Claim the names" / "Release engineering" bullets | Mostly done: the Homebrew formula ships in-repo (Formula/offshoot.rb, installable via brew tap), and release.yml publishes ghcr.io/sricola/offshoot images on every tagged release — docs/recipes/kubernetes.md's sidecar manifest tag is real now. Still open: a domain, and registry names (PyPI/npm) deliberately unclaimed for now |
| Trademark/collision check on "offshoot" in dev tooling | Milestone 1 | Not started this pass |
| Announce (HN "How offshoot is tested" post + Show HN with the parallel-attempts recording and the MCP walkthrough) | Tier 1 item 5 — copy drafted in docs/superpowers/specs/2026-09-26-launch-announce-draft.md | ROADMAP's own gate: one external person completes install → fork → promote without help; the maintainer presses post |
| Corruption bounty | Tier 1 item 5 — proposed terms: a reproducible case where a kill -9'd stock-sqlite3 writer under offshoot serve leaves a checkout that fails PRAGMA integrity_check or diverges from .dump of the source, on a supported backend, with the run script attached; one payout per distinct root cause |
the amount and the payout are the maintainer's call; goes live by moving these terms into docs/testing.md |
| OpenSSF Best Practices badge (Scorecard's CII-Best-Practices check, code-scanning alert 14) | 2026-10-01 — most of the passing-level questionnaire is already true: SECURITY.md with private reporting, CONTRIBUTING.md, GOVERNANCE.md, CI tests on every PR, CodeQL SAST, signed and attested releases, a changelog | the maintainer registers the project at https://www.bestpractices.dev, answers the questionnaire, and adds the badge to the README; the alert closes on the next Scorecard run |
| Scorecard Code-Review (code-scanning alert 13, dismissed "won't fix" 2026-10-01) | Changes land via PRs since #61 | a second human maintainer to approve PRs (GOVERNANCE.md); revisit the dismissal then |
| Scorecard Maintained (code-scanning alert 12) | n/a — nothing to do | self-heals: Scorecard scores 0 while the repo is under 90 days old; created 2026-07-30, so the alert clears on the first Scorecard run after 2026-10-28 |
Not on this list on purpose: the launch demo assets (asciinema of
parallel attempts, the MCP-in-Claude-Code walkthrough) are explicitly not
user-gated — they were recorded
alongside Task 1 and merged to main already, no further action needed.
Also not on this list: the module-path/org mismatch that used to block go install — the repo lives at github.com/sricola/offshoot, not the
aspirational offshoot-db org, so go.mod and every import/doc/workflow
reference were retargeted to sricola instead of waiting on an org
transfer (go install github.com/sricola/offshoot/cmd/offshoot@latest
resolves today). PyPI/npm registry names are unaffected — those stay
offshoot-db/@offshoot-db and are tracked in the row above.
Core model
| Guarantee / feature | Status | Notes |
|---|---|---|
| Branch = named ref (lineage, epoch, head txid, checkpoints, TTL, protected flag) | shipped-and-tested | internal/ops/ops.go, internal/store |
Protected flag settable per branch (offshoot protect/unprotect <db>[@branch]) |
shipped-and-tested | internal/ops/protect_test.go's TestSetProtectedFlipsTheFlagAndGatesDestroy; CLI-only (no daemon op, no MCP tool) — an agent sees the flag via offshoot_list but can't flip it; doesn't touch the branch's TouchedAt/TTL clock; refuses on a branch already claimed by a reaper or destroy; v0.2.11 |
| One writer per lineage, enforced by lease + epoch fencing | shipped-and-tested | internal/ops/fencing_test.go; every object write lands under the epoch current at write time |
| CAS on every ref mutation | shipped-and-tested | Local backend: O_CREAT|O_EXCL lock file. S3-compatible: conditional PutIf, gated by an attach-time capability probe |
| Copy-on-write fork (shared by default via a durable base pointer; materialized fallback at the fork-time snapshot floor) | shipped-and-tested | v0.2.0 (internal/ops/fork_cow_test.go, cow_divergence_test.go): the common-case fork writes data/{lineage}/base.json and zero data objects; store.Chain resolves base pointers transitively under a strict never-merge-across-lineages rule; two automatic snapshot floors (fork-time ops.ForkShareMaxDepth/-snapshot-every, plus the divergence floor) keep reads bounded; status/branches report storage=shared|materialized per branch. The materialize fallback (and compact, and promote/rollback when they copy — see the shared rollback and promote row) keeps the CopyObject fast path for single-snapshot chains (local: reflink; S3: server-side copy — single-request ≤5 GiB, multipart UploadPartCopy up to 5 TiB since v0.2.4), falling back to materialize-and-re-encode otherwise — see Fork performance below |
offshoot compact (make a shared fork self-contained; ancestor storage becomes reclaimable) |
shipped-and-tested | v0.2.0 (internal/ops/compact_test.go, compact_cow_test.go); no-op on an already self-contained branch; daemon op + SDK compact() both languages; the daemon refuses while the branch has an open session (v0.2.1) |
Compact preserves every checkpoint (rollback-style: each is copied into the new self-contained lineage and rewritten to epoch 1, plus an added compact checkpoint at head) |
shipped-and-tested | internal/ops/compact_test.go's TestCompactPreservesCheckpoints; earlier versions dropped every other checkpoint and left only a single compact one — this row now describes the current, preserving behavior. Cost: one snapshot copy per distinct checkpoint txid kept, on top of the head copy compact already paid; v0.2.11 |
| Checkpoint (named state within a branch, not inherited by children) | shipped-and-tested | |
| Rollback (repoint at new lineage seeded from a checkpoint) | shipped-and-tested | Since v0.2.12 the shared case copies nothing: the new lineage is a base pointer at the kept checkpoint (see the shared rollback and promote row), so the checkpoints kept at or below it resolve through that pointer rather than being copied forward. --materialize (daemon/SDK materialize), and the fork-time depth floor, still copy the kept checkpoints' snapshots into the new lineage as before |
Rollback safety fork (<branch>-pre-rollback: the branch's previous head kept as a shared, TTL'd fork before the repoint; rolling one-per-branch; marker-guarded so a user's branch is never replaced; --no-backup/--backup-ttl, daemon no_backup/backup_ttl, SDK backup/backup_ttl; always on for MCP, which names it as backup in the result) |
shipped-and-tested | internal/ops/rollback_backup_test.go's TestRollbackKeepsSafetyFork; mirrors promote's safety fork below — same safetyFork implementation, default TTL 24h; v0.2.11 |
| Promote (repoint target at source's head via fork machinery) | shipped-and-tested | Protected targets require --force |
Promote safety fork (<target>-pre-promote: the target's previous head kept as a shared, TTL'd fork before the repoint; rolling one-per-target; marker-guarded so a user's branch is never replaced; --no-backup/--backup-ttl, daemon no_backup/backup_ttl, SDK backup/noBackup; always on for MCP, which names it in the result) |
shipped-and-tested | internal/ops/promote_backup_test.go, daemon TestPromoteKeepsSafetyForkThroughDaemon, MCP TestPromoteKeepsSafetyForkAndSaysSo, both SDK suites. Prompted by discussion #40. The fork's base pointer pins the target's old lineage until the TTL reaps it. Since v0.2.12 promote itself shares too (see the shared rollback and promote row) |
Clone-based checkouts from a content-addressed by-chain cache (checkouts-ro/<db>/~by-chain/<chainID>.db; checkout, internal checkout refreshes, session open and checkout --at clone from an immutable entry built first — from a cached chain prefix plus ltxio.ApplySegments, or from the store; plain materialize where the filesystem cannot clone) |
shipped-and-tested | internal/ops/clone_test.go: TestForkCheckoutsCloneFromByChainCache, TestByChainEntriesAreEvictedAndRecreated, TestClonePathDegradesWhenUnsupported (always runs: forces reflink.ErrUnsupported), TestByChainEntryWithoutUsableSidecarIsAMiss, TestDivergedChildClonesPrefixAndAppliesSegments, TestCheckoutAtSharesTheByChainCache, TestMaterializeMissBuildsImmutableEntryFirst, TestByChainAreaIsBoundedByDefault, TestByChainAndShadowModesArePrivate; internal/ltxio's TestApplySegmentsMatchesMaterializeChain; internal/reflink's TestCloneProducesIdenticalFile; internal/session's TestOpenOnClonedCheckoutSkipsSettlingFlush. By-chain entries are capped at 64 per database by default (ops.DefaultByChainMaxEntries, LRU-pruned after each populate, no daemon needed; destroy does not remove them), count toward -ro-cache-budget at logical size (a --at miss counts twice) and are evicted as branch ~by-chain; entries are 0400 in 0700 directories; checkouts-ro stays safe to rm -rf. The clone tests skip on a filesystem that cannot clone — see the CI row under Platform; v0.2.12 |
O(1) clean check for an unchanged checkout (.sum sidecar fingerprint: size, mtime_ns, SQLite change_counter, stamped_ns; clean only when identity and fingerprint match and the mtime is more than 1 s older than the stamp; the change counter is not evidence in WAL mode; any mismatch hashes and re-stamps) |
shipped-and-tested | internal/ops/sidecar_test.go: TestCleanSkipDoesNotHashWhenFingerprintMatches, TestModifiedCheckoutStillDetectedRollbackJournalAndWAL, TestModifiedCheckoutWALForcedMatchingMtimeStillDetectedViaRacilyCleanGuard, TestRacilyCleanGuardHashesWithinMarginThenFastPathOnceSettled, TestTouchedButUnchangedCheckoutHashesAndStaysClean, TestOldSidecarWithoutFingerprintHashes, TestShortHeaderStampSumSucceedsAndCheckoutStateHashes, TestNoCacheFallbackWriteDuringHashRecordsNoFingerprint, TestSandwichedStampAnchorsStampedNSBeforeTheHash. Every stamp whose hash is computed here sandwiches it between two fingerprints (a write during the hash records no fingerprint) and anchors stamped_ns at the first. Filesystem-independent. Known blind spot, shared with git status: an edit that restores both size and mtime and (outside WAL mode) the change counter is invisible until the next hash. offshoot status's dirty check still always hashes; v0.2.12 |
At-rest segment checkpoints (offshoot checkpoint diffs the checkout against a reflinked .shadow and writes an LTX segment of the changed pages when the shadow verifies, identity matches, the chain is below the snapshot bound and under half the pages changed; else a snapshot; --snapshot forces one; ref checkpoint entries record kind; MCP result carries kind) |
shipped-and-tested | internal/ops/checkpoint_delta_test.go: TestSecondCheckpointWritesASegment, TestDeltaCheckpointsRoundTripUnderRandomMutations (30 rounds of random insert/update/delete, VACUUM and incremental vacuum; every checkpoint materialized and compared byte for byte), TestShadowIdentityMismatchFallsBackToSnapshot, TestDivergedShadowFallsBackToSnapshot, TestEveryBoundthCheckpointIsASnapshot, TestLargeChangeFractionForcesSnapshot, TestSnapshotOptionForces, TestNoShadowWhenReflinkUnsupported, TestForkFromSegmentHeadMaterializes, TestLateSnapshotLoserDeletesItsSnapshot, TestSnapshotLoserKeepsSharedKeyWhenWinnerIsAlsoSnapshot, TestStaleSnapshotAtNextTXIDForcesASnapshot (a snapshot a failed attempt left at the next txid is found by a one-request probe before the write, and the checkpoint overwrites it with a snapshot instead of writing a segment beside it), TestDestroyRemovesShadowAndItsTemp; internal/session's TestAtRestCheckpointAfterCloseWritesASegment; internal/store's TestCheckpointKindRoundTripsAndDefaultsEmpty; cmd/offshoot's TestCheckpointOutputNamesItsKind. The diff still reads the whole checkout and shadow (O(size) local I/O); changed pages are buffered in memory until upload, up to half the database. On a filesystem that cannot clone there is no shadow and every at-rest checkpoint is a snapshot; v0.2.12 |
Shared rollback and promote (a base pointer at the kept checkpoint / the source's head via fork's newLineageAt, not a snapshot copy; --materialize / daemon materialize / SDK materialize forces the copy; the fork-time depth floor still copies; Store.CollapseBase keeps repeated rollbacks' base spine one hop deep; results carry shared; offshoot_rollback_total{mode} / offshoot_promote_total{mode}) |
shipped-and-tested | internal/ops/repoint_shared_test.go: TestRollbackToKeptCheckpointIsShared, TestPromoteIsSharedAndSourceStaysResolvable, TestGCAfterSharedRollbackReclaimsTheAbandonedFuture, TestCompactAfterSharedRollbackDetaches, TestRepeatedRollbacksHitTheDepthFloor, TestRepeatedRollbacksToOneCheckpointKeepTheSpineFlat, TestMaterializeOptionForcesCopy; internal/store's TestCollapseBase; cmd/offshoot's TestRollbackAndPromoteOutputNameTheirMode; internal/daemon's TestMetricsSmokeOpenWriteFlushFork; both SDK suites assert the materialize wire field. Filesystem-independent. Behaviour change: offshoot rollback prints a second line after the checkout path. Compact still always copies; v0.2.12 |
Two concurrent at-rest offshoot checkpoint calls on one branch (same-kind overwrite, mixed-kind snapshot beside a winning segment, and a write between encode and stamp, all detected after the ref CAS) |
shipped-and-tested | The at-rest path takes no lease, so concurrent checkpoints remain unsupported; they are now survivable. After its CAS the winner establishes the checksum of what the store resolves the head to: (1) same-kind overwrite: store.Store.Head (optional store.Header capability: HeadObject on S3, a content hash on Local, a Get fallback otherwise) is compared with the etag its own upload returned, and on a mismatch the object's trailer checksum decides, so two byte-different encodes of the same state are not flagged; (2) mixed kind: a segment winner lists the snapshot key at its txid (snapshotMayExist, one LIST) and, when a racer's snapshot is there (it anchors the head), reads that snapshot's trailer; this covers a snapshot loser that died, or has not yet reached, its own kind-based cleanup; (3) a write between encode and stamp: the checkout's fingerprint is taken right after quiesce and compared at stamp time. The stamp is trusted only when the checkout's checksum equals the head's (so a racer whose encode equals the checkout keeps a trusted stamp and shadow); otherwise it records PostApplyChecksum=0 and a sentinel hash no file can match (untrusted:<txid>), so checkoutState reads "modified", fork warns, the shadow is dropped and the next checkpoint is a snapshot. A racer's content replacing or anchoring the head, or an object that cannot be re-read after an etag mismatch (logged), increments offshoot_checkpoint_overwrite_detected_total. A failing Head is logged and never fails the checkpoint. Remaining: a third racer that overwrites after the winner's check (ROADMAP Tier 2). internal/ops/checkpoint_overwrite_test.go: TestOverwriteWithIdenticalBytesStaysTrusted and TestOlderEncodeOverwriteReadsModified (local and fake S3), TestMixedKindRaceDistrustsTheSegmentWinner (local and fake S3; the snapshot racer is held before its CAS, so it never reaches its cleanup while the segment wins), TestWriteBetweenEncodeAndStampDistrusts, TestUnverifiableOverwriteIsLoggedAndDistrusted, TestIdenticalOverwriteIsNotFlagged, TestOverwriteDetectionSkipsWhenHeadUnsupported, TestOrphanOverwriteIsNotFlagged; internal/ops/checkpoint_delta_test.go's TestLateSnapshotLoserDeletesItsSnapshot (the live loser's cleanup); internal/store/storetest's conformance subtests HeadReturnsEtagAndSize, HeadMissingIsErrNotFound, HeadEtagChangesWhenContentChanges (Local, fake S3, live providers in CI); internal/store's TestStoreHeadFallsBackToGet; internal/daemon's TestCheckpointOverwriteCounterIsWired (limitations) |
LTX decoder hardening (internal/ltxio: frameGuard on every decode path; checkGrowth/CheckGrowth in applySegments and session.recordApply) |
shipped-and-tested | Found by the Tier 2 fuzz targets (make fuzz, nightly fuzz job). The pinned decoder, github.com/superfly/ltx, accepts objects that make it allocate up to 4 GiB from an unchecked size field, or that grow the database past the pages they carry. offshoot now refuses, before decoding them: a header page size that is not a power of two in [512, 65536]; a block-format page frame whose compressed size exceeds the LZ4 worst-case bound for one page; an LZ4-frame-format page in any shape other than the one ltx v0.5.1 wrote (FLG 0x64, BD 0x40, exactly one block that decompresses to exactly one page, then the end mark and content checksum), including legacy and skippable LZ4 magics, which pierrec/lz4 would accept; and a segment whose commit grows the database without carrying every new page (except the lock page), which ltx would apply by zero-filling (ErrUncarriedGrowth; a session whose capture would produce one fails rather than writing it). Every object this project ever wrote is unaffected: ltx v0.5.1 (frame format, 2026-07-30 to 2026-09-26, verified at every page size 512 to 65536) and v0.5.3 (block format) are the only versions shipped. internal/ltxio/guard_test.go: TestDecodeRejectsOversizedPageFrame, TestDecodeRejectsIllegalPageSize, TestFrameGuardPinsTheV051FrameShape (each refusal allocates under 8 MiB), TestApplyRejectsUncarriedGrowth, TestCheckGrowth, TestDecodesLTXv051FrameFormat (committed v0.5.1 fixtures materialize byte-identical); internal/ltxio/fuzz_test.go's FuzzDecodeSnapshot, FuzzApplySegments; internal/session/growth_test.go's TestUncarriedGrowthFailsTheSession. Not yet covered by a fixture: a 64 KiB-page or incompressible-page v0.5.1 object (ROADMAP Tier 2) |
| Object-granular reachability GC (mark every live branch's resolved chain at head + every checkpoint, transitively through base pointers; two-phase tombstone → grace → re-mark → sweep per object) | shipped-and-tested | Rewritten for copy-on-write in v0.2.0 (internal/ops/gc_test.go, gc_chain_test.go, gc_batch_test.go); counts objects, not lineages; sweep batch-deletes via store.BatchDeleter on S3 (v0.2.2); the never-delete-above-a-live-head compensating rule is epoch-aware since v0.2.4 (a provably fenced orphan is swept once its tombstone clears grace); GC fails closed, with offshoot_gc_errors_total as the alertable signal (v0.2.1) |
| TTL reaping (from the later of last-touch and lease expiry) | shipped-and-tested | internal/ops/reap_test.go, reap_cas_test.go |
Layout versioning (offshoot.json, refuse-newer-than-known) |
shipped-and-tested |
Storage backends
| Guarantee / feature | Status | Notes |
|---|---|---|
| Local directory backend | shipped-and-tested | Quickstart's default; no bucket required |
| RustFS | shipped-and-tested | rustfs/rustfs:1.0.0 (digest-pinned in ci.yml), make test-s3 passes the conformance suite for real on every PR (see README's provider table) — probe, full conformance, and the real multipart-upload subtest whose CompleteMultipartUpload precondition/checksum handling is the one thing the in-process fake can never substitute for. CI's local provider since v0.2.10 |
| MinIO | shipped | Verified through v0.2.9 (minio/minio:latest, last green 2026-09-10); MinIO withdrew its community images (Docker Hub, quay.io) and binaries (dl.min.io) in 2026, so nothing re-verifies it. Same code path as RustFS/AWS — downgraded from shipped-and-tested for honesty, not because anything broke |
| AWS S3 | shipped-and-tested | TestS3RealProvider (probe + full conformance + the multipart subtest) passes against a real AWS S3 bucket (us-east-1, 2026-08-13) — including the CompleteMultipartUpload precondition/checksum semantics the in-process fake cannot exercise |
| Google Cloud Storage | not supported | GCS's S3-interop API has no conditional writes; the attach probe refuses it outright rather than degrading |
Daemon and durability
| Guarantee / feature | Status | Notes |
|---|---|---|
| Live WAL capture (continuous, foreign-connection-safe) | shipped-and-tested | internal/capture/torture_test.go — kill -9 mid-write, repeated, verifies checksum + state equivalence after every bounce |
| Incremental LTX segments (flush ships only changed pages) | shipped-and-tested | |
| Bounded replay (snapshot every 16th flush; a read applies ≤1 snapshot + 15 segments) | shipped-and-tested | Cadence is Options.SnapshotEvery in the embeddable session library; offshoot serve -snapshot-every N exposes it to the daemon (Milestone 4 Task 6a) — see the Resource-behavior section below. The bound survives copy-on-write sharing via the two snapshot floors (v0.2.0): a shared child's counter seeds from its durable divergence, and a fork already at the depth bound materializes instead of sharing; the daemon's fork floor tracks its configured -snapshot-every (v0.2.1), while an at-rest CLI fork always uses the library default of 16 (documented in v0.2.4) |
Explicit flush durability (session status reports durable-through txid) |
shipped-and-tested | Durability advances only on flush/checkpoint; nothing ships to the store automatically between them |
Background flush interval (serve -flush-every, on by default) |
shipped-and-tested | internal/session/flush_test.go (TestAutoFlushShipsWritesWithoutManualFlush, TestIdleAutoFlushWritesNothing, TestAutoFlushFailureSurfacesAndRecovers), internal/daemon/lifecycle_test.go (TestServeFlushesAutomaticallyWithoutAnExplicitFlushOp). Default 30s, 0 disables; bounds data loss on daemon crash to at most one -flush-every interval — see README's What a flush costs. One daemon-wide cadence applied to every session it opens — see "Per-session FlushEvery override" below for what this is not |
| Settling-flush checksum-compare suppression (skip the mandatory first-open full snapshot when content is provably unchanged since the last one) | shipped-and-tested | rebaseline's very first call (internal/session/session.go) skips the settling flush only when ALL of: the checkout Open received was already proven byte-identical to the branch's head AT OPEN TIME (ops.CheckoutProven's .sum sidecar clean-and-current fast path, cleanAtOpen) AND the checksum RECORDED IN THAT SAME SIDECAR (ops.CheckoutResult.PostApplyChecksum — the LTX postApplyChecksum the checkout embodied when the sidecar was last stamped, by Checkout/Checkpoint/Rollback/Promote, or by a session's own clean Close, see the row below) exactly equals what rebaseline freshly computes from the replica once the engine's REAL startup rebase actually runs. The second condition is load-bearing, not redundant: Open can return to its caller before that real rebase finishes (see Session.rebaseline's doc comment), so a write landing in that window is folded in by the rebase's own checkpoint without ever passing through Apply; the checksum comparison is what catches exactly that case and forces a real settle instead of silently losing the write. Reading the checksum out of the LOCAL sidecar — rather than fetching it fresh from the store, which an earlier, since-reverted version of this fix did — costs Open NO store read at all beyond the two tiny ref-metadata Gets it already needed (TestReadOnlySessionWithCleanCheckoutMakesNoStoreWrites asserts exactly 2, never a Get on the head object itself, which the earlier version paid unconditionally — a full download whenever the head was a snapshot, permanently for a read-only branch that never flushes). Trust-boundary note: this reads the sidecar's recorded checksum as trusted input the same way every other same-host decision in this codebase already does (the checkout directory, .sum sidecar included, is fully trusted under the same-host/same-user unix-socket threat model — see docs/reference.md's daemon-ops threat-model note) — but it widens the CONSEQUENCE of a forged or corrupted local .sum sidecar, not the trust boundary itself: before this suppression existed, a bad sidecar could only make a checkout serve stale local bytes (a read-side effect, confined to this host); now, since a suppressed settle skips the real re-verification snapshot and lets subsequent ordinary flushes encode deltas straight off whatever the replica already held, a forged sidecar can seed what becomes the branch's durable baseline in the store, not just what's served locally. A sidecar that never recorded a checksum (older format) reads back as absent (0, fail-toward-settling) rather than being trusted — TestSettleStillHappensWithOldFormatSidecarLackingChecksum. A first-ever open, or one against a dirty/stale checkout, still settles exactly as before. internal/session/flush_test.go: TestReadOnlySessionWithCleanCheckoutMakesNoStoreWrites, TestSettleStillHappensWhenCheckoutWasStaleAtOpen, TestSettleStillHappensWithOldFormatSidecarLackingChecksum, TestSettlingSuppressionCatchesRaceWindowFold (white-box pins the race-window case directly; fails against the pre-fix cleanAtOpen-only gate), TestCleanAtOpenSessionsFirstRealFlushIsASegment (the suppressed session's first REAL flush, once one happens, is a segment continuing directly from the pre-session head — a previously-unreachable shape); M2's TestFlushLoopFlushesRebaseFoldedContentWhenOtherwiseIdle and TestFlushLoopRetriesAfterRebaseDuringUpload stay green unmodified — a mid-session rebase-on-divergence is untouched by this change. BenchmarkSessionOpen (internal/ops/fork_bench_test.go), re-run: 64MB db ~30.0ms/op, 512MB db ~199.5ms/op, B/op/allocs/op flat across that 8x size range — see docs/benchmarks.md |
Sidecar refresh on clean Close (rewrite the checkout's .sum sidecar to the branch's current head txid — and its LTX checksum, see the row above — when a session closes cleanly, not just on Checkout/Checkpoint/Rollback/Promote) |
shipped-and-tested | Session.Close now re-stamps the sidecar (ops.StampSum, including PostApplyChecksum = s.flushChecksum — the row above is what actually reads it back on the next Open) when the close is provably clean: no session error, nothing left unflushed (autoFlushPending()), at least one flush actually succeeded, the branch head hasn't moved past what this session flushed, the replica was never rebuilt by anything beyond its own mandatory startup rebase (Session.singleStartupRebase), AND — the stamped hash itself — the capture engine's OWN post-shutdown fingerprint (capture.State.Clean/MainHash, persisted only when the engine's shutdown fully verified drain+checkpoint(RESTART)+no-WAL-race+checkpoint(TRUNCATE)) is reused directly rather than independently re-derived; any of shutdown's four early-return paths (a foreign write racing the final checkpoint, most directly) leaves Clean=false and the sidecar is left unstamped. Split around the capture engine's own shutdown join: the pending decision needs the engine alive (a final DrainNow), the physical stamp needs the engine's shutdown to have already run (successfully or not — its verdict is exactly what gates this). internal/session/flush_test.go: TestCloseRefreshesSidecarSoReopenCleanSkips, TestCloseAfterFailedFlushDoesNotStampSidecar, TestSidecarNotStampedAfterMidSessionRebase, TestCloseDoesNotStampSidecarAfterUnverifiedShutdown (forces a real shutdown-verification failure via capture.ShutdownRaceHook; fails against the pre-fix version that independently re-quiesced+re-hashed instead of consulting capture.State); internal/capture/engine_test.go: TestEngineShutdownLeavesUnverifiedStateAfterRacedRestart |
| Clean-and-current checkout served without chain validation (an intentional, accepted tradeoff of the clean-skip fast path above, not a defect) | shipped-and-tested (documented tradeoff) | Once a checkout's .sum sidecar is clean-and-current — whether stamped by Checkout/Checkpoint/Rollback/Promote (Milestone 2 Task 1) or, as of the row above, by a session's own clean Close — the NEXT Checkout call for it is served straight from disk without ever touching the object store's chain, including when that chain has since been corrupted or had a member deleted out from under it. This widens (in reach, not in kind) with the sidecar-refresh row above: it now also applies after an ordinary session's clean close, not only after the four ops entry points. internal/session/flush_test.go's TestCleanCheckoutServedWithoutChainValidationAcrossClose pins this directly (positive test: a corrupted chain does NOT fail a clean-skip Checkout); TestMissingSegmentIsLoud (internal/session/segment_stress_test.go) was updated to remove the checkout+sidecar before asserting a missing chain member fails loudly, since it otherwise now legitimately clean-skips past exactly the corruption it means to detect — controller sign-off granted on this modification |
Per-session FlushEvery override (a distinct cadence per session open call, over the wire) |
deliberately deferred (YAGNI) | -flush-every is one setting for the whole daemon (internal/daemon/server.go's SetFlushEvery), applied to every session it opens; the daemon protocol's open op has no field for a caller to request a different cadence for just one session. Not scoped in Milestone 2 or planned elsewhere yet — revisit if a real caller needs mixed cadences on one daemon |
| Connection contract enforcement (live WAL-mode probe, rollback-journal detection) | not implemented — documented assumption | No live polling or journal watcher exists; the contract's WAL-only and shared-kernel clauses are trusted, and a mid-session violation is caught only by the restart-time divergence check (WAL-emptiness + main-file hash continuity in internal/capture's resume gate), which marks the checkout dirty rather than pretending continuity. Live enforcement is future work |
Reflink/clonefile fork, server-side S3 copy fork (design spec's ~40ms figure was the target this work aimed at, not a number reproduced below — see notes). Since v0.2.0 this fast path serves the materialize cases only — the floor-tripped fork, rollback or promote, --materialize, and compact — because the default fork shares instead of copying |
shipped-and-tested (single-snapshot-chain window) | Tasks 6a+6b of Milestone 2 — ops.Workspace.Fork's copySnapshotToNewLineage copies the source snapshot object directly (local: internal/reflink, clonefile(2) on darwin / FICLONE on Linux, silent plain-copy fallback otherwise; S3: store.S3.CopyObject, a real server-side CopyObject API call, no download or re-upload through this process) instead of materializing and re-encoding, when the source checkpoint's chain resolves to exactly one snapshot. Falls back to the pre-6a path once a daemon session has flushed segments past the last snapshot. S3's former ≤5GB CopyObject gate is gone: since v0.2.4, sources over 5 GiB take a multipart server-side copy (UploadPartCopy) up to S3's 5 TiB per-object ceiling, and store.ErrCopyUnsupported fires only beyond that. Measured (MinIO-local, not an AWS claim) 512MB fork ~198ms on APFS (was 2.87s pre-6a) and ~1.03s over S3 (was 4.57s pre-6b); ~9.3ms for the local copy alone in isolation from an unrelated pre-existing O(size) check, the closest this suite gets to the design spec's ~40ms figure directly — see docs/benchmarks.md |
Async fork-point upload (pending marker + fork pin on GC) |
deliberately deferred — and largely superseded by copy-on-write | Scoped in Milestone 2's original fork-performance bullet but not built: making the upload async introduces a new pending chain state that readers, GC, and fencing would all have to understand mid-upload. Since v0.2.0 the common-case fork shares and uploads nothing at all — there is no fork-point upload to make async and no pending window to caveat (noted in the v0.2.1 changelog). Only the materialize fallback (fork-floor trip) still pays a synchronous snapshot copy, made cheap by the reflink/CopyObject fast path above |
TTL, GC, and janitor
| Guarantee / feature | Status | Notes |
|---|---|---|
offshoot serve -reap-every / -gc-grace (janitor loop) |
shipped-and-tested | -reap-every 0 disables the janitor; offshoot gc remains available on demand |
| Protected branches never reaped | shipped-and-tested | |
| Live lease always defers reaping | shipped-and-tested | |
CAS-conditional ref delete (close the Destroy GetRef→lease-check→delete TOCTOU) |
shipped-and-tested | Milestone 4 Task 6b — ops.Workspace.Destroy (internal/ops/gc.go) CAS-writes a Deleting claim on the ref (store.Ref.Deleting/DeletingAt) before it does anything irreversible; store.AcquireLease refuses outright once it sees the claim (store.ErrDeleting), closing the window in which a lease acquired between Destroy's initial GetRef and its actual delete could have its branch deleted out from under it — the exact race an earlier design review documented for the M2 Destroy path. Scoped per the task's own timebox as a sibling claim field, not a unification with Reap's existing Reaping claim: Reaping's CAS mechanics are torture/race-tested and deliberately untouched here (internal/ops/reap.go/reap_test.go/reap_cas_test.go all pass unmodified). --force bypasses the protected/live-lease pre-checks only, never the claim itself — a lease that wins the underlying CAS race still survives a concurrent forced destroy. Backend split: Local gets a TRUE conditional delete (Local.DeleteIf, the same per-key lock file PutIf already uses) on top of the claim, exposed via Store.DeleteRefIf and the new store.ConditionalDeleter optional-capability interface; S3's DeleteObject has no compare-and-delete precondition at all, so DeleteRefIf falls back to an unconditional delete there and the CAS-written claim marker is the entire safety mechanism on that backend, documented as such rather than faked. A crashed Destroy (claimed, never deleted) self-heals via ops.Workspace.ClearStaleDeleteClaims (age-based, 30s — no TTL/deadline concept exists for a delete claim the way Reap's ReapDeadline gives it one), wired into the daemon janitor's janitorTick and the CLI offshoot gc on the same "report and press on" convention as a reap failure. Tests: internal/ops/destroy_claim_test.go's TestConcurrentDestroyAndAcquireLeaseHaveExactlyOneWinner (20-iteration race loop, mirroring TestConcurrentTouchAndReapHaveExactlyOneWinner: exactly one of Destroy/AcquireLease wins, the branch is never left half-deleted with a live lease on it), TestForceDestroyStillClaimGuards (same race with force=true), TestDestroySelfHealsStaleDeletingClaim (stale claim cleared, branch untouched, AcquireLease succeeds again; a fresh claim survives a pass); internal/store/local_test.go's TestLocalDeleteIfConditionalDelete/TestStoreDeleteRefIfUsesConditionalDeleteOnLocal (true CAS delete exercised directly); internal/daemon/destroy_claim_test.go's TestJanitorTickClearsStaleDeleteClaim (through a real janitor pass, not the ops call in isolation). go test ./internal/ops ./internal/store ./internal/daemon ./cmd/offshoot -count=1 -race clean; no internal/session/internal/capture changes, so no torture run was required |
Integration surface
| Guarantee / feature | Status | Notes |
|---|---|---|
| CLI (create/checkout/checkpoint/fork/rollback/promote/compact/destroy/touch/protect/unprotect/export/diff/gc/status/path/lease/serve/session/mcp) | shipped-and-tested | See docs/reference.md for every command |
| MCP server, 9 tools, at rest | shipped-and-tested | offshoot mcp; protected-branch rules enforced same as CLI; no offshoot_protect/offshoot_unprotect tool (CLI-only by design, see the Core model section above) |
MCP force gate (offshoot mcp -allow-force, off by default): an agent's force:true against a protected target/branch is refused before any mutation unless the server was started with the flag; offshoot_destroy's live-lease bypass is disabled the same way; the gate fails closed on a transient GetRef error. The gate also reaches a protected branch's own rollback/promote safety fork (<target>-pre-rollback/<target>-pre-promote), which is never itself protected: without -allow-force, offshoot_destroy, TTL-changing offshoot_touch, and offshoot_promote (when the fork is the target) calls on it are refused (guardProtectedSafetyFork) — promoting onto it would repoint (destroy) the undo point just as surely as an actual destroy — and a second offshoot_rollback of the protected branch is refused outright while its safety fork from an earlier rollback still exists, rather than silently replacing it; a plain offshoot_touch that only extends the fork's life is never blocked. This guard fails CLOSED on a transient error reading the target's own ref, since (unlike refuseForceOnProtected) there is no downstream backstop — the fork itself is never protected, so an unguarded ops call would sail through |
shipped-and-tested | internal/mcp/tools_test.go's TestMCPRefusesForceOnProtectedByDefault, TestMCPForceFailsClosedOnTransientGetRefError, TestMCPDestroyNeedsAllowForceForLiveLeaseToo, TestGuardProtectedSafetyFork (covers destroy/touch/second-rollback/promote-onto on a protected branch's fork, all proceeding once -allow-force is set, and the converse on an unprotected branch's fork), TestGuardProtectedSafetyForkFailsClosedOnTransientReadError (a one-shot injected GetRef error on the protected target's ref, or on the rollback safety-fork name itself, refuses rather than proceeding). Behavior change: earlier versions of offshoot mcp honored an agent-supplied force:true unconditionally, same as the CLI's --force; see docs/demo/mcp-walkthrough.md for the refusal firing for real, followed by offshoot_diff and a human-run CLI promote; v0.2.11 |
| MCP forks carry a TTL | shipped-and-tested | offshoot_fork takes ttl (explicit always wins) and falls back to offshoot mcp -default-ttl (default 24h; 0/none disables) when omitted; ttl:"none" overrides even a configured default. The response echoes the applied TTL and computed expiry. Reaping is offshoot mcp -reap-every (see the row below), a reachable offshoot serve daemon's janitor, or offshoot gc by hand |
offshoot mcp -reap-every DURATION|none (default 60s): daemonless TTL reaping |
shipped-and-tested | internal/mcp/tools_test.go's TestReapOnceReapsExpiredForksWhenNoDaemon, TestStartReaperTicksAndStopsOnCancel. Reaps expired forks and self-heals stranded delete claims on this cadence for as long as the MCP process is up — a bare offshoot mcp used to leave TTL expiry entirely to offshoot gc run by hand; this row is the daemonless fallback that changes that. Defers entirely (skips its own pass, logs once) when a real offshoot serve daemon is reachable at the same socket, so it's never a second writer against one store. Runs no GC either way — reclaiming a reaped branch's storage still needs offshoot gc or a running daemon; v0.2.11 |
| MCP rides the daemon (live capture, session-aware checkpoint) | shipped-and-tested | internal/mcp/daemon_test.go; offshoot_checkpoint/offshoot_fork/offshoot_checkout each probe the daemon per call and take its live path when a session is already open (and healthy) on the branch (opened by a harness — no MCP tool opens one itself, i.e. the good path requires a harness-opened session); a fenced session falls back to at-rest with a warning naming its error; no session, or no daemon, and the tool runs exactly at rest as before. offshoot_rollback/offshoot_promote (target)/offshoot_destroy refuse — even with force, which has no effect on this refusal — rather than proceed at rest when the daemon has any session (healthy or fenced) on the affected branch; offshoot_promote's source is not guarded the same way (TestPromoteFromOpenSourceProceedsAtRest) — an open session there doesn't block the promote, but the promoted state is the source's last-flushed head, not any unflushed write in that session. offshoot mcp -socket PATH names the daemon to ride |
MCP registry submission (server.json) |
drafted, not yet submitted | server.json (repo root) is a draft manifest; the registry's current schema will be validated against at submission time |
| LangGraph community-integration listing | drafted, not submitted | a community-integration PR for offshoot.langgraph.ThreadForks is drafted; blocked on PyPI publication (the install command it quotes needs to work) — see the SDK publish-pipeline row above |
| Python SDK (stdlib-only, over the unix-socket lifecycle API) | shipped-and-tested | sdk/python; publish pipeline prepared, actual PyPI publication user-gated (see the publish-pipeline row below and Milestone 3) |
| TypeScript SDK (zero runtime deps) | shipped-and-tested | sdk/typescript; publish pipeline prepared, actual npm publication user-gated (see the publish-pipeline row below and Milestone 3) |
| SDK typed surfaces (typing polish, Milestone 4 Task 7) | shipped-and-tested | TypeScript: _call's internal wire responses are typed (RawResponse/RawBranchInfo/RawCheckpointInfo/RawSessionInfo/RawAck/RawEvent mirroring internal/daemon/protocol.go's Response), not any; _call itself is @internal + stripInternal-stripped from the published .d.ts (still a real, fully-typed method at runtime and for this package's own tests, which compile against source).Verified additive: a dist/client.d.ts before/after diff shows exactly one removal (the now-internal _call line) and no other symbol changed; dist/testkit.d.ts byte-identical.Python: offshoot/py.typed (PEP 561) ships in the wheel ([tool.setuptools.package-data], confirmed via the dry-run wheel's contents); mypy --strict offshoot is clean across all four modules, fixed with annotations only (no behavior changes) — typing.cast() on JSON-derived returns, Mapping[str, str] narrowing, dict/Popen generic args, a module-private _TTL/_Seed type alias each. make test-sdks (Python 41/41, TS 49/49) and make test-pytest-plugin (58/58) green; tsc --noEmit clean; no Go changes |
SDK publish pipeline (.github/workflows/publish.yml: PyPI Trusted Publishing/OIDC, npm --provenance) |
shipped | Triggered by sdk-v* tags or workflow_dispatch; gated on the PUBLISH_ENABLED repository variable, default off — both jobs build the real sdist/wheel/npm-tarball, run twine check, and install-test them regardless of the gate, only skipping the upload step when it's off. That dry-run tier (make dry-run-sdks) runs on every PR via ci.yml's sdks job (verified locally too: python3 -m build + twine check + venv install/import for the wheel, npm pack + tarball install + import() test for the npm package, all green — see the Task 7 report). The upload path itself (pypa/gh-action-pypi-publish, npm publish --provenance) cannot be exercised without live PyPI/npm credentials that don't exist yet — actual publication is user-gated: claiming the offshoot-db PyPI name and @offshoot-db npm scope, configuring Trusted Publishing/NPM_TOKEN, and flipping PUBLISH_ENABLED are all manual, one-time, out-of-band actions (see CONTRIBUTING.md's Release process, ROADMAP.md). Manifests filled out for real publication (sdk/python/pyproject.toml: readme, urls, classifiers, authors, SPDX license; sdk/typescript/package.json: repository/bugs/homepage, files whitelist, prepublishOnly build). SDK version is a single source of truth (sdk/VERSION) checked against both manifests by scripts/check_sdk_versions.py |
| LangGraph integrations | shipped-and-tested | sdk/python-langgraph's OffshootSaver is a real BaseCheckpointSaver companion around the stock SqliteSaver, exercised by make test-python-langgraph against a compiled StateGraph (9 tests, wired into CI through its explicit [test] extra); the core SDK's offshoot.langgraph.ThreadForks keeps an existing checkpointer and maps threads to branches of the agent's separate application database, exercised end to end by examples/langgraph-rewind/ |
List databases (daemon dbs op, CLI offshoot session dbs, SDK dbs()) |
shipped-and-tested | Milestone 3 Task 1 — store.Store.ListRefs's keys, sorted; internal/daemon/metadata_test.go (TestOpDbsListsSortedDatabases, TestOpDbsOnEmptyStoreIsEmptyNotError), sdk/python/tests/test_client.py::test_dbs_lists_every_database_sorted, sdk/typescript/test/client.test.ts ("dbs lists every database sorted") |
Branch/checkpoint metadata and timestamps (Ref.Meta, per-checkpoint CreatedAt/Meta, fork/checkpoint/flush meta params, BranchInfo.touched_at/checkpoints_v2) |
shipped-and-tested | Milestone 3 Task 1 — capped at the ops layer (ops.ValidateMeta: ≤32 keys, keys ≤64 bytes, values ≤512 bytes; internal/ops/ops_test.go's TestValidateMetaCaps and the *RejectsMetaOverCap tests). Fork's meta sets the child branch's Ref.Meta (branch-level lineage — no row-level provenance); Checkpoint's meta sets that one checkpoint's own Meta. Every checkpoint-creating call site (Create, Checkpoint, Fork, Promote, and a daemon session's named flush) stamps CreatedAt (RFC3339 UTC) — internal/ops/ops_test.go's Test*StampsCreatedAt* tests, internal/session/flush_test.go's TestNamedFlushStampsCheckpointMetaAndCreatedAt. No store schema bump: new fields are omitempty, round-trip-and-old-shape-decodes-clean tested exactly like the TTL fields (internal/store/store_test.go's TestRefMetaAndCheckpointFieldsRoundTripAndOldRefsDecode, including a hand-written-JSON old-shaped-ref decode, not just a Go zero-value round trip). Wire compat: BranchInfo's pre-existing checkpoints (names-only) field is untouched; checkpoints_v2 ({name, txid, created_at}) rides alongside it; an old-client-shaped request (no meta field at all, constructed as literal JSON bytes, not a Go zero value) still works against every touched op — internal/daemon/metadata_test.go's TestWireCompatOldShaped*RequestWorks. CLI: fork/checkpoint gain repeatable --meta k=v; SDK parity both languages. Bug fixed in the same pass: Rollback's kept-checkpoint epoch relocation used to silently drop CreatedAt/Meta — internal/ops/ops_test.go's TestRollbackPreservesCheckpointCreatedAtAndMeta |
MCP tool metadata exposure (offshoot_fork/offshoot_checkpoint taking a caller-supplied meta map) |
shipped-and-tested | offshoot_fork/offshoot_checkpoint take meta (TestForkAndCheckpointCarryMeta); v0.2.11 |
| MCP tool annotations (explicit read-only/destructive/idempotent hints, openWorld false) | shipped-and-tested | TestToolAnnotationsClassifyEveryTool; v0.2.11 |
| MCP structuredContent on every success | shipped-and-tested | TestStructuredContentAccompaniesProse; v0.2.11 |
offshoot_touch |
shipped-and-tested | TestTouchExtendsALeasedAttempt; v0.2.11 |
Claude Code plugin + marketplace (plugin/, .claude-plugin/marketplace.json) |
shipped | make check-plugin in CI; hooks advisory only; v0.2.11 |
create --from reach: daemon protocol op, SDK create(..., from_path=...)/{fromPath}, .db-file seeds in both fixtures |
shipped-and-tested | The daemon's create op now accepts an optional path request field (a server-side, absolute path to an existing SQLite file to import via ops.CreateFrom instead of minting an empty database); a relative path is refused with an error containing "absolute", and create with path set is refused over HTTP with daemon: create with path is not available over HTTP; use the local socket — the same same-host path-trust story export already uses (the daemon reads a path the caller names, trusted because same-host/same-user unix-socket access already implies that trust), not the upload-channel alternative once considered; plain create (no path) keeps working over HTTP unchanged. internal/daemon/lifecycle_test.go's TestCreateFromPathImportsAnExistingFile, TestCreateWithRelativePathRefused, TestCreateFromPathToNonSQLiteFileErrors; internal/daemon/http_test.go's TestHTTPCreateWithPathRejectedPlainCreateStillWorks. Python Client.create(db, from_path=None) and TypeScript create(db, { fromPath }) resolve the path to absolute client-side before sending it as path (sdk/python/tests/test_client.py, sdk/typescript/test/client.test.ts). Both fixtures ride the same reach: offshoot.pytest_plugin's offshoot_seed/_SeedFactory and the TypeScript testkit's seedOnce accept a path to an existing SQLite .db file (detected by its SQLite format 3\0 header, never by extension alone), import it via create(..., from_path=...), and fork from the imported init checkpoint with fingerprint db:<sha256 of file bytes> — sdk/python/tests/test_pytest_plugin.py::test_db_file_seed_imports_and_forks_from_init and the TypeScript twin in sdk/typescript/test/testkit.test.ts ("seedOnce: a path to an existing SQLite db file imports it and forks from its init checkpoint"). MCP deliberately does not get this — excluded by design, not deferred: an MCP tool must never import an arbitrary host file named by an agent, so no offshoot_create-style tool exists and no from/path argument was added to any MCP tool. CLI offshoot create <db> --from file remains available too, unchanged; v0.2.11 |
MCP session open/close (an offshoot_open/offshoot_close tool pair) |
deliberately deferred | Milestone 2's "MCP rides the daemon" bullet named this gap and pointed at Milestone 3's fixture plugin as the eventual lifecycle owner; that fixture (offshoot.pytest_plugin, the row above) and its TypeScript counterpart (testkit, the row below) both now EXIST and ship in this milestone — but neither is an MCP tool, and no MCP open/close tool pair was built alongside them. The reasoning stands exactly as amended: an MCP-opened session has no natural owner responsible for closing it — a bare tool call has no equivalent of a fixture's guaranteed teardown or a with block's guaranteed exit, so an agent that opens and then gets interrupted, crashes, or simply forgets to call the matching close tool leaks a lease and a branch, precisely the class of leak this whole milestone's TTL/background-flush work exists to prevent. The good path for MCP daemon-mode capture remains what docs/reference.md's offshoot mcp entry and the README's MCP section already document: a harness (the SDKs, offshoot session open, or a custom loop) opens the session and stays responsible for closing it; MCP's checkpoint/fork/checkout tools ride that session without ever opening or closing one themselves. See docs/recipes/claude-agent-sdk.md for the concrete "open before, close after" wiring this implies for an agent harness |
Export (ops.Workspace.Export, CLI offshoot export, daemon export op, SDK export() both languages) |
shipped-and-tested | Milestone 3 Task 2 — materializes any checkpoint (or head) to a plain SQLite file anywhere, with zero ongoing relationship to the store: no .sum sidecar, no lease. Refuses to overwrite an existing destination unless force; the write is atomic (temp file in the destination's OWN directory, renamed into place only once every chain member is fetched and verified — reuses materializeChainAt/ltxio.MaterializeChain's existing atomicity, so a failed export never leaves a partial file, pinned by internal/ops/export_ops_test.go's TestExportAtomicOnMidWriteFailureLeavesNoTempOrPartialFile, which corrupts a segment's checksum mid-chain and asserts the destination directory ends up completely empty). Reads the branch's last DURABLE state from the store, never a live session's checkout — an open daemon session's unflushed writes are NOT in the export, asserted directly over the wire by internal/daemon/export_test.go's TestOpExportMissesUnflushedSessionWrites (write through a live session's checkout without flushing, export, assert the row count matches the pre-write durable state; then flush and export again, assert it now includes the write). Daemon export op requires an ABSOLUTE destination path (same-host/same-user unix-socket trust model — see docs/reference.md's daemon-ops section) and is deliberately NOT guarded by the open-session refusal checkout/rollback/promote use, since it never touches the live checkout. CLI parses the db[@branch[@checkpoint]] triple-@ target form (ops.ParseExportTarget) |
Read-only historical checkout (ops.Workspace.CheckoutAt, CLI offshoot checkout --at --read-only, daemon checkout-at op, SDK checkout_at()/checkoutAt()) |
shipped-and-tested | Milestone 3 Task 2 — materializes a NAMED checkpoint (no head alias) into a dedicated cache path, <store-root>/checkouts-ro/<db>/<branch>@<checkpoint>.db, chmod 0444, distinct from and never touching the writable checkouts/<db>/<branch>.db path or its .sum sidecar/lease — safe to call alongside a live daemon session on the SAME branch (internal/daemon/export_test.go's TestOpCheckoutAtSafeAlongsideOpenSessionOnSameBranch), unlike checkout/rollback/promote's refuseIfClaimed guard. Nothing in this codebase opens a checkouts-ro file through a live capture engine, so internal/dbfile's stray-close lock hazard doesn't apply to it either. A repeat call with force=false is a pure cache hit — no store access at all, not even a GetRef — because a checkpoint's content is immutable once created (internal/ops/export_ops_test.go's TestCheckoutAtWithoutForceIsACacheHitWithNoStoreAccess, which destroys the branch entirely between calls and proves the second call still succeeds); force=true re-reads the store and can fail once the checkpoint is gone (TestCheckoutAtForceRematerializesAndCanFail). Known, documented tradeoff: if a branch is destroyed and recreated with a same-named checkpoint, the cache path can serve stale content until force or a rm -rf of checkouts-ro — see README's Resource behavior |
offshoot.pytest_plugin fixture plugin (offshoot-db[pytest], sdk/python/offshoot/pytest_plugin.py, pytest11 entry point) |
shipped-and-tested | Milestone 3 Task 4 — the seed-once-fork-many paved road: offshoot_daemon (session-scoped: locates the binary via OFFSHOOT_BIN env or PATH, pytest.skips with install instructions when neither has one; temp store + socket; terminates at session end), offshoot_db (session-scoped NAMED-SEED FACTORY — offshoot_db(name="default", seed=None), memoized per name; creates eval-{name}, runs seed — a callable(path) or a SQL string — checkpoints it seed; seed=None falls back to the offshoot_seed ini option, a path to a .sql file, the zero-code default-seed case), offshoot_fork (function-scoped — offshoot_fork(seed_handle=None), forks a fresh worker-safe-named branch (t-{worker}-{testname-hash}-{n}, sanitized via offshoot.langgraph._sanitize) from the seed checkpoint with a TTL (default 1h, offshoot_ttl ini-overridable), opens a session, returns an object with .path/.client/.db/.branch; teardown closes the session then destroys the branch — a destroy failure is a UserWarning, never a test failure). offshoot_dump(path) -> str is the golden-file helper (sqlite3 .dump text) — deliberately: SQLite's on-disk bytes are not deterministic for identical logical content, so golden assertions compare offshoot_dump output, NEVER raw bytes (sdk/python/tests/test_pytest_plugin.py::test_offshoot_dump_is_the_right_comparison_not_bytes proves a vacuum changes the bytes but not the dump). Package extra offshoot-db[pytest] (pytest>=7); the pytest11 entry point is registered unconditionally in pyproject.toml but the module import-guards pytest, so the base offshoot-db install stays stdlib-only (verified: make test-sdks — the plain-unittest suites — passes with no pytest installed at all; make dry-run-python-sdk's built wheel import offshoot-tests clean in a pytest-free venv too).xdist stance (a locked design decision): one offshoot daemon + temp store per xdist worker (pytest has no cross-worker fixture-sharing mechanism), so seed cost is paid once per worker, not once total — measured on this repo's own smoke test (test_xdist_two_worker_run_passes_and_measures_seed_cost, a CREATE TABLE + 200-row seed, macOS arm64): ~80-90ms per worker, ~170ms of total seed work under -n2 against ~85-90ms of wall-clock time (workers seed concurrently). Full rationale and the numbers live in pytest_plugin.py's module docstring and sdk/python/README.md's pytest-fixture-plugin section.Found+fixed in passing: _run_seed's SQL-string path now wraps an unwrapped multi-statement seed in one transaction — sqlite3.Connection.executescript otherwise runs every statement as its own autocommit transaction, measured ~1.6s for 200 unwrapped INSERTs vs. ~17ms wrapped (~100x) — pinned by test_seed_factory_wraps_multi_statement_seed_in_one_transaction/test_seed_factory_wraps_unterminated_single_statement_seed/test_seed_factory_respects_seed_with_its_own_begin_commit.Testing: 50 direct-daemon logic tests (naming, TTL, teardown ordering close-then-destroy, factory memoization, skip-when-no-binary, destroy-failure-warns, seed-transaction-wrapping, dump-shaped/comment-prefixed seeds, dead-daemon-connection wrapping, seed-mismatch detection) plus 8 pytester-driven scenarios (entry-point autoload, fork-per-test isolation, the xdist 2-worker run that also measures the seed-cost numbers above, OFFSHOOT_BIN misconfigured fails loud, offshoot_require_binary ini fails loud, offshoot_require_binary ini defaults to a skip, offshoot_seed ini resolves relative to rootdir not cwd, teardown warnings don't escalate under -W error) — 58 tests total (re-counted directly for this row: grep -c '^def test_' sdk/python/tests/test_pytest_plugin.py and a live make test-pytest-plugin run both report 58/58 passing; the pytester tier grew from the plan's originally-scoped 2-3 scenarios to 8 during Task 4's review round) — make test-pytest-plugin, wired into ci.yml's sdks job (installs offshoot-db[pytest] + pytest-xdist AFTER make test-sdks already proved pytest-free, so the ordering itself is part of the proof) |
TypeScript testkit (sdk/typescript/src/testkit.ts, @offshoot-db/client/testkit) |
shipped-and-tested | Milestone 3 Task 5 — the vitest/jest counterpart of offshoot.pytest_plugin: framework-agnostic FUNCTIONS, not fixtures (no vitest/jest runtime dependency — this SDK stays zero-runtime-deps — and nothing registers itself automatically; the caller wires these into their own beforeAll/afterEach). startDaemon(opts?) -> Promise<DaemonHandle> locates the binary via OFFSHOOT_BIN env or PATH (a clear error naming both when neither has one — no skip tier, since this module has no test-framework integration to skip through, unlike the pytest plugin), starts it on a fresh temp store + socket, returns { sock, store, proc, stderrTail(), stop() }.seedOnce(daemon, {name?, seed}) -> Promise<SeedHandle> is a NAMED-SEED cache memoized per (daemon, name); seed is a SQL string, a path to a .sql file (detected: no newline, ends .sql, file exists — there's no ini file here to hold that decision separately the way pytest's offshoot_seed option does), or an async (dbPath) => void callback; a later call for the same name with a different seed (fingerprinted: SQL/path text by content hash, callables by identity) throws rather than silently keeping the first one, mirroring offshoot_db's mismatch semantics exactly. Ports _skip_leading_noise/_seed_opens_own_transaction verbatim, so a sqlite3 .dump's text (PRAGMA before BEGIN TRANSACTION) works as a seed unmodified and a plain multi-statement seed is wrapped in one transaction rather than paying one autocommit transaction per statement.forkPerTest(daemon, seedHandleOrName, opts?) -> Promise<ForkedSession> forks a fresh, worker-safe-named branch (t-{worker}-{sanitized-hint}-{n}; worker id from VITEST_POOL_ID/JEST_WORKER_ID when present, else "local" — sanitize is offshoot.langgraph._sanitize ported verbatim, since there's no TypeScript langgraph module to import it from) from the seed checkpoint with a TTL (default "1h", opts.ttl overrides), opens a session, returns { path, db, branch, client, flush(name?), close() }; close() closes the session and destroys the branch, and either failing is a console.warn, never a throw. Each ForkedSession owns its own connection and teardown independently of every other one, so — unlike the pytest plugin's shared-per-test _ForkFactory.teardown() loop — one fork's cleanup trouble can never block another's, by construction rather than an explicit try/except around each step. dump(path) -> Promise<string> is sqlite3 <path> .dump's text output, the same golden-comparison method as offshoot_dump: SQLite's on-disk bytes are not deterministic across writes with identical logical content, so golden assertions must compare dump output, never raw bytes (test/testkit.test.ts's "is the right comparison, not raw bytes" test proves a VACUUM changes the bytes but not the dump, mirroring the Python plugin's own such test). Since this SDK ships no SQLite driver, seeding and dumping both shell out to the sqlite3 CLI, same as test/client.test.ts already does.Exported as a new package.json exports subpath, @offshoot-db/client/testkit (the root export gained an explicit exports entry alongside it; main/types kept for back-compat with tools that don't read exports).Testing: sdk/typescript/test/testkit.test.ts, run via node:test against a real daemon — mirrors the Python plugin's direct-daemon tier (binary resolution naming both OFFSHOOT_BIN and PATH, naming, TTL default+override, seedOnce memoization + fingerprint mismatch, a .dump-shaped seed, a path-to-.sql seed, an async-callback seed, fork-per-test isolation, the string-name shorthand, teardown-warn-not-throw against a killed daemon, a golden-file-not-bytes scenario) plus one integration test wiring startDaemon/seedOnce/forkPerTest into node:test's own before/after/beforeEach/afterEach the way a real suite would — 22 tests in testkit.test.ts itself (re-counted directly: node --test test-dist/test/testkit.test.js in isolation reports tests 22/pass 22), wired into the existing make test-ts-sdk target alongside client.test.ts's own suite (no new Makefile target needed: it already globs test-dist/test/*.test.js; the CI-verified combined TS count lives in the typed-surfaces row above). make dry-run-ts-sdk's tarball exact-match assertion updated to expect dist/testkit.js/dist/testkit.d.ts alongside dist/client.*, plus a second install-then-import() check for the @offshoot-db/client/testkit subpath specifically (not just the package root) |
Branch diff, content-aware and everywhere (offshoot diff, ops.Workspace.MaterializeForDiff, ops.DiffSummary/ops.FormatDiffSummary, daemon diff op, SDK diff(), MCP offshoot_diff) |
shipped-and-tested | Milestone 3 Task 6 — offshoot diff <db>[@branch[@checkpoint]] <db>[@branch[@checkpoint]] [--summary] [--table T] materializes both sides READ-ONLY (never a live checkout, never a lease) and either streams sqldiff's output over them (default) or prints a stdlib-only, sqldiff-free content-aware per-table summary (--summary): for a table present on both sides with matching columns, rows added/removed/changed by the table's declared primary key when it's actually unique on both sides, else its internal rowid (TestDiffSummaryIsContentAware, TestDiffSummaryNonUniqueDeclaredPKFallsBackToRowid), with schema changes flagged (a SchemaChanged table's STATUS cell gets " (schema)"); a table whose column list differs between sides is reported — with row counts — but not compared. --table restricts either mode to one table. This is no longer CLI-only: the daemon protocol's diff op (left, right, table, full, max_bytes request fields; a diff response object with tables/totals and, when full was set, capped sqldiff SQL) returns the same summary as JSON over the wire — allowed over the HTTP surface (unlike export) since its response is always JSON built from store data, never a filesystem path — pinned by TestDiffOpReturnsContentAwareSummaryOverTheWire. Both SDKs expose diff(left, right, *, table=None, full=False, max_bytes=None) (Python) / diff(left, right, opts) with DiffOptions{table?, full?, maxBytes?} (TypeScript), tested by sdk/python/tests/test_client.py::test_diff_is_content_aware_and_needs_no_sqldiff and sdk/typescript/test/client.test.ts's "diff: content-aware summary over the wire, no sqldiff". MCP's ninth tool, offshoot_diff (database, left, right, table?, full?, max_bytes?, read-only annotations, full capped at 32768 bytes by default and 262144 at most), is tested by TestDiffToolComparesAttemptsContentAware. Materialization split (the staleness decision, locked and documented on MaterializeForDiff's own doc comment): a named checkpoint uses the existing CheckoutAt read-only cache (immutable content, legitimately reusable across diffs); a bare head target CANNOT be cached that way — head moves — so it is always freshly Exported to a private temp file removed after the diff, proven directly by internal/ops/diff_test.go's TestMaterializeForDiffHeadSideAlwaysReflectsANewWrite and the CLI-level TestDiffCLIHeadSideReflectsNewWriteNotStaleCache (a write + checkpoint between two --summary calls against the same head target changes the reported delta, not a stale cache). Both sides are opened file:<abs>?mode=ro&immutable=1 — verified empirically against a real chmod 0444 file (internal/ops/diff_test.go's TestTableRowCountsOpensReadOnlyEvenOnA0444File) that SQLite accepts a mode=ro URI parameter alongside the driver's always-requested SQLITE_OPEN_READWRITE|SQLITE_OPEN_CREATE flags (SQLite only rejects a mode LESS restrictive than the flags argument; ro is strictly more restrictive, so this is not a documented-but-untested claim); the right side is ATTACHed to the same connection so both sides are compared in one query, SetMaxOpenConns(1) pinning every query in a call to that attachment. sqldiff's absence (a SEPARATE binary from sqlite3 on most platforms) produces a clear, per-OS-hinted error rather than a bare "executable file not found" — the macOS hint (brew install sqldiff) was corrected mid-task after installing Homebrew's general sqlite formula locally and confirming it does NOT ship sqldiff (only the dedicated sqldiff formula does); the Debian/Ubuntu hint (sqlite3-tools) was verified against a real ubuntu:24.04 container's apt-cache show sqlite3-tools output listing /usr/bin/sqldiff, and .github/workflows/ci.yml's Linux job now installs that package (alongside the pre-existing plain sqlite3) so the CLI tests that exercise a REAL sqldiff invocation (TestDiffCLISqldiffPresentStreamsOutput, TestDiffCLIIdenticalSidesProduceNoSqldiffOutput) run in CI rather than permanently skipping (they gate on exec.LookPath("sqldiff") the same way requireSQLite3 gates on sqlite3 itself, so a local dev machine without sqldiff still skips cleanly); the daemon's full mode fails loud instead when sqldiff isn't on the daemon host's PATH. Cross-db diff (two entirely different db names) is explicitly supported and tested (TestDiffSummaryCLIWorksAcrossTwoDifferentDatabases) — a legitimate shape for eval comparisons, everywhere except MCP (one database; left/right are branch[@checkpoint] there, not db@branch[@checkpoint]). docs/diff.md documents the command, the raw by-hand export-twice-then-sqldiff recipe, and links the FAQ's no-merge stance; v0.2.11 |
BranchBench topologies harness (cmd/branchbench, make bench-branchbench; five macrobenchmark workflows from BranchBench, arXiv:2604.17180, each run against its own fresh local store) |
shipped-and-tested | go test ./cmd/branchbench's TestEveryWorkflowRunsAtQuickScale runs all five topologies at -quick scale in the normal go test ./... run, so this stays exercised on every change, not just when someone remembers to run the full benchmark by hand. The full-scale table — every workflow completing every step, 2,310/2,310 worker-steps, 0 CAS retries, nothing aborted or timed out — is pasted verbatim in docs/benchmarks.md's "BranchBench topologies (v0.2.12)" section. The parameter tuples are BranchBench's own; the numbers are ours, re-derived against our own generator/schema/SQL (the upstream harness at github.com/ElaineAng/db-fork carries no license file, so none of it is vendored here). The paper's hosted-Postgres-family numbers (Dolt 170/1000 worker-steps on MCTS before a 2h timeout; Neon capped by its 20-live-branch ceiling) are quoted for context in that section, never re-run by us; v0.2.11 |
Observability and security
| Guarantee / feature | Status | Notes |
|---|---|---|
offshoot status (branch state, checkpoints, TTL remaining) |
shipped-and-tested | |
session status (durable-through txid, epoch, holder, errors) |
shipped-and-tested | |
| Structured branch-state-transition logging | shipped-and-tested | internal/session/session_test.go (TestSessionTransitionLogsOpenedFlushedClosed), internal/session/flush_test.go (TestAutoFlushTransitionLogsRecordKindAutoAndFailure) — every session state transition (opened; flushed, tagged kind=manual/kind=auto with its txid; flush-failed with the error, ErrClosed races excluded; fenced with the terminal cause; closed) writes one offshoot: session: db@branch: event key=value ... line to stderr, matching the daemon janitor's existing offshoot: janitor: ... line family |
Metrics registry + instrumentation (internal/metrics, offshoot_* names locked) |
shipped-and-tested | Milestone 4 Task 2 — hand-rolled, zero-dependency, concurrent-safe Prometheus text-exposition registry (Counter/Gauge/Histogram with fixed buckets, CounterVec/GaugeVec for bounded label sets, Registry.WritePrometheus); zero-dep rationale is one sentence on the package doc comment. Every metric on the plan's LOCKED list is registered by internal/daemon's newMetrics — offshoot_build_info{version}, offshoot_sessions_open, offshoot_capture_lag_bytes{db,branch}/offshoot_durable_age_seconds{db,branch} (open sessions only, computed at SCRAPE TIME from the sessions map via a Registry.Collect callback, not continuously), offshoot_flush_total{result,kind}/offshoot_flush_duration_seconds, offshoot_fork_total{path}/offshoot_fork_duration_seconds, offshoot_checkpoint_duration_seconds, offshoot_reap_total, offshoot_gc_tombstoned_total/offshoot_gc_deleted_total/offshoot_gc_backlog, offshoot_ro_cache_bytes/offshoot_ro_cache_evictions_total (driven by the janitor's ro-cache pass as of Task 5 — see the Resource-behavior section below), offshoot_janitor_runs_total{result}.Two families were added after the lock, in v0.2.1 (additions are fine; renames are not): offshoot_fork_mode_total{mode="shared"|"materialized"} (the copy-on-write storage mode at fork time, orthogonal to offshoot_fork_total{path}) and offshoot_gc_errors_total (janitor GC passes that returned an error — the alertable signal for a fails-closed GC that would otherwise stall silently), bringing the family count to eighteen; offshoot_rollback_total{mode}/offshoot_promote_total{mode} (v0.2.12) and offshoot_checkpoint_overwrite_detected_total (Tier 2) bring it to twenty-one — see docs/operations.md.Instrumentation: internal/ops's ObserveFork/ObserveCheckpoint are package-level, nil-checked injected hooks (ops must not import internal/metrics) the daemon assigns once at NewServer construction; internal/session's OnTransition hook rides the SAME transition-log call site Milestone 2 added (logTransition), not a new one, feeding flush counters/durations from the existing "flushed"/"flush-failed" events. WritePrometheus output is golden-file-tested (internal/metrics/metrics_test.go) and validated against promtool check metrics (TestPromtoolCheckMetrics/TestPromtoolCheckRealMetrics, loud-skip-if-absent locally, installed explicitly in ci.yml's new metrics-lint job). Exposed over HTTP as of Task 3 — see the next row.Concurrent-scrape safety (T2 review carried item, closed in Task 3): Registry.WritePrometheus now serializes end to end via a scrapeMu mutex — a multi-step Collect callback (the session-gauge collector below Resets then repopulates two GaugeVecs) is otherwise not atomic across two concurrent scrapes, which could tear the output (e.g. offshoot_sessions_open reporting N while offshoot_capture_lag_bytes carries fewer than N samples in the SAME response). internal/metrics/metrics_test.go's TestWritePrometheusSerializesConcurrentCollectorRuns and internal/daemon/http_test.go's TestConcurrentMetricsScrapesWithSessionChurn (real HTTP, real session open/close churn) both pin the invariant and are confirmed to fail reliably without the mutex |
Prometheus /metrics HTTP endpoint |
shipped-and-tested | Milestone 4 Task 3 — GET /metrics (internal/daemon/http.go's handleMetrics) calls the registry above directly; token-gated like everything but /healthz. See the row above for the concurrent-scrape fix and its tests |
| HTTP binding + single-token auth | shipped-and-tested | Milestone 4 Task 3 — internal/daemon/http.go. Off by default; serve -http 127.0.0.1:PORT binds loopback with no further ack needed; a non-loopback bind additionally requires -http-allow-non-loopback AND an explicit -token/OFFSHOOT_TOKEN (two DISTINCT startup errors if either is missing — ValidateHTTPBind, TestValidateHTTPBindErrorsAreDistinct). Token: -token/OFFSHOOT_TOKEN, else auto-generated (GenerateToken, 256-bit) and printed once to stderr (loopback only). Every request but GET /healthz requires Authorization: Bearer <token>, checked with crypto/subtle.ConstantTimeCompare (checkAuth); a token is never logged in full again after that one startup line — only its 8-character TokenFingerprint appears in ongoing output (TestHTTPTokenRedaction, TestHTTPAutoGeneratedTokenPrintedOnceThenOnlyFingerprint). Routes: POST /rpc (the same Request/Response JSON and Server.dispatch the unix socket uses — TestHTTPRPCParityWithSocket; 1MiB body cap via http.MaxBytesReader, 413 on overflow, connection still usable after — TestHTTPRPCBodySizeLimit; Content-Type: application/json required), GET /debug/pprof/* (net/http/pprof, same Bearer auth — TestHTTPPprofBehindAuth), GET /healthz (unauthenticated, {"ok":true,"sessions":N}). http.Server timeouts are explicit and justified in http.go's doc comment (ReadHeaderTimeout 5s, ReadTimeout 30s, WriteTimeout 90s — sized to leave headroom for /debug/pprof/profile's default 30s capture window, IdleTimeout 2m). Shutdown ordering: the HTTP listener is closed (http.Server.Close, immediate — matching the unix socket's own force-close-live-connections philosophy) alongside the unix listener in Server.Shutdown; an HTTP op=shutdown request gets the same respond-then-shutdown-trigger ordering fix the unix socket got in Milestone 3 (TestHTTPShutdownRespondsBeforeClosingRequestingConn), and a generic hammer test (TestHTTPShutdownWhileRequestsInFlightIsSafe) proves no panic/hang under -race while Shutdown races in-flight requests. Threat model: single-tenant, same-host-or-trusted-network auth — the token is a shared secret, not a multi-tenant isolation boundary; see docs/reference.md's -http ADDR section and docs/operations.md for the full operator-facing writeup (Milestone 4 Task 8) |
Branch state taxonomy (active / pending / error / dirty / detached / idle) |
shipped-and-tested | Milestone 4 Task 1 — ops.BranchStateAt (internal/ops/status.go) computes active/dirty/detached/idle from ref+lease+checkout-sidecar truth alone (no daemon dependency); a daemon layers pending/error on top from its own in-memory session map (internal/daemon/server.go's Server.branchState) — see docs/reference.md's Branch states table for the full precedence (error > pending > active > dirty > detached > idle) and why idle is a deliberate addition the design spec's original taxonomy lacked (it assumed a daemon was always present). Computed only, nothing persisted. Surfaced in offshoot status (CLI/at-rest), the daemon branches op's BranchInfo.state (always present, no omitempty — wire-additive so pre-Milestone-4 SDKs against a new daemon are unaffected), and both SDKs' Branch.state (defaults to "" against a pre-Milestone-4 daemon that never sends the field). internal/ops/states_test.go constructs each ops-level state directly (including detached via a Promote whose best-effort checkout refresh is forced to fail, per that op's own doc comment) plus an exactly-one-state invariant test across a matrix workspace; internal/daemon/branchstate_test.go covers pending (the openDelay test hook holding an open in flight) and error (lease-theft fencing, mirroring internal/session/renew_test.go's own technique) including the precedence case where a fenced session's ref shows an active lease held by the thief but the daemon still correctly reports error |
Protected-branch flag (default on for main) |
shipped-and-tested | Enforced by CLI, daemon, and MCP alike |
Operator documentation (docs/operations.md) |
shipped | Milestone 4 Task 8 — the metrics reference table, branch-states table, event schema, budget mechanics, and HTTP/auth threat model in one operator-focused page, cross-linked from docs/reference.md's wire-level sections; its own metrics table is verified against a real scrape (go build + serve -http + curl /metrics, # TYPE line count matched against the table row count). docs/recipes/kubernetes.md ships alongside it: a real sidecar manifest (docs/recipes/k8s/offshoot-sidecar.yaml), schema-validated with both kubectl apply --dry-run=client and --dry-run=server against a disposable local rancher/k3s control plane (no cluster was otherwise available in this environment). "shipped" not "shipped-and-tested" per this page's own legend — no automated test enforces documentation accuracy going forward |
| TLS on the HTTP listener | deliberately deferred | Milestone 4's self-review scoped -http to loopback-by-default plus a shared-secret token, explicitly not TLS — a non-loopback bind is documented as needing a trusted network boundary (VPN, private subnet) of its own instead. Revisit with real non-loopback demand; see docs/operations.md |
| Per-branch at-rest metrics by default | deliberately deferred | A gauge per branch that has never been opened this process would scale label cardinality with total branch count rather than open-session count — offshoot_capture_lag_bytes/offshoot_durable_age_seconds stay open-sessions-only by design (see docs/operations.md). A dbs-scoped scrape option for at-rest branches was named as a future addition in the Milestone 4 plan, not built this pass |
| Metrics push/remote-write | deliberately deferred | GET /metrics is pull-only, matching every other route on the opt-in HTTP listener; no push-gateway or remote-write integration was scoped for Milestone 4 |
Eventing: event bus, socket subscribe op, GET /events SSE |
shipped-and-tested | Milestone 4 Task 4a — internal/daemon/events.go's in-daemon bus fans one versioned event schema out to subscribers over two transports that share the SAME encoder (encodeEvent): the unix socket's subscribe op and HTTP's GET /events (SSE). type is session_opened/flushed/flush_failed/fenced/session_closed (all fed from the SAME internal/session transition-callback call site Task 2 already hooked — session.OnTransition is a single package-level func var, so wireEvents composes onto whatever Task 2's wireHooks already assigned rather than clobbering it; internal/session itself is unmodified), reaped (fed from the janitor's Reap pass in janitorTick), and evicted (as of Milestone 4 Task 5, fed from the janitor's ro-cache LRU eviction pass in that same janitorTick — see the Resource-behavior section below). Never blocks the daemon (Global Constraint): eventBus.publish is entirely non-blocking — a subscriber whose bounded buffer (eventSubscriberBuffer, 64) is full is immediately removed, sent one terminal {type:"dropped_slow_consumer"} event, and has its channel closed, all without the publisher (a session transition or the janitor) ever waiting — TestSlowSubscriberDroppedSessionKeepsFlushing proves a subscriber that never reads its channel is dropped while a concurrently write-heavy session keeps flushing successfully throughout. Socket subscribe acks THEN the connection permanently leaves request/response mode and streams line-per-event JSON until disconnect (handle() special-cases it exactly like it already special-cases shutdown, subscribing to the bus before sending the ack to close an ack-vs-transition race); SDKs must use a fresh, dedicated connection — it is unix-socket-only, refused with a pointer to GET /events if sent over HTTP POST /rpc (TestSubscribeOverHTTPRPCRefused). GET /events requires the same Bearer auth as everything but /healthz, emits a periodic : ping keepalive comment and — the loud warning Task 3's http.go left for this task — bounds its writes so Task 3's 90s http.Server.WriteTimeout can never hard-cut a long-lived stream: rather than clearing the write deadline once, permanently (a review round caught that this let a subscriber that stayed CONNECTED but simply stopped reading pin the handler goroutine and its connection's file descriptor open forever, since nothing ever re-armed a bound on it again), handleEvents re-arms a fresh eventWriteDeadline-out deadline (default 45s) immediately before every write — the header write, each event, each ping — via http.ResponseController; a live stream re-arms this forward on every successful write (and at least every keepalive tick) so it's never affected, while a genuinely stalled reader's write now times out and the handler gives up. streamEvents (the unix socket side) gets the identical per-write re-arm before every c.Write. TestSSEStreamSurvivesPastWriteTimeout proves the live-stream side structurally and cheaply (a test-only httpWriteTimeout var shrunk to 150ms, stream proven alive well past it via a real event delivered afterward — no 90-second sleep); TestStalledSocketSubscriberConnectionIsClosedWithinWriteDeadline/TestStalledSSESubscriberConnectionIsClosedWithinWriteDeadline prove the stalled-reader side through the REAL write path (an oversized event forces an actual blocking write, not a simulated one), both confirmed to fail reliably against the pre-fix code. TestSSEParityWithSocketSubscribe subscribes both transports before a real session lifecycle and asserts identical observed event sequences. No internal/session changes, so no torture run was required for this task (internal/daemon/events_test.go, full suite green under -race). Task 4b — SDK stream helpers: both SDKs ship a thin events() helper over the subscribe op — Python Client.events() (sdk/python/offshoot/client.py) is a generator yielding Event dataclass instances, TypeScript Client.events() (sdk/typescript/src/client.ts) is an AsyncGenerator<OffshootEvent> — each opening its OWN fresh, dedicated socket connection (never the caller's own Client connection, per Task 4a's dedicated-connection requirement above). dropped_slow_consumer is yielded like any other event and then the stream simply ends (not raised/thrown) — an explicit, documented decision; a caller checks the last event's type if it cares. Stopping iteration early (Python break/.close(); TS break/.return()) closes the dedicated socket with no leaked fd, proved against a real daemon via lsof -p <pid> filtered to TYPE unix (a raw total-fd count is confounded by internal/dbfile's deliberately-never-closed checkout descriptors, so the tests isolate socket fds specifically) returning to its pre-subscribe baseline after a helper connection closes mid-stream. Real end-to-end lifecycle tests drive session_opened/flushed/session_closed on a separate connection and assert the helper observes them in order; the slow-consumer-drop path is hard to force deterministically from the SDK side (it needs the real bus's buffer-overflow timing, already covered server-side by TestEventBusDropsSlowSubscriberWithTerminalEvent), so both SDKs instead unit-test the helper's decode/contract path end to end against a small scripted fake daemon. Both SDK suites green under make test-sdks; both confirmed stdlib-only/zero-dep |
OpenSSF Scorecard workflow (.github/workflows/scorecard.yml) + README badge |
shipped | Runs on push to main, a weekly cron (Mondays 06:17 UTC), branch_protection_rule changes, and workflow_dispatch; ossf/scorecard-action with publish_results: true (the badge and the scorecard.dev viewer page) and its SARIF uploaded to code scanning via github/codeql-action/upload-sarif. shipped; first run on main pending (badge shows no score until then). "shipped" not "shipped-and-tested" per this page's legend: no automated test enforces the workflow stays wired going forward; v0.2.11 |
Scorecard remediation: hash-pinned CI installs, digest-pinned base images, CodeQL SAST (.github/workflows/codeql.yml) |
shipped | 2026-10-01. Every third-party Python package CI installs comes from a hash-pinned lock (requirements/*.txt, generated from the .in beside it — see CONTRIBUTING.md's "Refreshing the hash-pinned CI requirements"), with the repo's own SDKs installed --no-deps -e; publish.yml uses npm ci; the Dockerfile's base images are image:tag@sha256: pinned and bumped by Dependabot's docker ecosystem; SECURITY.md carries absolute reporting/verification URLs; CodeQL analyses Go on every PR, push to main and weekly. Local scorecard --local: Pinned-Dependencies 6 → 9, Security-Policy 4 → 10. Residual (1, kept on purpose): publish.yml's packed-tarball import test, npm install --ignore-scripts ../dry-run-pack/offshoot-db-client-*.tgz, is flagged as an unpinned npmCommand, but it installs the local tarball built two steps earlier, not a registry download. SAST rises as merged PRs accumulate CodeQL check-runs. Fuzzing is tracked separately; CII-Best-Practices, Code-Review and Maintained are in the standing nag |
CodeQL code scanning: name-derived paths contained under Workspace.Root; client-named paths dismissed as by design |
shipped-and-tested | First run 2026-10-01 (PR #62): 14 high-severity go/path-injection alerts (15-28), every one traced from a daemon request field (the HTTP /rpc body). 7 fixed in code (15, 16, 24-28: os.Stat/os.Open/os.ReadFile/os.WriteFile on a checkout, its .sum sidecar, and dbfile's reads of it, reached from db/branch names): ops.Workspace.underRoot is the one containment check every name-derived local path goes through (CheckoutPath, CheckoutAtPath, byChainPath; sidecar and shadow paths are suffixes of those). It joins under Root, filepath.Cleans and requires the Root+separator prefix (filepath.IsLocal for a . root). Every input already passed store.ValidateName, so the check cannot fail. On a validation bypass it panics, naming the invariant. internal/ops/underroot_test.go's TestUnderRootAcceptsEveryValidNameShape (outputs identical to the old filepath.Join across absolute, relative, ., "" and / roots) and TestUnderRootRejectsEscapingInput. 7 dismissed "won't fix" (17-23: export's destination at ops/export.go and through ltxio.MaterializeChain's temp file, rename and -wal/-shm removal; create's import source at copyFile): client-named file paths by design. The CLI path is the invoking user's own, and the daemon's Request.Path is a trusted absolute server-side path (internal/daemon/protocol.go; SECURITY.md). The HTTP source CodeQL traced cannot reach either sink: export is in httpForbiddenOps, and create with path is refused before dispatch. No query was disabled and no path filtered. Open CodeQL alerts on the PR after the re-run: 0 |
| Signed and attested releases (keyless cosign signatures on every tarball, SLSA build provenance, an SPDX SBOM with its own attestation, a cosign-signed and attested GHCR image) | shipped-and-tested | release.yml's collect job signs every tarball with keyless cosign (<asset>.sigstore.json), emits an SPDX SBOM (offshoot_<tag>.spdx.json) with a matching attestation, and produces SLSA build provenance (the GitHub attestation store plus an offshoot_<tag>_provenance.intoto.jsonl release asset); the docker job cosign-signs the GHCR image digest and pushes its own provenance attestation. Verification recipe: docs/installation.md's "Verify what you downloaded", linked from SECURITY.md and docs/testing.md. Validated end to end by a workflow_dispatch dev build (run 36251100872, tag dev-0935a75, since deleted): gh attestation verify confirmed both the SLSA v1 provenance and the SPDX 2.3 SBOM predicates, cosign verify-blob reported Verified OK against the release workflow's certificate identity, the plain sha256 checked out, and both cosign verify and gh attestation verify oci:// passed against the image. Releases before v0.2.11 carry checksums only; v0.2.11 |
Resource behavior
| Guarantee / feature | Status | Notes |
|---|---|---|
| Checkout-cache disk budget + LRU eviction | shipped-and-tested | Milestone 4 Task 5 — offshoot serve -ro-cache-budget <bytes|0> (default 0 = unlimited) bounds checkouts-ro (the read-only CheckoutAt cache, Milestone 3 Task 2), never checkouts/ (writable, leased — see below). Server.janitorTick (internal/daemon/server.go) runs a new ro-cache pass on the same -reap-every cadence as reap/GC: always computes and republishes current usage into offshoot_ro_cache_bytes (the Task-2-registered gauge, previously always 0 — now updated once per janitor pass, not continuously; see docs/reference.md's -ro-cache-budget section for the resulting between-passes staleness), and once usage exceeds the budget, LRU-evicts (ops.Workspace.EvictROCache, internal/ops/rocache.go) oldest-first until back under it. LRU clock: a .last-used touch-on-HIT marker file, not the .db file's own mtime — CheckoutAt's force=false cache-hit path now calls touchLastUsed, Chtimes-ing a <cachefile>.last-used sidecar to now on every hit; necessary because materializeAt's rename-into-place is the LAST thing that ever touches the .db file's mtime (a hit is a pure read, never touching it again), so without a separate marker "least recently used" would collapse to "least recently created" — exactly backwards for a cache whose point is that a repeatedly-hit checkpoint stays hot. lruClock falls back to the .db file's own mtime as the floor for an entry materialized but never since hit. TestEvictROCacheLastUsedTouchBeatsCreationOrder (internal/ops) and TestJanitorTickEvictsLRUUnderBudgetAndFiresEvictedEvent (internal/daemon) both prove an OLDER-by-creation entry that gets hit survives while a NEWER-by-creation, never-hit entry is evicted instead. checkouts/ is never evicted by construction, not a runtime check: EvictROCache only ever walks/removes paths under the separate checkouts-ro tree, never joined with or reachable from checkouts/'s own path shape — TestEvictROCacheNeverTouchesWritableCheckout and TestJanitorROCacheEvictionNeverTouchesLeasedWritableCheckout (a real open, leased session survives an aggressive budget=1 pass and can still flush afterward) confirm this end to end. Eviction is LOUD: one offshoot: janitor: ro-cache: evicted <db>@<branch>@<checkpoint> (<bytes> bytes) stderr line per eviction, offshoot_ro_cache_evictions_total incremented (previously always 0), and Task 4a's reserved evicted event type gets its emitter here ({type:"evicted", db, branch, detail:{checkpoint, bytes}}, published non-blockingly to the T4a bus at the eviction call site). offshoot status gains an ro-cache usage summary line (ops.Workspace.ROCacheUsage) plus a display-only -ro-cache-budget flag of its own (the budget itself is never persisted, matching every other serve tuning flag, so this at-rest command has no other way to show usage against it without a live daemon). Each eviction removes both the .db and its .last-used marker together; a mid-pass os.Remove failure follows Reap/GC's own partial-progress-plus-first-error convention. go test ./internal/ops ./internal/daemon -race clean; no internal/session/internal/capture changes, so no torture run was required |
| FD budget with idle-checkout eviction | deliberately deferred | Milestone 4's self-review named this a consciously narrowed bullet, not a silently dropped one: internal/dbfile's file descriptors are deliberately unclosable by that package's own design (see its doc comment) specifically to avoid a stray-close lock hazard, which makes "evict a cold session's FD" a real design problem, not just an unimplemented budget loop — it needs its own pass rather than reusing the ro-cache budget's shape. See docs/operations.md |
SnapshotEvery tuning exposed via the daemon |
shipped-and-tested | Milestone 4 Task 6a — offshoot serve -snapshot-every N (default 16, unchanged if omitted; rejects < 1) plumbs into session.Options.SnapshotEvery at opOpen the same way -flush-every plumbs FlushEvery (Server.SetSnapshotEvery, single-writer-before-Serve contract). Unlike -flush-every 0, there is no "disabled" sentinel — every flush must eventually snapshot, so SetSnapshotEvery's n <= 0 means "unset, session.Open applies its own default" rather than "off," and the CLI only calls it when the flag was actually given. internal/daemon/snapshot_every_test.go's TestServeSnapshotEveryWiringDrivesCadence opens a session over the socket with SetSnapshotEvery(4), drives 9 flushes, and asserts both a bounded store.Chain (≤4 members, mirroring ops_test.TestReplayStaysBoundedAcrossManyFlushes) and more than one snapshot object across the lineage listing — proof the cadence actually recurred under daemon wiring, not just the mandatory first-ever snapshot. cmd/offshoot/main_test.go's TestServeNonPositiveSnapshotEveryIsRejected pins 0/negative/non-integer values as usage errors. See README's What a flush costs for the cadence's bandwidth-vs-replay-latency trade-off |
| Current per-session disk/FD costs documented | shipped | Documented in README's Resource behavior: per-session FD footprint is small and fixed; disk is the sharper cost via internal/dbfile's never-closed descriptors (see that package's doc comment), now mostly avoided on a clean re-open by Checkout's clean-skip fast path, but not reclaimed for a checkout that does get re-materialized. "shipped" rather than "shipped-and-tested" per this page's own legend — no automated test enforces documentation accuracy |
Platform
| Guarantee / feature | Status | Notes |
|---|---|---|
| Linux + macOS | shipped-and-tested | |
| Copy-on-write fast paths tested on every push (clone checkouts, checkpoint shadows, segment checkpoints) | shipped-and-tested | The push/PR test job runs on ubuntu-latest, whose ext4 cannot clone, so every clone/shadow/segment fast-path test skips there and only the fallback is exercised; the full macOS go test runs weekly (nightly.yml). The cow-paths job in .github/workflows/ci.yml closes that gap: go test ./internal/ops ./internal/ltxio ./internal/reflink plus internal/session's two clone tests (TestOpenOnClonedCheckoutSkipsSettlingFlush, TestAtRestCheckpointAfterCloseWritesASegment) on macos-latest (APFS) on every push and PR, with a 20-minute timeout, after a probe step that fails the job if TestCloneProducesIdenticalFile skips — so a runner that stops cloning cannot turn the job green on skips. Scoped that way to bound the 10x macOS bill and to keep the load-sensitive session-flush heartbeat test off every push; Linux reflink (btrfs/XFS FICLONE) is exercised by no CI leg; v0.2.12 |
| Windows | not supported (non-goal) | Capture path and lock/SHM probing are POSIX-dependent; see ROADMAP non-goals |
| Multi-node orchestration / clustering | not supported (non-goal) | Shared-bucket safety is guaranteed by fencing; placement/failover/routing are explicitly out of scope — see ROADMAP non-goals |
| Merge (three-way) | not supported (non-goal) | Forks are pick-a-winner via promote; the escape hatch is sqldiff over two checkouts — see docs/faq.md |