Implementation status
offshoot's original design describes a v1 scope larger than what's shipped so far. This page is the honest accounting: what's tested in anger, what's built but unverified in some dimension, and what's still just a plan. If a guarantee you're relying on isn't marked shipped-and-tested below, verify it yourself before depending on it in production.
Status legend:
- shipped-and-tested — implemented, and exercised by an automated test (unit, integration, property, or torture) that would fail if it broke.
- shipped — implemented and in the code path you'll actually hit, but without the same weight of testing (e.g. verified against one provider, not the class of providers it claims to support).
- not yet implemented — spec'd or planned, no code behind it yet. Each row links to the roadmap milestone tracking it.
- deliberately deferred — considered for this milestone and explicitly declined, with a stated reason, rather than simply not yet started; see the row's Notes for why.
Standing nag: user-gated launch items
Everything on this page above and below is code and docs a controller/agent loop can finish on its own. These cannot be — they're one-time, out-of-band actions only a human with the right credentials/accounts can take, and no amount of further engineering substitutes for them. This section exists so the list doesn't quietly disappear into individual rows scattered across the doc set — every item below is genuinely ready to go, blocked only on the button-press:
| Item | Ready since | Blocked on |
|---|---|---|
Claim offshoot-db on PyPI and the @offshoot-db npm scope |
Milestone 3 Task 7 — manifests filled out, publish pipeline built and dry-run-verified | This page's own SDK publish-pipeline row: real sdist/wheel/npm-tarball builds pass twine check/npm pack today; only the actual name claim + Trusted Publishing/NPM_TOKEN configuration + flipping the PUBLISH_ENABLED repository variable are outstanding (see CONTRIBUTING.md's Release process) |
| Submit the MCP registry listing | Milestone 3 — server.json drafted in-repo |
the current registry schema will be validated at submission time |
| Submit the LangGraph community-integration PR | Milestone 3 — PR title/description/listing-table entry drafted | blocked on the PyPI publication above, since the listed install command needs to resolve |
Claim a domain, the Homebrew formula name, a docker image namespace |
Milestone 1's "Claim the names" / "Release engineering" bullets | Mostly done: the Homebrew formula ships in-repo (Formula/offshoot.rb, installable via brew tap), and release.yml publishes ghcr.io/sricola/offshoot images on every tagged release — docs/recipes/kubernetes.md's sidecar manifest tag is real now. Still open: a domain, and registry names (PyPI/npm) deliberately unclaimed for now |
| Trademark/collision check on "offshoot" in dev tooling | Milestone 1 | Not started this pass |
Not on this list on purpose: the launch demo assets (asciinema of
parallel attempts, the MCP-in-Claude-Code walkthrough) are explicitly not
user-gated — they were recorded
alongside Task 1 and merged to main already, no further action needed.
Also not on this list: the module-path/org mismatch that used to block go install — the repo lives at github.com/sricola/offshoot, not the
aspirational offshoot-db org, so go.mod and every import/doc/workflow
reference were retargeted to sricola instead of waiting on an org
transfer (go install github.com/sricola/offshoot/cmd/offshoot@latest
resolves today). PyPI/npm registry names are unaffected — those stay
offshoot-db/@offshoot-db and are tracked in the row above.
Core model
| Guarantee / feature | Status | Notes |
|---|---|---|
| Branch = named ref (lineage, epoch, head txid, checkpoints, TTL, protected flag) | shipped-and-tested | internal/ops/ops.go, internal/store |
| One writer per lineage, enforced by lease + epoch fencing | shipped-and-tested | internal/ops/fencing_test.go; every object write lands under the epoch current at write time |
| CAS on every ref mutation | shipped-and-tested | Local backend: O_CREAT|O_EXCL lock file. S3-compatible: conditional PutIf, gated by an attach-time capability probe |
| Copy-on-write fork (shared by default via a durable base pointer; materialized fallback at the fork-time snapshot floor) | shipped-and-tested | v0.2.0 (internal/ops/fork_cow_test.go, cow_divergence_test.go): the common-case fork writes data/{lineage}/base.json and zero data objects; store.Chain resolves base pointers transitively under a strict never-merge-across-lineages rule; two automatic snapshot floors (fork-time ops.ForkShareMaxDepth/-snapshot-every, plus the divergence floor) keep reads bounded; status/branches report storage=shared|materialized per branch. The materialize fallback (and promote/rollback/compact) keeps the CopyObject fast path for single-snapshot chains (local: reflink; S3: server-side copy — single-request ≤5 GiB, multipart UploadPartCopy up to 5 TiB since v0.2.4), falling back to materialize-and-re-encode otherwise — see Fork performance below |
offshoot compact (make a shared fork self-contained; ancestor storage becomes reclaimable) |
shipped-and-tested | v0.2.0 (internal/ops/compact_test.go, compact_cow_test.go); no-op on an already self-contained branch; resets the checkpoint map to {"compact": head} (documented tradeoff); daemon op + SDK compact() both languages; the daemon refuses while the branch has an open session (v0.2.1) |
| Checkpoint (named state within a branch, not inherited by children) | shipped-and-tested | |
| Rollback (repoint at new lineage seeded from a checkpoint) | shipped-and-tested | Kept checkpoints' snapshots are copied forward into the new lineage |
| Promote (repoint target at source's head via fork machinery) | shipped-and-tested | Protected targets require --force |
| Object-granular reachability GC (mark every live branch's resolved chain at head + every checkpoint, transitively through base pointers; two-phase tombstone → grace → re-mark → sweep per object) | shipped-and-tested | Rewritten for copy-on-write in v0.2.0 (internal/ops/gc_test.go, gc_chain_test.go, gc_batch_test.go); counts objects, not lineages; sweep batch-deletes via store.BatchDeleter on S3 (v0.2.2); the never-delete-above-a-live-head compensating rule is epoch-aware since v0.2.4 (a provably fenced orphan is swept once its tombstone clears grace); GC fails closed, with offshoot_gc_errors_total as the alertable signal (v0.2.1) |
| TTL reaping (from the later of last-touch and lease expiry) | shipped-and-tested | internal/ops/reap_test.go, reap_cas_test.go |
Layout versioning (offshoot.json, refuse-newer-than-known) |
shipped-and-tested |
Storage backends
| Guarantee / feature | Status | Notes |
|---|---|---|
| Local directory backend | shipped-and-tested | Quickstart's default; no bucket required |
| MinIO | shipped-and-tested | minio/minio:latest, make test-s3 passes the conformance suite for real (see README's provider table). Since v0.2.3 the env-gated TestS3RealProvider suite includes a real multipart-upload subtest, and it passes against MinIO — a real provider's CompleteMultipartUpload precondition/checksum handling is the one thing the in-process fake can never substitute for |
| AWS S3 | shipped-and-tested | TestS3RealProvider (probe + full conformance + the multipart subtest) passes against a real AWS S3 bucket (us-east-1, 2026-08-13) — including the CompleteMultipartUpload precondition/checksum semantics the in-process fake cannot exercise |
| Google Cloud Storage | not supported | GCS's S3-interop API has no conditional writes; the attach probe refuses it outright rather than degrading |
Daemon and durability
| Guarantee / feature | Status | Notes |
|---|---|---|
| Live WAL capture (continuous, foreign-connection-safe) | shipped-and-tested | internal/capture/torture_test.go — kill -9 mid-write, repeated, verifies checksum + state equivalence after every bounce |
| Incremental LTX segments (flush ships only changed pages) | shipped-and-tested | |
| Bounded replay (snapshot every 16th flush; a read applies ≤1 snapshot + 15 segments) | shipped-and-tested | Cadence is Options.SnapshotEvery in the embeddable session library; offshoot serve -snapshot-every N exposes it to the daemon (Milestone 4 Task 6a) — see the Resource-behavior section below. The bound survives copy-on-write sharing via the two snapshot floors (v0.2.0): a shared child's counter seeds from its durable divergence, and a fork already at the depth bound materializes instead of sharing; the daemon's fork floor tracks its configured -snapshot-every (v0.2.1), while an at-rest CLI fork always uses the library default of 16 (documented in v0.2.4) |
Explicit flush durability (session status reports durable-through txid) |
shipped-and-tested | Durability advances only on flush/checkpoint; nothing ships to the store automatically between them |
Background flush interval (serve -flush-every, on by default) |
shipped-and-tested | internal/session/flush_test.go (TestAutoFlushShipsWritesWithoutManualFlush, TestIdleAutoFlushWritesNothing, TestAutoFlushFailureSurfacesAndRecovers), internal/daemon/lifecycle_test.go (TestServeFlushesAutomaticallyWithoutAnExplicitFlushOp). Default 30s, 0 disables; bounds data loss on daemon crash to at most one -flush-every interval — see README's What a flush costs. One daemon-wide cadence applied to every session it opens — see "Per-session FlushEvery override" below for what this is not |
| Settling-flush checksum-compare suppression (skip the mandatory first-open full snapshot when content is provably unchanged since the last one) | shipped-and-tested | rebaseline's very first call (internal/session/session.go) skips the settling flush only when ALL of: the checkout Open received was already proven byte-identical to the branch's head AT OPEN TIME (ops.CheckoutProven's .sum sidecar clean-and-current fast path, cleanAtOpen) AND the checksum RECORDED IN THAT SAME SIDECAR (ops.CheckoutResult.PostApplyChecksum — the LTX postApplyChecksum the checkout embodied when the sidecar was last stamped, by Checkout/Checkpoint/Rollback/Promote, or by a session's own clean Close, see the row below) exactly equals what rebaseline freshly computes from the replica once the engine's REAL startup rebase actually runs. The second condition is load-bearing, not redundant: Open can return to its caller before that real rebase finishes (see Session.rebaseline's doc comment), so a write landing in that window is folded in by the rebase's own checkpoint without ever passing through Apply; the checksum comparison is what catches exactly that case and forces a real settle instead of silently losing the write. Reading the checksum out of the LOCAL sidecar — rather than fetching it fresh from the store, which an earlier, since-reverted version of this fix did — costs Open NO store read at all beyond the two tiny ref-metadata Gets it already needed (TestReadOnlySessionWithCleanCheckoutMakesNoStoreWrites asserts exactly 2, never a Get on the head object itself, which the earlier version paid unconditionally — a full download whenever the head was a snapshot, permanently for a read-only branch that never flushes). Trust-boundary note: this reads the sidecar's recorded checksum as trusted input the same way every other same-host decision in this codebase already does (the checkout directory, .sum sidecar included, is fully trusted under the same-host/same-user unix-socket threat model — see docs/reference.md's daemon-ops threat-model note) — but it widens the CONSEQUENCE of a forged or corrupted local .sum sidecar, not the trust boundary itself: before this suppression existed, a bad sidecar could only make a checkout serve stale local bytes (a read-side effect, confined to this host); now, since a suppressed settle skips the real re-verification snapshot and lets subsequent ordinary flushes encode deltas straight off whatever the replica already held, a forged sidecar can seed what becomes the branch's durable baseline in the store, not just what's served locally. A sidecar that never recorded a checksum (older format) reads back as absent (0, fail-toward-settling) rather than being trusted — TestSettleStillHappensWithOldFormatSidecarLackingChecksum. A first-ever open, or one against a dirty/stale checkout, still settles exactly as before. internal/session/flush_test.go: TestReadOnlySessionWithCleanCheckoutMakesNoStoreWrites, TestSettleStillHappensWhenCheckoutWasStaleAtOpen, TestSettleStillHappensWithOldFormatSidecarLackingChecksum, TestSettlingSuppressionCatchesRaceWindowFold (white-box pins the race-window case directly; fails against the pre-fix cleanAtOpen-only gate), TestCleanAtOpenSessionsFirstRealFlushIsASegment (the suppressed session's first REAL flush, once one happens, is a segment continuing directly from the pre-session head — a previously-unreachable shape); M2's TestFlushLoopFlushesRebaseFoldedContentWhenOtherwiseIdle and TestFlushLoopRetriesAfterRebaseDuringUpload stay green unmodified — a mid-session rebase-on-divergence is untouched by this change. BenchmarkSessionOpen (internal/ops/fork_bench_test.go), re-run: 64MB db ~30.0ms/op, 512MB db ~199.5ms/op, B/op/allocs/op flat across that 8x size range — see docs/benchmarks.md |
Sidecar refresh on clean Close (rewrite the checkout's .sum sidecar to the branch's current head txid — and its LTX checksum, see the row above — when a session closes cleanly, not just on Checkout/Checkpoint/Rollback/Promote) |
shipped-and-tested | Session.Close now re-stamps the sidecar (ops.StampSum, including PostApplyChecksum = s.flushChecksum — the row above is what actually reads it back on the next Open) when the close is provably clean: no session error, nothing left unflushed (autoFlushPending()), at least one flush actually succeeded, the branch head hasn't moved past what this session flushed, the replica was never rebuilt by anything beyond its own mandatory startup rebase (Session.singleStartupRebase), AND — the stamped hash itself — the capture engine's OWN post-shutdown fingerprint (capture.State.Clean/MainHash, persisted only when the engine's shutdown fully verified drain+checkpoint(RESTART)+no-WAL-race+checkpoint(TRUNCATE)) is reused directly rather than independently re-derived; any of shutdown's four early-return paths (a foreign write racing the final checkpoint, most directly) leaves Clean=false and the sidecar is left unstamped. Split around the capture engine's own shutdown join: the pending decision needs the engine alive (a final DrainNow), the physical stamp needs the engine's shutdown to have already run (successfully or not — its verdict is exactly what gates this). internal/session/flush_test.go: TestCloseRefreshesSidecarSoReopenCleanSkips, TestCloseAfterFailedFlushDoesNotStampSidecar, TestSidecarNotStampedAfterMidSessionRebase, TestCloseDoesNotStampSidecarAfterUnverifiedShutdown (forces a real shutdown-verification failure via capture.ShutdownRaceHook; fails against the pre-fix version that independently re-quiesced+re-hashed instead of consulting capture.State); internal/capture/engine_test.go: TestEngineShutdownLeavesUnverifiedStateAfterRacedRestart |
| Clean-and-current checkout served without chain validation (an intentional, accepted tradeoff of the clean-skip fast path above, not a defect) | shipped-and-tested (documented tradeoff) | Once a checkout's .sum sidecar is clean-and-current — whether stamped by Checkout/Checkpoint/Rollback/Promote (Milestone 2 Task 1) or, as of the row above, by a session's own clean Close — the NEXT Checkout call for it is served straight from disk without ever touching the object store's chain, including when that chain has since been corrupted or had a member deleted out from under it. This widens (in reach, not in kind) with the sidecar-refresh row above: it now also applies after an ordinary session's clean close, not only after the four ops entry points. internal/session/flush_test.go's TestCleanCheckoutServedWithoutChainValidationAcrossClose pins this directly (positive test: a corrupted chain does NOT fail a clean-skip Checkout); TestMissingSegmentIsLoud (internal/session/segment_stress_test.go) was updated to remove the checkout+sidecar before asserting a missing chain member fails loudly, since it otherwise now legitimately clean-skips past exactly the corruption it means to detect — controller sign-off granted on this modification |
Per-session FlushEvery override (a distinct cadence per session open call, over the wire) |
deliberately deferred (YAGNI) | -flush-every is one setting for the whole daemon (internal/daemon/server.go's SetFlushEvery), applied to every session it opens; the daemon protocol's open op has no field for a caller to request a different cadence for just one session. Not scoped in Milestone 2 or planned elsewhere yet — revisit if a real caller needs mixed cadences on one daemon |
| Connection contract enforcement (live WAL-mode probe, rollback-journal detection) | not implemented — documented assumption | No live polling or journal watcher exists; the contract's WAL-only and shared-kernel clauses are trusted, and a mid-session violation is caught only by the restart-time divergence check (WAL-emptiness + main-file hash continuity in internal/capture's resume gate), which marks the checkout dirty rather than pretending continuity. Live enforcement is future work |
Reflink/clonefile fork, server-side S3 copy fork (design spec's ~40ms figure was the target this work aimed at, not a number reproduced below — see notes). Since v0.2.0 this fast path serves the materialize cases only — the floor-tripped fork and promote/rollback/compact — because the default fork shares instead of copying |
shipped-and-tested (single-snapshot-chain window) | Tasks 6a+6b of Milestone 2 — ops.Workspace.Fork's copySnapshotToNewLineage copies the source snapshot object directly (local: internal/ops/reflink, clonefile(2) on darwin / FICLONE on Linux, silent plain-copy fallback otherwise; S3: store.S3.CopyObject, a real server-side CopyObject API call, no download or re-upload through this process) instead of materializing and re-encoding, when the source checkpoint's chain resolves to exactly one snapshot. Falls back to the pre-6a path once a daemon session has flushed segments past the last snapshot. S3's former ≤5GB CopyObject gate is gone: since v0.2.4, sources over 5 GiB take a multipart server-side copy (UploadPartCopy) up to S3's 5 TiB per-object ceiling, and store.ErrCopyUnsupported fires only beyond that. Measured (MinIO-local, not an AWS claim) 512MB fork ~198ms on APFS (was 2.87s pre-6a) and ~1.03s over S3 (was 4.57s pre-6b); ~9.3ms for the local copy alone in isolation from an unrelated pre-existing O(size) check, the closest this suite gets to the design spec's ~40ms figure directly — see docs/benchmarks.md |
Async fork-point upload (pending marker + fork pin on GC) |
deliberately deferred — and largely superseded by copy-on-write | Scoped in Milestone 2's original fork-performance bullet but not built: making the upload async introduces a new pending chain state that readers, GC, and fencing would all have to understand mid-upload. Since v0.2.0 the common-case fork shares and uploads nothing at all — there is no fork-point upload to make async and no pending window to caveat (noted in the v0.2.1 changelog). Only the materialize fallback (fork-floor trip) still pays a synchronous snapshot copy, made cheap by the reflink/CopyObject fast path above |
TTL, GC, and janitor
| Guarantee / feature | Status | Notes |
|---|---|---|
offshoot serve -reap-every / -gc-grace (janitor loop) |
shipped-and-tested | -reap-every 0 disables the janitor; offshoot gc remains available on demand |
| Protected branches never reaped | shipped-and-tested | |
| Live lease always defers reaping | shipped-and-tested | |
CAS-conditional ref delete (close the Destroy GetRef→lease-check→delete TOCTOU) |
shipped-and-tested | Milestone 4 Task 6b — ops.Workspace.Destroy (internal/ops/gc.go) CAS-writes a Deleting claim on the ref (store.Ref.Deleting/DeletingAt) before it does anything irreversible; store.AcquireLease refuses outright once it sees the claim (store.ErrDeleting), closing the window in which a lease acquired between Destroy's initial GetRef and its actual delete could have its branch deleted out from under it — the exact race an earlier design review documented for the M2 Destroy path. Scoped per the task's own timebox as a sibling claim field, not a unification with Reap's existing Reaping claim: Reaping's CAS mechanics are torture/race-tested and deliberately untouched here (internal/ops/reap.go/reap_test.go/reap_cas_test.go all pass unmodified). --force bypasses the protected/live-lease pre-checks only, never the claim itself — a lease that wins the underlying CAS race still survives a concurrent forced destroy. Backend split: Local gets a TRUE conditional delete (Local.DeleteIf, the same per-key lock file PutIf already uses) on top of the claim, exposed via Store.DeleteRefIf and the new store.ConditionalDeleter optional-capability interface; S3's DeleteObject has no compare-and-delete precondition at all, so DeleteRefIf falls back to an unconditional delete there and the CAS-written claim marker is the entire safety mechanism on that backend, documented as such rather than faked. A crashed Destroy (claimed, never deleted) self-heals via ops.Workspace.ClearStaleDeleteClaims (age-based, 30s — no TTL/deadline concept exists for a delete claim the way Reap's ReapDeadline gives it one), wired into the daemon janitor's janitorTick and the CLI offshoot gc on the same "report and press on" convention as a reap failure. Tests: internal/ops/destroy_claim_test.go's TestConcurrentDestroyAndAcquireLeaseHaveExactlyOneWinner (20-iteration race loop, mirroring TestConcurrentTouchAndReapHaveExactlyOneWinner: exactly one of Destroy/AcquireLease wins, the branch is never left half-deleted with a live lease on it), TestForceDestroyStillClaimGuards (same race with force=true), TestDestroySelfHealsStaleDeletingClaim (stale claim cleared, branch untouched, AcquireLease succeeds again; a fresh claim survives a pass); internal/store/local_test.go's TestLocalDeleteIfConditionalDelete/TestStoreDeleteRefIfUsesConditionalDeleteOnLocal (true CAS delete exercised directly); internal/daemon/destroy_claim_test.go's TestJanitorTickClearsStaleDeleteClaim (through a real janitor pass, not the ops call in isolation). go test ./internal/ops ./internal/store ./internal/daemon ./cmd/offshoot -count=1 -race clean; no internal/session/internal/capture changes, so no torture run was required |
Integration surface
| Guarantee / feature | Status | Notes |
|---|---|---|
| CLI (create/checkout/checkpoint/fork/rollback/promote/compact/destroy/touch/export/diff/gc/status/path/lease/serve/session/mcp) | shipped-and-tested | See docs/reference.md for every command |
| MCP server, 7 tools, at rest | shipped-and-tested | offshoot mcp; protected-branch rules enforced same as CLI |
| MCP forks carry a TTL | shipped-and-tested | offshoot_fork takes ttl (explicit always wins) and falls back to offshoot mcp -default-ttl (default 24h; 0/none disables) when omitted; ttl:"none" overrides even a configured default. The response echoes the applied TTL and computed expiry. Reaping still requires a running janitor (offshoot serve) — a daemonless MCP setup sweeps expired branches only on offshoot gc |
| MCP rides the daemon (live capture, session-aware checkpoint) | shipped-and-tested | internal/mcp/daemon_test.go; offshoot_checkpoint/offshoot_fork/offshoot_checkout each probe the daemon per call and take its live path when a session is already open (and healthy) on the branch (opened by a harness — no MCP tool opens one itself, i.e. the good path requires a harness-opened session); a fenced session falls back to at-rest with a warning naming its error; no session, or no daemon, and the tool runs exactly at rest as before. offshoot_rollback/offshoot_promote (target)/offshoot_destroy refuse — even with force, which has no effect on this refusal — rather than proceed at rest when the daemon has any session (healthy or fenced) on the affected branch; offshoot_promote's source is not guarded the same way (TestPromoteFromOpenSourceProceedsAtRest) — an open session there doesn't block the promote, but the promoted state is the source's last-flushed head, not any unflushed write in that session. offshoot mcp -socket PATH names the daemon to ride |
MCP registry submission (server.json) |
drafted, not yet submitted | server.json (repo root) is a draft manifest; the registry's current schema will be validated against at submission time |
| LangGraph community-integration listing | drafted, not submitted | a community-integration PR for offshoot.langgraph.ThreadForks is drafted; blocked on PyPI publication (the install command it quotes needs to work) — see the SDK publish-pipeline row above |
| Python SDK (stdlib-only, over the unix-socket lifecycle API) | shipped-and-tested | sdk/python; publish pipeline prepared, actual PyPI publication user-gated (see the publish-pipeline row below and Milestone 3) |
| TypeScript SDK (zero runtime deps) | shipped-and-tested | sdk/typescript; publish pipeline prepared, actual npm publication user-gated (see the publish-pipeline row below and Milestone 3) |
| SDK typed surfaces (typing polish, Milestone 4 Task 7) | shipped-and-tested | TypeScript: _call's internal wire responses are typed (RawResponse/RawBranchInfo/RawCheckpointInfo/RawSessionInfo/RawAck/RawEvent mirroring internal/daemon/protocol.go's Response), not any; _call itself is @internal + stripInternal-stripped from the published .d.ts (still a real, fully-typed method at runtime and for this package's own tests, which compile against source).Verified additive: a dist/client.d.ts before/after diff shows exactly one removal (the now-internal _call line) and no other symbol changed; dist/testkit.d.ts byte-identical.Python: offshoot/py.typed (PEP 561) ships in the wheel ([tool.setuptools.package-data], confirmed via the dry-run wheel's contents); mypy --strict offshoot is clean across all four modules, fixed with annotations only (no behavior changes) — typing.cast() on JSON-derived returns, Mapping[str, str] narrowing, dict/Popen generic args, a module-private _TTL/_Seed type alias each. make test-sdks (Python 41/41, TS 49/49) and make test-pytest-plugin (58/58) green; tsc --noEmit clean; no Go changes |
SDK publish pipeline (.github/workflows/publish.yml: PyPI Trusted Publishing/OIDC, npm --provenance) |
shipped | Triggered by sdk-v* tags or workflow_dispatch; gated on the PUBLISH_ENABLED repository variable, default off — both jobs build the real sdist/wheel/npm-tarball, run twine check, and install-test them regardless of the gate, only skipping the upload step when it's off. That dry-run tier (make dry-run-sdks) runs on every PR via ci.yml's sdks job (verified locally too: python3 -m build + twine check + venv install/import for the wheel, npm pack + tarball install + import() test for the npm package, all green — see the Task 7 report). The upload path itself (pypa/gh-action-pypi-publish, npm publish --provenance) cannot be exercised without live PyPI/npm credentials that don't exist yet — actual publication is user-gated: claiming the offshoot-db PyPI name and @offshoot-db npm scope, configuring Trusted Publishing/NPM_TOKEN, and flipping PUBLISH_ENABLED are all manual, one-time, out-of-band actions (see CONTRIBUTING.md's Release process, ROADMAP.md). Manifests filled out for real publication (sdk/python/pyproject.toml: readme, urls, classifiers, authors, SPDX license; sdk/typescript/package.json: repository/bugs/homepage, files whitelist, prepublishOnly build). SDK version is a single source of truth (sdk/VERSION) checked against both manifests by scripts/check_sdk_versions.py |
LangGraph ThreadForks companion |
shipped-and-tested | examples/langgraph-rewind/ runs it end to end |
List databases (daemon dbs op, CLI offshoot session dbs, SDK dbs()) |
shipped-and-tested | Milestone 3 Task 1 — store.Store.ListRefs's keys, sorted; internal/daemon/metadata_test.go (TestOpDbsListsSortedDatabases, TestOpDbsOnEmptyStoreIsEmptyNotError), sdk/python/tests/test_client.py::test_dbs_lists_every_database_sorted, sdk/typescript/test/client.test.ts ("dbs lists every database sorted") |
Branch/checkpoint metadata and timestamps (Ref.Meta, per-checkpoint CreatedAt/Meta, fork/checkpoint/flush meta params, BranchInfo.touched_at/checkpoints_v2) |
shipped-and-tested | Milestone 3 Task 1 — capped at the ops layer (ops.ValidateMeta: ≤32 keys, keys ≤64 bytes, values ≤512 bytes; internal/ops/ops_test.go's TestValidateMetaCaps and the *RejectsMetaOverCap tests). Fork's meta sets the child branch's Ref.Meta (branch-level lineage — no row-level provenance); Checkpoint's meta sets that one checkpoint's own Meta. Every checkpoint-creating call site (Create, Checkpoint, Fork, Promote, and a daemon session's named flush) stamps CreatedAt (RFC3339 UTC) — internal/ops/ops_test.go's Test*StampsCreatedAt* tests, internal/session/flush_test.go's TestNamedFlushStampsCheckpointMetaAndCreatedAt. No store schema bump: new fields are omitempty, round-trip-and-old-shape-decodes-clean tested exactly like the TTL fields (internal/store/store_test.go's TestRefMetaAndCheckpointFieldsRoundTripAndOldRefsDecode, including a hand-written-JSON old-shaped-ref decode, not just a Go zero-value round trip). Wire compat: BranchInfo's pre-existing checkpoints (names-only) field is untouched; checkpoints_v2 ({name, txid, created_at}) rides alongside it; an old-client-shaped request (no meta field at all, constructed as literal JSON bytes, not a Go zero value) still works against every touched op — internal/daemon/metadata_test.go's TestWireCompatOldShaped*RequestWorks. CLI: fork/checkpoint gain repeatable --meta k=v; SDK parity both languages. Bug fixed in the same pass: Rollback's kept-checkpoint epoch relocation used to silently drop CreatedAt/Meta — internal/ops/ops_test.go's TestRollbackPreservesCheckpointCreatedAtAndMeta |
MCP tool metadata exposure (offshoot_fork/offshoot_checkpoint taking a caller-supplied meta map) |
deliberately deferred | Explicitly out of scope for Milestone 3 Task 1's dispatch: ops.Workspace.Fork/Checkpoint support meta, and both MCP tool call sites (internal/mcp/tools.go) pass nil — no offshoot_* tool argument threads a caller-supplied map through yet. Revisit if an agent workflow needs to set eval-run/git-SHA metadata from inside an MCP tool call rather than the CLI/SDK |
create --from reach: daemon protocol op, SDK create(..., from_=...)/from, MCP tool argument |
deliberately deferred | Decided at Task 8 dispatch time, not discovered as a gap late: create --from stays a CLI-only import path for Milestone 3. Reaching the daemon protocol means accepting a source SQLite file from a caller that isn't the daemon's own host process — that needs either an upload channel (stream the file's bytes over the unix socket, a new op shape none of Tasks 1/2 built) or a same-host path-trust story like export's (the daemon reads a path the caller names, trusted because same-host/same-user unix-socket access already implies that trust — see docs/reference.md's daemon-ops threat-model note). Either design has real edges (export's is "write here," which is a strictly smaller trust surface than "read this arbitrary path and copy its bytes into the store," and an upload channel is a new wire primitive this milestone never needed elsewhere) that deserve their own pass rather than a bolt-on. CLI offshoot create <db> --from file (local-filesystem-only, no daemon involved) remains the only import path; SDK/MCP callers wanting a daemon-managed import wait for that design |
MCP session open/close (an offshoot_open/offshoot_close tool pair) |
deliberately deferred | Milestone 2's "MCP rides the daemon" bullet named this gap and pointed at Milestone 3's fixture plugin as the eventual lifecycle owner; that fixture (offshoot.pytest_plugin, the row above) and its TypeScript counterpart (testkit, the row below) both now EXIST and ship in this milestone — but neither is an MCP tool, and no MCP open/close tool pair was built alongside them. The reasoning stands exactly as amended: an MCP-opened session has no natural owner responsible for closing it — a bare tool call has no equivalent of a fixture's guaranteed teardown or a with block's guaranteed exit, so an agent that opens and then gets interrupted, crashes, or simply forgets to call the matching close tool leaks a lease and a branch, precisely the class of leak this whole milestone's TTL/background-flush work exists to prevent. The good path for MCP daemon-mode capture remains what docs/reference.md's offshoot mcp entry and the README's MCP section already document: a harness (the SDKs, offshoot session open, or a custom loop) opens the session and stays responsible for closing it; MCP's checkpoint/fork/checkout tools ride that session without ever opening or closing one themselves. See docs/recipes/claude-agent-sdk.md for the concrete "open before, close after" wiring this implies for an agent harness |
Export (ops.Workspace.Export, CLI offshoot export, daemon export op, SDK export() both languages) |
shipped-and-tested | Milestone 3 Task 2 — materializes any checkpoint (or head) to a plain SQLite file anywhere, with zero ongoing relationship to the store: no .sum sidecar, no lease. Refuses to overwrite an existing destination unless force; the write is atomic (temp file in the destination's OWN directory, renamed into place only once every chain member is fetched and verified — reuses materializeChainAt/ltxio.MaterializeChain's existing atomicity, so a failed export never leaves a partial file, pinned by internal/ops/export_ops_test.go's TestExportAtomicOnMidWriteFailureLeavesNoTempOrPartialFile, which corrupts a segment's checksum mid-chain and asserts the destination directory ends up completely empty). Reads the branch's last DURABLE state from the store, never a live session's checkout — an open daemon session's unflushed writes are NOT in the export, asserted directly over the wire by internal/daemon/export_test.go's TestOpExportMissesUnflushedSessionWrites (write through a live session's checkout without flushing, export, assert the row count matches the pre-write durable state; then flush and export again, assert it now includes the write). Daemon export op requires an ABSOLUTE destination path (same-host/same-user unix-socket trust model — see docs/reference.md's daemon-ops section) and is deliberately NOT guarded by the open-session refusal checkout/rollback/promote use, since it never touches the live checkout. CLI parses the db[@branch[@checkpoint]] triple-@ target form (ops.ParseExportTarget) |
Read-only historical checkout (ops.Workspace.CheckoutAt, CLI offshoot checkout --at --read-only, daemon checkout-at op, SDK checkout_at()/checkoutAt()) |
shipped-and-tested | Milestone 3 Task 2 — materializes a NAMED checkpoint (no head alias) into a dedicated cache path, <store-root>/checkouts-ro/<db>/<branch>@<checkpoint>.db, chmod 0444, distinct from and never touching the writable checkouts/<db>/<branch>.db path or its .sum sidecar/lease — safe to call alongside a live daemon session on the SAME branch (internal/daemon/export_test.go's TestOpCheckoutAtSafeAlongsideOpenSessionOnSameBranch), unlike checkout/rollback/promote's refuseIfClaimed guard. Nothing in this codebase opens a checkouts-ro file through a live capture engine, so internal/dbfile's stray-close lock hazard doesn't apply to it either. A repeat call with force=false is a pure cache hit — no store access at all, not even a GetRef — because a checkpoint's content is immutable once created (internal/ops/export_ops_test.go's TestCheckoutAtWithoutForceIsACacheHitWithNoStoreAccess, which destroys the branch entirely between calls and proves the second call still succeeds); force=true re-reads the store and can fail once the checkpoint is gone (TestCheckoutAtForceRematerializesAndCanFail). Known, documented tradeoff: if a branch is destroyed and recreated with a same-named checkpoint, the cache path can serve stale content until force or a rm -rf of checkouts-ro — see README's Resource behavior |
offshoot.pytest_plugin fixture plugin (offshoot-db[pytest], sdk/python/offshoot/pytest_plugin.py, pytest11 entry point) |
shipped-and-tested | Milestone 3 Task 4 — the seed-once-fork-many paved road: offshoot_daemon (session-scoped: locates the binary via OFFSHOOT_BIN env or PATH, pytest.skips with install instructions when neither has one; temp store + socket; terminates at session end), offshoot_db (session-scoped NAMED-SEED FACTORY — offshoot_db(name="default", seed=None), memoized per name; creates eval-{name}, runs seed — a callable(path) or a SQL string — checkpoints it seed; seed=None falls back to the offshoot_seed ini option, a path to a .sql file, the zero-code default-seed case), offshoot_fork (function-scoped — offshoot_fork(seed_handle=None), forks a fresh worker-safe-named branch (t-{worker}-{testname-hash}-{n}, sanitized via offshoot.langgraph._sanitize) from the seed checkpoint with a TTL (default 1h, offshoot_ttl ini-overridable), opens a session, returns an object with .path/.client/.db/.branch; teardown closes the session then destroys the branch — a destroy failure is a UserWarning, never a test failure). offshoot_dump(path) -> str is the golden-file helper (sqlite3 .dump text) — deliberately: SQLite's on-disk bytes are not deterministic for identical logical content, so golden assertions compare offshoot_dump output, NEVER raw bytes (sdk/python/tests/test_pytest_plugin.py::test_offshoot_dump_is_the_right_comparison_not_bytes proves a vacuum changes the bytes but not the dump). Package extra offshoot-db[pytest] (pytest>=7); the pytest11 entry point is registered unconditionally in pyproject.toml but the module import-guards pytest, so the base offshoot-db install stays stdlib-only (verified: make test-sdks — the plain-unittest suites — passes with no pytest installed at all; make dry-run-python-sdk's built wheel import offshoot-tests clean in a pytest-free venv too).xdist stance (a locked design decision): one offshoot daemon + temp store per xdist worker (pytest has no cross-worker fixture-sharing mechanism), so seed cost is paid once per worker, not once total — measured on this repo's own smoke test (test_xdist_two_worker_run_passes_and_measures_seed_cost, a CREATE TABLE + 200-row seed, macOS arm64): ~80-90ms per worker, ~170ms of total seed work under -n2 against ~85-90ms of wall-clock time (workers seed concurrently). Full rationale and the numbers live in pytest_plugin.py's module docstring and sdk/python/README.md's pytest-fixture-plugin section.Found+fixed in passing: _run_seed's SQL-string path now wraps an unwrapped multi-statement seed in one transaction — sqlite3.Connection.executescript otherwise runs every statement as its own autocommit transaction, measured ~1.6s for 200 unwrapped INSERTs vs. ~17ms wrapped (~100x) — pinned by test_seed_factory_wraps_multi_statement_seed_in_one_transaction/test_seed_factory_wraps_unterminated_single_statement_seed/test_seed_factory_respects_seed_with_its_own_begin_commit.Testing: 50 direct-daemon logic tests (naming, TTL, teardown ordering close-then-destroy, factory memoization, skip-when-no-binary, destroy-failure-warns, seed-transaction-wrapping, dump-shaped/comment-prefixed seeds, dead-daemon-connection wrapping, seed-mismatch detection) plus 8 pytester-driven scenarios (entry-point autoload, fork-per-test isolation, the xdist 2-worker run that also measures the seed-cost numbers above, OFFSHOOT_BIN misconfigured fails loud, offshoot_require_binary ini fails loud, offshoot_require_binary ini defaults to a skip, offshoot_seed ini resolves relative to rootdir not cwd, teardown warnings don't escalate under -W error) — 58 tests total (re-counted directly for this row: grep -c '^def test_' sdk/python/tests/test_pytest_plugin.py and a live make test-pytest-plugin run both report 58/58 passing; the pytester tier grew from the plan's originally-scoped 2-3 scenarios to 8 during Task 4's review round) — make test-pytest-plugin, wired into ci.yml's sdks job (installs offshoot-db[pytest] + pytest-xdist AFTER make test-sdks already proved pytest-free, so the ordering itself is part of the proof) |
TypeScript testkit (sdk/typescript/src/testkit.ts, @offshoot-db/client/testkit) |
shipped-and-tested | Milestone 3 Task 5 — the vitest/jest counterpart of offshoot.pytest_plugin: framework-agnostic FUNCTIONS, not fixtures (no vitest/jest runtime dependency — this SDK stays zero-runtime-deps — and nothing registers itself automatically; the caller wires these into their own beforeAll/afterEach). startDaemon(opts?) -> Promise<DaemonHandle> locates the binary via OFFSHOOT_BIN env or PATH (a clear error naming both when neither has one — no skip tier, since this module has no test-framework integration to skip through, unlike the pytest plugin), starts it on a fresh temp store + socket, returns { sock, store, proc, stderrTail(), stop() }.seedOnce(daemon, {name?, seed}) -> Promise<SeedHandle> is a NAMED-SEED cache memoized per (daemon, name); seed is a SQL string, a path to a .sql file (detected: no newline, ends .sql, file exists — there's no ini file here to hold that decision separately the way pytest's offshoot_seed option does), or an async (dbPath) => void callback; a later call for the same name with a different seed (fingerprinted: SQL/path text by content hash, callables by identity) throws rather than silently keeping the first one, mirroring offshoot_db's mismatch semantics exactly. Ports _skip_leading_noise/_seed_opens_own_transaction verbatim, so a sqlite3 .dump's text (PRAGMA before BEGIN TRANSACTION) works as a seed unmodified and a plain multi-statement seed is wrapped in one transaction rather than paying one autocommit transaction per statement.forkPerTest(daemon, seedHandleOrName, opts?) -> Promise<ForkedSession> forks a fresh, worker-safe-named branch (t-{worker}-{sanitized-hint}-{n}; worker id from VITEST_POOL_ID/JEST_WORKER_ID when present, else "local" — sanitize is offshoot.langgraph._sanitize ported verbatim, since there's no TypeScript langgraph module to import it from) from the seed checkpoint with a TTL (default "1h", opts.ttl overrides), opens a session, returns { path, db, branch, client, flush(name?), close() }; close() closes the session and destroys the branch, and either failing is a console.warn, never a throw. Each ForkedSession owns its own connection and teardown independently of every other one, so — unlike the pytest plugin's shared-per-test _ForkFactory.teardown() loop — one fork's cleanup trouble can never block another's, by construction rather than an explicit try/except around each step. dump(path) -> Promise<string> is sqlite3 <path> .dump's text output, the same golden-comparison method as offshoot_dump: SQLite's on-disk bytes are not deterministic across writes with identical logical content, so golden assertions must compare dump output, never raw bytes (test/testkit.test.ts's "is the right comparison, not raw bytes" test proves a VACUUM changes the bytes but not the dump, mirroring the Python plugin's own such test). Since this SDK ships no SQLite driver, seeding and dumping both shell out to the sqlite3 CLI, same as test/client.test.ts already does.Exported as a new package.json exports subpath, @offshoot-db/client/testkit (the root export gained an explicit exports entry alongside it; main/types kept for back-compat with tools that don't read exports).Testing: sdk/typescript/test/testkit.test.ts, run via node:test against a real daemon — mirrors the Python plugin's direct-daemon tier (binary resolution naming both OFFSHOOT_BIN and PATH, naming, TTL default+override, seedOnce memoization + fingerprint mismatch, a .dump-shaped seed, a path-to-.sql seed, an async-callback seed, fork-per-test isolation, the string-name shorthand, teardown-warn-not-throw against a killed daemon, a golden-file-not-bytes scenario) plus one integration test wiring startDaemon/seedOnce/forkPerTest into node:test's own before/after/beforeEach/afterEach the way a real suite would — 22 tests in testkit.test.ts itself (re-counted directly: node --test test-dist/test/testkit.test.js in isolation reports tests 22/pass 22), wired into the existing make test-ts-sdk target alongside client.test.ts's own suite (no new Makefile target needed: it already globs test-dist/test/*.test.js; the CI-verified combined TS count lives in the typed-surfaces row above). make dry-run-ts-sdk's tarball exact-match assertion updated to expect dist/testkit.js/dist/testkit.d.ts alongside dist/client.*, plus a second install-then-import() check for the @offshoot-db/client/testkit subpath specifically (not just the package root) |
Branch diff (offshoot diff, ops.Workspace.MaterializeForDiff, ops.DiffSummary/TableRowCounts) |
shipped-and-tested | Milestone 3 Task 6 — offshoot diff <db>[@branch[@checkpoint]] <db>[@branch[@checkpoint]] [--summary] materializes both sides READ-ONLY (never a live checkout, never a lease) and either streams sqldiff's output over them (default) or prints a stdlib-only, sqldiff-free table-level row-count summary (--summary). CLI-only: no daemon op, no SDK parity — the plan's file-structure list scopes this task to cmd/offshoot/docs/diff.md, unlike Tasks 1/2/4/5's daemon+SDK surface. Materialization split (the staleness decision, locked and documented on MaterializeForDiff's own doc comment): a named checkpoint uses the existing CheckoutAt read-only cache (immutable content, legitimately reusable across diffs); a bare head target CANNOT be cached that way — head moves — so it is always freshly Exported to a private temp file removed after the diff, proven directly by internal/ops/diff_test.go's TestMaterializeForDiffHeadSideAlwaysReflectsANewWrite and the CLI-level TestDiffCLIHeadSideReflectsNewWriteNotStaleCache (a write + checkpoint between two --summary calls against the same head target changes the reported delta, not a stale cache). --summary's row counts come from database/sql + the mattn/go-sqlite3 driver already vendored for internal/capture/internal/ops, opened as file:<abs>?mode=ro&immutable=1 — verified empirically against a real chmod 0444 file (internal/ops/diff_test.go's TestTableRowCountsOpensReadOnlyEvenOnA0444File) that SQLite accepts a mode=ro URI parameter alongside the driver's always-requested SQLITE_OPEN_READWRITE|SQLITE_OPEN_CREATE flags (SQLite only rejects a mode LESS restrictive than the flags argument; ro is strictly more restrictive, so this is not a documented-but-untested claim). sqldiff's absence (a SEPARATE binary from sqlite3 on most platforms) produces a clear, per-OS-hinted error rather than a bare "executable file not found" — the macOS hint (brew install sqldiff) was corrected mid-task after installing Homebrew's general sqlite formula locally and confirming it does NOT ship sqldiff (only the dedicated sqldiff formula does); the Debian/Ubuntu hint (sqlite3-tools) was verified against a real ubuntu:24.04 container's apt-cache show sqlite3-tools output listing /usr/bin/sqldiff, and .github/workflows/ci.yml's Linux job now installs that package (alongside the pre-existing plain sqlite3) so the CLI tests that exercise a REAL sqldiff invocation (TestDiffCLISqldiffPresentStreamsOutput, TestDiffCLIIdenticalSidesProduceNoSqldiffOutput) run in CI rather than permanently skipping (they gate on exec.LookPath("sqldiff") the same way requireSQLite3 gates on sqlite3 itself, so a local dev machine without sqldiff still skips cleanly). Cross-db diff (two entirely different db names) is explicitly supported and tested (TestDiffSummaryCLIWorksAcrossTwoDifferentDatabases) — a legitimate shape for eval comparisons. docs/diff.md documents the command, the raw by-hand export-twice-then-sqldiff recipe, and links the FAQ's no-merge stance |
Observability and security
| Guarantee / feature | Status | Notes |
|---|---|---|
offshoot status (branch state, checkpoints, TTL remaining) |
shipped-and-tested | |
session status (durable-through txid, epoch, holder, errors) |
shipped-and-tested | |
| Structured branch-state-transition logging | shipped-and-tested | internal/session/session_test.go (TestSessionTransitionLogsOpenedFlushedClosed), internal/session/flush_test.go (TestAutoFlushTransitionLogsRecordKindAutoAndFailure) — every session state transition (opened; flushed, tagged kind=manual/kind=auto with its txid; flush-failed with the error, ErrClosed races excluded; fenced with the terminal cause; closed) writes one offshoot: session: db@branch: event key=value ... line to stderr, matching the daemon janitor's existing offshoot: janitor: ... line family |
Metrics registry + instrumentation (internal/metrics, offshoot_* names locked) |
shipped-and-tested | Milestone 4 Task 2 — hand-rolled, zero-dependency, concurrent-safe Prometheus text-exposition registry (Counter/Gauge/Histogram with fixed buckets, CounterVec/GaugeVec for bounded label sets, Registry.WritePrometheus); zero-dep rationale is one sentence on the package doc comment. Every metric on the plan's LOCKED list is registered by internal/daemon's newMetrics — offshoot_build_info{version}, offshoot_sessions_open, offshoot_capture_lag_bytes{db,branch}/offshoot_durable_age_seconds{db,branch} (open sessions only, computed at SCRAPE TIME from the sessions map via a Registry.Collect callback, not continuously), offshoot_flush_total{result,kind}/offshoot_flush_duration_seconds, offshoot_fork_total{path}/offshoot_fork_duration_seconds, offshoot_checkpoint_duration_seconds, offshoot_reap_total, offshoot_gc_tombstoned_total/offshoot_gc_deleted_total/offshoot_gc_backlog, offshoot_ro_cache_bytes/offshoot_ro_cache_evictions_total (driven by the janitor's ro-cache pass as of Task 5 — see the Resource-behavior section below), offshoot_janitor_runs_total{result}.Two families were added after the lock, in v0.2.1 (additions are fine; renames are not): offshoot_fork_mode_total{mode="shared"|"materialized"} (the copy-on-write storage mode at fork time, orthogonal to offshoot_fork_total{path}) and offshoot_gc_errors_total (janitor GC passes that returned an error — the alertable signal for a fails-closed GC that would otherwise stall silently), bringing the family count to eighteen — see docs/operations.md.Instrumentation: internal/ops's ObserveFork/ObserveCheckpoint are package-level, nil-checked injected hooks (ops must not import internal/metrics) the daemon assigns once at NewServer construction; internal/session's OnTransition hook rides the SAME transition-log call site Milestone 2 added (logTransition), not a new one, feeding flush counters/durations from the existing "flushed"/"flush-failed" events. WritePrometheus output is golden-file-tested (internal/metrics/metrics_test.go) and validated against promtool check metrics (TestPromtoolCheckMetrics/TestPromtoolCheckRealMetrics, loud-skip-if-absent locally, installed explicitly in ci.yml's new metrics-lint job). Exposed over HTTP as of Task 3 — see the next row.Concurrent-scrape safety (T2 review carried item, closed in Task 3): Registry.WritePrometheus now serializes end to end via a scrapeMu mutex — a multi-step Collect callback (the session-gauge collector below Resets then repopulates two GaugeVecs) is otherwise not atomic across two concurrent scrapes, which could tear the output (e.g. offshoot_sessions_open reporting N while offshoot_capture_lag_bytes carries fewer than N samples in the SAME response). internal/metrics/metrics_test.go's TestWritePrometheusSerializesConcurrentCollectorRuns and internal/daemon/http_test.go's TestConcurrentMetricsScrapesWithSessionChurn (real HTTP, real session open/close churn) both pin the invariant and are confirmed to fail reliably without the mutex |
Prometheus /metrics HTTP endpoint |
shipped-and-tested | Milestone 4 Task 3 — GET /metrics (internal/daemon/http.go's handleMetrics) calls the registry above directly; token-gated like everything but /healthz. See the row above for the concurrent-scrape fix and its tests |
| HTTP binding + single-token auth | shipped-and-tested | Milestone 4 Task 3 — internal/daemon/http.go. Off by default; serve -http 127.0.0.1:PORT binds loopback with no further ack needed; a non-loopback bind additionally requires -http-allow-non-loopback AND an explicit -token/OFFSHOOT_TOKEN (two DISTINCT startup errors if either is missing — ValidateHTTPBind, TestValidateHTTPBindErrorsAreDistinct). Token: -token/OFFSHOOT_TOKEN, else auto-generated (GenerateToken, 256-bit) and printed once to stderr (loopback only). Every request but GET /healthz requires Authorization: Bearer <token>, checked with crypto/subtle.ConstantTimeCompare (checkAuth); a token is never logged in full again after that one startup line — only its 8-character TokenFingerprint appears in ongoing output (TestHTTPTokenRedaction, TestHTTPAutoGeneratedTokenPrintedOnceThenOnlyFingerprint). Routes: POST /rpc (the same Request/Response JSON and Server.dispatch the unix socket uses — TestHTTPRPCParityWithSocket; 1MiB body cap via http.MaxBytesReader, 413 on overflow, connection still usable after — TestHTTPRPCBodySizeLimit; Content-Type: application/json required), GET /debug/pprof/* (net/http/pprof, same Bearer auth — TestHTTPPprofBehindAuth), GET /healthz (unauthenticated, {"ok":true,"sessions":N}). http.Server timeouts are explicit and justified in http.go's doc comment (ReadHeaderTimeout 5s, ReadTimeout 30s, WriteTimeout 90s — sized to leave headroom for /debug/pprof/profile's default 30s capture window, IdleTimeout 2m). Shutdown ordering: the HTTP listener is closed (http.Server.Close, immediate — matching the unix socket's own force-close-live-connections philosophy) alongside the unix listener in Server.Shutdown; an HTTP op=shutdown request gets the same respond-then-shutdown-trigger ordering fix the unix socket got in Milestone 3 (TestHTTPShutdownRespondsBeforeClosingRequestingConn), and a generic hammer test (TestHTTPShutdownWhileRequestsInFlightIsSafe) proves no panic/hang under -race while Shutdown races in-flight requests. Threat model: single-tenant, same-host-or-trusted-network auth — the token is a shared secret, not a multi-tenant isolation boundary; see docs/reference.md's -http ADDR section and docs/operations.md for the full operator-facing writeup (Milestone 4 Task 8) |
Branch state taxonomy (active / pending / error / dirty / detached / idle) |
shipped-and-tested | Milestone 4 Task 1 — ops.BranchStateAt (internal/ops/status.go) computes active/dirty/detached/idle from ref+lease+checkout-sidecar truth alone (no daemon dependency); a daemon layers pending/error on top from its own in-memory session map (internal/daemon/server.go's Server.branchState) — see docs/reference.md's Branch states table for the full precedence (error > pending > active > dirty > detached > idle) and why idle is a deliberate addition the design spec's original taxonomy lacked (it assumed a daemon was always present). Computed only, nothing persisted. Surfaced in offshoot status (CLI/at-rest), the daemon branches op's BranchInfo.state (always present, no omitempty — wire-additive so pre-Milestone-4 SDKs against a new daemon are unaffected), and both SDKs' Branch.state (defaults to "" against a pre-Milestone-4 daemon that never sends the field). internal/ops/states_test.go constructs each ops-level state directly (including detached via a Promote whose best-effort checkout refresh is forced to fail, per that op's own doc comment) plus an exactly-one-state invariant test across a matrix workspace; internal/daemon/branchstate_test.go covers pending (the openDelay test hook holding an open in flight) and error (lease-theft fencing, mirroring internal/session/renew_test.go's own technique) including the precedence case where a fenced session's ref shows an active lease held by the thief but the daemon still correctly reports error |
Protected-branch flag (default on for main) |
shipped-and-tested | Enforced by CLI, daemon, and MCP alike |
Operator documentation (docs/operations.md) |
shipped | Milestone 4 Task 8 — the metrics reference table, branch-states table, event schema, budget mechanics, and HTTP/auth threat model in one operator-focused page, cross-linked from docs/reference.md's wire-level sections; its own metrics table is verified against a real scrape (go build + serve -http + curl /metrics, # TYPE line count matched against the table row count). docs/recipes/kubernetes.md ships alongside it: a real sidecar manifest (docs/recipes/k8s/offshoot-sidecar.yaml), schema-validated with both kubectl apply --dry-run=client and --dry-run=server against a disposable local rancher/k3s control plane (no cluster was otherwise available in this environment). "shipped" not "shipped-and-tested" per this page's own legend — no automated test enforces documentation accuracy going forward |
| TLS on the HTTP listener | deliberately deferred | Milestone 4's self-review scoped -http to loopback-by-default plus a shared-secret token, explicitly not TLS — a non-loopback bind is documented as needing a trusted network boundary (VPN, private subnet) of its own instead. Revisit with real non-loopback demand; see docs/operations.md |
| Per-branch at-rest metrics by default | deliberately deferred | A gauge per branch that has never been opened this process would scale label cardinality with total branch count rather than open-session count — offshoot_capture_lag_bytes/offshoot_durable_age_seconds stay open-sessions-only by design (see docs/operations.md). A dbs-scoped scrape option for at-rest branches was named as a future addition in the Milestone 4 plan, not built this pass |
| Metrics push/remote-write | deliberately deferred | GET /metrics is pull-only, matching every other route on the opt-in HTTP listener; no push-gateway or remote-write integration was scoped for Milestone 4 |
Eventing: event bus, socket subscribe op, GET /events SSE |
shipped-and-tested | Milestone 4 Task 4a — internal/daemon/events.go's in-daemon bus fans one versioned event schema out to subscribers over two transports that share the SAME encoder (encodeEvent): the unix socket's subscribe op and HTTP's GET /events (SSE). type is session_opened/flushed/flush_failed/fenced/session_closed (all fed from the SAME internal/session transition-callback call site Task 2 already hooked — session.OnTransition is a single package-level func var, so wireEvents composes onto whatever Task 2's wireHooks already assigned rather than clobbering it; internal/session itself is unmodified), reaped (fed from the janitor's Reap pass in janitorTick), and evicted (as of Milestone 4 Task 5, fed from the janitor's ro-cache LRU eviction pass in that same janitorTick — see the Resource-behavior section below). Never blocks the daemon (Global Constraint): eventBus.publish is entirely non-blocking — a subscriber whose bounded buffer (eventSubscriberBuffer, 64) is full is immediately removed, sent one terminal {type:"dropped_slow_consumer"} event, and has its channel closed, all without the publisher (a session transition or the janitor) ever waiting — TestSlowSubscriberDroppedSessionKeepsFlushing proves a subscriber that never reads its channel is dropped while a concurrently write-heavy session keeps flushing successfully throughout. Socket subscribe acks THEN the connection permanently leaves request/response mode and streams line-per-event JSON until disconnect (handle() special-cases it exactly like it already special-cases shutdown, subscribing to the bus before sending the ack to close an ack-vs-transition race); SDKs must use a fresh, dedicated connection — it is unix-socket-only, refused with a pointer to GET /events if sent over HTTP POST /rpc (TestSubscribeOverHTTPRPCRefused). GET /events requires the same Bearer auth as everything but /healthz, emits a periodic : ping keepalive comment and — the loud warning Task 3's http.go left for this task — bounds its writes so Task 3's 90s http.Server.WriteTimeout can never hard-cut a long-lived stream: rather than clearing the write deadline once, permanently (a review round caught that this let a subscriber that stayed CONNECTED but simply stopped reading pin the handler goroutine and its connection's file descriptor open forever, since nothing ever re-armed a bound on it again), handleEvents re-arms a fresh eventWriteDeadline-out deadline (default 45s) immediately before every write — the header write, each event, each ping — via http.ResponseController; a live stream re-arms this forward on every successful write (and at least every keepalive tick) so it's never affected, while a genuinely stalled reader's write now times out and the handler gives up. streamEvents (the unix socket side) gets the identical per-write re-arm before every c.Write. TestSSEStreamSurvivesPastWriteTimeout proves the live-stream side structurally and cheaply (a test-only httpWriteTimeout var shrunk to 150ms, stream proven alive well past it via a real event delivered afterward — no 90-second sleep); TestStalledSocketSubscriberConnectionIsClosedWithinWriteDeadline/TestStalledSSESubscriberConnectionIsClosedWithinWriteDeadline prove the stalled-reader side through the REAL write path (an oversized event forces an actual blocking write, not a simulated one), both confirmed to fail reliably against the pre-fix code. TestSSEParityWithSocketSubscribe subscribes both transports before a real session lifecycle and asserts identical observed event sequences. No internal/session changes, so no torture run was required for this task (internal/daemon/events_test.go, full suite green under -race). Task 4b — SDK stream helpers: both SDKs ship a thin events() helper over the subscribe op — Python Client.events() (sdk/python/offshoot/client.py) is a generator yielding Event dataclass instances, TypeScript Client.events() (sdk/typescript/src/client.ts) is an AsyncGenerator<OffshootEvent> — each opening its OWN fresh, dedicated socket connection (never the caller's own Client connection, per Task 4a's dedicated-connection requirement above). dropped_slow_consumer is yielded like any other event and then the stream simply ends (not raised/thrown) — an explicit, documented decision; a caller checks the last event's type if it cares. Stopping iteration early (Python break/.close(); TS break/.return()) closes the dedicated socket with no leaked fd, proved against a real daemon via lsof -p <pid> filtered to TYPE unix (a raw total-fd count is confounded by internal/dbfile's deliberately-never-closed checkout descriptors, so the tests isolate socket fds specifically) returning to its pre-subscribe baseline after a helper connection closes mid-stream. Real end-to-end lifecycle tests drive session_opened/flushed/session_closed on a separate connection and assert the helper observes them in order; the slow-consumer-drop path is hard to force deterministically from the SDK side (it needs the real bus's buffer-overflow timing, already covered server-side by TestEventBusDropsSlowSubscriberWithTerminalEvent), so both SDKs instead unit-test the helper's decode/contract path end to end against a small scripted fake daemon. Both SDK suites green under make test-sdks; both confirmed stdlib-only/zero-dep |
Resource behavior
| Guarantee / feature | Status | Notes |
|---|---|---|
| Checkout-cache disk budget + LRU eviction | shipped-and-tested | Milestone 4 Task 5 — offshoot serve -ro-cache-budget <bytes|0> (default 0 = unlimited) bounds checkouts-ro (the read-only CheckoutAt cache, Milestone 3 Task 2), never checkouts/ (writable, leased — see below). Server.janitorTick (internal/daemon/server.go) runs a new ro-cache pass on the same -reap-every cadence as reap/GC: always computes and republishes current usage into offshoot_ro_cache_bytes (the Task-2-registered gauge, previously always 0 — now updated once per janitor pass, not continuously; see docs/reference.md's -ro-cache-budget section for the resulting between-passes staleness), and once usage exceeds the budget, LRU-evicts (ops.Workspace.EvictROCache, internal/ops/rocache.go) oldest-first until back under it. LRU clock: a .last-used touch-on-HIT marker file, not the .db file's own mtime — CheckoutAt's force=false cache-hit path now calls touchLastUsed, Chtimes-ing a <cachefile>.last-used sidecar to now on every hit; necessary because materializeAt's rename-into-place is the LAST thing that ever touches the .db file's mtime (a hit is a pure read, never touching it again), so without a separate marker "least recently used" would collapse to "least recently created" — exactly backwards for a cache whose point is that a repeatedly-hit checkpoint stays hot. lruClock falls back to the .db file's own mtime as the floor for an entry materialized but never since hit. TestEvictROCacheLastUsedTouchBeatsCreationOrder (internal/ops) and TestJanitorTickEvictsLRUUnderBudgetAndFiresEvictedEvent (internal/daemon) both prove an OLDER-by-creation entry that gets hit survives while a NEWER-by-creation, never-hit entry is evicted instead. checkouts/ is never evicted by construction, not a runtime check: EvictROCache only ever walks/removes paths under the separate checkouts-ro tree, never joined with or reachable from checkouts/'s own path shape — TestEvictROCacheNeverTouchesWritableCheckout and TestJanitorROCacheEvictionNeverTouchesLeasedWritableCheckout (a real open, leased session survives an aggressive budget=1 pass and can still flush afterward) confirm this end to end. Eviction is LOUD: one offshoot: janitor: ro-cache: evicted <db>@<branch>@<checkpoint> (<bytes> bytes) stderr line per eviction, offshoot_ro_cache_evictions_total incremented (previously always 0), and Task 4a's reserved evicted event type gets its emitter here ({type:"evicted", db, branch, detail:{checkpoint, bytes}}, published non-blockingly to the T4a bus at the eviction call site). offshoot status gains an ro-cache usage summary line (ops.Workspace.ROCacheUsage) plus a display-only -ro-cache-budget flag of its own (the budget itself is never persisted, matching every other serve tuning flag, so this at-rest command has no other way to show usage against it without a live daemon). Each eviction removes both the .db and its .last-used marker together; a mid-pass os.Remove failure follows Reap/GC's own partial-progress-plus-first-error convention. go test ./internal/ops ./internal/daemon -race clean; no internal/session/internal/capture changes, so no torture run was required |
| FD budget with idle-checkout eviction | deliberately deferred | Milestone 4's self-review named this a consciously narrowed bullet, not a silently dropped one: internal/dbfile's file descriptors are deliberately unclosable by that package's own design (see its doc comment) specifically to avoid a stray-close lock hazard, which makes "evict a cold session's FD" a real design problem, not just an unimplemented budget loop — it needs its own pass rather than reusing the ro-cache budget's shape. See docs/operations.md |
SnapshotEvery tuning exposed via the daemon |
shipped-and-tested | Milestone 4 Task 6a — offshoot serve -snapshot-every N (default 16, unchanged if omitted; rejects < 1) plumbs into session.Options.SnapshotEvery at opOpen the same way -flush-every plumbs FlushEvery (Server.SetSnapshotEvery, single-writer-before-Serve contract). Unlike -flush-every 0, there is no "disabled" sentinel — every flush must eventually snapshot, so SetSnapshotEvery's n <= 0 means "unset, session.Open applies its own default" rather than "off," and the CLI only calls it when the flag was actually given. internal/daemon/snapshot_every_test.go's TestServeSnapshotEveryWiringDrivesCadence opens a session over the socket with SetSnapshotEvery(4), drives 9 flushes, and asserts both a bounded store.Chain (≤4 members, mirroring ops_test.TestReplayStaysBoundedAcrossManyFlushes) and more than one snapshot object across the lineage listing — proof the cadence actually recurred under daemon wiring, not just the mandatory first-ever snapshot. cmd/offshoot/main_test.go's TestServeNonPositiveSnapshotEveryIsRejected pins 0/negative/non-integer values as usage errors. See README's What a flush costs for the cadence's bandwidth-vs-replay-latency trade-off |
| Current per-session disk/FD costs documented | shipped | Documented in README's Resource behavior: per-session FD footprint is small and fixed; disk is the sharper cost via internal/dbfile's never-closed descriptors (see that package's doc comment), now mostly avoided on a clean re-open by Checkout's clean-skip fast path, but not reclaimed for a checkout that does get re-materialized. "shipped" rather than "shipped-and-tested" per this page's own legend — no automated test enforces documentation accuracy |
Platform
| Guarantee / feature | Status | Notes |
|---|---|---|
| Linux + macOS | shipped-and-tested | |
| Windows | not supported (non-goal) | Capture path and lock/SHM probing are POSIX-dependent; see ROADMAP non-goals |
| Multi-node orchestration / clustering | not supported (non-goal) | Shared-bucket safety is guaranteed by fencing; placement/failover/routing are explicitly out of scope — see ROADMAP non-goals |
| Merge (three-way) | not supported (non-goal) | Forks are pick-a-winner via promote; the escape hatch is sqldiff over two checkouts — see docs/faq.md |