Changelog
All notable changes to this project are documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning once it reaches 1.0. Before 1.0, minor versions may include breaking changes.
Format stability: offshoot is pre-1.0. The on-disk/on-bucket storage
format (refs, checkpoints, segments, leases) may change in a
backward-incompatible way in a minor release before 1.0; patch releases do
not break it. Any such break ships in the same release with either an
in-place migration or a documented export → create --from path, and an
older binary refuses a newer store's layout outright rather than guessing.
Pin an exact version if you depend on format stability. The full contract:
docs/stability.md.
[Unreleased]
Nothing yet.
[0.2.22] - 2026-10-17
Checkpoints can now be deleted: offshoot prune removes them by name or
keeps the N newest, on the CLI, the daemon, MCP and both SDKs, so a
long-lived branch that names a checkpoint per step keeps each new
checkpoint fast without any change to the storage format.
Added
offshoot prune <db>[@branch] (<checkpoint>... | --keep N) [--force]deletes named checkpoints, or all but the N newest, from a branch, on every surface: the daemon'spruneop (names/keeprequest fields,pruned/keptin the response), the MCP tooloffshoot_prune_checkpoints(force gated by-allow-forcelike destroy's), andprunein both SDKs. Until now a checkpoint could not be removed at all, and every checkpoint reads and rewrites the branch's whole checkpoint list, so a branch that named one per step paid more for each new one (about 3.5 ms after 1,000). Only checkpoint metadata changes: the head and data are untouched, gc reclaims what only the pruned checkpoints kept, and the ref format is unchanged. An unknown name refuses the whole call; a protected branch needs--force; it is safe beside a live session (reference).
Changed
- Python SDK: request fields are left off the wire only when
None,Falseor"", compared by identity, so an integer0is sent (it used to be dropped, since0 == False;diff'smax_bytes=0now reaches the daemon, which reads it as the default, as before). - Tests drive the
sqlite3CLI with a 5 s busy timeout in the TypeScript SDK suite too, astestutil.SQLite3CLIdoes for the Go tests.
[0.2.21] - 2026-10-17
A specification-conformance initiative: every documented requirement was traced to its implementation and a test, the gaps were closed with mutation-verified regression tests, and the defects that surfaced are fixed below. The traceability summary, coverage and mutation results are in docs/testing.md.
Fixed
create --fromopens its source read-only, so importing a WAL-mode database whose last writer exited without a clean close no longer checkpoints and unlinks that database's-walon the way out; the source is byte-identical before and after, as the docs promised (TestCreateFromLeavesAWALModeSourceByteIdentical). A rollback-journal source with a hot journal, which the old read-write import silently rolled back (modifying the source), is now refused with the recovery step named.- A ref named like the local backend's lock or temp files (
x.lock,x.tmp-1) was invisible to listings there (so GC would have swept its lineage) and unlinked by a sibling's CAS release;ValidateNamenow refuses those shapes on every backend. And a ref underrefs/whose name fails the current rule (written by an older binary or by something else) now failsListRefsloudly, naming the key, instead of being skipped — a skipped ref's lineage was unmarked and swept by GC while the ref still existed (limitations). compactoverwrote an existing checkpoint namedcompact(one from an earlier compact, or one you named yourself — its metadata was lost); the entry is now kept as is and compact adds its own only when the name is free.promoteskipped minting the target's safety fork whenever the source was named<target>-pre-promote; the undo shortcut is now keyed on the safety-fork marker, so a branch of your own at that name meets the documented refusal instead.- A GC pass racing a checkpoint on a forked parent failed with a phantom "no contiguous run reaches target": the mark now reads every ref before listing any lineage, so a listing is never older than the ref it serves.
- The daemon's
shutdownop started a Shutdown with no deadline, so a janitor pass that outlivedserve's 30 s wait left every session open and every lease held; the op now applies the same 30 s budget as a signal-driven shutdown (daemon.ShutdownBudget), and the janitor half-budget rule holds on both paths. serverefused a short explicit token only after binding its socket and starting the janitor, leaving the socket file behind; the check now runs with the other startup validation, before anything is created. Negative-reap-every/-gc-gracevalues are refused instead of silently disabling the janitor or shortening grace.diff --table Tin the default (sqldiff) mode printed nothing for a table that exists on neither side; it is now the same error--summarygives.- A refused
create/create --fromof an existing name no longer uploads a whole snapshot first (an orphan per typo'd re-run, on S3 a full database PUT); the name is checked before the upload and still by the ref CAS. "Exists" now means any branch under the name: a database whosemainwas destroyed while other branches survive no longer accepts a fresh, unrelatedmainfromcreate. fork --atvalidates the checkpoint name like every other name.- MCP: the modern-era (2026-07-28)
tools/callenvelope droppedstructuredContent; it now carries it as the docs state. The fork tool's TTL note described reaping as requiring a runningoffshoot servejanitor; it now describes the server's own-reap-everytimer. - Python SDK: an
okresponse missing a documented key (checkoutfor open/checkout/rollback/checkout-at,txidfor fork/flush/promote/ compact,branch/namein listings) raisesOffshootErrorinstead ofKeyErroror a silent0/"";ttl=True/Falseis aTypeErrorrather than"True"/"none". TypeScript SDK: the same keys throwOffshootErrorinstead of yielding an undefined path or a0txid;forkwith conflictingat/fromthrowsOffshootError. Optional fields (backup,shared,session_id,dbfile_descriptors,state) still default, so older daemons keep working. scripts/check_sdk_versions.pyalso checks the LangGraph companion'spyproject.tomlagainstsdk/VERSION.
Added
rollbackreports the abandoned transaction range (abandoned txids N..M;RollbackResult.AbandonedFrom/To), as the architecture doc had promised.daemon.ValidateHTTPToken(the explicit-token length rule, shared byserveandStartHTTP) anddbfile.ReadOnlyDSN.- Seventy-six mutation-verified Go regression tests (plus 15 Python, 13 TypeScript) across every package for the
documented contracts that had none: claim-guarded checkpoint/protect
refusals, TTL-clock stamps on every repoint, the under-64-page segment
rule, safety-fork edge cases, the full event-type set with detail keys,
-token-fileandOFFSHOOT_TOKENhandling, HTTP method/content-type codes, the per-request socket cap, snapshot-cadence seeding on a shared fork, equal-length wrong tokens, protected-branch refusals over every surface, local key containment, exact 0600 checkout modes, GC fail-closed on a mark error, truncated LTX objects, v1 layout read-without-bump, lease clock skew, MCP tool schemas and refusals, and SDK wire bytes and error contracts.
Changed
- Docs now say what is built: the architecture and limitations pages no
longer describe a crashed daemon's un-flushed tail as "preserved as an
orphan fork" (it is discarded after a warning; the orphan fork is
designed, not built), a fork no longer "finds its fork point tombstoned"
(grace plus re-mark are the protection), the local CAS lock is described
as
flock(2), the property-test and fault-injection bullets name what exists, and-token-filecounts as an explicit token everywhere.
[0.2.20] - 2026-10-12
Two windows in which the capture engine could silently lose a committed transaction are closed: the rebase snapshot copy and the shutdown clean-resume hash both run under the read lock now, so a foreign checkpoint can neither fold a write past the copy nor into the hash baseline. The local store's per-key lock is an flock, so a crash can no longer let two writers into one compare-and-swap; destroy removes the branch's read-only cache entries; the daemon refuses a socket a live daemon serves, survives a descriptor-ceiling accept error, closes sessions concurrently and caps requests per request rather than per connection; forking from a live session no longer stalls the agent. Release binaries are built with the toolchain go.mod pins (go1.27.2): v0.2.19 and earlier shipped on go1.26.0, which carries 29 reachable standard-library fixes. Also new: a nightly govulncheck job, a ruff gate for every Python file, and cmd/bench-ab with the nightly perf-ab leg.
Added
cmd/bench-ab, a tool that buildscmd/branchbenchfrom a git ref given as-before(required; the tool has no default and exits 0 with a line when none is given) and from the working tree, then alternates the two binaries for 4 rounds per workflow on one machine (default workflowsfailure_repro:1,simulation:8) and reports per-metric medians with a verdict. Only a gating metric (fork, checkout or checkpoint depth-1 p50) can fail the run: it regressed when the after median exceeds the before median by more than 15% AND every after sample exceeds every before sample; wall and eval are informational only. The built binary exits 0 (no-beforeref, or nothing regressed), 1 (a gating metric regressed) or 2 (bad flags, unknown ref, a build, run or parse failure);go runfolds any non-zero status into its own 1 andmakeinto its own 2, somake bench-aband the nightly buildbench-abfirst and run the binary, and undermakea regression shows as make's exit 2 (the report tells the two apart). Each raw per-run file starts with a# bench-ab side=<side> ref=<short rev> round=<n>line.make bench-ab [BEFORE=<ref>]runs it locally and, like the newperf-abjob in.github/workflows/nightly.yml, supplies the latest release tag (git describe --tags --abbrev=0 --match 'v[0-9]*') when no ref is given; the job runs on days main moves, on the Sunday leg and on manual dispatch, withsimulationat four workers to fit the four-vCPU runner. Its first run (a manual dispatch on 2026-10-09, still at eight workers) took 29 min 21 s, exited 0, and reported two improvements against v0.2.19 with every other gating row "ok" (benchmarks). 19 tests incmd/bench-ab:TestParseRowDepthOne,TestParseRowDeeperTreeReadsTheDepthOneP50,TestParseRowWallInMinutes,TestParseRowWallInMilliseconds,TestParseRowRejectsGarbage,TestParseRowRejectsAMalformedNumber,TestParseRowRejectsATimedOutRun,TestFindRowPicksTheWorkflow,TestMedian,TestVerdictRequiresEveryAfterAboveEveryBefore,TestVerdictRejectsTooFewRounds,TestImprovedMirrorsTheRegressionRule,TestExtractTarKeepsModesAndRefusesEscapes,TestRunOrderAlternates,TestTailLines,TestRawHeaderNamesSideAndRefAndKeepsTheRowFindable,TestOnlyGatingMetricsDecide,TestGatingRegressionStillFailsTheRun,TestParseSpecs.- A
vulncheckjob in.github/workflows/nightly.ymlrunsgovulncheck ./...(v1.8.0) against the toolchain go.mod pins and the module graph. It is deliberately not gated on thefreshoutput the other nightly jobs share: a Go point release that fixes a function the binary reaches arrives without any commit here, and this job is what notices it (Dependabot bumps modules, never thetoolchainline). make lint-pythonand a matching step in ci.yml'ssdksjob runruff checkandruff format --check(ruff 0.17.0, rules in the newruff.toml: pyflakes/pycodestyle core, isort, pyupgrade for the 3.10 floor, bugbear, unused-noqa) over both Python SDKs,scripts/,examples/and the pytest plugin's tests. Until now no Python file in the repo was linted or format-checked anywhere; the first run reformatted 16 files and fixed 63 findings (quoted annotations left over from beforefrom __future__ import annotations, unsorted imports, nine stale# noqadirectives, onetypingimportcollections.abcreplaces), none of which changed behaviour — the unit, pytest-plugin and LangGraph suites pass unchanged. Markdown is excluded: ruff 0.17 would otherwise reformat the Python blocks in the docs, whose trailing comments are aligned by hand..gitignorecovers/offshoot, the binarygo build ./cmd/offshoot(and CONTRIBUTING's-o offshootform) leaves at the repo root.
Changed
cmd/offshoot:rollbackandpromoteshare one flag parser (extractRepointFlags);mcp's-default-ttland-reap-everyshareparseDisableableDurationFlag(two copies of the same function with different defaults).internal/ops:reapOneuses thenowit is passed for the delete-claim check instead of a second clock read.cmd/offshoot'srunis now a dispatch table: each of the twenty commands (create…session) is its owncmd<Name>(w, rest)function (plusspecfor the three —mcp,serve,session— that need the store path), instead of a 946-lineswitchwhoseservearm alone ran 218 lines. Mechanical move, no behaviour change; the command suite inmain_test.gopasses unchanged.internal/metrics:NewCounterVecandNewGaugeVecshare onevecRows/writeVecRowspair instead of two identical 25-line render closures.internal/ops:copyFile, used only by the clone, export and materialize tests, lives incopyfile_test.gonow rather than inops.go.make lintpins staticcheck to a master commit (v0.7.0-0.dev.0.20261009230814-452d5bb86b45): the latest release, v0.8.1, cannot read Go 1.27 export data (export data version 5 is greater than maximum supported version 4) and the target's non-blocking||had hidden that on every run since the local toolchain moved to 1.27. The clean result is real again (zero findings across./...).scripts/check_sdk_versions.py,scripts/check_sdk_tag_version.py,scripts/bench-isolation.py,examples/eval-pass-k/run.pyandexamples/langgraph-rewind/agent.pycarry the executable bit their shebang lines implied.scripts/ci-local.shinitialises the two summary variables itsevalassigns, soshellcheck -S warningis clean over every tracked shell script.- Dependency bumps (Dependabot PRs 87 and 88):
aws-sdk-go-v2/service/s31.113.4 to 1.114.0,pierrec/lz4/v44.1.31 to 4.1.33 (the LTX frame fixtures and fuzz oracles ininternal/ltxiopass unchanged against it);anchore/sbom-action0.24.2 to 0.24.3 inrelease.yml.
Fixed
-
A capture-engine rebase could silently lose a committed transaction.
rebasecopied the snapshot of the main file with no read transaction held, on the reasoning that frames a foreign checkpoint folded mid-copy "stay physically present in the WAL" and would be re-applied. That holds only until the WAL restarts: a foreign commit T1 large enough to trip the agent connection's autocheckpoint was folded into pages the copy had already passed, the next commit T2 restarted the WAL under new salts, and the fresh reader bound to T2's generation never saw T1. The replica diverged withrebased=1, no error, a phantom lag, and every later flush shipped a self-consistent snapshot of the wrong content; a clean close then stamped the checkout's hash next to the replica's checksum, so the next open resumed over it.rebasenow begins the read transaction right aftercheckpoint(TRUNCATE): on the empty WAL that holdsREAD_LOCK(0), which every checkpoint needs exclusively to backfill and which a WAL restart requires first.internal/capture'sTestRebaseCopyHoldsReadLockAgainstFoldedTransactionreproduces the sequence through a mid-copy hook (1200 of 1200 updated rows missing on the unfixed engine) and pins the fix. -
Shutdown could record a clean-resume baseline over a write it never captured.
shutdownhashed the main file aftercheckpoint(TRUNCATE)with no lock held, so a commit whose autocheckpoint folded it in between enteredState{Clean:true, MainHash}without ever passing through the replica; once the agent's last connection closed (SQLite checkpoints and unlinks the WAL), the next start found a matching hash and resumed over the missing rows. The hash now runs under a read transaction begun on the empty WAL, after re-checking the WAL is still empty; a write that landed leaves frames the lock keeps from being folded, and the clean save is refused (TestShutdownRefusesCleanStateWhenAWriteLandsBeforeTheHash). Residual, documented at the site: a foreign TRUNCATE-mode checkpoint that both folds a write and empties the WAL in the gap before BEGIN. -
The local store's per-key lock could admit two writers, losing a ref update silently.
lock()broke a stale lock file by path (Stat, age > 30 s,Remove); two breakers racing the same stale file could each remove the other's fresh lock and both enterPutIf's read-compare-rename with the same etag, after which both renames landed and one update vanished with no error (invariant 2). The lock is now anflock(2)on the lock file: a process that dies inside the critical section releases it with its last descriptor, a leftover file is just a file (no stale-age heuristic), release unlinks the file while still holding the lock, and an acquirer re-checks that the inode it locked is the one at the path.TestLocalLockNeverAdmitsTwoHolders(eight contenders under a goroutine that keeps backdating the lock file's mtime) saw three simultaneous holders on the old lock. A lock file left by a dead process no longer costs anything (it used to block up to the 5 s deadline when younger than 30 s). The conformance suite's concurrent-CAS test now also asserts every loser getsstore.ErrCAS. -
Directory fsync failures were swallowed.
fsutil.SyncDirignored every error, so an EIO from the directory sync after a ref rename leftPutIfreporting success and the session reporting the txid durable (invariant 8). It returns the error now, except the errnos that mean "this filesystem does not fsync directories" (EINVAL, ENOTSUP, EOPNOTSUPP, ENOTTY). The capture state file,ltxio.Materializeandfsutil.WriteFileAtomicpropagate it. The local store does not: its writes promise (WritesSettled) that an error means nothing landed, and callers such asops.Destroy's claim handling re-read on that promise, so a directory-sync failure after a landed rename is reported on stderr (naming the directory and that durability across power loss is not assured) and the write still returns success. -
checkout --atanddiffcould serve a destroyed branch's state. The read-only checkout cache is keyed by (db, branch, checkpoint name) and a hit never reads the store;destroyleft the entries in place and every fork's auto checkpoint is namedfork, so the destroy-and-refork- under-the-same-name loop (pytest fixtures, eval harnesses) read the dead branch's fork point with no store access.destroy(and TTL reaping) now removescheckouts-ro/<db>/<branch>@*.dbwith the ref;TestCheckoutAtAfterDestroyAndReforkServesTheNewBranchpins the loop, and the old test that asserted the stale hit is flipped. docs/limitations and docs/reference no longer describe this as an accepted tradeoff. -
offshoot gc --gracewas honoured only as exactly--grace VALUE.--grace=2h,-grace 2h, a bare--graceor a stray positional all ran GC silently with the one-hour default and exit 0. gc, create (--from), destroy (--force), rollback and promote (--to/--onto,--force,--no-backup,--materialize,--backup-ttl),lease acquire --ttlandinitnow parse through the shared flag helpers: both dash spellings and--x=valuework everywhere, leftovers are a usage error, a flag without a value says so, and every duration error reads--flag "x": not a durationinstead of Go's wording (TestEveryCommandParsesFlagsInAnyFormAndRejectsLeftovers). -
A second
offshoot serveon a live daemon's socket took it over.NewServerunlinked whatever was at the path; a supervisor restartingservewhile the old process was still closing left the first daemon running with sessions nobody could reach, and its eventual Shutdown (UnixListener.Closeunlinks its path) removed the second daemon's socket too.NewServernow dials the path first and refuses one that answers ("a daemon is already serving at …"); unlink-on-close is off, and Shutdown removes the path only while it is still its own socket. -
A transient
accept(2)error (EMFILE/ENFILE) exited the daemon.Servereturned the first non-closing error, so a connection burst against the descriptor ceiling turned into a crash with every lease held and every checkout un-stamped. It backs off and keeps serving now (TestServeSurvivesATransientAcceptError). -
Shutdown closed sessions one after another, on one budget.
Session.Closeis slow in proportion to its checkout (the engine's shutdown hashes it), so with a handful of large checkouts a 30 s SIGTERM degraded into a timeout for the sessions that did not make the cut; and a janitor GC pass still running at SIGTERM spent the whole budget before any session was closed. Closes run concurrently, and the janitor wait gets half the remaining budget, after which sessions close anyway and the overrun is reported (TestShutdownClosesSessionsConcurrently). -
The daemon's 16 MiB request cap was per connection, not per request. Both SDKs keep one socket for the client's life, so a long-lived harness's ~140k-th small request was answered with EOF. The cap resets after every decoded request (
TestSocketConnectionCapIsPerRequest). -
forkandpromotefrom a branch with a live daemon session stalled the agent for 3 s. The un-checkpointed-changes warning probed the checkout with a TRUNCATE checkpoint, which busy-waited the full 3 s quiesce timeout against the capture engine's read transaction while holding the WAL writer lock, so the agent's commits froze for as long — to print a warning on the daemon's stderr. The probe is PASSIVE now, which never waits on readers; since PASSIVE also never reports busy, the probe reads the checkpoint's frame counts and treats an incomplete backfill as busy, so the warning stays "busy" rather than becoming a misleading "stale, run checkout" (refreshes still use TRUNCATE).TestForkFromLiveSessionSeesLatestWritewent from 3.3 s to 0.2 s and now bounds the latency;TestQuiesceWithPassiveReportsAReaderAsBusypins the frame-count rule. -
Local.CopyObjectchmods its result to 0600: reflink's plain-copy fallback (ext4) created 0644 objects where every other object is 0600. -
/metricsanswers 500 on a render failure instead of 200 with an empty body; the branch is unreachable with abytes.Buffer, but a scraper must never read a silent empty scrape as success. -
Python and TypeScript SDKs:
connect()/Client()against a socket nobody serves raisesOffshootError(naming the path andoffshoot serve) with the OS error as its cause, instead of a bareFileNotFoundError/ENOENT; a response line that is valid JSON but not an object is a malformed-responseOffshootErrorinstead of anAttributeError;Branchcarriesshared(the daemon already sent it); the TypeScriptSessionInfotype declaresdurable_age,last_flush_at,flush_errorandcapture_lag_bytes, which arrived at runtime but were invisible to the type checker. -
Docs:
offshoot_diffis read-only too (reference saidoffshoot_listwas the only one);rollback --forceis CLI-only (the parity table said full parity); theserveheading and--helpline list-token-file; the subscribe-connection comments state the real contract (any byte the client sends after the ack ends the stream silently); architecture's invariant 5 says what "live connection" means for the refresh probe (an idle open connection is not detected); CONTRIBUTING showsmake PYTHON=python3.12 …for a platform whosepython3is older than 3.10. -
Release binaries were built with go1.26.0.
go.moddeclaredgo 1.26.0with notoolchainline, andactions/setup-gohonours that literally, so everyrelease.ymlbuild since the 1.26 move (the v0.2.19 run of 2026-10-08 included) shipped a toolchain against whichgovulncheckreports 29 reachable standard-library vulnerabilities fixed in 1.26.1 through 1.26.9 (net/url,crypto/x509,net/httpand friends, all on the S3 and daemon HTTP paths).go.modandsite/gen/go.modnow pintoolchain go1.27.2, which setup-go v7 reads ahead of the language floor; thego 1.26.0floor itself is unchanged, sogo install ...@lateststill works from a 1.26 host (Go downloads the pinned toolchain on demand). The Docker image was never affected: its build stage already pinnedgolang:1.27. The nightlyfuzzjob's hand-writtengo-version: '1.27.x'is gone with it — every job reads the one pin from go.mod. -
Dependency bumps (Dependabot PRs 87 and 88):
aws-sdk-go-v2/service/s31.113.4 to 1.114.0,pierrec/lz4/v44.1.31 to 4.1.33 (the LTX frame fixtures and fuzz oracles ininternal/ltxiopass unchanged against it);anchore/sbom-action0.24.2 to 0.24.3 inrelease.yml.
[0.2.19] - 2026-10-08
The at-rest checkpoint's stamp verifies the checkout's content from the one read it already makes, which removes the second pass v0.2.16 added and brings the single-worker checkpoint back to about v0.2.15's cost with the content checked every time; a shared child lineage reuses its recorded chain until it writes its own snapshot; a repoint or create whose ref write landed but reported a failure keeps its lineage's objects instead of deleting them; and every reader of a SQLite database header refuses a header that is not one.
Fixed
- Every reader of a SQLite database header applies one verdict. The
shared header parser (
readDBHeader, behindChecksumDatabase,EncodeSnapshot,StreamChecksum,ApplySegmentsand the segment planner's compare of the checkout's header with the shadow's) now refuses a header without theSQLite format 3magic or with a page size that is not a power of two in [512, 65536]. Before, that check lived only inStreamChecksum:ChecksumDatabase, which a daemon session runs over its replica, divided by zero in the lock-page computation on a file whose header decoded to a page size of 0 (any all-zero file), and so did the segment planner when both the checkout and its shadow had been overwritten after the quiesce; a page size that is not a power of two gaveChecksumDatabasea wrong checksum with no error.EncodeSnapshotalready failed on such a header, through the LTX encoder, with a less specific message. An at-restcheckpointof an overwritten checkout already failed before any of this, when SQLite refused to open the file (TestReadDBHeaderValidatesMagicAndPageSize,TestChecksumDatabaseRejectsBadHeader,TestEncodeSnapshotRejectsBadHeader,TestApplySegmentsRejectsBadStartHeader). - A create, rollback, promote, compact or fork whose ref write landed
but came back as an error no longer deletes the new lineage it just
wrote. The S3 SDK's retry answering 412 to its own first attempt, or a
timeout that loses the response, used to make the verb treat its own
successful write as a lost race and delete the new lineage's base
pointer or snapshots, leaving the branch that now named that lineage
unreadable until something rewrote it (for
createandcreate --from,db@mainnamed a lineage with no snapshot, and the user was told the database already existed). Each of the five now re-reads the ref and compares it against the lineage ID it minted. A match proves the write landed, and the verb finishes as success. The ref still naming the lineage it named before the write proves the write did not land; only then does today's cleanup run. Anything else (a third lineage, no ref, or, forforkandcreate, any existing ref) leaves the objects for reachability GC to reclaim instead of deleting them. Which error comes back: a re-read that itself fails returns a new error, which says the write may have landed, names the kept lineage and tells the user to runoffshoot statusbefore retrying; every other outcome returns the same error text as before, byte for byte ("already exists", "fork db@branch: err", "create db: err", or "… lost a race (retry): …"). A race lost to a write that keeps the branch's lineage (a flush, a touch, a lease renewal) still deletes the new lineage's objects at once. A race lost to another repoint, a fork onto a taken name on the shared path, and a create onto a taken name instead leave the new lineage's objects (onebase.jsonon the shared path; full snapshot copies on a materialize or create path) until the nextoffshoot gc. A fork that materializes now reads the destination ref once (one GET of the ref key, no LIST) before copying a snapshot, so forking onto a taken name on that path writes nothing at all; the shared path still has no such pre-check and relies on its ref write's compare-and-swap to settle the race. Tests:internal/ops/repoint_landed_test.go'sTestForkLandedWriteKeepsBaseAndIsReadable,TestForkLostRaceLeavesOrphanForGC,TestRollbackLandedWriteKeepsObjects,TestPromoteLandedWriteKeepsObjects,TestCompactLandedWriteKeepsCopies,TestRepointSettleUnknownKeepsObjects,TestRollbackLandedThenRepointedAwayKeepsObjects,TestRollbackLostRaceLeavesObjectsForGC,TestRollbackLostToRefWriteRemovesObjects,TestForkLandedThenDestroyedKeepsObjects,TestRollbackSettleUnknownKeepsObjects,TestPromoteLostToRefWriteRemovesObjects,TestPromoteSettleUnknownKeepsObjects,TestCreateLandedWriteKeepsSnapshotAndIsReadable,TestCreateOntoTakenNameLeavesSnapshotForGC,TestForkMaterializeOntoFreeNameReadsRefOnce, andTestForkMaterializeOntoTakenNameWritesNothing.
Changed
- A checkpoint's stamp reads the checkout once again. The stamp now
verifies the checkout's content from the one read it already makes for
the sidecar hash:
ltxio.StreamChecksumfolds the LTX page checksum over that read while it feeds the SHA-256, the two computed concurrently, and the stamp is trusted only when the file's fingerprint did not change across that read, that checksum equals the head's, and no WAL frames sit beside the file. v0.2.16 applied the 1 s racily-clean margin to the stamp's matching-fingerprint shortcut, and a checkout written within that second, as one checkpointed right after its writes is, was then read a second time to checksum it; that second pass is gone, and the margin no longer applies to the stamp (it still governscheckout's fingerprint fast path). The content check now runs on every stamp; through v0.2.18 a matching fingerprint (from v0.2.16, only a settled one) was trusted without it. BranchBench's single-workerfailure_reprocheckpoint p50 went from 60.8 / 60.5 / 60.3 ms to 51.8 / 51.3 / 52.2 ms, alternating v0.2.18 with this change run by run (benchmarks). The page checksum is the slower of the two digests, soStreamChecksumfolds each block's pages in two halves on two hashers at once (the rolling checksum is an XOR fold, so the halves recombine in any order): the one pass over a 64 MiB file in the page cache went from 32.5 to 24.5 ms by the medians of three runs, against 22.1 ms for a SHA-256-only read of the same file (BenchmarkStreamChecksum,BenchmarkStreamChecksumSHAOnly;TestStreamChecksumSplitFoldMatchesForEveryPageCountpins the fold againstChecksumDatabasefor every page count from 1 to 40 with 16-page blocks). - A shared child lineage reuses its recorded chain. A lineage that
still resolves through a base pointer (a fresh fork, or a shared
rollbackorpromote) used to list its base lineage's epoch directories and its own on every segment checkpoint until it wrote its own snapshot, because the sidecar's recorded chain was accepted only when every key was the lineage's own. Its checkpoints, a fork at its head, a promote and a rollback to the head's own checkpoint now reuse the chain the sidecar recorded, validated against the ref's base pointer with no store read: the ancestor keys must end exactly atref.Base.TXID, the last of them on the base lineage, and the lineage's own must start atref.Base.TXID + 1. Promote and a rollback to the head's own checkpoint now take the recorded chain for every branch, shared or not, as a checkpoint and a fork at head already did. They therefore read the.sumsidecar of the checkout they resolve (promote's source, or the branch a rollback repoints), and fall back to listing the lineage when it records no usable chain. The first checkpoint after a checkout still lists, since no chain is recorded yet, as docheckout, a fork below head and materialize. The saving covers the checkpoints from the second up to and including the one that writes the child's own snapshot, which comes when the chain reachesSnapshotEveryif set, elseForkShareMaxDepth(16). A fresh child's second checkpoint after 1,000 checkpoints on its parent read 34.23 and 34.87 ms in two before series and 24.84 and 19.99 ms in two after series, run one after another, not alternating (benchmarks).
[0.2.18] - 2026-10-08
The at-rest checkpoint no longer lists its lineage to resolve its own chain
when the sidecar recorded the head's chain, so a local store's per-checkpoint
cost stops growing with the epochs the checkpoint lease mints (and S3 makes
one LIST fewer per segment checkpoint); emptied epoch and lineage
directories are removed as deletes empty them.
Changed
- At-rest checkpoints and forks at head no longer list the lineage to
resolve their chain when the
.sumsidecar already records the head's chain (any mismatch — a repoint, a session flush, an old or corrupt sidecar — still resolves by listing, as before). - A local store removes an epoch directory, then its lineage directory, once a delete empties it, instead of leaving it behind forever.
Fixed
- An at-rest checkpoint no longer lists the lineage to resolve its own
chain when the
.sumsidecar already recorded the head's chain (oneLISTfewer per checkpoint on S3 in that case; on a local store the listing grew with the epoch count — 26.3 → 36.4 ms from 1 to 1,000 checkpoints on one lineage, now 25.5 → 30.3 ms, a smaller climb, not a flat one). A first checkpoint after a fresh checkout, a daemon session, a repoint, or a distrusted stamp still lists once, before the sidecar has a recorded chain to reuse; a shared child lineage still lists until it writes its own snapshot (tracked as a ROADMAP follow-up). A separate diagnostic run found the largest measured contributor to what remains: the ref's ownCheckpointsmap, which grows by one entry per named checkpoint and is read and written on every at-rest checkpoint afterward, not yet addressed; part of that diagnostic's own climb is still unexplained (benchmarks).Store.Chainitself is unchanged forcheckout, a fork below head and materialize, which still list every epoch directory a kept checkpoint left.
[0.2.17] - 2026-10-05
The daemon's descriptors are pinned and budgeted so a garbage-collected
handle can no longer drop a SQLite lock; a closing session keeps its branch
and the daemon says so; the at-rest checkpoint takes the branch lease, so
two checkpoints on one branch are refused instead of racing; and every
daemon session holds its lease under a holder unique to that session, so
twin containers can no longer renew each other's leases, with
lease release --holder as the exact, safe way to free an orphaned one.
Fixed
- A re-materialized checkout's old descriptor could be closed by the
garbage collector. When
internal/dbfilefound a checkout path naming a new inode, it dropped its only reference to the descriptor for the old one, and Go's*os.Filefinalizer closed it at some later GC. Closing any descriptor releases every POSIX lock this process holds on that inode, so a capture engine orquiescestill on the old file could lose its SHARED lock with no error. The old descriptor is now kept, and closed only explicitly, once nothing pins its inode. - A long-lived process kept a full copy of every deleted checkout on
disk. Each
destroyor reap of a checked-out branch, and on a cloning filesystem each by-chain cache build and prune, stranded a descriptor on a file nothing would ever ask for again, holding the unlinked file's disk for the life of the daemon oroffshoot mcpprocess. Those descriptors are now closed as soon as nothing pins them: right after the destroy, reap, build or prune that strands them in the same process, and by a full sweep on every janitor tick and everyoffshoot mcp-reap-everytick, which also finds checkouts another process removed. A process run with-reap-every 0keeps descriptors another process stranded until it restarts. - On Linux, refreshing a checkout's shadow could drop a running
session's SQLite lock. After a
checkout, a rollback, promote or compact refresh, and a session's clean close, offshoot clones the checkout to its.shadowfor the next segment checkpoint. On Linux that clone opened and closed the checkout, even on ext4 where the clone then fails, and closing any descriptor releases every POSIX lock the process holds on the file. If another request had opened a session on that checkout by then (anopenright after acloseof the same branch, or racing an at-restcheckout), its capture engine lost its SHARED lock with no error, so a foreign writer's close-time checkpoint could later fold and unlink the WAL under it. The clone now goes through offshoot's own pinned descriptor and opens nothing on the checkout. macOS was not affected: clonefile(2) takes paths. - A checkout removed while
quiesceor a capture engine was opening it came back as an empty database. SQLite creates a missing file on open, so a checkout removed between a caller's check and the open (adestroyor reap racingbranches, or an engine started on a checkout that had just been removed) left an empty database at its path. Those opens now never create the file. Acheckout,checkpointordestroy(and a sessionopen, through its checkout) that loses this race to another operation fails withops: <op> <db>@<branch> lost a race with another operation (retry): ops: the checkout was replaced or removed while it was being opened, and none calls such a checkout "in use". A rollback, promote or compact whose post-repoint refresh loses it says the operation stands and namesoffshoot checkout, with no retry hint: retrying a promote or rollback would replace its safety fork with the already-repointed head. - A rollback, promote or compact that found its checkout busy said to
retry it. A busy checkout does not stop the repoint, so the refresh
after it fails whenever an app holds a read transaction on the checkout
while a write lands, and that error ended
close connections and retry. Retrying a promote or rollback replaced its safety fork (<target>-pre-promote,<branch>-pre-rollback) with the already-repointed head, which can leave the original head on no branch for GC to reclaim. The error now says the operation stands and to close the checkout's connections, then runoffshoot checkout <db>@<branch>.checkoutandcheckpointstill sayclose connections and retry, anddestroystill says to close them before destroying. create --froma path containing?imported an empty database. The import opened<path>?_busy_timeout=5000, and go-sqlite3 cut the name at the first?:create imp --from '/dir/q?x.db'created an empty/dir/qand imported that, with no error. In-process SQLite opens of an import source or a checkout now escape?,#and%, so the path names exactly that file.quiesceand the capture engine get the same fix for a store root containing those characters.- Reopening a branch while its session was still closing shared the
closing session's lease.
session closefreed the branch in the daemon before the close had finished, so anopenof the same branch from another client — a client killed mid-close that reconnected, or a supervisor reopening whateverstatusstopped listing — started a second session under the same lease holder and epoch. The closing session's release then cleared that lease, so the new session was fenced at its next renewal, and until then the branch looked unleased: a rollback, promote, compact or at-rest checkpoint could run under it. A closing session now keeps its branch until the close has released the lease. Anopenwaits up to 15 s for it and then takes a fresh epoch, or fails withdaemon: <db>@<branch> is still closing; retry. If the close failed to release the lease, the reopen is refused instead of renewing that lease in place: each daemon session now has a lease holder of its own (see this release's Changed and Added entries, below), so a reopen can never be granted under a closed session's holder and epoch. - A close retried after its session had closed could close another
client's session. A
closenamed onlydb@branch. A client that retried its close after the branch had been reopened closed the reopened session: its lease was released under it, its next flush failed withis not open, and its writes since its last flush were never shipped. The usual ways to get there are a client that gave up waiting for its close's answer, and a close that raised because its lease release failed (the session is closed all the same). The daemon'sopennow returns asession_id, 128 random bits minted for that session, and aclosethat sends it back closes only that session; once that session has closed, it fails withdaemon: session <id> on <db>@<branch> is not open. The Python and TypeScript SDKs'Session.close()send it.offshoot session close, and any client that sends nosession_id, still closes whatever session is open on the branch. offshoot session shutdowncould exit before every lease was released. The process exited as soon as the daemon stopped listening, while sessions were still closing; a close another client had started was not waited for at all, and a signal arriving during ashutdownended the process at once. The daemon now exits only after every session has closed and released its lease, bounded by the same 30 s asSIGINT/SIGTERM. The command still returns as soon as the daemon accepts it, and the daemon removes its socket at once and leaves the path alone after that, so a newserveon the same socket can start while the old process finishes; wait for that process to exit if every lease must be released first.- A slow close could let the session's lease lapse. Closing stopped renewing the lease before the capture engine's shutdown, the checkout hash, the sidecar stamp and the shadow refresh, which at default settings left 20 to 30 s. A close that ran longer let another writer take the branch while the old engine still owned the checkout. The lease is now renewed until the moment it is released.
- One failed lease release on close left the branch leased until the
lease expired. A release that lost its compare-and-swap to a concurrent
touchorprotect, or hit a transient store error, was not retried, so other writers saw the branch held for up to 30 s. Close now retries it up to three times; a release that landed but reported an error counts as released. destroy --forceon a local store failed when the lease holder renewed between the destroy's claim and its delete. A lease renewal, a daemon session's or (new in this release) an at-rest checkpoint's, writes over a destroy's claim and leaves it set, which moved the etag the local store's conditional delete compares against: a destroy whose checkout quiesce spanned a renewal tick unwound its claim and failed withcompare-and-swap conflict, and the session carried on, or the checkpoint it was meant to stop went on to commit. The delete now re-reads the ref and deletes again, up to 4 attempts in all, while the ref is still that destroy's claim with only the lease expiry moved; the holder's next renewal finds the branch gone, which ends the session or fails the checkpoint without committing. An unforceddestroy, which goes ahead only on a lapsed lease, is refused withhas a live leaseif the holder renewed it in that window. Any other write in between (a flush, atouch, a release) still fails the destroy, now withdestroy lost a race ... (retry). On S3 the delete was already unconditional.- A destroy that failed after its claim cleared another destroy's
claim, which on S3 could delete a branch under a fresh lease. Two
destroys of one branch (a second
destroy --force, or the reaper's on a lapsed lease) could both claim it, the second writing over the first. When either then failed (a busy checkout, a failed or lost delete), it cleared whatever claim was on the ref, so a session or a checkpoint could take the branch while the other destroy was still on its way to its delete; on S3, whose delete is unconditional, that destroy then deleted the branch under the fresh lease and reported success. A destroy now clears only its own claim, retrying past the holder's lease renewals, and a destroy of a branch another destroy claimed less than 30 s ago is refused, forced or not, withis already being destroyed(retryable;errors.Is(err, store.ErrDeleting)) and writes nothing. The reaper leaves such a branch to that destroy and writes nothing to its ref, even when it had claimed the branch for reaping first; if that destroy then fails, the reap claim stays until the next reap pass, which reaps the branch or clears the claim. A claim 30 s old, one stamped more than a minute ahead of the clock (written by a host whose clock runs ahead; the janitor clears these too), or one whose timestamp cannot be read, is still taken over. On S3, a destroy whose claim write landed but reported failure (the SDK's retry answering 412 to its own first attempt, or a timeout) recognises the claim as its own and goes on under it, deleting the branch if the claim landed in time (below), rather than reporting a lost race whose retry that claim would refuse; when the ref shows nothing landed yet (a 409 or a timeout, with the write possibly still in flight), it re-reads the ref for about 2 s before reporting the failure, and then says the claim may still land. A local store's failed write never lands, and is reported at once. A destroy sends its delete only while its claim has more than 10 s of its 30 s left, judged after the checkout quiesce: one held up past that (its claim write slowed by timeouts and SDK retries, whether the write then succeeded or landed after reporting failure, or a slow quiesce) sends no delete, removes its claim and fails withwas held up too long to delete under its claim(retryable;errors.Is(err, store.ErrCAS)), rather than send an S3 delete that could land after the janitor cleared the claim and a new session took the branch. On S3 a destroy whose delete request is itself held up for over 30 s after its claim can still delete a branch a new session has taken; see limitations. checkpoint --forceon a branch with a live session wrote a second writer's head under the session's epoch. The docs said--forcefenced the session; it did not. It wrote the head and a checkpoint entry under the session's own epoch and left the session's lease in place, so the session went on flushing on top of an object it had not written, a shape the chain resolver assumes cannot happen.--forceno longer overrides a live lease oncheckpoint: the checkpoint is refused, the session is untouched, and the message says to close the session and retry, or, if the holder is gone (a killed daemon, alease acquirenobody will release), to free the lease withoffshoot lease release <db>@<branch>. A lease left by asession closewhose release failed is not one to free that way: it lapses at its expiry, and once the branch is reopened it is the new session's (see the close fixes above).--forcestill checkpoints a detached checkout.- Two at-rest checkpoints on one branch could still leave the head on the wrong content. v0.2.13 made two racers survivable by checking, after the winner's ref write, whether the store still held its content. Two windows were left: a third racer overwriting the shared object key after that check, and a losing snapshot landing beside a winning segment from a writer that then died before cleaning up. An at-rest checkpoint now takes the branch lease before it writes its object and releases it in the ref write that advances the head. The acquire bumps the epoch, so the object key is private to the call and nothing else writes it; an object an earlier writer left at the same txid sits under an older epoch, never anchors the head, and is reclaimed by GC.
- Two daemons in look-alike containers renewed each other's leases and
shared an epoch, a split brain; a daemon restarted in such a container
picked up its dead predecessor's lease the same way. Every daemon
session shared one holder, bare
<host>/<pid>, andAcquireLeasetreats the same holder on a live lease as a self-renew under the same epoch. In containers, offshoot runs as PID 1 and two containers on host networking share a hostname, so two live daemons on one store held every lease as the same<host>/1: each one's open renewed the other's live lease, and both wrote under one epoch with nothing to fence either. A daemon that crashed and restarted under the same hostname did the same to its predecessor's lease. Each session now mints its own holder (see Changed, below), so no open can ever match another session's lease, live or orphaned: it is refused instead. The daemon records the exact holder of any of its own sessions whose lease release failed, and only a lease held by exactly that holder is called "held by an earlier session of this daemon", with its lapse time andoffshoot lease release <db>@<branch> --holder <holder>to free it now. Any othersession:holder, a twin daemon's included even though it shares thesession:<host>/<pid>/prefix, gets the conditional advice the checkpoint refusal gives: close that session, or, if its daemon has exited, free it by its exact--holder.
Changed
- Every in-process SQLite open of a checkout pins the file first. The
capture engine,
quiesce(behindcheckout,checkpoint,branches, rollback, promote and compact,fork's uncheckpointed-changes warning,destroyand reap) andcreate's read of an imported file pin the file and its path before opening it and release them only after the connection closes. Until then no descriptor offshoot caches under that path is closed, whatever file the path names by then. If the path is renamed over or removed between the pin and the open, the open is refused rather than left working on a file that is no longer the checkout: the operation fails with a lost-race error (see Fixed), andcreate --fromfails the import. A session's capture engine that loses this race still opens and then fails at once: the branch shows stateerrorwithcapture stopped: dbfile: path was replaced while a SQLite open held it, and closing and reopening the session recovers. A test fails on any newsql.Openin the tree until it is classified, and on a pinned one whose release could run before its connection closes. - A closing session holds its branch, and the daemon says so. While a
session is closing,
branchesreports the new stateclosing(it outrankserror),statuskeeps listing the session withstate: "closing", and a duplicateclosewaits for the first and returns its result instead of sayingis not open. Flush refuses withdaemon: <db>@<branch> is closing; fork, promote, rollback, compact, checkout and destroy refuse withdaemon: <db>@<branch> is closing; retry when the close finishes. MCP'soffshoot_checkoutandoffshoot_checkpointrefuse with "closing; retry in a few seconds" instead of falling back to at rest, and rollback, promote and destroy say to retry instead of to close the session. session_closedfires once the branch is free. It used to fire from inside the close; it now fires after the daemon has let go of the branch, so acting on it never meets aclosingrefusal, and always before thesession_openedof a reopen of the same branch. It andfencedcarry the session'sholderandepoch.holderis now unique per session (see the lease-holder entry further down in this section), so it never repeats across sessions, including a reopen after a close whose lease release failed: that reopen is refused rather than reopening under the closed session's pair (see Fixed, above). Onlyepochcan repeat it, since rollback, promote, compact, and destroying and recreating the branch all restart it./healthzandoffshoot_sessions_opencount open sessions only; a closing session is not counted, and reports nooffshoot_capture_lag_bytesoroffshoot_durable_age_seconds.- A second
offshoot checkpointon a branch is refused while the first runs, withhas a live lease held by "checkpoint:<host>/<pid>/<nonce>" ... (another checkpoint is in progress)and the advice to retry when it finishes, instead of racing it. So is an unforcedrollback,promote --onto,compactordestroy, each of which also says to retry when the checkpoint finishes, in a few seconds (a checkpoint's lease lapses on its own 30 s after its process dies, so none suggests closing a session orlease release); forced, a repoint clears the checkpoint's lease, and the checkpoint fails without committing and deletes its object. Every at-rest live-lease refusal now satisfieserrors.Is(err, store.ErrLeaseHeld). A session open on the branch (the daemon'sopen, both SDKs) fails at once withstore: branch lease is held by "checkpoint:..."; it does not wait, as it does for a session that is closing, so retry it once the checkpoint finishes. - Live-lease refusals no longer offer a
--forcethe caller does not have. MCP'soffshoot_rollback(which has noforce), andoffshoot_promoteandoffshoot_destroyon a server without-allow-force, drop ops'--forceadvice: under a checkpoint's lease they say to retry, as they do for a daemon session that is closing (above), and under any other leaseoffshoot_promotenow names the human's CLI command, asoffshoot_destroyalready did. The daemon'srollbackandcompactops (and so both SDKs'rollbackandcompact), which take no force, drop it too.offshoot_checkpoint's refusal of a detached checkout namesoffshoot_checkoutto refresh it, and the human'soffshoot checkpoint ... --forcefor edits that must win, instead of a--forcethe tool does not accept; Go callers test it witherrors.Is(err, ops.ErrDetachedCheckout). The plugin skill tells agents to retry a refusal that names acheckpoint:holder. - A checkpoint can now fail after it has started, in two new ways, and
can end without knowing whether it committed. If its lease ends while
it runs (a forced repoint,
offshoot lease release,destroy --force, or a reclaim after the process stalled past the 30 s TTL), it reports that itdid not commit, withits lease ended while it ran,store: branch lease lostand the branch's current state, orstore: branch is being deletedwhen its head write finds a forced destroy's claim, and deletes the object it uploaded; it leaves that claim alone, so the destroy goes through. Iftouch,protector a TTL change win the head write's compare-and-swap three times running, it reportsthe head write lost 3 compare-and-swaps to concurrent ref writes (retry; ...)and releases the lease. Neither moves the head. Once a head write has been sent, the checkpoint deletes its object only when the ref proves no write of its own landed and none still can. A store can apply a write after the client gave up on it: a timeout or a 5xx gets no verdict, and on S3 a 409 conflict, reported as a lost compare-and-swap, can be the SDK's retry colliding with its own first attempt still in flight. So the object is kept for GC after three lost compare-and-swaps; and when the head write failed and the ref then cannot show whether it landed, because the store may still apply it, the branch has moved on, the ref cannot be read, or a destroy or reap claimed the branch afterwards (an abandoned claim is undone to the very bytes the write was sent against), the checkpoint reports that itmay have committedand keeps its object: deleting it could leave the head, or a fork taken in between, naming an object that is gone. A retry under the same name is refused as already existing if it did commit. - An at-rest checkpoint makes two more ref requests and, for a
segment, two fewer
LISTs. It acquires the lease with a read and a write (its refusals, a live lease, a taken name, a missing or detached checkout, run on that read, before the write, so a refused checkpoint writes nothing), renews it everyLeaseTTL/3(10 s by default; one read and one write each) while it uploads, and commits with a read and a write that also releases the lease, where it used to read and write the ref once each. The probes for another checkpoint's snapshot at its txid, oneLISTbefore the write and one after, are gone. The head write makes up to three attempts when atouch,protector TTL change wins the compare-and-swap, and the lease acquire and the head write each recognise their own write when only the response was lost, as when the S3 SDK retries a write that had landed; an acquire recognised that way is adopted without another write. - An at-rest checkpoint is slower. The lease acquire is a second ref
write per checkpoint, and it has to be durable to fence anything. On a
local store that is one more fsync (
F_FULLFSYNCon macOS), and it is nearly all of the cost, since a ref read there takes about 0.1 ms. BranchBench'ssimulationworkflow on a local macOS store, alternating builds before and after this change, measured checkpoint p50 at 45.2 → 48.5 ms one at a time and 114.8 → 126.1 ms eight at a time (p99 170.5 → 187.6 ms), with the full workflow's wall time unchanged within noise; on a heavily loaded machine, where concurrent fsyncs queue, the eight-way gap measured 20 to 40 ms. On S3, where every request is a round trip, a snapshot checkpoint makes one moreGETand one more conditionalPUTof the ref than before, in sequence; a segment checkpoint makes the same number of requests as before, one moreGETandPUTin place of twoLISTs. The numbers are in benchmarks. - On a local store, resolving a branch's chain gets slower with every
at-rest checkpoint the branch's lineage has taken. Each checkpoint
writes under the epoch its acquire minted, so it leaves one more
data/<lineage>/<epoch>/directory, and a local listing of the lineage reads every one: a segment checkpoint's own chain resolve, and everycheckout,forkand materialize of the branch or of a fork resolving through it. Measured on the machine in benchmarks (BenchmarkChainAfterCheckpoints), resolving the head's chain took 1.6 ms after 100 checkpoints and 16 ms after 1,000, where it took 0.2 ms and 1.2 ms with every checkpoint in one directory before this change.compact(or a rollback or promote) starts a fresh lineage and resets it; a daemon session, which writes all its flushes under one epoch, adds one directory per session, as before. S3 lists a lineage flat and is unaffected. - Upgrade every
offshootbinary that touches a store together: an older binary'scheckpoint --forcestill goes past a live lease and writes under whatever epoch the ref carries, including one a newer checkpoint's lease holds, at the very key that checkpoint writes. A newer checkpoint that finds such a head under its own epoch keeps the object rather than delete what the head may name, and says the branch moved under its lease; but if a renewal moves the ref between the older binary's read and its head write, the older binary's own cleanup deletes the shared object and the newer checkpoint can then commit a head that names it, after whichcheckout,checkout --atandforkof the branch fail until it is rolled back to an earlier checkpoint. - A daemon session's lease holder is now unique per session, not one
bare
<host>/<pid>shared by every session (and every reopen) on the same host and pid. Eachsession.Openmintssession:<host>/<pid>/<8 hex nonce>;offshoot lease list,offshoot statusand the daemon'sbranchesop show it in place of the old<host>/<pid>. A script that matches a lease holder against"<host>/<pid>"to recognize a daemon session needs to match thesession:prefix instead. Holders now come in three shapes: a daemon session'ssession:<host>/<pid>/<8 hex>, an at-rest checkpoint'scheckpoint:<host>/<pid>/<8 hex>(unchanged), and the bare<host>/<pid>that onlyoffshoot lease acquirestill claims. A reopen after a close whose lease release failed is refused for up to the lease TTL instead of renewing that lease in place under the closed session's holder and epoch (see Fixed, above). - A
session closewhose lease release failed says what to do about the lease it left. The error was the store's bare release failure; it is nowsession <db>@<branch> closed but its lease release failed: <cause>; the lease (holder <holder>) lapses at <time>, or 'offshoot lease release <db>@<branch> --holder <holder>' frees it now, naming the closed session's own holder, which no session holds any more. An open that fails after its acquire and then cannot release says the same, and an open's checksum-baseline failure now retries that release like a failed checkout does. - Every lease expiry a refusal prints is rounded up to the second (RFC3339, UTC), so the time it names is never before the lease lapses; refusals used to print the stored nanosecond timestamp, or truncate it.
offshoot lease release --holder Hrefuses in its own words:lease release: <db>@<branch> is held by <current>, not H; nothing released, orlease release: <db>@<branch> is not leased; nothing released, instead of the store'sbranch lease lost: ... is held by "", not .... An emptyH(--holder=,--holder "") is a usage error rather than a release attempt.
Added
offshoot serve -fd-budget N(default64,0unlimited) bounds the checkout descriptors the daemon keeps cached. Each janitor tick closes the least recently used untilNremain, never one an open session or an in-flight read pins.- Four metrics:
offshoot_dbfile_descriptors,offshoot_dbfile_pins,offshoot_dbfile_stranded_pinned(stranded descriptors kept open by a pin on their file or a connection's hold on their path; non-zero across janitor passes means a pin leak, which the janitor also logs, naming the paths) andoffshoot_dbfile_evicted_total{reason="stranded"|"budget"}. - The daemon
statusop reportsdbfile_descriptors. PythonClient.daemon_status()and TypeScriptclient.daemonStatus()return the open sessions plus that count (None/undefinedfrom an older daemon that does not report it);status()is unchanged. session_idon the daemonopenresponse andcloserequest (see Fixed). Both are additive: a close without one closes by branch as before, and the SDKs send none when an older daemon'sopenreturns none.SessionInfo.state(openorclosing) in the daemonstatusop.offshoot session statusprintsstate=, TypeScriptSessionInfohasstate?, and Python'sstatus()dicts carry it. An older daemon sends none, which means open.- While an at-rest checkpoint runs, the branch shows as leased:
offshoot statusreportsstate=active, andoffshoot lease list, the daemon'sbranchesop and both SDKs'Branch.lease_holdernamecheckpoint:<host>/<pid>/<nonce>. The per-call nonce keeps two checkpoints in one process (an MCP server, a script) from sharing an epoch. offshoot lease release <db>[@branch] --holder Hreleases a lease only if its current holder is exactlyH— a compare-and-swap, not a blind release: a mismatch (including an unleased ref, which never equals a non-emptyH) refuses withlease release: <db>@<branch> is held by <current>, not H; nothing released(oris not leased) and writes nothing. This is the safe way to free a holder a refusal already named (asession:orcheckpoint:holder, unique to one session or checkpoint), since the exact match can never free a different session or checkpoint the same host and pid later holds.offshoot lease release, with or without--holder, now printsreleased <db>@<branch> held by <holder>on success.
[0.2.16] - 2026-10-03
Small hardening: one checkpoint-stamp fix for coarse-mtime filesystems, cheaper post-checkpoint verification on S3-compatible providers that reformat etags, and the LTX decoder pinned by test at the largest page size and the LZ4 worst case.
Fixed
- A
checkpointcould stamp its checkout trusted for content it never encoded. The stamp trusted the encode's checksum whenever the checkout's fingerprint (size, mtime, change counter) still matched the one taken right after quiesce. Checkouts are WAL mode, where SQLite can leave the change counter untouched across commits, so size and mtime alone carried that match; on a filesystem with 1 s mtime resolution a foreign same-size write landing in the same tick as the quiesce left the fingerprint identical with the content changed, and the sidecar then read "clean" (and the shadow was refreshed) for bytes the store does not hold. The stamp now applies the same 1 s racily-clean margin thatcheckoutalready applies to its fingerprint fast path: when the encode fingerprint's mtime is younger than the margin, the live database is checksummed instead, which costs one pass over the file and cannot be fooled by the tick.
Changed
- Etags are normalized before the post-checkpoint compare. After
winning its ref CAS, a checkpoint
Heads the object it just wrote and trusts it with that one request when the etag matches the one its create-only put returned; otherwise itGets the whole object to read the trailer. An S3-compatible provider (MinIO, RustFS, R2, B2, Ceph RGW) may return the same object's etag quoted onPUTand bare onHEAD, with a weakW/prefix, or with the hex in a different case, and every such checkpoint paid a needless fullGET. Both etags now go throughstore.NormalizeETag(trim, dropW/, strip one pair of quotes, lower-case) before the compare. What is sent to the provider asIf-Matchis unchanged, and so are the local backend's own etags.
Added
- A 64 KiB-page ltx v0.5.1 fixture (
internal/ltxio/testdata/ltx-v0.5.1-64k) beside the 4 KiB one. Its data page is random bytes LZ4 cannot shrink, so the page frame carries a stored block of exactly one page: the largest frame the pinned v0.5.1 shape can hold and the edge of the decode guard's block limit.TestDecodesLTXv051FrameFormatnow decodes both sets and asserts the 64 KiB one really is incompressible; both seed the fuzz targets.
[0.2.15] - 2026-10-02
Post-launch hardening: one concurrency fix in gc, a token-file flag for
serve, and two gaps in what the test suite proves closed (the Linux
reflink path now runs nightly on XFS; the capture engine is now
SIGKILLed, not just bounced, in the torture harness).
Fixed
gcpersisted its tombstone list with an unconditional write at the end of a pass, so a concurrent GC or janitor pass's prunes and additions could be clobbered, resurrecting an already-expired stone for a deleted key. The final write is now a compare-and-swap of exactly the pass's own removals, retried against the current list on a conflict.
Added
- A nightly
linux-reflinkCI job runs the copy-on-write tests on a loopback XFS filesystem with reflinks, so the LinuxFICLONEclone path (btrfs, XFS) is exercised and not only APFS. TestTortureCapturerKill: a second torture harness that runs the capture engine in a child process,SIGKILLs it mid-traffic every round, restarts it on the same state and replica, and requires the replica to equal the source after a graceful drain. Part ofmake test-tortureand the nightly torture job. The docs no longer say the capturer's crash is untested.offshoot serve -token-file PATHreads the HTTP bearer token from a file (whitespace trimmed), so it never appears inpsthe way-tokendoes; precedence is-token,-token-file,OFFSHOOT_TOKEN. The recommended form for a non-loopback bind.
[0.2.14] - 2026-10-02
Launch hardening. Every change below came out of a read-everything, run-everything review ahead of the first public announcement; the correctness items were reproduced before they were fixed.
Fixed
- An at-rest
checkpointof a detached checkout silently reverted a promote, rollback or compact. When a repoint could not refresh the checkout (it was busy, or the process died between the ref update and the refresh), the nextcheckpointsnapshotted the old content onto the new lineage. It is now refused, namingcheckout,exportand--forceas the ways out. - CLI
rollback,promote --onto,compactandcheckpointignored a live lease. A rollback of a branch a daemon session had open fenced the session and discarded its unflushed writes. All four now refuse a branch with a live lease unless--force(the ruledestroyalready applied), and refuse a branch mid-destroy or mid-reap outright. A lease on promote's source never blocks. checkoutwarns before replacing a stale checkout that holds un-checkpointed edits, by comparing the file with the hash its sidecar recorded; a checkout with no readable sidecar warns that it cannot be verified.- The MCP server told an agent to "use --force" on a protected branch and
then refused the forced retry. Every such refusal now says the server
does not honor force, not to retry, and the exact CLI command to ask the
human for.
offshoot_checkpointon a never-checked-out fork, a fork onto an existing name, and any tool on a missing database return the next tool call to make. MCPinitializereports the binary's version. offshoot --helpfailed with "store: not found" in a directory with no store, and afterinit,create --helpcreated a database named--helpwhilegc --helpran GC.help,--help,-h,help <cmd>,<cmd> --helpand--versionnever open the store; names cannot start with-.- "store: not found" and "compare-and-swap conflict: key exists" leaked to
users. Errors now say which database, branch or checkpoint is missing,
list a branch's own checkpoints when one is not found (and explain that
checkpoints belong to one branch), and say "already exists" for a
duplicate init, create or fork. S3 misconfiguration fails up front with
the variable to set, and the probe-cleanup warning no longer fires when
the probe's own put failed.
--ttland--graceerrors name the flag and value. - The promptfoo recipe used
require()on an ESM-only package and could not load; it is ESM now, and the TypeScript package also exports adefaultcondition. The pytest plugin's runtime hint no longer suggests a PyPI install that does not exist.examples/eval-pass-kfails with one line on Python < 3.10 instead of a traceback.
Changed
- Durability discipline on local stores. Every rename into place (store
objects and refs, materialized checkouts, capture state) is followed by a
directory
fsync(2)(plain, not macOS'sF_FULLFSYNC), and.sumsidecars are written atomically but not synced (they are a cache), so a flushed state is designed to survive power loss and not only process death. This is designed for, not torture-tested; the docs say so. create --fromimports throughVACUUM INTOunder one read transaction and normalizes the copy to WAL mode, instead of copying the bytes of a possibly live file.- The local store streams a put before taking its per-key lock and holds the lock only around compare and rename, so a multi-GB snapshot can no longer outlive the lock's 30 s stale-break horizon mid-write.
- Local store and checkout directories are created
0700(objects were already0600); existing directories keep their mode. createanddestroyconfirm what they did;-store/--store,--ttl/-ttland--flag=valueall parse; TTL remaining is shown rounded to seconds;offshoot_listrows carryttlandexpires_at.- Python
Client.fork(at=)and TypeScriptForkOptions.atalias the existingfrom_checkpoint/from; bothSessiontypes gaincheckpoint()as an alias offlush(). - Documentation: README rebuilt as a landing page; the "kill -9 durable"
wording is replaced everywhere by what is tested (a SIGKILLed writer
never diverges capture) with the flush window beside it; limitations no
longer claims that two nodes sharing a bucket cannot corrupt each other;
a "Recovering data without offshoot" section documents the store layout
and the upstream
ltxrebuild path; SECURITY.md states the threat model as enforced; internal plan documents and an unposted announcement draft are no longer in the public tree.
Security
- The release job holding the signing identity ran an unpinned
go-licenses@latestbefore signing the tarballs. License generation now runs in a read-only job, pinned, and reaches the signing job as an artifact. Releases are created as drafts and published only after the Docker image is pushed, signed and attested;:latestmoves in that same final step;workflow_dispatchsmoke builds create no public release. A version-surface check refuses a tag whose manifests still name the previous version. - Names read back from a store listing are re-validated before any path or reaper sees them; one unix-socket connection may send at most 16 MiB of requests; a plain-HTTP S3 endpoint on a non-loopback host prints a warning. Dependabot covers the TypeScript lockfile and the hash-pinned Python CI locks.
[0.2.13] - 2026-10-02
Hardening on top of v0.2.12's pointers-and-deltas release: concurrent
at-rest checkpoints detected and recovered, cache-backed refreshes for
rollback/promote/compact, a hardened LTX decoder found by new fuzz
targets, hash- and digest-pinned CI, CodeQL, a local-store Head that no
longer re-reads its object, and an HTTP shutdown response that can no
longer be lost to the shutdown it triggers.
Added
offshoot_checkpoint_overwrite_detected_total. Counts at-rest checkpoints that committed but found the store's head may not be their own content (see Changed). Registered at0; a stat panel on the Grafana dashboard.- Fuzz targets. Go native fuzz targets for store names, LTX snapshot
decoding, segment application, the checkout sidecar and the daemon
protocol;
make fuzzruns them all, and a nightlyfuzzjob runs each for 60 s and uploads any crasher. Store.Head. An optionalstore.Headerbackend capability (oneHeadObjecton S3) returning an object's etag and size, with aGetfallback for backends without it.- CodeQL static analysis on pushes to
main, pull requests and a weekly schedule. - Hash-pinned CI installs. Every third-party Python package CI installs
comes from a hash-pinned lock under
requirements/.
Changed
- Local store
Headno longer re-reads the object. Every object the local backend writes carries its content etag as an extended attribute (user.offshoot.etag), set on the temp file before the rename so it lands with the object.Head, which every at-rest checkpoint calls on the object it just committed, answers from a stat and that attribute instead of a second full read and hash of the snapshot; objects written by earlier versions, and filesystems without user xattrs, fall back to hashing as before. - Rollback, promote and compact refresh the checkout through the by-chain
cache, like
checkout: an identical or prefix-sharing file is cloned instead of decoded from the store. - Concurrent at-rest checkpoints on one branch are detected and recovered. After winning the ref CAS, a checkpoint verifies that the store resolves the head to its own content: its object was not replaced by a same-kind racer, and (for a segment) no racer's snapshot sits at the same txid. It also checks that the checkout did not change between its encode and its stamp. When the head and the checkout may differ, the checkout reads as modified, no checksum is trusted, and the next checkpoint writes a full snapshot. Two concurrent at-rest checkpoints are still not a supported pattern; see limitations.
- Flush CAS retry budget. A flush that loses its ref CAS to its own lease heartbeat (or to a metadata write) keeps retrying for up to a lease TTL instead of a fixed count, with a short jittered pause between retries.
internal/reflinkreplacesinternal/ops/reflink(package path only).
Stricter decoding
An LTX object is now refused, before it is decoded, when it has: an LZ4 frame shape other than the one ltx v0.5.1 wrote; a legacy or skippable LZ4 magic; a page frame larger than the LZ4 bound for one page; an illegal page size; or a segment that grows the database beyond the pages it carries. The pinned upstream decoder would accept some of these, at the cost of allocations of up to 4 GiB or zero-filled pages. No object this project ever wrote is affected.
Fixed
- A
shutdownrequest over HTTP could getEOFinstead of its response. The handler wrote the response and then triggered shutdown, but Go'snet/httponly hands a handler's buffered response to the connection when the handler returns, so when the shutdown goroutine closed the server first the client saw EOF before a single byte. The handler now flushes before it triggers shutdown. Seen as a recurringinternal/daemonCI failure on loaded Linux runners. - Three timing-sensitive tests (
internal/capture's resume Apply-count test andinternal/daemon's auto-flush and status tests) raced the engine's startup rebase, split one write into two autocommit transactions that an auto-flush could separate, or observed a transient injected failure through a window shorter than a descheduled test goroutine; each now waits on the state it asserts about.
[0.2.12] - 2026-09-30
Pointers and deltas, not copies: the operations that used to copy a whole database locally or in the store now clone, diff or point instead, where the filesystem and the chain allow it. Measured in docs/benchmarks.md (the "What changed in v0.2.12" paragraphs).
Added
- Segment checkpoints at rest.
offshoot checkpoint(no daemon) now writes an LTX segment of just the changed pages when it can, diffing the checkout against a reflinked<checkout>.shadowtaken at the last checkout or checkpoint. It falls back to a full snapshot when there is no usable shadow, when the branch's chain has reached the snapshot cadence (SnapshotEvery, default 16), or when half or more of the pages changed (databases under 64 pages skip that test). The shadow's rolling checksum is verified against the recorded head before a segment is derived from it. offshoot checkpoint --snapshotforces a full snapshot.- Checkpoint
kind. The CLI prints what was written —checkpoint "seeded" at txid 2 (segment, 2 pages, 0.4 KiB)orcheckpoint "v1" at txid 3 (snapshot, 0.0 MiB)— the MCPoffshoot_checkpointat-rest result carrieskind, and each checkpoint entry in a ref recordskind("snapshot"/"segment"; an additive JSON field, empty on older entries). A checkpoint that loses a concurrent compare-and-swap uses the winner's recorded kind to decide whether to delete its own object. - Shared rollback and promote.
rollbackto a kept checkpoint andpromoterepoint the branch through a base pointer — the same share path fork uses (newLineageAt) — instead of copying the snapshot. The fork-time depth floor still forces a copy.Store.CollapseBaseskips pass-through hops when writing the pointer, so repeated rollbacks to one checkpoint keep the base spine one hop deep. --materializeonoffshoot rollbackandoffshoot promote(daemon request fieldmaterialize; Pythonmaterialize=False; TypeScriptmaterialize?: boolean) keeps the old copying behaviour.sharedon the daemon'srollback/promoteresponses, and(shared)/(materialized)in the CLI output.- Metrics
offshoot_rollback_total{mode}andoffshoot_promote_total{mode}(shared/materialized, both pre-registered at 0). - CI: a
cow-pathsjob runsinternal/ops,internal/ltxioandinternal/ops/reflink, plusinternal/session's two clone tests, onmacos-lateston every push and PR (20-minute timeout), so the clone, shadow and segment paths — which skip on the ubuntu runner's ext4 — are tested before merge, not only weekly.
Changed
- Checkouts are clones.
checkout(and every internal checkout refresh) andcheckout --atbuild an immutable, content-addressed entry undercheckouts-ro/<db>/~by-chain/<chainID>.dband clone the checkout from it (APFSclonefile, LinuxFICLONE); a chain that extends a cached one clones the prefix and applies only the remaining segments. A second checkout of the same state — a fresh fork, a re-checkout after the file was deleted — is a clone instead of a decode. - By-chain entries count toward
-ro-cache-budgetat their logical size (they share blocks with the checkouts cloned from them, so this over-states real disk use), and acheckout --atmiss now leaves two entries.offshoot status'sro-cache:line andoffshoot_ro_cache_bytesinclude them. Janitor eviction lines andevictedevents for them carry branch~by-chainand the chain ID in the checkpoint position.rm -rf checkouts-rostays safe. - By-chain entries are capped at 64 per database by default
(
ops.DefaultByChainMaxEntries), pruned least-recently-used first after each checkout that builds one — no daemon or-ro-cache-budgetneeded, so a CLI-only user no longer keeps one full copy per chain forever.offshoot destroydoes not remove by-chain entries (another branch may share them); the bound ages them out. Entries are0400and thecheckouts-rodirectories0700, like the store's own0600objects; a checkout's.shadowis0600. - O(1) clean check. The
.sumsidecar gainedchain_id,size,mtime_ns,change_counter,stamped_nsandshadow. A repeatcheckoutof an unchanged checkout is proven clean from identity, size, mtime and SQLite's change counter without hashing, provided the file's mtime is more than 1 s older than the stamp (git's racily-clean rule); in WAL mode the change counter is not treated as evidence. Anything else hashes as before and re-stamps. Old sidecars hash once.offshoot statusstill always hashes. offshoot rollbackprints a second line. Line 1 is still the checkout path; line 2 is nowrolled back <db>@<branch> to "<cp>" (shared|materialized), ahead of the existing safety-fork line. A script that captured the whole output as the path must take the first line (offshoot rollback ... | head -1).offshoot promote's line gains a trailing(shared)/(materialized).statusstorage class. Shared rollbacks and promotes reportstorage=shared; they pin the lineage they point into until the branch diverges past it or is compacted (offshoot compact, which still always copies).offshoot destroyalso removes the checkout's.shadow(and a leftover.shadow.tmp).
Behaviour on non-reflink filesystems
Everything above that clones — by-chain entries, checkout clones, the
checkpoint shadow — needs a filesystem that supports reflinks (APFS,
btrfs, XFS with reflink=1). On one that doesn't (ext4, tmpfs, most
network filesystems), offshoot detects it and falls back without error: no by-chain entries, no shadows, checkouts
materialize from the store as before, and every at-rest checkpoint is a
full snapshot. The O(1) clean check, shared rollback/promote and the
kind field do not depend on the filesystem and apply everywhere.
Fixed
- Branch operations on a local store no longer slow down as checkouts
accumulate. Listing a lineage's objects walked the whole store
directory, which on a local store also holds every checkout,
.shadowand by-chain entry; it now reads only that lineage's directory. With chains that follow base pointers (one listing per hop), the old walk cost 1.1 s per chain resolution late in BranchBench'smctsrun (the "Diagnostic instrumentation" tables in docs/benchmarks.md) and made that topology 3x slower during this cycle (197.2 s). Measured after the fix:mcts41.9 s, against 63.3 s on v0.2.11, with fork, checkout and checkpoint at 32.6 / 78.5 / 67.3 ms p50 at depth 25; all five topologies are faster than on v0.2.11 (docs/benchmarks.md's "BranchBench topologies" section).
[0.2.11] - 2026-09-26
Added
create --fromreaches the daemon and both SDKs. The daemon'screateop now accepts an optionalpath(an absolute, server-side path to an existing SQLite file, imported viaops.CreateFrominstead of minting an empty database); refused, likeexport, over the HTTP surface whenpathis set, unix-socket-only. Pythoncreate(db, from_path=None)and TypeScriptcreate(db, { fromPath })resolve the path to absolute client-side. MCP is deliberately excluded, by design — an MCP tool must never import an arbitrary host file named by an agent..db-file seeds in the pytest plugin and the TypeScript testkit.offshoot_seed/offshoot_db'sseed=and the testkit'sseedOnce'sseednow also accept a path to an existing SQLite database file (detected by itsSQLite format 3\0header, never by extension alone), importing it via the newcreate/from_path/fromPathreach and forking each caller from itsinitcheckpoint with fingerprintdb:<sha256 of file bytes>.make bench-isolationand a measured per-test-isolation benchmark.scripts/bench-isolation.pytimes offshoot's fork+open+close against SQLite's backup API, a plain file copy, and Postgres template-clone / cold-container primitives on the same host; results are pasted verbatim intodocs/benchmarks.md's new "Per-test isolation primitives (v0.2.11)" section.- A runnable pass^k eval example and a recipe page.
examples/eval-pass-k/(run.py,golden.sql, README) computes pass@1/pass^k end to end against a seeded database, wired into CI asmake example-pass-k;docs/recipes/eval-harnesses.mdmaps the seed-once-fork-many pattern onto tau2-style environments, Inspect AI, and promptfoo. - Explicit MCP tool annotations. Every
offshoot_*tool'stools/listentry now carries a fully explicitannotationsobject (readOnlyHint/destructiveHint/idempotentHint/openWorldHint, the last alwaysfalse) instead of relying on the spec's defaults —destructiveHintdefaults totrue, so an unannotated tool used to read as destructive to a host that checks the hint.offshoot_listis the only read-only tool;checkout/fork/checkpoint/touchare non-destructive;rollback/promote/destroyare destructive. structuredContenton every successful MCP result. Alongside the human-readable text, each result now returns a snake_case JSON object (txid,path,ttl,expires_at,backup, …, tool-dependent) so a harness can read fields instead of parsing sentences.metaonoffshoot_fork/offshoot_checkpoint. Both MCP tools now accept an optionalmetaargument (string→string, at most 32 keys) that tags the resulting branch or checkpoint with a run id, git SHA, or agent name for later lookup — closing the gapdocs/status.mdhad tracked as deliberately deferred.offshoot_touchMCP tool. Resets a branch's activity clock so its TTL does not expire mid-task, and optionally changes the TTL (ttlomitted keeps the current one, a duration sets it,"none"clears it). The server now exposes eight tools, not seven.- Claude Code plugin and marketplace (
plugin/,.claude-plugin/marketplace.json):claude plugin marketplace add sricola/offshoot && claude plugin install offshoot@offshootinstalls the MCP server plus a skill that teaches the fork/checkpoint/rollback/promote loop and advisory hooks that only add context, never block a command. - Cursor one-click install link for
offshoot mcp(https://cursor.com/en/install-mcp?..., which hands off to the Cursor app), alongside the existingclaude mcp addcommand. - Content-aware
offshoot diff --summary. Rather than a bare row-count comparison,--summarynow counts rows added/removed/changed per table by row identity — the table's declared primary key when it's actually unique on both sides, otherwise its internal rowid — and flags schema changes; a table whose column list differs between the two sides is reported, with row counts, but not compared.offshoot diff ... --table Trestricts either mode (the defaultsqldiffor--summary) to one table. - Daemon
diffop.left,right,table,full,max_bytesrequest fields; adiffresponse object carrying the same content-aware summary as JSON, plus cappedsqldiffSQL whenfullis set (0 = 1 MiB default cap, 8 MiB ceiling). Allowed over the HTTP surface, unlikeexport, since it never returns a filesystem path. - SDK
diff(), both languages. Pythondiff(left, right, *, table=None, full=False, max_bytes=None) -> DiffResult; TypeScriptdiff(left, right, opts)withDiffOptions{table?, full?, maxBytes?}. offshoot_diffMCP tool (the ninth tool). Read-only;database,left,rightasbranch[@checkpoint],table?,full?,max_bytes?(default 32768, at most 262144) — compares two branches or checkpoints of one database to decide which attempt to promote.offshoot protect/offshoot unprotect <db>[@branch]. Any branch can now be givenmain's own protected-by-default treatment (refuse unforceddestroy/promote --onto, never TTL-reaped), or have it cleared. CLI-only by design — no daemon op, no MCP tool sets or clears the flag (an agent can still see it viaoffshoot_list/the daemon'sbranchesop); flipping it does not touch the branch's activity clock, so unprotecting a stale branch doesn't incidentally grant it a fresh TTL window.- Rollback safety fork.
offshoot rollbacknow keeps the branch's previous head first as a shared, TTL'd fork,<branch>-pre-rollback(one per branch, replaced by the next rollback of the same branch, marker-guarded so a user's own branch at that name is never replaced) — the same shape and implementation as promote's<target>-pre-promote. Undo a rollback by promoting the safety fork back onto the branch. Surface: CLI--no-backup/--backup-ttl DURplus a "kept … undo with" line; daemonno_backup/backup_ttlrequest fields and abackupresponse field; Pythonrollback(db, branch, to, *, backup=True, backup_ttl=None); TypeScriptrollback(db, branch, to, opts: RollbackOptions); MCPoffshoot_rollbackalways keeps it and names the undo handle (backup) in its result. offshoot mcp -allow-force. Off by default. Without it, an agent-suppliedforce: trueagainst a protected target/branch is refused before any mutation, andoffshoot_destroy's live-lease bypass is disabled too; with it, both behave exactly as before this flag existed. The refusal names-allow-forceand points the agent at the CLI or a fork. The gate fails closed on a transient error reading the branch's own protected flag (forceis downgraded tofalse, never let through).- MCP guards a protected branch's own safety fork, not just the branch
itself.
offshoot_destroy,offshoot_touch(when it would change the TTL), andoffshoot_promote(when the fork is the target) now all refuse, without-allow-force, on a branch that is itself<target>-pre-rollback/<target>-pre-promoteof some OTHER protected branch (guardProtectedSafetyFork) —main-pre-rollbackis not itself protected, so the plain protected-branch check never caught this before, and promoting onto it would repoint (destroy) the undo point just as surely as an actual destroy.offshoot_rollbackadditionally refuses outright, without-allow-force, when the protected branch it's rolling back already has a safety fork from an earlier rollback, rather than silently replacing an undo point the human may still need. A plainoffshoot_touchthat only extends a fork's life (nottlargument) is never blocked. The guard fails CLOSED on a transient error reading the target's own ref (there is no downstream backstop the wayrefuseForceOnProtectedhas, since the fork itself is never protected), and so doesoffshoot_rollback's own check of whether its safety-fork name already exists. SeeTestGuardProtectedSafetyForkandTestGuardProtectedSafetyForkFailsClosedOnTransientReadErrorininternal/mcp/tools_test.go. - MCP's safety-fork TTL floors at 24h.
offshoot_promoteandoffshoot_rollbacknow passmax(-default-ttl, ops.DefaultPromoteBackupTTL)as the safety fork's TTL, not-default-ttlalone — an operator runningoffshoot mcp -default-ttlshorter than 24h (to keep throwaway attempt forks small) no longer shrinks the undo window on<branch>-pre-rollback/<target>-pre-promotebelow a day as a side effect; a longer-default-ttlstill wins.TestPromoteKeepsSafetyForkAndSaysSoandTestRollbackKeepsSafetyForkAndSaysSonow cover both the floor (2h default → 24h fork) and a default above it (48h → 48h fork). offshoot mcp -reap-every DURATION|none(default60s). Reaps expired forks and self-heals stranded delete claims on this cadence for as long as the MCP process is up — the daemonless fallback a bareoffshoot mcpdidn't have before. Defers entirely to a reachableoffshoot servedaemon's own janitor (never a second writer against one store), logging once that it's skipping its own pass. Runs no GC either way — reclaiming storage still needsoffshoot gcor the daemon.- OpenSSF Scorecard workflow and README badge.
.github/workflows/scorecard.ymlrunsossf/scorecard-actionon push tomain, a weekly cron, andworkflow_dispatch, publishing results (the badge and the scorecard.dev viewer page) and uploading SARIF to code scanning. The badge shows no score until the workflow's first run onmainafter merge. - Signed and attested releases.
release.ymlnow signs every release tarball with keyless cosign (<asset>.sigstore.json, no maintainer-held key), attaches SLSA build provenance (a GitHub attestation plus anoffshoot_<tag>_provenance.intoto.jsonlrelease asset), and ships an SPDX SBOM of the Go module graph (offshoot_<tag>.spdx.json) with its own attestation. Thedockerjob cosign-signs the GHCR image digest and pushes a provenance attestation alongside it. A "Verify what you downloaded" recipe (gh attestation verify,cosign verify-blob,cosign verify/gh attestation verify oci://for the image) is indocs/installation.md, linked fromSECURITY.mdanddocs/testing.md. Releases before v0.2.11 remain checksum-only. cmd/branchbenchandmake bench-branchbench. Re-runs BranchBench's (arXiv:2604.17180) five macrobenchmark topologies —simulation,data_cleaning,software_dev,mcts,failure_repro— against a local offshoot store, each workflow getting its own fresh store.go test ./cmd/branchbenchruns all five at-quickscale as part of the normal suite;docs/benchmarks.md's new "BranchBench topologies (v0.2.11)" section has one pasted full-scale run (2,310/2,310 worker-steps, 0 CAS retries) alongside BranchBench's own published hosted-Postgres-family numbers for context.- README first screen and the testing page as a launch artifact. The
README's first screen now leads with Install, then "see it run", then
the stability paragraph, then the quickstart, instead of quickstart
first.
docs/testing.mdgained an "At a glance" evidence table, a "Signed releases" section, and a "What is not proven here" section naming this project's own testing gaps rather than leaving them implicit.
Changed
- Docs updated for the nine-tool MCP surface:
README.md,docs/agents.md,docs/reference.md,docs/status.md, anddocs/demo/mcp-walkthrough.md(itstools/listtranscript re-captured from a realoffshoot mcprun, twice — once foroffshoot_touch, again foroffshoot_diff) now describe annotations,structuredContent,meta, andoffshoot_touch/offshoot_diff, and document all three install paths (claude mcp add, the Claude Code plugin, and Cursor). offshoot diff --summary's output gainedADDED/REMOVED/CHANGEDcolumns: it's now a content-aware per-table comparison (rows added, removed, and changed by primary key or rowid, with schema changes flagged), not a bare row-count diff —STATUSfor a comparable table issame/changedbased on those counts (plus a schema change), suffixed " (schema)" when the schema itself changed.offshoot compactnow preserves every checkpoint instead of resetting to a singlecompactcheckpoint: each existing checkpoint's snapshot is copied into the new self-contained lineage and rewritten to epoch 1 (rollback-style), and acompactcheckpoint is added at head alongside them. Cost: one extra snapshot copy per distinct checkpoint txid kept, on top of the head copy compact already paid. Nothing needs exporting first to survive a compact anymore.- Behavior change:
offshoot mcphonors an agent'sforce: trueonly when the server was started with-allow-force. Earlier versions honored it unconditionally, the same as the CLI's--force. Harnesses that relied on an agent forcing past a protected branch or a live lease through MCP need to add-allow-forceexplicitly; the plugin's default.mcp.jsonwiring stays on the safe (refuse) default. Docs re-driven to match:docs/demo/mcp-walkthrough.md's transcript is a fresh capture (sections on promoting past the protected-branch refusal now showoffshoot_diffcomparing the attempt againstmain, followed by a human-run CLI--forcepromote — noforce: truetool call anywhere in it), plusdocs/agents.md,docs/reference.md,docs/status.md,docs/concepts.md,docs/architecture.md,README.md, andplugin/skills/offshoot/SKILL.md.
[0.2.10] - 2026-09-25
Added
- Promote keeps the target's previous head as a TTL'd safety fork.
Promote was the one verb whose inverse the user had to build by hand
(the demo forked
mainbefore promoting, because promote resets the target's checkpoint map). Now, before the repoint lands, the target's head is kept as a shared fork named<target>-pre-promote— two metadata objects, no data copy — markedoffshoot.pre-promote=<target>and TTL'd (24h by default). One rolling safety fork per target: the next promote replaces it, and only ever replaces a branch carrying the marker, so a user's branch at that name makes promote refuse instead. Undo a promote by promoting the safety fork back onto the target (that undo skips minting a new one). The accepted cost, documented in docs/reference.md: the fork's base pointer pins the target's old lineage until the TTL reaps it. Surface: CLI--no-backup/--backup-ttl DURplus a "kept … undo with" line; daemonno_backup/backup_ttlrequest fields and abackupresponse field (a session open on the previous safety fork is refused up front, like the target); MCPoffshoot_promotealways keeps it (TTL follows the configured fork default) and names the undo handle in its result; Pythonpromote(backup=, backup_ttl=); TypeScriptPromoteOptions.noBackup/backupTtl. Prompted by discussion #40. The parallel-attempts demo, README transcript, asciicast, and MCP walkthrough were re-captured from real runs. langgraph-checkpoint-offshootgrows an explicit[test]extra and its ownmake test-python-langgraphtier (wired into CI in a separate venv, with the no-skipOFFSHOOT_REQUIRE_LANGGRAPHpath armed), so the companion is exercised against a real compiledStateGraphon every push instead of skipping when langgraph is absent.- Release images are published for linux/amd64 and linux/arm64 (pinned QEMU), and the release verifies both platforms are present in the pushed index.
Fixed
- Flush could lose its final ref CAS to the session's own lease
heartbeat, surfacing to daemon users as a spurious "flush lost a race
(retry)" failure. Renewals stamp a fresh
LeaseExpiryinto the ref via the same CAS path, and a long flush (drain + encode + upload between GetRef and PutRef) can span a heartbeat. Flush now re-reads the ref on a CAS loss and, when it still carries our holder+epoch on the same lineage with the head untouched — proof the only rival write was lease bookkeeping — reapplies the head advance onto the fresh revision and retries (bounded). Foreign holders, new epochs, new lineages, or a moved head fail exactly as before. Pinned byTestFlushSurvivesOwnLeaseRenewalRace. TestForceDestroyStillClaimGuardsfailed ~1% of macOS runs (and the 2026-09-20 weekly nightly) on a legitimate interleaving — AcquireLease completing before a force Destroy's first read, which force then destroys by design. The test now judges that both-succeed case by the order the two ref writes reached the backend, so only a lease landing after the Deleting claim (the actual double-winner) fails it. No change to Destroy.offshoot_fork's MCP description states its real cost (two metadata objects; at-head forks hash the checkout) instead of "costs nothing".- Test-suite flakes: a busy-timeout for
mustExecagainst live-session targets, a structural (not timed) concurrent-takeover rebase assertion, and an observation race inTestFlushLoopRetriesAfterRebaseDuringUpload.
Changed
- CI's S3 conformance provider is RustFS, not MinIO. MinIO withdrew
its community images from Docker Hub and quay.io and its binaries from
dl.min.io in 2026 (the project archived its repo), which broke the
s3-conformancejob on every push.rustfs/rustfs:1.0.0(digest-pinned) passes the identical probe + conformance + multipart suite, so it is now the every-PR provider;make ci-local-s3(old nameci-local-miniostill works) andmake bench-s3follow.TestS3RealProvidergainsOFFSHOOT_S3_CREATE_BUCKET=1(creates the bucket on a disposable local server, so no client image is needed either). The provider table now lists MinIO as verified through v0.2.9 and no longer re-verified, and AWS S3 as verified nightly: theNIGHTLY_S3real-provider job is configured and green as of 2026-09-25, which starts the sustained-green window the 1.0 criteria ask for. - Build floor moved from Go 1.25 to Go 1.26 (
go.mod, Dockerfile build stage, documented from-source requirement):golang.org/x/sysv0.48.0 declaresgo 1.26. Dependency bumps in one pass (superseding Dependabot PRs 42–44, 51–57): x/sys 0.48.0, aws-sdk-go-v2 1.46.0 (s3 1.111.0, config 1.33.3), smithy-go 1.28.1; actions/checkout 7.0.1, actions/setup-go 7.0.0, docker/setup-buildx 4.3.0, docker/setup-qemu 4.3.0, actions/deploy-pages 5.0.1 (SHA pins + version comments). - Docs site:
llms.txtandllms-full.txtemitted for LLM crawlers; the README got a centered header, badges, nav, and scannable long sections. Makefilegains acheck-python-versionguard (PYTHON=override) so an old systempython3fails loudly instead of confusingly.
Removed
flake.nix(the Nix dev shell): unmaintained and untested next to the four supported install paths — Homebrew, prebuilt binaries, Docker, andgo install/source.
[0.2.9] - 2026-08-13
Changed
- CI/release supply-chain hardening: every GitHub Action SHA-pinned (with
Dependabot keeping pins fresh), least-privilege per-job release
permissions, checksum-verified promtool download, digest-pinned MinIO
images, no
${{ }}interpolation in workflow shell,:latestDocker tag only on real tag pushes, and one shared sqlite setup action. - Hygiene batch (audit pass 3): corrected the three stale CI-cadence
claims (docs/testing.md is now the canonical cadence statement;
stability.md links to it), bumped the site version chip to v0.2.8 and
dropped the duplicate footer copy (chip bump added to the release
checklist), added the S3 timeout-model index comment, listed the trust
docs + Grafana dashboard on README's front door, exported a public
offshoot.client.TTLalias (_TTLstill works), added a delegation tripwire test + directlanggraph-checkpoint>=4.2,<5pin to sdk/python-langgraph, and single-sourced that package's version in pyproject.toml. - Bumped
golang.org/x/sysv0.40.0 → v0.44.0 (clears GO-2026-5024 from module-level govulncheck output; the affected code is Windows-only and never called here). x/sys v0.44.0 requires Go 1.25, so the module'sgodirective, the Dockerfile build stage, and the documented build-from-source floor moved from Go 1.24 to Go 1.25 with it.
[0.2.8] - 2026-08-12
Added
- Packaging surfaces: in-repo Homebrew formula (
Formula/offshoot.rb), Nix flake (flake.nix, vendorHash pending firstnix build), a README install matrix (brew/docker/nix/prebuilt/WSL2), andsqldiffin the Docker image sooffshoot diffworks in-container. - Copy-on-write fork benchmark harness (
make bench-cow,internal/ops/cow_bench_test.go) and published measured numbers for shared-fork latency, added bytes per fork, divergence cost, and the read path — the new "Copy-on-write fork cost (v0.2.x)" section at the top of docs/benchmarks.md. - Trust-and-adoption docs batch:
docs/stability.md(the pre-1.0 stability contract: same-release migration-or-export/create --frompromise, the LayoutVersion gate, proposed v1.0 criteria),docs/testing.md(how offshoot is tested — real torture-run numbers, fencing/CAS model, conformance suite, CI gates),docs/ci-recipes.md(copy-pasteable GitHub Actions seed-once/fork-per-attempt workflows + pytest-fixture variant),docs/grafana-dashboard.json(ready-to-import dashboard over all 18 metric families), a CLI/daemon/SDK parity table indocs/reference.md, and a rewritten merge FAQ entry ("Can I merge two branches?"). langgraph-checkpoint-offshoot(sdk/python-langgraph) 0.1.0: a LangGraphBaseCheckpointSaverthat wraps the stockSqliteSaveron an offshoot-managed checkout, addingfork_thread/checkpoint/rollback/promote/destroy(each 1:1 with an offshoot op) so LangGraph threads can be forked per attempt, rolled back, promoted, and TTL-reaped.
[0.2.7] - 2026-08-11
Fixed
- Every single-shot S3 call is now bounded too — a backend that stalls
mid-body (or stops reading an upload) can no longer wedge Get, Put, or
the flush path's below-threshold snapshot upload. 0.2.6 bounded the
multipart RPCs and added a 60-second response-header timeout to every
call, but the header timeout only bounds waiting for a response to
BEGIN: a request that was accepted and then stalled mid-body still hung
its caller forever, several of which sit on the daemon's flush path
under the flush lock (so
Session.Closehung too). Now every buffered single-shot RPC —Get(including its body read),Put,PutIf, eachListpage,Delete, eachDeleteObjectsbatch,CopyObject's HEAD and single-request copy, and the below-threshold singlePutObjectinsidePutReader/PutReaderIf— runs under a per-call deadline. Calls with a known payload size (Put/PutIf, the below-threshold singlePutObject, the single-request copy) get a size-scaled deadline: a generous 15-minute base plus transfer time at the same pessimistic 1 MiB/s throughput floor the multipart per-part deadline is derived from, so a legitimate just-under-5-GiB single-request transfer (~85 minutes at that floor) always fits — "eventually unwedges", never "fails fast". Calls with no meaningful payload (List pages, Delete, DeleteObjects batches, HEAD, and the bufferedGet, which on this backend carries small metadata objects — large chain members stream viaGetReader) get the flat 15-minute base.GetReader's stream gets no total deadline — that would kill legitimate long reads of large objects — but is wrapped in a progress watchdog instead: a singleReadthat blocks for 60 seconds with no bytes at all cancels the request and fails the read with a recognizable "read stalled" error, while any progress re-arms the window and a slow CONSUMER (long pauses betweenReads) is never affected, because the watchdog is only armed while aReadis actually blocked.
[0.2.6] - 2026-08-11
Changed
- Create-only multipart uploads (a
PutReaderIfwith no etag — in practice a >5 GiB snapshot flush) now issue a best-effortHeadObjectpreflight and fail fast withErrCASwhen the key already exists, instead of uploading every part only to haveCompleteMultipartUpload's condition reject them after the full transfer. The preflight is an optimization only: any Head error (network, 403) falls through to the normal upload, and Complete'sIfNoneMatchcondition remains authoritative for a key created in the Head-to-Complete race window. The etag-CAS path and the below-threshold single-PUT path are unchanged. - Internal:
store.S3's multipart machinery moved froms3.gointo a newinternal/store/s3_multipart.go, and its repeated mechanical fragments (abort defer,CompletedPartchecksum carry, part-length clamp, the fake's XML error writes, the tests' fault-injection closure) were deduplicated into helpers. No behavior change.
Fixed
- A stalled S3 response can no longer wedge flush via the multipart
path, and a response that never begins is now bounded on every S3
call. The S3 client's HTTP transport now applies a
60-second response-header timeout to EVERY S3 call (Get/Put/List
included, not just multipart): it bounds only how long a call waits for
a response to begin, never the body transfer, so a slow-but-progressing
large download or upload is never affected. On top of that, each
individual multipart RPC (part upload/copy, create, complete, and the
new preflight above) runs under its own 15-minute deadline — sized to a
worst-case ~550 MiB part at a pessimistic throughput floor — and the
cleanup
AbortMultipartUploadunder a 1-minute one, so a response that begins and then stalls mid-transfer fails the flush (which then retries normally) instead of blocking anUploadPartforever while holding the flush lock, which hung every laterFlushandSession.Close.
[0.2.5] - 2026-08-11
Fixed
-
Silent data loss when a fenced writer's snapshot shared a txid with the next writer's segment. Chain resolution could serve a dead writer's entire database and drop a committed transaction, with no error and no checksum failure. The scenario: session A uploads a snapshot for txid T, its ref write fails with an ambiguous (non-CAS) error — a timeout, a network blip — so the uploaded object is deliberately left behind (deleting it would risk a live ref pointing at a missing object, the worse bug), and A dies. Session B takes the branch over under a higher epoch, writes a segment for that same txid T, and wins the ref CAS. Both objects now sit at T.
Resolution deduplicates same-txid objects and keeps the highest epoch, which is what makes epoch fencing hold on the read path — but it keyed on each object's txid range and ran over snapshots and segments separately. A snapshot for txid T covers everything up to T, so its range key is
{first: 0, last: T}, while a segment for T covers only T itself, so its range key is{first: T, last: T}— different keys, and held in different lists, so A's snapshot and B's segment could never be compared, and the fenced snapshot always survived. Resolution then anchored on it as the newest snapshot at or below the target and, since it landed exactly on the target, returned it alone: B's committed transaction was silently gone, and a read returned A's state as though it were current. The fenced snapshot is internally valid — it is simply the wrong object — so nothing downstream could detect it.Deduplication now runs once over the combined set, keyed on the txid an object establishes state at, so a snapshot and a segment for the same txid do collide and the live (higher-epoch) writer wins. Reads are byte-identical for any store without such a conflict, which is every store that has not hit this race.
[0.2.4] - 2026-08-11
Added
-
S3 multipart uploads now upload parts concurrently.
store.S3'sputMultipart(backingPutReader/PutReaderIfabovemultipartThreshold) previously uploaded parts strictly sequentially — a 5 GiB snapshot at the 64 MiB default part size paid ~80 sequentialUploadPartround trips. Parts now upload in parallel, bounded tomultipartConcurrency(default 4) simultaneous workers, but ONLY on theio.ReaderAtpath (the production*os.Filecase):io.ReaderAtis documented safe for parallelReadAtcalls on the same source, so each worker can read its own part independently. The non-io.ReaderAtfallback — which reads every part sequentially into one reused buffer — is unaffected and stays strictly sequential; parallelizing it would either race the shared buffer or need a buffer per worker, multiplying memory for exactly the callers who couldn't afford onepartSizebuffer in the first place. Every part's result lands in a pre-sized slice at its ownPartNumberindex, so the finalCompletedPartlist stays correctly ordered regardless of completion order — no sort step needed. The existing abort-on-every-error-path guarantee and per-part checksum propagation carry over unchanged under concurrency; error handling is first-error-wins with prompt cancellation of every in-flight and not-yet-started part — the cancellation half is NEW with concurrency (the prior strictly-sequential code never had anything in flight to cancel), not carried over. -
store.S3.CopyObjectnow server-side-copies objects over 5 GiB instead of declining. Previously, any source over S3's 5 GiB single-requestCopyObjectlimit returnedErrCopyUnsupported, forcing callers (ops.Fork's fast fork path) to fall back to a full materialize-download-re-encode-upload of the whole database.CopyObjectnow performs a real multipart server-side copy for sources up to S3's actual 5 TiB per-object ceiling:CreateMultipartUpload+ a sequence ofUploadPartCopycalls (each with an inclusiveCopySourceRangebyte range, sized the same way multipart uploads size their parts) +CompleteMultipartUpload.ErrCopyUnsupportedis now reserved for sources genuinely beyond any S3 mechanism's reach (over 5 TiB) — the 5 GiB boundary is now a strategy choice (single-request vs. multipart), not a capability limit. Same abort-on-every-error-path discipline as the multipart upload path;CompleteMultipartUploadsets no conditions, matchingCopyObject's existing overwrite-on-existing-dst contract. Unlike the multipart upload path,CreateMultipartUploadhere deliberately does NOT declare aChecksumAlgorithm:UploadPartCopyhas no client-side body to checksum, and a source object's checksum may not be derivable at all (an older writer, a third-party object, a different algorithm, or a stored checksum that doesn't decompose at this copy's part boundaries) — declaring one would makeCompleteMultipartUploadreject exactly those copies. Whatever checksum fields S3 does return per part are still forwarded when present.
Fixed
- GC's compensating rule is now epoch-aware, closing a bounded space
leak. The rule that protects a member object above a live ref's head in
that ref's own lineage (guarding a session flush's ambiguous-ref-write
retry, which re-Puts the SAME
(lineage, epoch, txid)key) used to protect such an object regardless of its epoch. An orphan left by a writer that was later fenced (Ref.Epochbumped out from under it byAcquireLease, e.g. on lease reclaim) sits at an epoch strictly below the lineage's current writer generation and can never win a future ref CAS — the fenced writer's own flush refuses outright on the epoch mismatch (session/flush.go'sref.Epoch != lease.Epochcheck), and even setting that aside,AcquireLease's epoch bump is itself a CAS'dPutRefthat changes the ref's etag, so the fenced writer's terminalPutRef(built from its now-staleGetRef) fails unconditionally — so it can never become live, yet the epoch-blind rule protected it forever while the branch stayed alive. The rule now compares the object's epoch (ChainMember.Epoch) against the lineage's currentRef.Epoch(the MINIMUM read across every ref naming that lineage, same over-protect direction as the existing minimum-head rule) and protects only>=; a strictly older epoch is provably fenced and is now swept once its stone clears grace. A stale read ofRef.Epochonly ever protects more, never less: an object at an epoch newer than what was read (a new acquirer bumped it after the read) still satisfies>=and stays protected.
Changed
-
Documented the fork-floor/
-snapshot-everydivergence between a daemon and an at-rest CLI fork.-snapshot-every N(seeoffshoot serve) is a per-process flag, never persisted to the store, and it also sets the daemon's fork-time snapshot-floor bound (ops.Workspace. SnapshotEvery). An at-rest CLIfork— no daemon involved — has no way to learn a daemon's configured value and always uses the library default of 16 (ops.ForkShareMaxDepth) instead, even against a store a daemon elsewhere serves with a different cadence. No behavior change: the bound stays correct either way (materialization stays bounded), just DIFFERENT from what the daemon's cadence would have used — looser than a lower configured N, tighter than a higher one (Nhas no upper bound, only the>= 1floor). Documented in docs/reference.md (offshoot fork) and docs/operations.md (tuning flags). -
storetest.FakeS3now enforces declared-vs-supplied checksums atCompleteMultipartUpload. (Shipped in 0.2.4; this note was omitted from the entry at release and is recorded here after the fact.) The fake records theChecksumAlgorithmaCreateMultipartUploaddeclares, echoes eachUploadPart's checksum headers back on the response, and rejects aCompletewhose part manifest is missing the declared algorithm's checksum withInvalidRequest— the same declared-vs- supplied contract real S3 enforces. This supersedes the 0.2.3 entry's "the fake ignores checksum headers entirely" caveat (accurate when written): from 0.2.4 on, the fake does cover the checksum-mismatch case, which is what makes the multipart upload path's explicit CRC32 declaration and the multipart copy path's deliberate non-declaration both load-bearing in tests rather than untestable.
[0.2.3] - 2026-08-09
Added
-
S3 snapshot uploads over 5 GiB no longer fail.
store.S3.PutReader/PutReaderIfpreviously issued a singlePutObject, which S3 caps at 5 GiB — a snapshot of a larger database failed to upload. Both now switch to a real multipart upload (CreateMultipartUpload+ sequentialUploadPartCompleteMultipartUpload) once size exceedsmultipartThreshold(default: 5 GiB, matching the single-PUT ceiling it replaces). Part size is computed to respect S3's two hard limits — parts >= 5 MiB, at most 10,000 parts — so any object up to S3's 5 TiB per-object limit stays within the part-count ceiling.PutReaderIf's CAS condition (IfNoneMatch: "*"/IfMatch) is placed onCompleteMultipartUpload, not onCreate/UploadPart, and a precondition rejection there maps tostore.ErrCASexactly as the single-PUT path does — CAS semantics are indistinguishable to callers regardless of which path an object's size took. Below the threshold, behavior is byte-for-byte unchanged: the original single-PutObjectpath, untouched.
Cost-critical: an abandoned multipart upload leaves its uploaded parts billed on S3 indefinitely, so every error exit after a successful
CreateMultipartUpload— a part-upload failure, a part-read failure, aCompletefailure including a CAS rejection — runsAbortMultipartUploadvia adefer; only a successfulCompleteskips it. An abort failure is appended to (never masks) the original error.Part bodies avoid buffering the whole object: when the caller's reader is both an
io.ReaderAtAND anio.Seeker(true of the*os.Filethe only production caller —flush.go's snapshot upload — passes), each part streams directly from the file viaio.NewSectionReader, zero buffering — the reader's current position (viaSeek(0, SeekCurrent)) is captured as a base offset so a reader the caller already seeked into uploads the same bytes the single-PutObjectpath would have (that path reads fromr's current position viaBody: r);io.ReaderAtalone would silently read from absolute offset 0 instead. A reader missing either falls back to reading one part at a time into a reused, part-sized buffer.store.S3.CopyObject's separate 5 GiB server-side-copy limit is untouched — that's a documented, unrelated fallback (to the slow materialize-and-re-encode path), not a failure.Both
CreateMultipartUploadand everyUploadPartcall now declareChecksumAlgorithm: CRC32explicitly, and each part's returned checksum is carried fromUploadPartOutputinto itsCompletedPartentry. Declaring it on both calls (not just Create) matters regardless of the SDK'sRequestChecksumCalculationsetting: left to its default (WhenSupported) the SDK attaches a CRC32 checksum to everyUploadParton its own, but underwhen_required(the common workaround for third-party S3-compatible stores after the Jan-2025 default-checksum change — exactly the audienceS3Confignames via R2/Tigris/MinIO) it would not, leavingCreateMultipartUpload's declared algorithm unmatched by any part's supplied checksum. Either way, a real S3CompleteMultipartUploadrejects a mismatch withInvalidRequest— a case the in-process fake cannot reproduce (plain HTTP takes a different SDK code path than theaws-chunkedframing production traffic uses over TLS, and the fake ignores checksum headers entirely).CompleteMultipartUploadInputalso now setsMpuObjectSizeto the expected total, so a length mismatch from this backend's own offset arithmetic is rejected by S3 with a 400 instead of silently storing a wrong-length object.Part size defaults to 64 MiB (
defaultPartSize), not a smaller "AWS-CLI-like" default — parts upload strictly sequentially (no concurrency yet), so a smaller default part size only means more sequential round trips for no benefit. BothmultipartThresholdanddefaultPartSizeare overridable in tests only (export_test.go'sSetMultipartThresholdForTest/SetPartSizeForTest) since a real >5 GiB upload, or one with a production-sized 64 MiB part, can't be exercised in a test;storetest.FakeS3now models the full multipart API (Create/UploadPart/Complete/Abort) to make the path genuinely testable, not just compiled.TestS3RealProvider(env-gated onOFFSHOOT_S3_TEST_BUCKET) gained a multipart subtest — a real provider'sCompleteMultipartUploadprecondition/checksum handling is the only evidence the in-process fake can never substitute for.
Changed
- Doc note on
store.BatchDeleter/ReaderGetter/ReaderPutter: these optional capabilities are discovered by type assertion, so astore.Backendwrapper that embeds another Backend (e.g. for instrumentation/caching) silently hides them — callers fall back to the buffered/per-key path, which stays correct but loses the batching/streaming benefit. No code change:ops.markCache(the one in-repo wrapper today) implementsBackenddirectly rather than embedding, and needs none of these three, so nothing is affected today.
[0.2.2] - 2026-08-08
Added
make lint: gofmt -l (fails on any unformatted file) +go vet+ best-effort staticcheck. ci.yml and ci-local run the same gofmt/vet gate on every PR, so an unformatted file can no longer ship silently.
Changed
-
Materializing a chain (checkout/fork/rollback/promote/compact's read path) no longer holds every resolved object in memory at once (perf audit H3, read-side).
ops.materializeMembersAtnow type-asserts the backend for a new optionalstore.ReaderGettercapability (GetReader(key) (io.ReadCloser, etag, error), implemented by bothS3— the rawGetObjectresponse body, noio.ReadAll— andLocal— a plainos.Open) and, when present, applies the chain through lazily- opened, self-closing streams: each member's stream opens on first Read and closes on EOF, so at most one snapshot/segment is resident at a time instead of the whole chain. Safe becauseltxio.MaterializeChainalready consumes its inputs strictly sequentially (snapshot fully, then each segment fully, in order). A backend withoutReaderGetterfalls back to the prior fully-buffered path unchanged. Materialized output, the returned checksum, and every stream's close-on-every-path (success and mid-apply error) are unaffected — pinned by a counting-backend test asserting max concurrently-open streams <= 1 and open count == close count. Read-side only; flush/capture (H3's write side) is unchanged. -
A snapshot flush no longer buffers the whole encoded snapshot in memory before uploading it (perf audit H3, write-side companion to the read-side entry above).
Session.flush's snapshot branch nowltxio.EncodeSnapshots to a second per-flush scratch FILE (alongside task 8's replica-clone scratch) instead of abytes.Buffer, then streams that file to the backend via a new optionalstore.ReaderPuttercapability (PutReaderIf/PutReader, implemented by bothS3— a plainPutObjectwithBody: r/ContentLength: size, noio.ReadAll— andLocal— write-to-temp-then-rename streamed viaio.Copy, computing the sha256 etag incrementally alongside the write) instead of buffering the object as a[]byteforPutIf/Put.PutReaderIf/PutReadermirrorPutIf/Put's exact CAS/create-only/overwrite contract, including the orphan-overwrite retry (a fresh reader reopened from offset 0, since the first attempt's reader is exhausted afterPutReaderIf). The segment branch (already O(changed pages)) and any backend withoutReaderPutterare entirely unchanged — still buffered, byte-for-byte the prior behavior. Upload→ref-CAS ordering, the durable checksum, and every other flush invariant (task 8's clone-then-release-replicaMudiscipline,forceSnapshot/pageSet drain semantics) are unaffected — pinned by a counting-backend test asserting the streaming methods are used and the buffered ones never are, that both scratch files are cleaned up on every path, and that streamed output materializes byte-identically to a direct encode; a separate test pins the no-ReaderPutterfallback unchanged. S3'sPutReaderIf/PutReaderinheritPutObject's single-request 5 GiB ceiling (same limitCopyObjectalready documents); multipart upload to lift it is a noted follow-up, not implemented here.make test-torture: 3168 rounds, 316 kill -9 bounces, 316/316 resumed cleanly, zero corruption. -
Snapshot flushes no longer hold the replica lock across the full-file encode (perf audit M2).
Session.flush's snapshot branch now clones the replica to a per-flush scratch file underreplicaMu(a copy-on-write reflink — near-O(1) — on APFS/btrfs/xfs, plain byte copy elsewhere), releases the lock, and runsEncodeSnapshotover the scratch outside it, so the capture engine'sApply/Rebaseno longer stall for the O(DB size) encode+checksum on every snapshot flush (multi-GB replicas: seconds ofLag()spike per snapshot, gone). The clone is taken whilereplicaMufreezes the replica at the flush's transaction boundary, so the uploaded bytes and checksum are exactly what encoding the live replica under the lock produced before; upload/ref-CAS ordering,forceSnapshot/pageSet drain semantics, and the segment path (already O(changed pages), still encoded under the lock) are unchanged. The scratch is removed on every exit path. -
Capture no longer fsyncs its resume-offset state file per transaction (perf audit M3).
drainStepnow records the applied position in memory and the engine persists it once per drain burst (plus at shutdown's final drain), lifting the ~200 txn/s capture ceiling the per-commit ~5ms fsync imposed. Safe because the persistedOff/Salt1/Salt2are operator observability only — resume eligibility rests solely on the verified-clean marker (Clean+ empty WAL +MainHash), which is unchanged — and the persisted offset can now lag the replica by up to one in-flight burst but never lead it. Clean shutdown/restart behavior (resume without rebase) is unchanged; a kill -9 mid-burst still safely rebases on the next start. -
Chain resolution of a diverged shared fork now Lists the child's prefix once instead of twice (perf audit H1). The seam path — a shared child with its own segments but no divergence-floor snapshot yet, the normal working state of an active fork — reuses one parsed member listing for both the snapshot-anchored walk and the seam-range walk. One less List RPC per affected lineage per resolution (Checkout, Fork, Compact, flush's divergence seed); resolution results are byte-identical.
-
A materializing (snapshot-floor) fork resolves the source chain once, not twice (perf audit M1). Fork passes the chain it already resolved for the floor decision straight to the snapshot copy instead of letting the copy re-resolve the identical (lineage, txid) — the redundant resolution was most expensive exactly when the floor trips, i.e. on a long chain. Rollback and Promote are unchanged; fork results are byte-identical.
-
GC's phase-2 sweep batch-deletes on S3: 1000 objects per DeleteObjects RPC instead of one serial DeleteObject each (perf audit H2). Destroying a month-old branch (~80k objects) now sweeps in ~80 round trips instead of ~80k (~an hour inside a single janitor tick). Implemented as an optional
store.BatchDeletercapability — S3 chunks into 1000-key DeleteObjects requests; the local backend loops; a backend without the capability falls back to per-key deletes. The swept key set, tombstone pruning, and abort-on-error semantics are unchanged (a partial batch failure prunes exactly the succeeded keys' tombstones and keeps the rest for a later pass); only the RPC shape changed. -
A sweeping GC pass no longer re-enumerates refs a third time (perf audit M4). The compensating rule's live-head set is derived from the same ListRefs + GetRefs the phase-2 re-mark already performed, saving 1 ListRefs + one GetRef per ref per sweeping pass — and guaranteeing the "never delete above a live head" rule sees exactly the refs the re-mark saw (one consistency window). GC results are identical.
-
Repeated shared forks pay the layout-v2 manifest check once per process, not once per fork (perf audit L4). The layout version is monotonic, so the store memoizes the "manifest already >= v2" observation after the first EnsureLayoutV2; subsequent shared forks skip the manifest Get entirely — 1 RPC saved per fork, ~15% of a shared fork's total RPC bill.
-
A CI image that loses sqlite3/sqldiff/promtool now fails loudly instead of silently skipping the integration suite green. The ~96 copy-pasted
exec.LookPathskip guards across the test suite collapsed into a sharedinternal/testutilhelper (RequireSQLite3/RequireSQLdiff/RequirePromtool): locally a missing binary still skips the test, but under CI (CInon-empty;OFFSHOOT_REQUIRE_PROMTOOLfor promtool's dedicated metrics-lint job) it is a hard failure. Test/CI-only change. -
Best-effort orphan cleanup is now loud. The seven orphan-cleanup deletes in ops (lost ref-CAS paths in create, checkpoint, fork, rollback, promote, compact) and Destroy's checkout-file removals now log to stderr when the underlying delete/remove fails, instead of failing silently. Behavior is otherwise unchanged: cleanup remains best-effort and reachability GC still reclaims anything left behind.
-
Named previously-hardcoded values: the session
DrainNowbudget (30s, shared by flush and close), the capture engine's checkpoint-takeover thresholds (64 frames / 5s idle), and the two deliberately different SQLite busy timeouts (ops quiesce 3000ms, capture engine 5000ms). Values unchanged. -
internal: split the checkout-fingerprint (.sum sidecar) subsystem out of
internal/ops/ops.gointointernal/ops/sidecar.go(no behavior change). -
internal: split the janitor (reap/stale-claim healing/GC/ro-cache eviction loop) out of
internal/daemon/server.gointointernal/daemon/janitor.go(no behavior change). -
Refreshed doc comments that still described pre-daemon/pre-CoW behavior (
Ref.Base,Ref.Epoch,EnsureLayoutV2, the ops package doc, the captureSinkcontract andhashSrccost note).
Removed
store.Store.DeleteRef(unused in production;Destroyuses the claim-guardedDeleteRefIf). Internal API only.
Security
exportis no longer reachable over HTTP/rpc(400, "not available over HTTP; use the local socket"). It is the one op that writes to an unconfined, client-chosen path on the daemon host's filesystem — safe under the unix socket's same-host/same-user trust model, but an arbitrary-file-write/exfiltration primitive for an authenticated network client on a non-loopback bind. Unix-socket export is unchanged.- The daemon's unix socket is created 0600 from the first instant
(a 0o077 umask around the listen), closing the window between
net.Listenand the subsequent chmod in which — for a-socketpath in a world-accessible directory — another local user could connect and get a full daemon client. The default socket path (0700 parent dir) was never exposed.
[0.2.1] - 2026-08-08
Added
offshoot_fork_mode_total{mode="shared"|"materialized"}— a fork-time counter for the copy-on-write storage mode, completing the 0.2.0 observability story:status/branchesalready report shared-vs-materialized per branch at read time; this records it as a rate at fork time.shared= a base pointer into the parent's chain, zero data objects copied;materialized= a full snapshot copy (the fork-time snapshot floor). Orthogonal tooffshoot_fork_total{path}, which names the materialize path's copy strategy. Both label values pre-registered at0; documented in docs/operations.md.offshoot_gc_errors_total— counts janitor GC passes that returned an error. GC deliberately fails closed (an incomplete reachability mark deletes nothing), which previously left a stalled GC silent while the store bloated: the error went to stderr only, with no alertable signal. The stderr line still carries the cause; the counter is the signal to alert on.
Fixed
- Daemon
compactnow refuses while the branch has an open session on this daemon ("close the session first"), matchingrollbackandpromote: compact repoints the branch to a new self-contained lineage, so a live session's next flush would CAS-fail against the repointed ref. Previously the daemon flushed the session and compacted anyway, leaving the session fenced. With no session open the durable head is already current, so compact materializes exactly that head — behavior with no open session is unchanged. - The fork-time snapshot floor now tracks the configured snapshot
cadence instead of the hardcoded default (16). A daemon started with
-snapshot-every Nbelow 16 could previously mint shared forks whose resolved base chains reached up to 15 members — bounded, but looser than the session's ownlen(chain) <= SnapshotEverycadence. The daemon now passes its cadence through to the fork floor (Workspace.SnapshotEvery), so the fork floor and the divergence floor agree. The at-rest CLIforkand any daemon without-snapshot-everykeep the default bound of 16 — behavior unchanged. store.Chainnow detects base-pointer cycles in a corrupt or hand-edited store and returns a clearbase pointer cycle detectederror instead of recursing until the stack overflows. Not reachable through this binary's writers (WriteLineageBaseis create-only, rejects a self-base, and always points at an existing lineage) — this is crash-hardening for a corrupt store, mirroringBaseSpine's existing defensive cycle guard.- Docs:
offshoot compact's cost class in docs/reference.md no longer claims a single compact pays "the same N×G copy" — N×G is the aggregate across N materialized forks; one compact/promote/rollback pays ~G (one full copy). README's install line no longer says tagged binaries land "with the first 0.1.x release" (stale since 0.2.0 shipped; binaries are published per tagged release). The original design spec's launch-demo bullet gained a superseded-note: under copy-on-write there is no async-upload/pending window for a shared fork to caveat.
[0.2.0] - 2026-08-07
Copy-on-write forks. offshoot fork no longer materializes a full
copy of the source into the child's lineage. The common-case fork now
shares: it writes a durable base pointer into the parent's
already-durable chain — zero data objects of its own — so N forks of a
G-byte database cost near-zero added store bytes instead of up to N×G.
The rest of this release is what makes that safe: base-following chain
resolution under a strict never-merge-across-lineages rule, two automatic
snapshot floors that keep read materialization bounded on any fork spine,
an object-granular reachability GC, a new offshoot compact command to
cut a shared branch's cord, and honest per-branch cost reporting in
status/branches.
Storage-format change — the reason this release is 0.2.0, not 0.1.4.
The first shared fork CAS-bumps the store manifest from LayoutVersion 1
to 2, and every pre-0.2.0 binary then refuses the entire store up
front (CheckManifest). The lockout is deliberate: a 0.1.x binary's
lineage-granular GC cannot see base pointers and would sweep a shared
ancestor's objects out from under live children — silent data loss — so
old binaries are fenced out of the store rather than allowed to corrupt
it. There is no downgrade path once a base pointer exists; upgrade every
binary that touches a store before any of them forks. A 0.2.0 binary
reads and writes a v1 (no-base) store unchanged, and GC's reachability
handles a mixed store (old full-copy forks + new shared forks) uniformly.
The two cost models, stated honestly: fork shares (near-free);
promote, rollback, and compact each still materialize a full
independent copy — "fork at checkpoint X is free but rollback to X costs
a full copy" is a deliberate, documented asymmetry, not an oversight
(rollback/promote abandon their old lineage; base-pointing into a lineage
that is meant to die would pin it forever). Destroying a parent stays
instant and always allowed, but its bytes are now reclaimed only once
no surviving child's chain still reads through them — destroy lingers
until the last sharing child is destroyed or compacted.
Changed
- Fork is copy-on-write (Tasks 1–4). The share path writes only a
durable per-lineage base pointer (
data/<lineage>/base.json, written create-only viastore.WriteLineageBase) plus the child ref's newBase *BasePointerreporting mirror (BasePointer{Lineage, TXID}, deliberately separate from the human-facingParentbreadcrumb, and deliberately epoch-free — pinning an epoch could resurrect a fenced writer's orphan). Resolution:store.Chainfollows base pointers transitively under the one hard rule that members are never merged across lineages — each lineage's half is resolved wholly within its own prefix and concatenated at a seam verified contiguous, sokeepHighestEpochcan never let a higher-epoch parent object win a child's txid range. The base object (not the ref mirror) is the resolution source of truth, so a shared parent's ref can be destroyed while descendants still resolve through its lineage. Bounded materialization survives sharing via two automatic floors, both keyed on the existing snapshot cadence: a fork-time floor (a fork whose fully-resolved chain is already atops.ForkShareMaxDepthmaterializes one fresh snapshot instead of sharing — the pre-0.2.0 fork path, now the fallback rather than the default) and a divergence floor (a session's snapshot counter seeds from the branch's durable divergence, so theSnapshotEverybound is a property of the branch, not the process, and a shared child stops touching its parent once it has written its own snapshot). Tests cover divergence isolation both ways, fenced-orphan safety at the fork point, bounded replay across a 40-level fork spine with content verification, and the cross-lineage-merge hazard (mutation-verified: a resolver that unions across lineages fails). - GC is an object-granular reachability collector (Task 5), a
rewrite forced by sharing: liveness can no longer be decided per
lineage when one lineage can be live-below-a-fork-point for a
descendant and dead-above-it for everyone. The mark phase computes the
live set as the union, over every live branch, of its resolved chain at
head AND at every checkpoint — using
store.Chainitself, never a reimplementation, so mark == read-path resolution by construction (property-tested) — plus everybase.jsonalong each branch's base spine. Sweep follows at object granularity:gc/tombstonesmaps individual object keys (stale lineage-keyed entries are pruned harmlessly); the two-phase tombstone → grace → re-mark → delete shape, mint-this-run skip, and clobber-safe stone persist all survive at the new grain.GC'stombstoned/deletedcounts, the CLIgcsummary line, and theoffshoot_gc_*metrics now count objects, not lineages. A per-pass memoized resolver keeps a shared ancestor's prefix listed once per pass regardless of descendant count. Two review-driven hardenings: a tombstone rescued by the re-mark is pruned (a later legitimate death earns a fresh grace period), and the sweep never deletes above a live branch's head in its own lineage (a retrying flush can legitimately re-create exactly that key). Covered by reachability, mixed-store, checkpoint-survival, and ancestor-destroyed-mid-fork-of-fork fault-injection tests, each mutation-verified. offshoot statusoutput (Task 7): every branch line now carries astorage=shared|materializedfield betweenstate=andtxid=— the per-branch cost class, surfacing the fork-shares vs promote/rollback/compact-materialize asymmetry instead of hiding it.
Added
offshoot compact <db>[@branch](Task 6) — the manual cord-cutter: re-encodes a shared branch's full base-following chain at head as ONE self-contained snapshot in a fresh lineage, CAS-repoints the ref with its base pointer cleared, and best-effort refreshes the checkout. The previously shared ancestor storage becomes unreferenced and is reclaimed by GC (compact itself deletes nothing). An already self-contained branch is a no-op returning the head txid, so scripted "compact everything" loops never fail. Flagged tradeoff: the checkpoint map resets to{"compact": head}, like promote's repoint (old checkpoints anchored on the shared ancestor would not resolve in the new lineage; preserving them rollback-style is a noted follow-up). A CAS loss to a concurrent flush deletes the orphan snapshot and returns a retry error. Ships as a daemon op (an open session is flushed first, like fork's source flush) and SDKcompact()in Python and TypeScript. Review fix folded in:PromoteandRollbacknow clear the ref'sBasemirror when they repoint a formerly-shared branch (previously a stale mirror survived, misreporting the branch as shared — and compact trusting it would needlessly re-materialize and wipe checkpoints Rollback had preserved); compact's no-op decision consults the durable base spine (store.BaseSpine), not the mirror, so the destructive path stays a no-op on a self-contained branch even against a stale mirror.- Per-branch cost reporting on the wire (Task 7):
BranchInfo(the daemonbranchesop) gains a wire-additivesharedbool — true for a base-pointer fork (near-zero added storage), false for a materialized, self-contained branch — mirrored byops.BranchStatus.Sharedfor the at-rest CLIstatus. Noomitempty: false means "materialized," never "unknown"; old clients that don't read the key are unaffected. - Store plumbing for all of the above (Tasks 1–2):
store.LayoutVersion2 (new stores are stamped v2 at init;Store.EnsureLayoutV2CAS-bumps an existing v1 manifest exactly once, idempotent under concurrent callers, called before the first base pointer is written),store.BaseKey/WriteLineageBase(which refuses a self-referential base) andstore.BaseSpine(the durable base-ancestor walk), andMaterializeChain's per-memberPreApplyChecksumverification doing fail-closed duty on every resolved splice — a mis-resolved chain with divergent content is a loud materialization failure, never silent corruption.
[0.1.3] - 2026-08-07
Milestone 4: operable at scale. A platform running hundreds of agent
sessions can now see (a hand-rolled, zero-dependency Prometheus /metrics
endpoint plus six computed branch states), bound (a checkouts-ro disk
budget with LRU eviction), and automate (an opt-in, token-authenticated
HTTP listener alongside the unix socket, plus an event bus drainable over
either transport) offshoot — without any of it requiring multi-node
anything; see docs/operations.md's first paragraph
and ROADMAP.md for that standing scope
boundary. Metric names are locked as of this release: every
offshoot_* name below is now API — renaming one after this tag is a
breaking change (this was the one free rename window; see PM Amendment
2/12 in the Milestone 4 plan). Operator-facing docs land in this release
too: docs/operations.md (the metrics/states/events/
budgets/HTTP-auth reference in one page) and
docs/recipes/kubernetes.md (a real,
schema-validated sidecar manifest). See
docs/status.md's Standing nag section for what's still
user-gated (PyPI/npm publication, the offshoot-db org transfer, registry
listing submissions) — none of it is blocked on further engineering.
Added
-
Branch state taxonomy (Milestone 4 Task 1):
ops.BranchStateAt(internal/ops/status.go) computes a branch's state —active,dirty,detached,idle— from its ref, lease liveness, and checkout.sumsidecar alone, no daemon dependency; a daemon layers two more states (pending,error) on top from its own in-memory session map (internal/daemon/server.go'sServer.branchState), since only a daemon knows which branches have a session reserved or errored. Precedence:error>pending>active>dirty>detached>idle.idleis a deliberate addition beyond the design spec's original taxonomy, which assumed a daemon was always present — see docs/reference.md for the full table and rationale. Surfaced inoffshoot status(BranchStatus.State), the daemonbranchesop (BranchInfo.state, always present — additive, notomitempty), and both SDKs'Branch.state(Python dataclass field, TypeScript interface field; both default to""against an older daemon that never sends it, so existing SDK code is unaffected). Disclosed cost: determiningdirtyfor a checked-out, unleased branch whose sidecar identity already matches the ref requires a real WAL checkpoint (quiesce, up to a 3s busy timeout) followed by a full SHA-256 hash of the checkout — per branch, perstatus/branchescall; a checkout that's busy during that checkpoint attempt is itself reporteddirtyrather thanidle— see docs/reference.md's Cost/Known blind spot notes. -
Metrics registry + instrumentation (Milestone 4 Task 2):
internal/metricsis a hand-rolled, zero-dependency, concurrent-safe Prometheus text-exposition registry —Counter/Gauge/Histogram(fixed buckets, cumulative_bucket/+Inf/_sum/_count),CounterVec/GaugeVecfor bounded label sets,Registry.WritePrometheus(io.Writer). Keptinternaland zero-dep by design (one-sentence rationale on the package doc comment, per PM Amendment 7) so a laterclient_golangswap is a call-site-only change.internal/daemon'snewMetricsregisters every metric on the plan's LOCKED name list:offshoot_build_info{version},offshoot_sessions_open,offshoot_capture_lag_bytes{db,branch}/offshoot_durable_age_seconds{db,branch}(open sessions only, computed at SCRAPE TIME from the sessions map via aRegistry.Collectcallback, never continuously),offshoot_flush_total{result,kind}/offshoot_flush_duration_seconds,offshoot_fork_total{path}/offshoot_fork_duration_seconds,offshoot_checkpoint_duration_seconds,offshoot_reap_total,offshoot_gc_tombstoned_total/offshoot_gc_deleted_total/offshoot_gc_backlog,offshoot_ro_cache_bytes/offshoot_ro_cache_evictions_total(registered now at zero — Task 5 wires real numbers),offshoot_janitor_runs_total{result}. Instrumentation:internal/opsgainedObserveFork/ObserveCheckpoint, package-level nil-checked injected-hook vars (ops must not importinternal/metrics) the daemon assigns once atNewServerconstruction;internal/sessiongained anOnTransitionhook fired from the SAMElogTransitioncall site Milestone 2 already used for its transition logs (not a new call site — Task 2's brief was explicit: hook the existing sites, don't restructure), which the daemon uses to drive flush counters/durations from the existing "flushed"/"flush-failed" events (andflush()now times itself and reportsduration_secondsin that same kv). Janitor-loop metrics (reap/GC/backlog/janitor_runs_total) are updated fromServer.janitorTick, split out ofStartJanitor's ticker body so a test can drive one tick deterministically. PM Amendment 7 hard gate:WritePrometheusoutput is golden-file-tested (internal/metrics/testdata/golden.txt) and validated against Prometheus's ownpromtool check metricslinter (TestPromtoolCheckMetricsininternal/metrics,TestPromtoolCheckRealMetricsininternal/daemonagainst the real, fully-wired registry) — both skip loudly (not silently) whenpromtoolisn't onPATH;.github/workflows/ci.ymlgained a newmetrics-lintjob that downloads a pinnedpromtoolrelease and runs both.internal/daemon/metrics_test.go'sTestMetricsSmokeOpenWriteFlushForkopens a session, writes, flushes, and forks over the real unix-socket RPC path, then scrapes the registry directly (no HTTP yet — that's Task 3) and asserts the counters moved. Not yet exposed over HTTP (GET /metrics, Task 3) — see docs/status.md for the exact split. -
HTTP listener + token auth (Milestone 4 Task 3):
offshoot serve -http ADDR(off by default) starts an opt-in HTTP listener (internal/daemon/http.go) alongside the unix socket:POST /rpc(the sameRequest/ResponseJSON andServer.dispatchthe socket uses — byte-identical semantics,TestHTTPRPCParityWithSocket; 1MiB body cap viahttp.MaxBytesReader,413on overflow with the connection still usable afterward;Content-Type: application/jsonrequired),GET /metrics(the T2 registry's Prometheus text exposition),GET /healthz(unauthenticated,{"ok":true,"sessions":N}), andGET /debug/pprof/*(PM Amendment 6:net/http/pprof's standard handlers, same auth as everything else). Every route but/healthzrequiresAuthorization: Bearer <token>, compared withcrypto/subtle.ConstantTimeCompare(checkAuth) — never Go's built-in==. Token source:-token/OFFSHOOT_TOKEN, elseGenerateToken(32 random bytes, hex-encoded) printed to stderr exactly once at startup (loopback binds only); ongoing output (including a later status/log line) shows only an 8-characterTokenFingerprint, never the token again —TestHTTPTokenRedaction,TestHTTPAutoGeneratedTokenPrintedOnceThenOnlyFingerprint. A non-loopback bind additionally requires BOTH-http-allow-non-loopback(an explicit ack) AND an explicit token — two distinct startup errors if either is missing (ValidateHTTPBind,TestValidateHTTPBindErrorsAreDistinct), checked before any listener is created.http.Serverruns explicit, justified timeouts (ReadHeaderTimeout5s,ReadTimeout30s,WriteTimeout90s — sized to leave headroom for/debug/pprof/profile's default 30s capture window,IdleTimeout2m) instead of the stdlib's unbounded defaults, per PM Amendment 6.Shutdown ordering: the HTTP listener is closed (
http.Server.Close, immediate — matching the unix socket's own force-close-live-connections philosophy, not a graceful drain) alongside the unix listener inServer.Shutdown. An HTTPop=shutdownrequest over/rpcgets the same respond-then-shutdown-trigger fix the unix socket'shandlegot in Milestone 3 (dispatch, encode the response, flush it to the wire, and only then triggerShutdownin a fresh goroutine) —TestHTTPShutdownRespondsBeforeClosingRequestingConn, using the same delay-hook technique as the socket's own regression test, confirmed to fail reliably against a build with the ordering reversed. A generic hammer test (TestHTTPShutdownWhileRequestsInFlightIsSafe) proves no panic or hang under-racewhileShutdownraces real in-flight requests.T2 review carried item, closed: concurrent
GET /metricsscrapes used to be able to tear the session-gauge collector's output — itResets then repopulates twoGaugeVecs (offshoot_capture_lag_bytes/offshoot_durable_age_seconds) in two steps that were only atomic if no OTHER concurrent scrape's run of the same collector interleaved with it. Fixed by addingscrapeMutointernal/metrics.Registry, serializingWritePrometheusend to end (collector run AND render) — chosen over a Server-level scrape lock or making every collector independently concurrency-safe, since it's the single minimal choke point and concurrent scrapers simply queue at ordinary Prometheus scrape cadences.internal/metrics/metrics_test.go'sTestWritePrometheusSerializesConcurrentCollectorRuns(Registry-level, synthetic) andinternal/daemon/http_test.go'sTestConcurrentMetricsScrapesWithSessionChurn(real HTTP, real session open/close churn across several branches, 8 scraper goroutines x 40 scrapes) both assert the invariant that must hold in every single scrape (offshoot_sessions_open's value equals the count ofoffshoot_capture_lag_bytessample lines) and are both confirmed to fail reliably withscrapeMuremoved.Threat model (PM Amendment 3, stated plainly in docs/reference.md): single-tenant, same-host-or-trusted-network auth — the token is a shared secret, not a multi-tenant isolation boundary; no TLS (the operator-facing
docs/operations.mdpage itself is Task 8).docs/status.md's HTTP row flips to shipped-and-tested;docs/reference.mdgains the-httpflag/route table andOFFSHOOT_TOKEN. -
Eventing: bus + socket
subscribe+ SSE (Milestone 4 Task 4a):internal/daemon/events.goadds an in-daemon event bus and two ways to drain it — the unix socket's newsubscribeop and HTTP'sGET /events(Server-Sent Events) — both streaming the exact same versioned JSON (PM Amendment 4: one schema, one encoder):{v:1, ts, type, db, branch, detail{}},typeone ofsession_opened/flushed/flush_failed/fenced/session_closed(all five fed from the SAMEinternal/sessiontransition-callback call site Milestone 4 Task 2 already hooked —internal/sessionitself is untouched, per this task's "ride the existing callback" brief) /reaped(fed from the janitor'sReappass) /evicted(schema slot reserved now; nothing emits it yet — Milestone 4 Task 5's ro-cache LRU eviction path is the intended source and will publish to the bus at its own eviction call site once it exists).Never blocks the daemon (Global Constraint):
eventBus.publishis a non-blocking fan-out — a subscriber whose bounded buffer (64 events) is already full is immediately dropped: removed from the subscriber set, sent a single terminal{type:"dropped_slow_consumer"}event, and has its channel closed, all without ever blocking the publisher (the session transition callback or the janitor).TestSlowSubscriberDroppedSessionKeepsFlushingproves this end to end — a subscriber that never reads its channel is dropped with the terminal event while a concurrently write-heavy session keeps flushing successfully throughout, unaffected.Socket
subscribe: acks, then the connection PERMANENTLY LEAVES request/response mode and streams line-per-event JSON until the client disconnects —handle()'s per-connection loop special-cases this op exactly like it already special-casesshutdown(subscribing to the bus happens before the ack is sent, closing a race where a transition could fire in the gap between ack and subscription). SDKs and any other caller MUST use a fresh, DEDICATED connection forsubscribe— it is unix-socket-only; asubscribeop sent over HTTPPOST /rpcis refused with a message pointing atGET /eventsinstead.HTTP
GET /events: same Bearer auth as everything but/healthz;data: <event JSON>\n\nper event, plus a periodic: pingSSE comment (sseKeepaliveInterval, 15s default — PM Amendment 12: proxies/kubelets kill silent streams) to keep the connection alive across anything sitting in front of this daemon. Re-arms its own per-write deadline before every write (http.NewResponseController(w).SetWriteDeadline(now + eventWriteDeadline), 45s default) rather than clearing it once, permanently — Task 3'shttp.Server.WriteTimeout(90s) covers a handler's entire wall-clock run with no concept of "still legitimately sending, just slowly," and would otherwise hard-cut every long-lived SSE stream at 90 seconds; a live stream re-arms this deadline forward on every successful write (and at least every keepalive tick), so it's never affected, while a subscriber that's still connected but has stopped reading entirely now gets its stream torn down withineventWriteDeadlineinstead of pinning the handler goroutine and its connection's file descriptor open forever. The unix socket'sstreamEventsgets the identical per-write re-arm before everyc.Write, for the same reason.TestSSEStreamSurvivesPastWriteTimeoutproves the live-stream side structurally and cheaply:httpWriteTimeoutis a test-only var shrunk to 150ms, and the stream is proven alive well past that shrunk value via a real event delivered afterward — no test ever sleeps anywhere near the real 90s.Tested (
internal/daemon/events_test.go,-race): a subscriber seessession_opened→flushed→session_closedfor a real session lifecycle over the socket; the slow-subscriber-dropped-while-session- keeps-flushing proof above; SSE/socket parity (both transports subscribed before a real session lifecycle runs, asserted to observe the identical event sequence); the keepalive ping; the write-deadline proof; a genuinely STALLED (still-connected, never-reading) subscriber on either transport has its connection/handler torn down withineventWriteDeadline, forced through the real write path with an oversized event rather than merely asserted (TestStalledSocketSubscriberConnectionIsClosedWithinWriteDeadline,TestStalledSSESubscriberConnectionIsClosedWithinWriteDeadline— both confirmed to fail reliably against the pre-fix code);/eventsrequires auth (401 without, plus a path-trick matrix extension);subscribeoverPOST /rpcis refused; reusing a subscribed connection for an ordinary op gets silence, never a Response. Nointernal/sessionchanges — bus is fed purely from the daemon-side callback composition, so no torture run was required for this task. -
Eventing: SDK stream helpers (Milestone 4 Task 4b): both SDKs ship a thin
events()helper over Task 4a's socketsubscribeop — PythonClient.events()(sdk/python/offshoot/client.py) is a generator yieldingEvent(v, ts, type, db, branch, detail)dataclass instances; TypeScriptClient.events()(sdk/typescript/src/client.ts) is anAsyncGenerator<OffshootEvent>(a new versioned interface, mirroring the wire schema). Both open their own fresh, dedicated socket connection — never the caller's ownClientconnection, matching Task 4a's MANDATORY "subscribe permanently takes over its connection" contract (see both methods' doc comments and docs/reference.md's Eventing section) — sendsubscribe, read the ack, then yield one parsed event per line in publish order until the stream ends. Stdlib-only / zero-dep on both sides (Python: onlyjson/socket, already-imported stdlib; TypeScript: onlynode:net, already-imported).dropped_slow_consumercontract, decided and documented: yielded like any other event, then the stream simply ends (Python: the generator returns, noStopIteration-wrapped error; TypeScript: the async iterator'sdone: true) — NOT raised/thrown as anOffshootError. A drop ends the stream the same way an ordinary disconnect does; a caller that cares whether it was dropped checks the last event'stype. Only a genuine transport/protocol failure (a connect error, a malformed line, a daemonok:falseack) raises/throws.Closing early, no leaked fd: Python —
break/generator.close()deliversGeneratorExitat the suspendedyield, caught by afinallythat closes the buffered reader and the socket. TypeScript —break/.return()on the async iterator resumes the generator's suspendedyieldas an abrupt completion, running an identicalfinallythatsock.destroy()s the dedicated connection. Both proved against a real daemon: a helper connection sees one real event, then closes mid-stream, and the daemon's open unix-domain-socket fd count (lsof -p <pid>filtered toTYPE unix— a raw total-fd comparison would be confounded byinternal/dbfile's deliberately-never-closed checkout descriptors) returns to its pre-subscribe baseline, and the daemon keeps answering ordinary ops on an unrelated connection throughout.Real end-to-end lifecycle tests (
sdk/python/tests/test_client.py'stest_events_sees_session_lifecycle_in_order,sdk/typescript/test/ client.test.ts's matching test): the helper subscribes on its own connection;session_opened→flushed→session_closedis driven by real ops on a SEPARATE connection and observed in order with correctdb/branch/detailfields. Thedropped_slow_consumerpath is hard to force deterministically from the SDK side (it needs the real bus's internal buffer-overflow timing — already proven server-side by Task 4a'sTestEventBusDropsSlowSubscriberWithTerminalEvent); both SDKs instead unit-test the helper's own decode/contract path end to end against a small scripted fake daemon speaking the identical line-per-event wire shape (TestEventsDecodePath/ theevents(): dropped_slow_consumer ...node:testcases), covering the terminal event, a malformed line, and a not-ok ack.Both SDK suites green under
make test-sdks; both SDKs confirmed stdlib-only/zero-dep (import inspection, no new dependency added to eitherpyproject.tomlorpackage.json).docs/status.md's eventing row flips to shipped-and-tested;docs/reference.mdgains thesubscribeop andGET /eventsroute, with the dedicated-connection warning restated for operators. -
Ro-cache disk budget + LRU eviction (Milestone 4 Task 5):
offshoot serve -ro-cache-budget <bytes|0>(default0= unlimited) boundscheckouts-ro's total size — the read-only cacheCheckoutAt(Milestone 3 Task 2) materializes into and which, until now, grew without bound. The janitor'sServer.janitorTick(internal/daemon/server.go) runs a new ro-cache pass on the same-reap-everycadence as reap/GC: it always computes and republishes current usage (offshoot_ro_cache_bytes, the T2-registered gauge, previously always 0) — a budget-less or already-under-budget pass still reports a real number, it just never evicts — and, once usage exceeds a configured budget, LRU-evicts entries (oldest first) until it's back under. A bare integer is bytes (the contract);-ro-cache-budgetandstatus's own display flag of the same name also accept a trailing power-of-1024 size suffix (K/KB,M/MB,G/GB,T/TB) as a convenience (parseByteSize,cmd/offshoot/main.go) — tested directly (TestParseByteSize) since the brief called out "if you add size parsing, test it."LRU clock (PM Amendment 11, pre-decided): a
.last-usedtouch-on-HIT marker file, not the.dbfile's own mtime —CheckoutAt's force=false cache-hit path (internal/ops/export.go) now callstouchLastUsed, which creates/Chtimess a<cachefile>.last-usedsidecar to the current time on every cache HIT. This is necessary, not cosmetic:materializeAt's rename-into-place is the LAST thing that ever touches the.dbfile's own mtime (a cache hit is a pure read — it must never touch the.dbfile again), so without a separate marker "least recently used" would collapse to "least recently created," exactly backwards for a cache whose whole point is that a repeatedly-hit checkpoint should stay hot.internal/ops/rocache.go'slruClockranks by the marker's mtime when present, falling back to the.dbfile's own mtime as the documented floor for an entry materialized but never since hit.TestEvictROCacheLastUsedTouchBeatsCreationOrderand its daemon-level counterpartTestJanitorTickEvictsLRUUnderBudgetAndFiresEvictedEventboth prove this directly: an OLDER (by creation) entry that gets hit survives eviction while a NEWER, never-hit entry is evicted instead.checkouts/(writable, leased) is never evicted — by construction, not a runtime check:ops.Workspace.EvictROCacheonly ever walks and removes paths under the separatecheckouts-rotree (never joined with, or reachable from,checkouts/'s own path shape — seeCheckoutAtPath's existing doc comment on that separation); there is no code path here that can construct or be handed acheckouts/path at all.TestEvictROCacheNeverTouchesWritableCheckout(ops-level, budget=1 as aggressive as it gets) andTestJanitorROCacheEvictionNeverTouchesLeasedWritableCheckout(daemon-level: a real open, leased session survives an aggressive pass and can still flush afterward) both confirm this end to end, not just by code inspection.Eviction is LOUD: one
offshoot: janitor: ro-cache: evicted <db>@<branch>@<checkpoint> (<bytes> bytes)stderr line per eviction,offshoot_ro_cache_evictions_total(the T2-registered counter, previously always 0) incremented, and Task 4a's reserved-but-unwiredevictedevent type now has its emitter —janitorTickpublishes{type:"evicted", db, branch, detail:{checkpoint, bytes}}to the T4a bus at the eviction call site (eventBus.publishis non-blocking, so this can never stall the janitor loop).offshoot statusgains an ro-cache usage summary line (ROCacheUsage,internal/ops/rocache.go) plus an optional-ro-cache-budgetflag of its own — display-only, since (like every otherservetuning flag) the budget is never persisted, so this at-rest CLI command has no other way to show usage against it without a live daemon connection.Each eviction removes both the
.dbcache file and its.last-usedmarker together. Partial-progress-plus-first-error on a mid-passos.Removefailure, matchingReap/GC's own convention elsewhere ininternal/ops.Tests:
go test ./internal/ops ./internal/daemon -count=1 -race—internal/ops/rocache_test.go(usage accounting, budget=0 never evicts, already-under-budget never evicts, oldest-first eviction to under budget, both files removed together, the writable-checkout and touch-beats-creation-order proofs above) andinternal/daemon/rocache_test.go(the same LRU/budget-zero/writable- checkout scenarios driven through a realjanitorTickpass, plus the metrics move and theevictedevent fires withdb/branch/checkpoint/bytes). Nointernal/session/internal/capturechanges — janitor/ops only — so no torture run was required for this task.docs/status.md's Resource-behavior row flips to shipped-and-tested;docs/reference.mdgains-ro-cache-budget, the LRU-clock rationale, and reiterates the writable-never-evicted guarantee; theevictedevent row in the events table loses its "reserved" caveat. -
serve -snapshot-every N(Milestone 4 Task 6a): plumbs the embeddable session library's existingOptions.SnapshotEverycadence into every sessionoffshoot serveopens, the same way-flush-everyalready plumbsFlushEvery—Server.SetSnapshotEvery(internal/daemon/server.go), read byopOpenunder the same single- writer-before-Servecontract. Default16, unchanged if the flag is omitted (SetSnapshotEveryis simply never called, sosnapshotEverystays its zero value andsession.Openapplies its own default); rejects a value< 1at the flag-parsing layer — unlike-flush-every 0, there is no "disabled" sentinel, since every flush must eventually snapshot.internal/session.DefaultSnapshotEveryis now exported socmd/offshoot's error message references the same number rather than a second hardcoded copy of it.Tests:
internal/daemon/snapshot_every_test.go(TestServeSnapshotEveryWiringDrivesCadence) drives 9 flushes over the socket withSetSnapshotEvery(4)and asserts a boundedstore.Chain(≤4 members) plus more than one snapshot object across the lineage listing, proving the cadence recurs under daemon wiring and not just at session start;cmd/offshoot/main_test.go(TestServeNonPositiveSnapshotEveryIsRejected) pins0/negative/ non-integer as usage errors.go test ./internal/ops ./internal/store ./internal/daemon ./cmd/offshoot -count=1 -raceclean; nointernal/session/internal/capturechanges, so no torture run was required.docs/status.md's bounded-replay and tuning-surface rows flip to shipped-and-tested;docs/reference.mdgains-snapshot-everyand documents the flush-cost trade-off (fewer segments per snapshot = more frequent full-database uploads but cheaper/bounded reads, and vice versa); README's "What a flush costs" section updated to match. -
CAS-conditional ref delete (Milestone 4 Task 6b):
ops.Workspace.Destroynow CAS-writes aDeletingclaim (store.Ref.Deleting/DeletingAt) on the ref before doing anything irreversible, closing theGetRef-> lease-check -> unconditional-DeleteRefTOCTOU an earlier design review documented forDestroy— a lease acquired in that window could previously have its branch deleted out from under it.store.AcquireLeaserefuses outright (store.ErrDeleting, retryable) once it sees the claim, the same way it already refusesReap'sReapingclaim.--forcebypasses the protected/live-lease pre-checks only, never the claim: a lease that wins the underlying CAS still survives a concurrent forced destroy.Scoped per the task's own one-review-cycle timebox (PM Amendment 9) as a sibling claim field, not a unification with Reap's existing
Reapingclaim —Reaping's CAS mechanics are torture/race-tested and deliberately untouched (internal/ops/reap.go/its test files pass unmodified).Backend split (PM Amendment 5 — no pretending S3
DeleteObjecthas preconditions it doesn't): a newstore.ConditionalDeleteroptional capability interface (DeleteIf(key, ifMatch) error) backsStore.DeleteRefIf. Local implements it for real (Local.DeleteIf, the same per-key lock filePutIfalready uses) — a true compare-and-delete, belt-and-suspenders on top of the claim. S3 does not implement it at all (DeleteObjectignoresIf-Match/If-None-Match; those headers are GET/PUT-only there), soDeleteRefIffalls back to an unconditional delete on S3 and the claim marker is the entire safety mechanism on that backend — documented as such onS3.Delete, not silently pretended away.A Destroy that crashes after claiming but before deleting leaves the claim stranded (branch untouched, no partial delete);
ops.Workspace .ClearStaleDeleteClaimsself-heals it by age (30s — a delete claim has no TTL/deadline concept the way Reap'sReapDeadlinegives its own claim one), wired into both the daemon janitor (janitorTick) and the CLIoffshoot gc, same "report and press on" convention as a reap failure.Tests:
internal/ops/destroy_claim_test.go'sTestConcurrentDestroyAndAcquireLeaseHaveExactlyOneWinner(20-iteration race loop mirroring M2'sTestConcurrentTouchAndReapHaveExactlyOneWinner: exactly one of Destroy/AcquireLease wins, the branch is never left half-deleted under a live lease),TestForceDestroyStillClaimGuards(same race withforce=true),TestDestroySelfHealsStaleDeletingClaim;internal/store/local_test.go'sTestLocalDeleteIfConditionalDelete/TestStoreDeleteRefIfUsesConditionalDeleteOnLocal;internal/daemon /destroy_claim_test.go'sTestJanitorTickClearsStaleDeleteClaimthrough a real janitor pass.go test ./internal/ops ./internal/store ./internal/daemon ./cmd/offshoot -count=1 -raceclean; nointernal/session/internal/capturechanges, so no torture run was required.docs/status.md's TTL/GC/janitor table gains a shipped-and-tested row;docs/reference.md'soffshoot destroysection documents the claim-guard and the backend split. -
SDK typing polish (Milestone 4 Task 7, pre-publish list) — behavior- preserving, no public signature or wire changes.
TypeScript (
sdk/typescript/src/client.ts): the internal_callplumbing no longer returnsPromise<any>. A module-privateRawResponseinterface (plusRawBranchInfo/RawCheckpointInfo/RawSessionInfo/RawAck/RawEvent) mirrorsinternal/daemon/protocol.go'sResponseand its embedded structs exactly as JSON arrives off the wire — every_callconsumer (open/checkout/fork/rollback/promote/branches/dbs/checkoutAt/status/Session.flush, plusevents's ack/event-line parsing) now reads a real, typed field instead ofany, with the exact same??/non-null-assertion defaulting each call site already had (no behavior change — verified via a before/afterdist/ client.jsdiff showing only added comments)._callitself is tagged@internaland stripped from the published.d.tsviatsconfig.build.json's newstripInternal: true— it remains a real, callable, fully-typed method at runtime and for this package's own test files (which compile against source, not the stripped declaration output), sincetest/client.test.tsoverrides it directly as a test double. Verified additive: adist/client.d.tsbefore/after diff shows exactly one removal (the now-@internal_callline) and zero changes to any other exported symbol;dist/testkit.d.tsis byte-identical.Python (
sdk/python): added the PEP 561offshoot/py.typedmarker, shipped via a new[tool.setuptools.package-data]entry inpyproject.toml(verified present in the dry-run wheel viaunzip -l).mypy --strict offshootis clean across all four modules (client.py/langgraph.py/pytest_plugin.py/__init__.py) — fixed with annotations only: missing parameter/return types,dict/Popengeneric type arguments, a module-private_TTL/_Seedtype alias each inclient.py/pytest_plugin.py,typing.cast()(a no-op at runtime) on JSON-derived returns that were otherwise widening toAny, and aMapping[str, str]narrowing for_locate_binary'senvparameter (os.environisn't literally adict).pytest_plugin.py's guardedimport pytesttype-checks cleanly against pytest's own inline types (pytest ships PEP 561 support; no stub package ortype: ignoreneeded).Tests:
make test-sdks(Python 41/41, TypeScript 49/49) andmake test-pytest-plugin(58/58) all green;tsc --noEmit(bothtsconfig.jsonandtsconfig.build.json) clean;mypy --strict offshootclean; dry-run wheel/sdist (make dry-run-python-sdk) and tarball (make dry-run-ts-sdk) both pass unchanged — no file-list updates needed on either SDK. No Go changes. -
Operator docs + Kubernetes recipe + the M4 sweep (Milestone 4 Task 8): docs/operations.md is the new operator-facing page — a metrics reference table (all sixteen
offshoot_*metric families, grep-verified againstinternal/daemon/metrics.goand against a realcurl /metricsscrape of a locally built binary: sixteen# TYPElines, sixteen table rows, one-to-one), the six-state branch-state table with precedence, the eight-type event schema and drop-slow-consumer semantics, the-ro-cache-budgetmechanics (LRU.last-usedclock, writable-never-evicted guarantee, the eviction-vs-CheckoutAtTOCTOU), the HTTP/auth threat model (single-tenant same-host-or-trusted-network, not multi-tenant isolation; constant-time token compare; loopback default with an explicit non-loopback ack + token requirement; "treat stderr as sensitive at startup";-token's argv-visible-via-psgap and whyOFFSHOOT_TOKENis preferred on a shared host;/debug/pprofbehind the same auth), and a tuning-flags table for all fiveserveknobs including the-flush-every/-snapshot-everycost interaction. Its first paragraph restates the single-node scope explicitly (PM Amendment 12) and links the FAQ's no-cluster stance.docs/recipes/kubernetes.md is a new sidecar recipe: one
offshoot serve+ one agent container per Pod, a sharedemptyDirfor BOTH the unix socket and the checkout tree (restated why: a checkout is a real SQLite file the agent process opens directly —OFFSHOOT_CHECKOUTS/the socket path cannot cross a network filesystem),-http 127.0.0.1for/metrics//healthzwithexec-based liveness/ readiness probes (anhttpGetprobe is dispatched against the Pod IP, which a loopback-only bind never answers), a Prometheus scrape-annotation example with the honest caveat that it needs a deliberate non-loopback opt-in (-http-allow-non-loopback+ a real token) to actually be reachable from outside the Pod, and an explicit "no StatefulSet-for-HA story" scope note linking the FAQ. The manifest (docs/recipes/k8s/offshoot-sidecar.yaml) is real and apply-able, not pseudocode: schema-validated with bothkubectl apply --dry-run=clientand--dry-run=serveragainst a disposable localrancher/k3scontrol plane (no live cluster was otherwise available), both passing.Sweep: README gained an operator paragraph under "Daemon mode" (metrics/ HTTP/events/budgets, linking both new docs) and a fifth "operator surface" note in "Integration surface"; its stale "Resource behavior" claim that "there are no resource budgets yet" is corrected now that Task 5's
-ro-cache-budgetshipped.docs/reference.mdcross-linksdocs/operations.mdfrom its-http ADDRand eventing sections rather than duplicating the operator-focused tables.docs/status.mdgained three new deliberately-deferred rows this milestone's self-review named but hadn't yet recorded (TLS, per-branch at-rest metrics by default, metrics push/remote-write), reworded the FD-budget row from "not yet implemented" to "deliberately deferred" with theinternal/dbfilereasoning, retired two stale forward-references to "the operator-facing docs/operations.md page itself is Task 8" now that the page exists, and gained a new "Standing nag: user-gated launch items" section consolidating every button-press-only action (org transfer, PyPI/npm name claims, registry listing submissions, domain/Homebrew/container-image claims) that no further engineering resolves.ROADMAP.md's Milestone 4 bullets are checked off against what actually shipped.Verification: every command in every changed/new doc was run verbatim, not transcribed from memory — the metrics table's scrape example is a real
go build+serve -http+curl /metricsrun; the Kubernetes manifest was actually dry-run-applied (see above) rather than merely YAML-parsed.make test-sdksand a fullgo test ./... -count=1both green (no code changed this task — docs and CHANGELOG/ROADMAP/status.md only).
[0.1.2] - 2026-08-06
Milestone 3: the eval-harness release. The target persona's first hour is
now paved end to end — install, seed once, fork per test, inspect, export,
clean up — from Python (offshoot.pytest_plugin) or TypeScript
(testkit), with docs/eval-harness.md as the
serious tutorial and framework recipes for Claude Code, the OpenAI Agents
SDK, LlamaIndex, and CrewAI in docs/recipes/. New protocol
surface (dbs, metadata/timestamps, export, read-only historical
checkouts) lands first, the fixture plugin and TS testkit are built on top
of it, and offshoot diff closes the daily "what changed between these two
attempts" loop. Two Milestone-2 performance follow-ups (settling-flush
suppression; sidecar refresh on clean close) ride along because the
fixture's session-per-test pattern is exactly the workload they fix. SDK
publishing is prepared (workflows, manifests) but actual PyPI/npm
publication, and the MCP registry/LangGraph listing submissions, stay
user-gated — see docs/status.md and
ROADMAP.md's Milestone 3 section for exactly what shipped vs.
what's a stated, pre-written deferral.
Added
- Docs sweep (Milestone 3 Task 8): docs/eval-harness.md,
the serious tutorial — install, the pytest plugin end to end (named-seed
factory, fork-per-test,
pytest-xdistwith the measured per-worker number, mid-test flush checkpoints, golden assertions viaoffshoot_dump— never byte-compare, why), export for handoff/debug,offshoot difffor the failed-vs-passed loop, TTL hygiene, a CI recipe (this repo's ownci.ymlsdksjob as a live example), what it costs, and the TypeScripttestkitsection mirroring the same semantics for vitest/jest/node:test. Every pytest example in it was actually run against a real daemon while writing it (a scratch project, real output pasted in, not invented); the TypeScripttestkitsection'snode:testexample was likewise run against a real build ofsdk/typescript.sdk/python/README.md's pytest-fixture-plugin section is superseded by the tutorial as the primary teaching surface and now points to it, staying as the PyPI-landing-page-sized condensed reference.docs/recipes/claude-agent-sdk.md:claude mcp addconfig foroffshoot mcp, and a hooks pattern (fork onSessionStart, checkpoint onStop, roll back on failure) — explicitly checked against the MCP daemon-mode reality documented in the README's MCP section anddocs/reference.md:offshoot mcpnever opens a session itself, so the recipe's actual content is opening the session via the CLI/SDK before the agent's first tool call, with the hooks shown as one way to automate that open/close, not a substitute for it. The hooks JSON shape itself is marked illustrative (Claude Code's own surface, not offshoot's, and outside what this task could pin the way the rest of this repo's docs are).docs/recipes/openai-agents.md: the OpenAI Agents SDK'sSQLiteSession(session_id, db_path)pointed at an offshoot checkout path, fork-per-attempt via the Python SDK. Actually run end to end against a realpip install openai-agents(no API key needed to prove the storage integration —add_items/get_itemsstand in forRunner.run); real output pasted in, including two forked attempts' histories staying independent after diverging from a shared checkpoint.docs/recipes/frameworks.md: short, honest LlamaIndex/CrewAI notes — recipes, not adapters, per the plan's stance. Explicitly marked unverified against a live install this pass (LlamaIndex's SQL chat store needs theaiosqliteextra; CrewAI's package failed to build locally —tiktoken, one of its transitive dependencies, has no prebuilt wheel for this Python/platform combination) rather than presented with false confidence.- README: integration-surface refresh — pytest plugin, TypeScript
testkit,export/checkout_at/dbsall get their own lines under the Python/TypeScript SDK sections; a new "Other agent frameworks" section points atdocs/recipes/; the Docs line and Quickstart section both gain pointers to the new tutorial. docs/reference.md: verified complete forexport/diff/checkout --at(documented per-task already; no gaps found) — added one pointer line at the top to the tutorial for readers who want the narrative walkthrough instead of the flag-by-flag reference.docs/status.md: two new deferral rows, both pre-written per PM Amendment —create --fromdaemon/SDK/MCP reach (needs an upload channel or a same-host path-trust story likeexport's; CLI remains the import path) and MCP session open/close (the fixture plugin and TS testkit are now real lifecycle owners for their own harness workload, but neither is an MCP tool pair — the no-natural-owner reasoning from Milestone 2's amendment stands, confirmed rather than just carried forward).ROADMAP.md: Milestone 3 checked off item-by-item against what actually shipped, with the two deferrals above plus listings submission and actual SDK publication called out as pre-written, user-gated deferral rows rather than silently-dropped scope; Milestone 2's "MCP rides the daemon" bullet updated to note the fixture/testkit now exist without overclaiming they closed the MCP-tool-pair gap.
- List databases (Milestone 3 Task 1): new daemon protocol op
dbsreturns every database the store has at least one ref for (store.Store.ListRefs's keys, sorted). CLI:offshoot session dbs. SDK: PythonClient.dbs() -> list[str], TypeScriptClient.dbs(): Promise<string[]>. - Branch/checkpoint metadata and timestamps (Milestone 3 Task 1):
Ref.Meta map[string]string(branch-level lineage metadata) and per-checkpointstore.Checkpoint.CreatedAt/Meta— all new, omitempty fields, no store schema bump (verified with a round-trip + old-ref/ old-checkpoint decode test mirroring the existing TTL-fields test).ops.Workspace.Fork/Checkpointgain ameta map[string]stringparam (nil= none), capped at the ops layer via the newops.ValidateMeta(at most 32 keys, keys ≤ 64 bytes, values ≤ 512 bytes — Global Constraints; clear errors naming the exact limit hit).Fork's meta describes the new branch's lineage (Ref.Meta);Checkpoint's meta describes that one checkpoint (Checkpoint.Meta). Every checkpoint- creating call site (Create'sinit,Checkpoint,Fork'sfork,Promote'spromote, and a daemon session's namedflush) now stampsCreatedAt(RFC3339 UTC).Rollback's kept-checkpoint relocation used to silently dropCreatedAt/Metawhen rewriting a checkpoint's epoch to the new lineage's1— fixed to preserve both.- Daemon protocol:
BranchInfogainstouched_at(the ref's activity- clock stamp) andcheckpoints_v2([]{name, txid, created_at}) — the existingcheckpoints(bare names) field is untouched for wire compat.forkandflushops gain an optionalmetafield; there is no separate daemon "checkpoint" op — a live session's namedflushis how its checkpoints are created (parity note:flush'smetais rejected ifnameis empty — there's no checkpoint for it to attach to). All new fields are additive/optional: an old-client-shaped request (nometa, reading onlycheckpoints) still works against the new daemon, pinned by wire-compat tests using hand-written JSON (not just a Go zero-value round trip). - CLI:
fork/checkpointgain repeatable--meta k=v. - SDK parity (Python + TypeScript):
fork/Session.flushacceptmeta;branches()/Client.branchessurfacetouched_at/checkpoints_v2(newCheckpointInfo/Branchfields). - MCP tool metadata exposure is explicitly out of scope for this task —
offshoot_fork/offshoot_checkpointpassnilmeta; seedocs/status.md.
- Publish pipeline, prepared and gated (Milestone 3 Task 7): the SDKs
are ready to publish; actual publication needs the user to claim the
offshoot-dbPyPI name and@offshoot-dbnpm scope first (see CONTRIBUTING.md's new Release process section)..github/workflows/publish.yml: triggered onsdk-v*tags orworkflow_dispatch. Two jobs, PyPI (pypa/gh-action-pypi-publish, Trusted Publishing/OIDC,id-token: write) and npm (npm publish --provenance, registry auth viaNPM_TOKENtoday — npm's own OIDC Trusted Publishing is documented as the future swap). Both gated on thePUBLISH_ENABLEDrepository variable: off (the default) runs a full dry run — real sdist/wheel +twine check+ wheel install/import test; realnpm packtarball + install/import test — everything short of the upload step. The same dry-run tier now runs inci.yml'ssdksjob on every PR (make dry-run-sdks), so a manifest mistake is caught long before a release tag exists.sdk/python/pyproject.tomlfilled out for real publication: readme,project.urls(Homepage/Repository/Issues/Changelog/Documentation), authors, classifiers, SPDXlicense = "Apache-2.0"(no redundantLicense ::classifier — current PEP 639 practice), reserved[pytest]extra ahead of Milestone 3 Task 4's fixture plugin.sdk/python/README.mdadded (PyPI landing page).sdk/typescript/package.jsonfilled out:repository/bugs/homepage,fileswhitelist (dist,README.md— excludes tests/ tsconfig from the published tarball, verified withnpm pack --dry-run),publishConfig.access: public(required for a scoped package to publish non-private),prepublishOnlybuildsdist/before packing.sdk/typescript/README.mdadded.- Version discipline: both SDKs publish in lockstep from one
sdk-v<version>tag (not twosdk-py-v/sdk-ts-vtags) — simplest scheme for two SDKs on one wire protocol and one review cadence.sdk/VERSIONis the single source of truth;pyproject.toml's andpackage.json's literal version fields are checked against it by the newscripts/check_sdk_versions.py(make check-sdk-versions, and the first step ofmake dry-run-python-sdk/CI);scripts/ check_sdk_tag_version.pychecks a pushed tag against the same file. server.json(repo root): draft MCP registry manifest built only from fields this repo's own MCP docs establish (name/description/version/ repository/command/args) — not submitted; the exact registry schema was deliberately not assumed from outside the repo, seedocs/launch/mcp-registry.mdand the TODO row indocs/status.md.docs/launch/langgraph-listing.md: LangGraph community-integration PR text (title, description, listing-table entry) foroffshoot.langgraph.ThreadForks— drafted, clearly marked not submitted (blocked on PyPI).
- Export + read-only historical checkouts (Milestone 3 Task 2):
ops.Workspace.Export(db, branch, checkpoint, dstPath, force)materializes any checkpoint (or head, whencheckpoint == "") to a plain SQLite file atdstPath, anywhere on the local filesystem, with zero ongoing relationship to the store afterward: no.sumsidecar, no lease. Refuses to overwrite an existingdstPathunlessforce. ReusesmaterializeChainAt/materializeAt's existing atomic temp-file-in-the-destination's-own-directory + rename, so a failed export (fetch error, checksum mismatch) never leaves a partial or truncated file, and discards thePostApplyChecksumthat machinery now threads through (Task 3) since there is no sidecar to stamp with it.- CLI:
offshoot export <db>[@branch[@checkpoint]] <out.db> [--force](ops.ParseExportTargetparses the triple-@target form). - Daemon
exportop:db/branch/name(checkpoint)/path(destination)/force.pathmust be an ABSOLUTE path — refused otherwise — per the same-host/same-user unix-socket trust model documented indocs/reference.md's new daemon-ops section. Reads the branch's last DURABLE state from the store, never a live session's checkout: an open session's unflushed writes are NOT included, proven directly over the wire (internal/daemon/export_test.go'sTestOpExportMissesUnflushedSessionWrites). ops.Workspace.CheckoutAt(db, branch, checkpoint, force) (string, error)materializes a NAMED checkpoint (no head alias) into a dedicated read-only cache path,<store-root>/checkouts-ro/<db>/ <branch>@<checkpoint>.db,chmod 0444, distinct from and never touching the writable checkout path, its sidecar, or a live capture engine's file descriptors on it — safe alongside an open daemon session on the SAME branch. A repeat call withforce=falseis a pure cache hit (no store access at all — a checkpoint's content is immutable);force=truere-materializes and re-reads the store.- CLI:
offshoot checkout <db>[@branch] --at <checkpoint> --read-only [--force](must be given together). - Daemon
checkout-atop: same semantics, server-side cache path. - SDK parity (Python + TypeScript):
Client.export(db, branch, out_path, checkpoint=None, force=False),Client.checkout_at(db, branch, checkpoint, force=False)(TypeScript:checkoutAt, options-object style matching the rest of that client). - README's Resource behavior section gains the read-only-checkout-cache
paragraph, including the explicit "safe to
rm -rfthe entirecheckouts-rodirectory at any time" guarantee.
offshoot.pytest_pluginfixture plugin (Milestone 3 Task 4): the seed-once-fork-many paved road for pytest-based eval/test suites, shipped as theoffshoot-db[pytest]package extra and registered via apytest11entry point (sdk/python/offshoot/pytest_plugin.py) — nothing to import by hand.offshoot_daemon(session-scoped): locates theoffshootbinary (OFFSHOOT_BINenv, elsePATH), starts it on a fresh temp store + socket, terminates it at session end;pytest.skips with install instructions when no binary is found, rather than failing.offshoot_db(session-scoped): a NAMED-SEED FACTORY,offshoot_db(name="default", seed=None). The first call for a name creates databaseeval-{name}, runsseed(a callable given a writable sqlite path, or a SQL string run viasqlite3), and flushes it to a checkpoint namedseed; later calls for the same name are a pure memoization hit.seed=Nonefalls back to the newoffshoot_seedini option (a path to a.sqlfile) — the zero-code default-seed case.offshoot_fork(function-scoped):offshoot_fork(seed_handle=None)forks a fresh, worker-safe-named branch (t-{worker}-{testname-hash}- {n}, sanitized via the existingoffshoot.langgraph._sanitize) from the seed checkpoint with a TTL (default1h, newoffshoot_ttlini option overrides it), opens a session, and returns an object with.path/.client/.db/.branch. Teardown closes the session then destroys the branch; a destroy failure is aUserWarning, never a test failure — the TTL is the crashed-run backstop.offshoot_dump(path) -> str:sqlite3 .dumptext — THE way to compare two offshoot-materialized SQLite files in a golden-file test. SQLite's on-disk bytes are not deterministic for identical logical content (page layout, freelist order, vacuum state can all differ) — never byte-compare two exported/checked-out.dbfiles.- xdist stance (locked): one
offshootdaemon + temp store PER WORKER (pytest has no cross-worker fixture-sharing mechanism), so a named seed's cost is paid once per worker, not once total. Measured (aCREATE TABLE+ 200-row seed, macOS arm64): ~80-90ms per worker; a 2-workerpytest -n2run pays that twice, concurrently (~170ms of total seed work, ~85-90ms of wall-clock time). Documented in the module's docstring andsdk/python/README.md. - Found+fixed in passing: a SQL-string seed with no explicit
BEGIN/COMMITused to run every statement as its own autocommit transaction undersqlite3.Connection.executescript— measured ~1.6s for 200 unwrappedINSERTs vs. ~17ms wrapped in one transaction (~100x)._run_seednow wraps a bare multi-statement seed in one transaction automatically (a seed that already opens its own transaction, or a seed callable, is left alone). - Base SDK stays stdlib-only: the
pytest11entry point is registered unconditionally inpyproject.toml, but the plugin module import-guardspytest(a stray direct import without the extra fails with an install-instruction message, not a bare traceback) —pytestis only ever actually imported when pytest itself is already running. Verified:make test-sdks(the plain-unittestsuites) passes with no pytest installed at all. - Tests:
sdk/python/tests/test_pytest_plugin.py— fixture-logic tests (naming, TTL, teardown ordering, factory memoization, skip-when-no- binary, destroy-failure-warns) against a directly started daemon, plus 3pytester-driven smoke scenarios (plugin loads via its entry point, fork-per-test isolation actually isolates, an xdist 2-worker run passes — the last one is also where the xdist numbers above were measured). Newmake test-pytest-plugintarget (needsoffshoot-db[pytest]+pytest-xdist, wired intoci.yml'ssdksjob aftermake test-sdksalready proved the base SDK pytest-free).
- TypeScript testkit (Milestone 3 Task 5):
sdk/typescript/src/testkit.ts— the vitest/jest counterpart ofoffshoot.pytest_plugin, exported as a new subpath,@offshoot-db/client/testkit(package.jsonexportsmap added; previously only a baremain/typespair). Framework-agnostic FUNCTIONS, not fixtures — no vitest/jest runtime dependency (this SDK stays zero-runtime-deps) and nothing is registered automatically; wire them into your ownbeforeAll/afterEach.startDaemon(opts?) -> Promise<DaemonHandle>: locates theoffshootbinary (OFFSHOOT_BINenv, elsePATH— a clear error naming both when neither has one; there is no skip tier here, unlike the pytest plugin, since this module has no test-framework integration to skip through), starts it on a fresh temp store + socket. Returns{ sock, store, proc, stderrTail(), stop() }; the caller stops it themselves (typically from a top-levelafterAll).seedOnce(daemon, {name?, seed}) -> Promise<SeedHandle>: a NAMED-SEED memoization cache keyed on(daemon, name).seedis a SQL string, a path to a.sqlfile (detected: no newline, ends in.sql, and the file exists — there's no pytest.ini-style config file here to hold that decision separately), or an async(dbPath) => voidcallback. The first call for a name creates databaseeval-{name}, runs the seed, and checkpoints itseed; a later call for the same name with a different seed (fingerprinted: SQL/path text by content hash, callables by identity) throws a clear error rather than silently keeping the first one — same mismatch semantics as the pytest plugin'soffshoot_db. Ports_skip_leading_noise/_seed_opens_own_transactionverbatim: a SQL-string seed is wrapped in one transaction unless it already opens its own (so asqlite3 .dump's text —PRAGMAbeforeBEGIN TRANSACTION— works verbatim as a seed, and so a plain multi-statement seed doesn't pay one autocommit-transaction-per- statement). Since this SDK ships no SQLite driver, seeding (anddump, below) shells out to thesqlite3CLI, same astest/client.test.tsalready does.forkPerTest(daemon, seedHandleOrName, opts?) -> Promise<ForkedSession>: forks a fresh, worker-safe-named branch (t-{worker}-{sanitized-hint}-{n}; worker id fromVITEST_POOL_IDorJEST_WORKER_IDwhen present, else"local") from the seed's checkpoint with a TTL (default"1h",opts.ttloverrides), opens a session, and returns{ path, db, branch, client, flush(name?), close() }.seedHandleOrNamemay be aSeedHandleor a plain string naming an already-seedOnced name.close()closes the session and destroys the branch; either failing is aconsole.warn, never a throw — call it fromafterEachwith no surrounding try/catch. EachForkedSessionowns its own connection and teardown independently, so (unlike the pytest plugin's shared-per-test_ForkFactory.teardown()loop) one fork's cleanup trouble can never block another's by construction, not by an explicit try/except around each step.dump(path) -> Promise<string>:sqlite3 <path> .dump's text — THE golden-comparison method. SQLite's on-disk bytes are not deterministic across writes with identical logical content — never byte-compare two exported/checked-out.dbfiles; compareawait dump(a) === await dump(b)instead.- Tests:
sdk/typescript/test/testkit.test.ts, run vianode:testagainst a real daemon (mirroring the pytest plugin's direct-daemon tier): binary resolution (OFFSHOOT_BIN/PATH/neither), naming, TTL default+override,seedOncememoization + fingerprint mismatch, a.dump-shaped seed, a path-to-.sqlseed, an async-callback seed, fork-per-test isolation, the string-name shorthand, teardown-warn-not- throw against a killed daemon, and a golden-file (dump, not bytes) scenario — plus one integration test wiringstartDaemon/seedOnce/forkPerTestintonode:test's ownbefore/after/beforeEach/afterEach, the way a real suite would. Wired into the existingmake test-ts-sdktarget (it already globstest-dist/test/*.test.js) — same zero-extra-dependency constraint (typescript/@types/nodedevDeps only). make dry-run-ts-sdk's tarball exact-match assertion updated to expectdist/testkit.js/dist/testkit.d.tsalongside the existingdist/client.*files, and a second install-then-import()check added for the new@offshoot-db/client/testkitsubpath (not just the package root).
offshoot diff(Milestone 3 Task 6):offshoot diff <db>[@branch[@checkpoint]] <db>[@branch[@checkpoint]] [--summary]materializes both sides READ-ONLY (never a live checkout, never a lease) and either streamssqldiff's output over them or prints asqldiff-free table-level row-count summary. CLI-only for this task — no daemon op, no SDK parity.ops.Workspace.MaterializeForDiff(db, branch, checkpoint) (DiffSide, error): a named checkpoint reuses the existingCheckoutAtread-only cache (a checkpoint's content is immutable, so this is a legitimate cache hit across repeated diffs); a bare head target is always freshlyExported to a private temp file, because head moves and a head-keyed entry in the checkout-at cache could never be idempotently cached the way a checkpoint-keyed one is — this is the task's staleness decision, documented onMaterializeForDiff's doc comment and pinned byinternal/ops/diff_test.go'sTestMaterializeForDiffHeadSideAlwaysReflectsANewWrite(a write between two calls against the same head target is visible in the second one, never served stale) and the CLI-levelTestDiffCLIHeadSideReflectsNewWriteNotStaleCache.DiffSide.Close()is a no-op for the cache-backed checkpoint case and removes the temp file/directory for the head case.ops.TableRowCounts(path) (map[string]int, error)/ops.DiffSummary(leftPath, rightPath) ([]TableDiff, error): the--summaryengine, stdlib-database/sql+ themattn/go-sqlite3driver already vendored forinternal/capture/internal/ops(no new Go module dependency), opened asfile:<abs>?mode=ro&immutable=1— verified against a realchmod 0444file (TestTableRowCountsOpensReadOnlyEvenOnA0444File) that SQLite accepts amode=roURI parameter layered on top of the driver's always-requestedSQLITE_OPEN_READWRITE|SQLITE_OPEN_CREATEflags (SQLite only rejects a mode LESS restrictive than the flags argument passed tosqlite3_open_v2;rois strictly more restrictive). Lists tables fromsqlite_master(excludingsqlite_%internals) and counts rows per table on both sides; a table present on only one side is reportedadded/removedwholesale rather than as a row-count delta. The two materialized paths may be two entirely different databases — cross-db diff is a legitimate eval- comparison shape and is explicitly tested (TestDiffSummaryCLIWorksAcrossTwoDifferentDatabases).- CLI
--summaryoutput: an alignedtext/tabwritertable (TABLE/LEFT/ RIGHT/STATUS columns,statusone ofsame/added/removed/changed (+N)/changed (-N)) plus a trailing totals line. - Default mode requires the separate
sqldiffbinary (NOT included by installing plainsqlite3on every platform) and fails with a clear, per-OS-hinted error naming the install path when it's missing, plus--summaryas thesqldiff-free alternative — found and fixed a wrong hint mid-task: a real local Homebrew install of the generalsqliteformula (keg-only, 13 files) confirmed it does NOT includesqldiff; Homebrew shipssqldiffas its OWN separate formula (brew install sqldiff), corrected in both the error text anddocs/diff.md. The Debian/Ubuntu hint (sqlite3-tools) is verified against a realubuntu:24.04container'sapt-cache show sqlite3-toolsoutput (lists/usr/bin/sqldiff), and.github/workflows/ci.yml's Linux job now installs that package alongside plainsqlite3sooffshoot diff's real-sqldiffCLI tests (TestDiffCLISqldiffPresentStreamsOutput,TestDiffCLIIdenticalSidesProduceNoSqldiffOutput) exercise the actual binary in CI instead of permanently skipping — they gate onexec.LookPath("sqldiff")the same wayrequireSQLite3gates onsqlite3itself, so a machine without it still skips cleanly. docs/diff.md: the command, the raw by-handexport-twice-then-sqldiffrecipe, the staleness rule, and a link to the FAQ's no-merge stance —offshoot diffis a read tool; it never resolves anything.
Fixed
- Settling-flush checksum-compare suppression (Milestone 2 follow-up):
rebaseline's first call now skips a session's mandatory startup settling flush when BOTH the checkoutOpenreceived was already proven byte-identical to the branch's head at the momentOpenchecked AND the LTX checksum recorded in that same checkout's.sumsidecar exactly matches what the checkout actually contains once the session's real startup rebase finishes running — the second check catches a write landing in the window betweenOpenreturning and its own startup rebase actually finishing, which the first check alone cannot see. Reading the checksum out of the local sidecar, rather than fetching it fresh from the store, costsOpenno extra store call at all — critically, no download of the head object itself, which a full snapshot can be (post-Create, post-Fork, every 16th flush, and permanently for a read-only branch that never flushes, exactly defeating this optimization's own purpose).ops.Checkout/Checkpoint/Rollback/Promoteall now record this checksum when they stamp a sidecar (sumRecord.PostApplyChecksum,ltxio.EncodeSnapshot/MaterializeChainnow return it as a byproduct of work they already do); a sidecar that never recorded one settles exactly as before ("fail toward settling"). A read-only daemon session reopened against an unmodified checkout no longer uploads a full snapshot (previously measured 541.9MB at a 512MB db) for doing nothing. A first-ever open, or one against a dirty/stale checkout, still settles exactly as before. - Sidecar refresh on clean Close (Milestone 2 follow-up):
Session.Closenow re-stamps the checkout's.sumsidecar — including its LTX checksum (the row above is what reads it back on the nextOpen) — when the close is provably clean (no session error, nothing left unflushed, at least one flush succeeded, the branch head hasn't moved past what was flushed, the replica was never rebuilt by anything beyond its own startup rebase, and — the stamped hash itself — the capture engine's own post-shutdown fingerprint, persisted only once its shutdown fully verified the checkout's WAL was cleanly folded in, is reused directly rather than independently re-derived), so the nextOpen/Checkoutagainst the same db@branch clean-skips instead of re-materializing — restoring, for the daemon-reopen pattern, the disk/descriptor win Milestone 2 Task 1 already established forCheckout/Checkpoint/Rollback/Promote. Ledgered as a documented tradeoff, not a regression: a clean-and-current checkout is now served without chain validation across a session's clean close too, not only across those fouropsentry points (see docs/status.md).
[0.1.1] - 2026-08-06
Milestone 2: safe defaults for an unattended agent. The daemon now ships
work on its own cadence instead of relying on an explicit flush, forks and
session.Open are fast on the common single-snapshot-chain path (measured,
before and after, in docs/benchmarks.md), MCP rides an already-open daemon
session for live capture and now TTLs its own forks by default, status
answers "which branch is behind and why," and docs/status.md /
docs/reference.md / this file are the honest accounting of what shipped,
what's merely "shipped" (unverified in some dimension), and what was
deliberately deferred with a stated reason rather than just left undone.
Release tag lands after this branch merges.
Added
-
Fast-path fork (Task 6a, local half):
ops.Workspace.Fork/Rollback/Promotenow copy the source checkpoint's snapshot object directly, instead of materializing it to a temp file and re-encoding a fresh snapshot, whenever the checkpoint's chain (store.Chain) resolves to exactly one member and that member is a snapshot — the common case for an at-rest fork. Newinternal/ops/reflinkpackage backs this on the local backend: a filesystem clone (clonefile(2)on darwin, theFICLONEioctl on Linux via the newgolang.org/x/sysdependency) when the filesystem supports it, silently falling back to a plain byte copy otherwise.store.BackendgainsCopyObject(dst, src string) error; the local backend implements it (temp+rename for atomicity, matching its existing write pattern), and S3 returns the newstore.ErrCopyUnsupportedsentinel unconditionally for now — S3 server-sideCopyObject(gated to objects ≤5GB) is a separate follow-up (Task 6b), soForkagainst S3 takes exactly its pre-existing slow path, unchanged. Falls back to the slow path automatically whenever the precondition doesn't hold (a branch that has been flushed through a daemon session's segment cadence past its last snapshot) or a backend can't perform the copy — never a hard failure. Measured 512MB fork 2.87s → 198ms locally on APFS (~14.5x; ~9.3ms for the copy itself, isolated from a separate pre-existing O(size) check — see docs/benchmarks.md), and a real, if smaller, win even without clone support (a Linux container withoutFICLONE: 3.02s → 513ms).internal/store/storetest's shared conformance suite gained aCopyObjectround-trip +ErrNotFoundsubtest that skips (rather than fails) against a backend returning the unsupported sentinel, so it already covers local, the in-process S3 fake, and the MinIO-gated real-provider suite. -
Fast-path fork (Task 6b, S3 half):
store.S3.CopyObjectnow issues a real server-sideCopyObjectAPI call (aPUTcarryingX-Amz-Copy-Source, no request body) instead of returning theErrCopyUnsupportedsentinel unconditionally —ops.Fork's fast path (Task 6a) now fires against an S3 backend too, for any single-snapshot checkpoint at or under S3's 5GB single-requestCopyObjectlimit.CopyObjectHEADs the source first (one request, not a download) and returnsErrCopyUnsupportedfor anything over that limit — the same fallback signal Task 6a already wiredForkto treat as "use the slow, materialize-and-re-encode path" — rather than implementing the multipartUploadPartCopyAPI this backend doesn't support. The sharedstoretest.RunConformanceCopyObjectsubtest (added in Task 6a) now runs for real against S3 instead of skipping on the sentinel, covering both the in-process fake and the MinIO-gated real-provider suite (make test-s3);storetest.FakeS3gainedCopyObject-request handling (it previously had no concept of the header-only, bodyless copy request and would have silently overwritten the destination with an empty body) and aSetSizeOverridetest hook so the 5GB gate can be exercised without allocating a real multi-gigabyte object. Measured (MinIO-local, not an AWS claim;make bench-s3) 512MB fork over S3 4.57s → 1.03s (~4.4x) and 64MB 536.9ms → 153.0ms (~3.5x) — see docs/benchmarks.md. -
Benchmarks:
internal/ops/fork_bench_test.gomeasuresFork's current slow path (copySnapshotToNewLineagematerializes the source checkpoint and re-encodes a fresh snapshot for the child lineage — O(size)),Checkout's clean-skip fast path (still O(size): it still quiesces and SHA-256-hashes the whole file, just skips the rebuild), andsession.Open's latency, all against a shared 64MB/512MB size table (plus asize=4GBcase, skipped by default under-short) so before/after numbers read off identical subtest names once Task 6's fast path lands.BenchmarkSessionOpenalso measures, once per size, the stored size of a session's settling flush — the full snapshot every daemon session uploads on its first auto-flush tick, even read-only — confirming that cost is O(size) rather than fixed.make benchruns the default sweep locally;make bench-s3runs the same benchmarks against a real MinIO container. Measured baselines, method, and machine description (host + a Linux container) are recorded indocs/benchmarks.md. -
MCP rides a running daemon:
offshoot_checkpoint,offshoot_fork, andoffshoot_checkouteach probe the daemon fresh on every call (never cached — the daemon may start after the MCP server does) and take a live-capture path when one is up, falling back silently to exactly today's at-rest behavior otherwise.offshoot_checkpointon a branch with an open daemon session flushes it live (the daemon'sflushop — no quiesce, no full-snapshot re-encode);offshoot_forkroutes through the daemon'sforkop whenever a daemon is up, which flushes an open source session first so an unflushed write always lands in the new branch;offshoot_checkouton a branch with an open session returns that session's own live checkout path instead of materializing a separate at-rest copy — but not a FENCED session's path: a session that lost its lease (e.g. to another writer) stays listed until closed, sooffshoot_checkoutchecks its health and falls back to at-rest with a warning naming the session's error rather than handing over a path nothing is capturing anymore. No MCP tool opens a session itself — the good path requires a harness (the SDKs,offshoot session open, or your own loop) to have opened one already; without that, MCP checkpoints stay at-rest even with a daemon running.offshoot mcpgains-socket PATH(default: the same derivationoffshoot serveuses for the store), so an MCP server and a daemon started against the same store agree on where to look without either hardcoding the path.offshoot_rollback,offshoot_promote(on its target), andoffshoot_destroynow refuse (rather than silently fencing a live daemon session) when the daemon has any session — healthy or fenced — open on the affected branch: those ops repoint or delete a branch's ref outright at rest, bypassing the daemon entirely, so without the refusal they could clear a lease or repoint storage out from under a session the daemon still believes it owns. -
MCP forks expire by default:
offshoot_forkgains attlargument (a Go duration string, or"none"for a branch that never expires) and falls back tooffshoot mcp -default-ttl(default24h) whenttlis omitted — an explicitttlon the call always wins, andttl:"none"always yields no TTL even under a configured default.-default-ttl 0or-default-ttl nonedisables the default entirely. The fork tool's response now echoes the TTL it applied and, when there is one, the computed expiry timestamp, so both land in the agent's own transcript. Reaping still requires a running janitor (offshoot serve) — a daemonlessoffshoot mcpsetup only sweeps expired branches whenoffshoot gcis run by hand, and both the tool's Description and the README now say so. -
Background flush:
session.Options.FlushEvery, when > 0, flushes a session automatically on that cadence so an agent that never callsFlushstill gets bounded data loss on crash (library default stays0, manual only).Session.LastFlush()/LastFlushErr()expose the most recent successful flush and the most recent automatic-flush failure — a failure is recorded and retried next tick (never one already superseded by a fresher success), never kills the session, and a session-closed race never surfaces as a spurious flush error.offshoot servegains-flush-every(default30s,0disables, negative rejected as a usage error) and applies it to every session it opens — the daemon ships work on a cadence by default, the safe default lives at that boundary rather than in the library primitive. An idle tick (nothing committed via a captured transaction, AND no rebase, since the last successful flush) does nothing at all — no object write, no ref write — rather than uploading a pointless full snapshot everySnapshotEveryticks forever; the rebase half of that check matters because a rebase's checkpoint can fold a real commit into the baseline without it ever passing through ordinary WAL capture, so every session performs one settling flush shortly after its startup rebase lands, even if the agent never writes anything. -
Observability (Milestone 2, Task 7):
statusnow answers "which branch is behind and why."capture.EnginegainsLag(), WAL bytes a writer has committed but this engine's replica has not yet applied — tracked via a new atomic offset field (the same lock-free patternRebased/Resumedalready use, since the engine has no general mutex covering its reader state) so it is safe to call from any goroutine concurrently withRun.SessionInfo(the daemon's "status" op) gainsdurable_age(time since the last successful flush, empty if never),last_flush_at(RFC3339),flush_error(the most recent automatic-flush failure, mirroringLastFlushErr's clear-on-success semantics exactly —ErrClosedis never recorded, same as the library-level field it surfaces), andcapture_lag_bytes(always present, 0 is meaningful).offshoot session status's output line gainslag=, and — when set —last_flush=,age=, andflush_error=. Every session state transition (opened; flushed, taggedkind=manualorkind=auto, with its txid; flush-failed, with the error —ErrClosedraces excluded, since aClose-in-progress isn't an operational failure; fenced, with the terminal cause; closed) now writes one structuredoffshoot: session: db@branch: event key=value ...line to stderr, matching the daemon janitor's existingoffshoot: janitor: ...prefix family rather than a second format.
Changed
Checkoutno longer re-materializes a checkout that is already clean at the branch's current head (sidecar fingerprint matches the file, and the fingerprint's recorded lineage/epoch/txid matches the ref right now); it returns the existing file as-is. Everysession.OpencallsCheckout, so this avoids an O(size) temp-file-and-rename plus a strandeddbfiledescriptor on every open when the checkout hasn't drifted. Modified and stale checkouts are unaffected: they still warn and re-materialize exactly as before.
[0.1.0] - 2026-08-05
Initial pre-release. offshoot brings git-like branching to SQLite databases: create, fork, checkpoint, rollback, and promote — stock SQLite files, your choice of storage.
Added
- Core branch operations:
init,create(optionally--froman existing file),checkout,checkpoint,fork(optionally from a checkpoint, with an optional TTL),rollback --to,promote --onto,destroy,path,status, andgcfor unreachable-lineage collection. - Two storage backends behind a common
Backendinterface: a local directory store and an S3-compatible store (AWS S3, Cloudflare R2, Tigris, MinIO), selected via a store spec (./path,file:///abs/path,s3://bucket/prefix). - Compare-and-swap safety: every branch ref update is a conditional write.
OpenBackendruns a CAS probe against the store at attach time and refuses to operate if conditional writes are not enforced, rather than silently degrading. Google Cloud Storage's S3-interop API is unsupported for this reason; MinIO is verified via the real-provider conformance suite, AWS S3 and R2/Tigris are expected to pass but not yet run against for real. Seeinternal/store/s3_integration_test.go/make test-s3. - Leases with epoch fencing: a long-running writer claims a branch with a TTL'd lease; acquiring or reclaiming bumps the branch's epoch so a stale writer that resumes after losing its lease cannot corrupt the branch — its writes land in a superseded, unreferenced prefix instead.
- Daemon mode (
offshoot serve): holds branch leases and captures every committed transaction while a session stays open, removing the quiesce-to-checkpoint constraint of at-rest mode.offshoot session open/flush/status/close/shutdowndrive it from the CLI; flushes are incremental (only changed pages) with a full snapshot every 16th flush, so materializing a branch replays at most one snapshot plus fifteen segments (bounded replay). - TTL reaping: branches can carry a TTL (set at fork time or via
offshoot touch), measured from the last durable write or lease renewal. The daemon's janitor (-reap-every,-gc-grace) reaps expired, lease-free branches and periodically GCs tombstoned lineages. Protected branches (mainby default) are never reaped. - MCP server (
offshoot mcp): exposes list, checkout, checkpoint, fork, rollback, promote, and destroy as MCP tools over stdio, so an agent can branch on its own initiative — with the same protected-branch guardrails as the CLI. - Python and TypeScript SDKs (
sdk/python,sdk/typescript): thin, dependency-light clients over the daemon's lifecycle API. Not yet published to PyPI/npm; import from a checkout of this repo. Exercised against a real daemon bymake test-sdks. offshoot.langgraph.ThreadForks: a LangGraph checkpointer companion that maps each thread to its own offshoot branch, so rewinding and retrying a thread forks the database too. Seeexamples/langgraph-rewind/.offshoot versionprints the release version plus the Go runtime version and GOOS/GOARCH.