⑂ offshoot

Core concepts

The vocabulary offshoot's commands, docs, and error messages all share — each term defined once, with the model it belongs to. The deeper design document behind this page is Architecture; the exact flag-by-flag behavior of every command named here is the CLI reference.

One picture

              checkpoints
demo@main     ○ init ──── ○ seeded ──── ○ v2 ─────────▶ head    (lineage A)
                          │
                          │  fork: child lineage B writes a base pointer
                          │  naming A and the fork-point txid — no data
                          │  copy; reads below the fork point resolve
                          │  through A's own objects
demo@attempt-1            ╰──── ○ fork ──── ○ passed ─▶ head    (lineage B: shared)

promote attempt-1 onto main:
demo@main     ○ promote ──────────────────────────────▶ head    (lineage C: shared)
              main is repointed at a NEW lineage C whose base pointer names
              B at attempt-1's head — no data copy (promote shares, like
              fork; --materialize copies instead). Lineage A is orphaned:
              tombstoned, held for a grace period, then GC'd. attempt-1
              survives unchanged — typically forked with --ttl, so the
              janitor reaps it after expiry; B's objects that main still
              reads stay live until main diverges past them or is
              compacted.

The naming model

Store

The place everything durable lives: a local directory or an S3-compatible bucket (-store ./.offshoot, -store s3://bucket/prefix). A store holds refs, snapshots, and segments; checkouts are always real local SQLite files even when the store is a bucket. Every attach probes the store for conditional-write (compare-and-swap) support and refuses to run without it — see Installation and the storage layout.

Database

A named entity in a store (invoices). Names are unique per store, charset [a-z0-9-_.], max 128 characters. offshoot create invoices makes an empty one; offshoot create invoices --from existing.db imports an existing SQLite file (the source is never modified).

Branch

A named ref within a database, addressed db@branch (invoices@attempt-1); every database has a default branch main, and db alone means db@main. The ref records the branch's lineage id, epoch, head transaction id, checkpoint map, TTL, and protected flag — and every mutation to it is a compare-and-swapped conditional write (invariant 2).

Checkpoint

A name within a branch mapping to a transaction id — offshoot checkpoint app v1 — unique per branch. It's the unit you fork from (fork --at v1), roll back to (rollback --to v1), export, or diff against. Children never inherit a parent's checkpoints: a fork's own history begins at its fork point with an auto-created fork checkpoint, so pre-fork states are resolved on the parent instead.

Checkout (the working copy)

The materialized local SQLite file for a branch — what offshoot checkout app@attempt-1 prints, at the fixed path <store>/checkouts/<db>/<branch>.db for local stores. You open it with any stock SQLite client at native speed; offshoot never proxies SQL. The path is derived from db@branch, never used as an identifier, and a live connection is never rewritten out from under itself (invariant 5). Read-only historical checkouts of a named checkpoint live in a separate checkouts-ro tree (checkout --at v1 --read-only).

Where the filesystem can clone (APFS, btrfs, XFS with reflink), a checkout is a clone of an immutable by-chain entry under checkouts-ro/<db>/~by-chain/, one per distinct resolved chain, so forty forks of one checkpoint are forty clones of one file rather than forty decodes of the same snapshot. Beside each checkout sit two companions: a .sum sidecar that lets a repeat checkout prove the file unchanged from its size, mtime and SQLite change counter without hashing it, and a .shadow — a reflinked copy as of the last checkpoint, which an at-rest checkpoint diffs against to write only the changed pages (what each file is).

The storage model

Lineage

An append-only sequence of transaction segments in the store — the storage history behind a branch. The core invariant: a lineage is only ever written by one branch, for its entire life (invariant 1). Branches acquire new lineages via fork, rollback, and promote; they never share or adopt another branch's live lineage. Segments use LTX, Litestream v0.5's Apache-2.0 transaction-aware format (credit).

Snapshot

A full encoding of the database at one transaction id — the self-contained kind of object in a lineage. An at-rest offshoot checkpoint writes one when it has no shadow to diff against, when most pages changed, when the chain has reached the snapshot cadence, or when asked (--snapshot); a daemon session writes one every Nth flush (-snapshot-every, default 16). Either way replay stays bounded.

Segment

An incremental object: only the pages changed since the previous flush or checkpoint, at a specific transaction id. A daemon session writes them from continuous capture; since v0.2.12 an at-rest offshoot checkpoint writes them too, from a page diff against the checkout's shadow (the checkpoint's output and the ref's kind field say which it wrote). Measured cost of a session's segments is ~761 B per single-row transaction against a 100 MB database.

Chain

The resolved read path for a branch at some transaction id: one snapshot plus the segments after it, in order — never more than one snapshot plus N-1 segments at the default cadence, so materializing state never replays an unbounded history. Chain resolution follows base pointers across lineages but never merges members across them (fork mechanics).

Base pointer (copy-on-write)

What makes fork near-free. A forked child gets its own lineage for everything it writes after the fork, plus one durable base pointer (data/{lineage}/base.json) naming the parent lineage and the fork-point txid. Reads on the child resolve through the parent's objects below the fork point and the child's own above it — so a fork of a 100 MB database adds 377 bytes to the store, flat from 1 to 100 forks. Two automatic snapshot floors keep chains bounded on deep fork-of-fork spines (details).

Storage class: shared vs materialized

offshoot status labels every branch storage=shared (a base-pointer lineage: near-free to hold, but it pins whatever storage its chain still reads through) or storage=materialized (a fully self-contained lineage: branches made by create, and the results of compact and of --materialize). Fork, promote and rollback share; compact is the one operation that always materializes a copy — that is its job (the storage-cost ledger).

Promote, rollback, and compact: the repointing operations

All three are the fork machinery pointed at a new lineage. Since v0.2.12 promote and rollback make that lineage the way fork does — a base pointer at the kept state (the checkpoint, or the source's head), no data copied — and --materialize asks for the old self-contained copy instead; compact always copies. Rollback (rollback app@b --to v1) repoints the branch at a lineage seeded from a checkpoint, keeping checkpoints at or before the target — and, like promote below, keeps the branch's previous head first as a shared, TTL'd safety fork, <branch>-pre-rollback, so the rollback can be undone by promoting that fork back (--no-backup skips it). Promote (promote app@attempt-1 --onto main --force) repoints the target at a lineage seeded from the source's head; the source survives unchanged, and the target's checkpoint map resets to just promote — but the target's previous head is kept first as a shared, TTL'd safety fork, <target>-pre-promote, so the promote can be undone by promoting that fork back (--no-backup skips it; see the reference). Compact (compact app@b) turns a shared fork into a self-contained branch — the manual lever for releasing a destroyed ancestor's storage. Unlike promote, it does not reset the checkpoint map: every existing checkpoint's snapshot is copied into the new self-contained lineage (as rollback --materialize does), with a compact checkpoint added at head. Compact re-encodes the whole database (~G bytes for a G-byte database) plus one extra snapshot copy per distinct checkpoint txid kept; promote and rollback pay that only with --materialize, or when the kept state's chain is already at the fork-time depth floor. What sharing trades away is prompt reclaim: a shared rollback keeps the old lineage's objects at and below the checkpoint live (the abandoned future above it is reclaimed), and a shared promote keeps the source's lineage live under the target — until the branch diverges past them or is compacted.

The concurrency model

Session

A live, daemon-held writing context on one branch: offshoot session open app (against a running offshoot serve) acquires the branch's lease, materializes its checkout, and captures every committed WAL transaction continuously while your process keeps writing. session flush (and the daemon's background timer, default every 30s) makes captured writes durable in the store; session close releases the lease. Without a daemon, every command runs at rest — open, do the work, exit.

Lease

The claim that makes a branch's single writer explicit: at most one holder per branch, renewed continuously by a daemon session, inspectable and breakable via offshoot lease list/acquire/release. Expiry is wall-clock and advisory — the actual guarantee against a stale writer comes from the epoch fence and ref CAS, not the clock (fencing in two paragraphs).

Epoch

The fencing counter in the ref: acquiring or reclaiming a branch bumps it, and every object write lands under the epoch current at write time as a create-only put. A writer that pauses, loses its lease, and later resumes writes into a dead epoch prefix no ref points at — garbage, collected later, never corruption of the live chain (invariant 3).

TTL

A branch's self-destruct timer, set at fork time (fork app attempt-1 --ttl 2h) or later (touch app@attempt-1 --ttl 30m; --ttl none clears it). Measured from the later of the branch's last-touch time and its lease expiry — exactly what the janitor's reap logic uses; touch resets the clock. A branch with an active lease is never reaped, protected branches are never reaped, and branches without a TTL live until destroyed. Reaping is the janitor's job — offshoot serve's timer, offshoot mcp -reap-every's own background pass when no daemon is reachable (default 60s; defers to the daemon's janitor instead of running alongside it), or an on-demand offshoot gc. GC itself (reclaiming a reaped branch's storage) still needs the daemon's janitor or offshoot gc either way — -reap-every on its own only reaps.

Protected branch

A per-branch flag, on by default for main and settable for any other branch via offshoot protect/unprotect (CLI-only — no daemon op, no MCP tool): destroy and promote --onto refuse without --force, uniformly across the CLI, the daemon, and the MCP server. Through offshoot mcp, though, an agent's own force:true is not the CLI's --force in disguise: it's honored only when the server was started with -allow-force (off by default), so touching a protected branch through MCP is a permission the human running the server grants at startup, not an argument the agent can supply its own way past. This is what lets an agent fork and experiment freely without being able to vaporize main in a single unforced (or, by default, even forced) call (invariant 7).

GC

The two-step cleaner behind fork-spam being the expected workload. Reap destroys branches whose TTL expired; collect finds storage objects no live branch's chain can reach (following base pointers, so a destroyed parent's bytes stay live while a shared child still reads through them), tombstones them, and deletes them once a grace period passes (gc --grace, default 1h; the daemon janitor's -gc-grace, default 15m). Destroy is instant; the storage refund waits for the last sharing child (the full semantics).

Where next