Content-Addressable Storage and Merkle DAGs: Tamper-Proof Local State

Every reference your application holds is a small bet that something stays where it was. The bet usually pays—until it quietly does not. A bookmark points at a URL whose page moved. A settings file references assets/logo.png, which a teammate renamed last week. A folder of photos contains three identical copies because nobody could prove two files were the same without opening them. And when storage misbehaves—a bad sector, an interrupted write, a bug in an export routine—the damage surfaces as garbage characters weeks later, long after any backup worth restoring has rolled away.

These are three separate annoyances with one shared root: data addressed by where it lives rather than what it contains. Content-addressable storage (CAS) inverts that choice, and once combined into Merkle trees, it produces local state that is deduplicated by construction, self-verifying on every read, and naturally suited to the sync problems described in our CRDT explainer. Git has run on exactly this design for two decades; this article builds the core of it in TypeScript against IndexedDB.

Location addressing has three quiet failure modes

Naming files by path is convenient precisely because paths are mutable—and that mutability leaks into every system built on top:

Renames break references. Anything that captured the old path—an embedded link, a saved workspace layout, another document’s cross-reference—now dangles. The data still exists; the address rotted.
Duplicates waste space and invite drift. Without an identity independent of location, nothing tells you that report-final.pdf and archive/report-final(2).pdf hold identical bytes, so both persist, and edits applied to one fork the other.
Corruption goes unnoticed. A checksum-less copy verifies nothing about itself. Bit rot, truncated uploads, and failed writes all produce plausible-looking files that fail only when opened—at which point the original may be gone.

None of these failures announces itself. That is what makes location-based identity expensive over the lifetime of real data.

Content addressing fixes all three at once

The fix is one function: address = SHA-256(bytes). Store each blob under the hash of its own contents, and the properties fall out mechanically:

Immutability by construction. An address is a fingerprint of the bytes. Change one bit anywhere and the address changes—so an address can never lie about what it retrieves. There is no such event as “the file moved” or “the file changed under me.”
Deduplication for free. Identical content hashes identically. Writing the same blob twice collapses to a single stored block; the second write is a no-op keyed on an existing entry.
Self-verifying reads. Re-hash whatever comes back from storage and compare it to the requested address. Any mismatch proves corruption before the data reaches application logic. Verification travels with the data instead of living in a side table.

The mental shift is subtle but total: addresses stop being labels people choose and become consequences of content. Applications never invent keys; they compute them.

From blobs to graphs: Merkle DAGs

Flat blobs alone would be little more than a checksum scheme. The structure emerges when blocks are allowed to reference other blocks by their addresses. A directory node, for instance, is just serialized text mapping names to child hashes:

{ "chapter-1.txt": "9f2c…", "chapter-2.txt": "41ab…" }

That node is itself stored as a block and hashed like any other—which means the parent’s hash depends on its children’s hashes, recursively, all the way down to leaf data. The result is a Merkle directed acyclic graph: a tree-like structure where every internal node cryptographically seals its entire subtree.

Three consequences follow directly:

Tamper evidence composes upward. Flip a byte in any leaf and its hash changes, which changes every ancestor’s hash, which changes the root. Nobody can alter historical data without producing a visibly different root. One 32-byte root hash attests an arbitrarily large document.
History costs nothing extra. Editing chapter two creates a new block for its new content, a new parent node pointing at the new child plus the untouched chapter-one hash, and so on up to a new root. The old root still addresses the complete previous version. Immutable history falls out of the graph shape—no overwrite ever occurs, so no overwrite can corrupt anything.
Unchanged subtrees are shared, not copied. Because parents link children by content, two document versions that differ in one section physically share every other block. This structural sharing is why versioned CAS remains cheap even for large documents: storage grows with change, not with size times versions.

Astute readers may notice such documents form graphs rather than strict trees whenever substructures are shared—a template referenced from two chapters produces one subtree serving two parents. The directed acyclic qualifier exists precisely for this: cycles cannot form (a parent cannot contain its own address, since that address depends on contents that would include it), but sharing is free and encouraged. Verification, diffing, and collection all treat same address as same bytes wherever the shape branches.

Git’s object model—blobs, trees, commits—is the most familiar production instance of exactly this graph. IPFS applies the same principle across networks, where a content address names data independently of which server holds it.

Code walkthrough: a minimal CAS over IndexedDB

The whole foundation fits in one small class. Blocks go in keyed by their SHA-256; reads re-hash to verify integrity before returning:

// cas.ts — minimal content-addressable store over IndexedDB.
const HEX = [..."0123456789abcdef"];

async function sha256Hex(bytes: Uint8Array): Promise<string> {
  const digest = await crypto.subtle.digest("SHA-256", bytes);
  return [...new Uint8Array(digest)]
    .map(b => HEX[b >> 4] + HEX[b & 15]).join("");
}

export class Cas {
  #db: Promise<IDBDatabase>;

  constructor() {
    this.#db = new Promise((resolve, reject) => {
      const req = indexedDB.open("cas-demo", 1);
      req.onupgradeneeded = () => req.result.createObjectStore("blocks");
      req.onsuccess = () => resolve(req.result);
      req.onerror = () => reject(req.error);
    });
  }

  // Write returns the content's address. Duplicate writes are no-ops:
  // identical bytes already occupy this key.
  async put(bytes: Uint8Array): Promise<string> {
    const addr = await sha256Hex(bytes);
    const db = await this.#db;
    await new Promise((resolve, reject) => {
      const tx = db.transaction("blocks", "readwrite");
      tx.objectStore("blocks").put(bytes, addr);
      tx.oncomplete = resolve;
      tx.onerror = () => reject(tx.error);
    });
    return addr;
  }

  // Read verifies before trusting: re-hash and compare to the key.
  async get(addr: string): Promise<Uint8Array> {
    const db = await this.#db;
    const bytes: Uint8Array | undefined = await new Promise((resolve, reject) => {
      const req = db.transaction("blocks").objectStore("blocks").get(addr);
      req.onsuccess = () => resolve(req.result);
      req.onerror = () => reject(req.error);
    });
    if (!bytes) throw new Error(`missing block ${addr.slice(0, 12)}…`);
    if ((await sha256Hex(bytes)) !== addr)
      throw new Error(`corruption detected at ${addr.slice(0, 12)}…`);
    return bytes;
  }
}

Note what disappeared from the usual persistence code: no schema versioning of values, no update-in-place semantics, no invalidation logic. Writes are idempotent; reads are proofs.

Layering documents on top takes one recursive function—every node serializes to JSON whose child values are themselves CAS addresses:

type Tree = { [key: string]: string | Tree };   // leaves: strings; nodes: subtrees

// Snapshot a nested document; returns the address of its root node.
export async function snapshot(tree: Tree, cas: Cas): Promise<string> {
  const entries = await Promise.all(
    Object.entries(tree).map(async ([key, value]) =>
      [key,
       typeof value === "string"
         ? await cas.put(new TextEncoder().encode(value))   // leaf → block
         : await snapshot(value, cas)] as const));          // subtree → recurse
  const encoded = new TextEncoder()
    .encode(JSON.stringify(Object.fromEntries(entries)));
  return cas.put(encoded);                                  // node → block
}

Comparing two snapshots exploits the graph shape. If roots match, everything matches—one comparison ends the walk. Otherwise descend only where child addresses differ:

// Report every path whose content differs between two roots.
export async function diff(a: string, b: string, cas: Cas,
                           path = ""): Promise<string[]> {
  if (a === b) return [];                    // equal hashes ⇒ equal subtrees
  const decode = (x: Uint8Array) =>
    JSON.parse(new TextDecoder().decode(x));
  let mapA: Tree, mapB: Tree;
  try {
    [mapA, mapB] = await Promise.all([cas.get(a), cas.get(b)]).then(
      ([x, y]) => [decode(x), decode(y)]);
  } catch { return [path || "/"]; }          // leaf mismatch — record it
  const out: string[] = [];
  for (const key of new Set([...Object.keys(mapA), ...Object.keys(mapB)])) {
    const ca = mapA[key], cb = mapB[key];
    if (!ca || !cb) { out.push(`${path}/${key}`); continue; }
    out.push(...await diff(ca, cb, cas, `${path}/${key}`));
  }
  return out;
}

The walk’s cost tracks divergence, not document size: two snapshots differing in one chapter visit that chapter plus the spine down from the root, then stop. For sync protocols this distinction is everything—the comparison itself already produces the network-shaped answer, naming exactly which blocks must travel.

Why local-first apps care

For software whose state lives entirely in the browser, this model solves several problems at once:

Tamper-evident history. Keep a log of root hashes—optionally signed, using primitives covered in the Web Crypto AES-GCM walkthrough—and any modification of stored state becomes detectable, even retroactively. Root hashes are small enough to pin somewhere independent of the archive—a note in a password manager, a printed page—giving the storage an external anchor it cannot rewrite. A journal that can prove its past entries are unaltered is qualitatively different from one that merely hopes.
Sync becomes address exchange. Two devices comparing root hashes know instantly whether they agree; if they diverge, the diff above names exactly which blocks to exchange. Only novel content crosses the wire, and received blocks verify themselves on arrival. This slots cleanly beside CRDT convergence: the CRDT decides what the merged state is, content addressing provides proof of which state every replica holds.
Backups audit themselves. Verifying a backup means walking its graph and re-hashing—no external manifest required, partial corruption pinpoints itself, and because verification recurses through the same addresses, spot-checks of any subtree carry local meaning even inside enormous archives.

All of this runs comfortably in browser storage. Which engine to host the block store on—IndexedDB or the Origin Private File System—is measured honestly in our OPFS-versus-IndexedDB comparison; the CAS layer does not care, it just wants reliable byte storage underneath. The broader architectural context—why applications keep state local in the first place—is in what local-first software means.

Questions people often ask

How worried should I be about hash collisions?

Practically, not at all. SHA-256 output spans 256 bits, so the birthday-bound risk of two distinct blocks colliding is around one in 2¹²⁸—far below the probability of undetected hardware error in any realistic dataset. Git ran on a weaker hash (SHA-1, 160 bits) for over a decade before deliberate collision research forced migration; accidental collisions were never the threat.

How do I handle files larger than a sensible block?

Chunk them. Fixed-size splitting works but degrades when insertions shift boundaries; content-defined chunking cuts at positions derived from the data itself, so unchanged regions produce identical chunks regardless of edits elsewhere. Each chunk becomes a CAS block; the file is represented by a node listing chunk addresses.

When do orphaned blocks get deleted?

Never automatically—CAS has no native deletion, since removing a block referenced by any live graph would corrupt it. Real systems periodically run mark-and-sweep from current roots (and any retained historical roots), collecting unreachable blocks. Until then, orphaned data simply idles, which also functions as a safety net against over-eager cleanup.

How is this different from just checksumming files?

A checksum verifies one object after the fact; it does not name it, deduplicate it, or compose. Merkle addressing does all four at once: the hash is simultaneously the key, the integrity proof, the dedup identity, and—through the graph—a single verifiable summary of arbitrarily large structured state.

The takeaway

Content addressing replaces trust in location with proof from mathematics: name data by what it is, store it once, verify it on every read, and let graphs of hashes turn mutable documents into immutable, diffable, shareable history. The implementation above is a teaching skeleton—real systems add chunking, packing, and collection—but every one of those extensions preserves the invariant that made the toy correct: an address is a promise about bytes, and promises that cannot break make the rest of the architecture calmer. For the sync half of that calm, continue with how CRDTs converge without a server.