HYPERSCALE · STATE & RESHARDING

The moving day that never happens.

Every growing database eventually faces the dreaded resharding migration: copy everything, hope nothing drifts, cut over, pray. Hyperscale splits and merges entire shards — live, under load — without moving a single record. Not because the movers are fast, but because the data was shelved for it from the start.

01 · The problem

Why "resharding" is a word engineers dread

When a database outgrows one machine, its data must be divided across several — and later re-divided as it keeps growing. Ask anyone who has run one at scale: the re-division is the nightmare. The industry's standard answers all involve moving the data itself, and everything painful follows from that:

✕ COPY & CUT OVER

The maintenance window

Copy terabytes to new homes while writes continue, freeze, reconcile the drift, flip. "Service unavailable this weekend" is this, from the inside.

✕ DUAL-WRITE

Two versions of the truth

Write to both old and new homes during the transition, and now every inconsistency question has two possible answers until the old side is finally torn down.

✕ REBALANCE THE RING

Everything scatters

Hash-based partitioning spreads each user's records evenly — which means evenly apart. Every resize sends shrapnel of records flying between machines, and no user's data ever lives in one place.

For a blockchain the stakes are worse: there is no maintenance window on a chain that must never stop, no admin to reconcile drift, and the "database" must stay provably intact — thousands of machines that don't trust each other must agree, cryptographically, that nothing was lost or invented in the shuffle.

02 · The shelving rule

A library shelved for the split

Imagine a library that knows, from the day it opens, that it will someday outgrow its building — so it adopts one iron shelving rule: everything a family owns sits together, on one contiguous run of shelving. Not sorted by topic, or acquisition date, or the whims of whoever shelved it: by owner, always. The books never need to know; the rule lives in the shelf-marks.

Hyperscale's version: all state lives in one giant catalogue — a binary merkle tree over a vast space of shelf-marks — and every object's shelf-mark is built in two halves:

owner's hash item's hash places it on the owner's shelf tells the owner's things apart shelf-mark = ada's account · vault · savings bo's account · vault cy's account · 3 vaults …
The first half of every shelf-mark is the owner. A family's belongings are contiguous by construction — and stay that way forever.

The subtle part is that a smart-contract engine constantly creates internal objects — token vaults, key-value stores — whose native names have nothing to do with their owner. Shelved naively, your account would sit on one shard and your money on three others. The shelving rule reaches through that: every internal object is shelf-marked under the account that owns it. And because an object never changes owner once committed, no shelf-mark ever needs to change — nothing is ever re-shelved, for any reason, including the splits to come.

A shard is simply a wing of this library — everything under one prefix of the shelf-mark space. And the catalogue is a merkle tree, so every wing has a one-line cryptographic summary: its subtree root.

One structure is doing four jobs: per-block state commitments, the proofs that ship state between shards, verifiable fast-sync for newcomers — and, as follows, resharding.
03 · The split

Hand over the wing, keep the books still

When a wing grows past a size threshold, the shard itself asserts a split in its own blocks — a claim about committed facts that every validator checks against its own copy before voting, so it can't be faked or forced. Then the beautiful part. The wing's catalogue node has exactly two children: the left half-wing and the right half-wing. Splitting the shard means each child subtree becomes a new shard, whole:

ONE OVERFULL WING root R state crosses the size threshold… SPLIT TWO WINGS, SAME SHELVES L R' hash(L, R') = R — one operation proves nothing moved
Splitting hands each child an existing subtree. Merging recombines them with the same single hash. No record is copied, re-keyed, or lost — provably.

What does move is people: each child wing needs its own committee of validators. That handover is choreographed make-before-break, like staffing a new reading room before unlocking its door:

Recruit. The beacon — the network's coordination chain — draws fresh validators by lottery and assigns half to each child.

Copy, verifiably. While the parent shard keeps serving as if nothing were happening, each recruit downloads its child's half of the data. Every chunk arrives with a proof against the attested catalogue root — a proof that covers completeness, so a malicious server can't quietly omit a page — and the recruit's own government record (its copy of the beacon chain) tells it which root to demand. Then it tails the parent's new blocks, keeping its copy current.

Gate. Nothing flips until both children could seat a live, supermajority-strong committee — each recruit's readiness is witnessed on-chain, not self-declared. If readiness never comes, the split lapses harmlessly and the recruits return to the pool.

Flip, fail-closed. The parent runs to a terminal block fixed by attested time, then stops. Each child's first block is derived — computed by every member independently from the parent's frozen final block — and must hash to exactly the value the beacon recorded in advance. Any mismatch and the flip simply refuses to happen. Merging is the same movie in reverse: two sibling wings, keepers drawn half from each, one hash to recombine.

04 · The fine print

What keeps it honest

◦ NO OSCILLATION

Wide hysteresis

The merge threshold sits far below the split threshold, so a freshly split child can't immediately clamor to merge back. Split-merge flapping is priced out of the mechanics.

◦ NO OPERATOR

No choreography to botch

Trigger, recruitment, readiness, flip — every step is either a committed on-chain fact or a pure function of one. There is no runbook, no admin console, no 2 a.m. cutover call.

◦ NO TRUST IN THE MOVERS

Proofs at every step

Recruits verify every downloaded chunk against a root their own beacon copy attests; children verify their derived genesis against the beacon's recorded hash. A wrong byte anywhere fails closed.

And the hardest question of all — what happens to a cross-shard transaction caught mid-flight when one of its shards stops existing at a terminal block? That one is the will: the dying shard's frozen final records decide, and every survivor settles against them identically.

The lesson: the cheapest data migration is the one made unnecessary years earlier, by a shelving rule. Decide where things live so that any future partition line falls between owners, never through one — and "resharding" stops being a moving day and becomes an org-chart change.