Skip to content

slave (M2): add shard skeleton with ShardChain stub and genesis record#26

Open
syntrust wants to merge 13 commits into
slave-m1from
slave-m2
Open

slave (M2): add shard skeleton with ShardChain stub and genesis record#26
syntrust wants to merge 13 commits into
slave-m1from
slave-m2

Conversation

@syntrust

@syntrust syntrust commented Jul 3, 2026

Copy link
Copy Markdown

Milestone 2 of the slave-node bootstrap (design doc, tracking #17), stacked on #21. Adds qkc/shard — additions only: everything the slave needs to stand one shard up at its QKC genesis, with the real chain behind a stubbed seam.

What's here

  • Genesis descriptor (genesis.go) — per-shard genesis derived from the cluster config. Fingerprint() is the descriptor's identity and covers the full rule set, compiled-in fork schedule included — any drift forces a loud re-bootstrap instead of new semantics on an old db.
  • ShardChain seam (services.go) — the narrow boundary ({GenesisHash, Head, Stop}) the geth-core shard chain plugs into later; StubChainService stands in today. Options carries named injection points.
  • GenesisRecord (rawdb.go) — the temporary genesis persistence, below.
  • Shard (shard.go) — isolated pebble chaindb per shard under {datadir}/shard-0x{full_shard_id}/.

rawdb.go: the temporary GenesisRecord

One QKC-prefixed key (QKC-genesis-record) stands in for the genesis block until the QKC block format (#1) lands. It does three jobs:

  • carries the two genesis facts geth's stock block format cannot hold: HashPrevRootBlock and the initial XShardCursor ((root_height, 0, 0), pyquarkchain genesis.py:92);
  • guards reopen: ReconcileGenesisRecord hard-errors when the cluster config changed since initialization — where goquarkchain silently keeps the stored genesis;
  • marks bootstrap completion: written only after the chain stands at its genesis, so a failed boot leaves no record and the retry takes the fresh path.

Every symbol carries TODO: temporary — remove once #1 lands. When #1 merges, the fields move into the genesis block itself, the reopen check becomes geth-native, and the record is deleted, not migrated — the db holds only genesis-level data, so re-bootstrap is free.

Tests

go build ./cmd/slave && go test ./qkc/... ./cmd/slave/ — 124 tests, 11 packages, green. Shard tests are table-driven over the real mainnet/devnet fixtures.

  • Milestone demoTestShardNewAndReopen: boot shard 0x00000001 into a temp datadir; assert the datadir layout, head at height 0, and the stored record; stop and reopen the same directory cleanly.
  • Record — round-trip; a fresh-db reconcile writes nothing; a failed first boot leaves no record behind (TestShardFailedFirstBootLeavesNoRecord).
  • Loud mismatch — shard-genesis and root-genesis config changes both fail reopen, naming both hashes, the db path, and cluster config changed since initialization.
  • Fingerprint — deterministic, and sensitive to timestamp, ALLOC, and fork-schedule changes.
  • Edges — mem-db mode writes nothing to disk; idempotent Stop; unknown shard rejected; ETH_CHAIN_ID consistency incl. the uint32-overflow boundary; DBDirName round-trip.

🤖 Generated with Claude Code

…etadata

Add the per-shard host the slave boots each of its shards through:

- genesis.go: Genesis descriptor built from ShardGenesis (fields + parsed
  ALLOC) with a Petersburg-only ChainConfig, ChainID = BASE_ETH_CHAIN_ID +
  chain_id + 1 (pyquarkchain config.py:363); Fingerprint() gives the
  descriptor a deterministic identity the stub reports as its genesis hash.
- services.go: the ShardChain seam (genesis hash, head, Stop) between the
  slave skeleton and the geth-core shard-chain task, plus the Options
  injection points for #4 (engine/miner), #5 (master conn), and sync; the
  stub chain reports head height 0 at the descriptor's identity.
- rawdb.go: GenesisMeta record (root-genesis linkage + xshard cursor at
  (root_height, 0, 0), genesis.py:92) under a single QKC-prefixed key,
  encoded with qkc/serialize; ReconcileGenesisMeta passes reopen only on an
  exact match and hard-errors naming both genesis hashes and the db path.
  Marked temporary: re-implemented, not patched, once QKC block format (#1)
  lands.
- shard.go: Shard{Branch, cfg, db, chain}; New opens an isolated pebble
  chaindb under {datadir}/shard-0x{full_shard_id}/, reconciles the metadata,
  and constructs the chain through the seam; Stop is idempotent and closes
  chain then db.

qkc/config gains ETH_CHAIN_ID parsing on ChainConfig: pyquarkchain forces
ETH_CHAIN_ID = BASE_ETH_CHAIN_ID + CHAIN_ID + 1 on load (config.py:534), so
Validate() accepts a configured value only when consistent.

Tests cover the descriptor (table-driven over the mainnet/devnet fixtures),
fingerprint determinism/sensitivity, metadata round-trip, reconcile, the
milestone demo (boot one shard into a temp dir, reopen idempotently), and
the loud mismatch errors for shard- and root-genesis config changes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@syntrust
syntrust marked this pull request as ready for review July 14, 2026 09:57
syntrust added 5 commits July 15, 2026 10:10
Shard.New logs "existing genesis validated" on reopen and "genesis
metadata written" on first boot, using the new existed return value.
An empty DB_PATH_ROOT means an ephemeral in-memory database in pyquarkchain
(use_mem_db, cluster_config.py:247) and in goquarkchain (geth's ephemeral-node
convention). Previously it silently created a persistent pebble db in the
process working directory.
… rule set

- The ETH_CHAIN_ID derivation is computed in uint64, matching pyquarkchain's
  unbounded arithmetic; the uint32 sum wrapped silently (worst case to 0,
  which reads as "absent" and puts a wrong replay-protection chain id into
  the EVM rule set).
- Fingerprint() now encodes the whole ChainConfig instead of just the chain
  id, so a change to the compiled-in fork schedule changes every fingerprint
  and forces a loud re-bootstrap instead of executing new semantics on an
  old db.
ReconcileGenesisMeta no longer writes on a fresh db: Shard.New commits the
record once the chain stands at that genesis, so a boot that fails at chain
construction leaves no record behind and the retry re-runs the fresh path
instead of reporting a never-validated record as existing.
syntrust and others added 5 commits July 16, 2026 16:15
Backport the final shape of the shard package into the milestone that creates
it: DBDirName/ParseDBDirName canonicalize the shard chaindb directory names,
and the stub-genesis surrogate paths carry their follow-up TODO markers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The record is a bootstrap stamp standing in for the genesis block, not a
block "meta" in the pyquarkchain sense — half its fields live in the
header there — so free the word before the QKC block format (#1) ports
MinorBlockMeta. The chaindb key moves with it (QKC-genesis-meta ->
QKC-genesis-record); no migration by design: the record is delete-not-
migrate scaffolding, and a stub-era datadir simply re-stamps on its next
boot.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@syntrust syntrust changed the title slave (M2): add shard skeleton with ShardChain stub and genesis metadata slave (M2): add shard skeleton with ShardChain stub and genesis record Jul 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant