Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

State-only checkpoint state startup #4251

Merged
merged 3 commits into from
Nov 2, 2022
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
State-only checkpoint state startup
Currently, we require genesis and a checkpoint block and state to start
from an arbitrary slot - this PR relaxes this requirement so that we can
start with a state alone.

The current trusted-node-sync algorithm works by first downloading
blocks until we find an epoch aligned non-empty slot, then downloads the
state via slot.

However, current
[proposals](ethereum/beacon-APIs#226) for
checkpointing prefer finalized state as
the main reference - this allows more simple access control and caching
on the server side - in particular, this should help checkpoint-syncing
from sources that have a fast `finalized` state download (like infura
and teku) but are slow when accessing state via slot.

Earlier versions of Nimbus will not be able to read databases created
without a checkpoint block and genesis. In most cases, backfilling makes
the database compatible except where genesis is also missing (custom
networks).

* backfill checkpoint block from libp2p instead of checkpoint source,
when doing trusted node sync
* allow starting the client without genesis / checkpoint block
* perform epoch start slot lookahead when loading tail state, so as to
deal with the case where the epoch start slot does not have a block
* replace `--blockId` with `--state-id` in TNS command line
* when replaying, also look at the parent of the last-known-block (even
if we don't have the parent block data, we can still replay from a
"parent" state) - in particular, this clears the way for implementing
state pruning
* deprecate `--finalized-checkpoint-block` option (no longer needed)
  • Loading branch information
arnetheduck committed Nov 2, 2022
commit 68e54467b598dd2c41edc38614f46d3009693360
7 changes: 6 additions & 1 deletion AllTests-mainnet.md
Original file line number Diff line number Diff line change
Expand Up @@ -449,6 +449,11 @@ OK: 2/2 Fail: 0/2 Skip: 0/2
+ roundtrip OK
```
OK: 2/2 Fail: 0/2 Skip: 0/2
## Starting states
```diff
+ Starting state without block OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
## Sync committee pool
```diff
+ Aggregating votes OK
Expand Down Expand Up @@ -598,4 +603,4 @@ OK: 1/1 Fail: 0/1 Skip: 0/1
OK: 9/9 Fail: 0/9 Skip: 0/9

---TOTAL---
OK: 331/336 Fail: 0/336 Skip: 5/336
OK: 332/337 Fail: 0/337 Skip: 5/337
15 changes: 15 additions & 0 deletions beacon_chain/beacon_chain_db.nim
Original file line number Diff line number Diff line change
Expand Up @@ -1101,6 +1101,21 @@ proc getState*(
db.immutableValidators, db.statesNoVal[T.toFork], key.data, output,
rollback)

proc getState*(
db: BeaconChainDB, fork: BeaconStateFork, state_root: Eth2Digest,
state: var ForkedHashedBeaconState, rollback: RollbackProc): bool =
if state.kind != fork:
# Avoid temporary (!)
state = (ref ForkedHashedBeaconState)(kind: fork)[]

withState(state):
if not db.getState(state_root, forkyState.data, rollback):
return false

forkyState.root = state_root

true

proc getStateRoot(db: BeaconChainDBV0,
root: Eth2Digest,
slot: Slot): Opt[Eth2Digest] =
Expand Down
13 changes: 10 additions & 3 deletions beacon_chain/conf.nim
Original file line number Diff line number Diff line change
Expand Up @@ -312,6 +312,7 @@ type
name: "finalized-checkpoint-state" .}: Option[InputFile]

finalizedCheckpointBlock* {.
hidden
desc: "SSZ file specifying a recent finalized block"
name: "finalized-checkpoint-block" .}: Option[InputFile]

Expand Down Expand Up @@ -763,11 +764,17 @@ type
name: "trusted-node-url"
.}: string

blockId* {.
desc: "Block id to sync to - this can be a block root, slot number, \"finalized\" or \"head\""
defaultValue: "finalized"
stateId* {.
desc: "State id to sync to - this can be \"finalized\", a slot number or state hash or \"head\""
defaultValue: "finalized",
name: "state-id"
.}: string

blockId* {.
hidden
desc: "Block id to sync to - this can be a block root, slot number, \"finalized\" or \"head\" (deprecated)"
.}: Option[string]

backfillBlocks* {.
desc: "Backfill blocks directly from REST server instead of fetching via API"
defaultValue: true
Expand Down
Loading