[FEAT] Support genesis from a merged state

[parithosh]

Feature Description

Current testnet tools all generate a phase0 genesis state and are perfect for testing the merge transition. However, now that the merge transition is completed on mainnet, we'd like to deprecate these tools to save on testing time/complexity with TTD. This would require client teams to support starting from a merge genesis state, i.e - post Bellatrix.

Since such a feature is mostly needed for testnets and internal testing tools, it'd be great to gate the acceptance of such a merge genesis state behind a flag, perhaps --enable-merge-genesis-state. Once the flag is enabled, the merge genesis state can be passed to the Nimbus beacon node via the existing --network flag. In this manner, default behavior need not change (if that is a concern).

Additional context
The feature request was briefly discussed in the interop channel of the Eth R&D discord server: here.

Tagging @z3n-chada and @marioevz for visibility.

[tersec]

Nimbus should support this via https://nimbus.guide/trusted-node-sync.html#sync-from-checkpoint-files. One can capture a post-merge checkpoint state/block pair and start the test Nimbus client using it.

[parithosh]

Would this also apply to a genesis state?

I.e: automation/testing tools would create a genesis state, use an init step that runs the commands needed for trusted-node-sync and then as the last step start the client.

[tersec]

#4230 being merged helps here.

I'm specifically suggesting using the --finalized-checkpoint-block and --finalized-checkpoint-state command-line arguments to provide a post-merge, epoch-aligned state/block pair. This doesn't necessarily require that separate init step.

There are some caveats:

(1) it will attempt to backfill once it's done syncing to head, and in general, won't treat the state you provide as a true genesis state. It will start fine merged though;

(2) this epoch-alignment requirement means that it probably cannot be the merge transition block; and

(3) if the tests require any kind of restarting, the restarted nodes cannot use those --finalized-checkpoint-block and --finalized-checkpoint-state arguments anymore, because the database will already exist. It will error on startup if one attempts to do so.

If these caveats aren't too problematic, Nimbus might already support this use-case.

[arnetheduck]

hey @parithosh, as of #4251 (latest unstable) I believe this should work using the --network=path/to/genesis-folder - we don't have good tooling right now to create such a genesis, could you give it a spin / point us to a non-phase0 genesis?

[parithosh]

We have the Shandong testnet that runs all the shanghai EIPs (no change from the CL side, so it should just work): https://github.com/ethereumjs/consensus-deployment-ansible/tree/master/shandong-testnet/custom_config_data

Alternatively we also have a non-phase0 genesis in this verkle testnet, currently lighthouse/teku/lodestar are part of it: https://github.com/ethpandaops/verkle-testnet/tree/beverly-pos-relaunch/beverly-hills-testnet/custom_config_data

You can also generate a fresh one with this docker command:

docker run --rm -it -u $UID -v $PWD/data:/data ethpandaops/ethereum-genesis-generator:latest all

Guide on configuring it can be found here: https://github.com/ethpandaops/ethereum-genesis-generator/

[arnetheduck]

We have the Shandong testnet

I looked at this genesis, and there's an oddity in it which prevents nimbus from loading it: the fork version of the state says previous_version: 01337902, current_version: 21337902, epoch: 0 which seems wrong: the fork before bellatrix is altair which according to the config has version 11337902 - is there a spec for this somewhere? ie presumably a merged testnet also contains altair.

[arnetheduck]

changing theconfig.yaml to use 11337902 as GENESIS_FORK_VERSION allows nimbus to load the state and start looking for peers

[arnetheduck]

hey @parithosh did you have any success with this?

[parithosh]

I apologize for the delay, the eth2-testnet-genesis tool has been fixed and will now use the previous_version to refer to the actual previous fork (i.e, altair for bellatrix state) and it has been patched downstream on the genesis-generator tool used for testnets.

The fix works and has been tested using a local testnet consisting of nimbus and lighthouse nodes.

Related downstream PRs:
protolambda/eth2-testnet-genesis#10
ethpandaops/ethereum-genesis-generator#15
kurtosis-tech/zzz-deprecated-eth2-merge-kurtosis-module#159

[zah]

Closing as implemented and confirmed to work.