lido-snapshot.webp

Primary_Horizontal_LightBG.png

ToC

Overview

Hi everyone, thank you for joining the 3rd round of testing for Lido’s integration of Obol based DVT on Holesky. All participating members have been split into various clusters that represent your group to operate DVs (distributed validators) and will represent an entry in the Lido Node Operator registry. Each cluster will coordinate in a corresponding Discord channel thread that will be used to set up a SAFE Multisig, represent an entry in the Lido Node Operator registry, coordinate a DKG ceremony, and run distributed validators on Holesky.

Prerequisites

First Steps

When your team has chosen the cluster coordinator, please tag @kimonsh and @perrier. A form (linked below) will be used to collect each cluster member’s Holesky addresses that they plan to use to sign messages in the SAFE, sign the distributed validator config and to receive validator’s rewards to.

Each cluster member must submit the form and verify the address(es) they would like to use. Each cluster member should submit up to two addresses:

  1. Required: Your individual manager address. This will be your address used for the SAFE Multisig and for signing messages related to your distributed validator cluster in the DV Launchpad. The launchpad only supports Metamask at the moment so you need to be able to connect to Dapps via Metamask (which can also import a hardware wallet for example). If you would like to, it can also act as your individual reward address. Each cluster member must submit the address and verification (see “Address Verification” below) in the form.
  1. Optional: Your reward address. This will be the address added to the reward splitter contract. The splitter contract will evenly distribute rewards between all members of your cluster. This rewards address should be used if you would like to use an address other than the one used for your manager address to collect rewards. If you provide a reward address you will need to sign a message verifying ownership of it in addition to the manager address’s verification.

Each individual participant is responsible for the security and storage of the private key(s) related to these addresses. Please confirm that you have your seed phrase backed up and the address is secure before submitting.

Use this form to submit your address and verification link (see verification instructions below): https://forms.gle/rCKvBhU4AzDHgqpY8

Address Verification

Use the following guide to prepare and sign a message that will be used for verification of your chosen manager and (where applicable) reward address:

In case of using externally owned account (EOA):

  1. Sign the message on mainnet using the text (add your specific details within <>: with the private key you’ll use as the signing key. One of the options is using Etherscan.
    1. Go to https://etherscan.io/verifiedSignatures
    2. Press the “Sign Message” button and connect your wallet
    3. Input the address you’re verifying (your Individual Manager Address or Individual Rewards Address)
    4. Enter your message using the following template (add your specific details within <>: “<my name> is joining Lido x Obol cluster <cluster name> with address <public address>”
    5. Press sign message and if the message is successfully signed, publish it.
    6. Submit the verified signature URL via the form (e.g. https://etherscan.io/verifySig/27336).

SAFE Multisig

Once the form is filled out, please tag @kimonsh and @perrier and wait for confirmation to proceed further.

Once each cluster members has submitted the form and Lido NOM team has approved, the addresses will be shared to the group and the cluster coordinator will create the 5/7 threshold SAFE Multisig using each cluster member’s individual manager address on Holesky (https://holesky-safe.protofire.io/welcome). When the SAFE is created, share the SAFE Holesky URL for your cluster to review and tag @kimonsh and @perrier. This address will be your cluster’s representation in the Lido Node Operator Registry on Holesky.

Each cluster should also pin a message with basic info in their threads:

<participant name>   :  <ETH address here>   | <participant cluster identifier>
<participant2 name>  :  <ETH address here>   | <participant3 cluster identifier>
<participant3 name>  :  <ETH address here>   | <participant3 cluster identifier>

repeat above 'n' times where 'n' = cluster size

The cluster identifier will be obtained at the end of the DKG (steps below). Update the message to reflect it once done.

During this time, the Simple DVT Module Committee will add your cluster to the Lido Node Operator Registry using the SAFE multisig you provided.

Operator Joining Flow

Create your ENR

All operators should familiarize themselves with the Obol Launchpad Quickstart Guide and launchpad walkthrough video.

All cluster participants should have already shared their individual manager and reward addresses in their cluster-specific form.

To prepare for the distributed key generation (DKG) ceremony, you need to create an ENR. On the machine where you plan to run your Obol DVs, run the following commands to get your ENR.

# Clone this repo
git clone <https://github.com/ObolNetwork/lido-charon-distributed-validator-node.git>
 
# Change directory
cd lido-charon-distributed-validator-node
 
# Create your charon ENR private key, this will create a charon-enr-private-key file in the .charon directory
docker run -u $(id -u):$(id -g) --rm -v "$(pwd):/opt/charon" obolnetwork/charon:v0.17.2 create enr

You should see an output similar to the below:

Created ENR private key: .charon/charon-enr-private-key

enr:-JG4QGQpV4qYe32QFUAbY1UyGNtNcrVMip83cvJRhw1brMslPeyELIz3q6dsZ7GblVaCjL_8FKQhF6Syg-O_kIWztimGAYHY5EvPgmlkgnY0gmlwhH8AAAGJc2VjcDI1NmsxoQKzMe_GFPpSqtnYl-mJr8uZAUtmkqccsAx7ojGmFy-FY4N0Y3CCDhqDdWRwgg4u

<aside> ⚠️ Please make sure to create a backup of the private key create at .charon/charon-enr-private-key. Be careful not to commit it to git! If you lose this file you won't be able to take part in the DKG ceremony and start the DV cluster successfully. ⚠️

</aside>

Creating the Obol Distributed Validator

Each cluster config will be created by the Obol team who will send a specific invite code to each cluster as a URL link.

Operators will follow the invite link to their cluster set up page in the DV Launchpad, they will review the cluster configuration, input their previously-generated ENR, and sign a message using their individual manager address provided earlier to confirm the cluster configuration.

Here are the steps to follow in detail once on the invite link:

DKG Ceremony

Once all operators have successfully signed the operator configuration, the next step is to perform the Distributed Key Generation ceremony.

A command will be automatically displayed in the launchpad once all operators have signed the config. Running that command will start the DKG process for your node.

Please note this process requires semi-synchronous coordination between all cluster operators. Your cluster should determine a time window of ~ 6 hours that all members can meet to conduct this one-time process.

When you start the DKG ceremony, you shouldn’t stop the process until all members have completed the ceremony.

You can’t just run the command and then turn your machine off! You must wait for everyone to run the command and the DKG to finish (<60sec). However, you can let the command run in the background as it will retry the DKG until successful.

When the DKG ceremony has been completed, operators will see the following messages:

All peers connected, starting DKG ceremony
dkg Successfully completed DKG ceremony 🎉

The artifacts of the DKG ceremony will be created in the .charon folder for each member (inside their lido-distributed-validator-node repo).

💡 Please ensure that ALL the operators have the above files created successfully before moving forward.

<aside> ⚠️ Please ensure that you have backed up your validator_keys/ folder

</aside>

Additional required configs

Add Operator ID to .env

While the quick-start guide linked above is adequate for general-purpose deployments, this repository needs further setup to integrate with Lido’s smart contracts:

  1. grab your Operator ID from https://operators-holesky.testnet.fi/ (it will be present after Simple DVT Module Committee adds your cluster to the registry)
  2. Run the following three separate commands to add the Operator ID into the .env file in the validator-ejector config section:
cp .env.sample .env

export MY_OPERATOR_ID=your operator id here

sed -i.bak "s|#VE_OPERATOR_ID=|VE_OPERATOR_ID=$MY_OPERATOR_ID|g" .env

Reference the README for more information.

Monitoring

The cluster will need to push distributed validator metrics to Obol’s central Prometheus service to monitor, analyze and improve cluster performance. This data will also be made available to the operators via a login to the Grafana Dashboards.

The below token needs to be added in prometheus/prometheus.yml replacing $PROM_REMOTE_WRITE_TOKEN.

oboln!auNAZyJs!IYneXhQviJICT0H?mcxuZjO2g=WXqJTbjs-9r2P52q!vlDNpq?eLx7gbgmJgKDCmnxtoQMgZ5KmDeXTMttmlRsF/dNzxoePjkIbKWuGY25v2fc9RO

The final prometheus/prometheus.yml will look like:

global:
  scrape_interval: 30s # Set the scrape interval to every 30 seconds.
  evaluation_interval: 30s # Evaluate rules every 30 seconds.

remote_write:
  - url: <https://vm.monitoring.gcp.obol.tech/write>
    authorization:
credentials: oboln!auNAZyJs!IYneXhQviJICT0H?mcxuZjO2g=WXqJTbjs-9r2P52q!vlDNpq?eLx7gbgmJgKDCmnxtoQMgZ5KmDeXTMttmlRsF/dNzxoePjkIbKWuGY25v2fc9RO
    write_relabel_configs:
      - source_labels: [job]
        regex: "charon"
        action: keep # Keeps charon metrics and drop metrics from other containers.

scrape_configs:
  - job_name: "geth"
    metrics_path: /debug/metrics/prometheus
    static_configs:
      - targets: ["geth:6060"]
  - job_name: "lighthouse"
    static_configs:
      - targets: ["lighthouse:5054"]
  - job_name: "charon"
    static_configs:
      - targets: ["charon:3620"]
  - job_name: "lodestar"
    static_configs:
      - targets: [ "lodestar:5064" ]

Please let us know if you have any questions by using your cluster thread in the simple-dvt-testnet-cluster-coordination channel in Discord (either the general channel or group-specific thread).

Start your Distributed Node