Lido x Obol Simple DVT Testing Report
Introduction
Today we’re excited to share a performance report detailing the first wave of testing for Lido’s Simple DVT module, the first mainnet module to be added as part of Lido V2. Overall, Obol’s DV middleware satisfied the performance requirements to move forward with a mainnet deployment of Lido’s Simple DVT module!
📑 Read the full report here: obol.tech/lido_obol_3.pdf
The testing effort included:
🌐 32 DV clusters
🧑🤝🧑 196 participants
🛜 3,100 validators
🗓️ for 69 days
This test on Holesky was in preparation for the upcoming deployment of Lido’s SimpleDVT staking module on mainnet. Lido will use Obol DVT as a technology primitive to diversify and scale their node operator set, while significantly reducing risks and improving Lido’s resilience, scalability, and decentralization. DVT’s features boost participation and collaboration by allowing smaller operators to join forces with larger ones, fostering a more diverse and resilient Ethereum network.
This latest round of testing follows two previous successful testing rounds performed with Lido. Each of the tests ran for an extended period of time (over 59 days) and demonstrated the ability of DVs to maintain high uptime and effectiveness. The previous testing reports can be found below:
📑 Lido Testing Report #2 (March - May 2023, Görli testnet)
📑 Lido Testing Report #1 (Oct ‘22 - Jan 2023, Görli testnet)
See below the summary section of today’s newest report:
Summary - Testing Results of Simple DVT, Wave 1 on Holesky:
The Simple DVT testing program, conducted by the Obol and Lido teams on Ethereum’s Holesky testnet, spanned 69 days from November 3rd, 2023, to January 11th, 2024. The metrics presented here were derived from a 45-day monitoring period starting on November 18th, 2023. Throughout this monitoring phase, the aggregate metrics of Obol's DVT clusters surpassed both Lido’s minimum requirements and the Holesky network averages, as shown in the table below.
Testing included 32 DV clusters utilizing a “5-of-7” threshold configuration. Initially, this included 214 participants, which later fell to 196. Each cluster initially ran 5 validators, a number that was subsequently scaled up to 50 and then 100 for most clusters, except for Glacial Gull and Crimson Coyote, which remained with 50 validators due to performance constraints. The testing concluded with all DV clusters exiting on January 22nd, with node operators subsequently claiming their rewards with the Obol Splits smart contracts. These results show that the mainnet deployment of the Simple DVT module can safely and efficiently onboard a significant number of new Node Operators to the Lido operator set.
Throughout the testing, various challenges were encountered, primarily stemming from infrastructure limitations attributable to the nascent stage of the Holesky testnet. The absence of established community infrastructure components such as SAFE, Splits.org, The Graph, MEV-Boost, and MEV relay infrastructure further compounded these challenges. However, despite these hurdles, the testnet managed to achieve a Block Proposal Success Rate of 71.2% during the monitoring period from November 28th to January 11th, surpassing the targeted 70% and eventually exceeding 85% during the final week of observation across all clusters. The top performing clusters maintained a proposal rate of 100% across the entire testing period.
Key Learnings & Recommendations
Ensure Beacon Node Health: The Charon middleware exhibited no inherent flaws but required operators to carefully tailor their node settings. It is imperative to ensure the continuous health and stability of the beacon node to maintain overall network integrity and functionality.
Maintain Low Intra-cluster Latency: Challenges with latency were particularly pronounced for operators located in Asia and Australia, as well as for multi-continent clusters, impacting their consensus times and overall performance. It is recommended to keep intra-cluster latency below 235 milliseconds for all nodes.
Provision Sufficient Hardware, ideally Bare Metal or on Prem: Some participants’ inadequate hardware (low-performance virtualized servers and/or slow HDD storage) hindered performance. In order to optimize both performance and cost-effectiveness, it is recommended to prioritize physical over virtualised setups. (Note that the Charon client itself takes less than 500MB of RAM and minimal CPU load, the majority of the hardware requirements stem from the Execution and Consensus clients.)
Carefully Select MEV-Boost Relays: Integration of MEV-Boost and relay functionalities, especially with newly introduced support on Holesky, highlighted relay performance disparities and the potential for optimizations to mitigate latency. It is recommended that an analysis of performant, low latency relays be done before selecting which ones to include as part of a DV node configuration.
Lido Exit Sidecar Optimization: Testing also facilitated optimization of the Lido Exit Sidecar feature, presenting potential applications beyond its initial scope.
What’s Next?
With the Obol Simple DVT clusters passing the minimum requirements on Holesky, the Simple DVT module will move to mainnet deployment. For operators that were not able to participate in this last testnet, there will also be future rounds of Simple DVT testnet announced in the coming weeks.
The next round is expected to commence in the next 1-2 months. Please fill out this form if you are interested in participating.