Community Testing

Launching the Obol Operator Community

Collin Myers

Jul 13, 2022 • 6 min read

Today we are excited to launch the Obol Operator Community (OOC), in connection with the launch of our first public testing program on Görli, called Athena. The OOC is designed to onboard staking products, enterprise validators, and client teams into the Obol Ecosystem for deeper collaboration across research, development, and testing.

Our first collaborative task will be testing Charon, our distributed validator middleware. This will require solo deployments (trusted) and collaborative deployments (trust minimized).

You will run our Charon middleware client, along with a Goerli execution client(s), Prater consensus client(s), and validator client(s). This first testing program will not be incentivized and is for the public good of Distributed Validator Technology.

While our public testing is being designed to be as hands-off as possible, the OOC is designed to be hands-on and a highly collaborative initiative. This is our chance as a core team to find alignment with different participants in the industry that are aligned with our vision of DVT and helps us better determine how we can maintain and sustain distributed signing technology across the entire Ethereum stack.

What is the OOC?

The adoption of DVT is a critical post merge technology to help alleviate a number of technical failures found in current validator set-ups, such as crash faults, availability failures, key compromise threats, and byzantine attacks.

Today it does not matter if you are running at home or you are a medium to large scale provider....we are all at risk of certain technical failures that DVT can help alleviate.

As the Obol Core Team, it is our responsibility to start onboarding and collaborating with key infrastructure providers to enable the smooth transition and on-going build of the DVT primitive.

The Obol Operator Community (OOC) is being designed to lead, educate, and coordinate a collection of liquid staking pools, client teams, and validator operators towards a DVT-enabled future.

We will begin our collaboration together by launching a 30 day testing program for participants who opt-in to participate.

What are the Goals of the OOC?

At a high level our primary goals are:

1. Spread awareness of how DVT can be used to advance your products

2. Mature coordination in the infrastructure industry to protect post-merge Ethereum

3. Test Charon on a larger global scale to prepare for mainnet readiness

4. Advance long-term relationships with aligned LS Pools, staking providers, and client teams

This testing will be the first time our Charon client will be run on a larger scale beyond our at home validator efforts. We ultimately intend to scale our effort into a simulation scenario where ~25,000 validators are running in distributed validator clusters.

We intend to push the limits of DVT middleware to explore where it can be improved and broaden its scope of application.

What Are The Testing Requirements?

Depending on what you sign up to, participants will be required to run Charon on their own and/or required to collaborate with other OOC participants to form a distributed validator cluster together; each cluster will be responsible for running one or more distributed validators.

This effort will be divided up into phases which are explained below:

Phase 0 - Kick-Off Call
Phase 0 is designed to have a 45 minute onboarding call where we will walk through the below agenda.

1. Status update on the Obol Project
2. Mutually introduce team members that are accountable for execution of this work stream
3. Explore how you want to use DVT
4. Answer any questions that you may have
5. Communicate next steps for Phase 1

Phase 1 - R&D & Education
Phase 1 is a 60 minute workshop call focused on conducting a live DKG ceremony and an attempt to connect each Charon client together with you and your team to prepare for later collaborative phases and/or centralised use cases.

For this call, participants can take part in the DKG and connection testing exercise from a dev workstation with docker and git installed. To take this exercise further and activate the validators created during the ceremony, will generally need a dedicated machine to host the clients on full time. This machine should be 16GB RAM, with 2TB of disk if you intend to host both execution and consensus clients on it. A static public IP address and open port is also preferred if these machines need to communicate with one another across the public internet rather than a local network.

The Obol Core team will be present to troubleshoot and walk-through this process, explaining and familiarising your team with charon’s middleware architecture and how it might fit into your existing validating and monitoring stacks.

Historically it has been very beneficial to fast follow on Phase 1.

Phase 2: Cluster Groupings

Phase 2 is where we begin to collaborate heavily as a community to create shared validators across the globe. After consolidating feedback from Phase 0 and 1, we will host a town hall with all of the OOC members who wish to participate in collaborative clusters where we will announce testing teams!

As part of the collaborative testing program, operators will need to be able to expose their Charon client to the public internet on a static IP address for it to communicate with the other Charon nodes in the cluster. Consensus clients and execution clients also generally need to be exposed to the internet to perform effectively.

We will send an invite and agenda for this Townhall into our discord following the conclusion of Phase 0 and 1.

Further Testing Design Information

Testing will include two tracks. Please note that this information is subject to change.

Track #1: Charon for Single Operator Use

Purpose: At Obol we intend to incentivize and design DVT to be used in a collaborative manner amongst groups. However, using DVT in centralized/trusted products is also a net positive for the Ethereum network, and for the operator. Adding high availability to your validating setup reduces the risk of downtime, and this redundancy aims to allow you to run more 32 ether validators on a smaller number of servers, improving operating costs. We feel it is important to offer this testing track for those of you who want to test charon for single operator use.

Node Configuration:

Testing participants are required to run the following node configurations

A minimum of four execution client + consensus client pairs for the goerli/prater testnet, ideally running on independent machines (can be in the same k8s cluster)
The ability to expose each charon client on a static IP address and open port if cross-public internet availability is a requirement (or the ability to network these charon clients together via LAN or VLAN approaches otherwise)
Running one charon client and one validator client for each EL+ CL client pair to form a 4 node distributed validator cluster
Assuming a containerised approach or access to sufficient hardware, we can also test 7 key clusters, and 10 key clusters, leveraging the same EL+CL clients used by the four node cluster.

Duration Requirements:

Minimum of 30 days (from Activation)

Start Date:

August 15th - 19th (Deposit made)

Data Gathering Requirements:

Subjective feedback around documentation, clarity of errors, ease of operation etc.
Feature requests and platform support requests across the Obol/charon/DV stack.

Performance Monitoring:

Pushing prometheus metrics to Obol’s hosted collector for aggregate overview
Obol will analyse the performance of the validator publicly via tools like rated.network and beaconcha.in.

Communication Channel:

Private discord channel with core team

Feedback Mechanism:

Private Cluster Discord Channel
Log issue directly into github for technical profiles
Typeform based feedback

Obol Points of Contact:

Oisin Kyne
Chris Battenfield
Collin Myers
Aly Saleh
Dhruv Bodani
Abhishek Kumar

Track #2: Charon for Collaborative Use

Purpose: This track of testing is where things get interesting. Our objectives are to push the technical limits of distribution by pairing everyone up with a group of OOC members to deploy diverse clusters across the cloud and on prem configurations. The Obol Core team will pair participants up for this track, however we promote those of you who have multi-provider products to propose to Obol additional clusters that align with your current products.

Node Configuration:

Minimum one execution client + consensus client pair for the goerli/prater testnet
The ability to expose each charon client on a static IP address and open port
Running one charon client and one validator client to take part as one part of each of the following cluster configurations:
A four operator cluster
A seven operator cluster
A ten operator cluster
A twenty-two operator cluster