Why We Built Charon as a Middleware

We built the world’s first Distributed Validator Middleware, Charon

If Ethereum is to be the world computer, it must be credibly-neutral and trust-minimised. When the Obol journey started in 2021, we made it our mission to ensure that Ethereum maintains these fundamental qualities, despite risks of centralisation and correlated failure that came with its move to proof-of-stake. The most notable issues included: the high financial barrier of a 32 ETH bonding requirement, the lack of redundancy in a validator making it prone to going offline, the slashing risks of active-passive setups, and potential correlated failures associated with geographical, client, or staking centralisation.

In researching, developing, and now delivering distributed validators to the Ethereum ecosystem, we made design decisions to prioritise security, resiliency, and composability. We didn’t take short cuts on building a DV implementation that would reduce any of these qualities, because that would run counter to our mission of strengthening and decentralising Ethereum.

Today, we’re pleased to see momentum behind DVT, with adoption from liquid staking protocols, node operators, at-home validators, and other parts of the staking ecosystem. Now that several implementations of DVT exist, each with fundamentally different approaches and architectures, it’s a good time to explain why we built a DV middleware, and to communicate the thinking behind the design of Charon.

Distributed Validator Implementations: Three Main Categories

There are three different categories of DV implementations:

1. DV Middleware:  A new DV middleware client is introduced into the validator stack, sitting between the validator and consensus clients.

2. DV Operator Network: A network of node operators utilising DVT, with mesh networking and preprogrammed tokenomics to facilitate and incentivize operators and/or stakers.

3. DV-Based Staking Product: A full staking product - such as a liquid staking protocol or staking-as-a-service interface - that utilises DVT within their node operator set, allowing users to stake on a service operated by DVs.

These approaches are very different, and it’s important to understand that they each operate at different levels of the stack and don't necessarily conflict with each other. For example, a liquid staking protocol (LSP) could make use of an operator set run on a DV network. Likewise, a staking product could be built with node operators who use a DV middleware in their client stack. However, what gets built first has massive ramifications on the value (and potential risks) that are brought to the broader network. 

If the most important qualities to ensure are security, resiliency, and composability, the only implementation that makes sense to start with is a DV Middleware. Here’s the three reasons why:

1. A Distributed Validator Middleware is the most secure

When considering how to maximise the security of a validator, there are two key things to consider—minimising slashing risk and ensuring private key security.

Obol minimises slashing risk by running within on top of existing validator client infrastructure

One approach to DVT design is to replace the validator client with a DVT-native client, which has custody of the private keys and the capability to sign arbitrary data. However, in our opinion a full validator client capable of signing and exfiltrating arbitrary data without the oversight of a second software implementation has much higher risk of causing correlated slashing. 

Instead, we built Obol’s DVT as a secure and trust-minimised middleware architecture. Our middleware client, Charon, doesn’t replace anything in the client stack, instead it sits between the consensus and validator clients. Node operators integrating the Charon DVT middleware into their stack can continue to use the same clients and private key infrastructure as before, albeit with a different key generation method. 

This gives the benefit of having both Charon and the existing validator client as failsafes, greatly reducing the odds of unintended slashing. Even in the worst case scenario where Charon is compromised by a supply chain attack or a remote code execution attack, or the Obol team become bad actors and push a malicious release, Charon cannot do a lot of damage as a middleware. If a compromised Charon client proposes a potential double vote or surround vote for a validator to sign, the validator client will check its anti-slashing database, see that it has already signed something conflicting, and simply refuse to return a signature. Charon could propose that a validator should sign an invalid block, but the chain would reject this and simply consider the proposal missed  - a much better outcome than slashing.

Obol ensures key security via distributed generation of key shares

From a security standpoint, DVT is an opportunity to improve on the security of handling Ethereum validator keys, because node operators reduce slashing risk by running private key shares instead of entire private keys.

When creating a new Obol DV cluster using a distributed key generation ceremony (DKG), the full private key never exists in a single place, greatly reducing the risk of private key compromise. Ahead of a DKG-based cluster creation, each operator generates an Ethereum Node Record (ENR) private key. The ENR private key is only used for node identification, establishing encrypted peer-to-peer communication between the cluster’s nodes, and signing consensus messages. This model is much lower risk than the alternative, where the charon software would have direct access to the BLS private key shares that make up the validator. (In the case that someone wishes to distribute an existing validator, it’s also possible to split an existing validator private key into key shares.)

An alternative approach to doing this is to split it into shares, encrypt each share with the public key of a node operator, and publish the encrypted private key on chain. The operators’ node key could then decrypt the validator private key. In our opinion, this is not secure. We believe that the safest approach is to avoid the existence of a singular private key, and certainly never to post any private key to a public blockchain network.

2. A Distributed Validator Middleware is the most resilient

To make Ethereum more resilient, a DV implementation must improve the non-correlation of the entire network. A DV Middleware implementation does this in two ways—allowing for node diversity within a cluster, and enabling direct private communications between nodes. This direct private communication of nodes within a cluster with one another allows each DV cluster to exist independently from all other DV clusters. A cluster can run an older client version or go offline, without affecting any other DV cluster.

Obol allows for diversity of clients in a DV cluster

Because Obol’s Charon middleware sits in between the validator and consensus clients, the nodes of an Obol DV cluster can run on a diversity of validator clients, consensus clients, and execution clients. (And optionally, remote signers.) If each node is running a unique client stack, an issue with one client will not bring down the whole cluster, protecting distributed validators against the types of client bugs that have brought down large numbers of validators in the recent past.

Obol maximises performance and efficiency through direct P2P communications

In an Obol DV cluster, nodes use LibP2P to communicate directly with each other, and communications are end-to-end encrypted with SSL. This direct communication of nodes with one another has several advantages: when a new version of Obol’s Charon is released, Obol DV clusters can upgrade on their own time, individually from other DV clusters. This direct communication also improves latency, and makes cluster communications harder to attack with a denial of service (DOS) attack. It also allows an Obol DV cluster to be run within a private network. This may allow cost savings on data egress costs, for operators running cluster nodes across multiple locations of a single cloud provider, for example.

The alternative approach is to make use of a public gossip network that all clients connect to. (This approach is typical of a DV network, where all nodes make themselves known and available to be grouped into clusters.) One immediate drawback is that client updates have to activate at the same time across all nodes, since the entire network needs to be using the same protocol. Another drawback: the public gossip network relies on nodes to voluntarily open their ports and participate in peering and gossip, resulting in higher data egress costs, since each node is relaying many more messages than only the ones related to that node’s cluster. This may require some type of artificial incentivization to encourage nodes to participate in gossip messaging. In the event that nodes don’t broadly participate in peering, it may be necessary for the network developers to run a central messaging relayer in order to support the gossip network and ensure continued performance between all gossip nodes. (The costs associated with this central relaying would somehow need to be passed onto the network operators and users.)

3. A Distributed Validator Middleware is the most composable

One of Ethereum’s greatest assets is its composability. For a primitive like DVT to preserve that, it must be built as a middleware - this allows products and technology to build on top of DVT, while making their own design choices for token design, network design, etc.

Obol requires no additional bond, and directly pays out staking rewards as ETH

Obol is building a DVT primitive, and therefore makes no changes to Ethereum’s standard bonding and reward mechanism. Obol does not require nodes to post any bonds additional to the 32 ETH required for a validator. To pay out rewards to operators, splitter contracts like Obol Splits can be used to withdraw and share rewards on a continuous basis. This allows products like liquid staking protocols to be built on top of Obol, implementing a bond or unique token into their protocol, should they choose to do so.

The alternative approach is to make the entire set of design choices, not only around the DVT implementation but also around bonding and rewards. (In other words, to build an entire DV network or product around a DV implementation.) One temptation is to create a token and require stakers to pay operators in that token. This would require stakers to keep a balance of the network token ready for fee paying, in order to continue using the staking service. This mechanism would be informed by oracles, which decide when to post rewards and punish operators. This alternative model has some drawbacks. Namely, the varying price of the network’s unique token will change relative to the price of ETH: operators are not able to determine their commission as a percentage of ETH staked, and stakers likewise must consider the additional initial cost of purchasing the token to determine their long-term rate of return on their staked ETH.

Obol supports others to build DV-native staking products and form DV clusters

Obol chooses to build a DVT primitive instead of playing a central role in the coordination and creation of mainnet DV clusters. Instead of designing our own network, we are supporting staking protocols and node operators to integrate Obol DVT into their own products and infrastructure. (Lido’s Simple DVT is a good example of a 3rd party using DVT to coordinate the creation of Obol DV clusters.) 

The only direct user experience currently offered by Obol is the DV launchpad, which allows users to easily spin up a distributed validator cluster themselves. (The DV launchpad is currently available for beta use on Mainnet and is limited to one validator per cluster.) Staking projects wishing to build products on DVT can use the Obol SDK - a typescript package that allows developers to create and activate Obol DV clusters programmatically, within the user experience of a staking product. 

The alternative approach that a DV network takes is to play a central role in the organisation and formation of DV clusters, building a larger network of stakers and operators. This approach matches operators and stakers together, either randomly or depending on input from the user, instead of staking through a DVT-enabled protocol or staking provider. This approach is not necessarily  “better” or “worse” than our approach, as it depends on the needs of the staking service using DVs, but more managerial responsibility (and potential risk) is placed on the DVT technology provider when they take a central role in the use of their technology.

Obol Labs does issue Techne credentials to node operators who have proven their knowledge and experience with Obol DVs. This helps node operators approach potential partners and cluster-mates with credibility, during a time when many staking providers like Lido and Ether.fi are looking to onboard new node operators to DV clusters. This sees the DV technology provider taking a role in making decisions about the suitability of operators based on their performance, but stops short of making actual managerial decisions or directly determining node operators’ access to stake.

Building lego blocks, staying Ethereum aligned

In this article, we explained how and why we built Obol’s DVT as a middleware: to keep Ethereum secure, resilient, and composable. We’ve started with Obol v1, but we have broader ambitions: Obol v2 will be not only a DV middleware, but an entire DV protocol, supporting multiple middleware client implementations, built by teams other than Obol Labs. Our composable approach will allow us to enable an entire ecosystem of staking products backed by DVs, making Ethereum more resilient and decentralised than ever before!

For information about how DVT can help you, check out our blog articles on how Obol DVT can help at-home validators, liquid staking providers, custodians, and staking-as-a-service providers.

References:

• Obol docs - docs.obol.tech/docs/intro
• Oisín’s blog - 2018 - “The Staking Problem”
• Oisín’s blog - 2020 - “The Staking Problem: revisited”
• Obol blog - Jan 2022 - “Tackling the Staking Problem”
• Obol blog - Oct 2022 - “What is DVT and How Does It Improve Staking on Ethereum?”
• Obol blog - Nov 2022 - “Designing Non-Correlation: Deep Dive Into DVT and Charon’s Architecture”
• Obol blog - Dec 2022 - “Committee Aggregations with Distributed Validators”
• Diva docs - “Diva vs Obol and SSV”
• Diva docs - “How does Diva’s DVT compare to SSV and Obol”