Holesky Incident Update

Cryptocurrency News and Public Mining Pools

Holesky Incident Update

On Monday, Pectra activated on Holesky. An EL bug caused chain splits, resulting in a minority chain being valid, degrading the network health.

Holesky validators must take action ASAP, some Sepolia users must upgrade their client.

If you are reading this and run a Holesky validator, please get it back online ASAP and remove your slashing protection DB (to allow you to attest to the valid minority chain). More info here: https://github.com/ethereum/pm/blob/master/Pectra/holesky-postmortem.md

Sepolia node operators: the fork will happen as previously scheduled at epoch 222464 (Mar. 5, 7:29 UTC).

If you are using Geth, Besu, Nethermind or Lodestar, you must update your client to one of the releases listed here: https://blog.ethereum.org/2025/02/14/pectra-testnet-announcement

FAQ

Why are we coordinating mass slashings on Holesky?

While this strategy could not work on mainnet, the hope is that we can get enough Holesky validators online at once to finalize a block on the correct chain. Validators who had previously attested to the invalid chain will get slashed as a result of doing this. Having a finalized block will allow CLs to more easily find peers on and sync to the valid chain.

After the mass slashings, Holesky will go through a long period (2-3 weeks) of non-finality again as the slashed validators are existed and their stake drops to <33% of the overall network stake. Once that happens, the validators who never attested to the invalid chain will finalize the valid minority chain.

Again, this is not something we could do if such a situation happened on mainnet. CL client teams are already discussing better ways for users to force nodes onto a minority chain in the future. Expect a deeper discussion on the topic on next week's ACDC.

Why are we not delaying Sepolia's hard fork?

The root cause of this situation was trivial to fix. Some EL clients used the wrong deposit contract address for testnets. Releases patching this have already been made.

Sepolia's validator set, unlike Holesky or mainnet, is permissioned, with a large share being run by client and testing teams. This allows for quick coordination of the upgrade.

With this approach, only validators running Besu, Geth, Nethermind and/or Lodestar must change anything. Validators using other clients can keep running the previously announced versions.

Forking Sepolia sooner will give us a longer testing window for Pectra.

How does this affect the Pectra testing process and mainnet fork date?

Holesky was an especially useful testing ground for Pectra because many projects, including staking pools, use it as a staging environment for mainnet. It also has a validator count comparable to L1.

To test Pectra in a high validator count environment, ethPandaOps has already spun up a new devnet with 1M validators 🔥

That said, we'll need to discuss what other testing we were hoping to get out of Holesky and the best way to do that before moving forward with scheduling a mainnet fork date. We'll discuss this on next week's ACDC, but it's unlikely we'll be setting a mainnet fork date right then.

Where can I follow the incident?

Over the past couple days, I've been keeping a post-mortem updated here: https://github.com/ethereum/pm/blob/master/Pectra/holesky-postmortem.md#holesky-coordinated-slashings

Any changes to Pectra's deployment on testnets or mainnet will be announced at http://blog.ethereum.org

submitted by /u/timbeiko
[link] [comments]