libp2p & Ethereum (the Merge)

Prithvi Shahi
libp2p & Ethereum (the Merge)
#libp2p #Ethereum #Merge
Share this item:

If you've kept up with developments in Web3, you've likely heard of a significant event that occurred a few months ago: the Paris upgrade (opens new window), more widely known as The Merge (opens new window), on the Ethereum network (opens new window). This was a historic event for Ethereum and one that was years in the making (Google even had a countdown (opens new window) for it.)

You may be wondering... why is the libp2p blog writing about Ethereum and the Merge? Well, as a result of the Merge, we're excited to share that:

libp2p is integrated into the Ethereum network! 🎉

Today we'd like to highlight years long effort behind this integration, describe how Ethereum uses libp2p, and celebrate this win for Ethereum and libp2p!

Table of Contents:

# A Brief History of libp2p ⏳

Let's begin with a brief reflection on libp2p's history.

Protocol Labs (opens new window) first developed libp2p (opens new window) as a networking library inside of IPFS (opens new window). At the outset, their code and repositories were coupled. However, Protocol Labs soon realized libp2p's potential and utility beyond IPFS, and project maintainers split the two codebases apart. This enabled libp2p to truly become a product-agnostic, reusable, general-purpose networking library.

As a result, the libp2p project saw tremendous growth and adoption across the ecosystem; what started as a humble part of IPFS has come to power much of the decentralized web today. Besides IPFS and Filecoin (opens new window), libp2p is relied on by networks like Polkadot (opens new window), Polygon (opens new window), Mina (opens new window), Celestia (opens new window), Flow (opens new window), and many more.

Each network had varying requirements for their networking stack, but by virtue of its modularity (opens new window), libp2p provided an appropriate solution. However, when projects chose to integrate libp2p in their lifecycles differed:

Some networks relied on it at genesis, while others adopted it as a part of a network upgrade. A notable instance of the latter case, and this article's subject, is, of course, the Merge!

What did the Merge upgrade entail? Let's take a look 👉

# The Merge 🐼

Ethereum's genesis occurred on July 30, 2015 (opens new window) as a part of a milestone called Frontier (opens new window). Naturally, the network and its protocols evolved via numerous upgrades (opens new window) after Frontier. Of those upgrades, the most recent (and arguably the most anticipated) was Paris (opens new window).

Paris, a.k.a. The Merge (opens new window), was executed a few months ago on September 15, 2022 (opens new window).

The Merge encompassed major changes for the Ethereum mainnet. Chief among them were:

  1. Upgrading the consensus mechanism (opens new window) 🚀
  2. Reducing network energy consumption by 99.95% 🌲
  3. Integrating libp2p into the mainnet's networking layer 🤝

Upgrading to proof-of-stake was a multi-year effort. It was executed successfully thanks to diligent planning and a strong foundation laid by the Ethereum Foundation and community. As a result, the network massively reduced its energy footprint (opens new window) and eliminated mining (performing power-hungry computations to produce blocks) necessitated by proof-of-work.

Many congratulations on this historic accomplishment! 🎉

While we're excited about these accomplishments, the last point is most salient to us. Like the consensus upgrade, the libp2p integration in Ethereum was also a culmination of many years of hard work.

Let's take a closer look at the collaboration between Protocol Labs and the Ethereum community.

# How libp2p was integrated into Ethereum 🤝

In the early days of Ethereum and libp2p, some commonly asked questions were: "Does Ethereum use libp2p?" or "Why doesn't Ethereum use libp2p?"

Until recently, the answer to the first question was no. The reason for that was the answer to the second question: libp2p didn't exist when Ethereum was first developed, so it never got a chance to be evaluated and/or adopted.

Felix Lange (opens new window), developer of go-ethereum (Geth) (opens new window) at the Ethereum Foundation, reflected on this in an article titled "Ethereum ♥ libp2p" (opens new window):

"libp2p didn't exist when we started building the peer-to-peer networking stack for Ethereum. There were discussions about building something together very early on, but in the end we were more set on shipping a working system than to discussing how to make the most flexible and generic framework."

Thus, prior to the Merge, Ethereum solely used devp2p (opens new window), a dedicated networking stack and set of networking protocols, not unlike libp2p in some manners. And though there were talks between the Ethereum and IPFS/libp2p communities to have one solution instead of two, the timing didn't work, and Ethereum shipped with devp2p as its solution.

To learn more about differences between devp2p and libp2p, please checkout the libp2p documentation: Comparing devp2p and libp2p (opens new window).

The desire to integrate libp2p into the network never fizzled out. Because both of these projects are open-source (hooray!), much of the collaboration happened in public, and we can see how things evolved over the years.

# Early Days (2016-2017) 👶

Devcon 2 (opens new window) was one of the first instances where Protocol Labs presented the possibility of extending Ethereum with libp2p. There, David Dias (opens new window) gave a talk on September 2016 titled "libp2p ❤ devp2p: IPFS and Ethereum Networking (opens new window)".

In this talk, David gave an overview of libp2p and gave a demo. He ran the EVM in a browser and modified a go-ethereum/Geth node to use libp2p. The Geth node running libp2p connected to the Ethereum mainnet, and the EVM nodes in the browsers (also running libp2p) were able to fetch blocks from the Geth node.

At the time, this was a big breakthrough and a sign of things to come.

Following this presentation, Protocol Labs and Ethereum continued working together (opens new window) to see how it would be possible to integrate libp2p.

[GitHub Link.]

Conversations continued into Devcon 3 in 2017. There, Felix gave a talk on devp2p (opens new window) which mentioned integration options with libp2p. Following the event, he published the "Ethereum ♥ libp2p (opens new window)" article that expanded on what a libp2p integration could look like:

"I've been thinking about ways to integrate libp2p for quite a while...My initial goal was to join their DHT, which turned out to be impractical. There's a much easier thing we can do though: use libp2p's transports and multiplexing algorithm. This saves us a bunch of code and gives us a few new protocols...that we can try out without too much work...The ultimate goal is to adopt libp2p's transport protocols as the way Ethereum nodes communicate."

This showcased the desire of both communities to leverage the benefits of libp2p.

# Designing libp2p into the Beacon Chain (2018-2019) ✍️

The above discussions were about integrating libp2p into the Ethereum mainnet, either replacing or augmenting devp2p.

In 2018, the Ethereum community focused significantly on how to scale the network. As the year progressed, the plan became to deliver both scalability and proof-of-stake as a part of "Ethereum 2.0" (for more information, see: State of the Ethereum Protocol #1 (opens new window)). Both solutions were to be developed on a separate chain from the mainnet, called the Beacon Chain (opens new window).

At this time, the Ethereum 2.0 community began evaluating the networking requirements for the Beacon Chain. Designing a libp2p integration became a formal effort. Focus moved toward adding libp2p to the Beacon Chain instead of extending/replacing devp2p on the Ethereum mainnet.

Raul Kripalani (opens new window), then the libp2p team lead at Protocol Labs, joined in on many of the early Eth 2.0 implementers calls (opens new window), advocated for libp2p, and supported the Ethereum community in evaluations and decision-making.

[GitHub Link.]

As the implementer calls and technical designs progressed, it was decided that the Beacon Chain would use libp2p for networking. The Ethereum community and Protocol Labs drafted initial minimum requirements:

[GitHub Link.]

These efforts culminated in libp2p's formal addition to the Beacon Chain consensus specification in early 2019.

[GitHub Link.]

The "initial" standardization shown above was not static, and the networking specification went through many iterations before (opens new window) and afterward (opens new window), with people across the world putting in crazy hours!

Collaboration between these communities and the Beacon Chain's use of libp2p yielded many desirable outcomes.

  1. Several new libp2p implementations: In order to achieve a high degree of client diversity (opens new window), numerous clients existed in the Eth 2.0 ecosystem. They consisted of the following:

    libp2p implementations didn't exist for JVM, Python, Nim, and .NET, so clients in those languages had nothing to use. This resulted in the birth of new libp2p implementations! Protocol Labs and the Ethereum Foundation co-funded some of these implementations: py-libp2p (opens new window), nim-libp2p (opens new window), jvm-libp2p (opens new window).

  2. New libp2p specifications (and standardization of spec documents): Because libp2p was to be used in the Beacon Chain, the project specifications were given a lot more scrutiny. This empowered libp2p maintainers to invest a lot of time into making the specs the best possible. One example of this was adding a new Noise encryption spec.

    An open question posited in an early wire protocol draft (opens new window) considered whether to use Noise (not yet standardized in libp2p) in lieu of Secio or TLS 1.3 (both supported in libp2p). This created the motivation to create a spec for and standardize the Noise handshake in libp2p (opens new window).

    Many key developers of Ethereum 2.0 were also formally added (opens new window) as interest group members of the Noise spec per the libp2p spec process (opens new window). Noise was eventually added as a formal requirement (opens new window) of the networking spec, and teams started building new implementations (like js-libp2p-noise (opens new window).)

  3. Hardening & formalization of existing libp2p implementations: GossipSub was selected for use, resulting in efforts to test it extensively. The Ethereum Foundation and ConsenSys co-funded a grant (opens new window) to "analyze the libp2p gossipsub implementation, ...help refine the networking stack and specification, and advance interoperability efforts."

    This effort was led by Whiteblock (opens new window) who published their tests and results as a part of their gossipsub-tests (opens new window) and p2p-tests (opens new window).

    Here's a presentation given by Whiteblock on their efforts at Devcon V: Networking in ETH2.0 (opens new window) Around this time, Protocol Labs also started to formalize the GossipSub implementation.

  4. libp2p presence at Web3 events and increased community growth: In this period, libp2p technology was in the spotlight by virtue of its use in Eth 2.0.

  5. Analysis of the collaboration: The effort of integrating libp2p in Eth 2.0 also presented a valuable opportunity to reflect on what was going well in the collaboration and areas that Protocol Labs could improve on. In July 2019, a study of libp2p and Eth2 (opens new window) was shared.

    The report detailed struggles shared by Ethereum teams in the integration process and provided recommendations on overcoming these issues.

# Full speed ahead (2020-present) 🏎️

The Eth 2.0 team started hosting dedicated "networking" meetings (started in Dec 2019) in addition to implementer calls. These meetings were dedicated to discussing progress and unblocking libp2p-related endeavors (opens new window) (GossipSub paper & peer scoring, Noise, etc.) As before, libp2p core team members were key participants.

# GossipSub v1.1

Updates to GossipSub and its formalization efforts also made significant headway.

Protocol Labs:

Similarly, the Ethereum 2.0 team:

[GitHub Link.]

Finally, on December 1, 2020, the Beacon Chain genesis occurred!:

# Beacon Chain Networking Spec 📝

Since its inception, the networking specification of the Beacon Chain has seen a steady stream of changes and updates. After mid-2021 (August), the spec mainly had matured, and the libp2p requirements were all locked in (in fact, there were no further updates until January 2022.) (However, the specification is still a living document. Updates and tweaks to parameters are still being made today.)

The most up-to-date networking specification can be found here (opens new window).

As we saw, this spec resulted from years of close collaboration between the Ethereum community and Protocol Labs. The specification and the libp2p integration would not have been possible without the amazing contributions of all parties. Some of the key teams involved from the Ethereum community were:

During this long developmental cycle, libp2p was running in the above nodes on the Beacon Chain. And once the Beacon Chain was merged with the original execution, libp2p was fully integrated into the Ethereum mainnet!

# How Ethereum Beacon Nodes use libp2p 🔍

So which exact libp2p modules are utilized by Beacon Chain nodes? Per the specification, nodes use the following libp2p features:

All of the modules above, aside from yamux, are mandatory.

One notable omission is the libp2p Kademlia DHT (opens new window). The networking specification instead specifies the use of discv5 (opens new window). The rationale is provided here (opens new window).

The specific design details are out of scope for this post but are described in great detail here (opens new window). Please check it out to see why TCP was chosen (and not QUIC), why Noise, why GossipSub, etc.

# Network statistics 📊

The number of networked Beacon Chain nodes (i.e., nodes running libp2p) is around 5,000 at the current count (opens new window) with this client diversity (opens new window).

[Miga Labs dashboard.]
[Consensus client diversity.]

libp2p is running on all those consensus clients; a fantastic testament to 6+ years of hard work!

Additionally, there are over 495,000 validators (opens new window) amongst these nodes (validators are virtual entities (opens new window) that live on the Beacon Chain.)

# What's next? 🚀

Now that the Merge has gone smoothly, the primary collaboration efforts have been resolved successfully. At this time, Protocol Labs is supporting the Ethereum community in ensuring the libp2p is running smoothly and reliably.

Some of the future roadmap items for libp2p and Ethereum have been hinted at in the specification (like exploring the usage of QUIC). When the time comes for that, the collaboration will pick up steam once more!

Additionally, in the future, we hope to take on work to integrate libp2p into devp2p itself (opens new window). This would enable libp2p's use in Ethereum execution clients (opens new window) in addition to the libp2p's use in Ethereum consensus clients (opens new window) today.

We also hope there's an opportunity to collaborate on enabling access to Ethereum data from the browser (opens new window) by using libp2p transport protocols like WebTransport (opens new window) and WebRTC (opens new window).

In conclusion, the libp2p and Ethereum integration has been a massive effort. It could not have been possible without the grit and perseverance of many organizations and individuals. Thank you to everyone involved, and onwards to our next goals! We hope you enjoyed our summary of the journey and hope you learned a few things along the way.

Thank you for reading! 🙏

# Resources and how you can help contribute 💪

If you would like to learn more about libp2p, please see the libp2p:

If you would like to contribute, please connect with the libp2p maintainers (opens new window).