rust-libp2p in 2022

rust-libp2p in 2022

# The rust-libp2p Project: A Recap of 2022

The rust-libp2p (opens new window) project has made significant strides in 2022, with numerous technical advancements and improvements to the project itself. Let's take a closer look at some of the technical highlights and on the meta level developments of the project itself. This is the work of many across various organizations including Protocol Labs (opens new window), Parity Technologies (opens new window), Sigma Prime (opens new window), Iroh (opens new window), Actyx (opens new window) and Little Bear Labs (opens new window).

# Technical Highlights

# Decentralized Hole Punching

We started the year with the release of the various components needed for hole punching (opens new window). We added the Circuit Relay v2 protocol (opens new window), DCUtR protocol (opens new window) and AutoNAT protocol (opens new window). These features were all included together in rust-libp2p v0.43.0 (opens new window) released in February. This made v0.43.0 the first rust-libp2p release with hole punching capabilities.

Furthermore, to gain insight into how different variables (environments, transports, etc.) affect libp2p hole punching efficacy, the libp2p project started the Hole Punching Measurement Campaign aka. Hole Punching Month (opens new window) in collaboration with ProbeLab (opens new window). A multitude of clients using both go-libp2p and rust-libp2p (opens new window) are currently punching holes across the globe, providing valuable data that we can later on use to improve the libp2p specification and the two implementations.

# New Transports

Over the year we worked on two new transports, namely WebRTC (browser-to-server) (opens new window) and QUIC (opens new window), which we both released towards the end of the year as alpha/ experimental features.

Our implementation of WebRTC enables browsers to connect to rust-libp2p based servers (opens new window) without those servers needing to have signed TLS certificates. QUIC is a better libp2p transport than the combination of TCP+Noise+Yamux in almost every dimension as it provides faster connection establishment, better native multiplexing, native encryption, and higher hole punching success rates. Along the way, given that QUIC already requires TLS, rust-libp2p can now secure TCP connections with TLS as well (opens new window) (previously only Noise).

# User Experience Improvements

Along the way we tackled many smaller improvements, as a whole having a big impact on the user experience. To mention a couple: naming consistency across crates (opens new window), refactoing of the many inject_* into a single enum event handler in both NetworkBehaviour and ConnectionHandler (opens new window), deprecation of event-based PollParameters (opens new window), the rework of our Rust feature flags (opens new window), and removal of listener streams in favor of polling transports directly (opens new window). Still remember the old days with the NetworkBehaviourEventProcess trait? All gone (opens new window) in favor of the much simpler (generated) OutEvent mechanism. The StreamMuxer trait received numerous significant simplifications, basically rewriting (opens new window) the trait as well as the trait implementation in yamux, mplex and now QUIC and WebRTC.

# DoS Protection

Defense against denial-of-service attacks is a cornerstone of a networking library, especially in the peer-to-peer space. rust-libp2p saw a lot of related improvements in 2022. We enforce various limits (e.g. on the number of streams (opens new window) and bytes of a request) and prioritize local work over new incoming work from a remote (opens new window) across the many layers. Up next is a patch (opens new window) enabling NetworkBehaviour implementations to implement their own connection management strategies.

# Metrics and Observability

Understanding large systems is hard. Understanding distributed systems is even harder. We made understanding large libp2p networks a bit easier in 2022 introducing a metric crate for rust-libp2p exposing Prometheus metrics, e.g. the time to establish a connection (opens new window) or the protocols supported by peers (opens new window).

# Housekeeping & Quality of Life Changes

Since January the rust-libp2p monorepo has a handy CLI tool for libp2p key management (opens new window).

In general, we keep up with recent developments of the Rust language, and incorporate some of its new shiny features. We make use of Cargo's weak dependencies (opens new window). In November we introduced a patch using const generics (opens new window) (in tests). In December we merged our first pull request using GATs (opens new window).

At this point, rust-libp2p is a large complex codebase. In August we added a coding guideline (opens new window) to the project, allowing us to agree on a set of rules to enforce consistency across the project and enable newcomers to hit the ground running. Projects outside of the libp2p realm might find this guideline useful, as it documents an opinionated way of doing asynchronous networking in Rust.

# Meta - Improvements to the Project

libp2p is an open-community open-source project. What do I mean by that? libp2p's source code is open-source and that source code is written by an open community. libp2p is not developed by one or 10 people, but much rather >100 (part-time) people across many organizations.

The core rust-libp2p maintainer team grew from two engineers (myself (opens new window) and Thomas (opens new window)) to four, with Elena (opens new window) and João (opens new window) joining the team. Beyond the core maintainers, a total of 36 people contributed to rust-libp2p's master branch in 2022 (opens new window).

# Automation

One thing that allows us to be productive despite the large number of people contributing to the project is automation. We invested heavily into rust-libp2p's automation.

Big quality of life improvement was the introduction of mergify (opens new window). We enforce semver compliance via cargo-semver-checks (opens new window). We adopted the conventional commit (opens new window) specification. We did a large refactoring of the CI job structure (opens new window), testing crates individually, thus increasing parallelism, improving caching and catching interdependency issues.

# Interoperability

rust-libp2p is one implementation of many of the libp2p specification. How do we ensure we are compatible across implementations? In 2022 we started the libp2p interoperability project and, as of September 2022, we continuously test (opens new window) that the various versions of go-libp2p and rust-libp2p can connect. In December we added nim-libp2p, in 2023 we will add js-libp2p.

In 2022 we published 9 releases of the main libp2p crate and a total of 268 releases across the workspace including sub-crates and patch releases.

# What's Next?

To conclude this post, it's worth mentioning the rust-libp2p project roadmap (opens new window), which was added in October (opens new window) and outlines the planned developments for the project in the future. Some notable features to look forward to in 2023 include improved WASM support (opens new window), the WebRTC browser-to-browser feature (opens new window), and the potential implementation of the new WebTransport transport (opens new window).