Announcing the release of js-libp2p v1.0.0 🎉

Chad Nehemiah
Announcing the release of js-libp2p v1.0.0 🎉
#browser #transport #webrtc #js-libp2p
Share this item:

# Announcing the release of js-libp2p v1.0.0 🎉

js-libp2p has been used in production for many years in IPFS and Ethereum (opens new window), as well as a wide variety of other ecosystems (opens new window). Over the years we have worked tirelessly to improve its functionality and performance based on feedback and insights from real world usage in peer to peer networks. Today, we're excited to announce the release of js-libp2p v1.0.0 (opens new window) 🎉

# What's new? 🤩

# Smart Dialing

One of the major inefficiencies we recognized was how we dialed peers; particularly in the browser. Some peers were being dialed on the same address multiple times despite having recent failures, and some peers were being dialed when they were not even online. Dials were also being wasted on peers on unreachable parts of the network, as in the scenario where IPv6 is not supported. This excessive dialing which was wasting resources, lead us to the development of smart dialing, which is a dialing strategy that is more aware of the network topology and the reachability of the multiaddrs it is dialing. Smart dialing is able to make more meaningful decisions about which peers to dial, and when to dial them. This has led to a significant reduction in the number of dials made, and has also led to a significant reduction in the number of failed dials; which has resulted in a much more efficient network and better peer management.

# Circuit Relay v2

One of the major hurdles in establishing a connection on the web is NAT traversal. Circuit relay was introduced as a means to establish connectivity between libp2p nodes that wouldn't otherwise be able to establish a direct connection to each other. In many cases, peers were be unable to traverse their NAT and/or firewall in a way that made them publicly accessible.

To enable peer-to-peer architectures in the face of connectivity barriers like NAT, libp2p defined a protocol called p2p-circuit. When a peer isn’t able to listen on a public address, it can dial out to a relay peer, which will keep a long-lived connection open. Other peers will be able to dial through the relay peer using a p2p-circuit address, which will forward traffic to its destination.

The circuit relay protocol was inspired by TURN (opens new window), which is part of the Interactive Connectivity Establishment collection of NAT traversal techniques. Circuit Relay v2 has significant improvements over v1, including:

  • Support for resource reservation which allows for more efficient use of relay resources through explicit reservations.
  • Limitations placed durations and data caps on relayed connections.
  • Introduction of two sub-protocols being hop and stop which were used to reserve and govern these resources effectively.

You can read more about Circuit Relay v2 here (opens new window)

# NAT Hole punching with DCUtR

As mentioned in Circuit Relay v2, relays are used to traverse NATs by acting as proxies, but this can be expensive to scale and maintain; and may result in low-bandwidth, high-latency connections. Hole punching is another technique that enables NAT traversal by enabling two nodes behind NATs to communicate directly. The libp2p DCUtR (Direct Connection Upgrade through Relay) is a protocol for establishing direct connections between nodes through hole punching, without a signaling server. DCUtR involves synchronizing and opening connections to each peer’s observed external addresses. You can read more about DCUtR here (opens new window). Implementing this in js-libp2p has benefitted browser nodes significantly, as it has enabled them to establish direct connections with other nodes without the need for a relay.

# WebRTC Private-to-Private Connectivity

Currently js-libp2p is the only implementation that supports private-to-private browser connectivity using WebRTC. This is a major advantage for js-libp2p, and is a major advantage for the browser ecosystem as allows direct peer-to-peer connectivity within the browser regardless of whether nodes are located behind NATs / Firewalls. A lot of effort was invested in making this a stable and resource sensitive transport that can be reasonably used across browser nodes. For more information, check out the private-to-private WebRTC spec (opens new window).

To follow the private-to-private implementation in go-libp2p, please see this tracking issue (opens new window).

# WebTransport

Historically, WebSockets were the only way for browsers to establish full-duplex two-way communication with servers. One of the challenges with WebSockets is that they require a TLS certificate signed by a certificate authority tied to a hostname when a page is loaded over HTTPS — something that nodes in peer-to-peer network often don't have.

WebTransport is a new web standard that allows for the creation of bidirectional, multiplexed connections between a client and a server built on top of QUIC. One of the many benefits of WebTransport is support for verification of a TLS certificate hash. This allows establishing WebTransport connections to servers that only have self-signed certificates. The browser trusts the server if the hash of the certificate used during the handshake matches its expected hash. This facilitates the creation of a low-latency, high-bandwidth connection between two peers without relying on third party certificate authorities.

You can read more about WebTransport in this blog post (opens new window) and you can also read the spec (opens new window) for more technical details.

# Performance Improvements and Optimizations 🔧

Although many of the performance improvements to js-libp2p were driven by feedback we've received from the community, we've also been informed by our own performance benchmarking and profiling. We've removed a lot of async operations, fine-tuned our stream multiplexing, reduced round trip times and memory usage, and done many other optimizations. Here's some of the major milestones we've achieved:

# Perf Protocol

The perf protocol (opens new window) was created to help us analyze performance libp2p implementations through client-driven benchmarks. This revealed a variety of performance issues, and subsequently, allowed us to make a variety of optimizations. You can read more about the specifics of the test setup here (opens new window) but essentially two libp2p nodes are spun up on different AWS servers, and they then send data to each other continuously over a period of time. The data is then analyzed to determine the performance of the libp2p implementation, you can view these visualizations on our performance dashboard (opens new window). Here are some of the optimizations we made based on the results of the perf protocol:

# Reducing latency

We've made an 80%+ improvement in the time it takes to establish a connection between two js-libp2p nodes.

Latency of a connection between the two nodes.

# Increasing throughput

We've been continuously improving the throughput of js-libp2p, and currently js-libp2p has the highest throughput of any of the libp2p implementations, averaging 2.06GB/s on both uploads and downloads!

Throughput of a connection between the two nodes.

# Reducing Dependencies

We realized that the libp2p bundle size was unnecessarily big, and that we could reduce the bundle size by removing some of the modules that are tangential for many use cases, such as fetch , UPnP, keychain etc. This reduced the bundle size by over 40%! We still think there is room for even further reduction, and we will be working on this in the future.

# Package Provenance

There is no single answer to the problem of software supply chain integrity. However, there are a variety of techniques that can be used to mitigate the risk of supply chain attacks. One of the techniques we've implemented is package provenance. When you download libp2p from the npm registry you now have visibility into the process by which the source code was translated into the published artifact. You can verify the integrity of provenance attestations via the npm CLI. You can read more about package provenance here (opens new window)

# Developer Experience Improvements 🌈

As the codebase grew, a variety of pain points were introduced into the developer experience, which made it harder to get started with libp2p and begin contributing. Here are some of the major improvements we've made to enable faster and more seamless development:

# Monorepo Setup

The js-libp2p ecosystem once consisted of over 81 repositories! This made it the local development experience difficult when debugging many issues that involved multiple modules, and it also made it very difficult to maintain. Upgrading dependencies was a nightmare and our Github notifications were extremely bloated. We decided to consolidate all of the repositories into a single monorepo, and this has made it much easier to contribute to the project, and it has also made it much easier to maintain. There's a single point of entry for all issues, and there's a single point of entry for all pull requests. Dependencies are automatically upgraded in our release cycle and we've standardized our commits and tooling which makes releases seamless.

# Modernization of Tooling

We've completely re-written our codebase in TypeScript, I won't delve into all the benefits of TypeScript here, but suffice to say we've eliminated whole classes of bugs. We've standardized our modules through ESM when packaging our code. We've also upgraded our logs to be a lot more configurable, adding the ability to append or prefix peerIds, more formatting options and the ability to have a custom logger passed in at the component level. We've introduced dependency checking to ensure there are no unused dependencies, more standardized linting and formatting rules as well as PR templates. We're hoping these tools make it faster for developers to iterate.

# Documentation Improvements

This was perhaps one of the biggest hurdles for potential contributors, and it is still an area we are actively improving on. Over the last year we have introduced tools to automate and validate our docs. We introduced a JS-docs based documentation generator which is intended to keep the API docs up to date. We've also introduced a tool which allows us to statically analyze the TS code snippets in our docs to ensure that the code can be copy-pasted and then run. We've added architectural diagrams (opens new window) which help to explain the various components of js-libp2p, and how they interact. We're hoping these tools make it easier for developers to get started with libp2p, and we're hoping it makes it easier for developers to contribute to libp2p.

# What's next? 🚀

There's still a lot of work left to do, and we're excited to continue working on js-libp2p. Here are some of the things we're working on for the future:

If any of these things excite you, we'd love to have you hear from you! You can find out more about how to contribute here (opens new window) as well as the resources section below. Thank you for reading! 🙏

# Resources and how you can contribute 💪

If you would like to learn more about libp2p, a great place to start is always the docs (opens new window), but we have also included some additional resources below: