Replace `Multiaddr` & related types with substrate-specific types by dmitry-markin · Pull Request #4198 · paritytech/polkadot-sdk

dmitry-markin · 2024-04-18T14:22:59Z

This PR introduces custom types / substrate wrappers for Multiaddr, multiaddr::Protocol, Multihash, ed25519::* and supplementary types like errors and iterators.

This is needed to unblock libp2p upgrade PR #1631 after #2944 was merged. libp2p and litep2p currently depend on different versions of multiaddr crate, and introduction of this "common ground" types is needed to support independent version upgrades of multiaddr and dependent crates in libp2p & litep2p.

While being just convenient to not tie versions of libp2p & litep2p dependencies together, it's currently not even possible to keep libp2p & litep2p dependencies updated to the same versions as multiaddr in libp2p depends on libp2p-identity that we can't include as a dependency of litep2p, which has it's own PeerId type. In the future, to keep things updated on litep2p side, we will likely need to fork multiaddr and make it use litep2p PeerId as a payload of /p2p/... protocol.

With these changes, common code in substrate uses these custom types, and litep2p & libp2p backends use corresponding libraries types.

This is a companion PR to paritytech/polkadot-sdk#4198.

…ltiaddr

paritytech-cicd-pr · 2024-05-17T13:42:15Z

The CI pipeline was cancelled due to failure one of the required jobs.
Job name: cargo-clippy
Logs: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6238603

bkchr

Didn't look that closely, but these are mainly wrappers any way?

substrate/client/network/types/src/multiaddr.rs

bkchr · 2024-05-18T16:04:14Z

substrate/client/network/types/src/ed25519.rs

+
+	/// Convert public key to `PeerId`.
+	pub fn to_peer_id(&self) -> PeerId {
+		litep2p::PeerId::from(litep2p::crypto::PublicKey::Ed25519(self.clone().into())).into()


Can we not directly construct the PeerId?

This would require hashing PublicKey type serialized as protobuf, and I tried to avoid including the protobuf schema into the already bloated wrapper sources.

lexnv · 2024-05-20T13:31:43Z

substrate/client/network/types/src/multihash.rs

+	}
+}
+
+// TODO: uncomment this after upgrading `multihash` crate to v0.19.1.


Is there anything blocking us from updating to multihash 0.19.1? Maybe we could do that in a followup and raise an issue to not forget about it?

Yes, it should be done at the same time as upgrading libp2p. I will mention it in the libp2p PR.

lexnv

LGTM! Nice job here!

There are a few CI steps that are failing, but other than that we are good to go 👍

…ltiaddr

…ritytech#4198) This PR introduces custom types / substrate wrappers for `Multiaddr`, `multiaddr::Protocol`, `Multihash`, `ed25519::*` and supplementary types like errors and iterators. This is needed to unblock `libp2p` upgrade PR paritytech#1631 after paritytech#2944 was merged. `libp2p` and `litep2p` currently depend on different versions of `multiaddr` crate, and introduction of this "common ground" types is needed to support independent version upgrades of `multiaddr` and dependent crates in `libp2p` & `litep2p`. While being just convenient to not tie versions of `libp2p` & `litep2p` dependencies together, it's currently not even possible to keep `libp2p` & `litep2p` dependencies updated to the same versions as `multiaddr` in `libp2p` depends on `libp2p-identity` that we can't include as a dependency of `litep2p`, which has it's own `PeerId` type. In the future, to keep things updated on `litep2p` side, we will likely need to fork `multiaddr` and make it use `litep2p` `PeerId` as a payload of `/p2p/...` protocol. With these changes, common code in substrate uses these custom types, and `litep2p` & `libp2p` backends use corresponding libraries types.

nazar-pc · 2024-06-21T09:22:19Z

substrate/client/network/types/src/multiaddr.rs

+/// or litep2p's `Multiaddr` when passed to the corresponding network backend.
+
+#[derive(PartialEq, Eq, PartialOrd, Ord, Clone, Hash)]
+pub struct Multiaddr {


This version is missing impl From<IpAddr> for Multiaddr from libp2p's implementation, can you add it?

Added in #4855.

@nazar-pc

Add `From` implementation used by downstream project. Ref. #4198 (comment) CC @nazar-pc

…ritytech#4198) This PR introduces custom types / substrate wrappers for `Multiaddr`, `multiaddr::Protocol`, `Multihash`, `ed25519::*` and supplementary types like errors and iterators. This is needed to unblock `libp2p` upgrade PR paritytech#1631 after paritytech#2944 was merged. `libp2p` and `litep2p` currently depend on different versions of `multiaddr` crate, and introduction of this "common ground" types is needed to support independent version upgrades of `multiaddr` and dependent crates in `libp2p` & `litep2p`. While being just convenient to not tie versions of `libp2p` & `litep2p` dependencies together, it's currently not even possible to keep `libp2p` & `litep2p` dependencies updated to the same versions as `multiaddr` in `libp2p` depends on `libp2p-identity` that we can't include as a dependency of `litep2p`, which has it's own `PeerId` type. In the future, to keep things updated on `litep2p` side, we will likely need to fork `multiaddr` and make it use `litep2p` `PeerId` as a payload of `/p2p/...` protocol. With these changes, common code in substrate uses these custom types, and `litep2p` & `libp2p` backends use corresponding libraries types.

@nazar-pc

…4855) Add `From` implementation used by downstream project. Ref. paritytech#4198 (comment) CC @nazar-pc

@nazar-pc

…4855) Add `From` implementation used by downstream project. Ref. paritytech#4198 (comment) CC @nazar-pc

This is a companion PR to paritytech/polkadot-sdk#4198.

Introduce network backend agnostic Multiaddr & Multihash types

87be8e5

dmitry-markin added the T0-node This PR/Issue is related to the topic “node”. label Apr 18, 2024

dmitry-markin requested a review from lexnv April 18, 2024 14:22

dmitry-markin self-assigned this Apr 18, 2024

dmitry-markin mentioned this pull request Apr 18, 2024

Re-export multihash & multiaddr types paritytech/litep2p#79

Merged

dmitry-markin added 4 commits April 19, 2024 17:16

Extend substrate's Multihash, make substrate's PeerId use it

584038e

Allow constructing Multiaddr from iterator of protocols

8100f7f

Convert network config to use substrate's Multiaddr

ea85860

Add ed25519 types to sc-network-types

94af781

dmitry-markin changed the title ~~Introduce network backend agnostic Multiaddr & Multihash types~~ Replace libp2p-identity types with substrate-specific types Apr 23, 2024

dmitry-markin added a commit to paritytech/litep2p that referenced this pull request Apr 23, 2024

Re-export multihash & multiaddr types (#79)

08112ca

This is a companion PR to paritytech/polkadot-sdk#4198.

dmitry-markin added 14 commits April 23, 2024 19:30

cargo update litep2p

aa4d28d

Make Check licenses check happy

da59b29

Make network config use substrate-specific types

f614106

Convert substrate-specific types for libp2p backend

ce321a9

Convert substrate-specific types for litep2p backend

a0373ae

Cleaner separation between backend-specific types

2bb5448

Make authority-discovery use substrate-specific types

bd7df2e

Make telemetry use libp2p-only Multiaddr

7c29f74

Make cli use sc_network::config::ed25519

2162dac

Make authority-discovery tests use proper types

bb6f58f

Fix tests

9fc3f80

Merge remote-tracking branch 'origin/master' into dm-network-types-mu…

04a4096

…ltiaddr

Make clippy happy

036aac7

Add tests for ed25519 types conversion between libs

685de87

Make clippy happy again

dabbd96

dmitry-markin changed the title ~~Replace libp2p-identity types with substrate-specific types~~ Replace libp2p-identity related types with substrate-specific types May 17, 2024

dmitry-markin changed the title ~~Replace libp2p-identity related types with substrate-specific types~~ Replace Multiaddr & related types with substrate-specific types May 17, 2024

Add PRDoc

a66016c

dmitry-markin requested a review from bkchr May 17, 2024 14:13

dmitry-markin marked this pull request as ready for review May 17, 2024 14:13

Fix doctest

41ffe22

bkchr approved these changes May 18, 2024

View reviewed changes

dmitry-markin added 2 commits May 20, 2024 16:03

Apply review suggestions

086f389

Update semver in PRDoc

7547c49

lexnv reviewed May 20, 2024

View reviewed changes

dmitry-markin added 2 commits May 20, 2024 17:26

Fix CI

b1bea17

Merge remote-tracking branch 'origin/master' into dm-network-types-mu…

a87a92d

…ltiaddr

dmitry-markin requested a review from lexnv May 20, 2024 16:02

lexnv approved these changes May 21, 2024

View reviewed changes

dmitry-markin added this pull request to the merge queue May 21, 2024

Merged via the queue into master with commit d05786f May 21, 2024

dmitry-markin deleted the dm-network-types-multiaddr branch May 21, 2024 16:45

nazar-pc reviewed Jun 21, 2024

View reviewed changes

dmitry-markin mentioned this pull request Jun 21, 2024

Possible use of libp2p_identity::PeerId as PeerId type paritytech/litep2p#73

Closed

nazar-pc mentioned this pull request Jun 21, 2024

Update Substrate to 1.13.0 autonomys/subspace#2868

Merged

1 task

dmitry-markin mentioned this pull request Jun 21, 2024

sc-network-types: implement From<IpAddr> for Multiaddr #4855

Merged

github-merge-queue bot pushed a commit that referenced this pull request Jun 21, 2024

sc-network-types: implement From<IpAddr> for Multiaddr (#4855)

3b3a1d2

Add `From` implementation used by downstream project. Ref. #4198 (comment) CC @nazar-pc

TarekkMA pushed a commit to moonbeam-foundation/polkadot-sdk that referenced this pull request Aug 2, 2024

sc-network-types: implement From<IpAddr> for Multiaddr (paritytech#…

8071184

…4855) Add `From` implementation used by downstream project. Ref. paritytech#4198 (comment) CC @nazar-pc

This was referenced Aug 21, 2024

Update polkadot-sdk from v1.11.0 to stable2407 moondance-labs/tanssi#659

Closed

Update polkadot-sdk from v1.11.0 to stable2407 moonbeam-foundation/moonbeam#2912

Closed

This was referenced Oct 4, 2024

feat: ⏫ upgrade to Polkadot SDK v1.13.0 Moonsong-Labs/storage-hub#214

Closed

feat: ⏫ upgrade to Polkadot SDK v1.13.0 Moonsong-Labs/storage-hub#216

Merged

sfffaaa pushed a commit to peaqnetwork/polkadot-sdk that referenced this pull request Dec 27, 2024

sc-network-types: implement From<IpAddr> for Multiaddr (paritytech#…

d27196f

…4855) Add `From` implementation used by downstream project. Ref. paritytech#4198 (comment) CC @nazar-pc

Intel-driver added a commit to Intel-driver/litep2p that referenced this pull request Dec 24, 2025

Re-export multihash & multiaddr types (#79)

ba61886

This is a companion PR to paritytech/polkadot-sdk#4198.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Replace `Multiaddr` & related types with substrate-specific types#4198

Replace `Multiaddr` & related types with substrate-specific types#4198
dmitry-markin merged 26 commits intomasterfrom
dm-network-types-multiaddr

dmitry-markin commented Apr 18, 2024 •

edited

Loading

Uh oh!

paritytech-cicd-pr commented May 17, 2024

Uh oh!

bkchr left a comment

Uh oh!

Uh oh!

bkchr May 18, 2024

Uh oh!

dmitry-markin May 20, 2024

Uh oh!

lexnv May 20, 2024

Uh oh!

dmitry-markin May 20, 2024

Uh oh!

lexnv left a comment

Uh oh!

nazar-pc Jun 21, 2024

Uh oh!

dmitry-markin Jun 21, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

dmitry-markin commented Apr 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

paritytech-cicd-pr commented May 17, 2024

Uh oh!

bkchr left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bkchr May 18, 2024

Choose a reason for hiding this comment

Uh oh!

dmitry-markin May 20, 2024

Choose a reason for hiding this comment

Uh oh!

lexnv May 20, 2024

Choose a reason for hiding this comment

Uh oh!

dmitry-markin May 20, 2024

Choose a reason for hiding this comment

Uh oh!

lexnv left a comment

Choose a reason for hiding this comment

Uh oh!

nazar-pc Jun 21, 2024

Choose a reason for hiding this comment

Uh oh!

dmitry-markin Jun 21, 2024

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

dmitry-markin commented Apr 18, 2024 •

edited

Loading