[P2P] Integrate background router

[bryanchriswhite]

@Reviewer

This PR may be more digestible / reviewable on a commit-by-commit basis. Commits are organized logically and any given line is only modified in a single commit, with few exceptions*.

*(In the interest of preserving the git-time-continuum 👮🚨, this applies in batches of commits between comments or reviews by humans, only once "in review")

---

Description

Before

While backgroundRouter exists and implements the Router interface, it had yet to be connected to anything. Additionally, it was not able to handle incoming unicast messages.

classDiagram
    class p2pModule {
        -router Router
        -handlePocketEnvelope([]byte) error
    }

    class P2PModule {
        <<interface>>
        GetAddress() (Address, error)
        HandleEvent(*anypb.Any) error
        Send([]byte, Address) error
        Broadcast([]byte) error
    }
    p2pModule --|> P2PModule

    class RainTreeRouter {
        UnicastRouter
        -handler MessageHandler
        +Broadcast([]byte) error
        -handleRainTreeMsg([]byte) error
    }

    class BackgroundRouter {
        UnicastRouter
        -handler MessageHandler
        +Broadcast([]byte) error
        -handleBackgroundMsg([]byte) error
        -readSubscription(subscription *pubsub.Subscription)
    }

    class UnicastRouter {
        -messageHandler MessageHandler
        -peerHandler PeerHandler
        +Send([]byte, Address) error
        -handleStream(libp2pNetwork.Stream)
        -readStream(libp2pNetwork.Stream)
    }
    RainTreeRouter --* UnicastRouter : (embedded)

    p2pModule --o "1" Router
    p2pModule ..* RainTreeRouter : (`router`)
    
    class Router {
        <<interface>>
        +Send([]byte, Address) error
        +Broadcast([]byte) error
    }
    BackgroundRouter --|> Router
    RainTreeRouter --|> Router

Loading

After

backgroundRouter embeds UnicastRouter and is integrated into p2pModule such that it is considered when calling P2PModule#Send() and P2PModule#Broadcast().

classDiagram
    class p2pModule {
        -stakedActorRouter Router
        -unstakedActorRouter Router
        -handlePocketEnvelope([]byte) error
    }

    class P2PModule {
        <<interface>>
        GetAddress() (Address, error)
        HandleEvent(*anypb.Any) error
        Send([]byte, Address) error
        Broadcast([]byte) error
    }
    p2pModule --|> P2PModule

    class RainTreeRouter {
        UnicastRouter
        -handler MessageHandler
        +Broadcast([]byte) error
        -handleRainTreeMsg([]byte) error
    }

    class BackgroundRouter {
        UnicastRouter
        -handler MessageHandler
        +Broadcast([]byte) error
        -handleBackgroundMsg([]byte) error
        -readSubscription(subscription *pubsub.Subscription)
    }

    class UnicastRouter {
        -messageHandler MessageHandler
        -peerHandler PeerHandler
        +Send([]byte, Address) error
        -handleStream(libp2pNetwork.Stream)
        -readStream(libp2pNetwork.Stream)
    }
    RainTreeRouter --* UnicastRouter : (embedded)
    BackgroundRouter --* UnicastRouter : (embedded)

    p2pModule --o "2" Router
    p2pModule ..* RainTreeRouter : (`stakedActorRouter`)
    p2pModule ..* BackgroundRouter : (`unstakedActorRouter`)
    
    class Router {
        <<interface>>
        +Send([]byte, Address) error
        +Broadcast([]byte) error
    }
    BackgroundRouter --|> Router
    RainTreeRouter --|> Router

Loading

Issue

• [P2P] Background gossip using libp2p-pubsub #505

Dependencies

• [P2P] refactor: unicast router #844

Type of change

Please mark the relevant option(s):

• New feature, functionality or library
• Bug fix
• Code health or cleanup
• Major breaking change
• Documentation
• Other

List of changes

• added background message protobuf type
• added Router#Close() interface method
• separated raintree & bg protocol IDs
• move PocketEnvelope nonce generation to PackMessage()
• added Handler to router configs
• refactored raintree router to support P2P module & Router interface changes
• refactored raintree router bootstrapping
• refactored background router to support P2P module integration & Router interface changes
• integrated background router into P2P module
• refactored staked actor router (raintree) peer discovery
• updated tests, post-refactoring

Testing

• make develop_test; if any code changes were made
• Docker Compose LocalNet; if any major functionality was changed or introduced
• k8s LocalNet; if any infrastructure or configuration changes were made

Required Checklist

• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have added, or updated, godoc format comments on touched members (see: tip.golang.org/doc/comment)
• I have tested my changes using the available tooling
• I have updated the corresponding CHANGELOG

If Applicable Checklist

• I have updated the corresponding README(s); local and/or global
• I have added tests that prove my fix is effective or that my feature works
• I have added, or updated, mermaid.js diagrams in the corresponding README(s)
• I have added, or updated, documentation and mermaid.js diagrams in shared/docs/* if I updated shared/*README(s)

[codecov]

Codecov Report

Patch coverage: 66.66% and project coverage change: +0.45 🎉

Comparison is base (fe24824) 31.78% compared to head (d564ade) 32.23%.

Additional details and impacted files

@@                     Coverage Diff                     @@
##           refactor/unicast-router     #732      +/-   ##
===========================================================
+ Coverage                    31.78%   32.23%   +0.45%     
===========================================================
  Files                          106       87      -19     
  Lines                         8996     6099    -2897     
===========================================================
- Hits                          2859     1966     -893     
+ Misses                        5795     3861    -1934     
+ Partials                       342      272      -70     

Impacted Files	Coverage Δ
p2p/types/errors.go	0.00% <ø> (ø)
p2p/utils/host.go	0.00% <0.00%> (ø)
shared/messaging/envelope.go	62.50% <100.00%> (+8.65%)	⬆️

... and 19 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

[Olshansk]

As noted in the P2P knowledge transfer (#438) ticket, for hotstuff consensus, we don't use RainTree for messages propagation between staked actors (e.g. state sync) but for consensus, it's a star pattern (i.e. direct)

[Olshansk]

How about this:

 Sender         | Receiver       | Router          | Example Usage                                        |
|----------------|----------------|-----------------|------------------------------------------------------|
| Staked Actor   | Staked Actor   | No router  | Consensus (hotstuff) messages (to validators only) |
| Staked Actor   | Staked Actor   | Raintree only   | Consensus (state sync) messages (to validators only) |
| Unstaked Actor | Staked Actor   | Background only | Consensus (state sync) messages (to validators only) |
| Unstaked Actor | Unstaked Actor | Background only | Consensus (state sync) & Debug (CLI) messages        |

[Olshansk]

Let's 🚢 this!!

[Olshansk]

Appreciate the comments 🙌

[Olshansk]

Nice change