[P2P] refactor: message handling

[bryanchriswhite]

Description

Before

The P2P module is responsible for handling incoming messages from the router's host. When it receives a message, it calls into the router to unpack the RainTreeMessage (and facilitate further broadcast propagation). The resulting PocketEnvelope is returned to the P2P module to be emitted over the application event bus.

classDiagram
    class RainTreeMessage {
        <<protobuf>>
        +Level uint32
        +Data []byte
        +Nonce uint64
    }
    
    class PocketEnvelope {
        <<protobuf>>
        +Content *anypb.Any   
    }

    RainTreeMessage --* PocketEnvelope : serialized as `Data`
    
      
    class P2PModule {
        -handleNetworkData([]byte) error
        -handleStream(stream libp2pNetwork.Stream)
        -readStream(stream libp2pNetwork.Stream)
    }

    class RainTreeRouter {
        +HandleNetworkData func([]byte) ([]byte, error)
    }

    RainTreeRouter --> P2PModule
    P2PModule --> RainTreeRouter
    RainTreeRouter --o RainTreeMessage
    P2PModule --o PocketEnvelope

Loading

After

The router encapsulates handling incoming RainTreeMessages, unpacking them and passing them along to the P2P module as serialized PocketEnvelopes. The P2P module then deserializes and emits them over the application event bus.

classDiagram
    class RainTreeMessage {
        <<protobuf>>
        +Level uint32
        +Data []byte
        +Nonce uint64
    }
    
    class PocketEnvelope {
        <<protobuf>>
        +Content *anypb.Any
    }

    RainTreeMessage --* PocketEnvelope : serialized as `Data`
    
    class P2PModule {
        -handlePocketEnvelope([]byte) error
    }

    class RainTreeRouter {
        -handler RouterHandler
        -handleRainTreeMsg([]byte) ([]byte, error)
        -handleStream(stream libp2pNetwork.Stream)
        -readStream(stream libp2pNetwork.Stream)
    }

    RainTreeRouter --> P2PModule : `handler` == `handlePocketEnvelope`
    RainTreeRouter --o RainTreeMessage

    P2PModule --o PocketEnvelope

Loading

Issue

Second deliverable in #762

Type of change

Please mark the relevant option(s):

• New feature, functionality or library
• Bug fix
• Code health or cleanup
• Major breaking change
• Documentation
• Other

List of changes

• Added Handler field to RainTreeConfig & BackgroundConfig
• Refactored rainTreeRouter logging methods
• Renamed rainTreeRouter#HandleNetworkData() to #handleRainTreeMsg()
• Renamed p2pModule#handleNetworkData() to #handleAppData()
• Added -tags=test to all test make targets
• Fixed mockdns usage in TestP2PModule_Insecure_Error test
• Moved message handling from p2p module to router

Testing

• make develop_test; if any code changes were made
• make test_e2e on k8s LocalNet; if any code changes were made
• e2e-devnet-test passes tests on DevNet; if any code was changed
• Docker Compose LocalNet; if any major functionality was changed or introduced
• k8s LocalNet; if any infrastructure or configuration changes were made

Required Checklist

• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have added, or updated, godoc format comments on touched members (see: tip.golang.org/doc/comment)
• I have tested my changes using the available tooling
• I have updated the corresponding CHANGELOG

If Applicable Checklist

• I have updated the corresponding README(s); local and/or global
• I have added tests that prove my fix is effective or that my feature works
• I have added, or updated, mermaid.js diagrams in the corresponding README(s)
• I have added, or updated, documentation and mermaid.js diagrams in shared/docs/* if I updated shared/*README(s)

[Olshansk]

Wdyt about adding the mermaid diagram in the github description to the README for p2p?

[Olshansk]

Should we add a linter for this? Dima recently added a custom one in build/linters/tests.go and it looks like we have access to the File.Name via the dsl matcher, but I don't know if there's any easy way to check for the first line.

Can be done in a separate commit, not a blocker.

type File struct {
	// Name is a file base name.
	Name String

	// PkgPath is a file package path.
	// Examples: "io/ioutil", "strings", "github.com/quasilyte/go-ruleguard/dsl".
	PkgPath String
}

[bryanchriswhite]

It's only necessary to add the test build tag to:

1. non-test files in a non-internal package which exports a private member for testing purposes.
2. test files which depend on something exported by a file which uses the build tag (assumes at least one instance of case 1).

Are you suggesting a linter which enforces the test build tag on all test files or something?

I don't suppose there would really be a prohibitive downside to adding the test tag in places where it's not strictly necessary. I can imagine an argument for either but am leaning towards "ubiquitous build tags" at the moment (unexpectedly):

• ubiquitous build tags
  • pros
    • consistent
    • simpler
    • better justifies updates to the makefile (e.g. -tags test; necessary in either case)
  • cons
    • slightly unconventional
    • requires running tests via make targets OR adding tags manually (e.g. IDE run/test configs)
  • UX
    • go test runs no tests
• only necessary build tags
  • pros
    • minimal impact on the codebase
  • cons
    • more mental overhead
    • less obvious when test(s) didn't run because of a missing tag
  • UX
    • go test runs some but not all tests

[Olshansk]

Why did you go with Debug instead of Error here?

Ditto below

[bryanchriswhite]

(see below)

In this case, my thinking was that this error only happens in the context of this logging helper function which is not a critical function. I didn't imagine that this logging helper producing an error would be useful to the end user (i.e. not actionable nor a useful signal). Perhaps I assume too much.

[Olshansk]

I like this design 💯

[Olshansk]

Ditto. Noticed you're using Debug instead of Err for a lot of these. Why is that?

[bryanchriswhite]

Thank you for scrutinizing this, please continue to do so! 🙌

I was trying to be conservative with what we log to non-debug loggers. In this particular case, my thinking was that being unable to set the read deadline isn't actionable and isn't really useful as a signal for end-users.

[bryanchriswhite]

Wdyt about adding the mermaid diagram in the github description to the README for p2p?

@Olshansk I was planning on incorporating the diagrams from #505, which represents the composition of this and #769, into #732. My thinking was that the diagrams in this and #769 are trimmed down versions which only show what's relevant for understanding the change, whereas what I expect we want to keep in the repo is the representation of the desired state with sufficient context.

Do you imagine a step in between?

[github-actions]

Here's a brief review of the code patch:

1. A new version (0.0.0.52) is being added with today's date.
2. The RainTreeConfig and BackgroundConfig structs are updated to include a new field, Handler.
3. Logging methods in the rainTreeRouter have been refactored.
4. Methods rainTreeRouter#HandleNetworkData() and p2pModule#handleNetworkData() are renamed to #handleRainTreeMsg() and #handlePocketEnvelope(), respectively, for better consistency.
5. The build flags now include -tags=test to all test make targets.
6. An issue with mockdnswas fixed in the TestP2PModule_Insecure_Error test.
7. Message handling is moved from the p2p module to the router.

Overall, the changes seem reasonably-safe and no significant risks or bugs stand out. One possible improvement suggestion is to ensure that unit tests are updated accordingly to reflect method name changes such as #handleRainTreeMsg() and #handlePocketEnvelope(). Additionally, it may be helpful to review the impact of refactored logging methods on overall system performance or log readability.

[codecov]

Codecov Report

Patch and project coverage have no change.

Comparison is base (95cf73d) 0.00% compared to head (65ca42d) 0.00%.

❗ Current head 65ca42d differs from pull request most recent head 33b5b58. Consider uploading reports for the commit 33b5b58 to get more accurate results

Additional details and impacted files

@@     Coverage Diff     @@
##   main   #763   +/-   ##
===========================
===========================

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

[github-actions]

The code patch provided seems reasonable, but I have a few minor suggestions for improvement. Please consider the following:

1. In line 19, add a comment to explain the purpose of moving the import statement and why it has been separated from the other import statements.

2. Although not incorrect, you might want to change lines 272-276 to avoid using the pointer to 'mod'. Instead, directly use 'p2pMod'. This increases readability and reduces complexity:

   mod := p2pMod
   ...
   mod.router.(*raintree.RainTreeRouter).HandleStream(stream)

3. Make sure that there are unit tests (or create new unit tests) that cover the changes introduced in this patch. The current tests should pass, and any new edge cases should also be considered when testing.

4. Ensure that you follow the rest of the project's style and coding conventions when contributing your patch.

Overall, the code patch is generally fine, but addressing these points would lead to better quality code review.