ADR 001: Trustless Fisherman system

[DragonDmoney]

ADR 001: Trustless Fisherman system

Context

Currently, the planned fisherman V1 model only supports fishermen selected by the DAO, making them trusted actors. This is not desirable for a number of reasons, centralization, few points of failure, and doxing the fisherman becomes easier. The most desirable, decentralized, scalable, and secure system would be a system where anybody can be a fisherman.

Proposed Solution

This outlines a new system for fisherman session selection and a new incentive model that will economically force fishermen to be as accurate as possible.

Session Selection

There are two systems possible for Fisherman session selection:

1. Continuous random sessions initiated by fishermen
2. A fisherman session tied to a service node session initiated by the protocol

Continuous random sessions initiated by fishermen

After a new block has been created, and after FishermanSessionDelay has elapsed, a Fisherman will generate a random list of other node addresses (including itself) with a VRF.

type FishermanSession struct {
    Fishermen []Fisherman
    Initiatior Fisherman
    Seed int
}

The fisherman will propagate this information to the other fisherman, and this marks the beginning of the TestingPeriod. The TestingPeriod is a measure of time defined by TestingPeriodLength. Samples outside the timeframe of the TestingPeriod will not be accepted, and will not impact the fisherman's reputation.

The fisherman will begin sampling. The fisherman has no requirements for sampling other than sampling during the TestingPeriod, and sending more samples than MinimumTestingSamples.

Fisherman can stake their own applications or can receive ephemeral keys from applications. Applications can find the fisherman in their session by querying a fisherman RPC.

Pros:

• There are lots of moving parts, which makes identifying fishermen and blocking them very difficult (from the ServiceNode perspective)
• Allows for a larger group of fishermen to monitor ServiceNodes increasing accuracy.

Cons:

• There will be more chain bloat, as the more fisherman sessions the more TestScores fisherman need to submit
• It is harder for the applications to identify which Fisherman are monitoring their sessions in a trustless manner.

Note: After further review, this system does not fully abolish vulnerabilities in the Sampling Protocol, and the algorithm listed below is the proposed method for this ADR

Protocol Fisherman Sessions

This system works very similarly to the current session generation session, however, another fisherman session will be generated in parallel to a ServiceNode session. This means for the entirety of the session, the same fisherman will be monitoring the ServiceNodes.

Pros:

• It is easy for applications to identify fishermen monitoring their session
• The session generation is on the protocol level

Cons:

• It is easier to identify and dox fisherman

TestScore Reports

After a fisherman session has concluded, all the fisherman will submit their Testscores to the network. These Testscores are the same as defined in the previous spec, only with the addition of their SessionID, which is a hash that corresponds to their Fisherman session.

As brought up by Luis, some fishermen may wait to see other fisherman's test scores in the mempool, and then submit a test score as close to the other fishermen to maximize their rewards. To prevent this, there must be a "encode and reveal scheme" for all TestScore transactions. In the first transaction, the fisherman will submit a hash of all the data on the transaction, and the fisherman will reveal the data in the next transaction.

Grading Period

After the Testscores have been submitted to the network, the grading process begins. All fishermen begin with a reputation score of 0.00, this reputation score corresponds to how many rewards they get (as defined in the Rewarding Period). Fisherman's reputation is affected by how similar their results are to the rest of the fishermen. The protocol will aggregate all the reports from the same Fisherman session. For every ServiceNode in that session, it will aggregate all the fisherman's results for that ServiceNode and run them through an algorithm to determine the consensus average (run all the test-scores through an algorithm to find the most accurate score for ServiceNode in a particular session). This algorithm has not yet been defined, but as a placeholder, we can use median. The median of the fisherman's results will go on the ServiceNode's report card.

Fisherman Grading

The accuracy of a fisherman on a particular ServiceNode is defined as 1 - abs(median / ServiceNodeResults). The average of all of these individual accuracies is the fisherman's SessionAccuracy. The fisherman's reputation is the average of a trailing list of Session Accuracies of length NumRollingAverageScoringSessions (similar to a ServiceNode's report card).

PauseTxs

If a ServiceNode does not respond, a Fisherman can submit a PauseTx to pause the ServiceNodes from participating in sessions. A PauseTx is successful of the majority of fishermen submit a PauseTx as well. If a fisherman is incorrectly submits/doesn't submit a PauseTx they are punished as well. Their SessionAccuracy is reduced by NumOfInaccuratePauseTxs * InaccuratePauseTxReduction.

Fisherman Rewarding

Fishermen are rewarded in a similar manner to ServiceNodes, there is a TotalVolumeUsage that is evenly divided into buckets per Fisherman that are above ReputationCliff. This value is the MaxFishermanSalary. The fisherman's reputation is put into a network variable function called ReputationToSalary. This function would look something like this:

The salary of the Fisherman is the ReputationToSalary multiplied by MaxFishermanSalary (note: if the value is negative, the Fisherman is burned).

Status

In-progress

Author

Pierre

[Olshansk]

Thanks for the first proposal @DragonDmoney!

Overall, I like and support the direction of this proposal. I think there are a handful of technical intricacies we'd still need to discuss and iterate, but don't think it needs to be done all at once.

A. High Level Comments

1. For those who attended the V1 contributor hour (available here), it might be unnecessary, but I believe this lacks an introductory summary of what the overall goal of Fisherman sessions and fisherman scores is.

2. I think it'd be helpful to add the diagrams used in the presentation as well.

3. NIT: To make it easier for other readers, can you use backticks around parameters, variable names, keywords, etc...

B. Questions / Comments

Samples outside the timeframe of the TestingPeriod will not be accepted, and will impact the fisherman's reputation

1. Is this mean to be "will not impact the fisherman's reputation?"

Applications can find the fisherman in their session by querying a fisherman RPC.

2. Did you have in mind how the application would determine the fisherman's RPC?

It is easier to identify and dox fisherman

3. Could you explain why this is an issue for the second option but not the first?

Fisherman's reputation is affected by how similar their results are to the rest of the fishermen

4. Might be worth mentioning that we are following a "Wisdom of the Crowds" type of approach here which is pretty common in Oracle networks

through an algorithm to determine the consensus average. This algorithm has not yet been defined, but as a placeholder, we can use median

5. What do you mean by the consensus average here? Kind of like a quorum?

Their SessionAccuracy is reduced by NumOfInaccuratePauseTxs * InaccuratePauseTxReduction

6. If I'm understanding correctly, this mechanism is meant to be a secondary form of insurance in addition to the test scores. Personally, I think this really increases and complicates the scope, and a good scoring solution may be sufficient, so I would vote to remove it from this specific ADR.

C. Suggestions / Improvements

1. Algorithm (mean/median/etc)

In the context of a single session, rather than the mean or median, I recommend using the Z-score.

I believe it is a fair assumption that the fisherman reports will be normally distributed, so using the mean and variance follows naturally. The Z-score is a common analytical tool that can be used to determine a data point's divergence from the distribution's mean taking variance into account.

2. ReputationToSalary

In the context of the network's ongoing scores, I believe having a rolling average governed by some sort of parameter (e.g. NumRollingAverageScoringSessions) would be useful here.

This way, really old scores are not representative of the actor's rewards today, and newer scores will not be impacted by single session outliers.

[Olshansk]

Per a discussion held with the team, I wanted to capture some ideas for future iterations of the utility module summarized by @DragonDmoney:

However, I personally believe that there is an opportunity to improve the existing specification to go from permissioned Fisherman to permissionless, but potentially doxxable, fisherman and we can further investigate the ideas above in the future.

@luyzdeleon asked:

I have an incentive design question: If fishermen are graded on how similar they respond to other fishermen, wouldn't it create an incentive to instead of telling the truth, they will want their test scores to be as similar as possible to other fishermen? Meaning if a fishermen submits their score first, other fishermen are incentivized to submit a test score as similar as possible to that first score sent

If we go with the option where the Fisherman sessions are tied to the ServiceNode sessions, all the scores are submitted at the same time (i.e. the end of the session) and we'd be able to use the scoring algorithm above to capture the difference from the mean (using whatever method we choose).

I think this approach might be worth creating a sequence diagram for to clarify the spec and discuss next steps.

[DragonDmoney]

I believe I have addressed some of the issues discussed in the last community call, and Daniel's comments and updated the ADR correspondingly. Although I am still working on sequence diagrams

[Olshansk]

@DragonDmoney Could you specify which section you updated so we can just re-review that part?