diff --git a/generalized-gossipsub/README.md b/generalized-gossipsub/README.md new file mode 100644 index 000000000..b45905c0d --- /dev/null +++ b/generalized-gossipsub/README.md @@ -0,0 +1,344 @@ +# Generalized Gossipsub + +This is a generalization of the original [gossipsub protocol](../gossipsub/README.md) +that makes the protocol extensible through additional modules called strategies. +These strategies can be employed per-topic and allow the protocol to be +fine-tuned under a range of scenarios and circumstances and applied either +dynamically or statically on a per-application (topic) basis. + +NOTE: Any current gossipsub implementation can be fully compatible with this +protocol by simply implementing the CHOKE/UNCHOKE control messages. This protocol is +fully backwards compatible with original gossipsub implementations. + +## Table of Contents + +- [Generalized Gossipsub](#generalized-gossipsub) + - [Motivation](#motivation) + - [Overview](#overview) + - [Strategies](#strategies) + - [The RPC](#the-rpc) + - [Deviations from Gossipsub](#deviations-from-gossipsub) + - [Protocol State](#protocol-state) + - [Topic Membership](#topic-membership) + - [Control Messages](#control-messages) + - [Message Propagation](#message-propagation) + +## Motivation + +[Gossipsub](../pubsub/gossipsub/README.md) is designed to be a configurable protocol that allows users to adjust +configuration parameters to make trade-offs between resiliency, latency and +bandwidth. Over the years, these parameters have been found too constraining +resulting in a number of proposals to modify the specification to achieve better +results under specific circumstances. + +There are three major areas that are constrained by the gossipsub specification, that are a +prime targets to free from the specification and allow users to implement freely, these +are: + +- **Scoring** - A system used to score peers for security and performance. +- **Mesh construction/peer sampling** (and therefore network topologies). +- **Message dissemination** - The choice of how to send messages either direct or + via gossip and the timing. + +A further constraint is that gossipsub requires that any given choice of these +areas is enforced across all topics, which constrains the solution space for +users to one design choice for potentially different applications. + +There are mesh construction strategies that can select mesh's in such a way that +can form efficient broadcast trees. We could think of a topology that forms a +single tree spanning all nodes on the network. This would be very efficient, but +not very resilient. Another example could be small mesh sizes, which sacrifices +resiliency for bandwidth, which may be applicable to applications implementing +schemes like erasure coding which have redundancy built in. +There are different broadcast strategies that could favour gossip over direct +sending. This could be a trade-off between latency and bandwidth. In some +cases these strategies would be better suited for nodes under bandwidth +constraints. + +The generalization proposed here aims to specify these strategies and allow the core protocol +to select each strategy per topic. This allows for scenarios where an +application may have a topic where resiliency is not very important, so a +low-bandwidth strategy could be chosen, (i.e low-mesh, sparse topology) and at +the same time have a topic where resiliency is important so chooses a +high-bandwidth strategy (i.e high-mesh, dense topology). + +A node that is heavily resource constrained, might also wish to switch to a +combination of strategies that is known to perform better under those +conditions. + +The goal of this protocol is into increase the degrees of freedom in fine-tuning +p2p message dissemination in a general way that doesn't require specification +changes to apply. It also aims to minimize engineering overhead for +implementations that already have gossipsub. + +## Overview + +The protocol defined here is an extension/modification of the [original +gossipsub](../pubsub/gossipsub/README.md) protocol. It is aimed to increase the +configurability of the protocol to allow for a wider set of applications and +performance tunings. + +As this is based on the original gossipsub specification, this document does not compile all past versions of the specification here, +rather lists the main departures from the original specification and consolidates all the protobuf and control messages. + +## Strategies + +There are three strategies that can be selected at any given time on a per-topic +bases: + +- **Scoring** - How to score peers based on their behaviour/performance. This can + also be global (not necessarily per-topic) +- **Mesh** - Construction/Maintenance - This strategy, if combined across all nodes, + fundamentally controls the topology of the network. +- **Broadcast** - This strategy can dictate tradeoffs between resilience, bandwidth + and latency by deciding how aggressively to directly send messages vs + gossiping them. + +Each of these are detailed in the [Strategies](./strategies.md) section. + +Examples strategies are also provided: + +- [Original Gossipsub Broadcast](./strategies/broadcast/original-gossipsub.md) +- [Original Gossipsub Mesh](./strategies/mesh/original-gossipsub.md) +- [Random Choke Mesh](./strategies/mesh/random-choke.md) + +## The RPC + +All communication between peers happens in the form of exchanging protobuf RPC +messages between participating peers. + +The `RPC` protobuf is as follows: + +```protobuf +syntax = "proto2"; +message RPC { + repeated SubOpts subscriptions = 1; + repeated Message publish = 2; + + message SubOpts { + optional bool subscribe = 1; // subscribe or unsubscribe + optional string topic_id = 2; + } + + optional ControlMessage control = 3; +} + +message Message { + optional bytes from = 1; + optional bytes data = 2; + optional bytes seqno = 3; + required string topic = 4; + optional bytes signature = 5; + optional bytes key = 6; +} + +message ControlMessage { + repeated ControlIHave ihave = 1; + repeated ControlIWant iwant = 2; + repeated ControlGraft graft = 3; + repeated ControlPrune prune = 4; + repeated ControlChoke choke = 5; + repeated ControlUnChoke unchoke = 6; +} + +message ControlIHave { + optional string topic_id = 1; + repeated bytes message_ids = 2; +} + +message ControlIWant { + repeated bytes message_ids= 1; +} + +message ControlGraft { + optional string topic_id = 1; +} + +message ControlPrune { + optional string topic_id = 1; + repeated PeerInfo peers = 2; // gossipsub v1.1 PX + optional uint64 backoff = 3; // gossipsub v1.1 backoff time (in seconds) +} + +message ControlChoke { + required string topicID = 1; +} +message ControlUnChoke { + required string topicID = 1; +} + +message PeerInfo { + optional bytes peer_id = 1; + optional bytes signed_peer_record = 2; +} +``` + +## Deviations from Gossipsub Logic + +Logic has been split from what we call "core" protocol into strategies. Here the +distinction between the split is made. + +### Parameters + +The parameters that remain in the protocol are: + +| Parameter | Purpose | Reasonable Default | +| -------------------- | ----------------------------------------------------- | ------------------ | +| `heartbeat_interval` | Time between [heartbeats](#heartbeat) | 1 second | +| `fanout_ttl` | Time-to-live for each topic's fanout state | 60 seconds | +| `mcache_len` | Number of history windows in message cache | 5 | +| `mcache_gossip` | Number of history windows to use when emitting gossip | 3 | +| `seen_ttl` | Expiry time for cache of seen message ids | 2 minutes | + +### Protocol State + +The core protocol still handles the control messages such as +SUBSCRIBE/UNSUBSCRIBE/GRAFT/PRUNE/CHOKE/UNCHOKE and therefore must still +maintain a state of peers that form a mesh (how this is formed is up to any +given strategy), and keep track of which peers are subscribed to which topic. + +### Topic Membership + +The [original gossipsub](../pubsub/gossipsub/README.md) specification describes +how a router behaves when peers join or leave topics. The terminology used is +`JOIN(topic)` and `LEAVE(topic)`. The new logic of this behaves is described +here. + +When the application invokes `JOIN(topic)`, the router will form a topic mesh by +asking the selected [mesh strategy][./mesh-strategy.md] for the given topic. + +For all new peers that have formed the mesh, the router will inform them that they have been added to the +mesh by sending them a `GRAFT` control message. + +The application can invoke `LEAVE(topic)` to unsubscribe from a topic. The +router will inform the peers in `mesh[topic]` by sending them a `PRUNE` control +message, so that they can remove the link from their own topic mesh. + +After sending `PRUNE` messages, the router will forget `mesh[topic]` and delete +it from its local state. + +### Control Messages + +Control messages are exchanged to maintain topic meshes and emit gossip. This +section lists the control messages in the core gossipsub protocol + +#### SUBSCRIBE + +When a subscribe message is received, the router records that the new peer is +now subscribed to the topic and informs the mesh strategy. The mesh strategy may +choose to GRAFT this peer on this topic. + +#### GRAFT + +The `GRAFT` message grafts a new link in a topic mesh. The `GRAFT` informs a peer +that it has been added to the local router's mesh view for the included topic id. + +#### PRUNE + +The `PRUNE` message prunes a mesh link from a topic mesh. `PRUNE` notifies a +peer that it has been removed from the local router's mesh view for the +included topic id. + +#### IHAVE + +The `IHAVE` message provides the +remote peer with a list of messages that were recently seen by the local router. + +The remote peer may then request the full message content with an `IWANT` message. + +The speed and degree at which these messages are sent are entirely left to the +broadcast strategy. + +#### IWANT + +The `IWANT` message requests the full content of one or more messages whose IDs +were announced by a remote peer in an `IHAVE` message. + +#### The CHOKE Message + +Upon receiving a `CHOKE` message, the router MUST no longer forward messages to +the peer that sent the `CHOKE` message, while it is still in the mesh. Instead +it MUST always send an IHAVE message (provided there are messages to send and +it does not hit the IHAVE message limit) immediately to the peer. + +A peer MUST NOT send a `CHOKE` message to another peer that is not currently +grafted into it's mesh. + +A peer MUST NOT send a `CHOKE` message to another peer that is already choked +on a given mesh topic. + +##### Pruning + +If a mesh peer sends a `PRUNE`, the local router should consider itself also +unchoked by this peer. If that peer was choked by the local router, as it is no +longer in the mesh, it should also be considered unchoked. + +Therefore, when pruning a choked peer from the mesh, an `UNCHOKE` message is +not required to be sent. + +##### Publishing + +Messages that are published to mesh peers MUST only be published to non-choked +peers. If flood-publishing, messages can be sent to non-mesh peers, which are +unchoked by definition. + +#### The UNCHOKE Message + +Upon receiving an `UNCHOKE` message, the router MUST resume forwarding messages to +the peer that sent the `UNCHOKE` message and halt sending IHAVE messages. + +A peer MUST NOT send an `UNCHOKE` message to any peer that is not currently +grafted into it's mesh. + +A peer MUST NOT send an `UNCHOKE` message to a peer that is already unchoked on +a given mesh topic. + +### Message Processing + +#### Message + +When a message is received that is valid and was not published by the router itself, the router informs the broadcast module via the +Forward(Topic) interface. + +The broadcast module will return which peers to forward the message to and which +to gossip to (if any). + +#### GRAFT Message + +On receiving a `GRAFT(topic)` message, the router will check to see +if it is indeed subscribed to the topic identified in the message. If so, the +router will inform the Mesh strategy for the topic to determine if this peer +should stay in the mesh, or if the peer should be pruned and a PRUNE(topic) +should be sent to inform the peer to remove its mesh link. + +#### PRUNE Message + +On receiving a `PRUNE(topic)` message, the router will remove the sender from `mesh[topic]`. + +#### IHAVE Message + +On receiving an `IHAVE(ids)` message, the router will check its +`seen` cache. If the `IHAVE` message contains message IDs that have not been +the router will ask the [broadcast strategy](./strategies.md) whether it should request it via an +IWANT message. + +#### IWANT Message + +On receiving an `IWANT(ids)` message, the router will check its +`mcache` and will forward any requested messages that are +present in the `mcache` to the peer who sent the `IWANT` message. It does this +immediately. + +## Heartbeat + +Each peer runs a periodic stabilization process called the "heartbeat procedure" +at regular intervals. The frequency of the heartbeat is controlled by the +[parameter](#parameters) `heartbeat_interval`, with a reasonable default of 1 +second. + +The heartbeat serves three functions: + +- Mesh and fanout maintenance - As defined by the mesh strategy +- Gossip Emission - As defined by the broadcast strategy. + +Every heartbeat, the equivalent heartbeat is called for each strategy. diff --git a/generalized-gossipsub/strategies.md b/generalized-gossipsub/strategies.md new file mode 100644 index 000000000..896a8e08a --- /dev/null +++ b/generalized-gossipsub/strategies.md @@ -0,0 +1,79 @@ +# Strategies + +These are extensible "modules" that give extra degrees of freedom to customizing +the core gossipsub protocol on a per-topic and per-time basis. + +They control how meshes are formed (mesh strategy) and how messages are +disseminated through the formed networks (broadcast strategy). + +These strategies are defined below + +## Mesh Strategies + +These strategies define the general topology of a network. They decide how peers +get connected and how the overlay network gets formed. This can give designers a +balance between lean broadcast trees and heavily connected networks by selecting how many and who will form mesh peers. + +A host of network-specific strategies can be made to produce a range of varying overlay networks which can be distinct per-topic. + +### Interface + +This section defines the basic logical interface that defines a mesh strategy. The exact API of how a mesh strategy interfaces with the core +protocol is left to each specific implementation. + +#### JOIN(Topic) + +When our router joins a topic, a new mesh may need to be formed. This module +will need to define how a new mesh is constructed given the known set of peers subscribed to a topic. + +#### SUBSCRIBE(Topic) + +A new peer has subscribed to a topic. The mesh strategy may wish to include this +peer into the mesh and inform the router to send a GRAFT (given the current mesh +state). + +#### GRAFT(Topic) + +A graft message has been received. The mesh strategy needs to decide if the +newly grafted peer aligns with its strategy and whether a PRUNE should be sent +to remove this peer. + +#### Heartbeat + +The heartbeat is a periodic process that can be used by a mesh strategy to +perform mesh and fanout maintenance. Various mesh strategies may wish to +regularly churn, add or remove peers from the mesh and fanout lists. + +## Broadcast Strategies + +These strategies define how messages are broadcast through the network. They set +how eagerly we send messages, whether we direct send them or gossip them and how +often. These strategies can effectively provide tradeoffs between latency, +bandwidth and redundancy on a per-topic, per-time basis. + +### Interface + +This section defines the basic logical interface that defines a mesh strategy. The exact API of how a mesh strategy interfaces with the core +protocol is left to each specific implementation. + +#### Publish(Topic) and Forward(Topic) + +When a message is published, the core protocol will inform the broadcast +strategy of all connected peers on this topic and the mesh peers of this topic. +The strategy will inform the core protocol of the following + +- Which peers to directly publish the message to +- Which peers to gossip (send an IHAVE) the message to (immediately) + +#### IHAVE(msg-ids) + +This optional interface can opt to not request messages via IWANTs if desired. + +#### Heartbeat + +The heartbeat is a periodic process that can be used by a broadcast strategy to +emmit gossip or broadcast messages. + +## Scoring Strategies + +TODO: There is a pretty clean interface for the 1.1 scoring, will update here. diff --git a/generalized-gossipsub/strategies/broadcast/original-gossipsub.md b/generalized-gossipsub/strategies/broadcast/original-gossipsub.md new file mode 100644 index 000000000..b038cdd7d --- /dev/null +++ b/generalized-gossipsub/strategies/broadcast/original-gossipsub.md @@ -0,0 +1,57 @@ +# Original Broadcast Strategy + +This module defines the broadcast/routing mechanics of the original gossipsub design. + +This router is based on randomized topic meshes and gossip. It is a general purpose pubsub protocol +with moderate amplification factors and good scaling properties. + +## Parameters + +This section lists the configurable parameters that control the behavior of +gossipsub, along with a short description and reasonable defaults. Each +parameter is introduced with full context elsewhere in this document. + +| Parameter | Purpose | Reasonable Default | +| --------- | -------------------------------------------------- | ------------------ | +| `D_lazy` | (Optional) the outbound degree for gossip emission | `D` | + +Note that `D_lazy` is used to control the outbound +degree when [emitting gossip](#gossip-emission), which may be tuned separately +than the degree for eager message propagation. + +## Interface Implementation + +### Publish(Topic) and Forward(Topic) + +- If the router is subscribed to the topic, it will send the message to all + peers in `mesh[topic]`. +- If the router is not subscribed to the topic, it will examine the set of peers + in `fanout[topic]`. If this set is empty, the router will choose up to `D` + peers from `peers.gossipsub[topic]` and add them to `fanout[topic]`. Assuming + there are now some peers in `fanout[topic]`, the router will send the message + to each. + +### Forward(Topic) + +If the message has not been previously seen, the router will forward the message to every peer in its local topic mesh, contained in `mesh[topic]`. + +### Graft(Topic) + +All graft links are accepted, no Prune is necessary. + +### Heartbeat + +Gossip is emitted to a random selection of peers for each topic that are not +already members of the topic mesh: + +``` +for each topic in mesh+fanout: + let mids be mcache.get_gossip_ids(topic) + if mids is not empty: + select D_lazy peers from peers.gossipsub[topic] + for each peer not in mesh[topic] or fanout[topic] + emit IHAVE(mids) + +shift the mcache + +``` diff --git a/generalized-gossipsub/strategies/mesh/original-gossipsub.md b/generalized-gossipsub/strategies/mesh/original-gossipsub.md new file mode 100644 index 000000000..e77386206 --- /dev/null +++ b/generalized-gossipsub/strategies/mesh/original-gossipsub.md @@ -0,0 +1,105 @@ +# Original Gossipsub Mesh Strategy + +This module defines the mesh construction strategy of the original gossipsub design. + +This router is based on randomized topic meshes and gossip. It is a general purpose pubsub protocol +with moderate amplification factors and good scaling properties. + +## Parameters + +This section lists the configurable parameters that control the behavior of +gossipsub, along with a short description and reasonable defaults. Each +parameter is introduced with full context elsewhere in this document. + +| Parameter | Purpose | Reasonable Default | +| --------- | ------------------------------------------ | ------------------ | +| `D` | The desired outbound degree of the network | 6 | +| `D_low` | Lower bound for outbound degree | 4 | +| `D_high` | Upper bound for outbound degree | 12 | + +## Interface Implementation + +### JOIN(Topic) + +The router selects up to [`D`](#parameters) peers from its [local peering +state](#peering-state) first examining the `fanout` map. If there are peers in +`fanout[topic]`, the router will move those peers from the `fanout` map to +`mesh[topic]`. If the topic is not in the `fanout` map, or if `fanout[topic]` +contains fewer than `D` peers, the router will attempt to fill `mesh[topic]` +with peers from `peers.gossipsub[topic]` which is the set of all +gossipsub-capable peers it is aware of that are members of the topic. + +### SUBSCRIBE(Topic) + +If the mesh currently has less than `D_low` peers in this topic, send a GRAFT. + +### Graft(Topic) + +On receiving a [`GRAFT(topic)` message](#graft), the router will check to see +if it is indeed subscribed to the topic identified in the message. If so, the +router will add the sender to `mesh[topic]`. If the router is no longer +subscribed to the topic, it will respond with a [`PRUNE(topic)` +message](#prune) to inform the sender that it should remove its mesh link. + +### Heartbeat + +In the heartbeat, this strategy performs two major tasks: + +- Mesh maintenance +- Fanout maintenance + +Both are described in their respective subsections + +#### Mesh Maintenance + +Topic meshes are maintained with the following stabilization algorithm: + +``` +for each topic in mesh: + if |mesh[topic]| < D_low: + select D - |mesh[topic]| peers from peers.gossipsub[topic] - mesh[topic] + ; i.e. not including those peers that are already in the topic mesh. + for each new peer: + add peer to mesh[topic] + emit GRAFT(topic) control message to peer + + if |mesh[topic]| > D_high: + select |mesh[topic]| - D peers from mesh[topic] + for each new peer: + remove peer from mesh[topic] + emit PRUNE(topic) control message to peer +``` + +The [parameters](#parameters) of the algorithm are: + +- `D`: the desired outbound degree of the network +- `D_low`: an acceptable lower threshold for `D`. If there are fewer than + `D_low` peers in a given topic mesh, we attempt to add new peers. +- `D_high`: an acceptable upper threshold for `D`. If there are more than + `D_high` peers in a given topic mesh, we randomly select peers for removal. + +#### Fanout Maintenance + +The `fanout` map is maintained by keeping track of the last published time for +each topic. If we do not publish any messages to a topic within a configurable +TTL, the fanout state for that topic is discarded. + +We also try to ensure that each `fanout[topic]` set has at least `D` members. + +The fanout maintenance algorithm is: + +``` +for each topic in fanout: + if time since last published > fanout_ttl + remove topic from fanout + else if |fanout[topic]| < D + select D - |fanout[topic]| peers from peers.gossipsub[topic] - fanout[topic] + add the peers to fanout[topic] +``` + +The [parameters](#parameters) of the algorithm are: + +- `D`: the desired outbound degree of the network. +- `fanout_ttl`: the time for which we keep the fanout state for each topic. If + we do not publish to a topic within `fanout_ttl`, the `fanout[topic]` set is + discarded. diff --git a/generalized-gossipsub/strategies/mesh/random-choke.md b/generalized-gossipsub/strategies/mesh/random-choke.md new file mode 100644 index 000000000..3092944d9 --- /dev/null +++ b/generalized-gossipsub/strategies/mesh/random-choke.md @@ -0,0 +1,121 @@ +# Random Choke - Mesh Strategy + +This is an example strategy that resembles the gossipsub v2 proposal (mainly as +an example strategy). + +This strategy provides a scale that can trade latency for bandwidth for a wide +use of applications. + +This strategy randomly chokes a percentage of its mesh peers providing a scale +between direct sending of messages in the overlay network and gossiping. + +## Parameters + +This section lists the configurable parameters that control this strategy. + +| Parameter | Purpose | Reasonable Default | +| ------------ | ---------------------------------------------------------- | ------------------ | +| `D` | The desired outbound degree of the network | 6 | +| `D_low` | Lower bound for outbound degree | 4 | +| `D_high` | Upper bound for outbound degree | 12 | +| `choke` | The proportion of the mesh to choke | `0.5` | +| `cycle_time` | The number of heartbeats before shuffling the choked peers | `10` | + +## Interface Implementation + +### JOIN(Topic) + +The router selects up to [`D`](#parameters) peers from its [local peering +state](#peering-state) first examining the `fanout` map. If there are peers in +`fanout[topic]`, the router will move those peers from the `fanout` map to +`mesh[topic]`. If the topic is not in the `fanout` map, or if `fanout[topic]` +contains fewer than `D` peers, the router will attempt to fill `mesh[topic]` +with peers from `peers.gossipsub[topic]` which is the set of all +gossipsub-capable peers it is aware of that are members of the topic. + +The router select the `choke` fraction of peers in the mesh and sends them a +`CHOKE` message. + +### SUBSCRIBE(Topic) + +If the mesh currently has less than `D_low` peers in this topic, send a GRAFT. + +### Graft(Topic) + +On receiving a [`GRAFT(topic)` message](#graft), the router will check to see +if it is indeed subscribed to the topic identified in the message. If so, the +router will add the sender to `mesh[topic]`. If the router is no longer +subscribed to the topic, it will respond with a [`PRUNE(topic)` +message](#prune) to inform the sender that it should remove its mesh link. + +### Heartbeat + +In the heartbeat, this strategy performs two major tasks: + +- Mesh maintenance +- Fanout maintenance +- Choke maintenance + +Both are described in their respective subsections + +#### Mesh Maintenance + +Topic meshes are maintained with the following stabilization algorithm: + +``` +for each topic in mesh: + if |mesh[topic]| < D_low: + select D - |mesh[topic]| peers from peers.gossipsub[topic] - mesh[topic] + ; i.e. not including those peers that are already in the topic mesh. + for each new peer: + add peer to mesh[topic] + emit GRAFT(topic) control message to peer + + if |mesh[topic]| > D_high: + select |mesh[topic]| - D peers from mesh[topic] + for each new peer: + remove peer from mesh[topic] + emit PRUNE(topic) control message to peer +``` + +The [parameters](#parameters) of the algorithm are: + +- `D`: the desired outbound degree of the network +- `D_low`: an acceptable lower threshold for `D`. If there are fewer than + `D_low` peers in a given topic mesh, we attempt to add new peers. +- `D_high`: an acceptable upper threshold for `D`. If there are more than + `D_high` peers in a given topic mesh, we randomly select peers for removal. + +#### Fanout Maintenance + +The `fanout` map is maintained by keeping track of the last published time for +each topic. If we do not publish any messages to a topic within a configurable +TTL, the fanout state for that topic is discarded. + +We also try to ensure that each `fanout[topic]` set has at least `D` members. + +The fanout maintenance algorithm is: + +``` +for each topic in fanout: + if time since last published > fanout_ttl + remove topic from fanout + else if |fanout[topic]| < D + select D - |fanout[topic]| peers from peers.gossipsub[topic] - fanout[topic] + add the peers to fanout[topic] +``` + +The [parameters](#parameters) of the algorithm are: + +- `D`: the desired outbound degree of the network. +- `fanout_ttl`: the time for which we keep the fanout state for each topic. If + we do not publish to a topic within `fanout_ttl`, the `fanout[topic]` set is + discarded. + +#### Choke Maintenance + +Ensure that `choke` proportion of mesh peers are `choked` and either send CHOKE +or UNCHOKE to maintain the proportion. + +Every `choke_time` randomize the set of choked peers in the mesh, maintaining +the proportion.