Conversation
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
| let _ = tx.send(result); | ||
| }; | ||
|
|
||
| executor.spawn_blocking("substrate-rpc-subscription", Some("rpc"), blocking_fut.boxed()); |
There was a problem hiding this comment.
yeah right, spawn_blocking doesn't return a handle for the future that's why you need the extra oneshot channel
| /// Limit the RPC functionality to a single connection. | ||
| #[derive(Default, Clone)] | ||
| pub struct RpcConnections { | ||
| data: Arc<Mutex<HashMap<ConnectionId, HashSet<String>>>>, |
There was a problem hiding this comment.
could also be Arc<Mutex<HashSet<(ConnectionId, SubscriptionId)>>> I guess but perhaps you have plan for it to more generic
There was a problem hiding this comment.
mmm since ConnectionId is just a small u64, I'd just combine to Arc<Mutex<HashSet<(ConnectionId, SubscriptionId)>>> and that would be more efficient & slightly simpler code below, and probably small or no real extra space cost :)
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
| /// The maximum number of items reported by the `chainHead_storage` before | ||
| /// pagination is required. | ||
| operation_max_storage_items: usize, | ||
| /// Limit the RPC functionality to a single connection. |
There was a problem hiding this comment.
I would re-phrase this, I think you are trying to say that each connection has its own state and not that the RPC functionality is limited to a single connection?
This connection state is then used to determine whether a certain call is permitted on a particular connection such as "unstable_storage calls" or something like that...
There was a problem hiding this comment.
I've rephrased the documentation a bit, let me know if it makes more sense now 🙏
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
substrate/client/rpc-spec-v2/src/chain_head/subscription/mod.rs
Outdated
Show resolved
Hide resolved
| /// For transactionBroadcast, this represents the operation ID. | ||
| /// For transaction, this is empty and the number of active calls is tracked by | ||
| /// [`Self::num_identifiers`]. | ||
| identifiers: HashSet<String>, |
There was a problem hiding this comment.
Am I right to assume that we are not worries about subscription IDs and operation IDs colliding at all in our implementation here?
There was a problem hiding this comment.
Yep, IIRC, we'll either use this for chainHead with subscriptions ID only; or for the tx with operation ID only
| let mut data = self.data.lock(); | ||
|
|
||
| let entry = data.entry(connection_id).or_insert_with(ConnectionData::default); | ||
| entry.num_identifiers = entry.num_identifiers.saturating_sub(1); |
There was a problem hiding this comment.
Do we ever clear out entries completely when the number goes to 0? Else we might eventually leak memory
| connection_data.num_identifiers = connection_data.num_identifiers.saturating_sub(1); | ||
|
|
||
| if connection_data.num_identifiers == 0 { | ||
| data.remove(&connection_id); |
There was a problem hiding this comment.
Ah, I guess this is whewre we clean up :)
jsdw
left a comment
There was a problem hiding this comment.
Looks good to me; only very small comments in the end :)
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
… context and limit connections (#3481) This PR ensures that the chainHead RPC class can be called only from within the same connection context. The chainHead methods are now registered as raw methods. - paritytech/jsonrpsee#1297 The concept of raw methods is introduced in jsonrpsee, which is an async method that exposes the connection ID: The raw method doesn't have the concept of a blocking method. Previously blocking methods are now spawning a blocking task to handle their blocking (ie DB) access. We spawn the same number of tasks as before, however we do that explicitly. Another approach would be implementing a RPC middleware that captures and decodes the method parameters: - #3343 However, that approach is prone to errors since the methods are hardcoded by name. Performace is affected by the double deserialization that needs to happen to extract the subscription ID we'd like to limit. Once from the middleware, and once from the methods itself. This PR paves the way to implement the chainHead connection limiter: - #1505 Registering tokens (subscription ID / operation ID) on the `RpcConnections` could be extended to return an error when the maximum number of operations is reached. While at it, have added an integration-test to ensure that chainHead methods can be called from within the same connection context. Before this is merged, a new JsonRPC release should be made to expose the `raw-methods`: - [x] Use jsonrpsee from crates io (blocked by: paritytech/jsonrpsee#1297) Closes: #3207 cc @paritytech/subxt-team --------- Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
… context and limit connections (paritytech#3481) This PR ensures that the chainHead RPC class can be called only from within the same connection context. The chainHead methods are now registered as raw methods. - paritytech/jsonrpsee#1297 The concept of raw methods is introduced in jsonrpsee, which is an async method that exposes the connection ID: The raw method doesn't have the concept of a blocking method. Previously blocking methods are now spawning a blocking task to handle their blocking (ie DB) access. We spawn the same number of tasks as before, however we do that explicitly. Another approach would be implementing a RPC middleware that captures and decodes the method parameters: - paritytech#3343 However, that approach is prone to errors since the methods are hardcoded by name. Performace is affected by the double deserialization that needs to happen to extract the subscription ID we'd like to limit. Once from the middleware, and once from the methods itself. This PR paves the way to implement the chainHead connection limiter: - paritytech#1505 Registering tokens (subscription ID / operation ID) on the `RpcConnections` could be extended to return an error when the maximum number of operations is reached. While at it, have added an integration-test to ensure that chainHead methods can be called from within the same connection context. Before this is merged, a new JsonRPC release should be made to expose the `raw-methods`: - [x] Use jsonrpsee from crates io (blocked by: paritytech/jsonrpsee#1297) Closes: paritytech#3207 cc @paritytech/subxt-team --------- Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
This PR ensures that the chainHead RPC class can be called only from within the same connection context.
The chainHead methods are now registered as raw methods.
The concept of raw methods is introduced in jsonrpsee, which is an async method that exposes the connection ID:
The raw method doesn't have the concept of a blocking method. Previously blocking methods are now spawning a blocking task to handle their blocking (ie DB) access. We spawn the same number of tasks as before, however we do that explicitly.
Another approach would be implementing a RPC middleware that captures and decodes the method parameters:
However, that approach is prone to errors since the methods are hardcoded by name. Performace is affected by the double deserialization that needs to happen to extract the subscription ID we'd like to limit. Once from the middleware, and once from the methods itself.
This PR paves the way to implement the chainHead connection limiter:
Registering tokens (subscription ID / operation ID) on the
RpcConnectionscould be extended to return an error when the maximum number of operations is reached.While at it, have added an integration-test to ensure that chainHead methods can be called from within the same connection context.
Before this is merged, a new JsonRPC release should be made to expose the
raw-methods:Closes: #3207
cc @paritytech/subxt-team