Skip to content
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
2eb5d82
implement as not-dyn generic + part of PartialComponent
gui1117 Mar 28, 2025
e435d70
make statement store optional and not part of polkadot partial
gui1117 Mar 28, 2025
92057c9
also in manual seal
gui1117 Mar 28, 2025
cad2edc
add cli arg + remove from manual seal
gui1117 Apr 28, 2025
fd3a8ea
Merge remote-tracking branch 'origin/master' into gui-statement-store…
gui1117 Apr 28, 2025
4a63575
enable by default
gui1117 Apr 28, 2025
c8a0be6
fmt
gui1117 Apr 28, 2025
e556e9e
Update from github-actions[bot] running command 'prdoc --audience nod…
github-actions[bot] Apr 29, 2025
886ee56
prdoc + remove unused
gui1117 Apr 29, 2025
8909fb2
Merge branch 'master' into gui-statement-store-in-omni-node
gui1117 Apr 29, 2025
e92ee61
Merge remote-tracking branch 'origin/master' into gui-statement-store…
gui1117 May 6, 2025
90dd3a2
network GOOD + host functions WIP
gui1117 May 31, 2025
3ddf39a
actually only register host fn for offchain worker
gui1117 May 31, 2025
928d88f
zero size statement store when disabled
gui1117 May 31, 2025
263ced1
enable if runtime API present at best hash
gui1117 May 31, 2025
7d3ffc8
remove zero size statement store, inexistence is handled good.
gui1117 May 31, 2025
5ced8b5
some todo
gui1117 May 31, 2025
8710f06
register metrics
gui1117 Jun 2, 2025
bd71ce2
log warning when ValidateStatement exists but statement store disabled
gui1117 Jun 2, 2025
aab795e
Merge remote-tracking branch 'origin/master' into gui-statement-store…
gui1117 Jun 2, 2025
31f4e62
fix usage of modified api in test
gui1117 Jun 3, 2025
ae45a8b
implement test
gui1117 Jun 3, 2025
1445aa9
additional inner comments
gui1117 Jun 3, 2025
ce63cef
taplo format
gui1117 Jun 3, 2025
424ec19
address issues
gui1117 Jun 6, 2025
4b764ab
fix some broken usage + clippy
gui1117 Jun 6, 2025
d54947d
disable test in ci, by wrapping in module
gui1117 Jun 6, 2025
2d8fd4a
Apply suggestions from code review
gui1117 Jun 7, 2025
bf1c26a
format + fix
gui1117 Jun 7, 2025
1cb4e2d
Merge branch 'master' into gui-statement-store-in-omni-node
gui1117 Jun 10, 2025
e0f5921
Update cumulus/polkadot-omni-node/lib/src/common/statement_store.rs
gui1117 Jun 17, 2025
94a87c4
Update cumulus/polkadot-omni-node/lib/src/common/statement_store.rs
gui1117 Jun 17, 2025
8100ca8
nits
gui1117 Jun 17, 2025
6e74940
refactor zombienet test
gui1117 Jun 17, 2025
494be5d
fmt
gui1117 Jun 17, 2025
ade2a18
Merge branch 'master' into gui-statement-store-in-omni-node
gui1117 Jun 17, 2025
2d61b75
bump major for people westend runtime
gui1117 Jun 17, 2025
aba9bf9
Merge remote-tracking branch 'origin/master' into gui-statement-store…
gui1117 Jun 23, 2025
28afb7a
use cli arg: enable_statement_store
gui1117 Jun 23, 2025
32200e4
fmt
gui1117 Jun 23, 2025
b50d9b3
doc
gui1117 Jun 23, 2025
807f37d
prdoc
gui1117 Jun 23, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions cumulus/polkadot-omni-node/lib/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ sc-rpc = { workspace = true, default-features = true }
sc-runtime-utilities = { workspace = true, default-features = true }
sc-service = { workspace = true, default-features = false }
sc-sysinfo = { workspace = true, default-features = true }
sc-statement-store = { workspace = true, default-features = true }
sc-telemetry = { workspace = true, default-features = true }
sc-tracing = { workspace = true, default-features = true }
sc-transaction-pool = { workspace = true, default-features = true }
Expand All @@ -71,6 +72,7 @@ sp-keystore = { workspace = true, default-features = true }
sp-offchain = { workspace = true, default-features = true }
sp-runtime = { workspace = true }
sp-session = { workspace = true, default-features = true }
sp-statement-store = { workspace = true, default-features = true }
sp-storage = { workspace = true, default-features = true }
sp-timestamp = { workspace = true, default-features = true }
sp-transaction-pool = { workspace = true, default-features = true }
Expand Down
9 changes: 9 additions & 0 deletions cumulus/polkadot-omni-node/lib/src/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,14 @@ pub struct Cli<Config: CliConfig> {
#[arg(long)]
pub dev_block_time: Option<u64>,

/// Disable the statement store.
///
/// The statement store is an off-chain data-store for signed statements accessible via RPC
/// and OCW.
/// It uses the runtime api to get the allowance associated to an account.
#[arg(long)]
pub disable_statement_store: bool,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As per the offline discussion, should we disable the statement store by default ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we disable by default I fear nobody will enable it. But maybe it will be enough with the few that will enable it.

I feel going into a dedicated binary for people chain seems the better then no?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that having a dedicated binary for the people chain would create more complications. We would need to release one more binary, change all the tests that involve the rococo-people chain to use the dedicated binary, change the deployment process only for the people chain. I don't know.

Or maybe we should just enable the statement-store by default as it is done here. I'm not sure which solution is the best. I'm curious what others think as well.

cc: @skunert @iulianbarbu @seemantaggarwal

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have also this other PR #8421 that make the CLI configurable for the statement store. This way we wouldn't need to maintain a different service in a different binary. But it still requires a dedicated process for the release.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would go for support in the existing polkadot-omni-node-lib instead for a new binary too.

If we disable by default I fear nobody will enable it.

Would it be widely useful for parachains? If no, only the ones requiring it will enable it, and that should be fine. I would rather go for the reverse logic here.


/// DEPRECATED: This feature has been stabilized, pLease use `--authoring slot-based` instead.
///
/// Use slot-based collator which can handle elastic scaling.
Expand Down Expand Up @@ -231,6 +239,7 @@ impl<Config: CliConfig> Cli<Config> {
.then(|| AuthoringPolicy::SlotBased)
.unwrap_or(self.authoring),
export_pov: self.export_pov_to_path.clone(),
disable_statement_store: self.disable_statement_store,
max_pov_percentage: self.run.experimental_max_pov_percentage,
}
}
Expand Down
6 changes: 6 additions & 0 deletions cumulus/polkadot-omni-node/lib/src/common/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ use sp_runtime::{
OpaqueExtrinsic,
};
use sp_session::SessionKeys;
use sp_statement_store::runtime_api::ValidateStatement;
use sp_transaction_pool::runtime_api::TaggedTransactionQueue;
use std::{fmt::Debug, path::PathBuf, str::FromStr};

Expand Down Expand Up @@ -70,6 +71,7 @@ pub trait NodeRuntimeApi<Block: BlockT>:
+ OffchainWorkerApi<Block>
+ CollectCollationInfo<Block>
+ GetCoreSelectorApi<Block>
+ ValidateStatement<Block>
+ Sized
{
}
Expand All @@ -83,6 +85,7 @@ impl<T, Block: BlockT> NodeRuntimeApi<Block> for T where
+ OffchainWorkerApi<Block>
+ GetCoreSelectorApi<Block>
+ CollectCollationInfo<Block>
+ ValidateStatement<Block>
{
}

Expand Down Expand Up @@ -113,6 +116,9 @@ pub struct NodeExtraArgs {
/// If set, each `PoV` build by the node will be exported to this folder.
pub export_pov: Option<PathBuf>,

/// Disable the statement store.
pub disable_statement_store: bool,

/// The maximum percentage of the maximum PoV size that the collator can use.
/// It will be removed once <https://github.com/paritytech/polkadot-sdk/issues/6020> is fixed.
pub max_pov_percentage: Option<u32>,
Expand Down
13 changes: 11 additions & 2 deletions cumulus/polkadot-omni-node/lib/src/common/rpc.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,10 @@ use crate::common::{
ConstructNodeRuntimeApi,
};
use pallet_transaction_payment_rpc::{TransactionPayment, TransactionPaymentApiServer};
use sc_rpc::dev::{Dev, DevApiServer};
use sc_rpc::{
dev::{Dev, DevApiServer},
statement::{StatementApiServer, StatementStore},
};
use sp_runtime::traits::Block as BlockT;
use std::{marker::PhantomData, sync::Arc};
use substrate_frame_rpc_system::{System, SystemApiServer};
Expand All @@ -32,11 +35,12 @@ use substrate_state_trie_migration_rpc::{StateMigration, StateMigrationApiServer
/// A type representing all RPC extensions.
pub type RpcExtension = jsonrpsee::RpcModule<()>;

pub(crate) trait BuildRpcExtensions<Client, Backend, Pool> {
pub(crate) trait BuildRpcExtensions<Client, Backend, Pool, StatementStore> {
fn build_rpc_extensions(
client: Arc<Client>,
backend: Arc<Backend>,
pool: Arc<Pool>,
statement_store: Option<Arc<StatementStore>>,
) -> sc_service::error::Result<RpcExtension>;
}

Expand All @@ -47,6 +51,7 @@ impl<Block: BlockT, RuntimeApi>
ParachainClient<Block, RuntimeApi>,
ParachainBackend<Block>,
sc_transaction_pool::TransactionPoolHandle<Block, ParachainClient<Block, RuntimeApi>>,
sc_statement_store::Store,
> for BuildParachainRpcExtensions<Block, RuntimeApi>
where
RuntimeApi:
Expand All @@ -60,13 +65,17 @@ where
pool: Arc<
sc_transaction_pool::TransactionPoolHandle<Block, ParachainClient<Block, RuntimeApi>>,
>,
statement_store: Option<Arc<sc_statement_store::Store>>,
) -> sc_service::error::Result<RpcExtension> {
let build = || -> Result<RpcExtension, Box<dyn std::error::Error + Send + Sync>> {
let mut module = RpcExtension::new(());

module.merge(System::new(client.clone(), pool).into_rpc())?;
module.merge(TransactionPayment::new(client.clone()).into_rpc())?;
module.merge(StateMigration::new(client.clone(), backend).into_rpc())?;
if let Some(statement_store) = statement_store {
module.merge(StatementStore::new(statement_store).into_rpc())?;
}
module.merge(Dev::new(client).into_rpc())?;

Ok(module)
Expand Down
20 changes: 20 additions & 0 deletions cumulus/polkadot-omni-node/lib/src/common/spec.rs
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ use sc_consensus::DefaultImportQueue;
use sc_executor::{HeapAllocStrategy, DEFAULT_HEAP_ALLOC_STRATEGY};
use sc_network::{config::FullNetworkConfiguration, NetworkBackend, NetworkBlock};
use sc_service::{Configuration, ImportQueue, PartialComponents, TaskManager};
use sc_statement_store::Store;
use sc_sysinfo::HwBench;
use sc_telemetry::{TelemetryHandle, TelemetryWorker};
use sc_tracing::tracing::Instrument;
Expand Down Expand Up @@ -239,6 +240,7 @@ pub(crate) trait NodeSpec: BaseNodeSpec {
ParachainClient<Self::Block, Self::RuntimeApi>,
ParachainBackend<Self::Block>,
TransactionPoolHandle<Self::Block, ParachainClient<Self::Block, Self::RuntimeApi>>,
Store,
>;

type StartConsensus: StartConsensus<
Expand Down Expand Up @@ -284,6 +286,22 @@ pub(crate) trait NodeSpec: BaseNodeSpec {
.await
.map_err(|e| sc_service::Error::Application(Box::new(e) as Box<_>))?;

let statement_store = if !node_extra_args.disable_statement_store {
let statement_store = sc_statement_store::Store::new_shared(
&parachain_config.data_path,
Default::default(),
client.clone(),
params.keystore_container.local_keystore(),
parachain_config.prometheus_registry(),
&task_manager.spawn_handle(),
)
.map_err(|e| sc_service::Error::Application(Box::new(e) as Box<_>))?;

Some(statement_store)
} else {
None
};

let validator = parachain_config.role.is_authority();
let prometheus_registry = parachain_config.prometheus_registry().cloned();
let transaction_pool = params.transaction_pool.clone();
Expand Down Expand Up @@ -332,12 +350,14 @@ pub(crate) trait NodeSpec: BaseNodeSpec {
let client = client.clone();
let transaction_pool = transaction_pool.clone();
let backend_for_rpc = backend.clone();
let statement_store = statement_store.clone();

Box::new(move |_| {
Self::BuildRpcExtensions::build_rpc_extensions(
client.clone(),
backend_for_rpc.clone(),
transaction_pool.clone(),
statement_store.clone(),
)
})
};
Expand Down
9 changes: 9 additions & 0 deletions cumulus/polkadot-omni-node/lib/src/fake_runtime_api/utils.rs
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,15 @@ macro_rules! impl_node_runtime_apis {
unimplemented!()
}
}

impl sp_statement_store::runtime_api::ValidateStatement<$block> for $runtime {
fn validate_statement(
_source: sp_statement_store::runtime_api::StatementSource,
_statement: sp_statement_store::Statement,
) -> Result<sp_statement_store::runtime_api::ValidStatement, sp_statement_store::runtime_api::InvalidStatement> {
unimplemented!()
}
}
}
};
}
Expand Down
1 change: 1 addition & 0 deletions cumulus/polkadot-omni-node/lib/src/nodes/manual_seal.rs
Original file line number Diff line number Diff line change
Expand Up @@ -259,6 +259,7 @@ impl<NodeSpec: NodeSpecT> ManualSealNode<NodeSpec> {
client.clone(),
backend_for_rpc.clone(),
transaction_pool.clone(),
None,
)?;
module
.merge(ManualSeal::new(manual_seal_sink.clone()).into_rpc())
Expand Down
14 changes: 14 additions & 0 deletions prdoc/pr_8076.prdoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
title: 'Enable statement store in polkadot-omni-node and polkadot-parachain'
doc:
- audience: Node Operator
description: |-
This PR introduces and enables the statement store and add a new argument `disable_statement_store` in `polkadot-omni-node-lib`.
This takes effect in `polkadot-omni-node` and `polkadot-parachain` and any node depending on `polkadot-omni-node-lib`.

The statement store is an off-chain data-store for signed statements accessible via RPC and OCW.
It uses the runtime api to get the allowance associated to an account.

If the runtime doesn't implement the API `ValidateStatement` then all statement submission will safely fail with the error `Error calling into the runtime`.
crates:
- name: polkadot-omni-node-lib
bump: major
Loading