run frame-omni-bencher overhead command in CI for all runtimes in the runtime matrix

[clangenb]

As defined in #6347, we want to run the omni-benchers overhead command in the CI to be sure that it keeps working.

Closes #6347

[clangenb]

Some tests failed because not all chains have development genesis presets. I shall add them in the same PR.

[mordamax]

Some tests failed because not all chains have development genesis presets. I shall add them in the same PR.

may be this flag needs to be added?
https://github.com/paritytech/polkadot-sdk/blob/master/.github/workflows/runtimes-matrix.json#L108

if so you can reuse bench_flags from the matrix

[clangenb]

Thanks a lot for the hint! I thought/hoped so too initially, however the flags are different, unfortunately.

I can overwrite the development preset with another one, but for the overhead command "none" is unfortunately not an option. It does need a preset.

[clangenb]

I noticed that there is an open issue regarding the missing genesis presets. I extracted a separate PR for that:

• Add missing genesis preset configs for parachain binaries #7473

Edit: The PRs for adding genesis presets have now been merged. So this PR is unblocked.

[clangenb]

@ggwpez everything works now except for glutton. The error says that it can't look up the transaction, but I can only find that the overhead command is executing system.remark, which should be supported by glutton. Is there an extrinsic I am missing?

Error log

Running short overhead benchmarking for PACKAGE_NAME=glutton-westend-runtime and RUNTIME_BLOB_PATH=./target/release/wbuild/glutton-westend-runtime/glutton_westend_runtime.compact.compressed.wasm
-rw-r--r-- 1 root root 4840334 Mar  4 10:03 ./target/release/wbuild/glutton-westend-runtime/glutton_westend_runtime.compact.compressed.wasm
Running command: ./target/release/frame-omni-bencher v1 benchmark overhead --runtime ./target/release/wbuild/glutton-westend-runtime/glutton_westend_runtime.compact.compressed.wasm
2025-03-04T10:08:11.598970Z  INFO polkadot_sdk_frame::benchmark::overhead: Identified Chain type from metadata: Parachain(paraid = 100)    
2025-03-04T10:08:12.362275Z  INFO polkadot_sdk_frame::benchmark::overhead: Per-block execution overhead [ns]:
Total: 34427576
Min: 331742, Max: 410174
Average: 344275, Median: 342620, Stddev: 9655.69
Percentiles 99th, 95th, 75th: 373583, 353799, 346876    
Error: Client(ApplyExtrinsicFailed(Validity(TransactionValidityError::Unknown(UnknownTransaction::CannotLookup))))

Edit: Ah ok, it seems that the error is misleading. This error is also thrown if the extrinsic is not signed by the sudo key for glutton, and the sudo key is missing in the presets; fixing it now.

[clangenb]

I think the PR is now ready to be review. 🚀 And maybe give it a silent tag?

[clangenb]

Wonderful, thanks for the approval. I think the PR is green enough for the merge queue. Can someone do that?

[clangenb]

We can enable glutton-soon after #7846 is merged. So we should just wait a bit more.

[clangenb]

One more review needed for this tiny PR. @ggwpez, maybe you want to give me the honor of reviewing this PR, as it closes your issue? :D

[clangenb]

I merged master, which increased some timeouts in the CI. Let's try adding it to the merge queue again.