Enable benchmarking all pallets for a given runtime with bench-all

[seadanda]

Add a runtime parameter to bench-all to allow benchmarking all pallets for a given runtime with one command

[seadanda]

@mordamax I created this but for some reason the artifacts that are generated when it runs does not include a diff, only json output. Am I missing something?

[mordamax]

@seadanda could you please share the pipeline for the context? thx

[seadanda]

@seadanda could you please share the pipeline for the context? thx

Ah I cleaned it and just used the individual bench scripts, I'm working on a branch for coretime, so I'll run it on that and update here when I have the artifacts to show you.

[seadanda]

@mordamax I realised that I sent the job link to someone in element, so I've got the job I ran for paritytech/polkadot-sdk#2024 here: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/4238416

[mordamax]

@seadanda could you please resolve conflicts?

@mordamax I realised that I sent the job link to someone in element, so I've got the job I ran for paritytech/polkadot-sdk#2024 here: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/4238416

sorry, This job does not have a trace. is what it says by now

could you please re-run so I can see what you mean by only json output ?

[seadanda]

I've created paritytech/polkadot-sdk#2548 and run it for glutton-westend to reproduce it on a small runtime.

[seadanda]

@mordamax The bench-bot run for glutton finished and didn't show any diff, so I ran bench-all cumulus with no runtime argument to compare, but actually it doesn't return a diff either for glutton.

It did produce a diff for asset-hub-rococo, though so I ran bench bot with the runtime argument and it didn't produce a diff.

If you get a chance could you take a look please?

[mordamax]

@ggwpez i have no idea why that happens, may be you could help?
https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/4546953

[seadanda]

Bump - sorry this is very old but has anybody got any ideas? It would be nice to merge this, but more importantly it seems that something is wrong with bench-all on main too.

[mordamax]

@seadanda i've updated your branch on sdk and re-started the bench with runtime specified to see the fresh results

[seadanda]

Looks like it worked: paritytech/polkadot-sdk@bd47706

Thanks for that - could we merge this in that case?

[mordamax]

thanks for contribution @seadanda !