Avoid running subsystem-benchmarks with nextest

[AndreiEres]

Fixes #3704

[ggwpez]

(no opinion on the code but it fixes my issue)

[bkchr]

I don't get how this fix fixes the issue?

I also have the fundamental question, why do we have a benchmark registered as test? Registering the benchmark with harness = false should be achieve the same result?

Generally not sure why this is also not just wrapped into Criterion. I mean you have there logic for warmup etc that would all be done by criterion automatically.

[AndreiEres]

@bkchr
Tests require subsystem-benchmarks feature, but they also depend on polkadot-availability-recovery with the same feature, so nextest runs them. That was the problem.

About why it's done by tests: they are a bit specific, so we need to implement them in that way without criterion.

[bkchr]

About why it's done by tests: they are a bit specific, so we need to implement them in that way without criterion.

Okay fine, but you don't need to use criterion. If you use harness = false you basically have to write your main on your own, like you have done.

[AndreiEres]

If you use harness = false you basically have to write your main on your own, like you have done.

Yes, but what do you mean?

[sandreim]

It's kind of odd to have 2 features in use now. I also don't really get why this fixes the issue.

[sandreim]

subsystem-benchmarks feature seems to no longer be used at all.

[AndreiEres]

It's in use. We require that feature to run regression tests. That way we don't mix normal and regression tests.

https://github.com/paritytech/polkadot-sdk/pull/3716/files#diff-ccb20ed2e9c6555ef9cd2b6e0905839764e8eac5da8970f170957da9cc9efc8fR46-R50

[sandreim]

Can't the test just be skipped if feature not defined ?

[AndreiEres]

Can't the test just be skipped if feature not defined ?

No, cargo test runs all test then.

[bkchr]

Yes, but what do you mean?

I mean this: #3741

[bkchr]

Closing in favor of: #3741