rococo-runtime: RococoGenesisExt removed

[michalkucharczyk]

RococoGenesisExt is removed. It was the hack to allow overwriting EpochDurationInBlocks. Removal of RococGenesisExt prevents from manipulating the state to change the runtime constants.

Changes:

• Environment variable which controls the time::EpochDurationInBlocks value was added: ROCOCO_EPOCH_DURATION (epoch duration will be set to the value of env),
• 10,100,600 versions of rococo-runtime are built in CI and put into polkadot-debug docker image.

rococo-runtime building examples:

• to build runtime for versi_staging_testnet which had EpochDurationInBlocks set to 100:

  ROCOCO_EPOCH_DURATION=100 cargo build --features=fast-runtime -p rococo-runtime
  

• to build runtime for wococo_development

  ROCOCO_EPOCH_DURATION=10 cargo build --features=fast-runtime -p rococo-runtime
  

• to build versi-staging chain spec:

  ROCOCO_EPOCH_DURATION=100 cargo run -p polkadot --features=fast-runtime -- build-spec --chain versi-staging --raw 
  

• to build wococo-dev chain spec:

  ROCOCO_EPOCH_DURATION=10 cargo run -p polkadot  --features=fast-runtime  -- build-spec --chain wococo-dev --raw 
  

Chain-specs can also be built using provided sample script. As the epoch duration is hard-coded into wasm blob, it is enough to replace the code field.

Step towards: #25

[michalkucharczyk]

It is extracted part of #1256

[michalkucharczyk]

@paritytech/devrel , @the-right-joyce

I wanted to give you a heads-up about some changes that have just been made in this PR. I've modified the default EpochDurationInBlocks for several ChainSpecs, namely: versi-local, wococo-local, rococo-local, wococo-dev, versi-dev, rococo-dev(where duration was previously set to 10), andversi-staging` (where duration was previously set to 100).

You can find additional details in the PR description.

This change could affect tutorials (and docs maybe?) that rely on timing within these runtimes. Therefore, I request you to please review and update them accordingly, and include information on how to adjust timings based on these changes.

Thank you for your attention.

note: this is copy of comment previously made in polkadot repo.

[ordian]

cc @Sophia-Gold since you're using session_length_in_blocks in your test

[PierreBesson]

From the point of view of the Devops team that is generating those chainspec to launch testnets, this doesn't look very
practical to use as it will force us to work from source instead from a binary for just 1 config value tweak.

If we could simply have a --chain rococo-fast-runtime already available in the built node it would be most convenient to do our job and automate our processes for launching custom Rococo-based testnet such as Versi.

[bkchr]

From the point of view of the Devops team that is generating those chainspec to launch testnets, this doesn't look very
practical to use as it will force us to work from source instead from a binary for just 1 config value tweak.

Our test nets are running with at least 1 hour session length any way or not? And 1 hour is still the default.

[eskimor]

From the point of view of the Devops team that is generating those chainspec to launch testnets, this doesn't look very
practical to use as it will force us to work from source instead from a binary for just 1 config value tweak.

Our test nets are running with at least 1 hour session length any way or not? And 1 hour is still the default.

No, Versi has shorter sessions and we would like to keep it at that as session changes are something very crucial to test.

[bkchr]

No, Versi has shorter sessions and we would like to keep it at that as session changes are something very crucial to test.

Yeah and it will still be possible. It will just work differently. This entire changing of the session length was a hack from the beginning, that I should never have used :P

[PierreBesson]

I appreciate the change to make it fully configurable. However, being able to change runtime constants without building from source could be very useful. But I guess, we will have to accept that we won't have a way to build fast-runtime chainspecs from the released binary after this change.

@michalkucharczyk

• Would it be possible to keep the short default epoch duration for the Versi runtime ?
• Would it also be possible to offer a configurable epoch duration for the westend runtime as well ?

[michalkucharczyk]

However, being able to change runtime constants without building from source could be very useful.

We want to go away from configuring runtime parameters using state trie.

The only other solution I see, is to - by default - build the rococo runtime binary with smaller epochDuration value and name it versi-runtime. And hide it behind the feature as other runtimes. This will lengthen the build process.
If that is acceptable, I can do it. However I believe we should not aim into adding more pre-configured runtimes into default build. Adding versi seems fair to me.

The following values were used in runtimes, I assume that we are talking only about keeping staging versions?:

	dev	local	staging
rococo	10	10	600
versi	10	10	100
wococo	10	10

Would it also be possible to offer a configurable epoch duration for the westend runtime as well ?

Technically should be doable. The question is if we should build another runtime version by default or is it OK for you to build it on your side? Maybe we should move this request to new ticket?

[bkchr]

I appreciate the change to make it fully configurable. However, being able to change runtime constants without building from source could be very useful.

This is one of the only ones we support this. I have done this as a hack in 2020 or somewhere then to make it easy to test stuff locally. Versi for sure doesn't fall into this category and I see no problem for you to just rebuild the runtime. In the future we will also be able to support to only build the runtime to build a chain spec, so you will not need to build the entire node. This will then result in a quite fast turn around (assuming you only change something in the runtime lib.rs).

This said, we probably should take the performance hit and also build a fast runtime for rococo-local for now. Sadly parachains are still not able to onboard at genesis and thus, people would be confused why stuff isn't working as they expect it to work.

[michalkucharczyk]

@bkchr I've just realized that it may not be doable for native runtime.

We can't just build the other version of native rococo-runtime crate. We would have to somehow re-use the rococo-runtime sources in rococo-runtime-fast (or whatever the name will be).

Am I missing something?

[bkchr]

The native runtime isn't used anymore. So, this is fine.

We can do something like here.

[BulatSaif]

The current versi and wococo network is different from what we have in the code. We use Versi and wococo based on westend runtime, surprise!

The current process to launch a new testnet is as follows:

1. Run build-spec --chain westend-staging, which generates a plain JSON file.
2. Override fields such as session keys, sudo, balance, id, name... Almost all fields except for "genesis.runtime.system.code."
3. Generate a new raw chainspec using: build-spec --chain ./chainspec-plain.json --raw.
   You can find an example here.

I propose instead having:

	dev	local	staging
rococo	10	10	600
versi	10	10	100
wococo	10	10

Let's have:

	dev	local	staging
rococo	10	100	600

I hope this will work and the only difference between the dev, local, and staging runtime is the session time? I know that dev, local have different session keys, but we can override them.

[bkchr]

Yeah no, we will not support three different lengths for the epoch duration.

@michalkucharczyk can provide you a script for building the stuff in the correct way.

@BulatSaif given that you already patch the chainspec manually, you could also overwrite the system.code field. Then you would only need to recompile the runtime which is way faster.

[BulatSaif]

@BulatSaif given that you already patch the chainspec manually, you could also overwrite the system.code field. Then you would only need to recompile the runtime which is way faster.

We did it in the past as a result we had a chain that is impossible to update (current wococo). After we apply the runtime update chain will stop producing the blocks Error Corrupted state at.
I assume some migration depends on session length, or runtime which I built was incorrect: paritytech/polkadot#7416

I would like to avoid modifying the substrate code by the DevOps team as we are not familiar with the codebase.

But we need a way to deploy and update chains with small session time. It will speed up the development of the parachain team since they can onboard or off board parachains faster.

[bkchr]

I would like to avoid modifying the substrate code by the DevOps team as we are not familiar with the codebase.

Yeah that is reasonable. However, noting down which network uses which session length sounds like a doable task. You will not need to modify any code, you will just need to set an env variable at build time to the appropriate number.

[michalkucharczyk]

@michalkucharczyk can provide you a script for building the stuff in the correct way.

f84cd5d

[BulatSaif]

You will not need to modify any code, you will just need to set an env variable at build time to the appropriate number.

This is the good news, can we have similar option ROCOCO_EPOCH_DURATION for westend?

[michalkucharczyk]

I did some experiment, and added pre-built runtimes into polkadot-debug docker image under /polkadot/runtimes directory.

Rutimes are not explicitly built by cargo build ... command. I added some extra commands into CI in order to build 10 and 100 versions of rococo runtime:

polkadot-sdk/.gitlab/pipeline/build.yml

Lines 22 to 23 in 2eddd98

	- time ROCOCO_EPOCH_DURATION=10 ./polkadot/scripts/build-only-wasm.sh rococo-runtime $(pwd)/runtimes/rococo-runtime-10/
	- time ROCOCO_EPOCH_DURATION=100 ./polkadot/scripts/build-only-wasm.sh rococo-runtime $(pwd)/runtimes/rococo-runtime-100/

There is also another small script to update the code in chain-spec (2eddd98):

./polkadot/scripts/update-chain-spec-with-fast-rococo.sh paritypr/polkadot-debug:1490-657ae27f 100

If that is acceptable, we could keep this as final solution.
If not, we could merge it, and keep it until devops team finds out how to deal with change introduced by this PR.

@bkchr, @BulatSaif wdyt?

[BulatSaif]

If that is acceptable, we could keep this as final solution.
If not, we could merge it, and keep it until devops team finds out how to deal with change introduced by this PR.

The proposed alternative solution to build chainspec with shorter epoch duration looks good. I did not run the scripts but it looks clear and we can use it.

[bkchr]

Mainly nitpicks. For bash scripts IDK. They look very complicated and not sure what they are doing. I would have thought this is just some ROCOCO_EPOCH_DURATION=100 build-wasm-only.sh rococo-runtime script. However, I have no strong views around this, as long as someone finds it useful.