Skip to content

Elastic scaling: add e2e test#3929

Merged
sandreim merged 27 commits intomasterfrom
sandreim/elastic_zombies
Apr 5, 2024
Merged

Elastic scaling: add e2e test#3929
sandreim merged 27 commits intomasterfrom
sandreim/elastic_zombies

Conversation

@sandreim
Copy link
Copy Markdown
Contributor

@sandreim sandreim commented Apr 1, 2024

On top of #3879

I've also moved the previous test where we ensure multiple cores per para doesn't break non elastic parachains.

sandreim added 18 commits March 28, 2024 14:47
@sandreim
refactor: move claim queue fetch utils to utils crate
e3b5192 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
cache all groups per para and remove para:group 1:1 assumption
01ceb3b 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
Merge branch 'master' of github.com:paritytech/polkadot-sdk into sand…
5ed7841 
…reim/elastic_statement_dist
@sandreim
typo
5f0872f 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
fix test build
034b61a 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
testing some local test run weirdness
9c91193 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
fix doctest
f87a624 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
remove panic
70213c5 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
add new cluster test
4c66001 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
add grid test
21ae63d 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
Merge branch 'master' of github.com:paritytech/polkadot-sdk into sand…
e923dac 
…reim/elastic_statement_dist

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
fmt
27ee145 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
explicit max_candidate_depth
be1a2c7 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
fix test comment
b9f1afc 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
Add elastic scaling zombienet test
6340e32 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
js script
8495e6f 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
move mvp test
04dc5fd 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
fix cmd
bd431cb 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim sandreim added R0-no-crate-publish-required The change does not require any crates to be re-published. T10-tests This PR/Issue is related to tests. labels Apr 1, 2024
@sandreim sandreim requested a review from a team as a code owner April 1, 2024 14:52
@sandreim
fix path
1713be1 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
pepoviola
pepoviola reviewed Apr 1, 2024
View reviewed changes
alindima
alindima reviewed Apr 2, 2024
View reviewed changes
alindima
alindima reviewed Apr 2, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@alindima alindima left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you also make some changes to the adder-collator for this test to pass?

@sandreim
Copy link
Copy Markdown
Contributor Author

sandreim commented Apr 2, 2024

Did you also make some changes to the adder-collator for this test to pass?

No direct changes were needed as we just define the collator_fn which is used by collation-generation that has been already upgraded to support elastic scaling

sandreim added 4 commits April 2, 2024 13:47
@sandreim
rename mvp test
14fcb4b 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
proper js args
07b58a3 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
non elast scaling para assert
2b2d2ab 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
some tolerance
332fda9 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
properly call assign-core.js
86c22a4 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Base automatically changed from sandreim/elastic_statement_dist to master April 3, 2024 08:57
sandreim added 2 commits April 3, 2024 12:01
@sandreim
Merge branch 'master' of github.com:paritytech/polkadot-sdk into sand…
a32c2c7 
…reim/elastic_zombies

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
minor script change
7bde9d8 
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
@sandreim
Copy link
Copy Markdown
Contributor Author

sandreim commented Apr 4, 2024

Seems like deployment error: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5751023

Error: Timeout(0) for: waitLog(): pod: temp-collator, container: temp-collator, log: waiting for copy files to finish

CC @pepoviola

@pepoviola
Copy link
Copy Markdown
Contributor

pepoviola commented Apr 4, 2024

Seems like deployment error: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5751023

Error: Timeout(0) for: waitLog(): pod: temp-collator, container: temp-collator, log: waiting for copy files to finish

CC @pepoviola

Fixed in a4531235d2808a8a614d96df9bf62a31ec283da1. The config image was the problem.

Thx!

@sandreim
Copy link
Copy Markdown
Contributor Author

sandreim commented Apr 5, 2024

Seems like deployment error: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5751023
Error: Timeout(0) for: waitLog(): pod: temp-collator, container: temp-collator, log: waiting for copy files to finish
CC @pepoviola

Fixed in a4531235d2808a8a614d96df9bf62a31ec283da1. The config image was the problem.

Thx!

thank you @pepoviola I totally missed it.

eskimor
eskimor approved these changes Apr 5, 2024
View reviewed changes
Copy link
Copy Markdown
Member

@eskimor eskimor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I read this right, that we are doing elastic scaling with 3 collators here instead of just one connecting to multiple backing groups? I guess that works with the adder collator. Do we also support one collator collating on 3 cores?

@sandreim
Copy link
Copy Markdown
Contributor Author

sandreim commented Apr 5, 2024

Do I read this right, that we are doing elastic scaling with 3 collators here instead of just one connecting to multiple backing groups? I guess that works with the adder collator. Do we also support one collator collating on 3 cores?

Just single collator case, the test has 2 parachains each with one collator:

  • 2000 has 3 cores assigned
  • 2001 has 1 core assigned

@sandreim sandreim added this pull request to the merge queue Apr 5, 2024
Merged via the queue into master with commit 0dc0d40 Apr 5, 2024
@sandreim sandreim deleted the sandreim/elastic_zombies branch April 5, 2024 10:09
Ank4n pushed a commit that referenced this pull request Apr 9, 2024
@sandreim @pepoviola @Ank4n
Elastic scaling: add e2e test (#3929)
bfd7ae2 
On top of #3879

I've also moved the previous test where we ensure multiple cores per
para doesn't break non elastic parachains.

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Co-authored-by: Javier Viola <363911+pepoviola@users.noreply.github.com>
dharjeezy pushed a commit to dharjeezy/polkadot-sdk that referenced this pull request Apr 9, 2024
@sandreim @pepoviola @dharjeezy
Elastic scaling: add e2e test (paritytech#3929)
954ea07 
On top of paritytech#3879

I've also moved the previous test where we ensure multiple cores per
para doesn't break non elastic parachains.

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Co-authored-by: Javier Viola <363911+pepoviola@users.noreply.github.com>
This was referenced Jun 5, 2024
Closed
Closed
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pepoviola pepoviola pepoviola left review comments

@alindima alindima alindima left review comments

@tdimitrov tdimitrov tdimitrov approved these changes

@eskimor eskimor eskimor approved these changes

Assignees

No one assigned

Labels

R0-no-crate-publish-required The change does not require any crates to be re-published. T10-tests This PR/Issue is related to tests.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@sandreim @pepoviola @tdimitrov @eskimor @alindima