[WIP] tests: perform exponential backoff in integration tests #1389
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
peaBerberian
added
tests
peaBerberian
force-pushed
the
misc/test-integration-backoff
branch
12 times, most recently
from
8be330f to
34ff493
Compare
peaBerberian
force-pushed
the
misc/test-integration-backoff
branch
5 times, most recently
from
6e7e291 to
335631a
Compare
This is an implementation of an idea we had in the past to speed up
asynchronous tests.
Those often depend on some timeout, with a timeout set very high
(hundreds of milliseconds, sometimes seconds) to ensure a test pass even
on the slowest machine running up our tests.
This results in our integration tests taking a lot of time to finish
(more than 30 minutes), even more considering that those are not
parallelized, unlike our unit tests (and doing so is not straightforward
because we rely on a dependency, mocha, as a test runner and on another,
karma, as a browser launcher).
To make it less long, we wondered if we could not rely on another
strategy:
1. Perform the check almost immediately, on fast computers it has a
chance of very quickly passing
2. If the check fails, wait some amount of time and re-perform the
check.
3. If the check fails multiple times, it surely means that the current
device is slow, so we'll wait a longer and longer delay before
re-performing the check.
4. If the check still fails after some set time (or 4 seconds by
default), fail the test.
This is basically another implementation of an "exponential backoff"
algorithm in the player (we already use this type of algorithm at
several places in the player, like for media requests for example).
peaBerberian
force-pushed
the
misc/test-integration-backoff
branch
from
335631a to
3a0ae47
Compare
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is an implementation of an idea we had in the past to speed up asynchronous tests.
Those often depend on some timeout, with a timeout set very high (hundreds of milliseconds, sometimes seconds) to ensure a test pass even on the slowest machine running our tests.
This results in our integration tests taking a lot of time to finish (more than 30 minutes), even more considering that those are not parallelized, unlike our unit tests (and doing so is not straightforward because we rely on a dependency, mocha, as a test runner and on another, karma, as a browser launcher).
To make it less long, we wondered if we could not rely on another strategy:
This is basically another implementation of an "exponential backoff" algorithm in the player (we already use this type of algorithm at several places in the player, like for media requests for example).
This commit does not yet transition everything to that new way, because I didn't have the time yet.
TODO: