Skip to content

Fixed randomly failing tests #2026

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 18, 2024
Merged

Fixed randomly failing tests #2026

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions tests/test_import_giskard.py
View file
Open in desktop
Copy link
Member

@Inokinoki Inokinoki Sep 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the threshold defined on an ARM mac?
Our CI machines are mostly on Intel and it could import some extra libraries such as cudnn etc.
Could these be the root causes that we are exceeding the threshold?

Copy link
Member Author

@kevinmessiaen kevinmessiaen Sep 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I defined the threshold based on my machine (arm mac).

Seems like it only fail on Python 3.12 (linux). Might be due to a specific library that we import you're right, I need to get access to a x86 linux device to analyse the import time of each module and see what goes wrong.

Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,11 @@ def test_import_giskard():
start = time()

# Start subprocess to ensure that giskard is not already imported
subprocess.run(["python", "-c", "import giskard"])
# Average of 10 imports to avoid random issues
for _ in range(10):
subprocess.run(["python", "-c", "import giskard"])

end = time()
assert (
end - start < IMPORT_TIME_THRESHOLD_SECOND
), f"Import of Giskard took {end - start} seconds (maximum threshold is set to {IMPORT_TIME_THRESHOLD_SECOND} second)"
end - start
) / 10 < IMPORT_TIME_THRESHOLD_SECOND, f"Import of Giskard took {(end - start) / 10} seconds on average (maximum threshold is set to {IMPORT_TIME_THRESHOLD_SECOND} second)"