Skip to content

[WIP - testing integration] Switch lib-lint-and-test workflow to uv #4932

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 23 commits into
base: develop
Choose a base branch
from
Open

[WIP - testing integration] Switch lib-lint-and-test workflow to uv #4932

Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
0027e08
Switch lib-lint-and-test workflow to uv
ivanzati Oct 20, 2025
87e7247
Merge branch 'develop' into ivanzati/uv-lib-lint-workflow
ivanzati Oct 21, 2025
3e4310c
Update pre-commit hook for lib-lint-and-test workflow
ivanzati Oct 21, 2025
d2bf06e
Fix working directory
ivanzati Oct 21, 2025
a32a4b3
Fix requires-python
ivanzati Oct 21, 2025
15e6bba
Fix coverage report upload
ivanzati Oct 21, 2025
9d8c595
Run pre-commit with uv
ivanzati Oct 21, 2025
addd755
Run integration tests
ivanzati Oct 21, 2025
9653873
Add wandb required for integration tests
ivanzati Oct 21, 2025
cc46d8d
Check integration-tests
ivanzati Oct 21, 2025
ba7b21b
Set CUBLAS_WORKSPACE_CONFIG in integration tests
ivanzati Oct 21, 2025
dddd6a9
Add tensorboard dependency
ivanzati Oct 22, 2025
f99141b
Merge branch 'develop' into ivanzati/uv-lib-lint-workflow
ivanzati Oct 22, 2025
5479217
Run integration tests with uv --frozen
ivanzati Oct 22, 2025
0c528ea
Reharsh uv sync in lib-lint-and-test workflow
ivanzati Oct 22, 2025
b46957d
Fix unit tests
ivanzati Oct 23, 2025
2900225
Merge branch 'develop' into ivanzati/uv-lib-lint-workflow
ivanzati Oct 30, 2025
354c6ce
Fix onnxscript and python 3.13
ivanzati Oct 30, 2025
04c9e23
Merge branch 'develop' into ivanzati/uv-lib-lint-workflow
ivanzati Oct 30, 2025
62e4d8b
Fix uv.lock for imgaug
ivanzati Oct 30, 2025
32e3d7c
Merge branch 'develop' into ivanzati/uv-lib-lint-workflow
ivanzati Oct 31, 2025
0eace50
Update uv.lock
ivanzati Oct 31, 2025
ee3e46e
Merge branch 'develop' into ivanzati/uv-lib-lint-workflow
ivanzati Nov 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
112 changes: 61 additions & 51 deletions .github/workflows/lib-lint-and-test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,14 @@ on:
push:
branches:
- develop
- releases/**
- release/**
Copy link
Contributor

@AlexanderBarabanov AlexanderBarabanov Oct 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In other workflows we use releases/**, although in branch names we use release (e.g. https://github.com/open-edge-platform/training_extensions/tree/release/2.6)

Leonardo, could you please clarify what we plan to use? @leoll2

Copy link
Contributor Author

@ivanzati ivanzati Oct 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have intention to change this to "release/**" for other workflows as well

Copy link
Contributor

@leoll2 leoll2 Oct 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. I'd opt for release for compatibility with the existing release branches. The other workflows have to be updated accordingly.

tags:
- "v*"
pull_request:
merge_group:
branches:
- develop
- releases/**
- release/**
workflow_dispatch: # run on request (no need for PR)

permissions: {} # No permissions by default
Expand Down Expand Up @@ -66,21 +66,28 @@ jobs:
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
persist-credentials: false
- name: Set up Python
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0

- name: Install uv and set the python version
uses: astral-sh/setup-uv@d0cc045d04ccac9d8b7881df0226f9e82c39688e # v6.8.0
with:
enable-cache: false
python-version: "3.12"
- name: Install tox
version: "0.8.15"

- name: Installing dependencies
working-directory: library
run: |
pip install '.[dev]'
- name: Code quality checks
run: uv sync --locked --extra dev --extra cuda --no-extra xpu
Copy link
Contributor

@leoll2 leoll2 Oct 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the extra cuda necessary for the tests to pass? Does it even work on the standard GPU-less Github runners?

Copy link
Contributor Author

@ivanzati ivanzati Oct 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch, indeed the cuda is not required to pass the quality check. Wanted to keep it the same as next tests. Though would revert back. Thanks!


- name: Run pre-commit checks
working-directory: library
run: tox r -vv -e pre-commit
run: uv run --frozen --extra dev --extra cuda --no-extra xpu pre-commit run --all-files
Copy link
Contributor

@leoll2 leoll2 Oct 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You already install the dependencies in the previous step, I think --frozen --extra dev --extra cuda --no-extra xpu is redundant.


Unit-Test:
runs-on: [self-hosted, linux, x64, dev, dmount]
permissions:
contents: read
# The id-token permission is required by Codecov to use OIDC
id-token: write
needs:
- Code-Quality-Checks
- check_paths
Expand All @@ -89,52 +96,41 @@ jobs:
strategy:
fail-fast: false
matrix:
include:
- python-version: "3.12"
tox-env: "py312"
- python-version: "3.11"
tox-env: "py311"
python-version: ["3.11", "3.12"]
name: Unit-Test-with-Python${{ matrix.python-version }}
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
persist-credentials: false
- name: Install Python
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0

- name: Install uv and set the python version
uses: astral-sh/setup-uv@d0cc045d04ccac9d8b7881df0226f9e82c39688e # v6.8.0
with:
enable-cache: false
python-version: ${{ matrix.python-version }}
- name: Install tox
version: "0.8.15"

- name: Installing dependencies
working-directory: library
run: |
pip install '.[dev]'
- name: Run unit test
run: uv sync --locked --extra dev --extra cuda --no-extra xpu

- name: Unit testing
working-directory: library
run: tox r -vv -e unit-test-${{ matrix.tox-env }}
run: uv run --frozen --extra dev --extra cuda --no-extra xpu pytest tests/unit --cov --cov-report=xml

- name: Upload coverage reports to Codecov
working-directory: library
run: |
# If the workflow is triggered from PR then it gets the commit id from the PR.
# else it uses the commit id of the latest commit. This is because the commit
# of the checked-out branch/commit does not exist in the tree as it is grafted.
# Also note: GitHub does not pass secrets to pipelines triggered from a fork.
# This means that upload will fail for PRs from forks.
if [ -n "${{ github.event.pull_request.head.sha }}" ]
then
COMMIT_ID=${{ github.event.pull_request.head.sha }}
else
COMMIT_ID=${{ github.sha }}
fi
# current version of codecov-action does not support uploading reports through the proxy
# so we use the latest version of codecov uploader binary
curl -Os https://uploader.codecov.io/latest/linux/codecov
chmod +x codecov
./codecov -t ${{ secrets.CODECOV_TOKEN }} --sha $COMMIT_ID -U $HTTP_PROXY -f .tox/coverage_unit-test-${{ matrix.tox-env }}.xml -F ${{ matrix.tox-env }}
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
Copy link
Contributor

@AlexanderBarabanov AlexanderBarabanov Oct 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This step will fail for PR from fork (by design, GH token for pull_request trigger has read-only permissions).
I would propose to add condition to skip this step for PR from forks (something like https://github.com/open-edge-platform/training_extensions/blob/develop/.github/workflows/ui-lint-and-test.yaml#L259)

with:
directory: library
flags: coverage_unit-test_Python-${{ matrix.python-version }}
use_oidc: true

Integration-Test:
if: |
github.event.pull_request.draft == false &&
!(startsWith(github.event.pull_request.title, '[WIP]')) &&
needs.check_paths.outputs.run_workflow == 'true'
#if: |
# github.event.pull_request.draft == false &&
# !(startsWith(github.event.pull_request.title, '[WIP]')) &&
# needs.check_paths.outputs.run_workflow == 'true'
runs-on: [self-hosted, linux, x64, dev, dmount]
permissions:
contents: read
Expand All @@ -146,30 +142,44 @@ jobs:
matrix:
include:
- task: "multi_class_cls"
python-version: "3.12"
Copy link
Contributor

@mramotowski mramotowski Nov 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are setting python-version to 3.12 for all of matrix builds wouldn't it be better to set single env for whole job?

- task: "multi_label_cls"
python-version: "3.12"
- task: "h_label_cls"
python-version: "3.12"
- task: "anomaly"
python-version: "3.12"
- task: "keypoint_detection"
python-version: "3.12"
- task: "detection"
python-version: "3.12"
- task: "instance_segmentation"
python-version: "3.12"
- task: "semantic_segmentation"
python-version: "3.12"
name: Integration-Test-${{ matrix.task }}-py312
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
persist-credentials: false
- name: Install Python
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0

- name: Install uv and set the python version
uses: astral-sh/setup-uv@d0cc045d04ccac9d8b7881df0226f9e82c39688e # v6.8.0
with:
python-version: "3.12"
- name: Install tox
enable-cache: false
python-version: ${{ matrix.python-version }}
version: "0.8.15"

- name: Installing dependencies
working-directory: library
run: |
pip install '.[dev]'
- name: Run Integration Test
run: uv sync --locked --extra cuda --no-extra xpu --dev

- name: Integration testing
working-directory: library
run: tox r -vv -e integration-test-${{ matrix.task }} -- --task ${{ matrix.task }} --run-category-only
env:
CUBLAS_WORKSPACE_CONFIG: ":4096:8"
run: uv run --frozen --extra cuda --no-extra xpu pytest tests/integration -ra --showlocals --csv=${{ matrix.task }}_Python-${{ matrix.python-version }}.csv --task ${{ matrix.task }} --open-subprocess --run-category-only

required_check:
name: Required Check lib-lint-and-test
Expand Down
15 changes: 10 additions & 5 deletions library/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ name = "otx"
dynamic = ["version"]
description = "OpenVINO™ Training Extensions: Train, Evaluate, Optimize, Deploy Computer Vision Models via OpenVINO™"
readme = "README.md"
requires-python = ">=3.10"
requires-python = ">=3.10,<3.13"
Copy link
Contributor

@AlexanderBarabanov AlexanderBarabanov Oct 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In app we use 3.13 https://github.com/open-edge-platform/training_extensions/blob/develop/application/docker/Dockerfile#L6
Will it somehow affect lib usage in application?

Copy link
Contributor

@leoll2 leoll2 Oct 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OTX supports Python 3.13, so <3.13 is wrong here. I would leave this line unchanged: ">=3.10".

license = {file = "LICENSE"}
authors = [
{ name = "OpenVINO™ Training Extensions Contributors" },
Expand All @@ -39,8 +39,7 @@ dependencies = [
"einops==0.8.1",
"decord==0.6.0",
"typeguard>=4.3,<4.5",
# TODO(ashwinvaidya17): https://github.com/openvinotoolkit/anomalib/issues/2126
"setuptools<70",
"setuptools",
Copy link
Contributor

@AlexanderBarabanov AlexanderBarabanov Oct 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I propose to firstly merge this PR #4903, as there are some chages in toml as well, e.g.
"setuptools==78.1.1"

Copy link
Contributor Author

@ivanzati ivanzati Oct 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, indeed. Also this would help to integration test with adaptive batch size to succeed I reckon.

"lightning==2.4.0",
"torchmetrics==1.6.0",
"pytorchcv==0.0.67",
Expand All @@ -53,6 +52,7 @@ dependencies = [
"nncf==2.17.0",
"anomalib[core]==1.1.3",
"numpy<2.0",
"tensorboard==2.20.0",
"tensorboardx==2.6.4"
]

Expand All @@ -69,6 +69,7 @@ dev = [
"pytest-csv",
"pytest-cov",
"onnxruntime==1.21.1",
"wandb",
]

cuda = ["torch==2.8.0",
Expand All @@ -82,7 +83,7 @@ xpu = [

docs = [
"furo",
"myst-parser",
"myst-parser>=2.0.0",
"sphinx==7.2.6",
"pydata-sphinx-theme==0.12.0",
"sphinx-tabs",
Expand All @@ -92,7 +93,6 @@ docs = [
"sphinx-autoapi",
"sphinxemoji",
"nbsphinx",
"myst-parser>=2.0.0",
"linkify-it-py",
]

Expand All @@ -115,6 +115,11 @@ otx = "otx.cli:main"
Documentation = "https://open-edge-platform.github.io/training_extensions/"
Repository = "https://github.com/open-edge-platform/training_extensions/"

[tool.uv]
conflicts = [
[{extra="xpu"}, {extra="cuda"}]
]

[tool.uv.sources]
torch = [
{ index = "torch-xpu", extra = "xpu"},
Expand Down
Loading
Loading