Skip to content

[ci] ensure correct Python version is used in CI, fix compatibility with older pre-3.12 patch versions #355

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jameslamb merged 15 commits into from
Jan 10, 2026
Merged

[ci] ensure correct Python version is used in CI, fix compatibility with older pre-3.12 patch versions #355

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
c5d99fa
[ci] ensure correct Python version is used in CI, other small CI fixes
jameslamb Jan 5, 2026
55b72a4
where is Python
jameslamb Jan 7, 2026
6312c25
use correct version
jameslamb Jan 7, 2026
746fb4d
more logs
jameslamb Jan 7, 2026
50df4f1
check PATH
jameslamb Jan 7, 2026
ce5a86c
what is modifying PATH
jameslamb Jan 7, 2026
c1ee176
stop using login shell
jameslamb Jan 7, 2026
c059392
fix Python 3.19-3.11 compatibility
jameslamb Jan 7, 2026
de78fe2
fix patch
jameslamb Jan 7, 2026
47974e9
add a test
jameslamb Jan 7, 2026
daecaa5
fix tests
jameslamb Jan 7, 2026
d901fd9
activate environment
jameslamb Jan 10, 2026
a22854f
docs might need a login shell
jameslamb Jan 10, 2026
31378cf
just do not touch check-docs
jameslamb Jan 10, 2026
56b0bc9
'remove debugging stuff
jameslamb Jan 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/check-docs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ jobs:
- name: build docs
shell: bash -l {0}
run: |
pip install .
python -m pip install .
make -C ./docs html
python -m sphinx -b linkcheck -D linkcheck_timeout=5 --fail-on-warning ./docs ./linkcheck_output
check-links:
Expand Down
7 changes: 2 additions & 5 deletions .github/workflows/manual-run.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,11 +41,8 @@ jobs:
sudo apt-get update -y
sudo apt-get install -y \
llvm
export PATH="/usr/share/miniconda/bin:${PATH}"
conda install \
--yes \
-c conda-forge \
pipx
python -m pip install \
pipx
make install
mkdir -p ./tmp-dir
python bin/get-pypi-files.py\
Expand Down
7 changes: 2 additions & 5 deletions .github/workflows/smoke-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,10 +48,8 @@ jobs:
with:
fetch-depth: 1
persist-credentials: false
- uses: conda-incubator/setup-miniconda@835234971496cad1653abb28a638a281cf32541f # v3.1.0
- uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548 # v6.1.0
with:
activate-environment: pydistcheck-tests
miniforge-version: latest
python-version: ${{ matrix.python_version }}
- name: Install extra tools on Linux
if: startsWith(matrix.os, 'ubuntu')
Expand All @@ -62,8 +60,7 @@ jobs:
- name: run tests
shell: bash -l {0}
run: |
conda install \
--yes \
python -m pip install \
pipx \
requests
make install
Expand Down
9 changes: 1 addition & 8 deletions .github/workflows/static-analysis.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,4 @@ jobs:
with:
fetch-depth: 1
persist-credentials: false
- name: run linting
run: |
export PATH="/usr/share/miniconda/bin:${PATH}"
conda install \
--yes \
-c conda-forge \
pre-commit
make lint
- uses: pre-commit/action@2c7b3805fd2a0fd8c1884dcaebf91fc102a13ecd #v3.0.1
30 changes: 18 additions & 12 deletions .github/workflows/unit-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,11 +70,17 @@ jobs:
with:
fetch-depth: 1
persist-credentials: false
- uses: conda-incubator/setup-miniconda@835234971496cad1653abb28a638a281cf32541f # v3.1.0
- uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548 # v6.1.0
with:
activate-environment: pydistcheck-tests
miniforge-version: latest
python-version: ${{ matrix.python_version }}
python-version: '${{ matrix.python_version }}'
- name: check that correct Python version is used
run: |
which pip
pip --version
which python
python --version
which python3
python3 --version
- name: Install extra tools on Linux
if: startsWith(matrix.os, 'ubuntu')
run: |
Expand All @@ -88,12 +94,12 @@ jobs:
- name: build and check dists
shell: bash -l {0}
run: |
pip install -r ./requirements-tests.txt || exit 1
python -m pip install -r ./requirements-tests.txt || exit 1
Copy link
Owner Author

@jameslamb jameslamb Jan 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was an attempt to fix something weird I'm seeing, but it didn't work.

On the unit-tests (macOS-15-intel, Python-CPython 3.9) job, it looks like the correct Python is getting installed:

/Users/runner/hostedtoolcache/Python/3.9.25/x64/bin/pip
pip 25.3 from /Users/runner/hostedtoolcache/Python/3.9.25/x64/lib/python3.9/site-packages/pip (python 3.9)
/Users/runner/hostedtoolcache/Python/3.9.25/x64/bin/python
Python 3.9.25
/Users/runner/hostedtoolcache/Python/3.9.25/x64/bin/python3
Python 3.9.25

But pytest is picking up the Python 3.14 from the host

============================= test session starts ==============================
platform darwin -- Python 3.14.2, pytest-9.0.2, pluggy-1.6.0

For the unit-tests (windows-latest, Python-CPython 3.9) job, it appears to picking up the correct Python AND pytest is choosing the right version.

/c/hostedtoolcache/windows/Python/3.9.13/x64/Scripts/pip
pip 25.3 from C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pip (python 3.9)

/c/hostedtoolcache/windows/Python/3.9.13/x64/python
Python 3.9.13
/c/hostedtoolcache/windows/Python/3.9.13/x64/python3
Python 3.9.13
...
============================= test session starts =============================
platform win32 -- Python 3.9.13, pytest-8.4.2, pluggy-1.6.0

(build link)

Copy link
Owner Author

@jameslamb jameslamb Jan 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Welp, seeing all these Python 3.14 paths in the "install" step of that Python 3.9 macOS job looks like a problem:

  python -m pip install -v dist/*.whl
  shell: /bin/bash -l {0}
  env:
    pythonLocation: /Users/runner/hostedtoolcache/Python/3.9.25/x64
    PKG_CONFIG_PATH: /Users/runner/hostedtoolcache/Python/3.9.25/x64/lib/pkgconfig
    Python_ROOT_DIR: /Users/runner/hostedtoolcache/Python/3.9.25/x64
    Python2_ROOT_DIR: /Users/runner/hostedtoolcache/Python/3.9.25/x64
    Python3_ROOT_DIR: /Users/runner/hostedtoolcache/Python/3.9.25/x64
Using pip 25.3 from /Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/site-packages/pip (python 3.14)
Processing ./dist/pydistcheck-0.11.0.99-py3-none-any.whl
Requirement already satisfied: click>=8.0 in /Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/site-packages (from pydistcheck==0.11.0.99) (8.3.1)
Installing collected packages: pydistcheck
  changing mode of /Library/Frameworks/Python.framework/Versions/3.14/bin/pydistcheck to 755
Successfully installed pydistcheck-0.11.0.99

(build link)

I honestly don't understand what's happening here. According to https://github.com/actions/setup-python/blob/main/docs/advanced-usage.md#using-update-environment-flag, the Python installed by actions/seutp-python should be added to PATH by default.

Maybe it's added to the end and therefore not being found? But if that's what is happening, I don't know how to reconcile that with which pip / pip --version looking like this:

/Users/runner/hostedtoolcache/Python/3.9.25/x64/bin/pip
pip 25.3 from /Users/runner/hostedtoolcache/Python/3.9.25/x64/lib/python3.9/site-packages/pip (python 3.9)

Copy link
Owner Author

@jameslamb jameslamb Jan 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There it is!

/Library/Frameworks/Python.framework/Versions/Current/bin shows up earlier in PATH than any of the stuff added by setup-python.

...
:/usr/local/opt/pipx_bin
:...
:/usr/local/bin:/usr/local/sbin:/Users/runner/bin
:...
:/Library/Frameworks/Python.framework/Versions/Current/bin
:...
:/Users/runner/hostedtoolcache/Python/3.9.25/x64/bin
:/Users/runner/hostedtoolcache/Python/3.9.25/x64
:...
:/usr/local/opt/pipx_bin
:...

Copy link
Owner Author

@jameslamb jameslamb Jan 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So it does look like setup-python is sticking its own paths on the beginning of PATH as expected

image

And even immediately before the "install" step

image

I think I see the problem... the install step is using shell: bash - l {0}. -l (login shell) is reloading .bashrc and that updates PATH with stuff, putting more of the GitHub Actions paths at the beginning of PATH

The default shell GitHub uses is bash -e {0}, or bash --noprofile --norc -eo pipefail {0} if you set it to bash... notice that --noprofile --norc!!

docs: https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax#jobsjob_iddefaultsrunshell

Honestly, there's no reason CI is using bash -l here. I'm certain I just copied that from somewhere else when I set up this repo. I'm gonna remove that everywhere.

Copy link
Owner Author

@jameslamb jameslamb Jan 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pushed commits removing the -l and saw the correct Python versions being used!

make build check-dists
- name: install
shell: bash -l {0}
run: |
pip install dist/*.whl
python -m pip install dist/*.whl
- name: test
shell: bash -l {0}
run: |
Expand All @@ -116,9 +122,9 @@ jobs:
with:
fetch-depth: 1
persist-credentials: false
- uses: actions/setup-python@v6
- uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548 # v6.1.0
with:
python-version: 'pypy${{ matrix.python_version}}'
python-version: 'pypy${{ matrix.python_version }}'
- name: Install extra tools on Linux
if: startsWith(matrix.os, 'ubuntu')
run: |
Expand All @@ -128,12 +134,12 @@ jobs:
- name: build and check dists
shell: bash -l {0}
run: |
pip install -r ./requirements-tests.txt
python -m pip install -r ./requirements-tests.txt
make build check-dists
- name: install
shell: bash -l {0}
run: |
pip install dist/*.whl
python -m pip install dist/*.whl
- name: test
shell: bash -l {0}
run: |
Expand All @@ -156,9 +162,9 @@ jobs:
with:
fetch-depth: 1
persist-credentials: false
- uses: actions/setup-python@v6
- uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548 # v6.1.0
with:
python-version: "${{ matrix.python_version}}"
python-version: "${{ matrix.python_version }}"
- name: run tests
run: |
bin/check-test-packages.sh ${GITHUB_WORKSPACE}/tests/data
Expand Down
2 changes: 1 addition & 1 deletion .readthedocs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ build:
pre_build:
- python -m sphinx -b linkcheck -D linkcheck_timeout=5 --fail-on-warning ./docs/ ./linkcheck_output
post_create_environment:
- pip install .
- python -m pip install .
conda:
environment: docs/env.yml
formats:
Expand Down
4 changes: 0 additions & 4 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,10 +35,6 @@ clean:
install:
pipx install --force '.[conda]'

.PHONY: lint
lint:
pre-commit run --all-files

.PHONY: linux-wheel
linux-wheel:
docker run \
Expand Down
Loading