Skip to content

Include all test files in sdist via MANIFEST.in#8162

Open
VladimirGutuev wants to merge 1 commit intohuggingface:mainfrom
VladimirGutuev:fix/include-tests-in-sdist
Open

Include all test files in sdist via MANIFEST.in#8162
VladimirGutuev wants to merge 1 commit intohuggingface:mainfrom
VladimirGutuev:fix/include-tests-in-sdist

Conversation

@VladimirGutuev
Copy link
Copy Markdown

Fixes #8159.

What's wrong

The sdist on PyPI for datasets==4.8.5 only ships 37 of the 95 files under tests/. Without an explicit MANIFEST.in, setuptools' default sdist behavior includes only top-level tests/test_*.py files and drops everything else.

Files missing from the current sdist:

  • tests/conftest.py
  • tests/_test_patching.py
  • everything under tests/commands/, tests/features/, tests/fixtures/, tests/io/, tests/packaged_modules/, tests/distributed_scripts/
  • the audio / image / PDF / JSON test-data files under tests/features/data/ and tests/io/data/

Anyone installing from sdist (distros, mirrors, air-gapped builds) cannot run the test suite — pytest fails to even collect because tests/conftest.py is missing the pytest_plugins configuration that pulls in tests.fixtures.*.

Fix

Add a minimal MANIFEST.in that grafts the whole tests/ directory and excludes bytecode.

graft tests
global-exclude __pycache__
global-exclude *.py[cod]

Verification

Built the sdist locally on main before/after the change and compared:

$ python -m build --sdist
Before After
Files under tests/ in sdist 37 95
tests/conftest.py missing included
tests/_test_patching.py missing included
tests/fixtures/fsspec.py missing included
tests/features/test_audio.py missing included
tests/io/test_csv.py missing included
tests/packaged_modules/test_arrow.py missing included
tests/features/data/test_audio_16000.mp3 missing included
__pycache__ files 0 0

All 95 files under tests/ in the working tree are now in the sdist; nothing else is added.

Without an explicit MANIFEST.in, setuptools' default sdist filters tests
to top-level files matching `test_*.py`. As a result the sdist for
4.8.5 ships only 37 of the 95 files under `tests/`, omitting:

- `tests/conftest.py`, `tests/_test_patching.py`
- everything under `tests/commands/`, `tests/features/`, `tests/fixtures/`,
  `tests/io/`, `tests/packaged_modules/`, `tests/distributed_scripts/`
- audio/image/PDF/JSON test data files

Anyone installing from the sdist (e.g. distros, mirrors, air-gapped
builds) cannot run the test suite.

Add a minimal MANIFEST.in that grafts the entire `tests/` directory and
excludes bytecode. After this change `python -m build --sdist` includes
all 95 test files (verified locally).

Closes huggingface#8159
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tests/conftest.py, tests/_test_patching.py, tests/fixtures/fsspec.py and other files are missing from the PYPI distribution

1 participant