Skip to content

Update nightly recipe / setup for 2021.11.0 release #308

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jakirkham merged 14 commits into from
Nov 11, 2021
Merged

Update nightly recipe / setup for 2021.11.0 release #308

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
9d2d302
Update conda recipe to match grayskull output
charlesbluca Nov 11, 2021
8d05148
Update package description in setup.py
charlesbluca Nov 11, 2021
81fd84a
Unpin dask dependency for development
charlesbluca Nov 11, 2021
f5851ba
Try out python version variants
charlesbluca Nov 11, 2021
7e139e0
Remove noarch line
charlesbluca Nov 11, 2021
6d4593d
Apply suggestions from code review
charlesbluca Nov 11, 2021
6ae9d4a
Bring back manually computed py_version
charlesbluca Nov 11, 2021
38f03ce
Use python variant for package string name
charlesbluca Nov 11, 2021
903cbe6
Remove dot from python string version
charlesbluca Nov 11, 2021
3f239fc
Set python version string at top of file
charlesbluca Nov 11, 2021
60da6ca
Use jinja2 replace filter
charlesbluca Nov 11, 2021
a674972
Use HTTPS for project URL
charlesbluca Nov 11, 2021
5c4d069
Upload linux packages instead of noarch
charlesbluca Nov 11, 2021
ead3d94
Add maintainers to recipe
charlesbluca Nov 11, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions continuous_integration/recipe/conda_build_config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
py_version:
- 3.7
- 3.8
- 3.9
34 changes: 21 additions & 13 deletions continuous_integration/recipe/meta.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
{% set name = "dask-sql" %}
{% set version = environ.get('GIT_DESCRIBE_TAG', '0.0.0.dev') + environ.get('VERSION_SUFFIX', '') %}
{% set py_version=environ.get('CONDA_PY', 36) %}
Copy link
Collaborator Author

@charlesbluca charlesbluca Nov 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If not will probably need to add this back in and modify



package:
Expand All @@ -11,40 +10,49 @@ source:
git_url: ../..

build:
noarch: python
number: {{ GIT_DESCRIBE_NUMBER }}
string: py{{ py_version }}_{{ GIT_DESCRIBE_HASH }}_{{ GIT_DESCRIBE_NUMBER }}
skip: true # [py2k]
entry_points:
- dask-sql-server = dask_sql.server.app:main
- dask-sql = dask_sql.cmd:main
noarch: python
Copy link
Collaborator Author

@charlesbluca charlesbluca Nov 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we still build nightlies noarch even though the stable releases don't do this?

Copy link
Contributor

@jakirkham jakirkham Nov 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No this should be dropped. It doesn't work with the selector below

Copy link
Collaborator Author

@charlesbluca charlesbluca Nov 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for clarity, does this mean that this package will only be available to the architecture + Python version used to build here (linux-64, py3.8)? If so then we might want to open a larger issue on who is using the nightlies to assess if this is desired or if we have to rethink our workflow.

If not, then we should be good to just remove this line and proceed

Copy link
Contributor

@jakirkham jakirkham Nov 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will need to use variants to construct a build matrix for the Python versions

No easy answers for OS other than including more jobs on other OSes

Am assuming we are the primary users of nightlies. So would use that as a guiding principle until someone else asks for something more

Copy link
Collaborator Author

@charlesbluca charlesbluca Nov 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the link - I'll play around with variants for now, and check in with RAPIDS folk to see if we have any specific needs here.

string: py{{ py_version.replace(".", "") }}_{{ GIT_DESCRIBE_HASH }}_{{ GIT_DESCRIBE_NUMBER }}
Copy link
Collaborator Author

@charlesbluca charlesbluca Nov 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Forgot about this - is there a clean way to get a Python version for the package string from the build variant?

Copy link
Contributor

@jakirkham jakirkham Nov 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can just replace with python instead of py_version. Though maybe you have already done this :)

script: {{ PYTHON }} -m pip install . --no-deps -vv

requirements:
build:
- maven >=3.6.0
host:
- pip
- python >=3.6
- setuptools_scm
- python {{ py_version }}
- setuptools-scm
run:
- python >=3.6
- dask ==2021.10.0
- python
- dask >=2021.10.0
- pandas >=1.0.0
- jpype1 >=1.0.2
- openjdk >=8
- fastapi >=0.61.1
- uvicorn >=0.11.3
- tzlocal >=2.1
- prompt_toolkit >=3.0.8
- pygments >=2.7.3
- nest-asyncio >=1.0.0
- tabulate >=0.8.9
- prompt-toolkit
- pygments
- nest-asyncio
- tabulate
- importlib-metadata # [py<38]
Copy link
Contributor

@jakirkham jakirkham Nov 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the selector that makes noarch not an option


test:
imports:
- dask_sql
commands:
- pip check
- dask-sql-server --help
- dask-sql --help
requires:
- pip

about:
home: https://github.com/dask-contrib/dask-sql
summary: SQL Query Layer for dask
home: http://github.com/dask-contrib/dask-sql/
summary: SQL query layer for Dask
license: MIT
license_file: LICENSE.txt
4 changes: 2 additions & 2 deletions setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ def run(self):

setup(
name="dask_sql",
description="Dask SQL",
description="SQL query layer for Dask",
Copy link
Collaborator Author

@charlesbluca charlesbluca Nov 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just noting that this change maps to the conda recipe's summary.

url="http://github.com/dask-contrib/dask-sql/",
maintainer="Nils Braun",
maintainer_email="[email protected]",
Expand All @@ -74,7 +74,7 @@ def run(self):
python_requires=">=3.6",
setup_requires=["setuptools_scm"] + sphinx_requirements,
install_requires=[
"dask[dataframe,distributed]==2021.10.0",
"dask[dataframe,distributed]>=2021.10.0",
Copy link
Collaborator Author

@charlesbluca charlesbluca Nov 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll need this if we want to use unpinned versions of Dask with the nightlies for certain features (dask-sql-server in particular)

"pandas>=1.0.0", # below 1.0, there were no nullable ext. types
"jpype1>=1.0.2",
"fastapi>=0.61.1",
Expand Down