nbconvert version without lxml requirement and more

[shalberd]

fixes #3228
closes #3214

Also discussed but never merged in PR 3214

nbconvert version is in use at ODH and in my organization without any issues in relation to Jupyterlab and runtime images.

Also, putting in some recent jupyter* library updates that are still Jupyterlab < 4 compatible, no collisions with Jupyterlab
3.6.5

Keeping Python 3.8 compatibility in here, seperating discussion on that for later.

This is used in Red Hat Open Data Hub Notebook images

https://github.com/opendatahub-io/elyra/blob/dspv2/etc/generic/requirements-elyra.txt
https://github.com/opendatahub-io/elyra/blob/dspv2/pyproject.toml#L30
https://github.com/opendatahub-io/notebooks/blob/main/jupyter/datascience/ubi9-python-3.9/Pipfile#L29

and the Jupyter* updates (jupyter-client, jupyter-core ... ) are working well, stemming from ODH contrib, working well
and tested by me to not conflict with Jupyterlab 3.5, confirmed with pipenv.

https://github.com/opendatahub-io-contrib/workbench-images/blob/main/snippets/bundles/2-datascience/py39/requirements-jupyter.txt#L90

https://github.com/opendatahub-io-contrib/workbench-images/blob/main/DEVELOPMENT.md

@harshad16 have not put in any KFP 2.0 dependencies in here yet, as the idea would be for latest / 3.16.0 maybe, but not elyra 4.0.

What changes were proposed in this pull request?

There was a needed update to nbconvert cause it required lxml. fixing nbconvert to 7.1.0 makes sures that lxml itself is not needed anymore and also that it is still compatible with kfp less than 2 and jupyterlab less than 4in terms of python.

How was this pull request tested?

Having this running in an ODH Contrib Jupyterlab 3.6.5 environment. Gitlab plugin working fine, requirements-elyra working fine in runtime images executing iypnb notebooks in Airflow 2.x.

Developer's Certificate of Origin 1.1

   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.

[shalberd]

@harshad16 @lresende @romeokienzler @kevin-bates does Elyra still need to work with Python 3.8?

https://github.com/elyra-ai/elyra/actions/runs/10460646800/job/28967259508?pr=3240

I don't know anyone still using Python 3.8. Its end of life is in October 2024

https://devguide.python.org/versions/

With regards to Jupyter-Core, ipython and so on, it is now a similar situation with python 3.8 as it was back then with Python 3.7.

#3101 (comment)

In any case, everything from ipython 8.13.1 on https://pypi.org/project/ipython/8.13.1/ does not work with Python 3.8 anymore ... so, keeping ipython at just the version before that for now.

[shalberd]

reasoning for requests update:

#3232 (comment)

https://github.com/psf/requests/releases

2.32.0 was yanked

[shalberd]

Tested built wheel file with Jupyterlab 3.6.5 and elyra requirements baked into a runtime file with Python 3.9, no problem on Airflow. Python and ipynb executed without issues.