Update generic component operator to support python 3.7

[akchinSTC]

Dependent on #2721

What changes were proposed in this pull request?

These changes update the KFP generic component and introduces a separate requirements file for runtime images that still use python 3.7 as this causes issues with dependency resolution during installation of ipython >= 8.

How was this pull request tested?

Developer's Certificate of Origin 1.1

   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.

[elyra-bot]

Thanks for making a pull request to Elyra!

To try out this branch on binder, follow this link:

[kiersten-stokes]

I'm getting a failure after notebook execution in convert_notebook_to_html(notebook_output, notebook_html) in the bootstrapper. import nbconvert is causing errors (logs below).

Looks like the notebook executes fine though. And no issues at all with scripts.

Log Output

Executing: 100%|██████████| 2/2 [00:01<00:00,  1.55cell/s]
[I 15:17:56.042] 'helloworld':'helloworld' - notebook execution completed (1.779 secs)
[E 15:17:56.044] Unexpected error: 
Traceback (most recent call last):
  File "/var/lib/pandas/jupyter-work-dir/bootstrapper.py", line 356, in execute
    NotebookFileOp.convert_notebook_to_html(notebook_output, notebook_html)
  File "/var/lib/pandas/jupyter-work-dir/bootstrapper.py", line 377, in convert_notebook_to_html
    import nbconvert
  File "/usr/local/lib/python3.9/site-packages/nbconvert/__init__.py", line 4, in 
    from .exporters import *
  File "/usr/local/lib/python3.9/site-packages/nbconvert/exporters/__init__.py", line 1, in 
    from .base import (export, get_exporter, 
  File "/usr/local/lib/python3.9/site-packages/nbconvert/exporters/base.py", line 14, in 
    from ipython_genutils.py3compat import string_types
ModuleNotFoundError: No module named 'ipython_genutils'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/var/lib/pandas/jupyter-work-dir/bootstrapper.py", line 697, in 
    main()
  File "/var/lib/pandas/jupyter-work-dir/bootstrapper.py", line 687, in main
    file_op.execute()
  File "/var/lib/pandas/jupyter-work-dir/bootstrapper.py", line 364, in execute
    NotebookFileOp.convert_notebook_to_html(notebook_output, notebook_html)
  File "/var/lib/pandas/jupyter-work-dir/bootstrapper.py", line 377, in convert_notebook_to_html
    import nbconvert
  File "/usr/local/lib/python3.9/site-packages/nbconvert/__init__.py", line 4, in 
    from .exporters import *
  File "/usr/local/lib/python3.9/site-packages/nbconvert/exporters/__init__.py", line 1, in 
    from .base import (export, get_exporter, 
  File "/usr/local/lib/python3.9/site-packages/nbconvert/exporters/base.py", line 14, in 
    from ipython_genutils.py3compat import string_types
ModuleNotFoundError: No module named 'ipython_genutils'

Potentially related to jupyter/nbconvert#1727 and its linked issue?

[kevin-bates]

yeah, looks like we should bump nbconvert. Thanks for catching this!

[kiersten-stokes]

Working like a charm!

[ptitzler]

Shouldn't the same fix be applied to Airflow processing?