ESMPy pip install and test update

[rsdunlapiv]

No description provided.

[rsdunlapiv]

@rokuingh the docs on this branch state that this is the test procedure:

The ESMPy testing is done with the nose package, both in serial and
parallel.  The nose commands are wrapped in the following ESMPy targets:
.. code::
    python setup.py test
    python setup.py test_examples
    python setup.py test_regrid_from_file
    python setup.py test_parallel
    python setup.py test_examples_parallel
    python setup.py test_regrid_from_file_parallel

Is that still correct, or does this need to be updated?

[rsdunlapiv]

@rokuingh the ESMPy doc builds on CircleCI are failing like this:

Removing intermediate container ec75a8bf94fa
Step 28/29 : RUN make html latexpdf 2>&1 | tee ${ARTIFACTS}/esmpy-make-doc.out
 ---> Running in 1e6885dc6ff4
sphinx-build -b html -d esmpy_doc/doctrees   . esmpy_doc/html
Running Sphinx v5.1.1

Configuration error:
There is a programmable error in your configuration file:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/sphinx/config.py", line 347, in eval_config_file
    exec(code, namespace)
  File "/opt/****/src/addon/ESMPy/doc/conf.py", line 21, in <module>
    import ESMF
  File "/opt/****/src/addon/ESMPy/src/ESMF/__init__.py", line 69, in <module>
    from ESMF.api.esmpymanager import *
  File "/opt/****/src/addon/ESMPy/src/ESMF/api/esmpymanager.py", line 11, in <module>
    from ESMF.interface.cbindings import *
  File "/opt/****/src/addon/ESMPy/src/ESMF/interface/cbindings.py", line 13, in <module>
    from ESMF.interface.loadESMF import _ESMF
  File "/opt/****/src/addon/ESMPy/src/ESMF/interface/loadESMF.py", line 38, in <module>
    with open(****mkfile_local, 'w') as emfl:
FileNotFoundError: [Errno 2] No such file or directory: '/opt/****/src/addon/ESMPy/doc/src/ESMF/interface/****mkfile.py'

make: *** [Makefile:45: html] Error 2

See here.

How should the doc build procedure be changed?

Note that if you want to test the ESMPy doc build container locally, you can use these instructions:
https://github.com/esmf-org/esmf/wiki/Documentation#build-esmpy-docs-locally

[rokuingh]

This is still the correct test procedure (using nose) up through Python 3.9. For Python 3.10 we need to migrate to nose2 or something else like pytest, which I haven't finished yet. The installation at Python 3.10 is not yet finished either.

[rsdunlapiv]

I just had successful pip install and nose2 tests on Python 3.9.5. Did you not expect the nose2 tests to work on Python 3.9? In general, I think we should try to move away from two different install/test procedures based on Python version because it adds a lot of complexity to manage. Given that, what minimum version of Python do you recommend and will that use pip install and nose2?

[rokuingh]

Yes pip and nose2 should work for 3.9, and probably all the way back through the 3 series. The issue we have with nose2 is that it doesn't have the same features for separating and running tests in groups like examples, slow, data, parallel etc.

[rsdunlapiv]

@rokuingh there must be some workaround to this issue, such as defining the tests in different directories or specifying specific tests when we run nose2, etc. Can you propose a workaround?

[rsdunlapiv]

@rokuingh your latest commit that added setup.py back causes install issues on Cheyenne:

(venv) dunlap@cheyenne4:/glade/scratch/dunlap/esmf-testing/gfortran_10.1.0_openmpi_O_esmpy-feature-pip/esmf/src/addon/ESMPy> pip install .
+ pip install .
Processing /glade/scratch/dunlap/esmf-testing/gfortran_10.1.0_openmpi_O_esmpy-feature-pip/esmf/src/addon/ESMPy
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Collecting numpy
  Using cached numpy-1.21.6-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (15.7 MB)
Collecting nose2
  Using cached nose2-0.12.0-py2.py3-none-any.whl (152 kB)
Building wheels for collected packages: ESMPy
  Building wheel for ESMPy (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for ESMPy (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [43 lines of output]
      running bdist_wheel
      /glade/u/home/dunlap/.local/lib/python3.7/site-packages
      running build
      Traceback (most recent call last):
        File "/glade/scratch/dunlap/esmf-testing/gfortran_10.1.0_openmpi_O_esmpy-feature-pip/esmf/src/addon/ESMPy/venv/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
          main()
        File "/glade/scratch/dunlap/esmf-testing/gfortran_10.1.0_openmpi_O_esmpy-feature-pip/esmf/src/addon/ESMPy/venv/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/glade/scratch/dunlap/esmf-testing/gfortran_10.1.0_openmpi_O_esmpy-feature-pip/esmf/src/addon/ESMPy/venv/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 262, in build_wheel
          metadata_directory)
        File "/glade/scratch/dunlap/pip-build-env-0mfrx37w/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 413, in build_wheel
          wheel_directory, config_settings)
        File "/glade/scratch/dunlap/pip-build-env-0mfrx37w/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 397, in _build_with_temp_dir
          self.run_setup()
        File "/glade/scratch/dunlap/pip-build-env-0mfrx37w/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 335, in run_setup
          exec(code, locals())
        File "<string>", line 278, in <module>
        File "/glade/scratch/dunlap/pip-build-env-0mfrx37w/overlay/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
        File "/glade/scratch/dunlap/pip-build-env-0mfrx37w/overlay/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/glade/scratch/dunlap/pip-build-env-0mfrx37w/overlay/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 973, in run_commands
          self.run_command(cmd)
        File "/glade/scratch/dunlap/pip-build-env-0mfrx37w/overlay/lib/python3.7/site-packages/setuptools/dist.py", line 1217, in run_command
          super().run_command(command)
        File "/glade/scratch/dunlap/pip-build-env-0mfrx37w/overlay/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
          cmd_obj.run()
        File "/glade/scratch/dunlap/pip-build-env-0mfrx37w/normal/lib/python3.7/site-packages/wheel/bdist_wheel.py", line 299, in run
          self.run_command('build')
        File "/glade/scratch/dunlap/pip-build-env-0mfrx37w/overlay/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
          self.distribution.run_command(command)
        File "/glade/scratch/dunlap/pip-build-env-0mfrx37w/overlay/lib/python3.7/site-packages/setuptools/dist.py", line 1217, in run_command
          super().run_command(command)
        File "/glade/scratch/dunlap/pip-build-env-0mfrx37w/overlay/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
          cmd_obj.run()
        File "<string>", line 112, in run
        File "src/ESMF/__init__.py", line 69, in <module>
          from ESMF.api.esmpymanager import *
        File "src/ESMF/api/esmpymanager.py", line 9, in <module>
          from ESMF.api.constants import *
        File "src/ESMF/api/constants.py", line 3, in <module>
          import numpy as np
      ModuleNotFoundError: No module named 'numpy'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for ESMPy
Failed to build ESMPy
ERROR: Could not build wheels for ESMPy, which is required to install pyproject.toml-based projects