[BUG] testing within vscode

[wutaomsft]

Description

Currently using vscode to discover test failed on the repository.

In which platform does it happen?

Tested on VSCode + ssh into DSVM, but should be reproducibe on all plaforms.

How do we replicate the issue?

In VSCode, click "Discover Tests" in the bottom bar, it would result in "test discovery failed". I have experienced two kind of errors by checking Python Test Log window.

1. The first kind of errors is something like
   Test Discovery failed:
   Error: ImportError while loading conftest '/home/azureuser/src/recommenders/tests/conftest.py'.
   conftest.py:22: in
   from reco_utils.common.constants import (
   E ModuleNotFoundError: No module named 'reco_utils'
   Which can be addressed in where could I get the 'reco_utils' that imported in the code? #1065.

2. The second kind of errors is something like
   import file mismatch:
   imported module 'test_notebooks_gpu' has this file attribute:
   /home/azureuser/src/recommenders/tests/integration/examples/test_notebooks_gpu.py
   which is not the same as the test file we want to collect:
   /home/azureuser/src/recommenders/tests/unit/examples/test_notebooks_gpu.py
   HINT: remove pycache / .pyc files and/or use a unique basename for your test file modules
   ________ ERROR collecting tests/unit/examples/test_notebooks_pyspark.py ________

According to https://stackoverflow.com/questions/12582503/test-discovery-failure-when-tests-in-different-directories-are-called-the-same, init.py are needed to different subirs. Will do a PR on this.

Expected behavior (i.e. solution)

Other Comments

[miguelgfierro]

yeah, this happens because VSCode is taking the tests as if they were executed from the root, however, we never run it that way because we have 3 different environments, among other things. We will do something like: pytest tests/unit -m "not notebooks and not spark and not gpu" --durations 0 and pytest tests/smoke -m "smoke and spark and not gpu" --durations 0, etc

[laserprec]

Adding onto @miguelgfierro, this test discovery issue can be resolved if we add __init__.py to all of the subfolders under tests. This due to how pytests modify python sys.path to inject the test files without explicitly packaging them. From the pytest documentation:

prepend import mode (which is the default): your test files must have unique names, because pytest will import them as top-level modules since there are no packages to derive a full package name from...

If you need to have test modules with the same name, you might add init.py files to your tests folder and subfolders, changing them to packages:

[laserprec]

This issue will be address in an upcoming PR.

[laserprec]

Issue should be resolved by PR #1494