[CFME][Automation][Python_Quickstart_script] – When trying to run the python ‘quickstart’ script using python3, the script fails

[ikanias]

Summary: [CFME][Automation][Python_Quickstart_script] – When trying to run the python ‘quickstart’ script using python3, the script fails

Bug severity: Critical

Description:

1. Make sure you have Python 3.7 installed on your machine

2. Configure the automation environment according to the following steps:

   • mkdir cfme
   • cd cfme
   • git clone https://github.com/ManageIQ/integration_tests/
   • cd integration_tests/conf/
   • echo >.yaml_key
   • python3 -m cfme.scripting.quickstart

Expected Result:
The script will function properly and a new /cfme_venv folder will be created alongside the ‘integration_tests’ and the ‘cfme-qe-yamls’ folders. In addition the automation environment will be configured successfully.

Actual Result:
When running the ‘python3 -m cfme.scripting.quickstart’ command there is an error message saying that the ‘Feature’ name could not be imported from 'setuptools' and the script fails.

The error received is:

QS $ yum install -y python2-virtualenv gcc postgresql-devel libxml2-devel libxslt-devel zeromq3-devel libcurl-devel redhat-rpm-config gcc-c++ openssl-devel libffi-devel python-devel tesseract freetype-devel    yum-utils
QS $ debuginfo-install -y python2 python3
INFO: Virtualenv .cfme_venv already exists, skipping creation
QS $ .cfme_venv/bin/pip install -r requirements/frozen.py3.txt --no-binary pycurl --no-binary azure-cosmosdb-table -q # this may take some time to finish ...
    ERROR: Command errored out with exit status 1:
     command: /root/cfme/integration_tests/.cfme_venv/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-sxg3vblj/markupsafe/setup.py'"'"'; __file__='"'"'/tmp/pip-install-sxg3vblj/markupsafe/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-b2rlyb7h
         cwd: /tmp/pip-install-sxg3vblj/markupsafe/
    Complete output (5 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-sxg3vblj/markupsafe/setup.py", line 6, in <module>
        from setuptools import setup, Extension, Feature
    ImportError: cannot import name 'Feature' from 'setuptools' (/root/cfme/integration_tests/.cfme_venv/lib/python3.7/site-packages/setuptools/__init__.py)
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
WARNING: You are using pip version 20.2.2; however, version 20.2.3 is available.
You should consider upgrading via the '/root/cfme/integration_tests/.cfme_venv/bin/python3 -m pip install --upgrade pip' command.
Running command failed!
CalledProcessError(1, ['.cfme_venv/bin/pip', 'install', '-r', 'requirements/frozen.py3.txt', '--no-binary', 'pycurl', '--no-binary', 'azure-cosmosdb-table', '-q'])

[mshriver]

From the output it looks like this is on a re-run of quickstart, as the venv already existed. What OS and version are you running on? Quickstart's first logged line is the identified OS and version, which I don't see in your output

[ikanias]

Hello Mike and thank you for your reply. I am using RHEL 7.5 and CFME 5.11.8. Regards, Itsik

…

On Wed, Oct 7, 2020, 17:23 Mike Shriver ***@***.***> wrote: From the output it looks like this is on a re-run of quickstart, as the venv already existed. What OS and version are you running on? Quickstart's first logged line is the identified OS and version, which I don't see in your output — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#10335 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQUQ6XH2UX6YIDXTAEDKPYDSJR2WZANCNFSM4R6GFXOA> .

[jawatts]

Can you try removing cfme_venv and rerunning again. Please paste the entire output

[mshriver]

trying to recreate - btw RHEL 7.5 was EOL back in April, might be time for an upgrade.

[ikanias]

Hi, There is no cfme_venv folder. That's the strange thing... [image: image.png]

…

On Wed, Oct 7, 2020 at 8:58 PM Mike Shriver ***@***.***> wrote: trying to recreate - btw RHEL 7.5 was EOL back in April, might be time for an upgrade. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#10335 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQUQ6XEJ3TV5YIOLVDMK5X3SJST2PANCNFSM4R6GFXOA> .

[andyvesel]

@ikanias it looks like your development env hasn't been activated.
Activate the development environment by . ../cfme_venv/bin/activate
Check this out ^

[ikanias]

I cannot activate it because the script 'python3 -m cfme.scripting.quickstart' that should create the environment is not working (That's what this issue was opened for).

[andyvesel]

I tried to make steps from https://cfme-tests.readthedocs.io/en/master/getting_started.html, and run quickstart. Even though I haven't run further steps (selenium server, etc), the script itself worked without errors.

~/c/integration_tests (master|…) $ python -m cfme.scripting.quickstart                                                                                        Thu 08 Oct 2020 11:21:50 AM CEST
OS_NAME: Fedora, OS_VERSION: 31
QS $ sudo dnf install -y gcc postgresql-devel libxml2-devel libxslt-devel zeromq-devel libcurl-devel redhat-rpm-config gcc-c++ openssl-devel libffi-devel python3 python3-pip python3-devel tesseract freetype-devel  git 'dnf-command(debuginfo-install)'
[sudo] password for aveselov: 
INFO: Virtualenv .cfme_venv already exists, skipping creation
QS $ .cfme_venv/bin/pip3 install -U pip wheel setuptools setuptools_scm docutils pbr
INFO: skipping requirement installation as frozen ones didn't change
      to enforce please invoke pip manually
QS $ .cfme_venv/bin/python3 -m cfme.scripting.disable_bytecode
QS $ .cfme_venv/bin/pip3 install -q -e .
QS $ .cfme_venv/bin/python3 -m cfme.scripting.link_config ../cfme-qe-yamls/complete/ conf
QS $ .cfme_venv/bin/python3 -c 'import curl'
INFO: please remember to activate the virtualenv via
      . .cfme_venv/bin/activate

Please make sure that you have made touch conf/yaml.key and cp conf/env.yaml.template conf/env.yaml before running the script

[ikanias]

Try to run it with Python3 not Python. The problem happens when running the script with 'python3' at the beginning of the script instead of 'python'. Please make sure you have Python 3.7 installed. Regards, Itsik

…

On Thu, Oct 8, 2020, 14:45 Andrei Veselov ***@***.***> wrote: I tried to make steps from https://cfme-tests.readthedocs.io/en/master/getting_started.html, and quickstart. Even though I haven't run further steps (selenium server, etc), the script itself worked without errors. ~/c/integration_tests (master|…) $ python -m cfme.scripting.quickstart Thu 08 Oct 2020 11:21:50 AM CEST OS_NAME: Fedora, OS_VERSION: 31 QS $ sudo dnf install -y gcc postgresql-devel libxml2-devel libxslt-devel zeromq-devel libcurl-devel redhat-rpm-config gcc-c++ openssl-devel libffi-devel python3 python3-pip python3-devel tesseract freetype-devel git 'dnf-command(debuginfo-install)' [sudo] password for aveselov: INFO: Virtualenv .cfme_venv already exists, skipping creation QS $ .cfme_venv/bin/pip3 install -U pip wheel setuptools setuptools_scm docutils pbr INFO: skipping requirement installation as frozen ones didn't change to enforce please invoke pip manually QS $ .cfme_venv/bin/python3 -m cfme.scripting.disable_bytecode QS $ .cfme_venv/bin/pip3 install -q -e . QS $ .cfme_venv/bin/python3 -m cfme.scripting.link_config ../cfme-qe-yamls/complete/ conf QS $ .cfme_venv/bin/python3 -c 'import curl' INFO: please remember to activate the virtualenv via . .cfme_venv/bin/activate Please make sure that you have made touch conf/yaml.key and cp conf/env.yaml.template conf/env.yaml before running the script — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#10335 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQUQ6XHYPUFXRRBVOVYIVNDSJWQ6DANCNFSM4R6GFXOA> .

[ikanias]

Re-installed my BM with RHEL 7.9 and CFME 5.11.9.0 and Python 3.7.9.
Then needed to install docker too. After all of that ran /usr/src/Python-3.7.9/python -m cfme.scripting.quickstart and it worked.
Ran '. ../cfme_venv/bin/activate' then 'docker pull cfmeqe/sel_ff_chrome' and then ran 'miq selenium-container'.
Then I ran 'miq-runtest --collect-only' and got errors probably because no cloning of 'https://gitlab.cee.redhat.com/cfme-qe/cfme-qe-yamls/' was done. Currently I have a certificate issue - working on it.