Centos7 Python 3 Compatibility

[carlmontanari]

SUMMARY

Fresh install on Centos7 fails the python3_test pack

ISSUE TYPE

• Bug Report

STACKSTORM VERSION

st2 3.0.1, on Python 2.7.5

OS / ENVIRONMENT / INSTALL METHOD

OS: CentOS Linux release 7.6.1810 (Core)
Install Method: one-line installer
python3_test pack installed: st2 pack install python3_test --python3

STEPS TO REPRODUCE

st2 run python3_test.test_stdlib_import

EXPECTED RESULTS

.
id: 5cf7ef3f6cb8de144d02d91a
status: succeeded
parameters: None
result:
  exit_code: 0
  result: imports work correctly
  stderr: ''
  stdout: "Using Python binary: /opt/stackstorm/virtualenvs/python3_test/bin/python
Using Python version: 3.6.8 (default, Apr 25 2019, 21:02:35)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]
/usr/lib64/python3.6/concurrent/__init__.py
"

ACTUAL RESULTS

.
id: 5cf7eb8d6cb8de144d02d908
status: failed
parameters: None
result:
  exit_code: 1
  result: None
  stderr: "Traceback (most recent call last):
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/python_runner/python_action_wrapper.py", line 334, in <module>
    obj.run()
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/python_runner/python_action_wrapper.py", line 193, in run
    output = action.run(**self._parameters)
  File "/opt/stackstorm/packs/python3_test/actions/test_stdlib_import.py", line 21, in run
    from cassandra.cluster import Cluster  # NOQA
  File "cassandra/cluster.py", line 23, in init cassandra.cluster
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/concurrent/futures/__init__.py", line 8, in <module>
    from concurrent.futures._base import (FIRST_COMPLETED,
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/concurrent/futures/_base.py", line 414
    raise exception_type, self._exception, self._traceback
                        ^
SyntaxError: invalid syntax
"
  stdout: "Using Python binary: /opt/stackstorm/virtualenvs/python3_test/bin/python
Using Python version: 3.6.8 (default, Apr 25 2019, 21:02:35)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]

Fix (hacky as it is)

Insert /usr/lib64/python3.6 at the top of any python3 action:
sys.path.insert(0, '/usr/lib64/python3.6/')

Edit: also in the python 3 test pack to get it to succeed (also hacky):
if not any(path in concurrent.__file__ for path in ['lib/python3', 'lib64/python3']):

Related PRs

#4674
#4666

[arm4b]

Thanks for the report 👍 confirming the bug.

[arm4b]

@Kami Looks like we skip that test in our e2e, because we don't install additional python36 for some reason during CentOS 7 VM bootstrap.

Test:
https://github.com/StackStorm/st2tests/blob/a3a7b8c58e0b1ac66912beaa7378339c70468e49/cli/test_pack_python3.bats#L20-L32

Result:

ok 28 SETUP: Install and register examples pack # skip Python 3 binary not found, skipping tests | setup: 0s, test: 0s, teardown: 0s
ok 29 packs.setup_virtualenv without python3 flags works and defaults to Python 2 # skip Python 3 binary not found, skipping tests | setup: 0s, test: 1s, teardown: 0s
ok 30 packs.setup_virtualenv with python3 flag works # skip Python 3 binary not found, skipping tests | setup: 0s, test: 0s, teardown: 0s
ok 31 python3 imports work correctly # skip Python 3 binary not found, skipping tests | setup: 0s, test: 0s, teardown: 0s
ok 32 TEARDOWN: Uninstall examples and python3_test pack # skip Python 3 binary not found, skipping tests | setup: 0s, test: 1s, teardown: 0s"

was that intentionally or we just forgot about it?

[bhanukumar22]

Will I be able to install stackstorm 3.2 in centos7 with python3?
Python2, I was able to install. But see some vulnerabilities in python2.7.5. So, need to go with python3.

[nmaludy]

@bhanukumar22 We have recently release StackStorm 3.2 with support for RHEL 8 using Python 3.6. Due to limited bandwidth we have no plans of supporting Python 3.x on RHEL 7, if you need Python 3.x please switch over to RHEL 8.

Thanks!