fixing anaconda link in dockerfile by gramhagen · Pull Request #1120 · recommenders-team/recommenders

gramhagen · 2020-06-09T18:49:42Z

Description

Fix anaconda link in Dockerfile

Related Issues

#1116

Checklist:

I have followed the contribution guidelines and code style for this project.
I have added tests covering my contributions.
I have updated the documentation accordingly.
This PR is being made to staging and not master.

r-yanyo · 2020-06-10T06:21:51Z

docker/Dockerfile

 COPY --from=base ${HOME} .

 # Setup Conda environment
-ENV PATH="${HOME}/conda/bin:${PATH}"


This change happend python: not found error when I built full environment.

Oops. We need this line after since I'm pulling from Nvidia base

miguelgfierro

LGTM

r-yanyo · 2020-06-10T13:13:12Z

This is not about #1120, but docker build still failed at the following problem.
This may be related to aksnzhy/xlearn#211 .

I faced this problem before when I built environment without docker.
I installed cmake and, I fixed this error.

It seems to install cmake, but I don't know why this error happend.
https://github.com/microsoft/recommenders/blob/master/docker/Dockerfile#L20

If this problem should not be handled in this PR, Please ignore.

#22 378.0 Failed to build xlearn
#22 378.0 Installing collected packages: applicationinsights, azureml-telemetry, ruamel.yaml.clib, ruamel.yaml, azure-common, oauthlib, requests-oauthlib, isodate, msrest, PyJWT, adal, msrestazure, azure-mgmt-storage, jeepney, SecretStorage, contextlib2, websocket-client, docker, jmespath, pathspec, azure-graphrbac, azure-mgmt-keyvault, azure-mgmt-authorization, azure-mgmt-containerregistry, jsonpickle, azure-mgmt-resource, pyasn1, ndg-httpsclient, azureml-core, azureml-train-restclients-hyperdrive, azureml-train-core, azureml-train, azureml-dataprep-native, distro, dotnetcore2, fusepy, azureml-dataprep, azureml-pipeline-core, azureml-pipeline-steps, azureml-pipeline, azureml-widgets, azureml-contrib-notebook, azureml-tensorboard, azureml-sdk, azure-core, azure-storage-blob, portalocker, azure-nspkg, azure-cli-nspkg, azure-cli-telemetry, humanfriendly, msal, pkginfo, argcomplete, tabulate, colorama, knack, bcrypt, pynacl, paramiko, msal-extensions, azure-mgmt-core, azure-cli-core, azure-mgmt-cosmosdb, toml, regex, appdirs, typed-ast, black, patsy, statsmodels, category-encoders, pymongo, networkx, hyperopt, idna, zope.interface, zope.event, greenlet, gevent, Werkzeug, itsdangerous, flask, locustio, memory-profiler, nbconvert, pydocumentdb, pymanopt, xlearn, sentencepiece, sacremoses, filelock, docutils, botocore, s3transfer, boto3, tokenizers, transformers, tensorflow-estimator, h5py, keras-applications, termcolor, wrapt, protobuf, google-pasta, markdown, grpcio, absl-py, tensorboard, astor, opt-einsum, gast, keras-preprocessing, tensorflow-gpu, configparser, databricks-cli, nvidia-ml-py3, coverage, json-tricks, simplejson, PythonWebHDFS, schema, nni
#22 378.0   Attempting uninstall: idna
#22 378.0     Found existing installation: idna 2.9
#22 378.0     Uninstalling idna-2.9:
#22 378.0       Successfully uninstalled idna-2.9
#22 378.0   Attempting uninstall: nbconvert
#22 378.0     Found existing installation: nbconvert 5.6.1
#22 378.0     Uninstalling nbconvert-5.6.1:
#22 378.0       Successfully uninstalled nbconvert-5.6.1
#22 378.0     Running setup.py install for xlearn: started
#22 378.0     Running setup.py install for xlearn: finished with status 'error'
#22 378.0 Pip subprocess error:
#22 378.0   ERROR: Command errored out with exit status 1:
#22 378.0    command: /root/conda/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-uucssjaq/xlearn/setup.py'"'"'; __file__='"'"'/tmp/pip-install-uucssjaq/xlearn/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-1w__yva7
#22 378.0        cwd: /tmp/pip-install-uucssjaq/xlearn/
#22 378.0   Complete output (51 lines):
#22 378.0   /root/conda/lib/python3.6/site-packages/setuptools/dist.py:454: UserWarning: Normalizing '0.40.a1' to '0.40a1'
#22 378.0     warnings.warn(tmpl.format(**locals()))
#22 378.0   running bdist_wheel
#22 378.0   running build
#22 378.0   running build_py
#22 378.0   Traceback (most recent call last):
#22 378.0     File "/tmp/pip-install-uucssjaq/xlearn/setup.py", line 37, in silent_call
#22 378.0       subprocess.check_output(cmd, stderr=shut_up)
#22 378.0     File "/root/conda/lib/python3.6/subprocess.py", line 356, in check_output
#22 378.0       **kwargs).stdout
#22 378.0     File "/root/conda/lib/python3.6/subprocess.py", line 423, in run
#22 378.0       with Popen(*popenargs, **kwargs) as process:
#22 378.0     File "/root/conda/lib/python3.6/subprocess.py", line 729, in __init__
#22 378.0       restore_signals, start_new_session)
#22 378.0     File "/root/conda/lib/python3.6/subprocess.py", line 1364, in _execute_child
#22 378.0       raise child_exception_type(errno_num, err_msg, err_filename)
#22 378.0   FileNotFoundError: [Errno 2] No such file or directory: 'cmake': 'cmake'
#22 378.0
#22 378.0   During handling of the above exception, another exception occurred:
#22 378.0
#22 378.0   Traceback (most recent call last):
#22 378.0     File "<string>", line 1, in <module>
#22 378.0     File "/tmp/pip-install-uucssjaq/xlearn/setup.py", line 126, in <module>
#22 378.0       url='https://github.com/aksnzhy/xlearn')
#22 378.0     File "/root/conda/lib/python3.6/site-packages/setuptools/__init__.py", line 161, in setup
#22 378.0       return distutils.core.setup(**attrs)
#22 378.0     File "/root/conda/lib/python3.6/distutils/core.py", line 148, in setup
#22 378.0       dist.run_commands()
#22 378.0     File "/root/conda/lib/python3.6/distutils/dist.py", line 955, in run_commands
#22 378.0       self.run_command(cmd)
#22 378.0     File "/root/conda/lib/python3.6/distutils/dist.py", line 974, in run_command
#22 378.0       cmd_obj.run()
#22 378.0     File "/root/conda/lib/python3.6/site-packages/wheel/bdist_wheel.py", line 223, in run
#22 378.0       self.run_command('build')
#22 378.0     File "/root/conda/lib/python3.6/distutils/cmd.py", line 313, in run_command
#22 378.0       self.distribution.run_command(command)
#22 378.0     File "/root/conda/lib/python3.6/distutils/dist.py", line 974, in run_command
#22 378.0       cmd_obj.run()
#22 378.0     File "/root/conda/lib/python3.6/distutils/command/build.py", line 135, in run
#22 378.0       self.run_command(cmd_name)
#22 378.0     File "/root/conda/lib/python3.6/distutils/cmd.py", line 313, in run_command
#22 378.0       self.distribution.run_command(command)
#22 378.0     File "/root/conda/lib/python3.6/distutils/dist.py", line 974, in run_command
#22 378.0       cmd_obj.run()
#22 378.0     File "/tmp/pip-install-uucssjaq/xlearn/setup.py", line 104, in run
#22 378.0       compile_cpp()
#22 378.0     File "/tmp/pip-install-uucssjaq/xlearn/setup.py", line 75, in compile_cpp
#22 378.0       silent_call(cmake_cmd, raise_error=True, error_msg='Please install CMake first')
#22 378.0     File "/tmp/pip-install-uucssjaq/xlearn/setup.py", line 41, in silent_call
#22 378.0       raise Exception(error_msg);
#22 378.0   Exception: Please install CMake first
#22 378.0   ----------------------------------------
#22 378.0   ERROR: Failed building wheel for xlearn
#22 378.0 ERROR: azureml-core 1.0.69 has requirement ruamel.yaml<=0.15.89,>=0.15.35, but you'll have ruamel-yaml 0.16.10 which is incompatible.
#22 378.0 ERROR: nni 1.5 has requirement hyperopt==0.1.2, but you'll have hyperopt 0.1.1 which is incompatible.
#22 378.0 ERROR: nni 1.5 has requirement scikit-learn<0.22,>=0.20, but you'll have scikit-learn 0.22.1 which is incompatible.
#22 378.0     ERROR: Command errored out with exit status 1:
#22 378.0      command: /root/conda/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-uucssjaq/xlearn/setup.py'"'"'; __file__='"'"'/tmp/pip-install-uucssjaq/xlearn/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-_2lbp1d3/install-record.txt --single-version-externally-managed --compile --install-headers /root/conda/include/python3.6m/xlearn
#22 378.0          cwd: /tmp/pip-install-uucssjaq/xlearn/
#22 378.0     Complete output (55 lines):
#22 378.0     /root/conda/lib/python3.6/site-packages/setuptools/dist.py:454: UserWarning: Normalizing '0.40.a1' to '0.40a1'
#22 378.0       warnings.warn(tmpl.format(**locals()))
#22 378.0     running install
#22 378.0     running build
#22 378.0     running build_py
#22 378.0     Traceback (most recent call last):
#22 378.0       File "/tmp/pip-install-uucssjaq/xlearn/setup.py", line 37, in silent_call
#22 378.0         subprocess.check_output(cmd, stderr=shut_up)
#22 378.0       File "/root/conda/lib/python3.6/subprocess.py", line 356, in check_output
#22 378.0         **kwargs).stdout
#22 378.0       File "/root/conda/lib/python3.6/subprocess.py", line 423, in run
#22 378.0         with Popen(*popenargs, **kwargs) as process:
#22 378.0       File "/root/conda/lib/python3.6/subprocess.py", line 729, in __init__
#22 378.0         restore_signals, start_new_session)
#22 378.0       File "/root/conda/lib/python3.6/subprocess.py", line 1364, in _execute_child
#22 378.0         raise child_exception_type(errno_num, err_msg, err_filename)
#22 378.0     FileNotFoundError: [Errno 2] No such file or directory: 'cmake': 'cmake'
#22 378.0
#22 378.0     During handling of the above exception, another exception occurred:
#22 378.0
#22 378.0     Traceback (most recent call last):
#22 378.0       File "<string>", line 1, in <module>
#22 378.0       File "/tmp/pip-install-uucssjaq/xlearn/setup.py", line 126, in <module>
#22 378.0         url='https://github.com/aksnzhy/xlearn')
#22 378.0       File "/root/conda/lib/python3.6/site-packages/setuptools/__init__.py", line 161, in setup
#22 378.0         return distutils.core.setup(**attrs)
#22 378.0       File "/root/conda/lib/python3.6/distutils/core.py", line 148, in setup
#22 378.0         dist.run_commands()
#22 378.0       File "/root/conda/lib/python3.6/distutils/dist.py", line 955, in run_commands
#22 378.0         self.run_command(cmd)
#22 378.0       File "/root/conda/lib/python3.6/distutils/dist.py", line 974, in run_command
#22 378.0         cmd_obj.run()
#22 378.0       File "/tmp/pip-install-uucssjaq/xlearn/setup.py", line 93, in run
#22 378.0         install.run(self)
#22 378.0       File "/root/conda/lib/python3.6/site-packages/setuptools/command/install.py", line 61, in run
#22 378.0         return orig.install.run(self)
#22 378.0       File "/root/conda/lib/python3.6/distutils/command/install.py", line 545, in run
#22 378.0         self.run_command('build')
#22 378.0       File "/root/conda/lib/python3.6/distutils/cmd.py", line 313, in run_command
#22 378.0         self.distribution.run_command(command)
#22 378.0       File "/root/conda/lib/python3.6/distutils/dist.py", line 974, in run_command
#22 378.0         cmd_obj.run()
#22 378.0       File "/root/conda/lib/python3.6/distutils/command/build.py", line 135, in run
#22 378.0         self.run_command(cmd_name)
#22 378.0       File "/root/conda/lib/python3.6/distutils/cmd.py", line 313, in run_command
#22 378.0         self.distribution.run_command(command)
#22 378.0       File "/root/conda/lib/python3.6/distutils/dist.py", line 974, in run_command
#22 378.0         cmd_obj.run()
#22 378.0       File "/tmp/pip-install-uucssjaq/xlearn/setup.py", line 104, in run
#22 378.0         compile_cpp()
#22 378.0       File "/tmp/pip-install-uucssjaq/xlearn/setup.py", line 75, in compile_cpp
#22 378.0         silent_call(cmake_cmd, raise_error=True, error_msg='Please install CMake first')
#22 378.0       File "/tmp/pip-install-uucssjaq/xlearn/setup.py", line 41, in silent_call
#22 378.0         raise Exception(error_msg);
#22 378.0     Exception: Please install CMake first
#22 378.0     ----------------------------------------
#22 378.0 ERROR: Command errored out with exit status 1: /root/conda/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-uucssjaq/xlearn/setup.py'"'"'; __file__='"'"'/tmp/pip-install-uucssjaq/xlearn/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-_2lbp1d3/install-record.txt --single-version-externally-managed --compile --install-headers /root/conda/include/python3.6m/xlearn Check the logs for full command output.
#22 378.0
#22 378.0
#22 378.0 CondaEnvException: Pip failed
#22 378.0
#22 378.0
------
failed to solve with frontend dockerfile.v0: failed to build LLB: executor failed running [/bin/sh -c conda env update -f base.yaml &&     conda clean -fay &&     python -m ipykernel install --user --name 'python3' --display-name 'python3']: runc did not terminate sucessfully

gramhagen · 2020-06-10T15:01:42Z

hmm, odd. yeah. do you mind creating a separate issue for this?
tagging @yueguoguo he's more familiar with xlearn and cmake dependency. I think there was a previous issue around this?

fixing anaconda link in dockerfile

d8e69b1

gramhagen requested a review from yueguoguo June 9, 2020 18:49

gramhagen requested review from anargyri, loomlike and miguelgfierro as code owners June 9, 2020 18:49

r-yanyo reviewed Jun 10, 2020

View reviewed changes

miguelgfierro approved these changes Jun 10, 2020

View reviewed changes

Revert conda path removal

e39f518

r-yanyo mentioned this pull request Jun 11, 2020

[BUG] Docker build failed at 'Failed to build xlearn' #1121

Closed

anargyri approved these changes Jun 11, 2020

View reviewed changes

gramhagen merged commit c6093e3 into staging Jun 11, 2020

miguelgfierro deleted the scgraham/docker_fix branch June 11, 2020 10:22

gramhagen mentioned this pull request Jul 14, 2020

[ASK] Docker build failed at "RUN python" #1116

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fixing anaconda link in dockerfile#1120

fixing anaconda link in dockerfile#1120
gramhagen merged 2 commits intostagingfrom
scgraham/docker_fix

gramhagen commented Jun 9, 2020

Uh oh!

r-yanyo Jun 10, 2020

Uh oh!

gramhagen Jun 10, 2020

Uh oh!

miguelgfierro left a comment

Uh oh!

r-yanyo commented Jun 10, 2020 •

edited

Loading

Uh oh!

gramhagen commented Jun 10, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

gramhagen commented Jun 9, 2020

Description

Related Issues

Checklist:

Uh oh!

r-yanyo Jun 10, 2020

Choose a reason for hiding this comment

Uh oh!

gramhagen Jun 10, 2020

Choose a reason for hiding this comment

Uh oh!

miguelgfierro left a comment

Choose a reason for hiding this comment

Uh oh!

r-yanyo commented Jun 10, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gramhagen commented Jun 10, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

r-yanyo commented Jun 10, 2020 •

edited

Loading