Virtualenv created includes copy of python exectuable

[yrro]

Be sure to check the existing issues (both open and closed!).

The virtualev created by pipenv includes a copy of the python executable rather than a symlink. This means that I'll end up running out-of-date copies of the python interpreter, which will not include security updates; and since the virtualenv does include symlinks to the standard library, the two will diverge and I'll end up with an unsupported combination of old python interpreter and newer standard library.

$ python -m pipenv.help output

Pipenv version: '11.10.1'

Pipenv location: '/home/sam/.local/lib/python3.6/site-packages/pipenv'

Python location: '/usr/bin/python3'

Other Python installations in PATH:

• 2.7: /usr/bin/python2.7

• 2.7: /usr/bin/python2.7

• 3.6: /usr/bin/python3.6m

• 3.6: /usr/bin/python3.6

• 2.7.15: /usr/bin/python

• 2.7.15: /usr/bin/python2

• 3.6.5: /usr/bin/python3

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.6.5c1',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '4.15.0-3-amd64',
 'platform_system': 'Linux',
 'platform_version': '#1 SMP Debian 4.15.17-1 (2018-04-19)',
 'python_full_version': '3.6.5rc1',
 'python_version': '3.6',
 'sys_platform': 'linux'}

System environment variables:

• LS_COLORS
• LESSCLOSE
• XDG_MENU_PREFIX
• LANG
• GDM_LANG
• MALLOC_PERTURB_
• DISPLAY
• EDITOR
• COLORTERM
• DESKTOP_AUTOSTART_ID
• USERNAME
• XDG_VTNR
• SSH_AUTH_SOCK
• S_COLORS
• LESS_TERMCAP_so
• LESS_TERMCAP_se
• XDG_SESSION_ID
• USER
• DESKTOP_SESSION
• GNOME_TERMINAL_SCREEN
• PWD
• HOME
• SSH_AGENT_PID
• QT_ACCESSIBILITY
• XDG_SESSION_TYPE
• https_proxy
• XDG_DATA_DIRS
• http_proxy
• XDG_SESSION_DESKTOP
• LESS_TERMCAP_mb
• LESS_TERMCAP_md
• LESS_TERMCAP_me
• ENV_TEST_SAM
• no_proxy
• ANSIBLE_NOCOWS
• GTK_MODULES
• NO_PROXY
• VISUAL
• WINDOWPATH
• TERM
• SHELL
• VTE_VERSION
• DEBNAME
• XDG_CURRENT_DESKTOP
• GPG_AGENT_INFO
• GNOME_TERMINAL_SERVICE
• PYTHONSTARTUP
• COLORFGBG
• SHLVL
• XDG_SEAT
• DEBEMAIL
• GDMSESSION
• GNOME_DESKTOP_SESSION_ID
• LOGNAME
• DBUS_SESSION_BUS_ADDRESS
• XDG_RUNTIME_DIR
• XAUTHORITY
• LESS_TERMCAP_ue
• LESS_TERMCAP_us
• ANSIBLE_COW_SELECTION
• PATH
• SESSION_MANAGER
• LESSOPEN
• _
• OLDPWD
• PYTHONDONTWRITEBYTECODE
• PIP_PYTHON_PATH

Pipenv–specific environment variables:

Debug–specific environment variables:

• PATH: /home/sam/bin:/home/sam/.local/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
• SHELL: /bin/bash
• EDITOR: vim
• LANG: en_GB.UTF-8
• PWD: /home/sam/src/nexsan-exporter/nexsan-exporter

---

---

Expected result

A venv that uses symlinks as created by the standard python python3 -m venv /tmp/foo:

$ ls -l /tmp/foo/bin/python*
lrwxrwxrwx 1 sam sam  7 May  1 10:44 /tmp/foo/bin/python -> python3
lrwxrwxrwx 1 sam sam 16 May  1 10:44 /tmp/foo/bin/python3 -> /usr/bin/python3

Actual result

A venv that includes a copy of the python executable:

$ ls -l ~/.local/share/virtualenvs/nexsan-exporter-1-atX-z8/bin/python*
lrwxrwxrwx 1 sam sam       7 May  1 10:46 /home/sam/.local/share/virtualenvs/nexsan-exporter-1-atX-z8/bin/python -> python3
-rwxr-xr-x 1 sam sam 4576440 May  1 10:46 /home/sam/.local/share/virtualenvs/nexsan-exporter-1-atX-z8/bin/python3
lrwxrwxrwx 1 sam sam       7 May  1 10:46 /home/sam/.local/share/virtualenvs/nexsan-exporter-1-atX-z8/bin/python3.6 -> python3
-rwxr-xr-x 1 sam sam    2377 May  1 10:46 /home/sam/.local/share/virtualenvs/nexsan-exporter-1-atX-z8/bin/python-config

Steps to replicate

$ python3 -m pipenv install --verbose
Creating a virtualenv for this project…
Using /usr/bin/python3 (3.6.5) to create virtualenv…
⠋Already using interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in /home/sam/.local/share/virtualenvs/nexsan-exporter-1-atX-z8/bin/python3
Also creating executable in /home/sam/.local/share/virtualenvs/nexsan-exporter-1-atX-z8/bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: /home/sam/.local/share/virtualenvs/nexsan-exporter-1-atX-z8
Creating a Pipfile for this project…
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Using pip: -i https://pypi.org/simple

                          ROUND 1                           
Current constraints:

Finding the best candidates:

Finding secondary dependencies:
------------------------------------------------------------
Result of round 1: stable, done

Locking [packages] dependencies…
Using pip: -i https://pypi.org/simple

                          ROUND 1                           
Current constraints:

Finding the best candidates:

Finding secondary dependencies:
------------------------------------------------------------
Result of round 1: stable, done

Updated Pipfile.lock (ca72e7)!
Installing dependencies from Pipfile.lock (ca72e7)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 0/0 — 00:00:00
To activate this project's virtualenv, run the following:
 $ pipenv shell

[techalchemy]

The venv library is deprecated and unusable for cross platform package. Virtualenv is perfectly fine, and pipenv virtualenvs are specifically designed as reproducible throwaways. Basically this isn’t a bug

[yrro]

venv got this right back in Python 3.3 and was a useful addition to the standard library. I don't think it has been deprecated. The closest thing I see in the documentation is a statement that the pyvenv script has been deprecated in favour of running python3 -m venv; but regardless...

It is absolutely a bug that you end up with a copy of the python interpreter in the virtual environment. Please reconsider this, as having to deal with random unpatched copies of the python interpreter all over my systems is insane. Doubly so mixing a static copy of the python interpreter with a symlinked python standard library that will change over time as the Python installation is updated... that's not even internally consistent!

[uranusjr]

@yrro virtualenv is not the same as venv. While I agree that venv indeed does this more correctly, the reason is not that it allows you to swap out the linked Python underneath—that is always unrecommended, even with venv.

In any case, Pipenv uses virtualenv instead of venv for an obvious reason: It needs to support the creation of Python 2 virtual environments. It is possible to switch between virtualenv and venv when creating environments of different Python versions, but it would increase both the implementation and maintenance overhead significantly, and it is only reasonable to stick with only one backend with limited resources available.

If you really, truly think Pipenv should use venv, and link-based Python executable creation, here are your options:

1. Work on a patch to implement this in virtualenv (again, not venv), and submit a patch.
2. Work on adding venv support to Pipenv, so it can leverage its “correct” implementation.

There has been much interest in venv support in Pipenv, and virtualenv is still looking for a maintainer as far as I know. Lend a hand if you are interested in actually fixing this bug, instead of trying to persuade people this is a bug.

[yrro]

I really don't mind whether virtualenv or venv is used. It certainly makes sense for you to focus on just using virtualenv until you want to drop support for Python < 3.3. Until then, I am having a go at making virtualenv support symlinking the Python interpreter; comments welcome on pypa/virtualenv#1171.