Last week, I needed to work on an older project running on Python 3.4.2.
I use pyenv to build and manage my Pythons,
so I told it to build 3.4.2 - and quickly got an error:
pyenv build 3.4.2
ERROR: The Python ssl extension was not compiled. Missing the OpenSSL lib?
Please consult to the Wiki page to fix the problem.
https://github.com/pyenv/pyenv/wiki/Common-build-problems
BUILD FAILED (Debian 10 using python-build 1.2.18-19-gcf81e5a0)
Inspect or clean up the working tree at /tmp/python-build.20200920213941.14572
Results logged to /tmp/python-build.20200920213941.14572.log
Last 10 log lines:
(cd /home/USERNAME/.pyenv/versions/3.4.2/share/man/man1; ln -s python3.4.1 python3.1)
if test "xupgrade" != "xno" ; then \
case upgrade in \
upgrade) ensurepip="--upgrade" ;; \
install|*) ensurepip="" ;; \
esac; \
./python -E -m ensurepip \
$ensurepip --root=/ ; \
fi
Ignoring ensurepip failure: pip 1.5.6 requires SSL/TLS
This error was happening because Buster comes with OpenSSL 1.1, and Python 3 versions under 3.5.3 require version 1.0.
On older Debian releases, I'd have fixed this simply by installing the
libssl1.0-dev package. But the last Debian release providing that package was Debian 9
(Stretch) and it's gone from Buster:
sudo apt-get install libssl1.0-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
Package libssl1.0-dev is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
E: Package 'libssl1.0-dev' has no installation candidate
So, with the easy fix off the table, I decided to compile OpenSSL 1.0 and have pyenv install the vintage Python version against that.
Check out and compile OpenSSL 1.0
First, check out OpenSSL source and switch to the last release in the 1.0 series, 1.0.2u.
By the way, we'll only be compiling it to a folder - openssl/1.0.2 - not installing systemwide. So there's no reason or need to run any of the commands below as root or with sudo.
# make a folder for storing openssl source and compiled build
mkdir openssl
cd openssl
# clone openssl source code
git clone https://github.com/openssl/openssl source
cd source
# check out last 1.0.2 version, 1.0.2u
git checkout tags/OpenSSL_1_0_2u -b 1.0.2
# set build destination path
OPENSSL102_BIN_PATH=$(realpath ../../openssl/1.0.2)
# build 1.0.2 to local folder
./config shared --prefix=$OPENSSL102_BIN_PATH --openssldir=$OPENSSL102_BIN_PATH/openssl
make
make test
make install
echo "OpenSSL 1.0.2 is in $OPENSSL102_BIN_PATH"
The compilation will output a wall of text but the last line will be something like ...
OpenSSL 1.0.2 is in /home/USER/openssl/1.0.2
Make note of that path, we'll need it later.
Now that we have the right version of OpenSSL, we can move on to installing Python.
Compile Python 3.4.2 against OpenSSL 1.0
Before we try installing Python 3.4.2 again, we'll need to tell pyenv to build it using the OpenSSL we've just compiled, not the system version.
export LDFLAGS="-L$OPENSSL102_BIN_PATH/lib/"
export CPPFLAGS="-I$OPENSSL102_BIN_PATH/include -I$OPENSSL102_BIN_PATH/include/openssl"
export LD_LIBRARY_PATH="$OPENSSL102_BIN_PATH/lib/"
Then, build Python:
pyenv install 3.4.2
Installing Python-3.4.2...
patching file ./Lib/ssl.py
patching file ./Modules/_ssl.c
Installed Python-3.4.2 to /home/USERNAME/.pyenv/versions/3.4.2
... and you now have Python 3.4.2 available on your Debian Buster system!
There some caveats to using OpenSSL in a non-standard location though - more about those follows.
Keep pip pointed to OpenSSL 1.0
If you open a new terminal session (or unset the env vars we set earlier) and
try running pip from Python 3.4.2, you'll see an error like this:
pyenv shell 3.4.2
pip
Traceback (most recent call last):
File "/home/USERNAME/.pyenv/versions/3.4.2/bin/pip", line 7, in <module>
from pip import main
File "/home/USERNAME/.pyenv/versions/3.4.2/lib/python3.4/site-packages/pip/__init__.py", line 10, in <module>
from pip.util import get_installed_distributions, get_prog
File "/home/USERNAME/.pyenv/versions/3.4.2/lib/python3.4/site-packages/pip/util.py", line 18, in <module>
from pip._vendor.distlib import version
File "/home/USERNAME/.pyenv/versions/3.4.2/lib/python3.4/site-packages/pip/_vendor/distlib/version.py", line 14, in <module>
from .compat import string_types
File "/home/USERNAME/.pyenv/versions/3.4.2/lib/python3.4/site-packages/pip/_vendor/distlib/compat.py", line 66, in <module>
from urllib.request import (urlopen, urlretrieve, Request, url2pathname,
ImportError: cannot import name 'HTTPSHandler'
That's because pip doesn't know where to look for OpenSSL 1.0.2.
To fix that, we'll need to bring back the LD_LIBRARY_PATH environment
variable, and keep the compiled OpenSSL 1.0.2 around.
To set the variable again, run this line manually - or automate it by adding it to .bashrc or similar:
export LD_LIBRARY_PATH="<PATH-TO-OPENSSL-1.0.2>/lib/"
... where <PATH-TO-OPENSSL-1.0.2> is the path we saw after our OpenSSL
compilation finished, in our case /home/USER/openssl/1.0.2.
Set up virtual environments using OpenSSL 1.0
If you try creating a new virtual environment while the LD_LIBRARY_PATH variable isn't set, pyenv will create the environment without complaining:
pyenv virtualenv 3.4.2 MYVENV
We see no errors here, but no output either. Normally, the command returns couple lines of text ending with Cleaning up.... Did something go wrong?
Well, yes, it did:
pyenv shell MYVENV
pip
pyenv: pip: command not found
The `pip' command exists in these Python versions:
3.4.2
Note: See 'pyenv help global' for tips on allowing both
python2 and python3 to be found.
There's no pip!
Luckily for us, like the previous pip-related issue, this one, too, can be fixed with
LD_LIBRARY_PATH - just make sure it's set when you create the virtualenv.
export LD_LIBRARY_PATH="<PATH-TO-OPENSSL-1.0.2>/lib/"
pyenv virtualenv 3.4.2 MYVENV
Ignoring indexes: https://pypi.python.org/simple/
Requirement already satisfied (use --upgrade to upgrade): setuptools in /home/USERNAME/.pyenv/versions/3.4.2/envs/MYVENV/lib/python3.4/site-packages
Requirement already satisfied (use --upgrade to upgrade): pip in /home/USERNAME/.pyenv/versions/3.4.2/envs/MYVENV/lib/python3.4/site-packages
Cleaning up...
... and that output is what we want - it means that our virtual environment has been created successfully. Now, your new environment contains pip as expected!
And, of course, Python 3.4 has EOLed in 2019, so both you and I should upgrade to a newer version soon.