Ask Your Question
2

SSL error using sage -pip install to download a package

asked 2020-04-29 06:30:15 +0100

gbowling gravatar image

updated 2020-04-30 03:41:22 +0100

I have gotten the following error when trying to use sage -pip install "packagename" in terminal.

Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/multipolynomial-bases/
pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Could not fetch URL https ://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.")) - skipping

I have tried looking up this problem and it seems to have been somewhat common, but from what I understand this should work easily with the current version of sage. I have tried many "fixes" that I have not gotten to work, including downloading the package and trying to use pip to install it locally. Please let me know if anyone knows how I may fix this!

I am using sage 9.0 on OSX 10.11.6.

edit retag flag offensive close merge delete

2 Answers

Sort by » oldest newest most voted
5

answered 2020-04-29 15:42:03 +0100

slelievre gravatar image

updated 2020-07-25 18:23:53 +0100

Installing extra Python packages for Sage on macOS

To "pip install" extra packages, pip needs to be able to establish secure connections. This requires the underlying Python to have its SSL module available.

On most Linux distributions, people normally have OpenSSL installed as part of their distribution, so we ship Sage binaries whose Python is compiled with its SSL module, and it will generally work.

On macOS, Apple uses another way to deal with secure connection, and does not provide the openssl package as part of the operating system.

The binaries we ship for macOS don't have OpenSSL because it was up to now licensed under "the OpenSSL license" which did not allow to distribute it along with GPL-licensed software such as Sage.

This is about to change since OpenSSL 3 will be licensed under the Apache 2.0 license. which will allow distributing OpenSSL with binaries of GPL-licensed software such as Sage. So far (2020-04), only a preliminary version of OpenSSL 3 is out. Maybe by the end of 2020 this will be solved for good.

In the meantime, macOS users who want to expand their Sage with extra Python packages have a few choices, including:

a. download Python packages and install them locally

b. modify one's Sage installation to give its Python the SSL module

c. use a version of Sage whose Python has the SSL module

Below some more details about each of these options.

In terminal commands below, the initial $ represents the shell prompt, it should not be typed, so if you copy-paste the commands don't include that initial $.

a. Download Python packages and install them locally

First note that running sage --pip install packagename relies on packagename being available on PyPI (the Python Package Index).

The PyPI page for packagename is likely:

`https://pypi.org/project/packagename`

Visit that page in a web browser and find an option there to download packagename as a zip file, or maybe some other compressed format, but let's say zip. Download it, and get a file packagename.zip on your computer, at some location, let's say in the Downloads folder in your home.

Then run the following (adapt path and filename if needed):

$ sage --pip install ~/Downloads/packagename.zip

This whole "go visit some webpage, download a zip, and run sage --pip install /path/to/packagename.zip" is a little less convenient than the simple sage --pip install packagename, but at least it gets the job done and doesn't require any change to your Sage installation.

In particular, no risk of damaging your Sage installation and getting it to a non-working state.

b. Install OpenSSL on an existing Sage installation

There are two ways to do that: b.1 is the easiest, and b.2 is a little more involved.

b.1. Ready-made

The easy way is provided by the developers of computational topology software packages (for studying the topology of three-manifolds). That software can be made to work with Sage and they found a way to provide just the right pieces to fix a Sage installation.

Download "fix_mac_sage8.tar.gz" or "fix_mac_sage9.tar.gz" (respectively for Sage 8.x and Sage 9.x) from the page:

Then unpack and follow the instructions in the README file found in the unpacked folder.

b.2. Rebuilding a lot of things

This is a different way of obtaining the same result one would get by following b.1.

Warning: it takes a lot longer. It leaves you without a working Sage installation while it is running. In addition, if it fails, it could break your Sage installation, in which case you have to reinstall Sage. (In that case you might want to use option c).

There is a preliminary, which is to have Apple's "command-line tools for developers" installed.

To install those, run this command in a terminal:

$ xcode-select --install

This will either tell you that the command-line tools are already installed, or open a dialog to let you install them.

Once this is done, it might be a good idea to restart your computer. Not sure it's needed but it can't hurt.

Now we are almost at the crucial step. This step will take a long time (up to hours) as it forces the rebuild of Sage's python3 module, which in turn forces to rebuild all packages that depend on python3 (in particular Numpy, Scipy, etc.).

To mitigate that, optionally, set the number of jobs that make should run in parallel.

To check how many processors can be used, run this:

$ sysctl -n hw.ncpu

Then, to set make to run say 4 jobs in parallel, run:

$ export MAKE='make -j4`

Now these two commands should work (but to warn again, it's happened to me that it did not work and left me with a broken Sage which I did not know how to repair):

$ sage -i openssl
$ sage -f python3  # can take hours

If that worked, you should be able to run

$ sage --pip install packagename

to install any Python package packagename from PyPI.

c. Installing Sage with SSL support

We now mention ways to perform a complete installation of Sage with OpenSSL support.

There are several choices including:

  1. install Sage via Conda, see

  2. build Sage from source

    • in a Conda environment which has OpenSSL installed
    • or with OpenSSL installed via Homebrew
    • or without any of Conda or Homebrew

On several computers

If you have several computers with the same version of macOS, you might be able to transfer Sage installations from one to the other, as long as you put them at the exact same location, for example /Applications/SageMath.

Use your favourite way to transfer files:

  • (compress then copy then uncompress)
  • rsync
  • ...

Careful, a Sage installation can be quite big, so it might not fit on a USB drive especially if it is formatted as "FAT" (which does not allow files over 4 GB).

If needed, after compressing to zip or tar-gz or other, split the file into several parts, put the parts on a usb drive, and reassemble the parts on the other side.

See this for details:

edit flag offensive delete link more

Comments

Thank you for the detailed comment. I am still getting errors using the first method. Processing /Users/grant/Downloads/multipolynomial_bases-1.2.tar.gz Complete output from command python setup.py egg_info: Traceback (most recent call last): File "<string>", line 1, in <module> File "/private/var/folders/d0/5l10k_w88xlgch00r6b9hv6h0000gq/T/pip-req-build-_esll2si/setup.py", line 22, in <module> version = readfile("VERSION"), # the VERSION file is shared with the documentation File "/private/var/folders/d0/5l10k_w88xlgch00r6b9hv6h0000gq/T/pip-req-build-_esll2si/setup.py", line 11, in readfile with open(filename, encoding='utf-8') as f: File "/Applications/SageMath-9.0.app/Contents/Resources/sage/local/lib/python3.7/codecs.py", line 898, in o

gbowling gravatar imagegbowling ( 2020-04-29 22:20:23 +0100 )edit

pen file = builtins.open(filename, mode, buffering) FileNotFoundError: [Errno 2] No such file or directory: 'VERSION'

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/d0/5l10k_w88xlgch00r6b9hv6h0000gq/T/pip-req-build-_esll2si/ pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available. Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.")) - skipping

gbowling gravatar imagegbowling ( 2020-04-29 22:20:55 +0100 )edit

When trying the second option, with the command line tools installed, sage -i openssl, I got an error as well. [openssl-1.1.1b] ccache: error: Failed to create directory /Users/grant/.ccache/f/2: Permission denied [openssl-1.1.1b] Error building openssl.

The following package(s) may have failed to build (not necessarily during this run of 'make openssl'):

  • package: openssl-1.1.1b log file: /Applications/SageMath-9.0.app/Contents/Resources/sage/logs/pkgs/openssl-1.1.1b.log build directory: /Applications/SageMath-9.0.app/Contents/Resources/sage/local/var/tmp/sage/build/openssl-1.1.1b
gbowling gravatar imagegbowling ( 2020-04-30 00:35:40 +0100 )edit

It seems I was able to install the .whl version from the website using -pip install packagename.whl! This ended up working. For whatever reason the .tar.gz file would not. Thank you!

gbowling gravatar imagegbowling ( 2020-04-30 02:08:12 +0100 )edit

It would have been useful to warn that sage -f python3 will take hours, leaving one without the ability to work in the meantime. Sage can be such a bloody pain sometimes (not infrequently)

Szabolcs gravatar imageSzabolcs ( 2020-05-06 20:31:09 +0100 )edit
0

answered 2020-04-29 15:04:24 +0100

Sébastien gravatar image

If you already compiled sage from source or you downloaded the binaries, you need to:

make openssl
sage -f python3 # takes very long time

Maybe you need to run make after that or maybe also make pyopenssl but I am not sure. I do answer this question around me often but I do not have a OSX myself, so I am not 100% sure.

If you just downloaded the source, you can run make openssl first so that python3 gets built with its SSL module according to this post by Samuel Lelièvre:

make openssl
make

That being said, hopefully openssl will be included in SageMath as a standard package soon, see ticket #29555 which will make all of this much more easy for everyone.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

2 followers

Stats

Asked: 2020-04-29 06:30:15 +0100

Seen: 4,199 times

Last updated: Jul 25 '20