Ask Your Question
1

It's really hard to build sage... why???

asked 2024-05-25 09:30:38 +0100

nothingnoone gravatar image

updated 2024-05-26 21:09:12 +0100

FrédéricC gravatar image

I've been struggling to get sage to build on a Kali 2024 installation for about 6 hours today. It's really difficult for me, and I have been a linux user for 20+ years and consider myself to be a pretty knowledgeable person. I've been building code from source since before github was a thing, and . bootstrap and ./configure are well known to me.

But dang, this software is difficult to figure out. First there are loads of wrong descriptions on the internet and even in the README(!) which tells me to download a binary distribution if I can. Well, that's no longer supported so what am I supposed to do? There is no sage package in Kali (not debian) and if you try to use the debian mirror you get loads of package dependency errors.

So fine, I'll build from source! The README says it's the usual "./configure;make;make install" but then I find I need to run "make configure" first, which is quite different than what is normal (. bootstrap is 'normal' for old-school stuff, ./configure is generally accepted, and 'mkdir build; cd build; cmake ../' is not unheard of). Then I get a SHITLOAD of errors, with 'helpful' (not really helpful) command lines to apt-get install a bunch of packages. Well, I know it seems like debian but A) it's not and B) those packages don't exist. Also there are TWO listings of apt-get commands. Why are there two different listings??? What is this software trying to tell me? So far I've read "download binaries" and "build from source" but neither is actually possible.

Now here's a real treasure: I get errors when I can finally run ./configure because the install dir (e.g., /usr/local/bin) isn't writable by my user. Well yeah... The README told me to configure the install location, so I did. But when I run as sudo, I get an error telling me not to do that either! Then the helpful "--enable-build-as-root" option for DOCKER ONLY. But I can't seem to build this software without it (yeah, this is in a VM, so let me please do what I want). So I'm supposed to only run this software as a non-root user, unless I'm using docker. Okay, thanks for helping me out with my choices, even though no novice is ever going to be able to install this software from source anyways; it's just infuriating to someone who knows what they're doing (granted, I may not know what I'm doing in your opinion given my ranting thus far. But I can configure linux kernels for custom embedded devices and have that work, so I'm really not a n00b (I pinkie swear)).

BTW, here's an example of the shitty bash script I used to try to install the debian packages for this software based on the recommendation of the configure script: for i in 4ti2 gpgconf openssh-client default-jdk libavdevice-dev coinor-cbc coinor-libcbc-dev ffmpeg fricas libigraph-dev libisl-dev libgraphviz-dev libnauty-dev lrslib libtbb-dev pandoc pdf2svg libxml-libxslt-perl libxml-writer-perl libxml2-dev libperl-dev libfile-slurp-perl libjson-perl libsvg-perl libterm-readkey-perl libterm-readline-gnu-perl libmongodb-perl libterm-readline-gnu-perl polymake libpolymake-dev r-base-dev r-cran-lattice sbcl texlive-latex-extra texlive-xetex latexmk dvipng tex-gyre texlive-fonts-recommended texlive-lang-cyrillic texlive-lang-english texlive-lang-european texlive-lang-french texlive-lang-german texlive-lang-italian texlive-lang-japanese texlive-lang-polish texlive-lang-portuguese texlive-lang-spanish texlive-luatex xindy;do echo "trying: apt install -y $i";sudo apt install -y $i;done

note I had to do it one at a time because apt doesn't like it when packages don't exist.

Again, there were actually two lists of apt-get install package recommended. Doing both didn't help, exactly. One package in particular that caused issues is "fricas". The configure script on my system calls fricas in some way that hangs forever. Ctrl+C/Ctrl+Z doesn't work in the shell (fun!) so I have to kill all the processes related to fricas from another shell. Not necessarily the fault of sage, but why does a non-required dependency cause the whole thing to hang?

Another fun one: libbrial-dev doesn't exist in this distro. But it keeps being suggested I install it, even if the library itself is already installed.

This one isn't fun at all. Guess what? I'm building in a place sage doesn't like. I struggle with this for a while because the build output is super vague and doesn't actually preserve the error, but eventually I discover: /mnt/SOMEWHERE/sage/local/var/tmp/sage/build/mpfr-4.2.1/inst to /mnt/vmshared/ethical_hacking/sage/local cp: failed to preserve ownership for /mnt/SOMEWHERE/sage/local/./lib/libmpfr.so.6: No such file or directory cp: failed to preserve ownership for /mnt/SOMEWHERE/sage/local/./lib/libmpfr.so: No such file or directory

Guess what, it's not a filesystem that can handle xattrs or even linux permissions. So your software needs to know the permissions of the filesystem to build? Okay, I guess... but no, wait, that's insane. Just build the software, please. Compile and put your binaries where I tell you and I'll worry about permissions. Why are you assuming a particular file system when distributing your software?!?!?

My latest configure run. This is not ambiguous at all (that is sarcasm):

checking for the package system in use... debian configure:

hint: installing the following system packages, if not
already present, is recommended and may avoid having to
build them (though some may have to be built anyway):

  $ sudo apt-get update 
  $ sudo apt-get install libfplll-dev texinfo maxima-sage maxima palp

configure:

hint: installing the following system packages, if not
already present, may provide additional optional features:

  $ sudo apt-get update 
  $ sudo apt-get install 4ti2 gpgconf openssh-client default-jdk libavdevice-dev libigraph-dev polymake libpolymake-dev

configure:

hint: After installation, re-run configure using:

  $ make reconfigure

❯ sudo apt install texinfo maxima-sage Reading package lists... Done Building dependency tree... Done Reading state information... Done texinfo is already the newest version (7.1-3). maxima-sage is already the newest version (5.45.1-2+b1). 0 upgraded, 0 newly installed, 0 to remove and 1318 not upgraded.

Those packages are installed, so WTF mate? How can I get you to be happy? Apparently sage is just an unhappy package, that will never be satisfied, no matter how many things I install. Only if I run it as root and pretend I'm not root can it really thrive.

Oh, yes, I had to move the directory sage to a filesystem that was able to handle the "advanced super secure" permissions required by sage to build. Let's see if it builds!

Nope! First we get errors about writing to the directory (something make should never do. make install should be putting the binaries/docs/etc. into the right place; make should NOT EVER write outside it's current directory). This software is just insane.

Oh I guess I need to run 'make clean' (not found in the documentation) since I moved directories to somewhere sage would /allow/ me to run it's precious software. oops another CRITICAL FAILURE that blocks me from moving on: rm: cannot remove '/usr/local/bin/var/tmp/sage/build': Permission denied

Some on... so you're putting the build files into the bindir????? Every part of your build should be relative to the source directory. This is crazy, and GAAAAAAHHHHHHH. Any make command other than 'make install' should just do it's own thing, in the directory it's currently executed in. --prefix doesn't mean "put your source and .o files here" it means, "put your install files here".

So now I'm trying ./configure with all paths that are on a ext4 filesystem, putting the --prefix, etc. options in the same path so no root is needed. I get the following relatively useless information again:

notice: the following SPKGs did not find equivalent system packages:

    brial cypari eclib fflas_ffpack fplll fpylll givaro info linbox lrcalc mathjax maxima palp pplpy primecountpy singular   4ti2 _develop _recommended bliss coxeter3 igraph libsemigroups polymake symengine

checking for the package system in use... debian configure:

hint: installing the following system packages, if not
already present, is recommended and may avoid having to
build them (though some may have to be built anyway):

  $ sudo apt-get update 
  $ sudo apt-get install libbrial-dev libbrial-groebner-dev libec-dev eclib-tools fflas-ffpack libfplll-dev libgivaro-dev texinfo liblinbox-dev liblrcalc-dev maxima-sage maxima palp singular singular-doc libsingular4-dev

configure:

hint: installing the following system packages, if not
already present, may provide additional optional features:

  $ sudo apt-get update 
  $ sudo apt-get install 4ti2 gpgconf openssh-client default-jdk libavdevice-dev libigraph-dev polymake libpolymake-dev

(Please remember, I already tried to install these, but this isn't a real debian system. However, many of them are installed... for example 4ti2, gpgconf, openssh-client, default-jdk, polymake and others are there. So what is it complaining about? And why are there two lists??)

Recommendations you likely don't want:

FIX YOUR README: it literally says "Like many other software packages, Sage is built from source using ./configure, followed by make. However, we strongly recommend to read the following step-by-step instructions for building Sage." - BUT you really need to run "make configure" first!!! - This is not clear, and no one is going to read to STEP 7 before trying to build your software. The README is way too verbose. Those of use who are building from source don't need/want a lecture (steps 1-6). - How many people with no understanding of build systems will be doing this? You can't clone the repo and run ./configure. This is wrong, so should be removed from the README.

Maybe just update your documentation to only recommend using docker with a known distro?

Perhaps, provide binaries again? Building from source is a nightmare I thought we'd escaped in 2012 or so. You (or someone else) could easily build binaries in Docker, but they would be trusted if you verify them (and supply hashes).

Work on the configure script. It's really nanny-ish (e.g., --enable-root-blah), and IMO trying to do too much (e.g., setting directory permissions). If someone is installing your software from source they likely know what they are doing. Don't penalize folks for trying to build your software on a FAT32 disk or something.

Add that you can't run the nanny software from a shared directory, or a filesystem without linux file permission support.

I still haven't gotten the software to build yet, and I'm a seriously seasoned person with extensive experience building software. I know you must take that with a grain of salt (get it, seasoned?) but I hope you understand my perspective.

edit retag flag offensive close merge delete

3 Answers

Sort by » oldest newest most voted
3

answered 2024-05-27 14:33:08 +0100

Sébastien gravatar image

updated 2024-05-27 14:40:21 +0100

First, we need to thank you for taking the time to share this experience and asking this question. I think we need more of these comments from external people. I think very few takes the time of sharing such experience which represents the struggling of many others which just don't bother to share their experience. So I am very thankful and very respectful for the person asking the question and I think we all deserve to be.

I know much less about normal installation of packages in Linux than the person asking the question here. But I have been using/developing for sage since 16 years, so let me share my explanation of the "why".

For years, compiling sage was relatively "easy". The idea was that people would just need to run make in the folder, wait and that's it. I think this was far from following the LInux approach (configure & make & make install), but for a mathematician just doing enough terminal, it was easier than trying to install all the dependencies separately. Also, all dependency packages were built including gcc and gfortran even if those packages were on the computer already. Having double copies of so many packages was time-consuming during compilation and sub-optimal. Therefore, at some point (5 to 8 years ago), it was suggested to use the packages from the system if they were available and their version were not too old. I think this was a good idea. This is now performed by the make configure / configure part which needs to be done beforehand. This script checks the packages installed on the system and tells whether it will use the package on the system or not. I used to always check at least whether it was using the gcc, gfortran and Python version from the system. So at least I knew I would gain a lot of time during compilation.

If gcc, gfortran, Python, etc. are not on the system, then it gives suggestions of packages to install. It turns out that these suggestions are broken in the sense that they suggest to install packages which were already installed, or to install packages that do not exist (I also needed to remove some packages by hand from the list which is painful, but not as painful as writing a for loop in bash since I don't know bash as well as the user asking the question).

I was involved at some point in trying to improve these suggestions and I was not alone. See for instance https://github.com/sagemath/sage/issu.... But 4 years later, the issue with these suggestions is still present. Why? At least, I can say that people caring did not succeed in improving this.

My answer to the question. After the change of sage installation to use this "configure" thing beforehand, my method became to install the packages suggested until a fixed-point is reached (the suggestion of packages is equal to the previous list after trying to install them). The suggestions in the fixed-point list need to be ignored. What is important is that I make sure with my eyes that it is using gfortran and gcc from the system. And then I run make in parallel (MAKE="make -j8" make) which is the nice part of sage and I think works great most of the time.

Taking a step back, note that suggestions of packages to install can be completely ignored. You can just run make and wait longer. In that sense, building sage is "easy".

edit flag offensive delete link more
2

answered 2024-05-25 13:22:36 +0100

Emmanuel Charpentier gravatar image

updated 2024-05-25 17:04:59 +0100

I'd suggest you to read, understand and apply the Sage installation guide before posting 10 kB worth of rant.

One valid point stands out in your rant : the README should be updated. To point out more strongly to the installation guide... Please feel free to open a ticket and propose an update...

Now, if you have a real question, please feel free to ask ; we'll be delighted to try and answer it.

HTH,

edit flag offensive delete link more
0

answered 2024-12-04 05:09:41 +0100

"I think we need more of these comments from external people." OK. It is obvious there is a massive amount of work in developing Sage, which I appreciate. I've used it before on other systems and it is great. I've also used SageManifolds, now part of the installation, which is also great. But I also agree with the OP that installing Sage (on Ubuntu for me) is ridiculously difficult. Over the years I have had about a 50% success rate. The wins were mainly with prebuilt binaries. All the great work in developing Sage goes to waste if it can't be installed. The current README is inconsistent in its instructions and overly complicated. I have no idea what the a corrected README should say because I couldn't install Sage, so I can't open a ticket to propose an update. When I try to build, ./bootstrap failed to open mpi4py so crashes. Yet mpi4py is installed, I can start Python and import it, so WTF is going on with bootstrap? ./configure complains "package directory is missing. Re-run ./bootstrap." But bootstrap wants mpi4py. And make throws the same error as ./configure. I don't have the patience to pursue this further - I wanted to use Sage for a specific problem, and I have alternatives, including Mathematica on my Raspberry Pi 5. I am happy to let the pi chug while I sleep rather than tear my hair out on trying to install Sage or spending days troubleshooting through back and forth on this forum. I don't know complex Linux build scripts and the OP's very informative comment that the build process is non-standard (aka here lie dragons) means even if I wanted to try to understand it using Google, it would most likely be further fruitless frustration. So yeah, there is a lot of room for improvement on the installation for Linux.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

Stats

Asked: 2024-05-25 09:30:38 +0100

Seen: 849 times

Last updated: May 27