Ask Your Question

Iguananaut's profile - activity

2019-10-17 08:48:50 -0500 answered a question Printing from parallel processes

Handling screen output (and other side-effects) during parallel processing is tricky business, because if you have multiple processes all writing to the same terminal simultaneously, the messages can become overlapped and garbled. It seems that what @parallel does in this case is it collects each sub-process's stdout (it should probably also do this for stderr) into a file, and later when the each process completes it outputs that process's stdout all at once to your main process's stdout.

There are several other ways to do this of course, and maybe this interface could provide more options, but there is no one general sane way to automatically overlap output from multiple simultaneous processes, at least not without some terminal-based GUI.

Here's one solution I came up with for doing this for functions decorated with Sage's @parallel decorator, though it was tricky-enough that I think it pointed the way toward some possible ways @parallel could be improved.

First of all, we create a queue for handling log messages from each sub-process. When each process wants to log a message it does so by putting it on a queue (we'll see this later). Meanwhile, since @parallel processes block the main process when called, we need to start a separate process (this might also work with a thread) to handle reading log messages off the queue and printing them one at a time as they come in. The nice thing about multiprocessing.Queue is it handles issues like IPC and synchronization for us:

import os

def handle_log_messages(q):
    while True:
        # This blocks until at least one message arrives on the queue
        message = q.get()
        if message is None:
            # We allow passing None as a "poison pill" to terminate the log handler
            break

        # Otherwise print the message and then wait for the next one
        print('[%d] %s' % (os.getpid(), message))

Next, we modify the parallelized function to take the log queue as an optional second argument, and add a simple wrapper to make it easy to "log" messages by putting them on the queue:

@parallel(1, timeout=10)
def fun(x, log_queue=None):
   def log(message):
        if log_queue is not None:
            log_queue.put(message)
    log('Starting %d' % x)
    sleep(5)
    log('Finished %d' % x)
    return x*x

Now when we call fun we also need to pass in the log_queue as an argument if we want logging. It would be nice if we could just call it like fun([1, 2, 3], log_queue=log_queue) where for keyword arguments the same value is passed in for each worker process. Unfortunately it does not currently work that way so I needed a kinda ugly workaround as you'll see later.

Now we instantiate the queue and the log handler:

from multiprocessing import Queue, Process
log_queue = Queue()
log_handler = Process(target=handle_log_messages, args=(log_queue,))
log_handler.start()  # Starts handle_log_messages in the background and starts waiting for messages

Then call our parallelized function. As mentioned previously we have to pass the same log_queue as the second argument to the function, which I did like this:

from itertools import repeat
values = [1, 2, 3,]
squared = fun(zip(values, repeat(log_queue)))

Now, while fun executes each log message is printed more-or-less in the order the arrive, along with the PID of the worker process it came from.

This could all probably be wrapped up in a more generic helper function, which might make a useful feature to add.

2019-10-17 07:25:44 -0500 edited answer Sage is not preparsing Python 3 underscored integers

Good point. Indeed, the preparser is not able to handle such literals yet. This is now trac ticket 28490.

2019-09-18 04:05:24 -0500 commented question Sage is not preparsing Python 3 underscored integers

Huh. I didn't even know this a feature of Python 3.

2019-09-18 03:55:39 -0500 commented question Difficulty managing to install SageMath on my laptop

Also, if you just want to try SageMath, and have good internet access, you can try it out through your web browser without installing anything by making a free account on https://cocalc.com/

2019-09-18 03:54:20 -0500 commented question Difficulty managing to install SageMath on my laptop

Hi Norman--if you have a more "computer literate" person near you you might want to ask them for help because it sounds like you're (understandably!) a bit confused, and it's hard to really understand what your problem is (often just knowing the right question to ask requires a degree of expertise...)

2019-09-16 07:40:05 -0500 received badge  Nice Answer (source)
2019-09-13 09:43:38 -0500 answered a question installing sage-8.8-Ubuntu_18.04-x86_64.tar.bz2

Here's what I did:

$ cd ~/Downloads
$ wget http://www-ftp.lip6.fr/pub/math/sagemath/linux/64bit/sage-8.8-Ubuntu_18.04-x86_64.tar.bz2
$ echo '171b62272a6bf869246fd9fdbc37eff1 *sage-8.8-Ubuntu_18.04-x86_64.tar.bz2' | md5sum -c
$ mkdir -p ~/.local/opt  # This is just where I opted to put it for myself; this is up to you
$ tar xf sage-8.8-Ubuntu_18.04-x86_64.tar.bz2 -C ~/.local/opt/
$ cd ~/.local/opt/SageMath
$ ./sage

Running ./sage for the first time outputs:

Rewriting paths for your new installation directory
===================================================

This might take a few minutes but only has to be done once.

patching /home/embray/.local/opt/SageMath/build/make/Makefile-auto
... many more lines like this ...
patching /home/embray/.local/opt/SageMath/src/sage/ext/interpreters/wrapper_rr.pyx
┌────────────────────────────────────────────────────────────────────┐
│ SageMath version 8.8, Release Date: 2019-06-26                     │
│ Using Python 2.7.15. Type "help()" for help.                       │
└────────────────────────────────────────────────────────────────────┘
Setting permissions of DOT_SAGE directory so only you can read and write it.
sage:

I also made a symlink:

$ ln -s ../opt/SageMath/sage ~/.local/bin/

Because I have ~/.local/bin on my $PATH, now I can run that Sage installation by just typing sage at the command prompt.

Also FWIW I have:

$ which python
/usr/bin/python
$ python --version
Python 2.7.15+
$ dpkg -S `which python`
python-minimal: /usr/bin/python

I also noticed in your original post that you were running as root. That might be part of your problem. If you're taking various arbitrary actions on your system as root then you can break almost anything.

2019-09-13 09:18:08 -0500 commented answer installing sage-8.8-Ubuntu_18.04-x86_64.tar.bz2

Yes but for beginners I would not recommend it. If you can't get a working Python on your system you definitely won't be able to install Sage from source. I have Ubuntu 18.04. I will try installing the Sage 8.8 binary on it and report back to you exactly the steps I took...

2019-09-13 09:14:25 -0500 commented question installing sage-8.8-Ubuntu_18.04-x86_64.tar.bz2

Unless, that is, you're trying to share your home directory between Windows and Linux partitions which, if so, would be worth actually specifying. Using an NFTS partition as your home directory may not work for some Linux software, as *nix permissions can't be emulated exactly (or at all) on NTFS partitions.

2019-09-13 09:11:59 -0500 commented question installing sage-8.8-Ubuntu_18.04-x86_64.tar.bz2

@ortollj I don't think dual booting has anything to do with it. Installing Sage on Ubuntu or Windows is totally different...

2019-09-13 09:09:54 -0500 commented question Sage Math can't find pandoc on MacOS Mojave

What happens if you run from sage.features.pandoc import Pandoc; Pandoc().is_present()?

2019-09-13 09:05:18 -0500 commented question SageAppliance not more maintained after 8.1?

FWIW it is possible now to install optional packages on Windows, though not all packages will install out-of-the-box, as they have not all been tested on Windows.

2019-09-09 10:49:23 -0500 commented answer Run sage notebook without sage.all imported

I'm quite certain it's possible to speed of Sage initialization by importing just a smaller set of core modules, and I would like to be able to do that too. Even with the import cycles (some of which I believe can be broken with appropriate refactoring, although some others are unavoidable), I think it should be possible to get imports from sage working without importing as much as sage.all does. It's something I've always wanted to work on but it is a daunting task no doubt.

2019-09-04 09:27:30 -0500 commented answer How do I install opencv in SageMath 8.8 on Windows 10

It appears that OpenCV only supplies pre-built binary wheels on PyPI, and unsurprisingly does not supply one built for Cygwin. This is a problem to ask of the OpenCV developers--at the very least the should also publish the source tarball on PyPI so it can be downloaded and (with any luck) built on platforms that they don't ship wheels for. Alternatively you can see if you can download the source tarball yourself, and use pip to install it manually like pip install <path/to/tarball>.

2019-09-04 09:21:44 -0500 edited question how do I enter 2sin|1−3√2|cos|1+ 2√3|

when I enter it in SageMath as 2 * sin * abs(1-3 * sqrt(2))*cos*abs(1+2 * sqrt(3)) it does not run the code? your help would be very appreciated :)

2019-09-04 09:20:05 -0500 commented question error for (3*e+2*I)*(-3+pi*I)

Is it possible you redefined e and pi? Because if you did something like e = 2.718281828; pi = 3.141592654 before computing the result, because then yes you'll get a numerical answer (and it would be a silly thing to do because e and pi are already built into Sage and can be given to high precision (e.g. pi.n(digits=100) => 3.141592653589793238462643383279502884197169399375105820974944592307816406286208998628034825342117068)`

2019-09-04 06:45:57 -0500 received badge  Nice Answer (source)
2019-09-04 05:54:23 -0500 commented question entering 2sin􏰀􏰀|1−3√2|􏰀􏰀cos|􏰀􏰀1+ 2√3|􏰀􏰀

(Probably you just want sin(abs(x)) which is $ \sin(|x|) $.

2019-09-04 05:44:52 -0500 commented question entering 2sin􏰀􏰀|1−3√2|􏰀􏰀cos|􏰀􏰀1+ 2√3|􏰀􏰀

Please re-post your question with a different title, and try entering the text of your problem into the body of the question. You can post equations using $ \LaTeX $ by enclosing it in dollar signs like $ \LaTeX $.

2019-09-04 05:42:31 -0500 commented answer Sagemath for Windows: a giant headache

Configuring PyCharm on Windows to use Sage: https://ask.sagemath.org/question/397...

2019-09-04 05:35:45 -0500 answered a question Sagemath for Windows: a giant headache

Piggy-backing on @vdelecroix's answer, I wrote a whole little tutorial on using Sage from PyCharm here: https://ask.sagemath.org/question/397...

I believe due to some recent changes in Sage its now even easier than that and requires setting fewer environment variables, but I'm not positive. Haven't tested it in a while.

Second, as (former; haven't needed it in a while) user of QuTiP, anaconda is not required to install it. You can pip install it into Sage's Python by running

$ pip install 'qutip<4.4.0'

(slight update: it appears QuTiP has at some point ceased continuous integration on Python 2 and have been letting Python 2 support break, despite not announcing anywhere a plan to cease Python 2 support, so it turned out the latest version contains Python 3-only syntax and is broken on Python 2; Sage 9.0 when it's released, will likely be the first to claim support for Python 3, and there will likely be a Python 3 Windows build as well.)

in the Sage shell. All of its dependencies are already included in Sage.

It does suck having to have two copies, and that Sage currently requires its own Python interpreter. But that's something that could be improved if more of Sage's dependencies were ported to native Windows (or if Anaconda added official support for packages built with Cygwin support; ask them they have millions of dollars in DoD funding).

2019-09-04 05:33:03 -0500 commented question Sagemath for Windows: a giant headache

If you have about 2 to 3 years' worth of funding available for a full time developer I can make Sage installable in anaconda on Windows :) Maybe that would be a better use of your money than that Mathematica license but I don't know!

2019-09-04 05:31:05 -0500 commented answer StopIteration raised during G.eulerian_circuit()

Worse comes to worse you could also catch the StopIteration exception and return an empty list or whatever the appropriate answer should be in this case.

2019-08-26 17:23:10 -0500 edited question LLL algorithm extended to linearly dependent vectors

Dear Sage community,

I am searching the way to compute Modified LLL Reduction algorithm ( by Michael Pohst) having in input a linear dependent vectors. Please could anyone help me to understand a way to do that in SAGE ?

Thank you in advance best regards AP

2019-08-22 15:43:47 -0500 commented question Slider for fractions of pi

So make a slider from 0 to 2 with step size 0.05 and then multiply that x by pi. Did you have some trouble doing that? What did you try?

2019-08-21 04:29:26 -0500 commented answer Computing a formula in SAGE

@Vochau I would suggest not simply copy/pasting. Try to type out the code that @rburing wrote manually and make sure, step by step, that you understand what it's doing. That way you'll be able to learn more, and be able to make your own modifications to solve variants of the problem.

2019-08-21 04:24:37 -0500 commented answer `sage -pip install --user ffmpeg` did not make the "`OSError: Error: ffmpeg does not appear to be installed.`" message go away

It looks like there is a poorly(?) documented optional dependency on ffmpeg for saving plot animations as video formats. That should probably be improved somehow. It looks like there isn't even an optional sage package for it...

2019-08-19 03:30:12 -0500 commented question `sage -pip install --user ffmpeg` did not make the "`OSError: Error: ffmpeg does not appear to be installed.`" message go away

It's not really clear what you mean by "I restarted everywhere I could restart", nor is it clear in what context you're getting this message. Did you try re-creating the Docker container? I don't think this is really a question about Sage unfortunately. Probably not related to anything you were trying to do with ffmpeg though.

2019-08-18 04:16:11 -0500 commented question cosmological spacetimes

As stated this is a little broad. Do you have some equations for what you'd like to do, and are there any specific stumbling blocks you are having getting started? I understand more-or-less what you're asking, but most people on here aren't cosmologists so the question would be more meaningful if you posted some math and/or code.

2019-08-18 04:08:32 -0500 commented answer How to generate normal noise for a function

Hi @lionel2000, please post a new question to ask a question. Do not ask questions as "answers" to different, unrelated questions.

2019-08-15 18:09:19 -0500 received badge  Good Answer (source)
2019-08-15 10:24:17 -0500 received badge  Nice Answer (source)
2019-08-15 08:02:58 -0500 answered a question asking real part of a function crash the kernel

I got

sage: n=var('n')
sage: assume(n>=0)
sage: assume(n,'integer')
sage: a_n=1/20*(1/2*sqrt(5) + 1/2)^n*(11*sqrt(5) + 25) - 1/20*(-1/2*sqrt(5) + 1/2)^n*(11*sqrt(5) - 25) - 1/12*
....: (1/2*I*sqrt(3) + 1/2)^n*(I*sqrt(3) + 3) - 1/12*(-1/2*I*sqrt(3) + 1/2)^n*(-I*sqrt(3) + 3)
sage: a_n.real_part()
------------------------------------------------------------------------
Unhandled SIGSEGV: A segmentation fault occurred.
This probably occurred because a *compiled* module has a bug
in it and is not properly wrapped with sig_on(), sig_off().
Python will now terminate.
------------------------------------------------------------------------
Segmentation fault (core dumped)

Will update my answer as I find out more, but obviously a segfault shouldn't happen here.

Confirmed on Linux and Windows. The segfault is coming from pynac. It seems to be specifically the assume(n, 'integer') part that does it. Without that it doesn't crash.

I spent a few minutes trying to narrow this down to a simpler example and got this

sage: n=var('n')
sage: assume(n, 'integer')
sage: (I^n).real_part()
<segfault>

It appears to go into an infinite recursion and eventually overflow the stack in

src/sagemath/sage/local/lib/libpynac.so.18(_ZNK5GiNaC5power9real_partEv+0x51)[0x7f7c45607bf1]

The correct answer for $ \mathfrak{Re}[i^n] $ should be $ \cos\left(\frac{\pi n}{2}\right) $ but I guess it's not that clever, and it certainly shouldn't crash...

If I don't stipulate that $ n \in \mathbb{Z} $ then it does give the correct answer, sort of...

sage: n=var('n')
sage: (I^n).real_part()
cos(1/2*pi*real_part(n))*e^(-1/2*pi*imag_part(n))

This is now tracked at https://trac.sagemath.org/ticket/28357

2019-08-15 07:58:28 -0500 commented question asking real part of a function crash the kernel

Have you tried it at the command prompt? In general it's good to isolate a problem from the specific environment in which it occurred, if possible.

2019-08-14 09:55:42 -0500 received badge  Nice Answer (source)
2019-08-14 07:17:17 -0500 commented question Where is defined __gmpq_cmp_z?

mpq_cmp_z (which gets mangled to __gmpq_cmp_z) is part of GMP, which is a dependency of sage. How did you install sage?

2019-08-14 07:17:06 -0500 commented question Where is defined __gmpq_cmp_z?

For that matter, those are just modules that reference that symbol, not the one where it's defined.

2019-08-14 07:15:02 -0500 edited answer var list with only one variable error

It seems for now you'll have to do an ugly workaround like:

sage: var(L[0] if len(L) == 1 else L)

I have opened a ticket with a proposed fix to the bug: https://trac.sagemath.org/ticket/28353