Szabolcs's profile - activity

It would have been useful to warn that sage -f python3 will take hours, leaving one without the ability to work in the meantime. Sage can be such a bloody pain sometimes (not infrequently)

@tmonteil

That @parallel works:

I cannot run this specific piece of code right now (no spare machine capacity to run for a day), but it should work. It is closely based on what I used these last few days for my actual work.

import datetime

limit = 60 # seconds

@parallel(ncpus=1, p_iter='fork')
def fes_fun(dg):
    alarm(limit)
    try:
        fes = dg.feedback_edge_set()
        print(len(fes))
    except AlarmInterrupt:
        print("timeout")    
    cancel_alarm()
    print(str(datetime.datetime.now()))
    print("\n")


fes_fun( (digraphs.RandomDirectedGNM(300,750) for _ in xrange(1000)) )

@tmonteil

Non-working alarm code is in the original post.

That GLPK is being used: this is based on timings, code is in the comments above. You could not reproduce it. I could reproduce it on two different computers. I would suggest: (1) try on a different instance (2) verify how many CPU cores are in use when you benchmark (GLPK: always one; Coin: possibly multiple)

@tmonteil Thanks for all the help! I do keep checking back here and I am still interested in understanding what is happening.

This looks like a bug in Sage or Sage's GLPK interface.

The practical solution that worked for me was to run the calculation in a separate process using Sage's parallelization features. It can be done by wrapping the feedback arc set calculation with a few function that has the @fork or @parallel decorators.

I can confirm that even after a several-day running time, the calculation has not stalled.

@dsejas No, I am referring to the Sage GUI available on macOS, not specific notebook formats.

@tmonteil I finally managed to compile Sage on a Mac, and compared Coin and GLPK again. Based on the timings, it still looks like solver=None defaults to GLPK in feedback_edge_set but _not_ in MixedIntergerLinearProgram. Coin is still slower than GLPK in the total time used, however, now it will sometimes use more than one CPU core, thus the wall time becomes competitive with GLPK (though typically still not faster). I do not know why it does not use multiple cores on Linux.

Since you seem to be involved with Sage, can you file an issue for this (solver=None should not be GLPK) if you think it is appropriate to do so?

I normally use the official (pre-compiled) app version of Sage on macOS. When I start the app, it adds a new dropdown menu to the menu bar.

Now I switched to building Sage from source, as this gives access to features I could not otherwise have (e.g. install Coin MIP solver with sage -i cbc). How can I run the same GUI interface that is available in the app version? I can run the command line sage program or I can use sage --notebook=jupyter. But is there a way to run the same GUI available in the app version?

I am trying to use Sage's parallelization with @parallel(p_iter='fork') (I need fork specifically to work around the problems brought up here.)

If I use print to show some messages, they only appear when the parallelized function has finished. Example:

@parallel(1, timeout=10)
def fun(x): 
    print('Starting %d' % x)
    sleep(5)
    print('Finished %d' % x)
    return x*x

list(fun([2,3,4]))

The "Starting" and the "Finished" messages show up simultaneously, even though they are issued with a 5 second time difference.

How can I print a message so that it will show up immediately?

I need this when I am running a Sage script in a terminal. I am not using Sage interactively in this case and I am not using the notebook interface.

@tmonteil I think I figured it out how to get this done with @parallel

@tmonteil Are you aware of any feasible workarounds to this stalling? I am trying to understand how parallelization works in Sage. It appears that it will spawn subprocesses which can be killed after a time limit. I am trying to find out if I can run each computation in a new process this way, and kill the process after the time limit (instead of using alarm).

@tomteil 2692 iterations, well over a day, and still going. We can safely say that with Coin it does not stall. However, using Coin would be a significant step-down because its worse performance. I am still puzzled about why there is no performance difference on your computer.

@tmonteil Wow, that's weird! Do you have any idea what could explain this difference? I verified that only a single CPU core is in use with both. I am using Sage 8.9, built from source without changing any options. The solutions that both GLPK and Coin return seem to be the same ones that you quote.

I also tried the calculation with GLPK on a Mac laptop (where I have not yet managed to install CBC), and it took 20 seconds, which is consistent with the 27 s timing I got on the Linux machine. Why would it be taking a full minute on your computer?

P.S. It does not look like it's going to stall with Coin, but let me keep it running for 24 hrs to be sure.

@tomteil It's running, I'll let you know. In the meantime, could you please try if 'Coin' is faster for you on smaller graphs too? I have not found a graph on which 'Coin would be faster than 'GLPK', but I only tried easy ones on which even Coin finishes within 5 min. Your example took much longer than that.

@tmonteil You are right that the code of feedback_edge_set does not seem to select (or mention) GLPK. Maybe MixedIntegerLinearProgram(solver=None) initializes to use GLPK by default despite default_mip_solver() returning 'Coin'. I don't know how to check what MixedIntegerLinearProgram() has actually initialized to. All I can see is that the timing with solver=None is the same as the timing with solver='GLPK'.

@tmonteil

sage: set_random_seed(1234)
sage: g=digraphs.RandomDirectedGNM(200,500)
sage: %time g.feedback_edge_set(solver='GLPK')
CPU times: user 27.6 s, sys: 28 ms, total: 27.7 s
Wall time: 27.7 s
sage: %time g.feedback_edge_set(solver='Coin')
CPU times: user 1min 19s, sys: 1.25 s, total: 1min 21s
Wall time: 1min 19s

@david Yes, I am sure. If I run %time g.feedback_edge_set(solver='GLPK') then the timing is the same as with %time g.feedback_edge_set(). If I run %time g.feedback_edge_set(solver='Coin') then the timing is several times longer. Therefore, I want to use GLPK for this application.

@tmonteil It's not because of a specific graph. The stalling appears to be random. I know this because I normally run this on a set of pre-computed graphs, so I could easily re-try on the graph where it stalled. Also, running it twice on the same data won't stall at the same step.

@tmonteil I added a minimal example. Can you try to reproduce the problem?

I was using SageMath 8.8's feedback_edge_set function on a large dataset. The calculation would take a couple of days to finish.

Unfortunately, Sage would keep stalling every few hours. The CPU usage of the process would simply drop to zero, and it would stop doing anything. I had to kill the process (a simple kill worked, no kill -9 necessary).

Has anyone else experienced this? Is it a known problem? Is there a workaround? It is very frustrating to have to keep manually killing and restarting the process. This problem also prevents me from running a calculation overnight: in the morning I might find that it stopped merely an hour after I started it (if I'm unlucky).

More information:

• I was using Sage on Linux
• This function uses MixedIntegerLinearProgram. I used the default GLPK (didn't change settings)
• I used alarm() to set a time limit on each computation.
• I always believed GLPK not to be interruptible, so I am not even sure which interruption (either manual or alarm) work on Sage with this function. This may be relevant.
• The memory usage of the process stayed low. It did not stall because the memory got filled and the machine started swapping.

Minimal example:

Put this in a source file:

import datetime
i=1
while True:
    alarm(60)
    try:    
        dg=digraphs.RandomDirectedGNM(300,750);
        fes = dg.feedback_edge_set()
        print(len(fes))
    except AlarmInterrupt:
        print("timeout")
    cancel_alarm()
    print(i)
    print(str(datetime.datetime.now()))
    print("\n")
    i = i+1

Run it as sage sourcefile.sage.

In my last test, it took 291 iterations and ~4-5 hours before it stalled.

It appears to stall only when GLPK is used as the MIP solver (i.e. feedback_edge_set(solver='GLPK'), which is also the default). If I use solver='Coin' (which needs to be installed first using sage -i cbc) then it does not stall. However, the calculation is several times slower with Coin than with GLPK.

Some comments on sage -i cbc: this appears to set the default_mip_solver() to "Coin", but feedback_edge_set still uses GLPK by default. I can set it to use Coin manually, but GLPK is far faster. GLPK is also much faster than Gurobi when using this specific function. Perhaps this function uses some feature which is GLPK-specific and that's why it'll have better performance with GLPK. I don't know.

Sure—I'll get back to you when I have a small example that reliably fails. It'll take time as it usually takes hours for it to fail.

I asked libreadline-dev, libncurses5-dev and libncursesw5-dev to be installed on this computer. Now Sage 8.9 compiles successfully. libreadline-dev alone was not sufficient. I did this based on the linked Google Groups thread before Dima posted here.

Checking the log file more carefully revealed this line:

*** WARNING: renaming "readline" since importing it failed: /home/szhorvat/sage-8.9/local/lib/libreadline.so.6: undefined symbol: UP

It fails with the same message.

The output of ls /lib/x86_64-linux-gnu/libreadline* is

/lib/x86_64-linux-gnu/libreadline.so.5 /lib/x86_64-linux-gnu/libreadline.so.6 /lib/x86_64-linux-gnu/libreadline.so.5.2 /lib/x86_64-linux-gnu/libreadline.so.6.3

make configure would say make: 'configure' is up to date. even if I start with a clean source tree. Did I miss anything in the build instructions? I now deleted configure, did make configure, ./configure, and now waiting for the build to finish ...

I am trying to build Sage 8.9 from source on Linux (Ubuntu 16.04, gcc 6). It fails. I could build Sage 8.8 on the same machine without problems.

It says that python2 and python3 failed to build, and suggests to look at the log files. The log files contain:

ImportError: No module named readline
readline module failed to import

and

ModuleNotFoundError: No module named 'readline'
readline module failed to import

Has anyone experienced the same and is there a simple solution?

Unfortunately, I do not have root access myself, thus I can't quickly try to install the readline dev package.

It is not that trivial to boil down the code to a minimal example. If I put in the work to do that, would you be willing to try to reproduce the problem? Trying to reproduce would also be some trouble: It would involve leaving Sage running for up to a day (or running multiple Sage processes for several hours to increase the likelihood that one of them would stall).