ASKSAGE: Sage Q&A Forum - Individual question feedhttps://ask.sagemath.org/questions/Q&A Forum for SageenCopyright Sage, 2010. Some rights reserved under creative commons license.Tue, 22 Oct 2019 04:29:13 -0500Sage stalls during computationhttps://ask.sagemath.org/question/48270/sage-stalls-during-computation/I was using SageMath 8.8's [feedback_edge_set](http://doc.sagemath.org/html/en/reference/graphs/sage/graphs/digraph.html#sage.graphs.digraph.DiGraph.feedback_edge_set) function on a large dataset. The calculation would take a couple of days to finish.
Unfortunately, Sage would keep stalling every few hours. The CPU usage of the process would simply drop to zero, and it would stop doing anything. I had to kill the process (a simple `kill` worked, no `kill -9` necessary).
Has anyone else experienced this? Is it a known problem? Is there a workaround? It is very frustrating to have to keep manually killing and restarting the process. This problem also prevents me from running a calculation overnight: in the morning I might find that it stopped merely an hour after I started it (if I'm unlucky).
----
More information:
- I was using Sage on Linux
- This function uses [MixedIntegerLinearProgram](http://doc.sagemath.org/html/en/reference/numerical/sage/numerical/mip.html#sage.numerical.mip.MixedIntegerLinearProgram). I used the default GLPK (didn't change settings)
- I used `alarm()` to set a time limit on each computation.
- I always believed GLPK [not to be interruptible](https://lists.gnu.org/archive/html/help-glpk/2006-04/msg00059.html), so I am not even sure which interruption (either manual or alarm) work on Sage with this function. This may be relevant.
- The memory usage of the process stayed low. It did not stall because the memory got filled and the machine started swapping.
----
Minimal example:
Put this in a source file:
import datetime
i=1
while True:
alarm(60)
try:
dg=digraphs.RandomDirectedGNM(300,750);
fes = dg.feedback_edge_set()
print(len(fes))
except AlarmInterrupt:
print("timeout")
cancel_alarm()
print(i)
print(str(datetime.datetime.now()))
print("\n")
i = i+1
Run it as `sage sourcefile.sage`.
In my last test, it took 291 iterations and ~4-5 hours before it stalled.
It appears to stall only when GLPK is used as the MIP solver (i.e. `feedback_edge_set(solver='GLPK')`, which is also the default). If I use `solver='Coin'` (which needs to be installed first using `sage -i cbc`) then it does not stall. However, the calculation is several times slower with Coin than with GLPK.Fri, 11 Oct 2019 02:27:37 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/Comment by tmonteil for <p>I was using SageMath 8.8's <a href="http://doc.sagemath.org/html/en/reference/graphs/sage/graphs/digraph.html#sage.graphs.digraph.DiGraph.feedback_edge_set">feedback_edge_set</a> function on a large dataset. The calculation would take a couple of days to finish.</p>
<p>Unfortunately, Sage would keep stalling every few hours. The CPU usage of the process would simply drop to zero, and it would stop doing anything. I had to kill the process (a simple <code>kill</code> worked, no <code>kill -9</code> necessary).</p>
<p>Has anyone else experienced this? Is it a known problem? Is there a workaround? It is very frustrating to have to keep manually killing and restarting the process. This problem also prevents me from running a calculation overnight: in the morning I might find that it stopped merely an hour after I started it (if I'm unlucky).</p>
<hr>
<p>More information:</p>
<ul>
<li>I was using Sage on Linux</li>
<li>This function uses <a href="http://doc.sagemath.org/html/en/reference/numerical/sage/numerical/mip.html#sage.numerical.mip.MixedIntegerLinearProgram">MixedIntegerLinearProgram</a>. I used the default GLPK (didn't change settings)</li>
<li>I used <code>alarm()</code> to set a time limit on each computation.</li>
<li>I always believed GLPK <a href="https://lists.gnu.org/archive/html/help-glpk/2006-04/msg00059.html">not to be interruptible</a>, so I am not even sure which interruption (either manual or alarm) work on Sage with this function. This may be relevant.</li>
<li>The memory usage of the process stayed low. It did not stall because the memory got filled and the machine started swapping.</li>
</ul>
<hr>
<p>Minimal example:</p>
<p>Put this in a source file:</p>
<pre><code>import datetime
i=1
while True:
alarm(60)
try:
dg=digraphs.RandomDirectedGNM(300,750);
fes = dg.feedback_edge_set()
print(len(fes))
except AlarmInterrupt:
print("timeout")
cancel_alarm()
print(i)
print(str(datetime.datetime.now()))
print("\n")
i = i+1
</code></pre>
<p>Run it as <code>sage sourcefile.sage</code>.</p>
<p>In my last test, it took 291 iterations and ~4-5 hours before it stalled.</p>
<p>It appears to stall only when GLPK is used as the MIP solver (i.e. <code>feedback_edge_set(solver='GLPK')</code>, which is also the default). If I use <code>solver='Coin'</code> (which needs to be installed first using <code>sage -i cbc</code>) then it does not stall. However, the calculation is several times slower with Coin than with GLPK.</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48319#post-id-48319aha, i was trying to idendify the culprit graphs.Sun, 13 Oct 2019 11:00:05 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48319#post-id-48319Comment by Szabolcs for <p>I was using SageMath 8.8's <a href="http://doc.sagemath.org/html/en/reference/graphs/sage/graphs/digraph.html#sage.graphs.digraph.DiGraph.feedback_edge_set">feedback_edge_set</a> function on a large dataset. The calculation would take a couple of days to finish.</p>
<p>Unfortunately, Sage would keep stalling every few hours. The CPU usage of the process would simply drop to zero, and it would stop doing anything. I had to kill the process (a simple <code>kill</code> worked, no <code>kill -9</code> necessary).</p>
<p>Has anyone else experienced this? Is it a known problem? Is there a workaround? It is very frustrating to have to keep manually killing and restarting the process. This problem also prevents me from running a calculation overnight: in the morning I might find that it stopped merely an hour after I started it (if I'm unlucky).</p>
<hr>
<p>More information:</p>
<ul>
<li>I was using Sage on Linux</li>
<li>This function uses <a href="http://doc.sagemath.org/html/en/reference/numerical/sage/numerical/mip.html#sage.numerical.mip.MixedIntegerLinearProgram">MixedIntegerLinearProgram</a>. I used the default GLPK (didn't change settings)</li>
<li>I used <code>alarm()</code> to set a time limit on each computation.</li>
<li>I always believed GLPK <a href="https://lists.gnu.org/archive/html/help-glpk/2006-04/msg00059.html">not to be interruptible</a>, so I am not even sure which interruption (either manual or alarm) work on Sage with this function. This may be relevant.</li>
<li>The memory usage of the process stayed low. It did not stall because the memory got filled and the machine started swapping.</li>
</ul>
<hr>
<p>Minimal example:</p>
<p>Put this in a source file:</p>
<pre><code>import datetime
i=1
while True:
alarm(60)
try:
dg=digraphs.RandomDirectedGNM(300,750);
fes = dg.feedback_edge_set()
print(len(fes))
except AlarmInterrupt:
print("timeout")
cancel_alarm()
print(i)
print(str(datetime.datetime.now()))
print("\n")
i = i+1
</code></pre>
<p>Run it as <code>sage sourcefile.sage</code>.</p>
<p>In my last test, it took 291 iterations and ~4-5 hours before it stalled.</p>
<p>It appears to stall only when GLPK is used as the MIP solver (i.e. <code>feedback_edge_set(solver='GLPK')</code>, which is also the default). If I use <code>solver='Coin'</code> (which needs to be installed first using <code>sage -i cbc</code>) then it does not stall. However, the calculation is several times slower with Coin than with GLPK.</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48316#post-id-48316@tmonteil It's not because of a specific graph. The stalling appears to be random. I know this because I normally run this on a set of pre-computed graphs, so I could easily re-try on the graph where it stalled. Also, running it twice on the same data won't stall at the same step.Sun, 13 Oct 2019 04:59:34 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48316#post-id-48316Comment by tmonteil for <p>I was using SageMath 8.8's <a href="http://doc.sagemath.org/html/en/reference/graphs/sage/graphs/digraph.html#sage.graphs.digraph.DiGraph.feedback_edge_set">feedback_edge_set</a> function on a large dataset. The calculation would take a couple of days to finish.</p>
<p>Unfortunately, Sage would keep stalling every few hours. The CPU usage of the process would simply drop to zero, and it would stop doing anything. I had to kill the process (a simple <code>kill</code> worked, no <code>kill -9</code> necessary).</p>
<p>Has anyone else experienced this? Is it a known problem? Is there a workaround? It is very frustrating to have to keep manually killing and restarting the process. This problem also prevents me from running a calculation overnight: in the morning I might find that it stopped merely an hour after I started it (if I'm unlucky).</p>
<hr>
<p>More information:</p>
<ul>
<li>I was using Sage on Linux</li>
<li>This function uses <a href="http://doc.sagemath.org/html/en/reference/numerical/sage/numerical/mip.html#sage.numerical.mip.MixedIntegerLinearProgram">MixedIntegerLinearProgram</a>. I used the default GLPK (didn't change settings)</li>
<li>I used <code>alarm()</code> to set a time limit on each computation.</li>
<li>I always believed GLPK <a href="https://lists.gnu.org/archive/html/help-glpk/2006-04/msg00059.html">not to be interruptible</a>, so I am not even sure which interruption (either manual or alarm) work on Sage with this function. This may be relevant.</li>
<li>The memory usage of the process stayed low. It did not stall because the memory got filled and the machine started swapping.</li>
</ul>
<hr>
<p>Minimal example:</p>
<p>Put this in a source file:</p>
<pre><code>import datetime
i=1
while True:
alarm(60)
try:
dg=digraphs.RandomDirectedGNM(300,750);
fes = dg.feedback_edge_set()
print(len(fes))
except AlarmInterrupt:
print("timeout")
cancel_alarm()
print(i)
print(str(datetime.datetime.now()))
print("\n")
i = i+1
</code></pre>
<p>Run it as <code>sage sourcefile.sage</code>.</p>
<p>In my last test, it took 291 iterations and ~4-5 hours before it stalled.</p>
<p>It appears to stall only when GLPK is used as the MIP solver (i.e. <code>feedback_edge_set(solver='GLPK')</code>, which is also the default). If I use <code>solver='Coin'</code> (which needs to be installed first using <code>sage -i cbc</code>) then it does not stall. However, the calculation is several times slower with Coin than with GLPK.</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48309#post-id-48309You can provide a short string representation of your graph with:
sage: G.dig6_string()Sat, 12 Oct 2019 16:29:20 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48309#post-id-48309Comment by tmonteil for <p>I was using SageMath 8.8's <a href="http://doc.sagemath.org/html/en/reference/graphs/sage/graphs/digraph.html#sage.graphs.digraph.DiGraph.feedback_edge_set">feedback_edge_set</a> function on a large dataset. The calculation would take a couple of days to finish.</p>
<p>Unfortunately, Sage would keep stalling every few hours. The CPU usage of the process would simply drop to zero, and it would stop doing anything. I had to kill the process (a simple <code>kill</code> worked, no <code>kill -9</code> necessary).</p>
<p>Has anyone else experienced this? Is it a known problem? Is there a workaround? It is very frustrating to have to keep manually killing and restarting the process. This problem also prevents me from running a calculation overnight: in the morning I might find that it stopped merely an hour after I started it (if I'm unlucky).</p>
<hr>
<p>More information:</p>
<ul>
<li>I was using Sage on Linux</li>
<li>This function uses <a href="http://doc.sagemath.org/html/en/reference/numerical/sage/numerical/mip.html#sage.numerical.mip.MixedIntegerLinearProgram">MixedIntegerLinearProgram</a>. I used the default GLPK (didn't change settings)</li>
<li>I used <code>alarm()</code> to set a time limit on each computation.</li>
<li>I always believed GLPK <a href="https://lists.gnu.org/archive/html/help-glpk/2006-04/msg00059.html">not to be interruptible</a>, so I am not even sure which interruption (either manual or alarm) work on Sage with this function. This may be relevant.</li>
<li>The memory usage of the process stayed low. It did not stall because the memory got filled and the machine started swapping.</li>
</ul>
<hr>
<p>Minimal example:</p>
<p>Put this in a source file:</p>
<pre><code>import datetime
i=1
while True:
alarm(60)
try:
dg=digraphs.RandomDirectedGNM(300,750);
fes = dg.feedback_edge_set()
print(len(fes))
except AlarmInterrupt:
print("timeout")
cancel_alarm()
print(i)
print(str(datetime.datetime.now()))
print("\n")
i = i+1
</code></pre>
<p>Run it as <code>sage sourcefile.sage</code>.</p>
<p>In my last test, it took 291 iterations and ~4-5 hours before it stalled.</p>
<p>It appears to stall only when GLPK is used as the MIP solver (i.e. <code>feedback_edge_set(solver='GLPK')</code>, which is also the default). If I use <code>solver='Coin'</code> (which needs to be installed first using <code>sage -i cbc</code>) then it does not stall. However, the calculation is several times slower with Coin than with GLPK.</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48307#post-id-48307Could you please isolate the graph that causes the issue and see if it is reproducible when the method is applied on that graph ?Sat, 12 Oct 2019 14:58:16 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48307#post-id-48307Answer by Szabolcs for <p>I was using SageMath 8.8's <a href="http://doc.sagemath.org/html/en/reference/graphs/sage/graphs/digraph.html#sage.graphs.digraph.DiGraph.feedback_edge_set">feedback_edge_set</a> function on a large dataset. The calculation would take a couple of days to finish.</p>
<p>Unfortunately, Sage would keep stalling every few hours. The CPU usage of the process would simply drop to zero, and it would stop doing anything. I had to kill the process (a simple <code>kill</code> worked, no <code>kill -9</code> necessary).</p>
<p>Has anyone else experienced this? Is it a known problem? Is there a workaround? It is very frustrating to have to keep manually killing and restarting the process. This problem also prevents me from running a calculation overnight: in the morning I might find that it stopped merely an hour after I started it (if I'm unlucky).</p>
<hr>
<p>More information:</p>
<ul>
<li>I was using Sage on Linux</li>
<li>This function uses <a href="http://doc.sagemath.org/html/en/reference/numerical/sage/numerical/mip.html#sage.numerical.mip.MixedIntegerLinearProgram">MixedIntegerLinearProgram</a>. I used the default GLPK (didn't change settings)</li>
<li>I used <code>alarm()</code> to set a time limit on each computation.</li>
<li>I always believed GLPK <a href="https://lists.gnu.org/archive/html/help-glpk/2006-04/msg00059.html">not to be interruptible</a>, so I am not even sure which interruption (either manual or alarm) work on Sage with this function. This may be relevant.</li>
<li>The memory usage of the process stayed low. It did not stall because the memory got filled and the machine started swapping.</li>
</ul>
<hr>
<p>Minimal example:</p>
<p>Put this in a source file:</p>
<pre><code>import datetime
i=1
while True:
alarm(60)
try:
dg=digraphs.RandomDirectedGNM(300,750);
fes = dg.feedback_edge_set()
print(len(fes))
except AlarmInterrupt:
print("timeout")
cancel_alarm()
print(i)
print(str(datetime.datetime.now()))
print("\n")
i = i+1
</code></pre>
<p>Run it as <code>sage sourcefile.sage</code>.</p>
<p>In my last test, it took 291 iterations and ~4-5 hours before it stalled.</p>
<p>It appears to stall only when GLPK is used as the MIP solver (i.e. <code>feedback_edge_set(solver='GLPK')</code>, which is also the default). If I use <code>solver='Coin'</code> (which needs to be installed first using <code>sage -i cbc</code>) then it does not stall. However, the calculation is several times slower with Coin than with GLPK.</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?answer=48449#post-id-48449This looks like a bug in Sage or Sage's GLPK interface.
The practical solution that worked for me was to run the calculation in a separate process using Sage's parallelization features. It can be done by wrapping the feedback arc set calculation with a few function that has the [`@fork` or `@parallel` decorators](http://doc.sagemath.org/html/en/reference/parallel/sage/parallel/decorate.html).
I can confirm that even after a several-day running time, the calculation has not stalled.Mon, 21 Oct 2019 06:37:50 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?answer=48449#post-id-48449Answer by tmonteil for <p>I was using SageMath 8.8's <a href="http://doc.sagemath.org/html/en/reference/graphs/sage/graphs/digraph.html#sage.graphs.digraph.DiGraph.feedback_edge_set">feedback_edge_set</a> function on a large dataset. The calculation would take a couple of days to finish.</p>
<p>Unfortunately, Sage would keep stalling every few hours. The CPU usage of the process would simply drop to zero, and it would stop doing anything. I had to kill the process (a simple <code>kill</code> worked, no <code>kill -9</code> necessary).</p>
<p>Has anyone else experienced this? Is it a known problem? Is there a workaround? It is very frustrating to have to keep manually killing and restarting the process. This problem also prevents me from running a calculation overnight: in the morning I might find that it stopped merely an hour after I started it (if I'm unlucky).</p>
<hr>
<p>More information:</p>
<ul>
<li>I was using Sage on Linux</li>
<li>This function uses <a href="http://doc.sagemath.org/html/en/reference/numerical/sage/numerical/mip.html#sage.numerical.mip.MixedIntegerLinearProgram">MixedIntegerLinearProgram</a>. I used the default GLPK (didn't change settings)</li>
<li>I used <code>alarm()</code> to set a time limit on each computation.</li>
<li>I always believed GLPK <a href="https://lists.gnu.org/archive/html/help-glpk/2006-04/msg00059.html">not to be interruptible</a>, so I am not even sure which interruption (either manual or alarm) work on Sage with this function. This may be relevant.</li>
<li>The memory usage of the process stayed low. It did not stall because the memory got filled and the machine started swapping.</li>
</ul>
<hr>
<p>Minimal example:</p>
<p>Put this in a source file:</p>
<pre><code>import datetime
i=1
while True:
alarm(60)
try:
dg=digraphs.RandomDirectedGNM(300,750);
fes = dg.feedback_edge_set()
print(len(fes))
except AlarmInterrupt:
print("timeout")
cancel_alarm()
print(i)
print(str(datetime.datetime.now()))
print("\n")
i = i+1
</code></pre>
<p>Run it as <code>sage sourcefile.sage</code>.</p>
<p>In my last test, it took 291 iterations and ~4-5 hours before it stalled.</p>
<p>It appears to stall only when GLPK is used as the MIP solver (i.e. <code>feedback_edge_set(solver='GLPK')</code>, which is also the default). If I use <code>solver='Coin'</code> (which needs to be installed first using <code>sage -i cbc</code>) then it does not stall. However, the calculation is several times slower with Coin than with GLPK.</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?answer=48271#post-id-48271I do not know the issue, but as a workaround, you can try:
sage -i cbc
and restart the computation.
Also, could you please provide the whole code so that someone can try to reproduce the issue ?Fri, 11 Oct 2019 03:55:26 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?answer=48271#post-id-48271Comment by tmonteil for <p>I do not know the issue, but as a workaround, you can try:</p>
<pre><code>sage -i cbc
</code></pre>
<p>and restart the computation.</p>
<p>Also, could you please provide the whole code so that someone can try to reproduce the issue ?</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48450#post-id-48450Yes i will, but i have to find some time to confirm your hypothesis.Mon, 21 Oct 2019 06:45:35 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48450#post-id-48450Comment by tmonteil for <p>I do not know the issue, but as a workaround, you can try:</p>
<pre><code>sage -i cbc
</code></pre>
<p>and restart the computation.</p>
<p>Also, could you please provide the whole code so that someone can try to reproduce the issue ?</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48451#post-id-48451Anyway, thanks a lot for your exploration of the problem.Mon, 21 Oct 2019 06:47:06 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48451#post-id-48451Comment by Szabolcs for <p>I do not know the issue, but as a workaround, you can try:</p>
<pre><code>sage -i cbc
</code></pre>
<p>and restart the computation.</p>
<p>Also, could you please provide the whole code so that someone can try to reproduce the issue ?</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48452#post-id-48452@tmonteil Thanks for all the help! I do keep checking back here and I am still interested in understanding what is happening.Mon, 21 Oct 2019 08:53:24 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48452#post-id-48452Comment by tmonteil for <p>I do not know the issue, but as a workaround, you can try:</p>
<pre><code>sage -i cbc
</code></pre>
<p>and restart the computation.</p>
<p>Also, could you please provide the whole code so that someone can try to reproduce the issue ?</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48453#post-id-48453Could you please provide all the pieces of code that lead to your conclusions (about `Glpk` being used despite `Coin` being installed, about the non-working use of `alarm`, about the working use of `@parallel`, etc) ?Mon, 21 Oct 2019 13:32:00 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48453#post-id-48453Comment by Szabolcs for <p>I do not know the issue, but as a workaround, you can try:</p>
<pre><code>sage -i cbc
</code></pre>
<p>and restart the computation.</p>
<p>Also, could you please provide the whole code so that someone can try to reproduce the issue ?</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48459#post-id-48459@tmonteil
Non-working alarm code is in the original post.
That GLPK is being used: this is based on timings, code is in the comments above. You could not reproduce it. I could reproduce it on two different computers. I would suggest: (1) try on a different instance (2) verify how many CPU cores are in use when you benchmark (GLPK: always one; Coin: possibly multiple)Tue, 22 Oct 2019 04:28:48 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48459#post-id-48459Comment by Szabolcs for <p>I do not know the issue, but as a workaround, you can try:</p>
<pre><code>sage -i cbc
</code></pre>
<p>and restart the computation.</p>
<p>Also, could you please provide the whole code so that someone can try to reproduce the issue ?</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48460#post-id-48460@tmonteil
That `@parallel` works:
I cannot run this specific piece of code right now (no spare machine capacity to run for a day), but it should work. It is closely based on what I used these last few days for my actual work.
import datetime
limit = 60 # seconds
@parallel(ncpus=1, p_iter='fork')
def fes_fun(dg):
alarm(limit)
try:
fes = dg.feedback_edge_set()
print(len(fes))
except AlarmInterrupt:
print("timeout")
cancel_alarm()
print(str(datetime.datetime.now()))
print("\n")
fes_fun( (digraphs.RandomDirectedGNM(300,750) for _ in xrange(1000)) )Tue, 22 Oct 2019 04:29:13 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48460#post-id-48460Comment by Szabolcs for <p>I do not know the issue, but as a workaround, you can try:</p>
<pre><code>sage -i cbc
</code></pre>
<p>and restart the computation.</p>
<p>Also, could you please provide the whole code so that someone can try to reproduce the issue ?</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48272#post-id-48272It is not that trivial to boil down the code to a minimal example. If I put in the work to do that, would you be willing to try to reproduce the problem? Trying to reproduce would also be some trouble: It would involve leaving Sage running for up to a day (or running multiple Sage processes for several hours to increase the likelihood that one of them would stall).Fri, 11 Oct 2019 04:00:38 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48272#post-id-48272Comment by tmonteil for <p>I do not know the issue, but as a workaround, you can try:</p>
<pre><code>sage -i cbc
</code></pre>
<p>and restart the computation.</p>
<p>Also, could you please provide the whole code so that someone can try to reproduce the issue ?</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48273#post-id-48273Could you reproduce the problem with a large random digraph ?Fri, 11 Oct 2019 04:57:42 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48273#post-id-48273Comment by tmonteil for <p>I do not know the issue, but as a workaround, you can try:</p>
<pre><code>sage -i cbc
</code></pre>
<p>and restart the computation.</p>
<p>Also, could you please provide the whole code so that someone can try to reproduce the issue ?</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48274#post-id-48274You can choose among different models:
sage: digraphs.Random<TAB_COMPLETION>Fri, 11 Oct 2019 04:59:38 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48274#post-id-48274Comment by Szabolcs for <p>I do not know the issue, but as a workaround, you can try:</p>
<pre><code>sage -i cbc
</code></pre>
<p>and restart the computation.</p>
<p>Also, could you please provide the whole code so that someone can try to reproduce the issue ?</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48290#post-id-48290Sure—I'll get back to you when I have a small example that reliably fails. It'll take time as it usually takes hours for it to fail.Fri, 11 Oct 2019 13:07:27 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48290#post-id-48290Comment by Szabolcs for <p>I do not know the issue, but as a workaround, you can try:</p>
<pre><code>sage -i cbc
</code></pre>
<p>and restart the computation.</p>
<p>Also, could you please provide the whole code so that someone can try to reproduce the issue ?</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48291#post-id-48291Some comments on `sage -i cbc`: this appears to set the `default_mip_solver()` to `"Coin"`, but `feedback_edge_set` still uses `GLPK` by default. I can set it to use `Coin` manually, but GLPK is far faster. GLPK is also much faster than Gurobi when using this specific function. Perhaps this function uses some feature which is GLPK-specific and that's why it'll have better performance with GLPK. I don't know.Fri, 11 Oct 2019 13:18:18 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48291#post-id-48291Comment by Szabolcs for <p>I do not know the issue, but as a workaround, you can try:</p>
<pre><code>sage -i cbc
</code></pre>
<p>and restart the computation.</p>
<p>Also, could you please provide the whole code so that someone can try to reproduce the issue ?</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48305#post-id-48305@tmonteil I added a minimal example. Can you try to reproduce the problem?Sat, 12 Oct 2019 13:04:00 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48305#post-id-48305Comment by tmonteil for <p>I do not know the issue, but as a workaround, you can try:</p>
<pre><code>sage -i cbc
</code></pre>
<p>and restart the computation.</p>
<p>Also, could you please provide the whole code so that someone can try to reproduce the issue ?</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48308#post-id-48308I am running it for 2 hours now.Sat, 12 Oct 2019 15:27:39 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48308#post-id-48308Comment by tmonteil for <p>I do not know the issue, but as a workaround, you can try:</p>
<pre><code>sage -i cbc
</code></pre>
<p>and restart the computation.</p>
<p>Also, could you please provide the whole code so that someone can try to reproduce the issue ?</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48310#post-id-48310I confirm that it stalled after 264 iterations and 3 hours 25.Sat, 12 Oct 2019 17:58:29 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48310#post-id-48310Comment by tmonteil for <p>I do not know the issue, but as a workaround, you can try:</p>
<pre><code>sage -i cbc
</code></pre>
<p>and restart the computation.</p>
<p>Also, could you please provide the whole code so that someone can try to reproduce the issue ?</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48321#post-id-48321Are you sure about your statement on Glpk vs Coin ? With a single test, i got:
sage: dg=digraphs.RandomDirectedGNM(300,750)
sage: %time fes = dg.feedback_edge_set(solver='Coin')
CPU times: user 19min 35s, sys: 9.29 s, total: 19min 45s
Wall time: 19min 39s
sage: %time fes = dg.feedback_edge_set(solver='Glpk')
CPU times: user 49min 23s, sys: 46.1 ms, total: 49min 23s
Wall time: 49min 23s
Perhaps could you try with Coin solver, and tell us if the problem can be reproduced ? It would tell us if the problem is specific to Glpk or not.Sun, 13 Oct 2019 17:13:55 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48321#post-id-48321Comment by David Coudert for <p>I do not know the issue, but as a workaround, you can try:</p>
<pre><code>sage -i cbc
</code></pre>
<p>and restart the computation.</p>
<p>Also, could you please provide the whole code so that someone can try to reproduce the issue ?</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48322#post-id-48322are you sure `feedback_edge_set` uses `GLPK` by default even after installing `cbc`? to check, set `verbose=1`, the displayed information are different from a solver to another.Mon, 14 Oct 2019 01:51:51 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48322#post-id-48322Comment by tmonteil for <p>I do not know the issue, but as a workaround, you can try:</p>
<pre><code>sage -i cbc
</code></pre>
<p>and restart the computation.</p>
<p>Also, could you please provide the whole code so that someone can try to reproduce the issue ?</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48324#post-id-48324I looked to the code, and there is nothing like this.Mon, 14 Oct 2019 05:41:04 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48324#post-id-48324Comment by Szabolcs for <p>I do not know the issue, but as a workaround, you can try:</p>
<pre><code>sage -i cbc
</code></pre>
<p>and restart the computation.</p>
<p>Also, could you please provide the whole code so that someone can try to reproduce the issue ?</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48325#post-id-48325@David Yes, I am sure. If I run `%time g.feedback_edge_set(solver='GLPK')` then the timing is the same as with `%time g.feedback_edge_set()`. If I run `%time g.feedback_edge_set(solver='Coin')` then the timing is several times longer. Therefore, I want to use GLPK for this application.Mon, 14 Oct 2019 05:52:29 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48325#post-id-48325Comment by Szabolcs for <p>I do not know the issue, but as a workaround, you can try:</p>
<pre><code>sage -i cbc
</code></pre>
<p>and restart the computation.</p>
<p>Also, could you please provide the whole code so that someone can try to reproduce the issue ?</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48326#post-id-48326@tmonteil
sage: set_random_seed(1234)
sage: g=digraphs.RandomDirectedGNM(200,500)
sage: %time g.feedback_edge_set(solver='GLPK')
CPU times: user 27.6 s, sys: 28 ms, total: 27.7 s
Wall time: 27.7 s
sage: %time g.feedback_edge_set(solver='Coin')
CPU times: user 1min 19s, sys: 1.25 s, total: 1min 21s
Wall time: 1min 19sMon, 14 Oct 2019 05:56:52 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48326#post-id-48326Comment by Szabolcs for <p>I do not know the issue, but as a workaround, you can try:</p>
<pre><code>sage -i cbc
</code></pre>
<p>and restart the computation.</p>
<p>Also, could you please provide the whole code so that someone can try to reproduce the issue ?</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48327#post-id-48327@tmonteil You are right that the code of `feedback_edge_set` does not seem to select (or mention) GLPK. Maybe `MixedIntegerLinearProgram(solver=None)` initializes to use GLPK by default despite `default_mip_solver()` returning `'Coin'`. I don't know how to check what `MixedIntegerLinearProgram()` has actually initialized to. All I can see is that the timing with `solver=None` is the same as the timing with `solver='GLPK'`.Mon, 14 Oct 2019 06:05:00 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48327#post-id-48327Comment by tmonteil for <p>I do not know the issue, but as a workaround, you can try:</p>
<pre><code>sage -i cbc
</code></pre>
<p>and restart the computation.</p>
<p>Also, could you please provide the whole code so that someone can try to reproduce the issue ?</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48328#post-id-48328OK. Could you reproduce the problem with explicitely set `solver='Coin'` ?Mon, 14 Oct 2019 06:47:53 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48328#post-id-48328Comment by Szabolcs for <p>I do not know the issue, but as a workaround, you can try:</p>
<pre><code>sage -i cbc
</code></pre>
<p>and restart the computation.</p>
<p>Also, could you please provide the whole code so that someone can try to reproduce the issue ?</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48333#post-id-48333@tomteil It's running, I'll let you know. In the meantime, could you please try if `'Coin'` is faster for you on smaller graphs too? I have not found a graph on which `'Coin` would be faster than `'GLPK'`, but I only tried easy ones on which even Coin finishes within 5 min. Your example took much longer than that.Mon, 14 Oct 2019 10:53:59 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48333#post-id-48333Comment by tmonteil for <p>I do not know the issue, but as a workaround, you can try:</p>
<pre><code>sage -i cbc
</code></pre>
<p>and restart the computation.</p>
<p>Also, could you please provide the whole code so that someone can try to reproduce the issue ?</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48335#post-id-48335With your own example:
sage: set_random_seed(1234)
sage: g=digraphs.RandomDirectedGNM(200,500)
sage: %time g.feedback_edge_set(solver='GLPK')
CPU times: user 1min 15s, sys: 16.1 ms, total: 1min 15s
Wall time: 1min 15s
[(6, 100),
(13, 15),
(13, 79),
(16, 79),
...
sage: %time g.feedback_edge_set(solver='Coin')
CPU times: user 1min 4s, sys: 1.07 s, total: 1min 5s
Wall time: 1min 6s
[(0, 13),
(6, 100),
(15, 13),
(16, 79),
(24, 9),
...Mon, 14 Oct 2019 13:31:49 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48335#post-id-48335Comment by Szabolcs for <p>I do not know the issue, but as a workaround, you can try:</p>
<pre><code>sage -i cbc
</code></pre>
<p>and restart the computation.</p>
<p>Also, could you please provide the whole code so that someone can try to reproduce the issue ?</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48339#post-id-48339@tmonteil Wow, that's weird! Do you have any idea what could explain this difference? I verified that only a single CPU core is in use with both. I am using Sage 8.9, built from source without changing any options. The solutions that both GLPK and Coin return seem to be the same ones that you quote.
I also tried the calculation with GLPK on a Mac laptop (where I have not yet managed to install CBC), and it took 20 seconds, which is consistent with the 27 s timing I got on the Linux machine. Why would it be taking a full minute on your computer?
P.S. It does not look like it's going to stall with Coin, but let me keep it running for 24 hrs to be sure.Mon, 14 Oct 2019 14:36:23 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48339#post-id-48339Comment by Szabolcs for <p>I do not know the issue, but as a workaround, you can try:</p>
<pre><code>sage -i cbc
</code></pre>
<p>and restart the computation.</p>
<p>Also, could you please provide the whole code so that someone can try to reproduce the issue ?</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48357#post-id-48357@tomteil 2692 iterations, well over a day, and still going. We can safely say that with Coin it does not stall. However, using Coin would be a significant step-down because its worse performance. I am still puzzled about why there is no performance difference on your computer.Wed, 16 Oct 2019 02:05:55 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48357#post-id-48357Comment by Szabolcs for <p>I do not know the issue, but as a workaround, you can try:</p>
<pre><code>sage -i cbc
</code></pre>
<p>and restart the computation.</p>
<p>Also, could you please provide the whole code so that someone can try to reproduce the issue ?</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48380#post-id-48380@tmonteil Are you aware of any feasible workarounds to this stalling? I am trying to understand how parallelization works in Sage. It appears that it will spawn subprocesses which can be killed after a time limit. I am trying to find out if I can run each computation in a new process this way, and kill the process after the time limit (instead of using `alarm`).Thu, 17 Oct 2019 03:19:31 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48380#post-id-48380Comment by Szabolcs for <p>I do not know the issue, but as a workaround, you can try:</p>
<pre><code>sage -i cbc
</code></pre>
<p>and restart the computation.</p>
<p>Also, could you please provide the whole code so that someone can try to reproduce the issue ?</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48381#post-id-48381@tmonteil I think I figured it out how to get this done with `@parallel`Thu, 17 Oct 2019 04:04:21 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48381#post-id-48381Comment by Szabolcs for <p>I do not know the issue, but as a workaround, you can try:</p>
<pre><code>sage -i cbc
</code></pre>
<p>and restart the computation.</p>
<p>Also, could you please provide the whole code so that someone can try to reproduce the issue ?</p>
https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48386#post-id-48386@tmonteil I finally managed to compile Sage on a Mac, and compared Coin and GLPK again. Based on the timings, it still looks like `solver=None` defaults to GLPK in `feedback_edge_set` but _not_ in `MixedIntergerLinearProgram`. Coin is still slower than GLPK in the total time used, however, now it will sometimes use more than one CPU core, thus the wall time becomes competitive with GLPK (though typically still not faster). I do not know why it does not use multiple cores on Linux.
Since you seem to be involved with Sage, can you file an issue for this (solver=None should not be GLPK) if you think it is appropriate to do so?Thu, 17 Oct 2019 06:38:22 -0500https://ask.sagemath.org/question/48270/sage-stalls-during-computation/?comment=48386#post-id-48386