ASKSAGE: Sage Q&A Forum - RSS feedhttps://ask.sagemath.org/questions/Q&A Forum for SageenCopyright Sage, 2010. Some rights reserved under creative commons license.Wed, 08 Nov 2017 00:36:51 +0100How do I set solver_parameter to make Gurobi use more than one processor?https://ask.sagemath.org/question/37726/how-do-i-set-solver_parameter-to-make-gurobi-use-more-than-one-processor/As [suggested previously](https://ask.sagemath.org/question/25819) by Nathann Cohen, the method `solver_parameter` allows to do such a thing
sage: p = MixedIntegerLinearProgram(solver = "Gurobi")
sage: p.solver_parameter?
In the [documentation](http://doc.sagemath.org/html/en/reference/numerical/sage/numerical/mip.html#sage.numerical.mip.MixedIntegerLinearProgram.solver_parameter) that appears, it says:
> Gurobi’s parameters should all be
> available through this method. Their
> list is available on Gurobi’s website
> http://www.gurobi.com/documentation/5.5/reference-manual/node798.
but this link is broken and leads us to http://www.gurobi.com/documentation/ instead. How do I set solver_parameter to make Gurobi use more than one processor?Mon, 29 May 2017 13:29:29 +0200https://ask.sagemath.org/question/37726/how-do-i-set-solver_parameter-to-make-gurobi-use-more-than-one-processor/Answer by mforets for <p>As <a href="https://ask.sagemath.org/question/25819">suggested previously</a> by Nathann Cohen, the method <code>solver_parameter</code> allows to do such a thing </p>
<pre><code>sage: p = MixedIntegerLinearProgram(solver = "Gurobi")
sage: p.solver_parameter?
</code></pre>
<p>In the <a href="http://doc.sagemath.org/html/en/reference/numerical/sage/numerical/mip.html#sage.numerical.mip.MixedIntegerLinearProgram.solver_parameter">documentation</a> that appears, it says: </p>
<blockquote>
<p>Gurobi’s parameters should all be
available through this method. Their
list is available on Gurobi’s website
<a href="http://www.gurobi.com/documentation/5.5/reference-manual/node798">http://www.gurobi.com/documentation/5...</a>.</p>
</blockquote>
<p>but this link is broken and leads us to <a href="http://www.gurobi.com/documentation/">http://www.gurobi.com/documentation/</a> instead. How do I set solver_parameter to make Gurobi use more than one processor?</p>
https://ask.sagemath.org/question/37726/how-do-i-set-solver_parameter-to-make-gurobi-use-more-than-one-processor/?answer=37817#post-id-37817The list of available parameters for Gurobi 7.0 is available in [this link](https://www.gurobi.com/documentation/7.0/refman/parameters.html). (true that Sage help should better point to [the general doc](https://www.gurobi.com/documentation/), because the previous link changes among versions..)
The parameter that you are interested in seems to be [Threads](https://www.gurobi.com/documentation/7.0/refman/threads.html), which controls the number of threads to apply to parallel algorithms. Apparently Gurobi already uses, by default, all available ones (see in the Threads description, or [here](https://groups.google.com/forum/#!topic/gurobi/7Q0s080QDi8)). In any case, there are also other parameters to control parallelism.
So for example we can do:
sage: p = MixedIntegerLinearProgram(solver="Gurobi")
sage: p.solver_parameter("Threads") # query the actual value
0
sage: p.solver_parameter("Threads", 4) # set a new value
sage: p.solver_parameter("Threads") # query the new value
4
In Sage, the allowed solver parameters for the Gurobi backend are listed in the dictionary [parameters_type](https://github.com/sagemath/sage/blob/master/src/sage/numerical/backends/gurobi_backend.pyx#L1223) in `gurobi_backend.pyx`. Be careful because since keywords may be added or removed from version to version of Gurobi, the dictionary is not necessarily up-to-date and may brake, but as Nathan explains in that file, it is a matter of changing the source code and recompiling sage! (`make start`)Tue, 06 Jun 2017 20:45:40 +0200https://ask.sagemath.org/question/37726/how-do-i-set-solver_parameter-to-make-gurobi-use-more-than-one-processor/?answer=37817#post-id-37817Comment by Sébastien for <p>The list of available parameters for Gurobi 7.0 is available in <a href="https://www.gurobi.com/documentation/7.0/refman/parameters.html">this link</a>. (true that Sage help should better point to <a href="https://www.gurobi.com/documentation/">the general doc</a>, because the previous link changes among versions..)</p>
<p>The parameter that you are interested in seems to be <a href="https://www.gurobi.com/documentation/7.0/refman/threads.html">Threads</a>, which controls the number of threads to apply to parallel algorithms. Apparently Gurobi already uses, by default, all available ones (see in the Threads description, or <a href="https://groups.google.com/forum/#!topic/gurobi/7Q0s080QDi8">here</a>). In any case, there are also other parameters to control parallelism.</p>
<p>So for example we can do:</p>
<pre><code>sage: p = MixedIntegerLinearProgram(solver="Gurobi")
sage: p.solver_parameter("Threads") # query the actual value
0
sage: p.solver_parameter("Threads", 4) # set a new value
sage: p.solver_parameter("Threads") # query the new value
4
</code></pre>
<p>In Sage, the allowed solver parameters for the Gurobi backend are listed in the dictionary <a href="https://github.com/sagemath/sage/blob/master/src/sage/numerical/backends/gurobi_backend.pyx#L1223">parameters_type</a> in <code>gurobi_backend.pyx</code>. Be careful because since keywords may be added or removed from version to version of Gurobi, the dictionary is not necessarily up-to-date and may brake, but as Nathan explains in that file, it is a matter of changing the source code and recompiling sage! (<code>make start</code>)</p>
https://ask.sagemath.org/question/37726/how-do-i-set-solver_parameter-to-make-gurobi-use-more-than-one-processor/?comment=37924#post-id-37924It does not seem to work, the command `p.solve()` below uses only one thread:
sage: p
Mixed Integer Program ( maximization, 180000 variables, 178940 constraints )
sage: p.get_backend()
<type 'sage.numerical.backends.gurobi_backend.GurobiBackend'>
sage: p.solver_parameter('Threads',8)
sage: %time p.solve()
CPU times: user 12.6 s, sys: 0 ns, total: 12.6 s
Wall time: 12.6 s
1.0
But, I just realized that the bottleneck of my problem is the creation of the object p because calling `add_constraint` hundreds of thousands of times takes much more time (some minutes) than the one call to `p.solve()` (some seconds).Tue, 13 Jun 2017 11:39:01 +0200https://ask.sagemath.org/question/37726/how-do-i-set-solver_parameter-to-make-gurobi-use-more-than-one-processor/?comment=37924#post-id-37924Comment by mforets for <p>The list of available parameters for Gurobi 7.0 is available in <a href="https://www.gurobi.com/documentation/7.0/refman/parameters.html">this link</a>. (true that Sage help should better point to <a href="https://www.gurobi.com/documentation/">the general doc</a>, because the previous link changes among versions..)</p>
<p>The parameter that you are interested in seems to be <a href="https://www.gurobi.com/documentation/7.0/refman/threads.html">Threads</a>, which controls the number of threads to apply to parallel algorithms. Apparently Gurobi already uses, by default, all available ones (see in the Threads description, or <a href="https://groups.google.com/forum/#!topic/gurobi/7Q0s080QDi8">here</a>). In any case, there are also other parameters to control parallelism.</p>
<p>So for example we can do:</p>
<pre><code>sage: p = MixedIntegerLinearProgram(solver="Gurobi")
sage: p.solver_parameter("Threads") # query the actual value
0
sage: p.solver_parameter("Threads", 4) # set a new value
sage: p.solver_parameter("Threads") # query the new value
4
</code></pre>
<p>In Sage, the allowed solver parameters for the Gurobi backend are listed in the dictionary <a href="https://github.com/sagemath/sage/blob/master/src/sage/numerical/backends/gurobi_backend.pyx#L1223">parameters_type</a> in <code>gurobi_backend.pyx</code>. Be careful because since keywords may be added or removed from version to version of Gurobi, the dictionary is not necessarily up-to-date and may brake, but as Nathan explains in that file, it is a matter of changing the source code and recompiling sage! (<code>make start</code>)</p>
https://ask.sagemath.org/question/37726/how-do-i-set-solver_parameter-to-make-gurobi-use-more-than-one-processor/?comment=37927#post-id-37927how do you know that the program is using only one thread? is it because the time doesn't change if you change the `Thread` parameter?Tue, 13 Jun 2017 13:49:17 +0200https://ask.sagemath.org/question/37726/how-do-i-set-solver_parameter-to-make-gurobi-use-more-than-one-processor/?comment=37927#post-id-37927Comment by mforets for <p>The list of available parameters for Gurobi 7.0 is available in <a href="https://www.gurobi.com/documentation/7.0/refman/parameters.html">this link</a>. (true that Sage help should better point to <a href="https://www.gurobi.com/documentation/">the general doc</a>, because the previous link changes among versions..)</p>
<p>The parameter that you are interested in seems to be <a href="https://www.gurobi.com/documentation/7.0/refman/threads.html">Threads</a>, which controls the number of threads to apply to parallel algorithms. Apparently Gurobi already uses, by default, all available ones (see in the Threads description, or <a href="https://groups.google.com/forum/#!topic/gurobi/7Q0s080QDi8">here</a>). In any case, there are also other parameters to control parallelism.</p>
<p>So for example we can do:</p>
<pre><code>sage: p = MixedIntegerLinearProgram(solver="Gurobi")
sage: p.solver_parameter("Threads") # query the actual value
0
sage: p.solver_parameter("Threads", 4) # set a new value
sage: p.solver_parameter("Threads") # query the new value
4
</code></pre>
<p>In Sage, the allowed solver parameters for the Gurobi backend are listed in the dictionary <a href="https://github.com/sagemath/sage/blob/master/src/sage/numerical/backends/gurobi_backend.pyx#L1223">parameters_type</a> in <code>gurobi_backend.pyx</code>. Be careful because since keywords may be added or removed from version to version of Gurobi, the dictionary is not necessarily up-to-date and may brake, but as Nathan explains in that file, it is a matter of changing the source code and recompiling sage! (<code>make start</code>)</p>
https://ask.sagemath.org/question/37726/how-do-i-set-solver_parameter-to-make-gurobi-use-more-than-one-processor/?comment=37930#post-id-37930for the second issue, this experiment seems to indicate that it's more efficient to push constraints "directly in the backend" :
sage: p = MixedIntegerLinearProgram(solver='Gurobi')
sage: x = p.new_variable()
sage: timeit('p.add_constraint(x[0] + 0.2*x[1] <= 4)')
625 loops, best of 3: 53.1 µs per loop
sage: p = MixedIntegerLinearProgram(solver='Gurobi')
sage: pb = p.get_backend()
sage: pb.add_variables(2)
sage: timeit("pb.add_linear_constraint(zip(range(2), [1.0, 0.2]), None, 4.0)")
625 loops, best of 3: 21.1 µs per loopTue, 13 Jun 2017 15:52:24 +0200https://ask.sagemath.org/question/37726/how-do-i-set-solver_parameter-to-make-gurobi-use-more-than-one-processor/?comment=37930#post-id-37930Comment by Sébastien for <p>The list of available parameters for Gurobi 7.0 is available in <a href="https://www.gurobi.com/documentation/7.0/refman/parameters.html">this link</a>. (true that Sage help should better point to <a href="https://www.gurobi.com/documentation/">the general doc</a>, because the previous link changes among versions..)</p>
<p>The parameter that you are interested in seems to be <a href="https://www.gurobi.com/documentation/7.0/refman/threads.html">Threads</a>, which controls the number of threads to apply to parallel algorithms. Apparently Gurobi already uses, by default, all available ones (see in the Threads description, or <a href="https://groups.google.com/forum/#!topic/gurobi/7Q0s080QDi8">here</a>). In any case, there are also other parameters to control parallelism.</p>
<p>So for example we can do:</p>
<pre><code>sage: p = MixedIntegerLinearProgram(solver="Gurobi")
sage: p.solver_parameter("Threads") # query the actual value
0
sage: p.solver_parameter("Threads", 4) # set a new value
sage: p.solver_parameter("Threads") # query the new value
4
</code></pre>
<p>In Sage, the allowed solver parameters for the Gurobi backend are listed in the dictionary <a href="https://github.com/sagemath/sage/blob/master/src/sage/numerical/backends/gurobi_backend.pyx#L1223">parameters_type</a> in <code>gurobi_backend.pyx</code>. Be careful because since keywords may be added or removed from version to version of Gurobi, the dictionary is not necessarily up-to-date and may brake, but as Nathan explains in that file, it is a matter of changing the source code and recompiling sage! (<code>make start</code>)</p>
https://ask.sagemath.org/question/37726/how-do-i-set-solver_parameter-to-make-gurobi-use-more-than-one-processor/?comment=37933#post-id-37933I conclude Gurobi is not using all processors and using only one thread by looking at `htop` during the execution. Coin backend uses all processors (by looking at htop), but it is also much slower and just can't solve the above problem. In the end, I am quite impressed by what Gurobi can do with only one processor but I am still wondering if it can do even better:)Tue, 13 Jun 2017 22:48:21 +0200https://ask.sagemath.org/question/37726/how-do-i-set-solver_parameter-to-make-gurobi-use-more-than-one-processor/?comment=37933#post-id-37933Comment by Sébastien for <p>The list of available parameters for Gurobi 7.0 is available in <a href="https://www.gurobi.com/documentation/7.0/refman/parameters.html">this link</a>. (true that Sage help should better point to <a href="https://www.gurobi.com/documentation/">the general doc</a>, because the previous link changes among versions..)</p>
<p>The parameter that you are interested in seems to be <a href="https://www.gurobi.com/documentation/7.0/refman/threads.html">Threads</a>, which controls the number of threads to apply to parallel algorithms. Apparently Gurobi already uses, by default, all available ones (see in the Threads description, or <a href="https://groups.google.com/forum/#!topic/gurobi/7Q0s080QDi8">here</a>). In any case, there are also other parameters to control parallelism.</p>
<p>So for example we can do:</p>
<pre><code>sage: p = MixedIntegerLinearProgram(solver="Gurobi")
sage: p.solver_parameter("Threads") # query the actual value
0
sage: p.solver_parameter("Threads", 4) # set a new value
sage: p.solver_parameter("Threads") # query the new value
4
</code></pre>
<p>In Sage, the allowed solver parameters for the Gurobi backend are listed in the dictionary <a href="https://github.com/sagemath/sage/blob/master/src/sage/numerical/backends/gurobi_backend.pyx#L1223">parameters_type</a> in <code>gurobi_backend.pyx</code>. Be careful because since keywords may be added or removed from version to version of Gurobi, the dictionary is not necessarily up-to-date and may brake, but as Nathan explains in that file, it is a matter of changing the source code and recompiling sage! (<code>make start</code>)</p>
https://ask.sagemath.org/question/37726/how-do-i-set-solver_parameter-to-make-gurobi-use-more-than-one-processor/?comment=39430#post-id-39430Today I used Gurobi for another problem, and it did use 4 processors in parallel for doing the computation. So, I think Gurobi do choices on parallel computation inline depending on the problem. So, I am now marking the answer given by mforest as correct:)Tue, 07 Nov 2017 21:48:43 +0100https://ask.sagemath.org/question/37726/how-do-i-set-solver_parameter-to-make-gurobi-use-more-than-one-processor/?comment=39430#post-id-39430Comment by mforets for <p>The list of available parameters for Gurobi 7.0 is available in <a href="https://www.gurobi.com/documentation/7.0/refman/parameters.html">this link</a>. (true that Sage help should better point to <a href="https://www.gurobi.com/documentation/">the general doc</a>, because the previous link changes among versions..)</p>
<p>The parameter that you are interested in seems to be <a href="https://www.gurobi.com/documentation/7.0/refman/threads.html">Threads</a>, which controls the number of threads to apply to parallel algorithms. Apparently Gurobi already uses, by default, all available ones (see in the Threads description, or <a href="https://groups.google.com/forum/#!topic/gurobi/7Q0s080QDi8">here</a>). In any case, there are also other parameters to control parallelism.</p>
<p>So for example we can do:</p>
<pre><code>sage: p = MixedIntegerLinearProgram(solver="Gurobi")
sage: p.solver_parameter("Threads") # query the actual value
0
sage: p.solver_parameter("Threads", 4) # set a new value
sage: p.solver_parameter("Threads") # query the new value
4
</code></pre>
<p>In Sage, the allowed solver parameters for the Gurobi backend are listed in the dictionary <a href="https://github.com/sagemath/sage/blob/master/src/sage/numerical/backends/gurobi_backend.pyx#L1223">parameters_type</a> in <code>gurobi_backend.pyx</code>. Be careful because since keywords may be added or removed from version to version of Gurobi, the dictionary is not necessarily up-to-date and may brake, but as Nathan explains in that file, it is a matter of changing the source code and recompiling sage! (<code>make start</code>)</p>
https://ask.sagemath.org/question/37726/how-do-i-set-solver_parameter-to-make-gurobi-use-more-than-one-processor/?comment=39439#post-id-39439alright! in the second comment i was playing with different ways of creating your object `p`, and it seemed that it's more efficient to use `add_variables` to the backend, in this particular case it was like a factor of 2 faster, so it can be relevant if you have a huge number of constraints. although i don't know if this observation is general or just a particular case.. best, MarceloWed, 08 Nov 2017 00:36:51 +0100https://ask.sagemath.org/question/37726/how-do-i-set-solver_parameter-to-make-gurobi-use-more-than-one-processor/?comment=39439#post-id-39439