Ask Your Question
2

Kill the thread in a long computation

asked 2013-05-10 06:40:44 -0500

Moataz Elmasry gravatar image

updated 2013-05-11 10:57:04 -0500

tmonteil gravatar image

Hello

I'm trying to run a long factor() computation and kill it if it took more than 10 seconds. As a simple example to kill a thread in python using signals consider the following code:

def loop_forever():
    import time
    while True:
        time.sleep(1)
    return 1;

def handler(signum, frame):
    raise Exception("end of time")

def timeout(func, args=(), kwargs={}, timeout_duration=10):
    signal.signal(signal.SIGALRM, handler)
    signal.alarm(timeout_duration)
    try:
        func(*args, **kwargs)
    except Exception:
        pass
    signal.alarm(0)

timeout(loop_forever)

This would run fine and the thread would be killed after 10 seconds

If on the other hand instead of calling loop_forever() I call

def crackKey():
    return factor(Primes().next(2^400) * Primes().next(2^500))

timeout(crackKey)

The thread is not killed after 10 seconds and it continues the computation Any idea why this is happening?

cheers

edit retag flag offensive close merge delete

2 answers

Sort by ยป oldest newest most voted
2

answered 2013-05-10 22:31:38 -0500

tmonteil gravatar image

updated 2013-05-13 10:02:08 -0500

In my previous answer, i guessed that the problem comes from cython, that does not handle the Python signal correctly, as if it runs too fast to hear it ;) Here is a workaround: the timetout() function should operate at a lower level. We will launch the computation as a new process, and kill the process after timeout_duration. The kill signal is not sent at the Python level, but at the kernel level.

To compute a function in a separate process, Sage has a @fork decorator, here is how to use it in your framework:

sage: def timeout(func, args=(), kwargs={}, timeout_duration=10):
....:    @fork(timeout=timeout_duration, verbose=True)
....:    def my_new_func():
....:        return func(*args, **kwargs)
....:    return my_new_func()

Now you can see that both

sage: timeout(loop_forever)

and

sage: timeout(my_func)

stop after time_duration !

edit flag offensive delete link more

Comments

wow it works!!! brilliant. A noob like me didn't realize the existence of these decorations at all. many thanks

Moataz Elmasry gravatar imageMoataz Elmasry ( 2013-05-11 00:31:20 -0500 )edit
1

Tiny remark: @fork creates another *process*, not another thread.

William Stein gravatar imageWilliam Stein ( 2013-05-11 13:20:15 -0500 )edit

You are right, i corrected this.

tmonteil gravatar imagetmonteil ( 2013-05-13 10:02:43 -0500 )edit
1

answered 2013-05-10 09:35:17 -0500

tmonteil gravatar image

updated 2013-05-10 10:00:50 -0500

The first reason why this does not work is that the expression factor(Primes().next(2^400)*Primes().next(2^500)) is an object of the class sage.structure.factorization_integer.IntegerFactorization, but not a function. Hence, if you send this expression to the timeout() function, it will be computed before being sent to the timeout() function, which won't kill the computation. Hence, you will experience the same problem as if you did:

sage: timeout(loop_forever())

Which will loop infinitely. So, you should do something like:

sage: def my_func():
....:     return factor(Primes().next(2^400) * Primes().next(2^500))
sage: timeout(my_func)

And, surprise, it doesn't work better ! Here is a possible reason. As you can check by typing:

sage: a = 123
sage: a.factor??

the factor() function is made of cython (see the use of cdef). So this may be the problem, let's investigate this track further. For that, open a notebook and create one cell with:

%cython
def loop_forever():
    import time
    while True:
        time.sleep(1)
    return 1;

(this will create the loop_forever() function as a cython one), and another cell with:

import signal

def handler(signum, frame):
    raise Exception("end of time")

def timeout(func, args=(), kwargs={}, timeout_duration=10):
    signal.signal(signal.SIGALRM, handler)
    signal.alarm(timeout_duration)
    try:
        func(*args, **kwargs)
    except Exception:
        pass
    signal.alarm(0)

Then try:

timeout(loop_forever)

As you can see, the computation does not stop either ! I guess cython is the reason why your problem happened.

edit flag offensive delete link more

Comments

Thanks for the investigation. I guess I prefer to avoid the notebook for now, since it adds a level of complexity while tracking the problem. As I mentioned loop forever can be killed in a sage session in the terminal in contrast to the factor() function. In reality the call to factor is never made directly from the timeout() but through another function that calls factor() and is itself passed to the timeout()

Moataz Elmasry gravatar imageMoataz Elmasry ( 2013-05-10 10:43:36 -0500 )edit
1

I proposed to use the notebook to ease the use of the `%cython` markup (not available in the Sage command line), see [this page](http://www.sagemath.org/doc/developer/coding_in_cython.html#writing-cython-code-in-sage). You can reproduce the investigation completely from the Sage command line by doing the following: Create a file named `/tmp/loop.spyx` and add the following content to it: def loop_forever(): import time while True: time.sleep(1) return 1; Then in the Sage command line, do sage: %attach /tmp/loop.spyx Compiling /tmp/loop.spyx... and do your test. You will see that `timeout(loop_forever)` will not stop either !

tmonteil gravatar imagetmonteil ( 2013-05-10 10:56:06 -0500 )edit

ahh I see. I updated the question to reflect a correct code

Moataz Elmasry gravatar imageMoataz Elmasry ( 2013-05-10 23:25:00 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

Stats

Asked: 2013-05-10 06:40:44 -0500

Seen: 1,738 times

Last updated: May 13 '13