Revision history [back]

Thanks Karl! That did help a lot, and I think I have a workaround. First, to be clear, I think the basic answer to my question is:

"No; a method always implicitly prepends its arguments with self (the class instance it is bound to), and @parallel only works properly if the first argument is a list or tuple. Therefore the two are fundamentally incompatible."

Of course maybe someone clever can think of a way to improve @parallel. I spent some time on this, but I couldn't come up with a reliable way to test whether the first argument was the class instance to which the method was bound.

But I did think of a reasonable workaround: Write a method for the class which will produce the parallelizable function. This function can still take self as its first argument and thereby access any other attributes in the class; the only difference will be that the user will have to explicitly give the class instance as the first argument. Here's a demo:

class PTest(SageObject):
    attr = 'red'
    def meth0(self,n):
        return n
    def generate_parallel_method(self):
        @parallel
        def meth1(self,n):
            "long and complicated class method"
            sleep(2)
            return [prime_pi(n),self.attr]
        return meth1

And here's how it works:

sage: N = PTest()
sage: L = [(N,x) for x in [1000,2000,3000,4000,5000]]
sage: pmeth = N.generate_parallel_method()

sage: pmeth(100)
Traceback (most recent call last)
...
TypeError: meth1() takes exactly 2 arguments (1 given)
sage: pmeth(N,100)
[25, 'red']

sage: r = pmeth(L)
sage: for t in r:
....:     print "%s --> %s"%(t[0],t[1])
....:     
((<class '__main__.PTest'>, 2000), {}) --> [303, 'red']
((<class '__main__.PTest'>, 1000), {}) --> [168, 'red']
((<class '__main__.PTest'>, 3000), {}) --> [430, 'red']
((<class '__main__.PTest'>, 4000), {}) --> [550, 'red']
((<class '__main__.PTest'>, 5000), {}) --> [669, 'red']

Thanks Karl! That did help a lot, and I think I have a workaround. First, to be clear, I think the basic answer to my question is:

"No; a method always implicitly prepends its arguments with self (the class instance it is bound to), and @parallel only works properly if the first argument is a list or tuple. Therefore the two are fundamentally incompatible."

Of course maybe someone clever can think of a way to improve @parallel. I spent some time on this, but I couldn't come up with a reliable way to test whether the first argument was the class instance to which the method was bound. bound.

UPDATE: This is now ticket 11461 :)

But I did think of a reasonable workaround: Write a method for the class which will produce the parallelizable function. This function can still take self as its first argument and thereby access any other attributes in the class; the only difference will be that the user will have to explicitly give the class instance as the first argument. Here's a demo:

class PTest(SageObject):
    attr = 'red'
    def meth0(self,n):
        return n
    def generate_parallel_method(self):
        @parallel
        def meth1(self,n):
            "long and complicated class method"
            sleep(2)
            return [prime_pi(n),self.attr]
        return meth1

And here's how it works:

sage: N = PTest()
sage: L = [(N,x) for x in [1000,2000,3000,4000,5000]]
sage: pmeth = N.generate_parallel_method()

sage: pmeth(100)
Traceback (most recent call last)
...
TypeError: meth1() takes exactly 2 arguments (1 given)
sage: pmeth(N,100)
[25, 'red']

sage: r = pmeth(L)
sage: for t in r:
....:     print "%s --> %s"%(t[0],t[1])
....:     
((<class '__main__.PTest'>, 2000), {}) --> [303, 'red']
((<class '__main__.PTest'>, 1000), {}) --> [168, 'red']
((<class '__main__.PTest'>, 3000), {}) --> [430, 'red']
((<class '__main__.PTest'>, 4000), {}) --> [550, 'red']
((<class '__main__.PTest'>, 5000), {}) --> [669, 'red']