owari's profile - activity

Hello all,

It appears to me that Python has a number of tools to work with multi cores/nodes for parallel calculations: multiprocessing, MPI for Python, parallel Python, Celery, Scoop and etc. (If I am right!). However, right now I am living inside Sage and see the Python world from its sight, and from Sage I see this question asked in the present forum two years before, wherein W.Stein explains that "The Sage roadmap, such as it is, doesn't address parallel computing" and "Check out the multiprocessing module in Python but watch out -- sometimes you really have to understand how certain things in Sage works to safely use it". I am almost new to Sage and don't know really what's going on inside it, and also I have a number (I mean really great number) of functions over which some heavy symbolic calculations (integrations and etc.) are to be applied, however, the calculations on each function is independent of the others, so that I was thinking about now that Sage has both Python and Maxima included in one place maybe I can handle independent tasks of the symbolic world using the paralleling facilities of the Python world?

Is it possible or you may suggest SymPy instead? I already have a code inside Sage so prefer to continue living inside Sage but I just doubt if it is as compatible with Python as might be SymPy (?), also I am not sure if SymPy (which has not reached its first stable release yet) would be as strong in calculations as is Maxima in Sage ?

Any guide would be appreciated,

Best Regards

Hi,

What's the difference between the method notation and function notation? Is it possible to define a differential or integral operators using either of these notations or else?

I'm a beginner, just tried a bit codes like the followings

reset
var('x,a,b')
L='diff(x,a,b)'
f=function('f',x)
f.L

or

reset
var('x,a,b,c')
L={c:'diff(x,a,b)'}    # defining a dicionary
f=function('f',x)
f.L[c]

and codes like these … but none worked. Any idea?

I was trying to define a function of form f_{i_1,i_2,i_3,i_4}(x,y,z) using the command

function('f_{i_1,i_2,i_3,i_4}',x,y,z)

but sage gave me back the error that 'tuple' object is not callable. I can define it like

function('f_{i_1}_{i_2}_{i_3}_{i_4}',x,y,z)

but it is not a good way when the indices are many … or maybe even I should use

function('f_i_1_i_2_i_3_i_4',x,y,z)

instead?

There are two comments in order here:

A) as you state yourself, the integrands of your expression, e.g.

U1*phi1[0]
y |--> (y - 1)*((y - 1)*(y + 1)*a12*y + (y - 1)*(y^2 + 1)*a13*y + (y - 1)*a11*y)*y

contains lots of symbolic varibles: a12,a13,a11 which are no valid floats, even after substituting a valid float for the integration variable y. That's what the error tells you ;-) . SciPy's quad has to know the numerical values of such parameters - how else could it do a numerical integral? How to supply float parameters to quad is described at http://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.quad.html#scipy.integrate.quad E.g.:

f(y,a12,a13,a11)=U1*phi1[0]
quad(f,0.0,1.0,args=(1,1,1,))
(0.1261904761904762, 1.4009957215507927e-15)

B) having said that, and not knowing what your actual problem is, I however presume that the preceeding is not what you actually want, and that rather you want to set up some kind of linear equation (eq1 ?) with symbolic variables and numerically evaluated 'constants'. If that is so, I would disect the numerical integration of each of those constant off from the symbolic part of the code.

very true, thank you ...

I have a simple code that seems to work fine (although I cannot be sure when there are several constants like a13 that seem to be integration constants while the integrals are all definite?), at least runs with no error, when the symbolic "integral" function is used instead of "quad", but now using "quad" the code runs into error. This is an example code:

reset()
forget()

from scipy.integrate import quad

var('y')
var('kx')

alpha1=var('a11,a12,a13')

phi11(y)=y*(y-1);    phi12(y)=y*(y-1)*(y+1);    phi13(y)=y*(y-1)*(y^2+1)

phi1=[phi11,phi12,phi13]

U1(y)=sum(alpha1[j]*phi1[j](y) for j in range(3))

eq1=[
     quad(U1.diff(y,y)*phi1[j],0.0,1.0)[0]
     -kx*quad(y*U1*phi1[j],0.0,1.0)[0]
     -kx^2*quad(U1*phi1[j],0.0,1.0)[0]
     for j in range(3)]

print eq1

The error is:

quadpack.error: Supplied function does not return a valid float.

Any idea how to resolve it? I have tested many different usages of the functions inside the argument of quad, using lambda, explicitly specifying (y) as the argument of the functions, defining a function Y(y)=y so to use Y instead of y in writing the expression "yU1phi1[j]", and etc but all with no success. Searching the web also didn't help much. Please bare with me I am not an expert.

Spawned from this question and the answer given to it I wrote a piece of code using the functools.partial() but it doesn't work properly. The code is:

var('x,y,z,t,x1,y1,z1,t1,var1')
U=function('U0',x,y,z,t,x1,y1,z1,t1)
U0(x,y,z,t,x1,y1,z1,t1)=sin(x-x1)*sin(y-y1)*sin(z-z1)*sin(t-t1)
    # if write U0=... instead of U0(...)=... the order of arguments of U0 is not
    # specified and e.g. sin(x-x1) might become sin(x-y) from the system's viewpoint

num=2.0
def numint(func, var1, h):
    #global var1
    integ = 0.5*h*(func(var1=0) + func(var1=1))
    for i in range(1,num):
        integ = integ + h * func(var1=i*h)
    return integ

from functools import partial
numericint = partial(numint, h=1/num)


print numericint(U0,var1=t)+numericint(U0,var1=t1)

Any idea why it gives 2*sin(z - z1)*sin(y - y1)*sin(x - x1)*sin(t - t1) as output instead of integrating U0 once with respect to t and once with respect to t1?

thanks.

Using Sage the following simplified code is to recursively compute two functions starting from their initial estimations. When I try to introduce a "numerical integral operator" to the code, instead of using the built-in functions, it gives an error:

PicklingError: Can't pickle <type 'function'="">: attribute lookup __ builtin__.function failed

the code is:

reset()
forget()
from multiprocessing import Pool, cpu_count

#variables
var('x,y, x1,y1')
N=5   #number of iterations
var('q')

#functions' name definition (without specifying the rules)  # n is the level of estimation
U0=[]
U1=[]
for n in range(N+1):
    U0.append(function('U0_%s' %i, x,y))
    U1.append(function('U1_%s' %i, x,y, x1,y1))

#initial estimation of the functions
U0[0]=sin(x+y)
U1[0]=sin(x1+y1)*cos(x-y)


#numerical integrator
num=20.0  # at each call of the int function h is assigned (b-a)/num, so assigned to num here
def numint(f, x, a, b, h):
    #pickle_function(f)
    integ = 0.5*h*(f(x=a) + f(x=b))
    for i in range(1,num):
        integ = integ + h * f(x=a+i*h)
    return integ


#the integral operators
def Ix(f,x):
    return pool.apply( numint, (f,x,-1.0,+1.0,2.0/num) )
def Iy(f,y):
    return pool.apply_async( numint, (f,y,-1.0,+1.0,2.0/num) )
def IR(f,x,y):
    return Iy(Ix(f,x),y)
def N0(n,f0,f1):
    return f0[n] + IR(f1[n],x1,y1) + IR(IR(f1[n],x,y),x1,y1)
def N1(n,f0,f1):
    return f1[n] + IR(f0[n],x,y) - IR(IR(f1[n],x,y),x1,y1)


#the calculations
pool = Pool(processes=cpu_count())
for n in range(N):
    worker0 = N0(n,U0,U1)
    worker1 = N1(n,U0,U1)
    U0[n+1] = U0[n] - worker0.get()
    U1[n+1] = U1[n] - worker1.get()
    show(U0[n+1])
    show(U1[n+1])

As far as I have understood from searching the web and reading the documents this pickling error is either due to the integral operator (a function anyway) not being Picklable or due to its arguments not being picklable. I tried to make the operator itself picklable at no success, but I guess the problem should be due to the integral operator itself being a functional of its integrand, and as it is an operator the function in its integrand is not to be determined since the very beginning, so maybe the first argument of the operator (which itself is a function) not being clearly defined at top level is the reason for the operator not being in overall picklable? Any idea how to overcome this error?

NOTE. the main code is much more complex than the minimalistic code provided here, so I rather need to define an integral operator to prevent the code's readability.

And have you suggestion on which method is best to do so? Thanks

I don't think it be easy to manage parallel processing for computing also each heavy symbolic integration, existing inside some series within each function-update during each step of the iteration loop, but if it is then it may help reducing the time consumption by the code very considerably! [My real code, albeit in its simplest and smallest version with 12 functions being iterated for, is the one I linked to above in my question, if it may help to see also its details.]

If your problem is trivially parallel (independent tasks) then you can just use the @parallel decorator:

sage: @parallel
....: def f(x):
....:     return x^2
....: 
sage: list(f(range(10)))
[(((0,), {}), 0), (((1,), {}), 1), (((2,), {}), 4), (((3,), {}), 9), (((4,), {}), 16), (((5,), {}), 25), (((6,), {}), 36), (((7,), {}), 49), (((8,), {}), 64), (((9,), {}), 81)]

This will use all cores of your computer to work in parallel. Of course you can use any of the other Python tools you mentioned to distribute your work tasks.

Ok, suppose I have an iteration loop during each iteration a number of functions being updated within some expressions containing a number of symbolic differentiation and integration. These updating processes are not coupled, so that inside each step of the loop there are plenty of functions that stand in a queue to be updated, one after another, each being considerably time consuming, whereas each of them can be updated exploiting one cpu core/node cluster so that the whole updating step occurs in a time of the order of one function update time. This was, if the number of functions to be updated inside the loop is small such a parallelization will be useless but if they are many then it will be a big advantage to use a parallel algorithm.