Experimenting with @parallel resulted in unexpected performance issues in Sage 6.4.1. Here is a very simple example:
@parallel(p_iter='multiprocessing', ncpus=6) def f(n): return factor(n) t=walltime() r = range(1,1000000) p = sorted(list( f(r))) print walltime(t) 82.0724880695
t=walltime() for i in range(1,1000000): factor(i) print walltime(t) 12.1648099422
I have 6 physical cores, yet the serial calculation runs more than 6 times faster, even though I can see 6 instances of python running on my computer. Maybe it is pilot error, I have the following questions: 1) Does Sage require a special way of compiling it in order to take full advantage of @parallel? 2) In this case using 'fork' is even worse, it never completes the calculation. 3) How does @parallel distribute the calculations? Since, in general, it takes significantly longer for factor() to process larger numbers, it seems that assigning the case n=1,7,13,... to core_0, n=2,8,14,... to core_1, etc., makes sense. Shuffling the original serial list given to f(n) also seems plausible. However, dividing the whole serial range to 6 intervals and assigning them to the 6 cores, respectively, would be a bad choice and for most of the time only one or two python processes would do anything. Does anyone know what scheme is used in Sage?
Thanks for any suggestions.
