Ask Your Question

doegox's profile - activity

2013-07-18 05:30:06 +0200 received badge  Supporter (source)
2013-07-18 05:29:48 +0200 commented answer numpy/libatlas crashing on numpy.ma.corrcoeff()

Thank you! I was suspecting something like that and launched a compilation before leaving the office. Now everything works as expected! Generally speaking is there a way to check a platform against the CPU flags chosen in the compilation of the official releases? I imagine a little script reading cpuinfo or sth like that... Phil

2013-07-16 11:58:37 +0200 received badge  Editor (source)
2013-07-16 11:56:26 +0200 asked a question numpy/libatlas crashing on numpy.ma.corrcoeff()

Hi,

This simple python program runs fine under OS' python (Debian 6.0.1, numpy 1:1.4.1-5, libblas3gf 1.2-8, libatlas3gf-base 3.8.3-27, alternatives set to use libatlas rather than libblas)

#!/usr/bin/env python
import numpy
M=numpy.random.random((4,4))
print numpy.ma.corrcoef(M)
M=numpy.random.random((40,40))
print numpy.ma.corrcoef(M)

Now if I run it under sage's python (Sage v5.10) I get

/home/opt/sage-5.10-linux-64bit-ubuntu_10.04.4_lts-x86_64-Linux/sage -python test.py 
[[1.0 0.6938641917200964 0.24027636395823246 0.6460879199569344]
 [0.6938641917200964 1.0 -0.5029534684621264 0.6829105082413892]
 [0.24027636395823246 -0.5029534684621264 1.0 -0.3398172349895916]
 [0.6460879199569344 0.6829105082413892 -0.3398172349895916 1.0]]
/home/opt/sage-5.10-linux-64bit-ubuntu_10.04.4_lts-x86_64-Linux/
  local/bin/sage-python: line 2: 
  26485 Segmentation fault      sage -python "$@"

Any call to numpy.ma.corrcoef() on a matrix >= 40x40 fails.

GDB Trace:

(gdb) bt
#0  0x00007f031953b8cd in ATL_dJIK40x40x40TN40x40x0_a1_b0 ()
   from /home/opt/sage-5.10-linux-64bit-ubuntu_10.04.4_lts-x86_64-Linux/local/lib/libatlas.so
#1  0x00007f03195ad811 in ATL_dmmJIK2 ()
   from /home/opt/sage-5.10-linux-64bit-ubuntu_10.04.4_lts-x86_64-Linux/local/lib/libatlas.so
#2  0x00007f03195ae2f7 in ATL_dmmJIK ()
   from /home/opt/sage-5.10-linux-64bit-ubuntu_10.04.4_lts-x86_64-Linux/local/lib/libatlas.so
#3  0x00007f03195a5e17 in ATL_dgemm ()
   from /home/opt/sage-5.10-linux-64bit-ubuntu_10.04.4_lts-x86_64-Linux/local/lib/libatlas.so
#4  0x00007f031984ae86 in ATL_dptgemm_nt ()
   from /home/opt/sage-5.10-linux-64bit-ubuntu_10.04.4_lts-x86_64-Linux/local/lib/libatlas.so
#5  0x00007f031984afa9 in ATL_dptgemm ()
   from /home/opt/sage-5.10-linux-64bit-ubuntu_10.04.4_lts-x86_64-Linux/local/lib/libatlas.so
#6  0x00007f0319a9fb58 in dotblas_matrixproduct (__NPY_UNUSED_TAGGEDdummy=Unhandled dwarf expression opcode 0xf3
)
    at numpy/core/blasdot/_dotblas.c:804
#7  0x00007f031b30fb92 in call_function (f=Unhandled dwarf expression opcode 0xf3
) at Python/ceval.c:4021
#8  PyEval_EvalFrameEx (f=Unhandled dwarf expression opcode 0xf3
) at Python/ceval.c:2666
#9  0x00007f031b3109c2 in PyEval_EvalCodeEx (co=Unhandled dwarf expression opcode 0xf3
) at Python/ceval.c:3253

I also tried on the online Sage servers (v5.4) and there it works without issue.

Before trying v5.10 I was running 5.0.1, crashing too.

Actually given that trace the problem can be nailed down to:

#!/usr/bin/env python
import numpy
M=numpy.random.random((40,40))
print numpy.dot(M,M)

and the mangled function name ATL_dJIK40x40x40TN40x40x0_a1_b0 () seems to indicate Atlas is indeed doing something special for matrices >= 40x40...

I must confess I don't know what to try now...

Thanks for any help!

Phil