Ask Your Question
0

numpy/libatlas crashing on numpy.ma.corrcoeff()

asked 2013-07-16 11:56:26 +0100

doegox gravatar image

updated 2013-07-16 12:25:03 +0100

Hi,

This simple python program runs fine under OS' python (Debian 6.0.1, numpy 1:1.4.1-5, libblas3gf 1.2-8, libatlas3gf-base 3.8.3-27, alternatives set to use libatlas rather than libblas)

#!/usr/bin/env python
import numpy
M=numpy.random.random((4,4))
print numpy.ma.corrcoef(M)
M=numpy.random.random((40,40))
print numpy.ma.corrcoef(M)

Now if I run it under sage's python (Sage v5.10) I get

/home/opt/sage-5.10-linux-64bit-ubuntu_10.04.4_lts-x86_64-Linux/sage -python test.py 
[[1.0 0.6938641917200964 0.24027636395823246 0.6460879199569344]
 [0.6938641917200964 1.0 -0.5029534684621264 0.6829105082413892]
 [0.24027636395823246 -0.5029534684621264 1.0 -0.3398172349895916]
 [0.6460879199569344 0.6829105082413892 -0.3398172349895916 1.0]]
/home/opt/sage-5.10-linux-64bit-ubuntu_10.04.4_lts-x86_64-Linux/
  local/bin/sage-python: line 2: 
  26485 Segmentation fault      sage -python "$@"

Any call to numpy.ma.corrcoef() on a matrix >= 40x40 fails.

GDB Trace:

(gdb) bt
#0  0x00007f031953b8cd in ATL_dJIK40x40x40TN40x40x0_a1_b0 ()
   from /home/opt/sage-5.10-linux-64bit-ubuntu_10.04.4_lts-x86_64-Linux/local/lib/libatlas.so
#1  0x00007f03195ad811 in ATL_dmmJIK2 ()
   from /home/opt/sage-5.10-linux-64bit-ubuntu_10.04.4_lts-x86_64-Linux/local/lib/libatlas.so
#2  0x00007f03195ae2f7 in ATL_dmmJIK ()
   from /home/opt/sage-5.10-linux-64bit-ubuntu_10.04.4_lts-x86_64-Linux/local/lib/libatlas.so
#3  0x00007f03195a5e17 in ATL_dgemm ()
   from /home/opt/sage-5.10-linux-64bit-ubuntu_10.04.4_lts-x86_64-Linux/local/lib/libatlas.so
#4  0x00007f031984ae86 in ATL_dptgemm_nt ()
   from /home/opt/sage-5.10-linux-64bit-ubuntu_10.04.4_lts-x86_64-Linux/local/lib/libatlas.so
#5  0x00007f031984afa9 in ATL_dptgemm ()
   from /home/opt/sage-5.10-linux-64bit-ubuntu_10.04.4_lts-x86_64-Linux/local/lib/libatlas.so
#6  0x00007f0319a9fb58 in dotblas_matrixproduct (__NPY_UNUSED_TAGGEDdummy=Unhandled dwarf expression opcode 0xf3
)
    at numpy/core/blasdot/_dotblas.c:804
#7  0x00007f031b30fb92 in call_function (f=Unhandled dwarf expression opcode 0xf3
) at Python/ceval.c:4021
#8  PyEval_EvalFrameEx (f=Unhandled dwarf expression opcode 0xf3
) at Python/ceval.c:2666
#9  0x00007f031b3109c2 in PyEval_EvalCodeEx (co=Unhandled dwarf expression opcode 0xf3
) at Python/ceval.c:3253

I also tried on the online Sage servers (v5.4) and there it works without issue.

Before trying v5.10 I was running 5.0.1, crashing too.

Actually given that trace the problem can be nailed down to:

#!/usr/bin/env python
import numpy
M=numpy.random.random((40,40))
print numpy.dot(M,M)

and the mangled function name ATL_dJIK40x40x40TN40x40x0_a1_b0 () seems to indicate Atlas is indeed doing something special for matrices >= 40x40...

I must confess I don't know what to try now...

Thanks for any help!

Phil

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
1

answered 2013-07-16 17:08:36 +0100

Volker Braun gravatar image

The ATLAS in your binary distribution was built with CPU instruction set extensions that are not supported by your own CPU, or you are hitting one of the bugs in the old ATLAS that we still haven't upgraded in Sage. Try compiling from source. See also http://trac.sagemath.org/14754

edit flag offensive delete link more

Comments

Thank you! I was suspecting something like that and launched a compilation before leaving the office. Now everything works as expected! Generally speaking is there a way to check a platform against the CPU flags chosen in the compilation of the official releases? I imagine a little script reading cpuinfo or sth like that... Phil

doegox gravatar imagedoegox ( 2013-07-18 05:29:48 +0100 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

Stats

Asked: 2013-07-16 11:56:26 +0100

Seen: 601 times

Last updated: Jul 16 '13