12

I wrote this test script:

import numpy as np
import scipy.linalg

n = 130
r = np.array(np.random.normal(size=(n, n)), dtype=np.float32)
e = scipy.linalg.eig(r, left=False, right=False)
print e.mean()

Running it using IPython, the code always succeeds in a fraction of a second (I tried it about a dozen times)

With Python, the code always fails to converge (or just hangs, for some larger n) with a message like

Traceback (most recent call last):
  File "strange.py", line 6, in <module>
    e = scipy.linalg.eig(r, left=False, right=False)
  File "/usr/lib/python2.7/dist-packages/scipy/linalg/decomp.py", line 179, in eig
    "with order >= %d have converged)" % info)
numpy.linalg.linalg.LinAlgError: eig algorithm did not converge (only eigenvalues with order >= 130 have converged)

What explains this difference in the behavior of Python and IPython? The relevant software versions are:

  • Ubuntu 12.04, 64-bit
  • Numpy 1.6.1
  • SciPy 0.9.0
  • Python 2.7.3
  • IPython 0.12.1

Edit

I observed this behavior only with single precision and n >= 130. If n = 129, the code works in both Python and IPython.

Adding np.random.seed(1234) after the imports gives the same result: IPython converges, while Python does not.

scipy.linalg.__file__ = '/usr/lib/python2.7/dist-packages/scipy/linalg/__init__.pyc' in both. Despite this, I would guess that IPython and Python somehow manage to pull in different LAPACK versions, but how?

The way I noticed this oddity is that I was experimenting in IPython, and then pasting the code into a *.py file that I run with Python. You can imagine how confused I was for a while.

Edit 2.

np.geterr() is {'divide': 'warn', 'invalid': 'warn', 'over': 'warn', 'under': 'ignore'} in both Python and IPython

$ ls -l /etc/alternatives/libblas.so
lrwxrwxrwx 1 root root 37 Jun 29 18:21 /etc/alternatives/libblas.so -> /usr/lib/openblas-base/libopenblas.so
19
  • 4
    By the way, why numpy and scipy are so outdated?
    – alko
    Commented Dec 10, 2013 at 13:13
  • 3
    you do not seed the random number generator (np.seed(1234)).
    – ev-br
    Commented Dec 10, 2013 at 14:15
  • 2
    @alko: Those are the versions in the Ubuntu 12.04 repository. Commented Dec 10, 2013 at 15:08
  • 2
    @Dougal I gave in to peer pressure to upgrade iPython, NumPy and SciPy, so I wiped them, and installed the latest versions from source instead of Ubuntu 12.04. I don't seem have this problem anymore. Thanks to everyone for trying to help!
    – MWB
    Commented Dec 11, 2013 at 7:41
  • 2
    I made a virtual Ubuntu 12.04 64bit machine and installed the cited versions of Numpy, SciPy, Python, and IPython, and both versions of the script worked in both ipython and python (see https://gist.github.com/tsbertalan/8173209). However, after installing libopenblas-base, only the ipython one worked, as described.
    – tsbertalan
    Commented Dec 29, 2013 at 18:47

1 Answer 1

1

It could be that the LD_LIBRARY_PATH is different when you are using IPython. This could result in using different libraries. You can check this in both Python and IPython and see if they are identical:

import os
print os.environ['LD_LIBRARY_PATH']

Not the answer you're looking for? Browse other questions tagged or ask your own question.