Wednesday, 30 December 2009

Installation of Numpy and Scipy, 64 bit Koala

I am losing posts from this blog. Previous installations were ok, and documented on here, but they've gone. My doubts about the stability of this blog are growing.

Anyway, here's a working process for Koala 64:

Numpy etc, Dec 09

Remove conflicting packages and install requirements:

sudo apt-get remove python-nose g77
sudo apt-get install build-essential python-dev swig gfortran python-setuptools
sudo easy_install nose


Build lapack
=======

mkdir -p ~/downloads/lapack
cd ~/downloads/lapack
wget http://www.netlib.org/lapack/lapack-3.1.1.tgz
tar zxvf lapack-3.1.1.tgz
cd lapack-3.1.1
cp INSTALL/make.inc.gfortran make.inc

Edit the make.inc file: Set OPTS and NOOPT for position independent code: I used,
OPTS = -O2 -fPIC -m64
NOOPT = -O0 -fPIC -m64

where the 64 is for 64 bit.

cd SRC
make

This creates lapack_.a ; its location is required for ATLAS.
My version made ~/downloads/lapack/lapack-3.1.1/lapack_LINUX.a


BUILD ATLAS
========

mkdir -p ~/downloads/ATLAS
cd ~/downloads/ATLAS
wget http://downloads.sourceforge.net/math-atlas/atlas3.8.2.tar.bz2?modtime=1212787160&big_mirror=0

tar jxvf atlas3.8.2.tar.bz2
cd ATLAS
mkdir ATLAS_ubuntu64 # my choice
cd ATLAS_ubuntu64

sudo cpufreq-selector -g performance # error, nomatter. Possibly for laptops.
../configure -b 64 -Fa alg -fPIC --with-netlib-lapack=/home/ms/downloads/lapack/lapack-3.1.1/lapack_LINUX.a
(note the double dots at the start of that)

Heron: Start = 12:06, Finish = 13:55
Koala: Start = 13:33, Finish = 13:42

Ocelot:  Failed here on Hector.


odd.

make time

Reference clock rate=2394Mhz, new rate=2399Mhz
Refrenc : % of clock rate achieved by reference install
Present : % of clock rate achieved by present ATLAS install

single precision double precision
******************************** *******************************
real complex real complex
--------------- --------------- --------------- ---------------
Benchmark Refrenc Present Refrenc Present Refrenc Present Refrenc Present
========= ======= ======= ======= ======= ======= ======= ======= =======
kSelMM 561.0 563.4 532.2 500.6 319.7 315.9 298.5 303.1
kGenMM 173.6 155.4 177.0 173.3 165.3 143.0 165.4 164.8
kMM_NT 134.3 128.8 129.3 166.9 126.9 120.4 133.2 148.7
kMM_TN 159.7 161.0 156.8 161.0 149.2 140.4 157.3 154.7
BIG_MM 546.6 559.8 559.5 564.4 312.3 313.3 308.1 314.0
kMV_N 108.2 117.5 209.2 233.0 53.7 68.5 86.6 100.9
kMV_T 87.8 93.8 96.2 80.4 46.5 57.0 77.7 74.9
kGER 132.6 173.8 108.7 136.9 27.4 33.6 43.6 54.6



Copy the libraries to a lib directory:

cd lib
mkdir -p ~/scipy_build/lib
cp * ~/scipy_build/lib




Build UMFPACK
=========

Get the latest versions of AMD, UFconfig and UMFPACK. Untar them:

mkdir ~/downloads/UMFPACK
cd ~/downloads/UMFPACK
wget http://www.cise.ufl.edu/research/sparse/umfpack/current/UMFPACK.tar.gz
gunzip < UMFPACK.tar.gz | tar xvf -

wget http://www.cise.ufl.edu/research/sparse/amd/current/AMD.tar.gz
gunzip < AMD.tar.gz | tar xvf -

wget http://www.cise.ufl.edu/research/sparse/UFconfig/UFconfig-3.2.0.tar.gz
gunzip < UFconfig-3.2.0.tar.gz | tar xvf -

Check which version of gcc is in use. This machine shows

/usr/lib/gcc/x86_64-linux-gnu/4.4.1

so, UFconfig/UFconfig.mk contains:

------
CC = gcc
CFLAGS = -O3 -fexceptions -m64 -fPIC
F77 = gfortran
F77FLAGS = -O -m64 -fPIC
-------
BLAS = -L/usr/lib/gcc/x86_64-linux-gnu/4.4.1 -L/home/ms/scipy_build/lib -llapack -lf77blas -lcblas -latlas -lgfortran
LAPACK = -L/usr/lib/gcc/x86_64-linux-gnu/4.4.1 -L/home/ms/scipy_build/lib -llapack -lf77blas -lcblas -latlas -lgfortran


Run make in the UMFPACK directory:
cd /home/ms/downloads/UMFPACK/UMFPACK/
make

Copy the resulting libraries and include files:
cd ..
mkdir /home/ms/scipy_build/lib/include
cp AMD/Lib/libamd.a ~/scipy_build/lib
cp UMFPACK/Lib/libumfpack.a ~/scipy_build/lib
cp AMD/Include/amd.h ~/scipy_build/lib/include
cp UFconfig/UFconfig.h ~/scipy_build/lib/include
cp UMFPACK/Include/*.h ~/scipy_build/lib/include

Copy libgfortran into scipy library directory (doesn't work if the umfpack libs aren't together)
cp /usr/lib/gcc/x86_64-linux-gnu/4.4.1/libgfortran.* ~/scipy_build/lib/


Build FFTW
=======
These steps should be executed one at a time. If the download file does not exist,
check the website for the latest version name.

mkdir ~/downloads/FFTW
cd ~/downloads/FFTW
wget http://www.fftw.org/fftw-3.2.2.tar.gz
gunzip < fftw-3.2.2.tar.gz | tar xvf -
cd fftw-3.2.2
./configure

Note: This part didn't work -----------------------------------
Examine the makefile for flags: vi Makefile
Look for CFLAGS, FFLAGS, CXXFLAGS. In mine, there were no CxxFLAGS. I added -fPIC -m64 to CFLAGS and FFLAGS,
from past experience.

./configure --enable-sse2 --enable-threads --with-combined-threads CFLAGS = "-O3 -fomit-frame-pointer -malign-double -fstrict-aliasing -ffast-math -march=core2 -fPIC -m64" FFLAGS = "-g -O2 -fPIC -m64"

Now, I get:
configure: error: unrecognized option: -O3 -fomit-frame-pointer -malign-double -fstrict-aliasing -ffast-math -march=core2

Odd, indeed. I performed ./configure again, and this time didn't specify anything manually.
end of non-working case ---------------------------------------------------

./configure
make
sudo make install


Build numpy and scipy
==============

Download and extract:

cd ~/downloads; mkdir scipy; mkdir numpy
cd numpy; wget http://downloads.sourceforge.net/numpy/numpy-1.2.1.tar.gz
gunzip < numpy-1.2.1.tar.gz | tar xvf -

cd ../scipy; wget http://downloads.sourceforge.net/scipy/scipy-0.7.1.tar.gz
gunzip < scipy-0.7.1.tar.gz | tar xvf -

Set site.cfg
cd ~/downloads/numpy/numpy-1.2.1/
cp site.cfg.example site.cfg


Execute the next few lines as one command, to update the file:
------------------ *snip* ------------------

echo '[DEFAULT]
library_dirs = /usr/local/lib:/home/ms/scipy_build/lib
include_dirs = /usr/local/include:/home/ms/scipy_build/lib/include

[atlas]
atlas_libs = lapack, f77blas, cblas, atlas

[amd]
amd_libs = amd
[umfpack]
umfpack_libs = umfpack, gfortran

[fftw]
libraries = fftw3

' | tee -a site.cfg


------------------ *snip* ------------------

Copy this site.cfg to scipy, also (maybe not required)

python setup.py build > buildlog.log
sudo python setup.py install >> installlog.log

installs numpy. Then,

cd ~/downloads/scipy/scipy-0.7.1
python setup.py build > buildlog.log

Warning about no config file. In previous installations, I copied the numpy
site.cfg file to this directory, and it worked ok. That is a worry. Continue for now.

sudo python setup.py install >> installlog.log



Success, indicated by:

$ python
import numpy
import scipy

no errors.

No comments:

Post a Comment