Installation of Numpy and Scipy, 64 bit Koala

I am losing posts from this blog. Previous installations were ok, and documented on here, but they've gone. My doubts about the stability of this blog are growing.

Anyway, here's a working process for Koala 64:

Numpy etc, Dec 09

Remove conflicting packages and install requirements:

sudo apt-get remove python-nose g77
sudo apt-get install build-essential python-dev swig gfortran python-setuptools
sudo easy_install nose


Build lapack
=======

mkdir -p ~/downloads/lapack
cd ~/downloads/lapack
wget http://www.netlib.org/lapack/lapack-3.1.1.tgz
tar zxvf lapack-3.1.1.tgz
cd lapack-3.1.1
cp INSTALL/make.inc.gfortran make.inc

Edit the make.inc file: Set OPTS and NOOPT for position independent code: I used,
OPTS = -O2 -fPIC -m64
NOOPT = -O0 -fPIC -m64

where the 64 is for 64 bit.

cd SRC
make

This creates lapack_.a ; its location is required for ATLAS.
My version made ~/downloads/lapack/lapack-3.1.1/lapack_LINUX.a


BUILD ATLAS
========

mkdir -p ~/downloads/ATLAS
cd ~/downloads/ATLAS
wget http://downloads.sourceforge.net/math-atlas/atlas3.8.2.tar.bz2?modtime=1212787160&big_mirror=0

tar jxvf atlas3.8.2.tar.bz2
cd ATLAS
mkdir ATLAS_ubuntu64 # my choice
cd ATLAS_ubuntu64

sudo cpufreq-selector -g performance # error, nomatter. Possibly for laptops.
../configure -b 64 -Fa alg -fPIC --with-netlib-lapack=/home/ms/downloads/lapack/lapack-3.1.1/lapack_LINUX.a
(note the double dots at the start of that)

Heron: Start = 12:06, Finish = 13:55
Koala: Start = 13:33, Finish = 13:42

Ocelot:  Failed here on Hector.


odd.

make time

Reference clock rate=2394Mhz, new rate=2399Mhz
Refrenc : % of clock rate achieved by reference install
Present : % of clock rate achieved by present ATLAS install

single precision double precision
******************************** *******************************
real complex real complex
--------------- --------------- --------------- ---------------
Benchmark Refrenc Present Refrenc Present Refrenc Present Refrenc Present
========= ======= ======= ======= ======= ======= ======= ======= =======
kSelMM 561.0 563.4 532.2 500.6 319.7 315.9 298.5 303.1
kGenMM 173.6 155.4 177.0 173.3 165.3 143.0 165.4 164.8
kMM_NT 134.3 128.8 129.3 166.9 126.9 120.4 133.2 148.7
kMM_TN 159.7 161.0 156.8 161.0 149.2 140.4 157.3 154.7
BIG_MM 546.6 559.8 559.5 564.4 312.3 313.3 308.1 314.0
kMV_N 108.2 117.5 209.2 233.0 53.7 68.5 86.6 100.9
kMV_T 87.8 93.8 96.2 80.4 46.5 57.0 77.7 74.9
kGER 132.6 173.8 108.7 136.9 27.4 33.6 43.6 54.6



Copy the libraries to a lib directory:

cd lib
mkdir -p ~/scipy_build/lib
cp * ~/scipy_build/lib




Build UMFPACK
=========

Get the latest versions of AMD, UFconfig and UMFPACK. Untar them:

mkdir ~/downloads/UMFPACK
cd ~/downloads/UMFPACK
wget http://www.cise.ufl.edu/research/sparse/umfpack/current/UMFPACK.tar.gz
gunzip < UMFPACK.tar.gz | tar xvf -

wget http://www.cise.ufl.edu/research/sparse/amd/current/AMD.tar.gz
gunzip < AMD.tar.gz | tar xvf -

wget http://www.cise.ufl.edu/research/sparse/UFconfig/UFconfig-3.2.0.tar.gz
gunzip < UFconfig-3.2.0.tar.gz | tar xvf -

Check which version of gcc is in use. This machine shows

/usr/lib/gcc/x86_64-linux-gnu/4.4.1

so, UFconfig/UFconfig.mk contains:

------
CC = gcc
CFLAGS = -O3 -fexceptions -m64 -fPIC
F77 = gfortran
F77FLAGS = -O -m64 -fPIC
-------
BLAS = -L/usr/lib/gcc/x86_64-linux-gnu/4.4.1 -L/home/ms/scipy_build/lib -llapack -lf77blas -lcblas -latlas -lgfortran
LAPACK = -L/usr/lib/gcc/x86_64-linux-gnu/4.4.1 -L/home/ms/scipy_build/lib -llapack -lf77blas -lcblas -latlas -lgfortran


Run make in the UMFPACK directory:
cd /home/ms/downloads/UMFPACK/UMFPACK/
make

Copy the resulting libraries and include files:
cd ..
mkdir /home/ms/scipy_build/lib/include
cp AMD/Lib/libamd.a ~/scipy_build/lib
cp UMFPACK/Lib/libumfpack.a ~/scipy_build/lib
cp AMD/Include/amd.h ~/scipy_build/lib/include
cp UFconfig/UFconfig.h ~/scipy_build/lib/include
cp UMFPACK/Include/*.h ~/scipy_build/lib/include

Copy libgfortran into scipy library directory (doesn't work if the umfpack libs aren't together)
cp /usr/lib/gcc/x86_64-linux-gnu/4.4.1/libgfortran.* ~/scipy_build/lib/


Build FFTW
=======
These steps should be executed one at a time. If the download file does not exist,
check the website for the latest version name.

mkdir ~/downloads/FFTW
cd ~/downloads/FFTW
wget http://www.fftw.org/fftw-3.2.2.tar.gz
gunzip < fftw-3.2.2.tar.gz | tar xvf -
cd fftw-3.2.2
./configure

Note: This part didn't work -----------------------------------
Examine the makefile for flags: vi Makefile
Look for CFLAGS, FFLAGS, CXXFLAGS. In mine, there were no CxxFLAGS. I added -fPIC -m64 to CFLAGS and FFLAGS,
from past experience.

./configure --enable-sse2 --enable-threads --with-combined-threads CFLAGS = "-O3 -fomit-frame-pointer -malign-double -fstrict-aliasing -ffast-math -march=core2 -fPIC -m64" FFLAGS = "-g -O2 -fPIC -m64"

Now, I get:
configure: error: unrecognized option: -O3 -fomit-frame-pointer -malign-double -fstrict-aliasing -ffast-math -march=core2

Odd, indeed. I performed ./configure again, and this time didn't specify anything manually.
end of non-working case ---------------------------------------------------

./configure
make
sudo make install


Build numpy and scipy
==============

Download and extract:

cd ~/downloads; mkdir scipy; mkdir numpy
cd numpy; wget http://downloads.sourceforge.net/numpy/numpy-1.2.1.tar.gz
gunzip < numpy-1.2.1.tar.gz | tar xvf -

cd ../scipy; wget http://downloads.sourceforge.net/scipy/scipy-0.7.1.tar.gz
gunzip < scipy-0.7.1.tar.gz | tar xvf -

Set site.cfg
cd ~/downloads/numpy/numpy-1.2.1/
cp site.cfg.example site.cfg


Execute the next few lines as one command, to update the file:
------------------ *snip* ------------------

echo '[DEFAULT]
library_dirs = /usr/local/lib:/home/ms/scipy_build/lib
include_dirs = /usr/local/include:/home/ms/scipy_build/lib/include

[atlas]
atlas_libs = lapack, f77blas, cblas, atlas

[amd]
amd_libs = amd
[umfpack]
umfpack_libs = umfpack, gfortran

[fftw]
libraries = fftw3

' | tee -a site.cfg


------------------ *snip* ------------------

Copy this site.cfg to scipy, also (maybe not required)

python setup.py build > buildlog.log
sudo python setup.py install >> installlog.log

installs numpy. Then,

cd ~/downloads/scipy/scipy-0.7.1
python setup.py build > buildlog.log

Warning about no config file. In previous installations, I copied the numpy
site.cfg file to this directory, and it worked ok. That is a worry. Continue for now.

sudo python setup.py install >> installlog.log



Success, indicated by:

$ python
import numpy
import scipy

no errors.

Protect yourself from Google record keeping

http://mycroft.mozdev.org/search-engines.html?name=scroogle

General ubuntu setup.

That last post concerning setting up parts of ubuntu on the eee,
triggered an interest in a single script to configure ubuntu from
start to finish.

This is that command. For clarity, it may be split into a few commands.

Before execution, the required repositories should be added.

The following is performed:

Installation of vim, konsole, fluxbox, latex, skype, conky, gkrellm, eterm (wallpaper), fortune:

sudo apt-get install vim konsole fluxbox texlive-latex-base skype conky gkrellm eterm fortune-mod

Update of fluxbox keyboard shortcuts: Copy and paste the multiple lines as one command.

echo '# Applications
Mod4 Z :Exec konsole
Mod4 F :Exec firefox
Mod4 S :Exec skype

' | tee -a ~/.fluxbox/keys


Update /etc/hosts to prevent infuriating web shit.

sudo echo '
# Auto added during config
127.0.0.1 push-analytics.com
127.0.0.1 view.atdmt.com
' | tee -a /etc/hosts

koala, eee, config

echo deb http://packages.medibuntu.org/ karmic free non-free | sudo tee -a /etc/apt/sources.list
wget -q http://packages.medibuntu.org/medibuntu-key.gpg -O- | sudo apt-key add -
sudo apt-get update

skype:
It should have been present in the medibunto repo's, but it wouldn't install.
I downloaded a .deb file, attempted to install with

dpkg -i filename.deb

but there were dependency problems, that were not easily fixed.

Doing

sudo apt-get -f install

downloaded stuff and fixed everything.

koala, eee, config

UK news agencies have nothing useful to say: turn to kids for help

http://news.bbc.co.uk/1/hi/england/8334216.stm

Being made redundant meant I couldn't look after my hamster that much.

vim notes 2

Paragraph forward / back: ESC { and ESC }


pasting:

vim --version

Check whether xterm_clipboard has a + or - preceding it. If -, add it by running
a version with it enabled. gvim has it enabled. I installed vim-python, to install
gvim, which then affected my normal vim.

Once enabled,

ESCAPE " + p

pastes from the xterm clipboard.

From http://vim.wikia.com/wiki/Accessing_the_system_clipboard

* "+2yy – copy two lines to X11 clipboard
* "+dd – cut line to X11 clipboard
* "+p – paste X11 clipboard

-----------------


Code folding.

V
select block
ZF

marks the code for folding

-------------------

multiple tabs:

vi -p *.tex

or

:tabnew filename

tab navigate with CTRL pg-up and pg-down

Gavin: tabdo command

executes command on all tabs.
-----------------------

Bookmarking:

m a : set a mark
` a : jump to mark. `a' can be anything.

-------------------

Mapping keys in Vim:

To map to control-V, the paste function whilst in insert mode:

 

:imap  pi 

or something like that
-----------------

Search / replace (from http://www.linux.com/learn/tutorials/8255-vim-tips-the-basics-of-search-and-replace):

:/string search
:?string backwards search
n next
N previous

:8,10 s/search/replace/g from line 8 to 10, g(lobal = repeats on lines)

Or, use visual select and execute the search. v is visual, V is line visual, CRTL-v is block visual.

:%s/search/replace/g over entire file

----------------------

Vim white spaces on pasting from GUI

Pre paste, do

:set paste

Post paste, do

:set nopaste

to prevent automatic indentation. From http://aymanh.com/a-collection-of-vim-tips

-------------------------

Copy, paste multiple buffers.


" a y to place into register `a'.
" a p to place into register `a'.

So, the `+' is the global buffer....

--------------------------

To pull the results of a command into the editor:

:r ! command

for example

:r ! ls gabvo*

;)
----------------------------

Speech marks when using vim-latex:

Insert mode, CTRL-V, "

CTRL-V indicates literal mode, I think.

http://vim-latex.sourceforge.net/index.php?subject=faq&title=FAQ#faq-not-loaded-vim7

-------------