Friday, 30 January 2009

Ordered 3-D mesh with Gmsh, no extrusion, hexahedra


// Gmsh project created on Tue Aug 5 18:29:53 2008
l = 0.02; // length
h = 0.01; // duct height
w = 0.001; // width
my = 1; // y mesh
mz = 1; // mesh density across width.
mx = 40; // mesh along the length.

// Bottom surface; extrude from here, up.=========================NODES
Point(1) = { 0, h, 0, 1}; // bottom, rear right corner.
Point(2) = { l, h, 0, 1}; // bottom, front right corner.
Point(3) = { 0, 0, 0, 1}; // bottom, rear right corner.
Point(4) = { l, 0, 0, 1}; // bottom, front right corner.

Point(5) = { 0, h, w, 1}; // bottom, rear right corner.
Point(6) = { l, h, w*2, 1}; // bottom, front right corner.
Point(7) = { 0, 0, w, 1}; // bottom, rear right corner.
Point(8) = { l, 0, w*2, 1}; // bottom, front right corner.


// Cube-ish creation. Duct. ==============================================
Line(1) = {1, 2};
Line(2) = {2, 4};
Line(3) = {4, 3};
Line(4) = {3, 1};
Line(5) = {5, 6};
Line(6) = {6, 8};
Line(7) = {8, 7};
Line(8) = {7, 5};
Line(9) = {1, 5};
Line(10) = {2, 6};
Line(11) = {3, 7};
Line(12) = {4, 8};




// Define closed loops and planes for the shapes just created. These were
// selected in GMSH, rather than in the script as the negative values control
// the direction of the path. ============================================Planes
// Duct.
Line Loop(1) = {1,2,3,4};
Plane Surface(2) = {1};
Line Loop(3) = {5,6,7,8};
Plane Surface(4) = {3};
Line Loop(5) = {9,-8,-11,4};
Plane Surface(6) = {5};
Line Loop(7) = {10,6,-12,-2};
Plane Surface(8) = {7};
Line Loop(9) = {1,10,-5,-9};
Plane Surface(10) = {9};
Line Loop(11) = {3,11,-7,-12};
Plane Surface(12) = {11};


// The following code is required to make the mesh ordered =====================
// Set the lines above, as 'transfinite'. These are line numbers
Transfinite Line{1, 3, 5, 7} = mx + 1 Using Progression 1.0;
Transfinite Line{2, 4, 6, 8} = my + 1 Using Progression 1.0;
Transfinite Line{9, 10, 11, 12} = mz + 1 Using Progression 1.0;


// Create new, 'transfinite' surfaces. Matching plane surface numbers above and
// referencing same node numbers. Had to inspect
// these from gmsh. Note, curly brackets, not parentheses.
// {surface} = {nodes}
Transfinite Surface{2} = {1, 2, 3, 4};
Transfinite Surface{4} = {5, 6, 7, 8};
Transfinite Surface{6} = {1, 5, 3, 7};
Transfinite Surface{8} = {6, 2, 8, 4};
Transfinite Surface{10} = {2, 6, 1, 5};
Transfinite Surface{12} = {8, 4, 7, 3};

// Without the following recombine, the elements are tri primative.
// Duct.
Recombine Surface{2, 4, 6, 8, 10, 12};


// End of extra code to get ordered mesh ***************************************
//
// Extrude {vector} { Surface{n}; Layers{mz} ; Recombine; }
// with surface number(s) n extruded, mesh density mz1.
// Assign result to var[], to allow further work with the new surface, in var[0]
// Entries 0 to 5 are returned for these extruded faces, so steps of 6 to
// get the faces for different volumes appended to the list.
//
// Extrude to form duct width: -------------------------
//
// num[] = Extrude { 0, 0, -w} {
// Surface {
// 2
// };
// Layers{mz}; Recombine;
// };
// Can't extrude, as I want to change the thickness in that direction.
// New nodes, instead.


// Now, define a volume.
Surface Loop(13) = {4,10,2,8,12,6};
Volume(14) = {13};

// And define as transfinite. The points were selected, clockwise for both
// faces, looking in the same direction.
Transfinite Volume{14} = {5,6,8,7,1,2,4,3};

Thursday, 29 January 2009

Ordered 3-D mesh with Gmsh. Extrude 2-D surface.


// Gmsh project created on Tue Aug 5 18:29:53 2008
l = 0.02; // length
h = 0.01; // duct height
w = 0.001; // duct width
mx = 20; // mesh along the length.
my = 12; // y mesh. Divisible by 4
mz = 1; // mesh density across width.

// Tranfinite surfaces have 3 or 4 points, so can't lump the square face.
// Split it, then, into 4 sections.

Point(1) = { 0, h, h, 1}; // left wall, top to bottom.
Point(2) = { 0, h * 0.75, h, 1}; // ...
Point(3) = { 0, h * 0.5, h, 1}; // ...
Point(4) = { 0, h * 0.25, h, 1}; // ...
Point(5) = { 0, h * 0, h, 1}; // ...

Point(6) = { l, h, h, 1}; // right wall, top to bottom.
Point(7) = { l, h * 0.75, h, 1}; // ...
Point(8) = { l, h * 0.5, h, 1}; // ...
Point(9) = { l, h * 0.25, h, 1}; // ...
Point(10) = { l, h * 0.00, h, 1}; // ...

Point(11) = { l + h/4, h * 0.75, h, 1}; // inner square nodes
Point(12) = { l + h/4, h * 0.5, h, 1}; // ...
Point(13) = { l + h/4, h * 0.25, h, 1}; // ...

Point(14) = { l + 0.707 * h/2 , h/2 + 0.707 * h/2, h, 1}; // Curve.
Point(15) = { l + 1.000 * h/2 , h/2 + 0.000 * h/2, h, 1}; // Curve.
Point(16) = { l + 0.707 * h/2 , h/2 - 0.707 * h/2, h, 1}; // Curve.


// Square creation. Duct. ==============================================Squares
Line(1) = {1, 2}; // left side
Line(2) = {2, 3}; // ...
Line(3) = {3, 4}; // ...
Line(4) = {4, 5}; // ...

Line(5) = {6, 7}; // right side
Line(6) = {7, 8}; // ...
Line(7) = {8, 9}; // ...
Line(8) = {9, 10}; // ...

Line(9) = {14, 11}; // inner square, vertical
Line(10) = {11, 12}; // ...
Line(11) = {12, 13}; // ...
Line(12) = {13, 16}; // ...

Circle(13) = {6, 8, 14}; // curved end
Circle(14) = {14, 8, 15}; // ...
Circle(15) = {15, 8, 16}; // ...
Circle(16) = {16, 8, 10}; // ...

Line(17) = {1, 6}; // horizontal lines, left to right.
Line(18) = {2, 7}; // ...
Line(19) = {7, 11}; // ...
Line(20) = {3, 8}; // ...
Line(21) = {8, 12}; // ...
Line(22) = {12, 15}; // square to circle
Line(23) = {4, 9}; // ...
Line(24) = {9, 13}; // ...
Line(25) = {5, 10}; // ...


// Define closed loops and planes for the shapes just created. These were
// selected in GMSH, rather than in the script as the negative values control
// the direction of the path. ============================================Planes
// Duct.
Line Loop(26) = {17,5,-18,-1};
Plane Surface(27) = {26};
Line Loop(28) = {18,6,-20,-2};
Plane Surface(29) = {28};
Line Loop(30) = {20,7,-23,-3};
Plane Surface(31) = {30};
Line Loop(32) = {23,8,-25,-4};
Plane Surface(33) = {32};
Line Loop(34) = {13,9,-19,-5};
Plane Surface(35) = {34};
Line Loop(36) = {19,10,-21,-6};
Plane Surface(37) = {36};
Line Loop(38) = {21,11,-24,-7};
Plane Surface(39) = {38};
Line Loop(40) = {24,12,16,-8};
Plane Surface(41) = {40};
Line Loop(42) = {14,-22,-10,-9};
Plane Surface(43) = {42};
Line Loop(44) = {22,15,-12,-11};
Plane Surface(45) = {44};


// The following code is required to make the mesh ordered =====================
// Set the lines above, as 'transfinite'. These are line numbers
Transfinite Line{1, 2, 3, 4, 5, 6, 7, 8, 10, 11} = 0.25*my Using Progression 1.0; // vertical lines

Transfinite Line{17, 18, 20, 23, 25} = mx Using Progression 1.0; // horizontal lines
Transfinite Line{19, 21, 24} = 0.25*my Using Progression 1.0; // horizontal lines
Transfinite Line{9, 22, 12} = 0.25*my Using Progression 1.0; // diagonal lines
Transfinite Line{13, 14, 15, 16} = 0.25*my Using Progression 1.0; // Circle portions.

// Create new, 'transfinite' surfaces. Matching plane surface numbers above and
// referencing same node numbers. Had to inspect
// these from gmsh. Note, curly brackets, not parentheses.
// {surface} = {nodes}
Transfinite Surface{27} = {1, 2, 6, 7}; // duct, left side, top to bottom.
Transfinite Surface{29} = {2, 7, 3, 8}; // ...
Transfinite Surface{31} = {3, 8, 4, 9}; // ...
Transfinite Surface{33} = {4, 9, 5, 10}; // ...

Transfinite Surface{35} = {6, 14, 7, 11}; // duct, middle, top to bottom.
Transfinite Surface{37} = {7, 11, 8, 12}; // ...
Transfinite Surface{39} = {8, 12, 9, 13}; // ...
Transfinite Surface{41} = {9, 13, 10, 16}; // ...

Transfinite Surface{43} = {11, 14, 12, 15}; // last two of circle,top to bottom.
Transfinite Surface{45} = {12, 15, 13, 16}; // ...


// Without the following recombine, the elements are tri primative.
// Duct.
Recombine Surface{27, 29, 31, 33, 35, 37, 39, 41, 43, 45};
// End of extra code to get ordered mesh =======================================


// Extrude {vector} { Surface{n}; Layers{mz} ; Recombine; }
// with surface number(s) n extruded, mesh density mz1.
// Assign result to var[], to allow further work with the new surface, in var[0]
// Entries 0 to 5 are returned for these extruded faces, so steps of 6 to
// get the faces for different volumes appended to the list.
//
// Extrude to form duct width: -------------------------
//
num[] = Extrude { 0, 0, -w} {
Surface {
27 , 29, 31, 33, 35, 37, 39, 41, 43, 45
};
Layers{mz}; Recombine;
};

Tuesday, 27 January 2009

Tex plots, using pgfplots. Non-numeric characters on axes.


\begin{figure}[htbp]\begin{center} % plot. Z is changed by complex amounts. No good.
\mbox{
\tikzstyle{every axis legend}+= [at={(0.98,0.98)},anchor=north east] % im is 0,0 to 1,1
\begin{tikzpicture}
\begin{axis}[width=\plotw,height=\ploth,
xlabel={$x/l$}, ylabel={$\theta$},grid=major,
ytick ={0, 0.79, 1.57, 2.36, 3.14, 3.93, 4.71, 5.50, 6.28},
yticklabels={0, , , , $\pi$, , , ,$2 \pi$},
xmin = -0.1, xmax = 1.1, ymin=0, ymax=6.28]

% reference.dumptex(r1[0] / 0.02, reference.ang(r1[1]))
% 1000 Hz, 0 flow

\addplot[color = black] plot coordinates {(0.00000e+00,4.71241e+00) (5.00000e-02,4.69414e+00) (1.00000e-01,4.67588e+00) (1.50000e-01,4.65761e+00) (2.00000e-01,4.63934e+00) (2.50000e-01,4.62109e+00) (3.00000e-01,4.60281e+00) (3.50000e-01,4.58455e+00) (4.00000e-01,4.56629e+00) (4.50000e-01,4.54803e+00) (5.00000e-01,4.52976e+00) (5.50000e-01,4.51149e+00) (6.00000e-01,4.49323e+00) (6.50000e-01,4.47496e+00) (7.00000e-01,4.45670e+00) (7.50000e-01,4.43843e+00) (8.00000e-01,4.42016e+00) (8.50000e-01,4.40191e+00) (9.00000e-01,4.38364e+00) (9.50000e-01,4.36537e+00) (1.00000e+00,4.34711e+00)};

\legend{Phase.\\}

\end{axis} \end{tikzpicture}

}
\caption{Zero flow, one-dimensional results at $f=1kHz$.}
\label{fig:m95acou2a_pvf} \end{center} \end{figure}

Monday, 26 January 2009

Saturday, 24 January 2009

Mass file renaming.

Thankfully: A civilised way:

Rename all .xcf.png files, with .png:

rename 's/.xcf.png/.png/' ????.xcf.png



Or, the hard way:

Copied directly from http://www.linuxquestions.org/blog/ghostdog74-290378/2008/9/17/simple-mass-file-renamer-and-deleter-python-1190/
by 'ghostdog74':

This can:

1) Change upper case to lower case and vice versa
2) Change file names by pattern substitution
3) Change file names by number sequence
4) Insert pattern in front of files
5) Insert pattern at the end of files.
6) Simple sort by number.
7) Manual revert of changed files.
8) Deletion of files by pattern match only.
9) Ability to recurse directory
10) Inverse match pattern, like grep -v
11) Case sensitivity search
12) Remove string by character positions. eg 0:10 => remove from start to 10th character.



#!/usr/bin/env python
import sys,os,getopt,glob,string,re,operator,fnmatch,time

################################## FUNCTIONS #################################################
def usage(name):
print """
%s
Mini File Renamer and Deleter v1.0.2 - Copyright 2008
Author: ghostdog74

%s
usage: %s [-h] [-D dir] [-t f|d ][-[s|p] oldpat] [-e newpat ] [ -i pattern ][ -b patten ] [-d depth] [-Z|-v|-n|-r|-I] [-l] [files]
-D Directory to start rename/delete. Use quotes for directories with spaces
If no directory is specified, current working directory is assumed
Eg: Use -D c:\\test\\ (for Windows, end with trailing back slash)
-h Prints help page
-t Type of files. Either files(f) or directories(d). Default is files, so can omit -t f
-s Sequence substitution. Specify pattern to be substituted. Must be specified with -e.
-p Pattern substitution. Must be specified with -e (pattern to change to)
eg.
1) To change whole jpeg file name to upper case ==> -p ".*" -e "A-Z" "*.jpeg"
2) To change the word "TEST" to lower case in jpeg file ==> -p "TEST" -e "a-z" "*.jpeg"
3) To remove all numbers from directory name ==> -p "[0-9]+" -e '' -t d -l "*"
4) To remove first character from file/directory ==> -p "^." -e '' -l "*.jpeg"
5) To remove special characters ==> -p "[\"^']" -e "back" "*"

-e * When used with -s, indicates ending sequence pattern. Can include alphanumberic. Must use ":" to specify range
Eg
1) -s "test" -e "01:10" will replace 'test' in files from '01' to '10'. If more than 1 files with 'test',
will go by sequence, ie '02' , '03' etc.
2) -s "test" -e "###01:11@@@" will replace 'test' from '###01@@@' , followed by '###02@@@' and so on to ###11@@@
* When used with -p, indicates pattern to change to.
* When used with -c, specify -e "[A-Z]" to change to uppercase, -e "[a-z]" to change to lower case.

-i Insert pattern to infront of file name.
-b Insert pattern to back of file name.
-c Remove characters in file name by position, position index start from 0. Always use -l to verify files to be changed.
eg
1) -c 1 ==> remove 2nd character
2) -c -1 ===> remove last character.
3) -c -2: ===> remove from last 2nd character onwards.
4) -c 3: ==> remove from 4th character onwards
5) -c 4:10 ==> remove from 5th to 10th character
6) -c :3 ==> remove from start to 3rd character.
7) -c 1:3 -e "[A-Z]" ==> change positions specified to uppercase

-l List all files with pattern only. No renaming. Useful for verifying what will be changed.
-r Used alone. Enable restoration of previous commands. eg %s -r
-n Simple Numerical sort. Specify -n to turn numerical sorting on. Only works on files of the same structure.
-d # of directories down to do rename (default = 0). ie: Directory depth level
-I Case insensitive pattern search. eg -p "rot" -e "" -I -l "*.txt". Find rot,ROT,roT,RoT ..
-v Wildcard pattern reversal. eg -v "*.bat" : Files that doesn't end with .bat.
-Z Do deletion of files.
eg -d 4 -Z ".*01*" -l -v "*.txt" ==> delete all files that doesn't end with .txt and with the pattern "01" in the
file name, 4 levels deep into current directory
[files] List of files to be renamed/deleted. Can have wildcards. eg test*.txt
To specify all files ==> "*". Will not work if not specified.
""" % ("=" *100 , "=" * 100 ,name,name )


def pathChecker(path):
''' Function to determine correct path or file exists
and then returns the number of count of the path separator
as an indicator of the depth of the path.
'''
if os.path.exists(path):
pathcount=path.count(os.sep)
if pathcount > 1:
return int(pathcount) , path , 0
else:return 1,1,1
else:
return 0,0,-1

def combocheck(s,k):
'''Function to check the options user keyed in against a set of bad options
Input : s => bad options
k => list of user supplied keys...eg -S -s -N..etc
'''
v = len(s); t = 0 ##store lenght of predefined bad options, t=0 to count matched bad ops
for x in s:
if x in k:
t = t + 1
## if all bad options found
if v == t : return True
else: return False


def doWalk(DIR=None,maxdepth=1,TYPE="f",ACTION="sub",patold=None,patnew=None,DEBUG=1,INVERSE=0,CASE=1,SORT=0,FileNamesArgs=None):
''' Traverse the directory specified until depth level,
looking for files with the correct patterns and rename them
accordingly
'''
GlobbedTypeList=[]
AllFilesGlobbed=[]


# convert into reg expression syntax in order to search. eg "*.txt" to ".*txt$"

regex = fnmatch.translate(FileNamesArgs)
reobj = re.compile(regex)
for ROOT,DIRECTORY,FILES in os.walk(DIR,True):
#do for files less or equal to maxdepth
if ROOT.count(os.sep) <= int(maxdepth): if FileNamesArgs is not None: if INVERSE: for FI in os.listdir( ROOT ): if not reobj.search(FI): AllFilesGlobbed.append( os.path.join(ROOT,FI )) else: try: allfiles = glob.glob(os.path.join(ROOT,FileNamesArgs)) except Exception,e: pass else: if allfiles: for found in allfiles: if found not in AllFilesGlobbed: AllFilesGlobbed.append(found) if TYPE=="d": for dirname in AllFilesGlobbed: if os.path.isdir(dirname): if not dirname in GlobbedTypeList: GlobbedTypeList.append([dirname,dirname.count(os.sep)]) else: for filenames in AllFilesGlobbed: if os.path.isfile(filenames): if not filenames in GlobbedTypeList: GlobbedTypeList.append(filenames) if SORT==1: GlobbedTypeList = sorted_copy(GlobbedTypeList) else: GlobbedTypeList=sorted(GlobbedTypeList, key=(operator.itemgetter(0))) # do various actions doAction(GlobbedTypeList,TYPE,ACTION,patold,patnew,CASE,SORT,DEBUG) def brake(): raw_input("Enter") def clearscreen(rows): for i in range(rows): print def rename(FROM,TO="",DEBUG=1): if FROM == TO:return if DEBUG==0 : if TO : try: os.rename(FROM,TO) except Exception,e: print "Error : ",e else: print FROM , " is renamed to ", TO # store to restore file. simple mechanism. Use pickle/shelve ?? open(restorefile,"a").write("""%s,%s\n""" %( TO,FROM )) elif not TO : print "Deleting " ,FROM if os.path.isdir(FROM): try: os.removedirs(FROM) #or use os.rmdir except Exception,e: print "Error: ",e elif os.path.isfile(FROM): try : os.remove(FROM) except Exception,e: print "Error: ",e else: print "==>>>> ", "[" ,FROM ,"]==>[",TO,"]"

def changecase(patnew,thefile):
# if -c and -e option and -e [A-Z] or -e [a-z]
try:
foundlowercase = re.findall( "\[a-z\]|a-z" , patnew)[0]
except:
try:
founduppercase = re.findall( "\[A-Z\]|A-Z" , patnew)[0]
except: notfound=1
else:
newname = thefile.replace(thefile,thefile.upper())
else:
newname = thefile.replace(thefile,thefile.lower())


def doAction(FILES,TYPE="d", ACTION="sub", patold="" ,patnew="", CASE=1, SORT=0,DEBUG=1):
'''Function to do sequence substitution
Input: FILES => A list of globbed files to be processed.
TYPE => f = files, d = directories
patold => The pattern in the file to be substituted
patnew => The new pattern to replace the old
DEBUG => 1 : Do a listing only
0 : Do substitution, and renaming of files
'''

notfound=0 #flag for doing case change.
if TYPE=="d":
# if renaming directories, rename from the last(highest) level. So have to sort according to maxdepth
FILES=sorted(FILES, key=(operator.itemgetter(1)),reverse=True)

if CASE == 0:
patold_re = re.compile(patold,re.IGNORECASE)
elif CASE:
patold_re = re.compile(patold)

if ACTION=="seq":
# see if in format 001:020 ...this indicates sequence
patnew_re =re.compile("(\w*\d*\D*)(\d+)[:](\d+)(\D*\w*\d*)")
seq = patnew_re.findall(patnew)[0]
startseq=seq[1]; endseq=seq[-2]
patendseqfront = seq[0]
patendseqback = seq[-1]
if endseq and int(endseq) < endseq =" endseq,startseq" endseq="startseq" endseq ="="" endseq="len(FILES)" length_startseq =" len(str(startseq))" type="="" fn="fn[0]" type="="" fn="fn" thefile =" os.path.split(FN)" action="="" patnew ="="" newname =" patold+thefile" patnew ="="" newname =" thefile+patold" action="="">:
patdigit=re.compile("(-)*(\d*):(-)*(\d*)")
b= list(thefile)
if patnew and re.search( "\[a-z\]|a-z" , patnew) :
caseflag=1
elif patnew and re.search( "\[A-Z\]|A-Z" , patnew):
caseflag=2
else:caseflag=0

# if single digit
if ":" not in patold and ( int(patold) < caseflag="="1" caseflag="="2:" newname =" ''.join(b)" foundit =" patdigit.findall(patold)[0]" first = "" second = "" caseflag="="1:" newname =" thefile[0:int(first)]+thefile[int(first):].lower()" caseflag="="2:" newname =" thefile[0:int(first)]+thefile[int(first):].upper()" newname =" thefile[" caseflag="="1:" newname =" thefile[:int(second)].lower()+thefile[int(second):]" caseflag="="2:" newname =" thefile[:int(second)].upper()+thefile[" newname =" thefile[" caseflag="="1:" newname =" thefile[0:int(first)]+thefile[int(first):int(second)].lower()+thefile[int(second):]" caseflag="="2:" newname =" thefile[0:int(first)]+thefile[int(first):int(second)].upper()+thefile[int(second):]" newname =" ''.join(b)" action="="" repl=" patendseqfront+str(startseq).zfill(length_startseq)+patendseqback" newname =" re.sub(patold,repl,thefile)" n="="">Number of files less than ending sequence number...Exiting.."
break
#increment sequence
startseq = int(startseq) + 1
elif ACTION=="sub":

#changing case. changing all filename to upper/lower case.
try:
foundlowercase = re.findall( "\[a-z\]|a-z" , patnew)[0]
except:
try:
founduppercase = re.findall( "\[A-Z\]|A-Z" , patnew)[0]
except: notfound=1
else:
newname = thefile.replace(thefile,thefile.upper())
else:
newname = thefile.replace(thefile,thefile.lower())

if notfound:
newname = patold_re.sub(patnew,thefile)
rename(FN, os.path.join(thepath,newname),DEBUG )
elif ACTION=="delete":
rename(FN, None ,DEBUG )

# Taken from Python recipe
def sorted_copy(alist):
# http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/52234
indices = map(_generate_index, alist)
decorated = zip(indices, alist)
decorated.sort()
return [ item for index, item in decorated ]

def _generate_index(str):
"""
Splits a string into alpha and numeric elements, which
is used as an index for sorting"
"""
#
# the index is built progressively
# using the _append function
#
index = []
def _append(fragment, alist=index):
if fragment.isdigit(): fragment = int(fragment)
alist.append(fragment)

# initialize loop
prev_isdigit = str[0].isdigit()
current_fragment = ''
# group a string into digit and non-digit parts
for char in str:
curr_isdigit = char.isdigit()
if curr_isdigit == prev_isdigit:
current_fragment += char
else:
_append(current_fragment)
current_fragment = char
prev_isdigit = curr_isdigit
_append(current_fragment)
return tuple(index)

#--------------------------------------END Functions -------------------------#

## these are options not allowed
bad_options = [ ['-s','-p'],
['-Z','-p'],['-Z','-s'],['-Z','-c'],
['-c','-b'],['-c','-i'],['-c','-p'],['-c','-s'],
#can add somemore...
]

################################## END FUNCTIONS ##################################################

if __name__ == '__main__':

basename = os.path.basename(sys.argv[0])

# create restore directory if doesn't exists
restorepath = ".restore"
if not os.path.exists(restorepath): os.mkdir(restorepath,777)
# design the restoration filename
TIME=list(time.localtime())
# get month and day into double digits string
if len( str(TIME[1]) ) < time="'-'.join(map(str,TIME))" restorefile =" os.path.join(" debug="1" inverse="0" type="f" action="sub" case="1" sort="0" args =" getopt.gnu_getopt" args =" None" function ="args[0]" filenames="args[-1]" opts ="="" options =" dict(opts)" keys =" options.keys()" h="{};rh="{}" num="num+1" choice =" raw_input(" ofilename =" h[choice]" ofilename ="="" to="olines.strip().split(" n="n+1">>> %s" %( k,rh[k][0],rh[k][1])
restoreyesno = raw_input( "Continue to restore [y|n]?: ")
if restoreyesno in ["n","N"]: break
elif restoreyesno in ["y","Y"]:
rchoice = raw_input("Enter choice to restore: ")
for n, olines in enumerate(open( os.path.join(restorepath,ofilename))):
From,To = olines.strip().split(",")
n=n+1
if "All" in rh[int(rchoice)] or "Original" in rh[int(rchoice)] :
try:
os.rename(From,To)
except Exception,e:
print "Error restoring: ",e

elif n==int(rchoice):
try:
os.rename(From,To)
except Exception,e:
print "Error restoring: ",e

# directory key
if options.has_key('-D') and options['-D'] != []:
depthcnt ,newpath, ret = pathChecker(options['-D']) #check the root, whether exists
if ret == -1:
print "%s does not exists. " % (options['-D'] )
sys.exit(2)
options['-D'] = newpath
else :
# if not -D specified, take current directory
depthcnt ,newpath, ret = pathChecker(os.getcwd())
options['-D'] = os.getcwd()

DIR=options['-D']



# check bad options...
for bad_ops in bad_options :
if combocheck( bad_ops, keys) :
usage(basename)
sys.exit(0)

# Check for maxdepth
if options.has_key('-d'):
maxdepth = int(options['-d']) + depthcnt
else:
maxdepth = depthcnt

# check for list flag - debug mode , 0 for commit.
if not options.has_key('-l'): DEBUG = 0

# check for inverse pattern search,similar to grep's -v
if options.has_key('-v'): INVERSE = 1

# check numerical sorting
if options.has_key('-n'): SORT = 1

#check file type, whether search file or directory
if options.has_key('-t'):
TYPE=options['-t']

if options.has_key('-i') :
patold=options['-i']
patnew="front"
ACTION="insert"

#insert at back of file name
if options.has_key('-b') :
patold=options['-b']
patnew="back"
ACTION="insert"

if options.has_key('-c') and options.has_key('-e') :
patold=options['-c']
patnew=options['-e']
ACTION="char"
elif options.has_key('-c') and not options.has_key('-e'):
patold=options['-c']
patnew=""
ACTION="char"

if options.has_key('-s') and options.has_key('-e') :
patold=options['-s']
patnew=options['-e']
if patnew == "" : usage(basename); sys.exit(1)
ACTION="seq"
elif options.has_key('-s') and not options.has_key('-e'):usage(basename) ;sys.exit()


if options.has_key('-p') and options.has_key('-e') :
patold=options['-p'];patnew=options['-e']
elif options.has_key('-p') and not options.has_key('-e'): usage(basename) ;sys.exit()

if options.has_key('-v'):
INVERSE=1

# ignore case sensitivity
if options.has_key('-I'): CASE=0

if options.has_key('-Z'):
patold=options['-Z']
if not patold : usage(basename);sys.exit(1)
patnew=""
ACTION="delete"

# do the walking.
try:
doWalk(DIR,maxdepth,TYPE,ACTION,patold,patnew,DEBUG,INVERSE,CASE,SORT,FileNames)
except:
usage(basename)
sys.exit()


With examples:

Quote:
Code:
Some examples:
1) Changing cases:
a) Change case of 2nd to 3rd character of all files starting with "test" to upper case.
---> filerenamer.py -c 1:3 -e "[A-Z]" -l "test*"
b) Change case of 2nd till last 4th character of all files to lower case
---> filerenamer.py -c 2:-4 -e "[a-z]" -l "*"
c) Change file name to all upper case
---> filerenamer.py -c 0: -e "[A-Z]" -l "*"

2) Change file names by substitution:
a) Change the word "test" to "foo" for all files starting with "test"
---> filerenamer.py -p "test" -e "foo" -l "test*"

3) Change file names to a number sequence
a) Change the word "test" in files starting with test to number sequence 001 to 100
---> filerenamer.py -s "test" -e "001:100" -l "test*"
b) Change the word "test" in files starting with test to "foo" and number sequence 001 to 100
---> filerenamer.py -s "test" -e "foo001:100" -l "test*"
* changes testbar.txt to foo001bar.txt
c) Change the word "test" in files starting with test to number sequence 001 to 100 followed by "foo"
---> filerenamer.py -s "test" -e "01:20foo" -l "test*"
* changes testbar.txt to 001foobar.txt

4) Removing characters by position
a) Remove the first 5 characters for files starting with "test"
---> filerenamer.py -c 0:5 -l "test*"
b) Remove the last 5 characters for files starting with "test"
---> filerenamer.py -c -5: -l "test*"
c) Remove 2nd character from all files
---> filerenamer.py -c 1 -l "*"

NB: uses the python indexing convention.
and my examples:

Finding text in files / replacing text in files.

Finding:


for i in `find -name "*.tex"`; do grep -H "sample string" $i; done


or even,


grep -r --include=*.tex "ipcluster" ./


(but don't tell Gavbo)

To show all lines containing, 'flow' but not 'viscous':

grep -r --include=*.tex "flow" ./ | grep --invert-match "viscous"




Finding, and displaying a few lines after: Use -A. -B for 'before'


for i in `find -name "*.tex"`; do grep -H "sample string" -A 2 $i; done



Replacing: From http://www.linuxforums.org/forum/linux-programming-scripting/39664-search-replace-text-within-several-files-2.html

for file in /home/clients/*/*/conf/httpd.include ; do
sed -e 's/192.168.0.5/192.168.0.10/g' "$file"
echo ----------------------------------------------------------------
done


to show the effect without making changes, and

for file in /home/clients/*/*/conf/httpd.include ; do
sed -e 's/192.168.0.5/192.168.0.10/g' "$file" > tmp_file
mv tmp_file "$file"
done


Using find:
Check operation:
$ for i in `find -name "*.tex"`; do sed -e 's/sample string/replace string/g' "$i" ; done

Commit to changes:
$ for i in `find -name "*.tex"`; do sed -e 's/sample string/replace string/g' "$i" > tmp_file; mv tmp_file "$i"; done

Tuesday, 20 January 2009

ATLAS, LAPACK installation (unfinished)

ATLAS

Check http://math-atlas.sourceforge.net/atlas_install/
Read section 8 in particular. This document was taken from section 8, and modified (as little as possible) for beans.

Download from http://downloads.sourceforge.net/math-atlas/atlas3.8.2.tar.bz2?modtime=1212787160&big_mirror=0

unpack:
$ tar xvf atlas3.8.2.tar.bz2

This is suse10.3 64 using gcc4.2.1 which is ok, according to section 8
Like section 8, I'll build dynamic libraries (position independent sounds good to me).

$ cat /proc/cpuinfo

tells me 2.00GHz. I'll try (sect 8)

-D c -DXeonCPS=2000

I'll install the libraries to /home/m.starnes/local/atlas (like sect 8 did)
so pass

-prefix=/home/me/local/atlas

64 bit installation and full LAPACK library so pass

-b 64

and

--with-netlib-lapack=

The something will be known once LAPACK is installed.

Section 8.2

$ cd ~/
$ mkdir numerics
$ cd numerics
$ bunzip2 -c ~/downloads/ATLAS/atlas3.8.2.tar.bz2 | tar xfm -
$ mv ATLAS ATLAS3.8.2
$ gunzip -c ~/downloads/python/lapack-3.1.1.tgz | tar xfm -
$ ls
ATLAS3.8.2/ lapack-3.1.1/

Set the LAPACK Make.inc appropriately.
$ cd lapack-3.1.1/
$ cp INSTALL/make.inc.LINUX make.inc

Change to the ATLAS source directory, and produce a dry-run BLDdir in order to get information for the make.inc file:

$ cd ../ATLAS3.8.2/
$ mkdir bogus
$ cd bogus
$ ../configure -b 64 -D c -DXeonCPS=2000 -Fa alg -fPIC
.....
.....
$ fgrep "F77 =" Make.inc
F77 = gfortran
$ fgrep "F77FLAGS =" Make.inc
F77FLAGS = -fomit-frame-pointer -mfpmath=387 -O2 -falign-loops=4 -fPIC -m64

OK. Delete the bogus dire, and fill in make.inc

$ cd ..
$ rm -rf bogus/
$ cd ../lapack-3.1.1/
$ vi make.inc

Set the following:
FORTRAN = gfortran
OPTS = -fomit-frame-pointer -mfpmath=387 -O2 -falign-loops=4 -fPIC -m64
DRVOPTS = $(OPTS)
NOOPT = -fomit-frame-pointer -mfpmath=387 -m64
LOADER = $(FORTRAN)
LOADOPTS = $(OPTS)
TIMER = INT_ETIME

and install LAPACK:
$ make lib

lots of output.
$ ls
BLAS html lapack_LINUX.a make.inc manpages SRC tmglib_LINUX.a
COPYING INSTALL Makefile make.inc.example README TESTING

That's a succesfull LAPACK library. Now install ATLAS.



8.3. Creating BLDdir and installing ATLAS

$ cd ../ATLAS3.8.2
$ mkdir beans64 (not animal64)
$ cd beans64
$ mkdir ~/local/atlas (my extra, this, as the directory wasn't there)
$ ../configure -b 64 -D c -DXeonCPS=2000 -Fa alg -fPIC --prefix=/home/me/local/atlas --with-netlib-lapack=/home/me/numerics/lapack-3.1.1/lapack_LINUX.a
$ make

lots of output, including some timing stuff....

$ make check

Says 0 fails. Same for $ make ptcheck (parallel)

Test performance.

$ make time

Beans shows a table with only 4 columns but it hows the clock rate to be 1995MHz, which is correct.

Clock rate=1995Mhz
single precision double precision
********************* ********************
real complex real complex
Benchmark % Clock % Clock % Clock % Clock
========= ========= ========= ========= =========
kSelMM 606.4 556.8 317.0 303.2
kGenMM 99.6 99.6 99.6 99.6
kMM_NT 99.6 98.2 98.2 96.9
kMM_TN 99.6 98.2 98.2 96.2
BIG_MM 593.4 588.3 310.1 310.8
kMV_N 58.1 98.6 52.6 72.6
kMV_T 83.3 90.9 52.7 64.0
kGER 95.4 97.4 46.7 66.1


I'll rewind beans, and put the correct time in, see if it makes any difference to this result:

$ cd ~/numerics/ATLAS3.8.2
$ mkdir bogus
$ cd bogus

$ ../configure -b 64 -D c -DXeonCPS=2000 -Fa alg -fPIC
$ fgrep "F77 =" Make.inc
F77 = gfortran
$ fgrep "F77FLAGS =" Make.inc
F77FLAGS = -fomit-frame-pointer -mfpmath=387 -O2 -falign-loops=4 -fPIC -m64
(no change)
$ cd ..
$ rm -rf bogus/
$ cd ../lapack-3.1.1/
$ vi make.inc (this time, I set NOOPTS to include -fPIC, to fix the make shared problem below)
(no changes)
$ make clean (essential on the second pass, otherwise make lib doesnt do tests)
$ make lib (takes ages)

$ cd ../ATLAS3.8.2
$ mkdir beans64 (not animal64. remove all files if this is 2nd try.)
$ cd beans64
$ mkdir ~/local/atlas (my extra, this, as the directory wasn't there. removal all files if already there.)
$ rm Make.inc (my extra, as the first try complained. Same error 2nd time, too.)
$ ../configure -b 64 -D c -DXeonCPS=2000 -Fa alg -fPIC --prefix=/home/m.starnes/local/atlas --with-netlib-lapack=/home/m.starnes/numerics/lapack-3.1.1/lapack_LINUX.a
$ make (lots of output including timing.)
$ make time

Clock rate=1995Mhz
single precision double precision
********************* ********************
real complex real complex
Benchmark % Clock % Clock % Clock % Clock
========= ========= ========= ========= =========
kSelMM 606.4 556.8 317.0 303.2
kGenMM 99.6 99.6 99.6 99.6
kMM_NT 99.6 98.2 98.2 96.9
kMM_TN 99.6 98.2 98.2 96.2
BIG_MM 586.6 584.1 311.1 310.8
kMV_N 58.1 98.6 52.6 72.6
kMV_T 83.3 90.9 52.7 64.0
kGER 95.4 97.4 46.7 66.1

Same as before. Balls. Continue.

$ make install

$ cd ~/local/atlas
$ ls
include lib
$ ls include
atlas cblas.h clapack.h
$ ls include/atlas
atlas_buildinfo.h atlas_ctrsmXover.h atlas_smv.h atlas_zdNKB.h cXover.h
atlas_cacheedge.h atlas_dmv.h atlas_smvN.h atlas_zmv.h dmm.h
atlas_cmv.h atlas_dmvN.h atlas_smvS.h atlas_zmvN.h dXover.h
atlas_cmvN.h atlas_dmvS.h atlas_smvT.h atlas_zmvS.h smm.h
atlas_cmvS.h atlas_dmvT.h atlas_sNCmm.h atlas_zmvT.h sXover.h
atlas_cmvT.h atlas_dNCmm.h atlas_sr1.h atlas_zNCmm.h zmm.h
atlas_cNCmm.h atlas_dr1.h atlas_ssysinfo.h atlas_zr1.h zXover.h
atlas_cr1.h atlas_dsysinfo.h atlas_strsmXover.h atlas_zsysinfo.h
atlas_csNKB.h atlas_dtrsmXover.h atlas_trsmNB.h atlas_ztrsmXover.h
atlas_csysinfo.h atlas_pthreads.h atlas_type.h cmm.h
$ ls lib
libatlas.a libcblas.a libf77blas.a liblapack.a libptcblas.a libptf77blas.a

lib has only, '.a' files = static libraries. Dynamic libs are not automated by ATLAS. Build the shared objects:

$ cd ~/numerics/ATLAS3.8.2/beans64/lib
$ make shared

rm -f libatlas.so liblapack.so
make libatlas.so liblapack.so libf77blas.so libcblas.so liblapack.so
make[1]: Entering directory `/home/me/numerics/ATLAS3.8.2/beans64/lib'
ld -melf_x86_64 -shared -soname libatlas.so -o libatlas.so \
--whole-archive libatlas.a --no-whole-archive -lc -lpthread -lm
ld -melf_x86_64 -shared -soname liblapack.so -o liblapack.so --whole-archive \
liblapack.a --no-whole-archive -L/usr/lib64/gcc/x86_64-suse-linux/4.2.1 -l gfortran
ld: liblapack.a(slaruv.o): relocation R_X86_64_32S against `a local symbol' can not be used when making a shared object; recompile with -fPIC
liblapack.a(slaruv.o): could not read symbols: Bad value
make[1]: *** [liblapack.so] Error 1
make[1]: Leaving directory `/home/me/numerics/ATLAS3.8.2/beans64/lib'
make: *** [shared] Error 2

Same problem with quad.

Second time, set NOOPTS to include -fPIC.