Friday, 30 January 2009
Ordered 3-D mesh with Gmsh, no extrusion, hexahedra
// Gmsh project created on Tue Aug 5 18:29:53 2008
l = 0.02; // length
h = 0.01; // duct height
w = 0.001; // width
my = 1; // y mesh
mz = 1; // mesh density across width.
mx = 40; // mesh along the length.
// Bottom surface; extrude from here, up.=========================NODES
Point(1) = { 0, h, 0, 1}; // bottom, rear right corner.
Point(2) = { l, h, 0, 1}; // bottom, front right corner.
Point(3) = { 0, 0, 0, 1}; // bottom, rear right corner.
Point(4) = { l, 0, 0, 1}; // bottom, front right corner.
Point(5) = { 0, h, w, 1}; // bottom, rear right corner.
Point(6) = { l, h, w*2, 1}; // bottom, front right corner.
Point(7) = { 0, 0, w, 1}; // bottom, rear right corner.
Point(8) = { l, 0, w*2, 1}; // bottom, front right corner.
// Cube-ish creation. Duct. ==============================================
Line(1) = {1, 2};
Line(2) = {2, 4};
Line(3) = {4, 3};
Line(4) = {3, 1};
Line(5) = {5, 6};
Line(6) = {6, 8};
Line(7) = {8, 7};
Line(8) = {7, 5};
Line(9) = {1, 5};
Line(10) = {2, 6};
Line(11) = {3, 7};
Line(12) = {4, 8};
// Define closed loops and planes for the shapes just created. These were
// selected in GMSH, rather than in the script as the negative values control
// the direction of the path. ============================================Planes
// Duct.
Line Loop(1) = {1,2,3,4};
Plane Surface(2) = {1};
Line Loop(3) = {5,6,7,8};
Plane Surface(4) = {3};
Line Loop(5) = {9,-8,-11,4};
Plane Surface(6) = {5};
Line Loop(7) = {10,6,-12,-2};
Plane Surface(8) = {7};
Line Loop(9) = {1,10,-5,-9};
Plane Surface(10) = {9};
Line Loop(11) = {3,11,-7,-12};
Plane Surface(12) = {11};
// The following code is required to make the mesh ordered =====================
// Set the lines above, as 'transfinite'. These are line numbers
Transfinite Line{1, 3, 5, 7} = mx + 1 Using Progression 1.0;
Transfinite Line{2, 4, 6, 8} = my + 1 Using Progression 1.0;
Transfinite Line{9, 10, 11, 12} = mz + 1 Using Progression 1.0;
// Create new, 'transfinite' surfaces. Matching plane surface numbers above and
// referencing same node numbers. Had to inspect
// these from gmsh. Note, curly brackets, not parentheses.
// {surface} = {nodes}
Transfinite Surface{2} = {1, 2, 3, 4};
Transfinite Surface{4} = {5, 6, 7, 8};
Transfinite Surface{6} = {1, 5, 3, 7};
Transfinite Surface{8} = {6, 2, 8, 4};
Transfinite Surface{10} = {2, 6, 1, 5};
Transfinite Surface{12} = {8, 4, 7, 3};
// Without the following recombine, the elements are tri primative.
// Duct.
Recombine Surface{2, 4, 6, 8, 10, 12};
// End of extra code to get ordered mesh ***************************************
//
// Extrude {vector} { Surface{n}; Layers{mz} ; Recombine; }
// with surface number(s) n extruded, mesh density mz1.
// Assign result to var[], to allow further work with the new surface, in var[0]
// Entries 0 to 5 are returned for these extruded faces, so steps of 6 to
// get the faces for different volumes appended to the list.
//
// Extrude to form duct width: -------------------------
//
// num[] = Extrude { 0, 0, -w} {
// Surface {
// 2
// };
// Layers{mz}; Recombine;
// };
// Can't extrude, as I want to change the thickness in that direction.
// New nodes, instead.
// Now, define a volume.
Surface Loop(13) = {4,10,2,8,12,6};
Volume(14) = {13};
// And define as transfinite. The points were selected, clockwise for both
// faces, looking in the same direction.
Transfinite Volume{14} = {5,6,8,7,1,2,4,3};
Thursday, 29 January 2009
Ordered 3-D mesh with Gmsh. Extrude 2-D surface.
// Gmsh project created on Tue Aug 5 18:29:53 2008
l = 0.02; // length
h = 0.01; // duct height
w = 0.001; // duct width
mx = 20; // mesh along the length.
my = 12; // y mesh. Divisible by 4
mz = 1; // mesh density across width.
// Tranfinite surfaces have 3 or 4 points, so can't lump the square face.
// Split it, then, into 4 sections.
Point(1) = { 0, h, h, 1}; // left wall, top to bottom.
Point(2) = { 0, h * 0.75, h, 1}; // ...
Point(3) = { 0, h * 0.5, h, 1}; // ...
Point(4) = { 0, h * 0.25, h, 1}; // ...
Point(5) = { 0, h * 0, h, 1}; // ...
Point(6) = { l, h, h, 1}; // right wall, top to bottom.
Point(7) = { l, h * 0.75, h, 1}; // ...
Point(8) = { l, h * 0.5, h, 1}; // ...
Point(9) = { l, h * 0.25, h, 1}; // ...
Point(10) = { l, h * 0.00, h, 1}; // ...
Point(11) = { l + h/4, h * 0.75, h, 1}; // inner square nodes
Point(12) = { l + h/4, h * 0.5, h, 1}; // ...
Point(13) = { l + h/4, h * 0.25, h, 1}; // ...
Point(14) = { l + 0.707 * h/2 , h/2 + 0.707 * h/2, h, 1}; // Curve.
Point(15) = { l + 1.000 * h/2 , h/2 + 0.000 * h/2, h, 1}; // Curve.
Point(16) = { l + 0.707 * h/2 , h/2 - 0.707 * h/2, h, 1}; // Curve.
// Square creation. Duct. ==============================================Squares
Line(1) = {1, 2}; // left side
Line(2) = {2, 3}; // ...
Line(3) = {3, 4}; // ...
Line(4) = {4, 5}; // ...
Line(5) = {6, 7}; // right side
Line(6) = {7, 8}; // ...
Line(7) = {8, 9}; // ...
Line(8) = {9, 10}; // ...
Line(9) = {14, 11}; // inner square, vertical
Line(10) = {11, 12}; // ...
Line(11) = {12, 13}; // ...
Line(12) = {13, 16}; // ...
Circle(13) = {6, 8, 14}; // curved end
Circle(14) = {14, 8, 15}; // ...
Circle(15) = {15, 8, 16}; // ...
Circle(16) = {16, 8, 10}; // ...
Line(17) = {1, 6}; // horizontal lines, left to right.
Line(18) = {2, 7}; // ...
Line(19) = {7, 11}; // ...
Line(20) = {3, 8}; // ...
Line(21) = {8, 12}; // ...
Line(22) = {12, 15}; // square to circle
Line(23) = {4, 9}; // ...
Line(24) = {9, 13}; // ...
Line(25) = {5, 10}; // ...
// Define closed loops and planes for the shapes just created. These were
// selected in GMSH, rather than in the script as the negative values control
// the direction of the path. ============================================Planes
// Duct.
Line Loop(26) = {17,5,-18,-1};
Plane Surface(27) = {26};
Line Loop(28) = {18,6,-20,-2};
Plane Surface(29) = {28};
Line Loop(30) = {20,7,-23,-3};
Plane Surface(31) = {30};
Line Loop(32) = {23,8,-25,-4};
Plane Surface(33) = {32};
Line Loop(34) = {13,9,-19,-5};
Plane Surface(35) = {34};
Line Loop(36) = {19,10,-21,-6};
Plane Surface(37) = {36};
Line Loop(38) = {21,11,-24,-7};
Plane Surface(39) = {38};
Line Loop(40) = {24,12,16,-8};
Plane Surface(41) = {40};
Line Loop(42) = {14,-22,-10,-9};
Plane Surface(43) = {42};
Line Loop(44) = {22,15,-12,-11};
Plane Surface(45) = {44};
// The following code is required to make the mesh ordered =====================
// Set the lines above, as 'transfinite'. These are line numbers
Transfinite Line{1, 2, 3, 4, 5, 6, 7, 8, 10, 11} = 0.25*my Using Progression 1.0; // vertical lines
Transfinite Line{17, 18, 20, 23, 25} = mx Using Progression 1.0; // horizontal lines
Transfinite Line{19, 21, 24} = 0.25*my Using Progression 1.0; // horizontal lines
Transfinite Line{9, 22, 12} = 0.25*my Using Progression 1.0; // diagonal lines
Transfinite Line{13, 14, 15, 16} = 0.25*my Using Progression 1.0; // Circle portions.
// Create new, 'transfinite' surfaces. Matching plane surface numbers above and
// referencing same node numbers. Had to inspect
// these from gmsh. Note, curly brackets, not parentheses.
// {surface} = {nodes}
Transfinite Surface{27} = {1, 2, 6, 7}; // duct, left side, top to bottom.
Transfinite Surface{29} = {2, 7, 3, 8}; // ...
Transfinite Surface{31} = {3, 8, 4, 9}; // ...
Transfinite Surface{33} = {4, 9, 5, 10}; // ...
Transfinite Surface{35} = {6, 14, 7, 11}; // duct, middle, top to bottom.
Transfinite Surface{37} = {7, 11, 8, 12}; // ...
Transfinite Surface{39} = {8, 12, 9, 13}; // ...
Transfinite Surface{41} = {9, 13, 10, 16}; // ...
Transfinite Surface{43} = {11, 14, 12, 15}; // last two of circle,top to bottom.
Transfinite Surface{45} = {12, 15, 13, 16}; // ...
// Without the following recombine, the elements are tri primative.
// Duct.
Recombine Surface{27, 29, 31, 33, 35, 37, 39, 41, 43, 45};
// End of extra code to get ordered mesh =======================================
// Extrude {vector} { Surface{n}; Layers{mz} ; Recombine; }
// with surface number(s) n extruded, mesh density mz1.
// Assign result to var[], to allow further work with the new surface, in var[0]
// Entries 0 to 5 are returned for these extruded faces, so steps of 6 to
// get the faces for different volumes appended to the list.
//
// Extrude to form duct width: -------------------------
//
num[] = Extrude { 0, 0, -w} {
Surface {
27 , 29, 31, 33, 35, 37, 39, 41, 43, 45
};
Layers{mz}; Recombine;
};
Tuesday, 27 January 2009
Tex plots, using pgfplots. Non-numeric characters on axes.
\begin{figure}[htbp]\begin{center} % plot. Z is changed by complex amounts. No good.
\mbox{
\tikzstyle{every axis legend}+= [at={(0.98,0.98)},anchor=north east] % im is 0,0 to 1,1
\begin{tikzpicture}
\begin{axis}[width=\plotw,height=\ploth,
xlabel={$x/l$}, ylabel={$\theta$},grid=major,
ytick ={0, 0.79, 1.57, 2.36, 3.14, 3.93, 4.71, 5.50, 6.28},
yticklabels={0, , , , $\pi$, , , ,$2 \pi$},
xmin = -0.1, xmax = 1.1, ymin=0, ymax=6.28]
% reference.dumptex(r1[0] / 0.02, reference.ang(r1[1]))
% 1000 Hz, 0 flow
\addplot[color = black] plot coordinates {(0.00000e+00,4.71241e+00) (5.00000e-02,4.69414e+00) (1.00000e-01,4.67588e+00) (1.50000e-01,4.65761e+00) (2.00000e-01,4.63934e+00) (2.50000e-01,4.62109e+00) (3.00000e-01,4.60281e+00) (3.50000e-01,4.58455e+00) (4.00000e-01,4.56629e+00) (4.50000e-01,4.54803e+00) (5.00000e-01,4.52976e+00) (5.50000e-01,4.51149e+00) (6.00000e-01,4.49323e+00) (6.50000e-01,4.47496e+00) (7.00000e-01,4.45670e+00) (7.50000e-01,4.43843e+00) (8.00000e-01,4.42016e+00) (8.50000e-01,4.40191e+00) (9.00000e-01,4.38364e+00) (9.50000e-01,4.36537e+00) (1.00000e+00,4.34711e+00)};
\legend{Phase.\\}
\end{axis} \end{tikzpicture}
}
\caption{Zero flow, one-dimensional results at $f=1kHz$.}
\label{fig:m95acou2a_pvf} \end{center} \end{figure}
Monday, 26 January 2009
Saturday, 24 January 2009
Mass file renaming.
Thankfully: A civilised way:
Rename all .xcf.png files, with .png:
rename 's/.xcf.png/.png/' ????.xcf.png
Or, the hard way:
Copied directly from http://www.linuxquestions.org/blog/ghostdog74-290378/2008/9/17/simple-mass-file-renamer-and-deleter-python-1190/
by 'ghostdog74':
This can:
1) Change upper case to lower case and vice versa
2) Change file names by pattern substitution
3) Change file names by number sequence
4) Insert pattern in front of files
5) Insert pattern at the end of files.
6) Simple sort by number.
7) Manual revert of changed files.
8) Deletion of files by pattern match only.
9) Ability to recurse directory
10) Inverse match pattern, like grep -v
11) Case sensitivity search
12) Remove string by character positions. eg 0:10 => remove from start to 10th character.
With examples:
and my examples:
Rename all .xcf.png files, with .png:
rename 's/.xcf.png/.png/' ????.xcf.png
Or, the hard way:
Copied directly from http://www.linuxquestions.org/blog/ghostdog74-290378/2008/9/17/simple-mass-file-renamer-and-deleter-python-1190/
by 'ghostdog74':
This can:
1) Change upper case to lower case and vice versa
2) Change file names by pattern substitution
3) Change file names by number sequence
4) Insert pattern in front of files
5) Insert pattern at the end of files.
6) Simple sort by number.
7) Manual revert of changed files.
8) Deletion of files by pattern match only.
9) Ability to recurse directory
10) Inverse match pattern, like grep -v
11) Case sensitivity search
12) Remove string by character positions. eg 0:10 => remove from start to 10th character.
#!/usr/bin/env python
import sys,os,getopt,glob,string,re,operator,fnmatch,time
################################## FUNCTIONS #################################################
def usage(name):
print """
%s
Mini File Renamer and Deleter v1.0.2 - Copyright 2008
Author: ghostdog74
%s
usage: %s [-h] [-D dir] [-t f|d ][-[s|p] oldpat] [-e newpat ] [ -i pattern ][ -b patten ] [-d depth] [-Z|-v|-n|-r|-I] [-l] [files]
-D Directory to start rename/delete. Use quotes for directories with spaces
If no directory is specified, current working directory is assumed
Eg: Use -D c:\\test\\ (for Windows, end with trailing back slash)
-h Prints help page
-t Type of files. Either files(f) or directories(d). Default is files, so can omit -t f
-s Sequence substitution. Specify pattern to be substituted. Must be specified with -e.
-p Pattern substitution. Must be specified with -e (pattern to change to)
eg.
1) To change whole jpeg file name to upper case ==> -p ".*" -e "A-Z" "*.jpeg"
2) To change the word "TEST" to lower case in jpeg file ==> -p "TEST" -e "a-z" "*.jpeg"
3) To remove all numbers from directory name ==> -p "[0-9]+" -e '' -t d -l "*"
4) To remove first character from file/directory ==> -p "^." -e '' -l "*.jpeg"
5) To remove special characters ==> -p "[\"^']" -e "back" "*"
-e * When used with -s, indicates ending sequence pattern. Can include alphanumberic. Must use ":" to specify range
Eg
1) -s "test" -e "01:10" will replace 'test' in files from '01' to '10'. If more than 1 files with 'test',
will go by sequence, ie '02' , '03' etc.
2) -s "test" -e "###01:11@@@" will replace 'test' from '###01@@@' , followed by '###02@@@' and so on to ###11@@@
* When used with -p, indicates pattern to change to.
* When used with -c, specify -e "[A-Z]" to change to uppercase, -e "[a-z]" to change to lower case.
-i Insert pattern to infront of file name.
-b Insert pattern to back of file name.
-c Remove characters in file name by position, position index start from 0. Always use -l to verify files to be changed.
eg
1) -c 1 ==> remove 2nd character
2) -c -1 ===> remove last character.
3) -c -2: ===> remove from last 2nd character onwards.
4) -c 3: ==> remove from 4th character onwards
5) -c 4:10 ==> remove from 5th to 10th character
6) -c :3 ==> remove from start to 3rd character.
7) -c 1:3 -e "[A-Z]" ==> change positions specified to uppercase
-l List all files with pattern only. No renaming. Useful for verifying what will be changed.
-r Used alone. Enable restoration of previous commands. eg %s -r
-n Simple Numerical sort. Specify -n to turn numerical sorting on. Only works on files of the same structure.
-d # of directories down to do rename (default = 0). ie: Directory depth level
-I Case insensitive pattern search. eg -p "rot" -e "" -I -l "*.txt". Find rot,ROT,roT,RoT ..
-v Wildcard pattern reversal. eg -v "*.bat" : Files that doesn't end with .bat.
-Z Do deletion of files.
eg -d 4 -Z ".*01*" -l -v "*.txt" ==> delete all files that doesn't end with .txt and with the pattern "01" in the
file name, 4 levels deep into current directory
[files] List of files to be renamed/deleted. Can have wildcards. eg test*.txt
To specify all files ==> "*". Will not work if not specified.
""" % ("=" *100 , "=" * 100 ,name,name )
def pathChecker(path):
''' Function to determine correct path or file exists
and then returns the number of count of the path separator
as an indicator of the depth of the path.
'''
if os.path.exists(path):
pathcount=path.count(os.sep)
if pathcount > 1:
return int(pathcount) , path , 0
else:return 1,1,1
else:
return 0,0,-1
def combocheck(s,k):
'''Function to check the options user keyed in against a set of bad options
Input : s => bad options
k => list of user supplied keys...eg -S -s -N..etc
'''
v = len(s); t = 0 ##store lenght of predefined bad options, t=0 to count matched bad ops
for x in s:
if x in k:
t = t + 1
## if all bad options found
if v == t : return True
else: return False
def doWalk(DIR=None,maxdepth=1,TYPE="f",ACTION="sub",patold=None,patnew=None,DEBUG=1,INVERSE=0,CASE=1,SORT=0,FileNamesArgs=None):
''' Traverse the directory specified until depth level,
looking for files with the correct patterns and rename them
accordingly
'''
GlobbedTypeList=[]
AllFilesGlobbed=[]
# convert into reg expression syntax in order to search. eg "*.txt" to ".*txt$"
regex = fnmatch.translate(FileNamesArgs)
reobj = re.compile(regex)
for ROOT,DIRECTORY,FILES in os.walk(DIR,True):
#do for files less or equal to maxdepth
if ROOT.count(os.sep) <= int(maxdepth): if FileNamesArgs is not None: if INVERSE: for FI in os.listdir( ROOT ): if not reobj.search(FI): AllFilesGlobbed.append( os.path.join(ROOT,FI )) else: try: allfiles = glob.glob(os.path.join(ROOT,FileNamesArgs)) except Exception,e: pass else: if allfiles: for found in allfiles: if found not in AllFilesGlobbed: AllFilesGlobbed.append(found) if TYPE=="d": for dirname in AllFilesGlobbed: if os.path.isdir(dirname): if not dirname in GlobbedTypeList: GlobbedTypeList.append([dirname,dirname.count(os.sep)]) else: for filenames in AllFilesGlobbed: if os.path.isfile(filenames): if not filenames in GlobbedTypeList: GlobbedTypeList.append(filenames) if SORT==1: GlobbedTypeList = sorted_copy(GlobbedTypeList) else: GlobbedTypeList=sorted(GlobbedTypeList, key=(operator.itemgetter(0))) # do various actions doAction(GlobbedTypeList,TYPE,ACTION,patold,patnew,CASE,SORT,DEBUG) def brake(): raw_input("Enter") def clearscreen(rows): for i in range(rows): print def rename(FROM,TO="",DEBUG=1): if FROM == TO:return if DEBUG==0 : if TO : try: os.rename(FROM,TO) except Exception,e: print "Error : ",e else: print FROM , " is renamed to ", TO # store to restore file. simple mechanism. Use pickle/shelve ?? open(restorefile,"a").write("""%s,%s\n""" %( TO,FROM )) elif not TO : print "Deleting " ,FROM if os.path.isdir(FROM): try: os.removedirs(FROM) #or use os.rmdir except Exception,e: print "Error: ",e elif os.path.isfile(FROM): try : os.remove(FROM) except Exception,e: print "Error: ",e else: print "==>>>> ", "[" ,FROM ,"]==>[",TO,"]"
def changecase(patnew,thefile):
# if -c and -e option and -e [A-Z] or -e [a-z]
try:
foundlowercase = re.findall( "\[a-z\]|a-z" , patnew)[0]
except:
try:
founduppercase = re.findall( "\[A-Z\]|A-Z" , patnew)[0]
except: notfound=1
else:
newname = thefile.replace(thefile,thefile.upper())
else:
newname = thefile.replace(thefile,thefile.lower())
def doAction(FILES,TYPE="d", ACTION="sub", patold="" ,patnew="", CASE=1, SORT=0,DEBUG=1):
'''Function to do sequence substitution
Input: FILES => A list of globbed files to be processed.
TYPE => f = files, d = directories
patold => The pattern in the file to be substituted
patnew => The new pattern to replace the old
DEBUG => 1 : Do a listing only
0 : Do substitution, and renaming of files
'''
notfound=0 #flag for doing case change.
if TYPE=="d":
# if renaming directories, rename from the last(highest) level. So have to sort according to maxdepth
FILES=sorted(FILES, key=(operator.itemgetter(1)),reverse=True)
if CASE == 0:
patold_re = re.compile(patold,re.IGNORECASE)
elif CASE:
patold_re = re.compile(patold)
if ACTION=="seq":
# see if in format 001:020 ...this indicates sequence
patnew_re =re.compile("(\w*\d*\D*)(\d+)[:](\d+)(\D*\w*\d*)")
seq = patnew_re.findall(patnew)[0]
startseq=seq[1]; endseq=seq[-2]
patendseqfront = seq[0]
patendseqback = seq[-1]
if endseq and int(endseq) < endseq =" endseq,startseq" endseq="startseq" endseq ="="" endseq="len(FILES)" length_startseq =" len(str(startseq))" type="="" fn="fn[0]" type="="" fn="fn" thefile =" os.path.split(FN)" action="="" patnew ="="" newname =" patold+thefile" patnew ="="" newname =" thefile+patold" action="="">:
patdigit=re.compile("(-)*(\d*):(-)*(\d*)")
b= list(thefile)
if patnew and re.search( "\[a-z\]|a-z" , patnew) :
caseflag=1
elif patnew and re.search( "\[A-Z\]|A-Z" , patnew):
caseflag=2
else:caseflag=0
# if single digit
if ":" not in patold and ( int(patold) < caseflag="="1" caseflag="="2:" newname =" ''.join(b)" foundit =" patdigit.findall(patold)[0]" first = "" second = "" caseflag="="1:" newname =" thefile[0:int(first)]+thefile[int(first):].lower()" caseflag="="2:" newname =" thefile[0:int(first)]+thefile[int(first):].upper()" newname =" thefile[" caseflag="="1:" newname =" thefile[:int(second)].lower()+thefile[int(second):]" caseflag="="2:" newname =" thefile[:int(second)].upper()+thefile[" newname =" thefile[" caseflag="="1:" newname =" thefile[0:int(first)]+thefile[int(first):int(second)].lower()+thefile[int(second):]" caseflag="="2:" newname =" thefile[0:int(first)]+thefile[int(first):int(second)].upper()+thefile[int(second):]" newname =" ''.join(b)" action="="" repl=" patendseqfront+str(startseq).zfill(length_startseq)+patendseqback" newname =" re.sub(patold,repl,thefile)" n="="">Number of files less than ending sequence number...Exiting.."
break
#increment sequence
startseq = int(startseq) + 1
elif ACTION=="sub":
#changing case. changing all filename to upper/lower case.
try:
foundlowercase = re.findall( "\[a-z\]|a-z" , patnew)[0]
except:
try:
founduppercase = re.findall( "\[A-Z\]|A-Z" , patnew)[0]
except: notfound=1
else:
newname = thefile.replace(thefile,thefile.upper())
else:
newname = thefile.replace(thefile,thefile.lower())
if notfound:
newname = patold_re.sub(patnew,thefile)
rename(FN, os.path.join(thepath,newname),DEBUG )
elif ACTION=="delete":
rename(FN, None ,DEBUG )
# Taken from Python recipe
def sorted_copy(alist):
# http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/52234
indices = map(_generate_index, alist)
decorated = zip(indices, alist)
decorated.sort()
return [ item for index, item in decorated ]
def _generate_index(str):
"""
Splits a string into alpha and numeric elements, which
is used as an index for sorting"
"""
#
# the index is built progressively
# using the _append function
#
index = []
def _append(fragment, alist=index):
if fragment.isdigit(): fragment = int(fragment)
alist.append(fragment)
# initialize loop
prev_isdigit = str[0].isdigit()
current_fragment = ''
# group a string into digit and non-digit parts
for char in str:
curr_isdigit = char.isdigit()
if curr_isdigit == prev_isdigit:
current_fragment += char
else:
_append(current_fragment)
current_fragment = char
prev_isdigit = curr_isdigit
_append(current_fragment)
return tuple(index)
#--------------------------------------END Functions -------------------------#
## these are options not allowed
bad_options = [ ['-s','-p'],
['-Z','-p'],['-Z','-s'],['-Z','-c'],
['-c','-b'],['-c','-i'],['-c','-p'],['-c','-s'],
#can add somemore...
]
################################## END FUNCTIONS ##################################################
if __name__ == '__main__':
basename = os.path.basename(sys.argv[0])
# create restore directory if doesn't exists
restorepath = ".restore"
if not os.path.exists(restorepath): os.mkdir(restorepath,777)
# design the restoration filename
TIME=list(time.localtime())
# get month and day into double digits string
if len( str(TIME[1]) ) < time="'-'.join(map(str,TIME))" restorefile =" os.path.join(" debug="1" inverse="0" type="f" action="sub" case="1" sort="0" args =" getopt.gnu_getopt" args =" None" function ="args[0]" filenames="args[-1]" opts ="="" options =" dict(opts)" keys =" options.keys()" h="{};rh="{}" num="num+1" choice =" raw_input(" ofilename =" h[choice]" ofilename ="="" to="olines.strip().split(" n="n+1">>> %s" %( k,rh[k][0],rh[k][1])
restoreyesno = raw_input( "Continue to restore [y|n]?: ")
if restoreyesno in ["n","N"]: break
elif restoreyesno in ["y","Y"]:
rchoice = raw_input("Enter choice to restore: ")
for n, olines in enumerate(open( os.path.join(restorepath,ofilename))):
From,To = olines.strip().split(",")
n=n+1
if "All" in rh[int(rchoice)] or "Original" in rh[int(rchoice)] :
try:
os.rename(From,To)
except Exception,e:
print "Error restoring: ",e
elif n==int(rchoice):
try:
os.rename(From,To)
except Exception,e:
print "Error restoring: ",e
# directory key
if options.has_key('-D') and options['-D'] != []:
depthcnt ,newpath, ret = pathChecker(options['-D']) #check the root, whether exists
if ret == -1:
print "%s does not exists. " % (options['-D'] )
sys.exit(2)
options['-D'] = newpath
else :
# if not -D specified, take current directory
depthcnt ,newpath, ret = pathChecker(os.getcwd())
options['-D'] = os.getcwd()
DIR=options['-D']
# check bad options...
for bad_ops in bad_options :
if combocheck( bad_ops, keys) :
usage(basename)
sys.exit(0)
# Check for maxdepth
if options.has_key('-d'):
maxdepth = int(options['-d']) + depthcnt
else:
maxdepth = depthcnt
# check for list flag - debug mode , 0 for commit.
if not options.has_key('-l'): DEBUG = 0
# check for inverse pattern search,similar to grep's -v
if options.has_key('-v'): INVERSE = 1
# check numerical sorting
if options.has_key('-n'): SORT = 1
#check file type, whether search file or directory
if options.has_key('-t'):
TYPE=options['-t']
if options.has_key('-i') :
patold=options['-i']
patnew="front"
ACTION="insert"
#insert at back of file name
if options.has_key('-b') :
patold=options['-b']
patnew="back"
ACTION="insert"
if options.has_key('-c') and options.has_key('-e') :
patold=options['-c']
patnew=options['-e']
ACTION="char"
elif options.has_key('-c') and not options.has_key('-e'):
patold=options['-c']
patnew=""
ACTION="char"
if options.has_key('-s') and options.has_key('-e') :
patold=options['-s']
patnew=options['-e']
if patnew == "" : usage(basename); sys.exit(1)
ACTION="seq"
elif options.has_key('-s') and not options.has_key('-e'):usage(basename) ;sys.exit()
if options.has_key('-p') and options.has_key('-e') :
patold=options['-p'];patnew=options['-e']
elif options.has_key('-p') and not options.has_key('-e'): usage(basename) ;sys.exit()
if options.has_key('-v'):
INVERSE=1
# ignore case sensitivity
if options.has_key('-I'): CASE=0
if options.has_key('-Z'):
patold=options['-Z']
if not patold : usage(basename);sys.exit(1)
patnew=""
ACTION="delete"
# do the walking.
try:
doWalk(DIR,maxdepth,TYPE,ACTION,patold,patnew,DEBUG,INVERSE,CASE,SORT,FileNames)
except:
usage(basename)
sys.exit()
With examples:
Quote:
Code:
Some examples:
1) Changing cases:
a) Change case of 2nd to 3rd character of all files starting with "test" to upper case.
---> filerenamer.py -c 1:3 -e "[A-Z]" -l "test*"
b) Change case of 2nd till last 4th character of all files to lower case
---> filerenamer.py -c 2:-4 -e "[a-z]" -l "*"
c) Change file name to all upper case
---> filerenamer.py -c 0: -e "[A-Z]" -l "*"
2) Change file names by substitution:
a) Change the word "test" to "foo" for all files starting with "test"
---> filerenamer.py -p "test" -e "foo" -l "test*"
3) Change file names to a number sequence
a) Change the word "test" in files starting with test to number sequence 001 to 100
---> filerenamer.py -s "test" -e "001:100" -l "test*"
b) Change the word "test" in files starting with test to "foo" and number sequence 001 to 100
---> filerenamer.py -s "test" -e "foo001:100" -l "test*"
* changes testbar.txt to foo001bar.txt
c) Change the word "test" in files starting with test to number sequence 001 to 100 followed by "foo"
---> filerenamer.py -s "test" -e "01:20foo" -l "test*"
* changes testbar.txt to 001foobar.txt
4) Removing characters by position
a) Remove the first 5 characters for files starting with "test"
---> filerenamer.py -c 0:5 -l "test*"
b) Remove the last 5 characters for files starting with "test"
---> filerenamer.py -c -5: -l "test*"
c) Remove 2nd character from all files
---> filerenamer.py -c 1 -l "*"
NB: uses the python indexing convention.
Finding text in files / replacing text in files.
Finding:
or even,
(but don't tell Gavbo)
To show all lines containing, 'flow' but not 'viscous':
Finding, and displaying a few lines after: Use -A. -B for 'before'
Replacing: From http://www.linuxforums.org/forum/linux-programming-scripting/39664-search-replace-text-within-several-files-2.html
to show the effect without making changes, and
Using find:
Check operation:
$ for i in `find -name "*.tex"`; do sed -e 's/sample string/replace string/g' "$i" ; done
Commit to changes:
$ for i in `find -name "*.tex"`; do sed -e 's/sample string/replace string/g' "$i" > tmp_file; mv tmp_file "$i"; done
for i in `find -name "*.tex"`; do grep -H "sample string" $i; done
or even,
grep -r --include=*.tex "ipcluster" ./
(but don't tell Gavbo)
To show all lines containing, 'flow' but not 'viscous':
grep -r --include=*.tex "flow" ./ | grep --invert-match "viscous"
Finding, and displaying a few lines after: Use -A. -B for 'before'
for i in `find -name "*.tex"`; do grep -H "sample string" -A 2 $i; done
Replacing: From http://www.linuxforums.org/forum/linux-programming-scripting/39664-search-replace-text-within-several-files-2.html
for file in /home/clients/*/*/conf/httpd.include ; do
sed -e 's/192.168.0.5/192.168.0.10/g' "$file"
echo ----------------------------------------------------------------
done
to show the effect without making changes, and
for file in /home/clients/*/*/conf/httpd.include ; do
sed -e 's/192.168.0.5/192.168.0.10/g' "$file" > tmp_file
mv tmp_file "$file"
done
Using find:
Check operation:
$ for i in `find -name "*.tex"`; do sed -e 's/sample string/replace string/g' "$i" ; done
Commit to changes:
$ for i in `find -name "*.tex"`; do sed -e 's/sample string/replace string/g' "$i" > tmp_file; mv tmp_file "$i"; done
Tuesday, 20 January 2009
ATLAS, LAPACK installation (unfinished)
ATLAS
Check http://math-atlas.sourceforge.net/atlas_install/
Read section 8 in particular. This document was taken from section 8, and modified (as little as possible) for beans.
Download from http://downloads.sourceforge.net/math-atlas/atlas3.8.2.tar.bz2?modtime=1212787160&big_mirror=0
unpack:
$ tar xvf atlas3.8.2.tar.bz2
This is suse10.3 64 using gcc4.2.1 which is ok, according to section 8
Like section 8, I'll build dynamic libraries (position independent sounds good to me).
$ cat /proc/cpuinfo
tells me 2.00GHz. I'll try (sect 8)
-D c -DXeonCPS=2000
I'll install the libraries to /home/m.starnes/local/atlas (like sect 8 did)
so pass
-prefix=/home/me/local/atlas
64 bit installation and full LAPACK library so pass
-b 64
and
--with-netlib-lapack=
The something will be known once LAPACK is installed.
Section 8.2
$ cd ~/
$ mkdir numerics
$ cd numerics
$ bunzip2 -c ~/downloads/ATLAS/atlas3.8.2.tar.bz2 | tar xfm -
$ mv ATLAS ATLAS3.8.2
$ gunzip -c ~/downloads/python/lapack-3.1.1.tgz | tar xfm -
$ ls
ATLAS3.8.2/ lapack-3.1.1/
Set the LAPACK Make.inc appropriately.
$ cd lapack-3.1.1/
$ cp INSTALL/make.inc.LINUX make.inc
Change to the ATLAS source directory, and produce a dry-run BLDdir in order to get information for the make.inc file:
$ cd ../ATLAS3.8.2/
$ mkdir bogus
$ cd bogus
$ ../configure -b 64 -D c -DXeonCPS=2000 -Fa alg -fPIC
.....
.....
$ fgrep "F77 =" Make.inc
F77 = gfortran
$ fgrep "F77FLAGS =" Make.inc
F77FLAGS = -fomit-frame-pointer -mfpmath=387 -O2 -falign-loops=4 -fPIC -m64
OK. Delete the bogus dire, and fill in make.inc
$ cd ..
$ rm -rf bogus/
$ cd ../lapack-3.1.1/
$ vi make.inc
Set the following:
FORTRAN = gfortran
OPTS = -fomit-frame-pointer -mfpmath=387 -O2 -falign-loops=4 -fPIC -m64
DRVOPTS = $(OPTS)
NOOPT = -fomit-frame-pointer -mfpmath=387 -m64
LOADER = $(FORTRAN)
LOADOPTS = $(OPTS)
TIMER = INT_ETIME
and install LAPACK:
$ make lib
lots of output.
$ ls
BLAS html lapack_LINUX.a make.inc manpages SRC tmglib_LINUX.a
COPYING INSTALL Makefile make.inc.example README TESTING
That's a succesfull LAPACK library. Now install ATLAS.
8.3. Creating BLDdir and installing ATLAS
$ cd ../ATLAS3.8.2
$ mkdir beans64 (not animal64)
$ cd beans64
$ mkdir ~/local/atlas (my extra, this, as the directory wasn't there)
$ ../configure -b 64 -D c -DXeonCPS=2000 -Fa alg -fPIC --prefix=/home/me/local/atlas --with-netlib-lapack=/home/me/numerics/lapack-3.1.1/lapack_LINUX.a
$ make
lots of output, including some timing stuff....
$ make check
Says 0 fails. Same for $ make ptcheck (parallel)
Test performance.
$ make time
Beans shows a table with only 4 columns but it hows the clock rate to be 1995MHz, which is correct.
Clock rate=1995Mhz
single precision double precision
********************* ********************
real complex real complex
Benchmark % Clock % Clock % Clock % Clock
========= ========= ========= ========= =========
kSelMM 606.4 556.8 317.0 303.2
kGenMM 99.6 99.6 99.6 99.6
kMM_NT 99.6 98.2 98.2 96.9
kMM_TN 99.6 98.2 98.2 96.2
BIG_MM 593.4 588.3 310.1 310.8
kMV_N 58.1 98.6 52.6 72.6
kMV_T 83.3 90.9 52.7 64.0
kGER 95.4 97.4 46.7 66.1
I'll rewind beans, and put the correct time in, see if it makes any difference to this result:
$ cd ~/numerics/ATLAS3.8.2
$ mkdir bogus
$ cd bogus
$ ../configure -b 64 -D c -DXeonCPS=2000 -Fa alg -fPIC
$ fgrep "F77 =" Make.inc
F77 = gfortran
$ fgrep "F77FLAGS =" Make.inc
F77FLAGS = -fomit-frame-pointer -mfpmath=387 -O2 -falign-loops=4 -fPIC -m64
(no change)
$ cd ..
$ rm -rf bogus/
$ cd ../lapack-3.1.1/
$ vi make.inc (this time, I set NOOPTS to include -fPIC, to fix the make shared problem below)
(no changes)
$ make clean (essential on the second pass, otherwise make lib doesnt do tests)
$ make lib (takes ages)
$ cd ../ATLAS3.8.2
$ mkdir beans64 (not animal64. remove all files if this is 2nd try.)
$ cd beans64
$ mkdir ~/local/atlas (my extra, this, as the directory wasn't there. removal all files if already there.)
$ rm Make.inc (my extra, as the first try complained. Same error 2nd time, too.)
$ ../configure -b 64 -D c -DXeonCPS=2000 -Fa alg -fPIC --prefix=/home/m.starnes/local/atlas --with-netlib-lapack=/home/m.starnes/numerics/lapack-3.1.1/lapack_LINUX.a
$ make (lots of output including timing.)
$ make time
Clock rate=1995Mhz
single precision double precision
********************* ********************
real complex real complex
Benchmark % Clock % Clock % Clock % Clock
========= ========= ========= ========= =========
kSelMM 606.4 556.8 317.0 303.2
kGenMM 99.6 99.6 99.6 99.6
kMM_NT 99.6 98.2 98.2 96.9
kMM_TN 99.6 98.2 98.2 96.2
BIG_MM 586.6 584.1 311.1 310.8
kMV_N 58.1 98.6 52.6 72.6
kMV_T 83.3 90.9 52.7 64.0
kGER 95.4 97.4 46.7 66.1
Same as before. Balls. Continue.
$ make install
$ cd ~/local/atlas
$ ls
include lib
$ ls include
atlas cblas.h clapack.h
$ ls include/atlas
atlas_buildinfo.h atlas_ctrsmXover.h atlas_smv.h atlas_zdNKB.h cXover.h
atlas_cacheedge.h atlas_dmv.h atlas_smvN.h atlas_zmv.h dmm.h
atlas_cmv.h atlas_dmvN.h atlas_smvS.h atlas_zmvN.h dXover.h
atlas_cmvN.h atlas_dmvS.h atlas_smvT.h atlas_zmvS.h smm.h
atlas_cmvS.h atlas_dmvT.h atlas_sNCmm.h atlas_zmvT.h sXover.h
atlas_cmvT.h atlas_dNCmm.h atlas_sr1.h atlas_zNCmm.h zmm.h
atlas_cNCmm.h atlas_dr1.h atlas_ssysinfo.h atlas_zr1.h zXover.h
atlas_cr1.h atlas_dsysinfo.h atlas_strsmXover.h atlas_zsysinfo.h
atlas_csNKB.h atlas_dtrsmXover.h atlas_trsmNB.h atlas_ztrsmXover.h
atlas_csysinfo.h atlas_pthreads.h atlas_type.h cmm.h
$ ls lib
libatlas.a libcblas.a libf77blas.a liblapack.a libptcblas.a libptf77blas.a
lib has only, '.a' files = static libraries. Dynamic libs are not automated by ATLAS. Build the shared objects:
$ cd ~/numerics/ATLAS3.8.2/beans64/lib
$ make shared
rm -f libatlas.so liblapack.so
make libatlas.so liblapack.so libf77blas.so libcblas.so liblapack.so
make[1]: Entering directory `/home/me/numerics/ATLAS3.8.2/beans64/lib'
ld -melf_x86_64 -shared -soname libatlas.so -o libatlas.so \
--whole-archive libatlas.a --no-whole-archive -lc -lpthread -lm
ld -melf_x86_64 -shared -soname liblapack.so -o liblapack.so --whole-archive \
liblapack.a --no-whole-archive -L/usr/lib64/gcc/x86_64-suse-linux/4.2.1 -l gfortran
ld: liblapack.a(slaruv.o): relocation R_X86_64_32S against `a local symbol' can not be used when making a shared object; recompile with -fPIC
liblapack.a(slaruv.o): could not read symbols: Bad value
make[1]: *** [liblapack.so] Error 1
make[1]: Leaving directory `/home/me/numerics/ATLAS3.8.2/beans64/lib'
make: *** [shared] Error 2
Same problem with quad.
Second time, set NOOPTS to include -fPIC.
Check http://math-atlas.sourceforge.net/atlas_install/
Read section 8 in particular. This document was taken from section 8, and modified (as little as possible) for beans.
Download from http://downloads.sourceforge.net/math-atlas/atlas3.8.2.tar.bz2?modtime=1212787160&big_mirror=0
unpack:
$ tar xvf atlas3.8.2.tar.bz2
This is suse10.3 64 using gcc4.2.1 which is ok, according to section 8
Like section 8, I'll build dynamic libraries (position independent sounds good to me).
$ cat /proc/cpuinfo
tells me 2.00GHz. I'll try (sect 8)
-D c -DXeonCPS=2000
I'll install the libraries to /home/m.starnes/local/atlas (like sect 8 did)
so pass
-prefix=/home/me/local/atlas
64 bit installation and full LAPACK library so pass
-b 64
and
--with-netlib-lapack=
The something will be known once LAPACK is installed.
Section 8.2
$ cd ~/
$ mkdir numerics
$ cd numerics
$ bunzip2 -c ~/downloads/ATLAS/atlas3.8.2.tar.bz2 | tar xfm -
$ mv ATLAS ATLAS3.8.2
$ gunzip -c ~/downloads/python/lapack-3.1.1.tgz | tar xfm -
$ ls
ATLAS3.8.2/ lapack-3.1.1/
Set the LAPACK Make.inc appropriately.
$ cd lapack-3.1.1/
$ cp INSTALL/make.inc.LINUX make.inc
Change to the ATLAS source directory, and produce a dry-run BLDdir in order to get information for the make.inc file:
$ cd ../ATLAS3.8.2/
$ mkdir bogus
$ cd bogus
$ ../configure -b 64 -D c -DXeonCPS=2000 -Fa alg -fPIC
.....
.....
$ fgrep "F77 =" Make.inc
F77 = gfortran
$ fgrep "F77FLAGS =" Make.inc
F77FLAGS = -fomit-frame-pointer -mfpmath=387 -O2 -falign-loops=4 -fPIC -m64
OK. Delete the bogus dire, and fill in make.inc
$ cd ..
$ rm -rf bogus/
$ cd ../lapack-3.1.1/
$ vi make.inc
Set the following:
FORTRAN = gfortran
OPTS = -fomit-frame-pointer -mfpmath=387 -O2 -falign-loops=4 -fPIC -m64
DRVOPTS = $(OPTS)
NOOPT = -fomit-frame-pointer -mfpmath=387 -m64
LOADER = $(FORTRAN)
LOADOPTS = $(OPTS)
TIMER = INT_ETIME
and install LAPACK:
$ make lib
lots of output.
$ ls
BLAS html lapack_LINUX.a make.inc manpages SRC tmglib_LINUX.a
COPYING INSTALL Makefile make.inc.example README TESTING
That's a succesfull LAPACK library. Now install ATLAS.
8.3. Creating BLDdir and installing ATLAS
$ cd ../ATLAS3.8.2
$ mkdir beans64 (not animal64)
$ cd beans64
$ mkdir ~/local/atlas (my extra, this, as the directory wasn't there)
$ ../configure -b 64 -D c -DXeonCPS=2000 -Fa alg -fPIC --prefix=/home/me/local/atlas --with-netlib-lapack=/home/me/numerics/lapack-3.1.1/lapack_LINUX.a
$ make
lots of output, including some timing stuff....
$ make check
Says 0 fails. Same for $ make ptcheck (parallel)
Test performance.
$ make time
Beans shows a table with only 4 columns but it hows the clock rate to be 1995MHz, which is correct.
Clock rate=1995Mhz
single precision double precision
********************* ********************
real complex real complex
Benchmark % Clock % Clock % Clock % Clock
========= ========= ========= ========= =========
kSelMM 606.4 556.8 317.0 303.2
kGenMM 99.6 99.6 99.6 99.6
kMM_NT 99.6 98.2 98.2 96.9
kMM_TN 99.6 98.2 98.2 96.2
BIG_MM 593.4 588.3 310.1 310.8
kMV_N 58.1 98.6 52.6 72.6
kMV_T 83.3 90.9 52.7 64.0
kGER 95.4 97.4 46.7 66.1
I'll rewind beans, and put the correct time in, see if it makes any difference to this result:
$ cd ~/numerics/ATLAS3.8.2
$ mkdir bogus
$ cd bogus
$ ../configure -b 64 -D c -DXeonCPS=2000 -Fa alg -fPIC
$ fgrep "F77 =" Make.inc
F77 = gfortran
$ fgrep "F77FLAGS =" Make.inc
F77FLAGS = -fomit-frame-pointer -mfpmath=387 -O2 -falign-loops=4 -fPIC -m64
(no change)
$ cd ..
$ rm -rf bogus/
$ cd ../lapack-3.1.1/
$ vi make.inc (this time, I set NOOPTS to include -fPIC, to fix the make shared problem below)
(no changes)
$ make clean (essential on the second pass, otherwise make lib doesnt do tests)
$ make lib (takes ages)
$ cd ../ATLAS3.8.2
$ mkdir beans64 (not animal64. remove all files if this is 2nd try.)
$ cd beans64
$ mkdir ~/local/atlas (my extra, this, as the directory wasn't there. removal all files if already there.)
$ rm Make.inc (my extra, as the first try complained. Same error 2nd time, too.)
$ ../configure -b 64 -D c -DXeonCPS=2000 -Fa alg -fPIC --prefix=/home/m.starnes/local/atlas --with-netlib-lapack=/home/m.starnes/numerics/lapack-3.1.1/lapack_LINUX.a
$ make (lots of output including timing.)
$ make time
Clock rate=1995Mhz
single precision double precision
********************* ********************
real complex real complex
Benchmark % Clock % Clock % Clock % Clock
========= ========= ========= ========= =========
kSelMM 606.4 556.8 317.0 303.2
kGenMM 99.6 99.6 99.6 99.6
kMM_NT 99.6 98.2 98.2 96.9
kMM_TN 99.6 98.2 98.2 96.2
BIG_MM 586.6 584.1 311.1 310.8
kMV_N 58.1 98.6 52.6 72.6
kMV_T 83.3 90.9 52.7 64.0
kGER 95.4 97.4 46.7 66.1
Same as before. Balls. Continue.
$ make install
$ cd ~/local/atlas
$ ls
include lib
$ ls include
atlas cblas.h clapack.h
$ ls include/atlas
atlas_buildinfo.h atlas_ctrsmXover.h atlas_smv.h atlas_zdNKB.h cXover.h
atlas_cacheedge.h atlas_dmv.h atlas_smvN.h atlas_zmv.h dmm.h
atlas_cmv.h atlas_dmvN.h atlas_smvS.h atlas_zmvN.h dXover.h
atlas_cmvN.h atlas_dmvS.h atlas_smvT.h atlas_zmvS.h smm.h
atlas_cmvS.h atlas_dmvT.h atlas_sNCmm.h atlas_zmvT.h sXover.h
atlas_cmvT.h atlas_dNCmm.h atlas_sr1.h atlas_zNCmm.h zmm.h
atlas_cNCmm.h atlas_dr1.h atlas_ssysinfo.h atlas_zr1.h zXover.h
atlas_cr1.h atlas_dsysinfo.h atlas_strsmXover.h atlas_zsysinfo.h
atlas_csNKB.h atlas_dtrsmXover.h atlas_trsmNB.h atlas_ztrsmXover.h
atlas_csysinfo.h atlas_pthreads.h atlas_type.h cmm.h
$ ls lib
libatlas.a libcblas.a libf77blas.a liblapack.a libptcblas.a libptf77blas.a
lib has only, '.a' files = static libraries. Dynamic libs are not automated by ATLAS. Build the shared objects:
$ cd ~/numerics/ATLAS3.8.2/beans64/lib
$ make shared
rm -f libatlas.so liblapack.so
make libatlas.so liblapack.so libf77blas.so libcblas.so liblapack.so
make[1]: Entering directory `/home/me/numerics/ATLAS3.8.2/beans64/lib'
ld -melf_x86_64 -shared -soname libatlas.so -o libatlas.so \
--whole-archive libatlas.a --no-whole-archive -lc -lpthread -lm
ld -melf_x86_64 -shared -soname liblapack.so -o liblapack.so --whole-archive \
liblapack.a --no-whole-archive -L/usr/lib64/gcc/x86_64-suse-linux/4.2.1 -l gfortran
ld: liblapack.a(slaruv.o): relocation R_X86_64_32S against `a local symbol' can not be used when making a shared object; recompile with -fPIC
liblapack.a(slaruv.o): could not read symbols: Bad value
make[1]: *** [liblapack.so] Error 1
make[1]: Leaving directory `/home/me/numerics/ATLAS3.8.2/beans64/lib'
make: *** [shared] Error 2
Same problem with quad.
Second time, set NOOPTS to include -fPIC.
Friday, 16 January 2009
Inserting images in LaTeX
% With 28.6 pixels per cm in my screengrabs, use px / 28.6 to get viewport dimensions.
\begin{figure}[htbp]
\begin{center}
\mbox{
%\subfigure[a] { \fbox{
\includegraphics[natwidth=10cm,scale=0.2,clip=true,
% Use pixels / 28.9:
viewport=0cm 0cm 12cm 12.5cm] % edges cut from default plot size.
{m94acou1a_mesh} } %} }
\caption{The mesh used to test the effect of the damping face, on the impedance of
the drive face.}
\label{fig:m94acou1a_mesh} \end{center} \end{figure}
\begin{figure}[htbp]
\begin{center}
\mbox{
%\subfigure[a] { \fbox{
\includegraphics[natwidth=10cm,scale=0.2,clip=true,
% Use pixels / 28.9:
viewport=0cm 0cm 12cm 12.5cm] % edges cut from default plot size.
{m94acou1a_mesh} } %} }
\caption{The mesh used to test the effect of the damping face, on the impedance of
the drive face.}
\label{fig:m94acou1a_mesh} \end{center} \end{figure}
Gmsh 2.2.3: Ordered mesh, tube of elements.
gmsh version 2.2.3. Version 2.3.0 produces extra elements; I dunno how to tame it, yet.
// Gmsh project created on Tue Aug 5 18:29:53 2008
l = 0.01; // length
h = 0.01; // duct height
my = 1; // y mesh
mz = 1; // mesh density across width.
mx = 10; // mesh along the length.
// Bottom surface; extrude from here, up.=========================NODES
Point(1) = { 0, 0, 0, 1}; // bottom, rear left corner.
Point(2) = { l, 0, 0, 1}; // bottom, front left corner.
Point(3) = { 0, 0, h, 1}; // bottom, rear right corner.
Point(4) = { l, 0, h, 1}; // bottom, front right corner.
// Square creation. Duct. ==============================================Squares
Line(1) = {1, 2}; // left, bottom edge.
Line(2) = {1, 3}; // rear, bottom.
Line(3) = {4, 2}; // front, bottom.
Line(4) = {3, 4}; // right, bottom.
// Define closed loops and planes for the shapes just created. These were
// selected in GMSH, rather than in the script as the negative values control
// the direction of the path. ============================================Planes
// Duct.
Line Loop(5) = {3,-1,2,4}; // bottom line loop.
Plane Surface(6) = {5}; // bottom surface.
// The following code is required to make the mesh ordered =====================
// Set the lines above, as 'transfinite'. Duct (setting most of the mesh):
Transfinite Line{1,4} = mx + 1 Using Progression 1.0; // lines 1 and 4 (sides). Set mesh.
Transfinite Line{2,3} = mz Using Progression 1.0; // lines 2 and 3 (width) set mesh.
// Create new, 'transfinite' surfaces. Matching surface numbers above and
// referencing same node numbers (waste of time, it seems). Had to inspect
// these from gmsh. Note, curly brackets, not parentheses.
// {surface} = {nodes}
Transfinite Surface{6} = {1,2,3,4};
// Without the following recombine, the elements are tri primative.
// Duct.
Recombine Surface{6};
// End of extra code to get ordered mesh ***************************************
//
// Extrude {vector} { Surface{n}; Layers{mz} ; Recombine; }
// with surface number(s) n extruded, mesh density mz1.
// Assign result to var[], to allow further work with the new surface, in var[0]
// Entries 0 to 5 are returned for these extruded faces, so steps of 6 to
// get the faces for different volumes appended to the list.
//
// Extrude to form duct thickness: -------------------------
// upper middle.
num[] = Extrude { 0, h, 0} {
Surface {
6
};
Layers{my}; Recombine;
};
Tuesday, 13 January 2009
Space within arrays
Latex doesn't handle spacing around fractions inside arrays well. Get round this using, for example, \vspace{1 mm}, as follows:
\[ \left( \begin{array}{cccc}
\vspace{1 mm}
\frac{4}{9} & \frac{2}{9} & \frac{1}{9} & \frac{2}{9} \\ \vspace{1 mm}
\frac{2}{9} & \frac{4}{9} & \frac{2}{9} & \frac{1}{9} \\ \vspace{1 mm}
\frac{1}{9} & \frac{2}{9} & \frac{4}{9} & \frac{2}{9} \\ \vspace{1 mm}
\frac{2}{9} & \frac{1}{9} & \frac{2}{9} & \frac{4}{9} \\
\end{array} \right)\]
It's still not quite right, but is better than default.
Replacing text in files.
for file in m93acou10?.pyFE ; do
# test case: Run first, check screen output looks ok.
sed -e 's/splits:16/splits:32/g' "$file"
# Real case: Run only when sure all is well.
# sed -e 's/splits:16/splits:32/g' "$file" > tmp_file
# mv tmp_file "$file"
echo replaced in $file----------------------------------------------
done
from http://www.linuxforums.org/forum/linux-programming-scripting/39664-search-replace-text-within-several-files-2.html
# test case: Run first, check screen output looks ok.
sed -e 's/splits:16/splits:32/g' "$file"
# Real case: Run only when sure all is well.
# sed -e 's/splits:16/splits:32/g' "$file" > tmp_file
# mv tmp_file "$file"
echo replaced in $file----------------------------------------------
done
from http://www.linuxforums.org/forum/linux-programming-scripting/39664-search-replace-text-within-several-files-2.html
Latex packages
Easiest way to install. Unzip in ~/downloads/packagename
Append ~/downloads/packagename// to TEXINPUTS. Keep the double '/'
Job done!
This was from the pgfplots manual.
Append ~/downloads/packagename// to TEXINPUTS. Keep the double '/'
Job done!
This was from the pgfplots manual.
Monday, 12 January 2009
Installing Scipy etc
Suse 10.3, beans. The rpms have been downloaded already.
Install gcc42-fortran from repository. (no help, exlude next time)
Install gcc-fortran from repository. (essential)
$ rpm -i bzr-1.6.1-0.pm.1.x86_64.rpm
libgfortran41 was essential. Oddly, libgfortran42 did not satisfy.
blas and libblas3 were installed (reported by yast). Removed them,
along with matplotlib, numpy, scipy (none of which worked).
# rpm -i refblas3-3.0-11.1.x86_64.rpm
now worked.
# rpm -i lapack3-3.0-19.1.x86_64.rpm
worked.
# rpm -i python-configobj-4.5.2-0.pm.1.x86_64.rpm
didn't work. there was a newer version already. left as was.
rpm -i python-numpy-1.0.4-4.1.x86_64.rpm
worked
rpm -i python-scipy-0.6.0-2.1.x86_64.rpm
worked
python-numeric was installed with yast
python-gtk and python-gtk-devel were installed with yast.
rpm -i python-matplotlib-0.91.2-0.pm.1.x86_64.rpm
worked.
import numpy, scipy still fails. A working numpy shows
>>> print numpy.__file__
'/usr/local/lib/python2.5/site-packages/numpy/__init__.pyc'
and that's not part of $PATH, nor $PYTHONPATH. How is Python
getting to it?
Suse is installing python packages to directories with 64 appended. Python's search path uses the non-64 version, so no installed packages
are found. The readline error maybe indicative of this.
The 64 bit paths can be added to PYTHONPATH, or directly added by
editing the site.py file. (http://docs.python.org/install/index.html).
I'll update PYTHONPATH, to include in particular, /usr/lib64/python
sys.path shows the search path. I copied it's contents into PYTHONPATH as defined in .bashrc, and changed the entries to 64 versions.
/usr/local/lib64 is empty and should be /usr/lib64/
In fact, removing /local/ from all the python paths including 64, fixed readline, numpy and scipy. Job done.
Summary:
Suse10.3 was installing to /usr/lib64/python/site-packages
but the default pythonpath (in sys.path) did not use this path. The
paths in sys.path were added to PYTHONPATH in .bashrc, and modified to use /usr/lib64/.
Install gcc42-fortran from repository. (no help, exlude next time)
Install gcc-fortran from repository. (essential)
$ rpm -i bzr-1.6.1-0.pm.1.x86_64.rpm
libgfortran41 was essential. Oddly, libgfortran42 did not satisfy.
blas and libblas3 were installed (reported by yast). Removed them,
along with matplotlib, numpy, scipy (none of which worked).
# rpm -i refblas3-3.0-11.1.x86_64.rpm
now worked.
# rpm -i lapack3-3.0-19.1.x86_64.rpm
worked.
# rpm -i python-configobj-4.5.2-0.pm.1.x86_64.rpm
didn't work. there was a newer version already. left as was.
rpm -i python-numpy-1.0.4-4.1.x86_64.rpm
worked
rpm -i python-scipy-0.6.0-2.1.x86_64.rpm
worked
python-numeric was installed with yast
python-gtk and python-gtk-devel were installed with yast.
rpm -i python-matplotlib-0.91.2-0.pm.1.x86_64.rpm
worked.
import numpy, scipy still fails. A working numpy shows
>>> print numpy.__file__
'/usr/local/lib/python2.5/site-packages/numpy/__init__.pyc'
and that's not part of $PATH, nor $PYTHONPATH. How is Python
getting to it?
Suse is installing python packages to directories with 64 appended. Python's search path uses the non-64 version, so no installed packages
are found. The readline error maybe indicative of this.
The 64 bit paths can be added to PYTHONPATH, or directly added by
editing the site.py file. (http://docs.python.org/install/index.html).
I'll update PYTHONPATH, to include in particular, /usr/lib64/python
sys.path shows the search path. I copied it's contents into PYTHONPATH as defined in .bashrc, and changed the entries to 64 versions.
/usr/local/lib64 is empty and should be /usr/lib64/
In fact, removing /local/ from all the python paths including 64, fixed readline, numpy and scipy. Job done.
Summary:
Suse10.3 was installing to /usr/lib64/python/site-packages
but the default pythonpath (in sys.path) did not use this path. The
paths in sys.path were added to PYTHONPATH in .bashrc, and modified to use /usr/lib64/.
Saturday, 10 January 2009
Linear regression example
Problem setup
-------------
y = mx + c
If m = 4, and c = 34, the values of y for x = [1,2,3,4,5] are:
In [68@19:52:03]:x = array([1, 2, 3, 4])
In [71@19:52:03]:y = 4 * x + 34
In [72@19:52:03]:y
Out[72]: array([38, 42, 46, 50])
Solve for m, c
--------------
In [73@19:52:03]:a = array( [[1, x[0]], [1, x[1]], [1, x[2]], [1, x[3]]])
In [80@19:52:03]:numpy.linalg.lstsq(a, y)
Out[80]: (array([ 34., 4.]), array([ 0.]), 2, array([ 5.77937881, 0.77380911]))
There in the first item returned, are c and m for a least squares solution. The second item is the 'Sums of residues'. Third is the rank. Fourth are the singular values of 'a'.
If the relation wasn't y = mx + c, but, say mx + nx^{2} + c, then a third column in a could be used, to hold the calculated square terms of x. n would then be returned also, in the first item.
Improvement
-----------
If x and y are are lists of coordinates:
a = numpy.ones((len(x), 2))
a[:, 1] = x
y = numpy.array(y)
out = numpy.linalg.lstsq(a, y)
c, m = out[0]
sum_residues = out[1]
out_rank = out[2]
singulars = out[3]
-------------
y = mx + c
If m = 4, and c = 34, the values of y for x = [1,2,3,4,5] are:
In [68@19:52:03]:x = array([1, 2, 3, 4])
In [71@19:52:03]:y = 4 * x + 34
In [72@19:52:03]:y
Out[72]: array([38, 42, 46, 50])
Solve for m, c
--------------
In [73@19:52:03]:a = array( [[1, x[0]], [1, x[1]], [1, x[2]], [1, x[3]]])
In [80@19:52:03]:numpy.linalg.lstsq(a, y)
Out[80]: (array([ 34., 4.]), array([ 0.]), 2, array([ 5.77937881, 0.77380911]))
There in the first item returned, are c and m for a least squares solution. The second item is the 'Sums of residues'. Third is the rank. Fourth are the singular values of 'a'.
If the relation wasn't y = mx + c, but, say mx + nx^{2} + c, then a third column in a could be used, to hold the calculated square terms of x. n would then be returned also, in the first item.
Improvement
-----------
If x and y are are lists of coordinates:
a = numpy.ones((len(x), 2))
a[:, 1] = x
y = numpy.array(y)
out = numpy.linalg.lstsq(a, y)
c, m = out[0]
sum_residues = out[1]
out_rank = out[2]
singulars = out[3]
Anemometer plan view.
\begin{figure}[htbp] \begin{center}
\mbox{
\subfigure[Case 1] {
\def\subfigcapskip{300pt}
\def\farskip{200pt}
\begin{tikzpicture} [scale=0.6]
\draw (0,0) circle (2.5cm); % anemometer body, top down.
\draw (120:1.1) circle (0.9cm) node {$a$};
\draw (0:1.1) circle (0.9cm) node {$b$}; % polar coordinates!
\draw (240:1.1) circle (0.9cm) node {$c$};
\draw [white] (5,-3); % forcing space below subfigure
\draw[->] (-5,0) -- node [above] {flow} (-3,0) ;
% Support pillars
\foreach \x in {0 + 30, 60 + 30, 120 + 30, 180 + 30, 240 + 30, 300 + 30}
\draw (\x-3:2.5) -- (\x-3:2.5-0.25) -- (\x + 3:2.5-0.25) -- (\x+3:2.5);
\end{tikzpicture} }
\subfigure[Case 2] {
\def\subfigcapskip{300pt}
\def\farskip{200pt}
\begin{tikzpicture} [scale=0.6]
\draw (0,0) circle (2.5cm); % anemometer body, top down.
\draw (120+30:1.1) circle (0.9cm) node {$a$};
\draw (0+30:1.1) circle (0.9cm) node {$b$}; % polar coordinates!
\draw (240 + 30:1.1) circle (0.9cm) node {$c$};
\draw [white] (0,-3); % forcing space below subfiure
% Support pillars
\foreach \x in {0 + 60, 60 + 60, 120 + 60, 180 + 60, 240 + 60, 300 + 60}
\draw (\x-3:2.5) -- (\x-3:2.5-0.25) -- (\x + 3:2.5-0.25) -- (\x+3:2.5);
\end{tikzpicture} } }
\caption{Orientation of the anemometer with respect to flow direction, for the
two experiments.}
\label{fig:TwoMeasurements}
\end{center}
\end{figure}
Thursday, 8 January 2009
LaTeX tables
\begin{table}[htbp]
\centering % this centres the whole of the table.
\begin{tabular}{ccc}
\toprule % new command, using booktabs
\multicolumn{2}{c}{two cols} & {one col} \\
$u_{x}\,(ms^{-1})$ & $u_{mean_{x, y, z}}\, (ms^{-1})$ & $\frac{u_{mean_{x}}}{u_{x}}$ \\
\midrule
10 & 7.4, 6E-5, 1E-4 & 0.74\\
20 & 14.9, 1E-4, 5E-4 & 0.75\\
30 & 23.5, 9E-5, 8E-4 & 0.78\\
40 & 32.3, 2E-4, 1E-3 & 0.81\\
50 & 40.0, 3E-4, 2E-2 & 0.80\\
\bottomrule
\end{tabular}
\caption{Effect of changing from uniform to steady, non-uniform flow, on the
mean velocity in the duct.}
\label{tab:uniform-to-steady-non-uniform-vels}
\end{table}
Tuesday, 6 January 2009
Replacing a hard-drive
*post note: do not use fdisk. It is limited to 2TB MBR systems. Use parted, instead*
Out of space again. The windows drive is 100 Gig or so, so replace it with a 1Tb drive and use the extra space with linux.
http://tlug.dnho.net/?q=node/91
looks promising.
The new drive is sdc, the old, sda. Copy the partition table:
$ sfdisk -d /dev/sda1 > sda1.out
where
$ cat sda.out
# partition table of /dev/sda
unit: sectors
/dev/sda1 : start= 63, size=156344517, Id= 7, bootable
/dev/sda2 : start= 0, size= 0, Id= 0
/dev/sda3 : start= 0, size= 0, Id= 0
/dev/sda4 : start= 0, size= 0, Id= 0
Copy the partition table to sdc:
$ sfdisk /dev/sdc < sda.out
Checking that no-one is using this disk right now ...
OK
Disk /dev/sdc: 121601 cylinders, 255 heads, 63 sectors/track
sfdisk: ERROR: sector 0 does not have an msdos signature
/dev/sdc: unrecognized partition table type
Old situation:
No partitions found
New situation:
Units = sectors of 512 bytes, counting from 0
Device Boot Start End #sectors Id System
/dev/sdc1 * 63 156344579 156344517 7 HPFS/NTFS
/dev/sdc2 0 - 0 0 Empty
/dev/sdc3 0 - 0 0 Empty
/dev/sdc4 0 - 0 0 Empty
Successfully wrote the new partition table
Re-reading the partition table ...
If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
to zero the first 512 bytes: dd if=/dev/zero of=/dev/foo7 bs=512 count=1
(See fdisk(8).)
Shit. Complexities. Is it DOS? No, it's NTFS, isn't it? Leave as is, for now, and continue with the low level copy. If on drive swap there's a problem, try the zeroing mentioned above.
Low level copy: Note, dd_rescue is just as good and wont fail on bad sectors.
$ dd if=/dev/sda1 of=/dev/sdc1
After approx. an hour:
156344517+0 records in
156344517+0 records out
80048392704 bytes (80 GB) copied, 3404.66 s, 23.5 MB/s
Now, swap the drive with the original and reboot. Did it work?
Of course not. The system freezes at the DMI Table update (or whatever).
OK, zero the first 512 bytes and retry.
$ dd if=/dev/zero of=/dev/sdc1 bs=512 count=1
1+0 records in
1+0 records out
512 bytes (512 B) copied, 0.000456196 s, 1.1 MB/s
And reboot. Joy? No. Maybe I should have copied the device, instead of the partition. Boot time shows this drive is the master, so the boot stuff is coming from it. OK, try:
------------------------------------ Start here ------------------------------
$ dd if=/dev/sda of=/dev/sdc
or
$dd_rescue /dev/sda /dev/sdc
It worked! The new drive can be plugged in, to replace the original. Now, to get to the extra space.
$ parted /dev/sda
GNU Parted 1.7.1
Using /dev/sda
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print
Disk /dev/sda: 1000GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
1 32.3kB 80.0GB 80.0GB primary ntfs boot, , , , , , , , , type=07, ,
(parted)
So, the windows partition is there. I want EXT2 / EXT3 on the remaining space. fdisk will partition the disk for me. From,
http://tldp.org/HOWTO/Partition/fdisk_partitioning.html
fdisk's units are the 'cylinder'. $ fdisk /dev/sda shows:
$ fdisk /dev/sda
The number of cylinders for this disk is set to 121601.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help): p
Disk /dev/sda: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 9732 78172258+ 7 HPFS/NTFS
Command (m for help):
So, I have a total of
121601 cylinders, of which 9732 are used for xp. Use the rest for ext2.
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (9733-121601, default 9733):
Using default value 9733
Last cylinder or +size or +sizeM or +sizeK (9733-121601, default 121601):
Using default value 121601
Command (m for help): p
Disk /dev/sda: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 9732 78172258+ 7 HPFS/NTFS
/dev/sda2 9733 121601 898587742+ 83 Linux
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
------------------------------------------------------------------
Done. Now to format the new partition, and mount it at startup.
First, is the partition present?
$ ls /dev | grep sda
sda
sda1
sda2
Yes! sda2 is new. To format as ext3:
$ mke2fs -j /dev/sda2
mke2fs 1.39 (29-May-2006)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
112328704 inodes, 224646935 blocks
11232346 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=226492416
6856 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
102400000, 214990848
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 25 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
And finally, edit /etc/fstab to mount at startup.
I added the line
/dev/sda2 /home/me/projects/d2 ext3 acl,user_xattr 1 3
Now, create the directory /home/me/projects/d2 and reset to test.
It worked.
Footnote: testdisk was the program I used earlier, to recover lost partitions. It's an executable that needs to be downloaded (it's not a part of linux, afaik).
If /etc/fstab references drives by ID, it /dev/disk/by-id/scsi_SATA_etc_etc,
this process will fail. I found this ID approach was terminated by a part number, for each line. Changing this to /dev/sda6 (for example) fixed this.
-----------------------------------------------------------------
Adding a harddrive. First, partition it with fdisk, then format the partition with mke2fs, then
set mounting behaviour.
Work out which device the new drive is. In this case, it's /dev/sdb
$ sudo fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x6c929856.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.
The number of cylinders for this disk is set to 243201.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
Command (m for help): p
Disk /dev/sdb: 2000.3 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x6c929856
Device Boot Start End Blocks Id System
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-243201, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-243201, default 243201):
Using default value 243201
Command (m for help): p
Disk /dev/sdb: 2000.3 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x6c929856
Device Boot Start End Blocks Id System
/dev/sdb1 1 243201 1953512001 83 Linux
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
--------------- Now format the new partition.
Is the new partition present?
$ ls /dev/sd*
/dev/sda /dev/sda2 /dev/sda4 /dev/sda6 /dev/sdb1
/dev/sda1 /dev/sda3 /dev/sda5 /dev/sdb
yes: sdb1 is new, and a result of the fdisk operation.
$ mke2fs -j /dev/sdb1
mke2fs 1.40.8 (13-Mar-2008)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
122101760 inodes, 488378000 blocks
24418900 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
14905 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
102400000, 214990848
Writing inode tables: count/14905
And finally, edit /etc/fstab to mount at startup.
I added the line
/dev/sdb1 /home/me/projects/d2 ext3 acl,user_xattr 1 3
Out of space again. The windows drive is 100 Gig or so, so replace it with a 1Tb drive and use the extra space with linux.
http://tlug.dnho.net/?q=node/91
looks promising.
The new drive is sdc, the old, sda. Copy the partition table:
$ sfdisk -d /dev/sda1 > sda1.out
where
$ cat sda.out
# partition table of /dev/sda
unit: sectors
/dev/sda1 : start= 63, size=156344517, Id= 7, bootable
/dev/sda2 : start= 0, size= 0, Id= 0
/dev/sda3 : start= 0, size= 0, Id= 0
/dev/sda4 : start= 0, size= 0, Id= 0
Copy the partition table to sdc:
$ sfdisk /dev/sdc < sda.out
Checking that no-one is using this disk right now ...
OK
Disk /dev/sdc: 121601 cylinders, 255 heads, 63 sectors/track
sfdisk: ERROR: sector 0 does not have an msdos signature
/dev/sdc: unrecognized partition table type
Old situation:
No partitions found
New situation:
Units = sectors of 512 bytes, counting from 0
Device Boot Start End #sectors Id System
/dev/sdc1 * 63 156344579 156344517 7 HPFS/NTFS
/dev/sdc2 0 - 0 0 Empty
/dev/sdc3 0 - 0 0 Empty
/dev/sdc4 0 - 0 0 Empty
Successfully wrote the new partition table
Re-reading the partition table ...
If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
to zero the first 512 bytes: dd if=/dev/zero of=/dev/foo7 bs=512 count=1
(See fdisk(8).)
Shit. Complexities. Is it DOS? No, it's NTFS, isn't it? Leave as is, for now, and continue with the low level copy. If on drive swap there's a problem, try the zeroing mentioned above.
Low level copy: Note, dd_rescue is just as good and wont fail on bad sectors.
$ dd if=/dev/sda1 of=/dev/sdc1
After approx. an hour:
156344517+0 records in
156344517+0 records out
80048392704 bytes (80 GB) copied, 3404.66 s, 23.5 MB/s
Now, swap the drive with the original and reboot. Did it work?
Of course not. The system freezes at the DMI Table update (or whatever).
OK, zero the first 512 bytes and retry.
$ dd if=/dev/zero of=/dev/sdc1 bs=512 count=1
1+0 records in
1+0 records out
512 bytes (512 B) copied, 0.000456196 s, 1.1 MB/s
And reboot. Joy? No. Maybe I should have copied the device, instead of the partition. Boot time shows this drive is the master, so the boot stuff is coming from it. OK, try:
------------------------------------ Start here ------------------------------
$ dd if=/dev/sda of=/dev/sdc
or
$dd_rescue /dev/sda /dev/sdc
It worked! The new drive can be plugged in, to replace the original. Now, to get to the extra space.
$ parted /dev/sda
GNU Parted 1.7.1
Using /dev/sda
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print
Disk /dev/sda: 1000GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
1 32.3kB 80.0GB 80.0GB primary ntfs boot, , , , , , , , , type=07, ,
(parted)
So, the windows partition is there. I want EXT2 / EXT3 on the remaining space. fdisk will partition the disk for me. From,
http://tldp.org/HOWTO/Partition/fdisk_partitioning.html
fdisk's units are the 'cylinder'. $ fdisk /dev/sda shows:
$ fdisk /dev/sda
The number of cylinders for this disk is set to 121601.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help): p
Disk /dev/sda: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 9732 78172258+ 7 HPFS/NTFS
Command (m for help):
So, I have a total of
121601 cylinders, of which 9732 are used for xp. Use the rest for ext2.
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (9733-121601, default 9733):
Using default value 9733
Last cylinder or +size or +sizeM or +sizeK (9733-121601, default 121601):
Using default value 121601
Command (m for help): p
Disk /dev/sda: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 9732 78172258+ 7 HPFS/NTFS
/dev/sda2 9733 121601 898587742+ 83 Linux
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
------------------------------------------------------------------
Done. Now to format the new partition, and mount it at startup.
First, is the partition present?
$ ls /dev | grep sda
sda
sda1
sda2
Yes! sda2 is new. To format as ext3:
$ mke2fs -j /dev/sda2
mke2fs 1.39 (29-May-2006)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
112328704 inodes, 224646935 blocks
11232346 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=226492416
6856 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
102400000, 214990848
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 25 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
And finally, edit /etc/fstab to mount at startup.
I added the line
/dev/sda2 /home/me/projects/d2 ext3 acl,user_xattr 1 3
Now, create the directory /home/me/projects/d2 and reset to test.
It worked.
Footnote: testdisk was the program I used earlier, to recover lost partitions. It's an executable that needs to be downloaded (it's not a part of linux, afaik).
If /etc/fstab references drives by ID, it /dev/disk/by-id/scsi_SATA_etc_etc,
this process will fail. I found this ID approach was terminated by a part number, for each line. Changing this to /dev/sda6 (for example) fixed this.
-----------------------------------------------------------------
Adding a harddrive. First, partition it with fdisk, then format the partition with mke2fs, then
set mounting behaviour.
Work out which device the new drive is. In this case, it's /dev/sdb
$ sudo fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x6c929856.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.
The number of cylinders for this disk is set to 243201.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
Command (m for help): p
Disk /dev/sdb: 2000.3 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x6c929856
Device Boot Start End Blocks Id System
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-243201, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-243201, default 243201):
Using default value 243201
Command (m for help): p
Disk /dev/sdb: 2000.3 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x6c929856
Device Boot Start End Blocks Id System
/dev/sdb1 1 243201 1953512001 83 Linux
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
--------------- Now format the new partition.
Is the new partition present?
$ ls /dev/sd*
/dev/sda /dev/sda2 /dev/sda4 /dev/sda6 /dev/sdb1
/dev/sda1 /dev/sda3 /dev/sda5 /dev/sdb
yes: sdb1 is new, and a result of the fdisk operation.
$ mke2fs -j /dev/sdb1
mke2fs 1.40.8 (13-Mar-2008)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
122101760 inodes, 488378000 blocks
24418900 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
14905 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
102400000, 214990848
Writing inode tables: count/14905
And finally, edit /etc/fstab to mount at startup.
I added the line
/dev/sdb1 /home/me/projects/d2 ext3 acl,user_xattr 1 3
Subscribe to:
Posts (Atom)