Stochastic Analysis - Reading 1

Today I made the plot of
x' = -y, x0 = x
y' = -(y-x)/epsilon, y0 = y

which, by what the paper states, leads to the following equation for x in the limit as epsilon -> 0:
x' = -x, x0 = x

in Matlab, using analycal tool, such as solve the equation for x, y analytically, then let epsilon approaching zero, in the hope that x_epsilon -> x, and the solution curve (x_epsilon, y) converges to (x,y) but no hope... dunno why.

Next step will be, using numerical tool.

PDE Qual

Old Qual from Purdue

Hermite Polynomials and Gaussian distribution Links

Gaussian functions from Wiki

Gaussian integrals from Wiki

Normal distribution from Wiki

Construct Hermite Polynomials from Gaussian distribution functions (see copy below)

Hermite polynomials from Wiki

Addition on Hermite polynomials


Construct Hermite Polynomials from Gaussian distribution functions, copied from link above:

Let us look at the Hermite polynomials in somewhat more detail. Consider a Gaussian function, .

.

We write

.

This defines the Hermite polynomials.

.

The parity of the Hermite polynomials is (-1)ⁿ.

H_n(z) is a nth degree polynomial in z.

Proof:

This statement holds for n = 1 and n = 2.

Let . Then

.

If H_n-1(z) is a polynomial of degree n-1, the H_n(z) is a polynomial of degree n. The statement therefore holds for all n.

The generating function

Consider F(z+l) = exp(-(z+l)²).

A Taylor series expansion, , yields

.

Let l’ = -l. Then

.

. (Taylor series expansion)

is called the generating function for the Hermite polynomials H_n(z).

Recurrence relations

We have already found

.

Other recurrence relations exist.

and combining the first two relations

.

[ is obtained by differentiating with respect to z,

, and equating terms of equal power of l.

is obtained by differentiating with respect to l.]

Summary

The Hermite polynomials H_n(z) are defined by

or .

The generating function of the Hermite polynomials is

.

The recurrence relations are

, ,

and

.

The differential equation satisfied by the Hermite polynomials is

.

The normalization and orthogonality condition are

Basic probability in Matlab

Mean value

A =
1 2 0
2 3 0

mean(A) = 1.5000 2.5000 0 % Mean of each column

mean(A, 2) = % Mean of each row
1.0000
1.6667


Variance

• V = var(X) returns

- for vectors X: its variance.
- for matrices: a row vector containing the variance of each column of X.

• V = var(X,w,dim)
  

takes the variance along the dimension dim of X.
Pass in 0 for w to use the default normalization by N-1, or 1 to use N.
The variance is the square of the standard deviation (STD)

Example:
A =
1 2 0
2 3 0

var(A) =
0.5000 0.5000 0 % variance of columns

var(A, 0, 2) = % variance of rows
1.0000
2.3333

Links for "fun"

Topics - Online mag. for English learners

List of traditional children games

Deep fun

Test strong convergence of Milstein Mthd for SDE

Below is the file

******************************************

%MILSTRONG  Test strong convergence of Milstein: vectorized
%
% Solves   dX = r*X*(K-X) dt + beta*X dW,  X(0) = Xzero,
%        where r = 2, K= 1, beta = 1 and Xzero = 0.5.
%
% Discretized Brownian path over [0,1] has dt = 2^(-11).
% Milstein uses timesteps 128*dt, 64*dt, 32*dt, 16*dt (also dt for reference).
%
% Examines strong convergence at T=1:  E | X_L - X(T) |.
% Code is vectorized: all paths computed simultaneously.

randn('state',100)
r = 2; K = 1; beta = 0.25; Xzero = 0.5;   % problem parameters
T = 1; N = 2^(11); dt = T/N;                       % from 0 to T=1. N = 2^11 steps, each of length dt            
M = 500;                                                            % number of paths sampled
R = [1; 16; 32; 64; 128];                               % Milstein stepsizes are R*dt, 5 R's for 5 paths

dW = sqrt(dt)*randn(M,N);                 % Brownian increments for each path each timestep
Xmil = zeros(M,5);                                  % preallocate array: M paths w. corresp. 5 R
for p = 1:5                                                   % Consider each time step separately
     Dt = R(p)*dt; L = N/R(p);                % Dt = new time step
                                                                       % L timesteps of size Dt = R dt
     Xtemp = Xzero*ones(M,1);            % All M paths begin at Xzero
     for j = 1:L                          % For each step
          Winc = sum(dW(:,R(p)*(j-1)+1:R(p)*j),2); 
          Xtemp = Xtemp + Dt*r*Xtemp.*(K-Xtemp) + beta*Xtemp.*Winc ...
              + 0.5*beta^2*Xtemp.*(Winc.^2 - Dt); % See below
end
     Xmil(:,p) = Xtemp;       % store Milstein solution at t = 1
end

Xref = Xmil(:,1);                                       % Reference solution: "exact" soln when timestep = dt 
Xerr = abs(Xmil(:,2:5) - repmat(Xref,1,4));   % Error in each path
mean(Xerr);                                              % Mean pathwise errors
Dtvals = dt*R(2:5);                                % Milstein timesteps used

subplot(224)                                            % lower RH picture
loglog(Dtvals,mean(Xerr),'b*-'), hold on
loglog(Dtvals,Dtvals,'r--'), hold off  % reference slope of 1
axis([1e-3 1e-1 1e-4 1])
xlabel('\Delta t')
ylabel('Sample average of | X(T) - X_L |')
title('milstrong.m','FontSize',10)

%%%% Least squares fit of error = C * Dt^q %%%%
A = [ones(4,1), log(Dtvals)]; rhs = log(mean(Xerr)');
sol = A\rhs; q = sol(2)
resid = norm(A*sol - rhs)

******************************************

Notes:

-

-

Colon (:) in Matlab

colon (:)

Create vectors, array subscripting, and for-loop iterators.

Description

The colon is one of the most useful operators in MATLAB. It can create vectors, subscript arrays, and specify for iterations.The colon operator uses the following rules to create regularly spaced vectors:

j:k is the same as [j,j+1,...,k] (empty if j > k)
j:i:k is the same as [j,j+i,j+2i, ...,k]
is empty if i == 0, if i > 0 and j> k, or if i <> where i, j, and k are all scalars.

Below are the definitions that govern the use of the colon to pick out selected rows, columns, and elements of vectors, matrices, and higher-dimensional arrays:

A(:,j) is the jth column of A
A(i,:) is the ith row of A
A(:,:) is the equivalent two-dimensional array. For matrices this is the same as A.
A(j:k) is A(j), A(j+1),...,A(k)
A(:,j:k) is A(:,j), A(:,j+1),...,A(:,k)
A(:,:,k) is the kth page of three-dimensional array A.
A(i,j,k,:) is a vector in four-dimensional array A. The vector includes A(i,j,k,1), A(i,j,k,2), A(i,j,k,3), and so on.
A(:) is all the elements of A, regarded as a single column. On the left side of an assignment statement, A(:) fills A, preserving its shape from before. In this case, the right side must contain
the same number of elements as A.

Examples

Using the colon with integers,
D = 1:4 results in D = 1 2 3 4

Using two colons to create a vector with arbitrary real increments between the elements,
E = 0:.1:.5 results in E = 0 0.1000 0.2000 0.3000 0.4000 0.5000

The command
A(:,:,2) = pascal(3)
generates a three-dimensional array whose first page is all zeros.

A(:,:,1) =
0 0 0
0 0 0
0 0 0

A(:,:,2) =
1 1 1
1 2 3
1 3 6

Brownian Path Simulation using Matlab

%BPATH1  Brownian path simulation

randn('state',100)           % set the state of randn to be 100
T = 1; N = 500; dt = T/N;    % Length = 1, do 500 steps, each step measures dt=1/500
dW = zeros(1,N);             % preallocate arrays ...
W = zeros(1,N);              % for efficiency

dW(1) = sqrt(dt)*randn;      % first approximation outside the loop ...
W(1) = dW(1);                % since W(0) = 0 is not allowed
for j = 2:N
 dW(j) = sqrt(dt)*randn;   % general increment, dW ~ sqrt(dt)
 W(j) = W(j-1) + dW(j);    % then go a step of dW(j) from W(j-1) to W(j)
end

plot([0:dt:T],[0,W],'r-')    % plot W against t: plot from time 0 to time T, increment dt
xlabel('t','FontSize',16)
ylabel('W(t)','FontSize',16,'Rotation',0)

Note: Matlab starts array from 1, not 0. So in order to include the starting point W(0) = 0 into the graph, we need to include it in the plotting command by doing plot([0:dt:T],[0,W])

Following is a more elegant and more efficient version of bpath1.m :
The "for" loop is replaced by a "vectorized" command
(KEY to writing efficient MATLAB)code (See DJHigham & NJHigha, MATLAB guide, SIAM, Philly 2000)


%BPATH2  Brownian path simulation: vectorized

randn('state',100)          % set the state of randn
T = 1; N = 500; dt = T/N;

dW = sqrt(dt)*randn(1,N);   % generate the array of increments
W = cumsum(dW);             % cumulative sum, W(j)= dW(1) + ... + dW(j)

plot([0:dt:T],[0,W],'r-')   % plot W against t, red line
xlabel('t','FontSize',16)
ylabel('W(t)','FontSize',16,'Rotation',0)

A higher simulation of Brownian path:
Average from about 1000 different paths. Sketch the average path and 5 of the brownian paths.
Here we evaluate the function u(W(t)) = exp(t + W(t)/2) along 1000 discretized Brownian paths.
The average of u(W(t)) over these paths is plotted with a solid blue line.
Five individual paths are also plotted using a dashed red line.

%BPATH3  Function along a Brownian path

randn('state',100)                                % set the state of randn
T = 1; N = 500; dt = T/N; t = [dt:dt:1];          % t = row vector, N components, from dt to 1, increments dt.

M = 1000;                                         % M paths simultaneously
dW = sqrt(dt)*randn(M,N);                         % increments for M paths. 1 path 1 row. Each row N increments ...
                                                % i.e., dW is an M-by-N array st dW(i,j) gives the increment dWj

W = cumsum(dW,2);                                 % cumulative sum accross the second dimension (column)
                                                % i.e., each row of W is the cumsum of the corresp. row of dW

U = exp(repmat(t,[M 1]) + 0.5*W);                 % repmat(t,[M 1]) form a "matrix" M row 1 column, each "entry" is t.
                                                % With t a N-dim row vector, this put M t's stack together,
                                                % forming a M by N matrix.
                                                % then add it to the matrix .5*W
                                                  % then take exp of this matrix
Umean = mean(U);                                  % compute columwise averages
plot([0,t],[1,Umean],'b-'), hold on               % plot mean over M paths
                                                 % x-axis is 0 and the vector t
                                                 % y-axis is 1. At time 0 the path begin at y=1. WHY???
plot([0,t],[ones(5,1),U(1:5,:)],'r--'), hold off  % plot 5 individual paths
                                                 % discrete "time" from 0 to 1
                                                 % over 5 paths, all begin at 1
                                                 % paths are first 5 rows of U
xlabel('t','FontSize',16)
ylabel('U(t)','FontSize',16,'Rotation',0,'HorizontalAlignment','right')
legend('mean of 1000 paths','5 individual paths',2)

averr = norm((Umean - exp(9*t/8)),'inf')          % sample error. Infinity norm of matrix
                                                 % largest row sum of A.
                                                 % w.o. 'inf' - 1-norm, largest column sum

To be explored....

Matlab - Basic Plotting

To plot one function in one graph (line plot)

t = 0 : pi/100 : 2*pi; %create a vector of values in the range [0, 2pi] in increments of pi/100
y = sin(t); %use this vector to evaluate the sine function over that range
plot(t,y)
grid on % Turn on grid lines for this plot

To plot several functions in one graph (several lines plot)

y = sin(t);
y2 = sin(t-0.25);
y3 = sin(t-0.5);
plot(t,y,t,y2,t,y3)

with lines type specified

t = 0:pi/100:2*pi;
y = sin(t);
y2 = sin(t-0.25);
y3 = sin(t-0.5);
plot(t,y,'-',t,y2,'--',t,y3,':')

Colors, Line Styles, and Markers

The basic plotting functions accepts character-string arguments that specify various line styles, marker symbols, and colors for each vector plotted. In the general form,

plot(x,y,'linestyle_marker_color')

linestyle_marker_color is a character string (delineated by single quotation marks) constructed from

1. A line style (e.g., dashed, dotted, etc.)
2. A marker type (e.g., x, *, o, etc.)
3. A predefined color specifier (c, m, y, k, r, g, b, w)

For example,

plot(x,y,':squarey')%plots a yellow dotted line and places square markers at each data point.

If you specify a marker type, but not a line style, MATLAB draws only the marker.
The specification can consist of one or none of each specifier in any order.

For example, the string'go--' defines a dashed line with circular markers, both colored green.

You can also specify the size of the marker and, for markers that are closed shapes, you can specify separately the colors of the edges and the face.

x = -pi:pi/10:pi; % x is from -pi to pi, with increments pi/10
y = tan(sin(x)) - sin(tan(x)); % definition of y
plot(x,y,'--rs',... % plot the graph y = f(x), a red dashed line with squared markers
'LineWidth',2,... % a line with width of two points
'MarkerEdgeColor','k',... % the edge of the marker colored BLACK
'MarkerFaceColor','g',... % the facr of the marker colored green
'MarkerSize',10) % the size of the marker set to 10 points

Note: the three dots '...' tells Matlab that something more will be added in the next line.
W.o. them, Matlab will interpret the commend to be ended right when the line ends.

Adding plot to an existing graph

semilogx(1:100,'+') % screate a semilogarithmic plot
hold all % hold plot and cycle line colors
plot(1:3:300,1:100,'--') % then add a linear plot
hold off % stop the holding process
grid on % Turn on grid lines for this plot

Plotting only the data points

For example, given two vectors

x = 0:pi/15:4*pi; % x from 1 to 4pi, with increments pi/15.
y = exp(2*cos(x)); % y is a function in x
plot(x,y, 'r+') % call plot with only a color and a marker specifier, here a red plus sigh at each data point.

Plotting Markers and Lines

x = 0:pi/15:4*pi;
y = exp(2*cos(x)); % x and y as above
plot(x,y,'-r',... % data as a red, solid line
x,y,'ok') % adds circular markers with black edges at each data point



Some options:
'a' - none
'b' = blue
'c' = cyan
'd' = looks like {}
'e' - none
'f' - none
'g'
'h' = star
'k' = black
'l' - none
'm' = pink/purple
'n' - none
'o' = circle
'p' = star
'q' - none
'r' = red
's' = square
't' - none
'u' - none
'v' - looks like gradient notation
'w' = white
'y' = yellow
'z' - none

'^' = upward triangle
'*' = snowflake
':' = tiny dashed
'<' = rightward triangle '>' = leftward triangle
'.' = tinny colored dot

Matlab - Arrays

zero

B = zeros(n) returns an n-by-n matrix of zeros. An error message appears if n is not a scalar.
B = zeros(m,n) or B= zeros([m n]) returns an m-by-n matrix of zeros.
B = zeros(m,n,p,...) or B= zeros([m n p ...]) returns an m-by-n-by-p-by-... arrayof zeros.
B = zeros(size(A)) returns an array the same size as A consisting of all zeros.

Remarks

The MATLAB language does not have a dimension statement;

MATLAB automatically allocates storage for matrices.

Nevertheless, for large matrices, MATLAB programs may execute faster if the zeros function is used to set aside storage for a matrix whose elements are to be generated one at a time,
or a row or column at a time.

For example

x = zeros(1,n);
for i = 1:n, x(i) = i; end

Matlab - randn

randn(n) : randomly produce an nxn matrix
using ... method of generator

randn(m,n) : randomly produce an mxn matrix
using ... method of generator

s = randn(method) returns in s the current internal state of the generator selected by method.
It does not change the generator being used.
'state' Uses Marsaglia's ziggurat algorithm (the default in MATLAB versions 5 and later).
The period is approximately 2^64.
'seed' Uses the polar algorithm (the default in MATLAB version 4).
The period is approximately (2^31-1)*(pi/8).
randn(method, n) initializes the state of this generator using the value of 100
Different initializers give different result.
Same initializer gives same result.

So, basically, if we want to keep the same "random" set of number through several calculation, we can do it by storing internal state.
Note that after each time we call randn, after the function generalize some random numbers, its internal state changes. If we want it to reproduce some set of "random" numbers again, we need to keep track of the internal state.

randn('state', 0) Set randn to its default initial state
randn('state', sum(100*clock)) Initialize randn to a different state each time

Example:
s = randn('state') % Save the current state
u1 = randn(3) % Generate 3x3 values
randn('state', s) % reset the state
u2 = randn(3) % contains exactly the same values as u1

Simulate SDEs using Matlab

http://www.maths.strath.ac.uk/~aas96106/algfiles.html

Numerical Computing with Matlab

Individual Chapters from Mathworks.

Octave - 1

Begin: octave
Interrupt: Ctrl + C
Exit: quit or exit
suspend: Ctrl + Z
EXAMPLES:

Matrices

octave:1> A = [ 1, 1, 2; 3, 5, 8; 13, 21, 34 ]  (Create matrix to store for later. printout)
octave:2> B = rand (3, 2);                      (Create matrix 3 rows 2 columns, random vars, no printout)
octave:3> B     (Display B)

octave:4> 2 * A     (Matrix A times scalar a)
octave:5> A * B     (Matrix matrix multiply)
octave:6> A' * A    (transpose (A) * A)

octave:7> A \ b     (solve Ax = b)

Basic Linux Commands - 4

Command	Function
ls	Short listing of directory contents
cd [dirname]	Change directory to [dirname]
cd ..	Change directory and go back a level
cd	Change back to your home directory
cd	Change back to your home directory
cp [oldfile] [newfile]	Copy a file
mv [oldpath/oldfile] [newpath/newfile]	Move or rename a file
rm [file_to_delete]	Delete a file (files are unrecoverable)
gzip [file_to_compress]	Compress a file using gzip
gunzip [file_to_uncompress]	Uncompress a gzipped file
mkdir [new_dir_name]	Create a directory in the current directory
rmdir [dir_to_delete]	Remove a directory (if it is empty)
man [command]	Get the manual pages for [command]

Exercises on basic Unix (Linux) commands

1.1 Introduction

Hints:

• green texts are comments/explanations,
• gray text is what cames on the screen,
• red text are the commands you have to type in (and [xyz] means to type the "xyz" key)

System configuration

shell bash;
different accounts ;
User needs to have personal groups (user radio belongs to group students, ..);
su;
virtual console

1.2 Access to the system and logout

Let's start with something wrong.

login: redio[Enter]

Even if the account doesn't exist you'll be asked for a password

Password: radio2001[Enter]

Login incorrect

Let's start again

login: radio[Enter]

Password: 2001[Enter]

Last login: Sun Nov 11 10:45:11 on tty1

You get information about your last access

The prompt shell means the system is ready to receive your commands.

$

1.2.1 Switching account

logging out and then logging in with a different account
using su.

$ su root[Enter]

Password: smr1301[Enter]

If it is the right password you get identity and the rights of the typed user.

1.2.2 Virtual Consoles

[Alt+Fn] (where n is in the range 1..6).

You can start another login session on a different console

1.2.3 Who am I?

When you have more then one account may be necessary to check which account you are using.

$ whoami[Enter]

root

whoami let you know under which identity you are working on.

$ logname[Enter]

radio

logname allows you to know what account you used when you first entered in the system (login).

1.2.4 Ending a session

Just need to end the shell, that is the program that shows the prompt.

$ whoami[Enter]

root

$ exit[Enter]

In case we had used su

$ whoami[Enter]

radio

This exit closes last shell, and logs the user out of the system.

$ exit[Enter]

login:

1.2.5 Shutting down

login: root[Enter]

Password: smr1301[Enter]

# shutdown -h now[Enter]

System is going down NOW!!

...

System halted

When this happens it is possible to reboot the system.

---------

You can use su to become root and shutdown the system.

$ su[Enter]

Password: smr1301[Enter]

# shutdown -h now[Enter]

1.3 Password management

1.3.1 User root changes password to regular user

# passwd radio[Enter]

New UNIX password: 123[Enter]

The password is too short, the system advises you but it doesn't stop you from using it.

BAD PASSWORD: it's a WAY too short

Retype new UNIX password: 123[Enter]

passwd: all authentication tokens updated successfully

1.3.2 Regular user changes its password

$ passwd[Enter]

You'll be asked for the old password.

Changing password for radio

(current) UNIX password: 123[Enter]

New UNIX password: elpoep[Enter]

BAD PASSWORD: it is based on a (reversed) dictionary word

passwd: Authentication token manipulation error

$ passwd[Enter]

Changing password for radio

(current) UNIX password: 123[Enter]

New UNIX password: I cannot understand PCs[Enter]

Retype new UNIX password: I cannot understand PCs[Enter]

passwd: all authentication tokens updated successfully

1.4 Moving around directories

The filesystem is organizated in directory and subdirectory.

1.4.1 Current Directory

$ cd /usr/bin[Enter]

$ pwd[Enter]

/usr/bin

1.4.2 Absolute and relative path

The absolute path starts from the root directory while the relative one starts from the current directory.

$ cd /usr/local[Enter]

This is an absolute path since it starts with /.

$ pwd[Enter]

/usr/local

$ cd bin[Enter]

This is a relative one: it starts from the local directory moving downward to bin.

$ pwd[Enter]

/usr/local/bin

1.4.3 Moving backward

Every directory holds two references to special subdirectory:

single (.) that is a reference to the current directory.

double (..) that is a reference to the previous directory.

These symbols are effective directory names.

$ cd ..[Enter]

$ pwd[Enter]

/usr/local

$ cd ../bin[Enter]

$ pwd[Enter]

/usr/bin

You may go backward for more then one level.

$ cd ../../var/tmp[Enter]

$ pwd[Enter]

/var/tmp

$ cd /usr/bin/../local/bin/..[Enter]

$ pwd[Enter]

/usr/local

1.4.4 The precise reference to the current directory

The current directory can be seen as a single dot. In practice all relative paths can start with the prefix ./.

$ cd ./bin[Enter]

$ pwd[Enter]

/usr/local/bin

1.4.5 Home Directory

Every user has his own personal directory, known as home, it keeps all the data owned by the user. Typing just cd the user can reach his home directory.

$ cd[Enter]

$ pwd[Enter]

/home/radio

Some shells replace tilde (~) at the begging of a path with the path of the home directory of the working user.

$ cd ~[Enter]

$ pwd[Enter]

/home/radio

In the same way if tilde is set before a user account it will be replaced with the path of user's home directory.

$ cd ~ftp[Enter]

$ pwd[Enter]

/home/ftp

Going back to home directory.

$ cd[Enter]

1.5 File content

1.5.1 Directory content

To list the directory content command ls is used

$ ls /bin[Enter]

arch           dd             gzip       netconf        sleep
ash            df             hostname   netstat        sort
ash.static     dmesg          igawk      nice           stty
aumix-minimal  dnsdomainname  ipcalc     nisdomainname  su
awk            doexec         kill       ping           sync
basename       domainname     linuxconf  ps             tar
bash           echo           ln         pwd            tcsh
bash2          ed             loadkeys   red            touch
bsh            egrep          login      remadmin       true
cat            ex             ls         rm             umount
chgrp          false          mail       rmdir          uname
chmod          fgrep          mkdir      rpm            userconf
chown          fsconf         mknod      rvi            usleep
consolechars   gawk           mktemp     rview          vi
cp             gawk-3.0.4     more       sed            view
cpio           grep           mount      setserial      vimtutor
csh            gtar           mt         sfxload        ypdomainname
date           gunzip         mv         sh             zcat

The command ls /bin lists the content of /bin/.

A more expressive listing can be obtened using option -l.

$ ls -l /bin[Enter]

-rwxr-xr-x    1 root     root         2612 Mar  7 11:29 arch
-rwxr-xr-x    1 root     root        60592 Feb  3 20:12 ash
-rwxr-xr-x    1 root     root       263064 Feb  3 20:12 ash.static
-rwxr-xr-x    1 root     root         9968 Feb  3 19:04 aumix-minimal
lrwxrwxrwx    1 root     root            4 Apr 13 23:28 awk -> gawk
-rwxr-xr-x    1 root     root         5756 Mar  7 12:15 basename
-rwxr-xr-x    1 root     root       316848 Feb 27 18:44 bash

... many lines ...

-rwxr-xr-x    1 root     root         4320 Mar  7 12:15 true
-rwsr-xr-x    1 root     root        26608 Feb  3 15:14 umount
-rwxr-xr-x    1 root     root         6196 Mar  7 12:15 uname
lrwxrwxrwx    1 root     root           14 Apr 13 23:49 userconf -> /bin/linuxco
nf
-rwxr-xr-x    1 root     root        16252 Mar  8 17:26 usleep
-rwxr-xr-x    1 root     root       346352 Mar  7 18:18 vi
lrwxrwxrwx    1 root     root            2 Apr 14 00:00 view -> vi
-rwxr-xr-x    1 root     root          362 Mar  7 18:18 vimtutor
lrwxrwxrwx    1 root     root            8 Apr 13 23:51 ypdomainname -> hostname
-rwxr-xr-x    3 root     root        46384 Feb 15 17:04 zcat

$ cd[Enter]

$ ls[Enter]

To list hidden files

$ ls -a[Enter]

.              .bash_history  .enlightenment       .mc
..             .bash_logout   .gnome               .tcshrc
.ICEauthority  .bash_profile  .gnome-desktop       .xsession-errors
.Xauthority    .bashrc        .gnome-help-browser
.Xdefaults     .cshrc         .gnome_private

1.5.2 File Content

In order to analyse the file content cat,less and more ca be used (it doesn't make sense if it is a binary file).

$ cat /etc/issue[Enter]

Red Hat Linux release 6.2 (Zoot)
Kernel 2.2.14-5.0 on an i586

1.5.3 Find out the file type

file is command the uses the so called magic number (Unix tradition) to figure out the file type..

$ file /etc/*[Enter]

/etc/CORBA: directory

/etc/DIR_COLORS:           English text
/etc/HOSTNAME:             ASCII text
/etc/X11:                  directory
/etc/adjtime:              ASCII text
/etc/aliases:              English text

...many lines...

/etc/shells:               ASCII text
/etc/skel:                 directory
/etc/smb.conf:             English text
/etc/smrsh:                directory
/etc/snmp:                 directory
/etc/sound:                directory
/etc/sysconfig:            directory
/etc/sysctl.conf:          English text
/etc/syslog.conf:          English text
/etc/termcap:              English text
/etc/up2date.conf:         can't read `/etc/up2date.conf' (Permission denied).
/etc/vga:                  directory
/etc/yp.conf:              English text
/etc/ypserv.conf:          English text

This method is not realible but can be useful.

1.5.4 Used and free space

In order to check the free space on disk you can use df.

$ df[Enter]

The result maybe something similar to this one.

Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/hda5              1511968    805300    629860  56% /
/dev/hda1                99521      2482     91900   3% /boot

To check the directory used space: du.

$ du /bin[Enter]

5240 /bin

In this case the /bin/ directory holds files for a total amount of 5240KB.

1.6 File creation, copy and deletion

1.6.1 File creation

There are different ways to create a file. The easiest way to create an empty file si to use touch. First move to the home directory, the best place where to play.

$ cd[Enter]

$ touch myfile[Enter]

$ ls -l myfile[Enter]

-rw-rw-r-- 1 radio radio 0 Dec 23 10:49 myfile

The file was created..

You can use cat too:

$ cat > myfile2[Enter]

there are better ways to write[Enter]

text.[Enter]

This is a oneway writing.[Enter]

[Ctrl+d]

$ cat myfile2[Enter]

1.6.2 File copy

$ cp myfile2 myfile3[Enter]

Group copy is possible only if the last file is an existing directory.

$ cp myfile myfile2 myfile3 /tmp[Enter]

$ cp myfile* /tmp[Enter]

1.6.3 File deletion

Becareful when you delete something as root!!

$ rm myfile myfile2[Enter]

There is noway to recover deleted files.

You can use the wild chars:* and ?.

$ ls myfile*[Enter]

myfile3

$ rm myfile*[Enter]

1.7 Working with directory

1.7.1 Directory creation

$ cd[Enter]

$ mkdir mydir[Enter]

Let's check with ls.

$ ls -l[Enter]

...

drwxr-xr-x 8 radio radio 1024 Dec 23 12:11 mydir

...

The d character at the beginning of the string tells has that the file is a directory.

1.7.2 Directory copy

cp with option -r or-R.

$ cp -r mydir mydir2[Enter]

1.7.3 Directory deletion

You may delete an empty directory using rmdir.

$ rmdir mydir2[Enter]

Let's try with something more complex.

$ mkdir carbon[Enter]

$ mkdir carbon/hydrogen[Enter]

$ mkdir carbon/oxygen[Enter]

$ mkdir carbon/hydrogen/helium[Enter]

$ rmdir carbon[Enter]

rmdir: carbon: Directory not empty

$ rm -r carbon[Enter]

1.8 Moving around directories and linking files

In Unix environment rename and moving a file are the same thing.

1.8.1 Moving and renaming

The command used is mv.

$ touch white[Enter]

$ touch green[Enter]

$ mkdir purple[Enter]

Let's check.

$ ls -l[Enter]

...

-rw-rw-r-- 1 radio radio 0 Dec 25 12:46 white

-rw-rw-r-- 1 radio radio 0 Dec 25 12:46 green

drwxrwxr-x 2 radio radio 1024 Dec 25 12:46 purple

...

Let's rename white file to make it brown.

$ mv white brown[Enter]

$ ls -l[Enter]

...

-rw-rw-r-- 1 radio radio 0 Dec 25 12:46 brown

...

To move more file all in once the destination must be a directory.

$ mv brown green purple[Enter]

$ ls -l purple[Enter]

-rw-rw-r-- 1 radio radio 0 Dec 25 12:46 green

-rw-rw-r-- 1 radio radio 0 Dec 25 12:46 brown

$ mv purple /tmp[Enter]

1.8.2 Linking

Instead of copying a file we may want to create a reference to it. There are two kind of links that can be created hard links and soft links. Let's see the soft one

The command is ln with the option -s.

Let's set the environment.

$ touch one[Enter]

$ touch two[Enter]

$ mkdir three[Enter]

Checking ..

$ ls -l[Enter]

...

-rw-rw-r-- 1 radio radio 0 Dec 25 12:46 two

drwxrwxr-x 2 radio radio 1024 Dec 25 12:46 three

-rw-rw-r-- 1 radio radio 0 Dec 25 12:46 one

$ ln -s one one.bis[Enter]

$ ls -l[Enter]

...

lrwxrwxrwx 1 radio radio 3 Dec 25 12:47 one.bis -> one

It's the same for directories.

$ ln -s /tmp miatemp[Enter]

$ ln -s /home/radio/one* /home/radio/two three[Enter]

$ ls -l three[Enter]

lrwxrwxrwx 1 radio radio 15 Dec 25 15:21 two -> /home/radio/two

lrwxrwxrwx 1 radio radio 15 Dec 25 15:21 one -> /home/radio/one

lrwxrwxrwx 1 radio radio 19 Dec 25 15:21 one.bis -> /home/radio/one.bis

1.9 The shell

The shell is the way to interact with the operative systme. The shell bash is the one used in our exercises.

1.9.1 Automatic fulfilling

The shell can fulfill a command using [Tab], this feature is particularly useful when you have file with long names.

$ touch microprocessor[Enter]

$ touch microscopic[Enter]

$ touch supersonic[Enter]

$ ls sup[Tab]

$ ls sup[Tab]ersonic[Enter]

$ ls mic[Tab]ro

$ ls mic[Tab]rop[Tab]rocessor[Enter]

1.9.2 Substitution: wild char

This is an alternative way to fulfill a command, it is the the shell that changes symbols with the right information.

1.9.2.1 Asterisk *

That symbol can be replaced with a sequence, from 0 to infinity, of symbols.

$ ls[Enter]

$ ls *[Enter]

This command is different, the shell changes the * with the list of files and directory held in the current directory. This means that if there is any subdirectory its content

would be displayed.

ls micro*[Enter]

microprocessor microscopic

* can be changed with null string:

$ touch millimicro[Enter]

$ ls *micro*[Enter]

microprocessor microscopic millimicro

1.9.2.2 Question mark ?

The question mark ? can be change with just one symbol

Let's create some files.

$ touch xy123j4[Enter]

$ touch xy456j5[Enter]

$ touch xy789j111[Enter]

$ touch xy78j67[Enter]

$ ls [Enter]

xy123j4
xy456j5
xy789j111
xy78j67

$ ls ?????j?[Enter]

xy123j4
xy456j5

We would have a different result using *

$ ls *j*[Enter]

xy123j4 xy456j5 xy789j111 xy78j67

1.9.2.3 Square brackets [ ]

The square brackets are used to have a range of symbols from which to choose the substitution symbols. Just one symbol from the one listed is used.

$ ls xy????[4567]*[Enter]

xy123j4 xy456j5

$ ls xy????[4-7]*[Enter]

1.9.2.4 Escape

For some special symbols you need to use escape

$ touch six\*height[Enter]

$ ls[Enter]

...

six*height

If you create a filename with a space inside

$ touch my\ letter[Enter]

$ ls[Enter]

...

my letter

six*height

1.9.3 Input/Output redirection and pipeline

The shell allows you to redirect command output from the standard output (usually the screen). The same for the input.

1.9.4 Redirection

$ ls -l > mylist[Enter]

$ cat mylist[Enter]

For the input .. instead of using the standard input (keyboard).

$ cat < mylist[Enter]

Appending to a file.

$ ls -l /tmp >> mylist[Enter]

$ cat mylist[Enter]

1.9.5 Pipeline

The pipeline is a way of redirecting input and output commands.

$ cat mylist | sort[Enter]

$ cat <>

Make it easier but without using pipeline.

$ sort <>

1.9.6 Alias

Aliases allows you to create an alternative name to an existing command.

$ alias ll='ls -l'[Enter]

$ ll[Enter]

This alias take options as an ordinary command.

$ ll micro*[Enter]

It's the same as ls -l micro*.

-rw-rw-r-- 1 radio radio 0 Dec 26 10:19 microprocessor

-rw-rw-r-- 1 radio radio 0 Dec 26 10:19 microscopic

This are the aliases that are usually created to avoid mistake

$ alias rm='rm -i'[Enter]

$ alias cp='cp -i'[Enter]

$ alias mv='mv -i'[Enter]

Now try to remove a file.

$ rm microprocessor[Enter]

rm: remove `microprocessor'?:

n[Enter]

In this way the file was not removed.

1.10 Searching

Searching files or directories it's an important task in a filesystem as complex as the Linux (or Unix) one.

1.10.1 Find

Searching a file or directory using its name or other external feature find command is used.

$ find / -name bash -print[Enter]

This command search for files and directory named bash inside all the directory starting from the root directory (/).

/bin/bash

...

find: /var/run/sudo: Permission denied

find: /var/spool/at: Permission denied

find: /var/spool/cron: Permission denied

...

You can use wild chars, in this case it will be the find command that will have to manage them, without using shell.

$ find / -name \*sh -print[Enter]

The escape char \ tells the shell not to translate the * as wild char.

/bin/bash

/bin/ash

/bin/sh

...

1.10.2 Grep
To search inside the content of a file grep is used.

$ grep radio /etc/*[Enter]

/etc/group:radio::500:radio

/etc/passwd:radio:Ide2ncPYY1234:500:500:radiolab:/home/radio:/bin/bash
grep: /etc/skel: Is a directory
grep: /etc/sudoers: Permission denied
...

1.16 References

Matt Chapman, Frankie Blaskovic, The Guide -- A Beginners Guide to UNIX

http://www.belgarath.demon.co.uk/guide/

Christopher C. Taylor, Unix is a Four Letter Word... and Vi is a Two Letter Abbreviation

http://www.linuxbox.com/~taylor/4ltrwrd/