STAT 6310, Introduction to Stochastic Processes , Spring 2006

Instructor: Prof. Jaimie Kwon (Homepage)

Lecture: TuTh ScC 205, 4:00 -5:50 PM

Textbook: Higgins and Keller-McNulty, Concepts in Probability and Stochastic Modeling, Duxbury Press, 1994, ISBN: 0534231365

Typos (utoronto, sdsu, ualberta)

Objectives: This course is an introduction to stochastic models and their applications to science and engineering. The emphasis will be on modeling random processes occurring or evolving in time. The major topics covered in this course are discrete-time Markov chains (Chapter 4), Bernoulli counting and Poisson processes (Chapter 7), queueing models and birth-death processes (Chapter 7), and continuous-time Markov chains (Chapter 9). Other possible topics include reliability, renewal processes, martingales, and Brownian motion.

Prerequisite: STAT/MATH 3401 or STAT/ENGR 3601. I will also assume typical first year Stat MS students background (State 3402 and 3502). Students will be mostly OK without 3402 and 3502 but must be willing to learn necessary materials as you go along.

Syllabus

Lecture Note (MS Word): See the disclaimer

Gray area in the following table is tentative:

Week Date Chapters coveres

HW and Announcement (Usually due in a week). Solutions will be posted/handed out on the due date

 Materials
1 3/27 Review of chapters 1-3, 4

Homework #1
Optional review exercises:
Ex 2.2-4, 5, 6,
Ex 2.7-1, 4, 5, 6
Ex 2.8-1, 4
Ex 2.9-1, 2, 3
Ex 3.2-1, 2, 3
Ex 3.6-1, 4
Try solving at least the last question in each section above; you won't see these problems on the exams since I assume you know how to do them; if they seem very difficult, you might want to consider changing to 4401 or take this course later.

Main exercises:
Ex 4.2-1, 2, 3, 4, 8, 9
Ex 4.3-1, 2, 3, 4, 5, 6, 7
Aim to do these by next Tuesday (4/4)

Review material (chapters 1-4, pdf)

 
2 4/3 4

Homework #2
Proofs are difficult but give your best try (<30 min each); using computers are OK but know what you're doing
Ex 4.4-1
Ex 4.6-1,2,5,7,8,9,10,12
Ex 4.7-1,2,4,5,6,7,8,9,10
Due next Tuesday (4/11)

 HW #1 solutions (pdf)
3 4/10  

Homework #3
Due ...

Quiz #1 Thursday (4/13, 5-6 PM) covers sections 4.1-4.6

quiz #1 solutions (pdf)

 

HW #2 solutions (pdf)

R code for Gambler's ruin Markov chain:

P <- matrix(c(1, 0, 0, 0, 0,
.6, 0, .4, 0, 0,
0, .6, 0, .4, 0,
0, 0, .6, 0, .4,
0, 0, 0, 0, 1), byrow=TRUE, nrow=5, ncol=5)
P
matrix.power(P, 2)
matrix.power(P, 3)
matrix.power(P, 4)
matrix.power(P, 100)
Q <- P[2:4, 2:4]
R <- P[2:4, c(1,5)]
diag(3)
solve(diag(3)-Q) %*% c(1,1,1) # gives Matrix Mu
solve(diag(3)-Q) %*% R # gives Matrix F
solve(diag(3)-Q) # Gives Matrix U
4 4/17 review 5, 6

Midterm #1 Thursday (4/20, 4-6 PM) covers chapters 4

midterm #1 (pdf)
solutions (pdf)

  
5 4/24 7 Homework #3
Proofs are difficult but give your best try (<30 min each); using computers are OK but know what you're doing
Ex 7.1-3, 4, 5, 8, 9, 10
Ex 7.2-1, 3, 4, 5
Ex 7.3-3, 4, 5
Ex 7.4-1, 3, 6, 7, 8
Ex 7.5-2, 3, 5, 6, 7, 8, 9
Ex 7.7-3, 4, 6
Due 5/9
p <- .07 # frame success probability
ntimes <- 100 # number of frames
## Run this first
successes <- rbinom(ntimes, 1, p) # 0-1 outcomes
N <- cumsum(successes) # cumulative number of successes
plot(1:ntimes, N, xlab='Frames n', ylab='Count X(n)',
main=paste('Bernoulli Counting Process with p=',p),
ylim=c(-1, ntimes*p*2), type='b')
abline(h=0)
points(1:ntimes, rep(-1, ntimes), col=successes, pch=15)
## Then run the following to add more simulation runs
nsims <- 100
for(i in 1:nsims){
successes <- rbinom(ntimes, 1, p)
N <- cumsum(successes)
# lines(1:ntimes, N)
lines(1:ntimes, jitter(N)) # jitter points to get clearer picture
}
6 5/1     HW #3 solutions (pdf)
7 5/8  

Quiz #2 Tuesday (5/9, 5-6 PM)

quiz #2 solutions (pdf)

  
8 5/15 selected part of 8-10

Thursday class (5/18) is off; I'm attending a conference then. There will be a reading assignment instead.

Midterm #2 Tuesday (5/16, 4-6 PM)

midterm #2 solutions (pdf) (error in question #2 is corrected.)

HW #4.
Ex 8.4-3
Ex 9.3-1,2,3,4
Ex 10.1-2,3
Ex 10.2-1,2,3,5
Ex 10.3-1,3,4
Ex 10.4-1,2
Ex 10.5-1

HW #4 solutions (pdf)

  
9 5/22      
10 5/29    
11 6/5  

Final: Tuesday 6/6/2006, 4:00 pm to 5:50 pm

See Final schedule

 

Last updated 05/30/2006