This is a course on stochastic processes, which involve collections of random variables indexed by time or by space. In this course you will learn the nomenclature and techniques needed for understanding the major types of stochastic processes, how to apply these processes in mathematical modeling, and how to effectively compute and simulate using these processes. We will cover materials including (not limited to) discrete-time and continurous-time Markov Chain, Reversible Markov Chain, hidden Markov Model (HMM) (see more details at a sylabus). For computing I will be teaching the basics of MATLAB, although you may utilize any environment you are familiar with for completing the assignments.

**Introduction to Stochastic Processes** 2nd edit G. F. Lawler.

Ruriko Yoshida, Assistant Professor of Statistics

- Office: 805A POT
- Phone: (859) 257-5698
- E-mail:
*ruriko.yoshida@uky.edu* - Office Hours: MW 10:00 AM - 11:00 AM

- MWF 12:00 PM - 12:50 PM, CB307

www.polytopes.net/courses/Stat624S07

We will cover Chapter 1, 2, 3, 4, 6, and 7. In addition, We will cover Hidden Markov Model and also applications to computational Biology (especially, to alignment problems and also evolutional model). For more details see PDF.

**Exam 1**Wed March 21.**Exam 2**Wed April 18.**Final**1 PM on Monday April 30. ROOM CB 307.

There will be two in-class exams, graded homeworks, programming assignments, and a final exam: these will count 40%, 20%, 15%, and 25% of your grade, respectively.

Students with excused absences will be given a make-up exam. No homework will be made up for credit, but it's important to make it up for your own benefit. The lowest scored HW will be discarded. Late homework will not be accepted. No make up final.

- Basic Probability (PDF)
- MATLAB Exercise (PDF). Matlab code for LU decomposition (ZIP file). Templete file (Change this file name to LUsolve.m). Test file.
- MATLAB Tutorial 1 (PDF)
- MATLAB Tutorial 2 (PDF)
- MATLAB Tutorial 3 (PDF)
- MATLAB Tutorial 4 (PDF)
- MATLAB Tutorial 5 (PDF)
- MATLAB Tutorial 6 (PDF)

Write each problem neatly. If you have any question about grading, you must prove why you think so. Regrading will be in 7 days after returning. After 7 days, regrading will NOT be accepted. All exams will be in class. Calculator is allowed. They are closed book exams.

**Exam 1**March 21. Summary for Exam 1 (PDF). Review discussed in a class for exam 1 (PDF). Mean = 91.71, SDT DIV = 4.8278, Highest = 98.**Exam 2**April 18. Review discussed in a class for exam 2 (PDF). Mean = 85.23, STD DIV = 9.3771, Highest = 99.**Final**April 30. Review discussed in a class for the final (PDF). Mean = 171.83, STD DIV = 14.987, Highest 196.**Common Exam**May 29th. Practice problems for DTMC for the exam (PDF). Also I found a nice problem set on the website..... Here is a PDF file.

For computing problems, I recommend you to use MATLAB. But if you want to use R or SAS it is fine if you do not use pre-existing functions. Please show me what you have before hand in if you are using R or SAS. If you do not get my permission then you will get no credits.

The problem session for HWs is at 5pm on Tue. Room: TBA.

The deadline for redo HWs is April 27th, Fri.

- Homework 1 (PDF) Due: January 19th, Fri.
- Homework 2 (PDF) Due:
**January 31st, Wed**. - Homework 3 (PDF) Due:
**February 7th, Wed**. - Homework 4 (PDF) Due: February 16th,
Fri.
**Cut off problem 4. You do not have to do problem 4.** - Homework 5 (PDF) Due: February 23th, Fri.
- Homework 6 (PDF) Due: March 5th, Mon.
- Homework 7 (PDF) Due: March 26rd, Mon.
- Homework 8 (PDF) Due: April 2nd. Mon.
- Homework 9 (PDF) Due: April 11th. Wed.

It counts 10% of your grades.

The deadline of submitting the topic is February 28th, 2007, Wed.

The deadline of submitting the first draft is March 30th, 2007, Fri.

The deadline of submission is April 30th, 2007 at the biginning of the final exam.

- Study Phylogenetic tree models, such as the Jukes-Cantor model, the kimura 2 or 4 parameter models, or the general time reversible (GTR) model (and how to compute the Maximum likelihood estimations).
- Study the paper [Evans/Speed, Annals of Statistics 1993] on the 3-PARAMETER KIMURA MODEL for trees.
- Study computing the exact p-value from a multi-dimansional contingency tables and Markov bases.
- Study enumeration methods of contingency tables.
- Study how to generate phylogenetic trees and DNA sequences via the birth-and-death process.
- Study the pairwise Hidden Markov model (HMM) and its application to the pairwise alignment problem.
- Study HMM and its application to the GpC island problem.
- Study HMM and its application to gene-finding problem.
- Study a generalized profile hidden markov model.
- Study The Coalescent And Population Structure.
- Study the codon based model for phylogenetic trees suggested by Ziheng Yang.
- Study Stochastic optimization and its applications to Financial economics.
- Study the the Ergodic Behavior of Stochastic Processes of Economic Equilibria by Blume, Lawrence.
- Read the survey paper by David J. Aldous.

- Finite Markov Chains and Algorithmic Applications, by Olle Haagstrom. London Mathematical Society, Student Texts 52, Cambridge University Press, 2002.
- Randomized Algorithms, by R. Motwani and P. Raghavan. Cambridge University Press, 1995.
- Algorithms for Random Generation and Counting - A Markov Chain Approach, by Alistair Sinclair. Birkhauser, 1993.
- Probability and Computing, by Michael Mitzenmacher and Eli Upfal. Cambridge University Press, 2005.
- Random Walks and Electrical Networks, by P. Doyle and J.L. Snell. Carus Mathematical Monograph no. 22, Mathematical Association of America, 1984. (Available on-line for free download.)
- Uniform Random Spanning Trees, by Robin Pemantle, in Topics in Contemporary Probability and Applications, Ed. J.L. Snell, CRC Press, 1995.
- Some New Games for your Computer, by Rick Durrett, in Topics in Contemporary Probability and Applications, Ed. J.L. Snell, CRC Press, 1995.
- Cellular Automata with Errors: Problems for Students of Probability, by Andrei Toom, in Topics in Contemporary Probability and Applications, Ed. J.L. Snell, CRC Press, 1995.
- How Many Times Should You Shuffle a Deck of Cards?, by Brad Mann, in Topics in Contemporary Probability and Applications, Ed. J.L. Snell, CRC Press, 1995.
- Random Graphs in Ecology, by Joel E. Cohen, in Topics in Contemporary Probability and Applications, Ed. J.L. Snell, CRC Press, 1995.
- A Matrix Perturbation View of the Small World Phenomenon, by D.J. Higham. SIAM J. Matrix Anal. Appl. 2003, Vol. 25, No. 2, pp. 429-444 (available on-line.)
- Random Graphs, by B. Bollobas. Academic Press, 1985.
- Stochastic Modelling for Systems Biology, by Darren J. Wilkinson. CRC, Mathematical and Computational Biology Series, 2006.
- Random Walks in Biology, by Howard C. Berg. Princeton U. Press, 1993.
- Computational Cell Biology, by C.P. Fall, E.S. Marland, J.M. Wagner, J.J. Tyson (editors). IAM volume 20 - Mathematical Biology, Springer-Verlag, 2002 (corrected third print, 2005).
- Branching processes in biology, by M. Kimmel and D.E. Axelrod. Springer, 2202.
- Evolutionary dynamics - exploring the equations of life, by M.A. Nowak. Belknap/Harvard, 2006.
- Stochastic Tools in Mathematics and Science, by A.J. Chorin and O.H. Hald. Springer, 2006.
- Theory and Applications of Stochastic Differential Equations, by Z. Schuss. John Wiley, 1980.
- Stochastic Differential Equations in Science and Engineering, by D. Henderson and P. Plaschko. World Scientific, 2006.
- Music and Probability, by David Temperley. MIT Press, 2007.
- Bayesian Methods for Music Signal Analysis, by A. Taylan Cemgil. (Look for PDF file on the web.)