Midterm study guide:
You will be expected to know the material in Stochastic Processes that
was covered in HWs 2,3,4,5, and 6, including
- Markov chains on finite S in discrete time
- Irreducibility, periodicity, communication classes, recurrent and transient states
- Positive recurrent and null recurrent Markov chains on countably infinite S
- Branching processes and extinction probability
- Definition and properties of superharmonic functions
- Computing state valuations for optimal stopping time with cost and discounting
- Applying the definition of martingale
- Applying the optional sampling theorem
There are 7 questions and 80 minutes to answer them. No books, notes,
calculators or other electronic devices may be used.