Math 5061: Theory of Statistics I
Fall 2016

Instructor: Todd Kuffner (

Lecture: 1:00-2:30pm, Monday/Wednesday, Cupples I, Room 199

Office Hours: Monday 8:00-9:00am, Tuesday 4:00-6:00pm; Cupples I Room 18 (basement).

Course Overview: This course is intended for Ph.D. students in Statistics and Mathematics. Math 5061-5062 together form a year-long sequence in mathematical statistics leading to the Ph.D. qualifying exam in statistical theory. The first semester will cover introductory measure-theoretic probability, decision theory, notions of optimality, principles of data reduction, and finite sample estimation and inference. We will discuss foundational issues, and consider several paradigms for testing, such as the Neyman-Pearson, neo-Fisherian, and Bayesian approaches. Roughly half of the first semester is devoted to the measure-theoretic foundations of probability theory and statistics. The second semester will cover asymptotic theory, including convergence in measure, limit theorems, integral and density approximations, and higher-order asymptotics. Maximum likelihood, Bayesian, and bootstrap methods will be considered. Empirical processes, large deviations, and modern topics (e.g. Bayesian nonparametric asymptotics) will be introduced as time permits. The style of the course is theorem-proof based; applications will not be emphasized, and examples will be theoretical. Statistical software is not part of the course.

Prerequisite: It is assumed that students have taken a first course in real analysis, probability, and mathematical statistics, and are familiar with basic topology, multivariate calculus, and matrix algebra. Ph.D. students are strongly encouraged to enroll in Math 5051 concurrently (Ph.D.-level measure theory and functional analysis).

If you are undecided about whether or not to take this course, it may be helpful to look at the Ph.D. qualifying exam from the last time I taught the course (2014-2015). This time there will be more measure theory and probability theory on exams.

Textbook: There are many excellent books and online resources for the material in this course. However, no single book is suitable. Due to the cost of purchasing several books, I will not require that students use any particular books. The recommended readings for each lecture are accompanied by sections of three books listed below, but students are welcome to look at other references for the same material. I will use the same books for Math 5061 and Math 5062. The links give electronic access to two of the books for Washington University students (logged in to library account) through SpringerLink, but I also recommend purchasing these books as they are excellent references for researchers.
Another suggestion: Essentials of Statistical Inference by G.A. Young and R.L. Smith, Cambridge University Press. This book is much shorter and not intended as an encyclopedic reference, but it is perhaps the most clearly-written, insightful treatment of modern statistical inference.

Homework: There will be weekly homework assignments. You are strongly encouraged to write your solutions in LaTeX. If not, then handwritten submissions must be clear and organized. Homework will be graded, but solutions will not be provided to students.

Homework grader: Qiyiwen Zhang (

Blackboard: During the semester, homework assignments, homework and midterm exam grades and any other course-related announcements will be posted to Blackboard or sent by email using Blackboard.

Attendance: Attendance is required for all lectures.  The student who misses a lecture is responsible for any assignments and/or announcements made.

Grades: 15% Homework, 20% Midterm 1, 20% Midterm 2, 45% Final

Exams: 2 in-class midterms and 1 final. The dates of the exams should not be considered fixed until the first day of class. What appears on Course Listings may be incorrect.

Homework: There will be weekly homework assignments. The lowest homework grade will be dropped. If you added the class late and missed the first homework, then that will count as your dropped homework.

Final Course Grade: The letter grades for the course will be determined according to the following numerical grades on a 0-100 scale.
impress me
[87, 90)
[77, 80)
[67, 70)
[83, 87)
[73, 77)
[63, 67)

[90, 93)
[80, 83)
[70, 73)
[60, 63)

Other Course Policies: Students are encouraged to look at the Faculty of Arts & Sciences policies.
Course Schedule: tentative; will be updated after lecture to reflect what was actually covered; AL=Athreya & Lahiri, TPE=Theory of Point Estimation (Lehmann & Casella); TSP=Testing Statistical Hypotheses (Lehmann & Romano)

Lecture 1
Review of set theory; algebras, sigma algebras; Borel sets
Reading: Appendix of AL
HW1 assigned
Lecture 2
Measures; extensions
Reading: AL 1.1-1.2
HW1 due
HW2 assigned
Labor Day; no class
Lecture 3
Completeness; measurable transformations
Reading: AL 1.3-1.4; AL 2.1-2.2
HW2 due Friday 09/09
Lecture 4
Induced measures; distribution functions;  Lebesgue and Riemann integration
Reading: AL 2.1-2.4
HW3 assigned
Lecture 5
Convergence for measurable functions
Reading: AL 2.5

Lecture 6 (guest lecturer)
Important Inequalities (Markov, Chebychev, Cramer, Jensen, Holder, Cauchy-Schwarz, Minkowski)
Reading: AL 3.1

Lecture 7 (guest lecturer)
L^p spaces, Banach spaces, and Hilbert spaces
Reading: AL 3.2-3.3

Lecture 8
Radon-Nikodym theorem; signed measures
Reading: AL 4.1-4.2
Lecture 9
Functions of bounded variation; absolutely continuous function on R; singular distributions; product spaces; product measures
Reading: AL 4.3-4.4, 5.1
Lecture 10
Fubini-Tonelli theorems; sample spaces; random variables; Kolmogorov's consistency theorem
Reading: AL 5.2 (note: 5.3-5.8 would be part of an analysis course), 6.1-6.3
Lecture 11
Expectation; moment generating functions; pi-lambda Theorem; independence
Reading: AL 6.1-6.3, 7.1

Lecture 12
Exchangeability; Representation Theorems; Borel-Cantelli lemmas; Kolmogorov's 0-1 Law
Reading: 7.1-7.2
Midterm 1 during class
Material: Lectures 1-10
Fall Break; no class
Lecture 13
Conditional expectations/probability; regular conditional distributions; Bayesian statistical experiments
Reading: 12.1-12.3
Recommended review material before next lecture: TPE 1.1-1.4
Lecture 14
Decision theory; data reduction via sufficiency
Reading: TSH 1.1-1.2, 1.4; TPE 1.6
Lecture 15
Exponential families; optimal data reduction via minimal sufficiency and completeness
Reading: TPE 1.5-1.6

Lecture 16
More data reduction; risk reduction
Reading: TPE 1.6-1.7, 2.1-2.3
Lecture 17
Optimal unbiased and location equivariant estimation; risk unbiasedness
Reading: TPE 2.1-2.3, 3.1-3.3

Midterm 2 during class
Material: Lectures 11-17
Lecture 18
Bayes estimators and average risk optimality
Reading: TPE 4.1-4.3
Lecture 19
Bayes estimators and average risk optimality
Reading: TPE 4.1-4.3
Lecture 20
Minimax estimators and worst-case optimality
Reading: TPE 5.1-5.2
Thanksgiving Break; no class
Lecture 21
Minimax estimators and worst-case optimality
Reading: TPE 5.1-5.2
Lecture 22
Minimax estimation; admissibility; simultaneous estimation
Reading: TPE 5.1-5.2, 4.7 (p. 272-277), 5.5 (p. 355-360)
Lecture 23
Robust estimation; high-dimensional estimation
Reading: handout
Last day of fall semester classes

Final Exam scheduled 6:00-8:00pm in Cupples I Room 199
Material: Lectures 1-23