Math 439: Linear Statistical Models - Fall 2013

Syllabus in .pdf format.

Instructor:  Dr. Todd Kuffner

    Office: Room 203A, Cupples I
    Email: kuffner at math.wustl.edu
    Office Hours: Mon and Tues 12-1pm, or by appointment
    Lectures: Mon and Wed 4-5:30pm, Cupples I, Room 218

Course Description:  Theory and practice of linear regression, analysis of variance (ANOVA) and their extensions, including testing, estimation, confidence interval procedures, modeling, regression diagnostics and plots, polynomial regression, colinearity and confounding, model selection, geometry of least squares. The theory will be approached mainly from the frequentist perspective and use of the computer (mostly R) to analyze data will be emphasized. We will cover most of the material corresponding to the first 11 chapters of the required text, along with some supplementary material where needed.

Course Goals: By the end of the course, students should demonstrate knowledge of the theory underlying linear statistical models, as well as some competence in applying the theory to the analysis of data using R. Students should understand the limitations and implications of key assumptions of linear models, and have a working knowledge of common methods of estimation, hypothesis testing and model diagnostics for linear models.

Prerequisites:  Math 3200 and a course in linear algebra, or permission of instructor.

Required Text: Introduction to Linear Regression Analysis, Montgomery, D.C., E.A. Peck and G.G. Vining, 5th Edition, Wiley, 2012, ISBN: 9780470542811.

Note: there is also a solutions manual to the above textbook.

References (which you may find useful--most supplementary material is contained in these books):

• Linear Models with R, by Faraway (this text should also have been required, you are strongly encouraged to purchase it)
  
• An R Companion to Linear Statistical Models, by Hay-Jahans
  
• Applied Linear Statistical Models, by Kutner, Nachstheim, Neter & Li
• A Primer on Linear Models, by Monahan (one of the best sources for the linear algebra and multivariate analysis relevant to linear models)
  
• Linear Statistical Models, by Stapleton
• Statistical Models: Theory and Practice, by Freedman
• Matrix Tricks for Linear Statistical Models, by Puntanen, Styan & Isotalo
• The R Book, by Crawley
  

Computing: Students are required to use R to complete homework assignments. It can be freely downloaded from here.
 
Grades: There will be assigned homework, two midterm exams and a cumulative final exam. Final grades will be determined according to the following percentages:

Homework	40%	6 homeworks, the lowest grade will be dropped; hence 5 homeworks will count 8% each
First Midterm Exam	15%	in class on Monday 7th October
Second Midterm Exam	15%	in class on Wednesday 13th November
Final Exam	30%	6-8pm, Friday 13th December

Lectures (updated periodically to reflect what was actually covered):
Note: all course materials are available on Blackboard

Week 1 (08/28)	Introduction (MPV Ch. 1)	Lecture 1
Week 2 (09/04)	Simple Linear Regression (MPV Ch. 2, Faraway Chapters 1-3)	Lecture 2 HW1 (due Monday 16th September)
Week 3 (09/09, 09/11)	Simple Linear Regression (MPV Ch. 2, Faraway Chapters 1-3)	Lecture 3 HW2 (due Monday 23rd September) Lecture 4
Week 4 (09/16, 09/18)	Multiple Linear Regression (MPV Ch. 3, Faraway Chapters 1-3) Linear Algebra and Geometry of Least Squares (Supplementary)	Lecture 5 Lecture 6
Week 5 (09/23, 09/25)	Identifiability, Estimability, Gauss-Markov Theorem (MPV Ch.3 and supplementary) Multivariate Normal Distributions (supplementary)	Lecture 7 HW3 (due Wednesday 2nd October) Lecture 8
Week 6 (09/30, 10/02)	Distributions of Quadratic Forms, Cochran's Theorem (supplementary) ANOVA, General Linear Hypothesis, Simultaneous Confidence Intervals (MPV Ch. 3, Faraway Ch. 3 and supplementary)	Lecture 9 Lecture 10
Week 7 (10/07, 10/09)	Hidden Extrapolation, Standardized Regression Coefficients (MPV Ch. 3 and Faraway 5.2)	First Midterm Exam Lecture 11
Week 8 (10/14, 10/16)	Scaled Residuals, Normal Probability Plots, Partial Regression/Residual Plots, PRESS Statistic (MPV Ch. 4 and Faraway 4.1) Heteroscedasticity Diagnostics, Box-Cox Transformations (MPV Ch. 4 and 5, Faraway Ch. 4 and 7)	Lecture 12 Lecture 13
Week 9 (10/21, 10/23)	Generalized and Weighted Least Squares (MPV Ch. 5 and Faraway Ch. 6) Non-normality, Robust Regression: M-Estimators, LAD, IRLS, LTS, Bootstrap (MPV Ch. 15, Faraway Ch. 6)	Lecture 14 HW4 (due Monday 4th November) Lecture 15
Week 10 (10/28, 10/30)	Leverage & Influence, Cook's distance, DFFITS, DFBETA (MPV Ch. 6, Faraway 4.2, supplementary) Model Structure, Lack of Fit, Linearization, Polynomial Regression (MPV 4.5, 5.3 and Ch. 7, Faraway 4.3, 6.3 and 7.2)	Lecture 16 Lecture 17
Week 11 (11/04, 11/06)	Indicator Variables, Missing Data (MPV Ch. 8, Faraway Ch. 12 and 13.2 and supplementary) Multicollinearity Diagnostics (MPV 9.1-9.4, Faraway 5.3)	Lecture 18 HW5 (due Monday 25th November) Lecture 19
Week 12 (11/11, 11/13)	Multicollineary, Orthogonalization and Cross-Validation (MPV Ch. 11, Faraway Ch. 9)	Lecture 20 Second Midterm Exam
Week 13 (11/18, 11/20)	Multicollinearity, Shrinkage, Ridge Regression, Principal-Component Regression (MPV 9.5, Faraway Ch. 9) Variable Selection: Criterion-Based Procedures, AIC, BIC, Mallow's (MPV 10.1, Faraway 8.1 and 8.3)	Lecture 21 HW6 (due Wednesday 4th December) Lecture 22
Week 14 (11/25)	Variable Selection: Testing-Based Procedures, Backward Elimination, Stepwise (MPV 10.2-10.3, Faraway 8.2)	Lecture 23
Week 15 (12/02, 12/04)	A Complete Example (Faraway Ch. 10 and 11) Special Topic: One-Way ANOVA (Faraway Ch. 14)	Lecture 24 Lecture 25
Week 16 (reading week)		Final Exam: Friday 13th December