Math
439: Linear Statistical Models - Fall 2013

Syllabus in .pdf format.

Instructor: Dr. Todd Kuffner

Office: Room 203A, Cupples I

Email: kuffner at math.wustl.edu

Office Hours: Mon and Tues 12-1pm, or by appointment

Lectures: Mon and Wed 4-5:30pm, Cupples I, Room 218

Course Description: Theory and practice of linear regression, analysis of variance (ANOVA) and their extensions, including testing, estimation, confidence interval procedures, modeling, regression diagnostics and plots, polynomial regression, colinearity and confounding, model selection, geometry of least squares. The theory will be approached mainly from the frequentist perspective and use of the computer (mostly R) to analyze data will be emphasized. We will cover most of the material corresponding to the first 11 chapters of the required text, along with some supplementary material where needed.

Course Goals: By the end of the course, students should demonstrate knowledge of the theory underlying linear statistical models, as well as some competence in applying the theory to the analysis of data using R. Students should understand the limitations and implications of key assumptions of linear models, and have a working knowledge of common methods of estimation, hypothesis testing and model diagnostics for linear models.

Prerequisites: Math 3200 and a course in linear algebra, or permission of instructor.

Required Text: Introduction to Linear Regression Analysis, Montgomery, D.C., E.A. Peck and G.G. Vining, 5th Edition, Wiley, 2012, ISBN: 9780470542811.

Note: there is also a solutions manual to the above textbook.

References (which you may find useful--most supplementary material is contained in these books):

- Linear Models with R, by
Faraway (this text should also have
been required, you are strongly encouraged to purchase it)

- An R Companion to Linear
Statistical Models, by Hay-Jahans

- Applied Linear Statistical Models, by Kutner, Nachstheim, Neter & Li
- A Primer on Linear Models,
by Monahan (one of the best sources for the linear algebra and
multivariate analysis relevant to linear models)

- Linear Statistical Models, by Stapleton
- Statistical Models: Theory and Practice, by Freedman
- Matrix Tricks for Linear Statistical Models, by Puntanen, Styan & Isotalo
- The R Book, by Crawley

Grades: There will be assigned homework, two midterm exams and a cumulative final exam. Final grades will be determined according to the following percentages:

Homework |
40% |
6 homeworks, the lowest grade
will be dropped; hence 5 homeworks will count 8% each |

First Midterm Exam |
15% |
in class on Monday 7th October |

Second Midterm Exam |
15% |
in class on Wednesday 13th
November |

Final Exam |
30% |
6-8pm, Friday 13th December |

Lectures (updated periodically to reflect what was actually covered):

Note: all course materials are available on Blackboard

Week 1 (08/28) |
Introduction (MPV Ch. 1) |
Lecture 1 |

Week 2 (09/04) |
Simple Linear Regression (MPV
Ch. 2, Faraway Chapters 1-3) |
Lecture 2 HW1 (due Monday 16th September) |

Week 3 (09/09, 09/11) |
Simple Linear Regression (MPV
Ch. 2, Faraway Chapters 1-3) |
Lecture 3 HW2 (due Monday 23rd September) Lecture 4 |

Week 4 (09/16, 09/18) |
Multiple Linear Regression (MPV
Ch. 3, Faraway Chapters 1-3) Linear Algebra and Geometry of Least Squares (Supplementary) |
Lecture 5 Lecture 6 |

Week 5 (09/23, 09/25) |
Identifiability, Estimability,
Gauss-Markov Theorem (MPV Ch.3 and supplementary) Multivariate Normal Distributions (supplementary) |
Lecture 7 HW3 (due Wednesday 2nd October) Lecture 8 |

Week 6 (09/30, 10/02) |
Distributions of Quadratic
Forms, Cochran's Theorem (supplementary) ANOVA, General Linear Hypothesis, Simultaneous Confidence Intervals (MPV Ch. 3, Faraway Ch. 3 and supplementary) |
Lecture 9 Lecture 10 |

Week 7 (10/07, 10/09) |
Hidden Extrapolation, Standardized Regression Coefficients (MPV Ch. 3 and Faraway 5.2) |
First Midterm Exam Lecture 11 |

Week 8 (10/14, 10/16) |
Scaled Residuals, Normal
Probability Plots, Partial Regression/Residual Plots, PRESS Statistic
(MPV Ch. 4 and Faraway 4.1) Heteroscedasticity Diagnostics, Box-Cox Transformations (MPV Ch. 4 and 5, Faraway Ch. 4 and 7) |
Lecture 12 Lecture 13 |

Week 9 (10/21, 10/23) |
Generalized and Weighted Least
Squares (MPV Ch. 5 and Faraway Ch. 6) Non-normality, Robust Regression: M-Estimators, LAD, IRLS, LTS, Bootstrap (MPV Ch. 15, Faraway Ch. 6) |
Lecture 14 HW4 (due Monday 4th November) Lecture 15 |

Week 10 (10/28, 10/30) |
Leverage & Influence, Cook's
distance, DFFITS, DFBETA (MPV Ch. 6, Faraway 4.2, supplementary) Model Structure, Lack of Fit, Linearization, Polynomial Regression (MPV 4.5, 5.3 and Ch. 7, Faraway 4.3, 6.3 and 7.2) |
Lecture 16 Lecture 17 |

Week 11 (11/04, 11/06) |
Indicator Variables, Missing
Data (MPV Ch. 8, Faraway Ch. 12 and 13.2 and supplementary) Multicollinearity Diagnostics (MPV 9.1-9.4, Faraway 5.3) |
Lecture 18 HW5 (due Monday 25th November) Lecture 19 |

Week 12 (11/11, 11/13) |
Multicollineary,
Orthogonalization and Cross-Validation (MPV Ch. 11, Faraway Ch. 9) |
Lecture 20 Second Midterm Exam |

Week 13 (11/18, 11/20) |
Multicollinearity, Shrinkage,
Ridge Regression, Principal-Component Regression (MPV 9.5, Faraway Ch.
9) Variable Selection: Criterion-Based Procedures, AIC, BIC, Mallow's (MPV 10.1, Faraway 8.1 and 8.3) |
Lecture 21 HW6 (due Wednesday 4th December) Lecture 22 |

Week 14 (11/25) |
Variable Selection:
Testing-Based Procedures, Backward Elimination, Stepwise (MPV
10.2-10.3, Faraway 8.2) |
Lecture 23 |

Week 15 (12/02, 12/04) |
A Complete Example (Faraway Ch.
10 and 11) Special Topic: One-Way ANOVA (Faraway Ch. 14) |
Lecture 24 Lecture 25 |

Week 16 (reading week) |
Final Exam: Friday 13th December |