Ma 309: Matrix Algebra Model Solutions to Homework Assignment 7 Prof. Wickerhauser Exercise 1 of Section 5.1, p.237. Formula: angle = arccos( /(||v||*||w||) ). We first check if the vectors are parallel. ||v|| ||w|| angle remark ----- ----- ----- ----- ------ a. 42 sqrt(14) 3*sqrt(14) 0 v,w are parallel b. 0 90 v,w are orthogonal c. 14 sqrt(17) sqrt(13) arcos(14/sqrt(13*17)) d. 8 sqrt(14) sqrt(21) arcos(8/(7*sqrt(6))) Exercise 2 of Section 5.1, p.238. Formulas (from p.227): scalar projection of v onto w is /||w|| vector projection of v onto w is (/)w scalar projection vector projection ----------------- ----------------- a. sqrt(14) (1/3)w b. 0 0 c. 14/sqrt(13) (14/13)w d. 8/sqrt(21) (8/21)w Exercise 6 of Section 5.1, p.238. Follow Example 5, p.227. The line y=2x+1 consists of the points {(x,y)=(t,2*t+1)=(0,1)+t*(1,2): t in R}. These have direction vector (1,2), so the point (x,y) on the line that is nearest to (5,2) must satisfy the orthogonality condition < (x,y)-(5,2), (1,2) > = 0. But this reduces to a single equation in the unknown t: <(t-5,2*t+1-2),(1,2)>=0 ==> t-5 + 2*(2*t-1) = 0 ==> 5*t=7 ==> t=7/5 ==> (x,y) = (7/5, 19/5) Exercise 9 of Section 5.1, p.238. Follow Example 7, p.228. A normal vector to the plane is N=(2,2,1), so for v=(1,1,1) the distance is given by the scalar projection of v onto N, or /||N|| = 5/3 Exercise 13 of Section 5.1, p.238. It is not true. We may take x1=x3=(1,0,0) and x2=(0,1,0). Then =0 and =0, but =1. Exercise 16 of Section 5.1, p.239. a. Check the inner product: = = - by the linearity of <,> in its second factor. But = ||p||^2 = [^2/^2] ||y||^2 = ^2/, since ||y||^2 = . Also, = (/) = ^2/ Thus =, and the result follows. b. By Pythagoras' theorem, since x=p+z is an orthogonal decomposition, we have ||x||^2 = ||p||^2 + ||z||^2 = 6^2+8^2, so ||x|| = sqrt(36+64) = 10. Exercise 1ab of Section 5.2, p.247. a. A' = 3 6. Columns are multiples of (3,4)', 4 8 so R(A') = Span{(3,4)'} and {(3,4)'} is a basis. By Theorem 5.2.1, N(A)=R(A')^, and (-4,3)' is perpendicular to (3,4)', so N(A)=Span{(-4,3)'}. A = 3 4. Columns are multiples of (1,2)', so R(A)=Span{(1,2)'} 6 8 and {(1,2)'} is a basis. By Theorem 5.2.1, N(A')=R(A)^, and (-2,1)' is perpendicular to (1,2)', so N(A')=Span{(-2,1)'} and {(-2,1)} is a basis. b. A' = 1 2. Columns are linearly independent, 3 4 1 0 so R(A')=Span{(1,3,1)',(2,4,0)'} and {(1,3,1)', (2,4,0)'} is a basis. By Theorem 5.2.1, N(A)=R(A')^, and (-4,2,-2)' is perpendicular to both (1,3,1)' and (2,4,0)'. But N(A) can be at most one dimensional by Theorem 5.2.2, so N(A)=Span{(-4,2,-2)'} and (-4,2,-2)' is a basis. A = 1 3 1. After row reduction to row echelon form we see that 2 4 0 columns 1 and 2 are linearly independent but column 3 is a linear combination of 1 and 2, so R(A)=Span{(1,2)', (3,4)'} and {(1,2)', (3,4)'} is a basis. Thus R(A) is all of R^2. By Theorem 5.2.1, N(A')=R(A)^={0} since R(A) is all of R^2. Exercise 4 of Section 5.2, p.248. Solve for elements y=(a,b,c,d) perpendicular to both x1=(1,0,-2,1)' and x2=(0,1,3,-2)': =0 ==> a+0-2c+d=0 =0 ==> 0+b+3c-2d=0 The associated matrix is already in reduced row echelon form: A = 1 0 -2 1 a = 2c-d 0 1 3 -2 b = -3c+2d There are two columns without leading ones, so there are two free variables and the nullspace is two-dimensional: N(A) = {(2c-d,-3c+2d,c,d)': c,d in R} = {c*(2,-3,1,0)'+d*(-1,2,0,1)': c,d in R} = Span{(2,-3,1,0)', (-1,2,0,1)'}. Hence {(2,-3,1,0)',(-1,2,0,1)'}, which is evidently a linearly independent set, forms a basis for the orthogonal complement of Span{x1,x2}. Exercise 5 of Section 5.2, p.248. a. Use the vector cross product N=(P2-P1)X(P3-P1)=(8,-2,1) to get a vector perpendicular to both P2-P1 and P3-P1. b. Use the normal equations =0 to get the equation of the plane: 8(x-1)-2(y-1)+(z-1)=0 ==> 8x-2y+z=7 Exercise 6 of Section 5.2, p.248. No. The row space of a matrix is orthogonal to its nullspace by Theorem 5.2.1, so vectors in one must be orthogonal to vectors in the other. Since <(3,1,2),(2,1,1)>=9 != 0, it is not possible for (3,1,2) to be in the row space of a matrix with (2,1,1)' in its nullspace. Exercise 9 of Section 5.2, p.248. By the rank+nullity theorem, Rank(A)+dim N(A)=n, so dim N(A)=n-r. Likewise, Rank(A')+dim N(A')=m, so dim N(A')=m-r. Exercise 13 of Section 5.2, p.248. a. x in N(A'A) ==> (A'A)x=0 ==> A'(Ax)=0 ==> Ax in N(A'). But also, Ax is in R(A) for every x, by definition. b. Combining part a and theorem 5.2.1, which that says R(A) and N(A') are orthogonal, we conclude that x in N(A'A) ==> Ax=0 ==> x in N(A), so N(A'A) is contained in N(A). But also, x in N(A) ==> Ax=0 ==> A'Ax=0 ==> x in N(A'A), so N(A) is contained in N(A'A). Thus N(A)=N(A'A). c. A is mxn and A'A is nxn, so both have the same number of columns. By the rank+nullity theorem, Rank(A)=n-dim N(A) and Rank(A'A)=n-dim N(A'A). From part b we have N(A)=N(A'A) ==> dim N(A)=dim N(A'A), so by substitution of equals we conclude that Rank(A)=Rank(A'A). d. A has linearly independent columns ==> Rank(A)=n ==> Rank(A'A)=n by part c. But A'A is nxn, so its columns are linearly independent and by Theorem 3.6.3 it is nonsingular. Exercise 1 of Section 5.3, p.258. a. Solve the normal equations using MatLab: >> A=[1 1;2 -3; 0 0]; b=[3 1 2]'; det(A'*A) ans = 25 A'*A is nonsingular, so we may use Theorem 5.3.2: >> inv(A'*A)*(A'*b) ans = 2.0000 1.0000 Hence the least-squares solution is x1=1, x2=2. b. Solve the normal equations using MatLab: >> A=[-1 1; 2 1; 1 -2]; b=[10 5 20]'; det(A'*A) ans = 35 A'*A is nonsingular, so we may use Theorem 5.3.2: >> inv(A'*A)*(A'*b) ans = 2.7143 -3.7143 Hence the least-squares solution is x1=2.7143, x2= -3.7143. c. Solve the normal equations using MatLab: >> A=[1 1 1; -1 1 1; 0 -1 1; 1 0 1]; b=[4 0 1 2]'; det(A'*A) ans = 30 A'*A is nonsingular, so we may use Theorem 5.3.2: >> inv(A'*A)*(A'*b) ans = 1.6000 0.6000 1.2000 Hence the least-squares solution is x1=1.6, x2= 0.6, x3=1.2. Exercise 2 of Section 5.3, p.258. For 1a: >> A=[1 1;2 -3; 0 0]; b=[3 1 2]'; >> x=(A'*A)\(A'*b); p=A*x; r=b-p; >> [p r] 3.0000 -0.0000 1.0000 0.0000 0 2.0000 >>A'*r 1.0e-14 * 0 -0.2220 This is within double-precision truncation error of 0. For 1b: >> A=[-1 1; 2 1; 1 -2]; b=[10 5 20]'; >> x=(A'*A)\(A'*b); p=A*x; r=b-p; >> [p r] -6.4286 16.4286 1.7143 3.2857 10.1429 9.8571 >> A'*r 1.0e-14 * -0.5329 0 This is within double-precision truncation error of 0. For 1c: >> A=[1 1 1; -1 1 1; 0 -1 1; 1 0 1]; b=[4 0 1 2]'; >> x=(A'*A)\(A'*b); p=A*x; r=b-p; >> [p r] 3.4000 0.6000 0.2000 -0.2000 0.6000 0.4000 2.8000 -0.8000 >> A'*r 1.0e-14 * -0.1110 0.0222 -0.1554 This is within double-precision truncation error of 0. Exercise 3 of Section 5.3, p.258. a. Get the augmented matrix from the normal equations with MatLab: >> A=[1 2; 2 4; -1 -2]; b=[3 2 1]'; [A'*A A'*b] C = 6 12 6 12 24 12 Reduce to row-echelon form: >> rref(C) 1 2 1 0 0 0 The second variable is free, so the solution set is {(1-2t,t):t in R} = (1,0)+Span{(-2,1)} b. Get the augmented matrix from the normal equations with MatLab: >> A=[1 1 3; -1 3 1; 1 2 4]; b=[-2 0 8]'; C=[A'*A A'*b] C = 3 0 6 6 0 14 14 14 6 14 26 26 Reduce to row-echelon form: >> rref(C) 1 0 2 2 0 1 1 1 0 0 0 0 The third variable is free, so the solution set is {(2-2t, 1-t, t):t in R} = (2,1,0)+Span{(-2,-1,1)}. Exercise 7 of Section 5.3, p.259. Rewrite the normal equations with the given notation: For x=[x1...xn], A=[1...1; x1...xn]' and y=[y1...yn]', writing x0,y0 for the respective averages of x's and y's, A'Ax=A'y <==> / n n*x0 \ / c0 \ = / n*y0 \ \ n*x0 x'x / \ c1 / \ x'y / If x0=0, then the off-diagonal terms in the matrix A'A vanish, resulting in a diagonal matrix. x0=0 ==> / n 0 \ / c0 \ = / n*y0 \ \ 0 x'x / \ c1 / \ x'y / We invert to get: / c0 \ = / 1/n 0 \ / n*y0 \ \ c1 / \ 0 1/(x'x) / \ x'y / ==> c0 = y0, c1 = (x'y)/(x'x) as claimed. Exercise 8 of Section 5.3, p.259. From exercise 7 above, x0=0 ==> least squares line has the equation Y = c1*X + c0 with c0=y0. This line evidently passes through the point (0,y0)=(x0,y0). Given any set of points P={(xi,yi):i=1,...,n}, we may change coordinates to x'=x-x0, y'=y, where x0 is the average of the xi's as above. Then the average of the x'i's wil be x'0=0, so by exercise 7 the least squares line through P'={(x'i,y'i): i=1,...,n} will have the equation Y'=c'1*X'+ c'0, and will pass through (0,y0), since c'0=y'0=y0. But then the least squares line determined by the original points P will have the equation Y=c'1*(X-x0) + c'0, which evidently passes through (x0,y0). Altenatively, we may prove this directly from the normal equations in exercise 7: A'Ax=A'y <==> / n n*x0 \ / c0 \ = / n*y0 \ \ n*x0 x'x / \ c1 / \ x'y / The 2x2 matrix is nonsingular if n>1 and there are at least two distinct values xi. We may therefore use Theorem 5.3.2 with the formula for 2x2 inverse matrices: / c0 \ = / n n*x0 \^(-1) / n*y0 \ \ c1 / \ n*x0 x'x / \ x'y / = (1/D) / x'x -n*x0 \ / n*y0 \ \ -n*x0 n / \ x'y /, where D = n*x'x-n*n*x0*x0 is the (nonzero) determinant of A'A. Multiplying this out gives c0 = n*[y0*x'x - x0*x'y]/D c1 = n*[x'y - n*x0*y0]/D The least squares line Y=c1*X+c0 therefore satisfies c1*x0+c0 = n*[(x'y - n*x0*y0)*x0 + y0*x'x - x0*x'y]/D = n*[x0*x'y - n*x0*x0*y0 + y0*x'x - x0*x'y]/D = n*[n*x0*x0*y0 + y0*x'x]/D = [n*n*x0*x0 + n*x'x]*y0/D = D*y0/D = y0 Thus, the least squares line passes through the center of mass (x0,y0). Exercise 9 of Section 5.3, p.259. a. First note that b in R(A) <==> b=Ax for some x in R^n, by the definition of R(A). But if we substitute this representation of b with P=A*inv(A'A)*A', we get Pb = A*inv(A'A)*A'(Ax) = A*inv(A'A)*(A'A)x = A*Ix = Ax = b, as claimed. b. If b is in the orthogonal complement of R(A), then b belongs to N(A') by the Fundamental Subspaces Theorem. Hence A'b=0, so Pb = A*inv(A'A)*A'b = A*inv(A'A)*0 = 0. c. See Figure 5.3.2 on p.251. Exercise 11 of Section 5.3, p.260. The matrix is partitioned, so we evaluate it in pieces; / A I \ /x\ = /b\ <==> Ax+r=b and \ 0 A'/ \r/ \0/ A'r=0 But the equation for the first coordinate follows from the definition of the residual r= b-Ax, while that of the second follows from equation 1 on page 251, namely that r belongs to N(A'). Exercise 13 of Section 5.3, p.260. We follow Application 3 on pages 257-258. The center (c1,c2) and radius r of the least squares circle through {(xi,yi):i=1,...,n} satisfy the normal equation A'Ac=b, where A=[2x1 2y1 1; ... ; 2xn 2yn 1] is nx3, c=[c1 c2 c3]' is the vector of unknowns with c3=r^2-c1^2-c2^2, and the right-hand side vector is b=[x1^2+y1^2 ... xn^2+yn^2]'. We use Matlab with the four points given in this exercise: >> x=[-1 0 1.1 2.4]; y=[-2 2.4 -4 -1.6]; A=[2*x; 2*y; 1 1 1 1]' A = -2.0000 -4.0000 1.0000 0 4.8000 1.0000 2.2000 -8.0000 1.0000 4.8000 -3.2000 1.0000 >> b = (x.*x+y.*y)' b = 5.0000 5.7600 17.2100 8.3200 >> det(A'*A) ans = 8.2692e+03 Hence we may use Theorem 5.3.2 to solve the normal equations for the coefficients c=[c1 c2 c3]': >> c=inv(A'*A)*(A'*b) c = 0.5755 -0.6426 6.6823 Now we extract the radius from the coefficients: >> r=sqrt( c(3) - c(1)^2 - c(2)^2 ) r = 2.4368 Hence the least-squares circle has center (0.5755, -0.6426) and radius r= 2.4368.