Reference no: EM132227711

Homework

Question 1. Let A ∈ R^p×p be a non-singular matrix and b ∈ R^p be a vector. Find the vector x₀ which maximizes f (x) ^def= ||Ax - b||² = (Ax - b)^T (Ax - b). [Hint: It suffices to solve the equation ∂f (x)/∂x = 0. You need not prove this, but think why.

Question 2. Let X be a n x p matrix with the full column rank, that is, rank(X) = p. Let H = X(X^T X)^-1X^T. Prove that

(a) H is an orthogonal projection, i.e. it is symmetric and idempotent.

(b) The range of H is the column space of X, i.e. C(H) = C(X).

(c) Prove that rank( H) = tr(H).

Question 3. Let A ∈ R^p×p be a symmetric matrix with eigenvalues λ₁ ≤ λ₂ ≤ .....≤ λ_p. [Hint: Use the spectral decomposition.]

(a) Find the maximum of x^T Ax when x ranges over {x : ||x|| = 1}. Note that eigenvalues are not necessarily positive.
(b) If λ₁ > 0, prove that A is positive definite, that is, x^T Ax > 0 for every x ∈ R\{0}.