Explain ridge regression, Applied Statistics

Using log(x1), log(x2) and log(x3) as the predictors, do pair wise scatterplots of all pairs of variables (including the response) and comment (use the pairs function). Do you think that multi collinearity might be a problem with these data?

Plot the ridge trace for a grid of 50 values for the shrinkage parameter  over the range [0; 1]. Based on this plot suggest a reasonable value for . Find the estimates of the coecients for a ridge re gression with your chosen value of  (using centred and scaled predictors).

(The following question is based on Exercise 8.5 of Myers (1990), Classical and Modern Regression with Applications (Second Edition)," Duxbury).

With centred and scaled predictor variables, the ridge regression estimator for the coecients of the predictors is where y is the vector of responses, X is the design matrix for the centred and scaled predictors, is

1709_basic linear models.png

the shirnkage parameter and I denotes the identity matrix. We write n for the number of observations and k for the number of predictors. Writing biR for the ith component of bR, we will prove in this question that where 2 is the variance of the responses, and vi, i = 1,.......k are the eigenvalues of XTX. The di erent parts of the question below lead you through the proof.

735_basic linear models1.png

(a) Write XTX = QDQT for the eigenvalue decomposition of XTX, where D = diag(v1,........vk) is the diagonal matrix of eigenvalues and Q is an orthogonal matrix (QTQ = I) where the columns are the eigenvectors of XTX. Show that XTX +I = Q(D+I)QT .

2344_basic linear models2.png

where V ar(bR) denotes the covariance matrix of bR. (Hint: recall the result from basic linear models that if Y is a k  1 random vector with V ar(Y ) = V and if A is a k  k matrix and Z = AY then V ar(Z) = AV AT ).

Posted Date: 2/28/2013 12:46:51 AM | Location : United States

Related Discussions:- Explain ridge regression, Assignment Help, Ask Question on Explain ridge regression, Get Answer, Expert's Help, Explain ridge regression Discussions

Write discussion on Explain ridge regression
Your posts are moderated
Related Questions
Question Following the general methodology used by econometricians as explained in the session for week 1 (eight steps), explain how you would proceed to determine if a good com

The manager of Pizza Hut provides a delivery service for customers who telephone in an order. The manager would like to give callers an idea of the time it will take to deliver an

You are given the differential equation dy/dx = y' = f(x, y) with initial condition y(0 ) 1 = . The following numerical method is also given: where  f n = f( x n , y n )

This question has two parts with multiple items to answer. You are a psychologist who has collected the subjective well-being scores of a number of elderly people aged 90 or abo

PCA is a linear transformation that transforms the data to a new coordinate system such that the greatest variance by any projection of the data comes to lie on the first coordinat

10. If a set of scores has a sample mean of 25 and a sample variance of 4, find the following: a. the z-score for a raw score of 31 b. the z-score for a raw score of 18 c. the raw

Meaning of Interpolation and Extrapolation Interpolation is a method of estimating the most probable  missing figure on  the basis of given data under certain assumptions. On t

1. For each of the following variables: major, graduate GPA, and height: a. Determine whether the variable is categorical or numerical. b. If the variable is numerical, deter

There exists an unclassified data set with hidden data structures in it. The task in this assignment is to perform comprehensive Cluster Analysis in order to reveal the structures