Explain ridge regression, Applied Statistics

Using log(x1), log(x2) and log(x3) as the predictors, do pair wise scatterplots of all pairs of variables (including the response) and comment (use the pairs function). Do you think that multi collinearity might be a problem with these data?

Plot the ridge trace for a grid of 50 values for the shrinkage parameter  over the range [0; 1]. Based on this plot suggest a reasonable value for . Find the estimates of the coecients for a ridge re gression with your chosen value of  (using centred and scaled predictors).

(The following question is based on Exercise 8.5 of Myers (1990), Classical and Modern Regression with Applications (Second Edition)," Duxbury).

With centred and scaled predictor variables, the ridge regression estimator for the coecients of the predictors is where y is the vector of responses, X is the design matrix for the centred and scaled predictors, is

1709_basic linear models.png

the shirnkage parameter and I denotes the identity matrix. We write n for the number of observations and k for the number of predictors. Writing biR for the ith component of bR, we will prove in this question that where 2 is the variance of the responses, and vi, i = 1,.......k are the eigenvalues of XTX. The di erent parts of the question below lead you through the proof.

735_basic linear models1.png

(a) Write XTX = QDQT for the eigenvalue decomposition of XTX, where D = diag(v1,........vk) is the diagonal matrix of eigenvalues and Q is an orthogonal matrix (QTQ = I) where the columns are the eigenvectors of XTX. Show that XTX +I = Q(D+I)QT .

2344_basic linear models2.png

where V ar(bR) denotes the covariance matrix of bR. (Hint: recall the result from basic linear models that if Y is a k  1 random vector with V ar(Y ) = V and if A is a k  k matrix and Z = AY then V ar(Z) = AV AT ).

Posted Date: 2/28/2013 12:46:51 AM | Location : United States

Related Discussions:- Explain ridge regression, Assignment Help, Ask Question on Explain ridge regression, Get Answer, Expert's Help, Explain ridge regression Discussions

Write discussion on Explain ridge regression
Your posts are moderated
Related Questions
The Quality Manager of a battery manufacturing plant reviewed the warranty records within his department and found that 4% of the low maintenance batteries produced at the plant ov

Binomial Distribution Binomial distribution  was discovered by swiss mathematician James  Bernonulli, so this distribution is called as Bernoulli distribution also, this is a d

Lifts usually have signs indicating their maximum capacity. Consider a sign in a lift that reads "maximum capacity 1400kg or 20 persons". Suppose that the weights of lift-users are

Frequency distribution A frequency distribution is a series where a number of items with similar values are put in separate groups or bunches. In other words a frequency distri

Mid year population 440000 Late fatal death          29 No. of live birth           5200 No. of infant death      423 No. of maternal death 89 No. of infant deaths i

Simple Linear Regression One calculate of the risk or volatility of an individual stock is the standard deviation of the total return (capital appreciation plus dividends) over

prepare a critical analysis of a quantitative study focusing on protection of human participants data collection data management and analysis problem statement and interpretation o

In an agricultural experiment, we wish to compare the yields of three different varieties of wheat. Call these varieties A, B and C. We have a ?eld that has been marked into a 3 *

a) What is meant by secular trend? Discuss any two methods of isolating trend values in a time series.

Difference between Correlation and Regression Analysis 1. Degree and Nature  of Relationship: Coefficient of correlation measures   the degree  of covariance  between two vari