Explain ridge regression, Applied Statistics

Assignment Help:

Using log(x1), log(x2) and log(x3) as the predictors, do pair wise scatterplots of all pairs of variables (including the response) and comment (use the pairs function). Do you think that multi collinearity might be a problem with these data?

Plot the ridge trace for a grid of 50 values for the shrinkage parameter  over the range [0; 1]. Based on this plot suggest a reasonable value for . Find the estimates of the coecients for a ridge re gression with your chosen value of  (using centred and scaled predictors).

(The following question is based on Exercise 8.5 of Myers (1990), Classical and Modern Regression with Applications (Second Edition)," Duxbury).

With centred and scaled predictor variables, the ridge regression estimator for the coecients of the predictors is where y is the vector of responses, X is the design matrix for the centred and scaled predictors, is

1709_basic linear models.png

the shirnkage parameter and I denotes the identity matrix. We write n for the number of observations and k for the number of predictors. Writing biR for the ith component of bR, we will prove in this question that where 2 is the variance of the responses, and vi, i = 1,.......k are the eigenvalues of XTX. The di erent parts of the question below lead you through the proof.

735_basic linear models1.png

(a) Write XTX = QDQT for the eigenvalue decomposition of XTX, where D = diag(v1,........vk) is the diagonal matrix of eigenvalues and Q is an orthogonal matrix (QTQ = I) where the columns are the eigenvectors of XTX. Show that XTX +I = Q(D+I)QT .

2344_basic linear models2.png

where V ar(bR) denotes the covariance matrix of bR. (Hint: recall the result from basic linear models that if Y is a k  1 random vector with V ar(Y ) = V and if A is a k  k matrix and Z = AY then V ar(Z) = AV AT ).


Related Discussions:- Explain ridge regression

Probability, 2.1 Modern hotels and certain establishments make use of an el...

2.1 Modern hotels and certain establishments make use of an electronic door lock system. To open a door an electronic card is inserted into a slot. A green light indicates that the

Regression analysis, Of the 6,325 kindergarten students who participated in...

Of the 6,325 kindergarten students who participated in the study, almost half or 3,052 were eligible for a free lunch program. The categorical variable sesk (1 == free lunch, 2 = n

Testing of hypothesis, Testing of Hypothesis One objective of sampling...

Testing of Hypothesis One objective of sampling theory is Hypothesis Testing. Hypothesis testing begins by making an assumption about the population parameter. Then we gather

Standard erro, practical application of standard error

practical application of standard error

Determine the subset of variables, Agency revenues. An economic consultant ...

Agency revenues. An economic consultant was retained by a large employment agency in a metropolitan area to develop a regression model for predicting monthly agency revenues ( y ).

Perform a dimensional analysis for the quantities, Show how the Normal bin ...

Show how the Normal bin width rule can be modi ed if f is skewed or kurtotic. Examine the eff ect of bimodality. Compare your rules to Doane's (1976) extensions of Sturges' rule.

Create the venn diagram, Create the Venn diagram: A   - you work for a...

Create the Venn diagram: A   - you work for an insurance company.  80% of your company's staff is sales force and 70% of your company's sales is force is male. in your company

Poisson distribution, Poisson Distribution The poisson Distribution  wa...

Poisson Distribution The poisson Distribution  was discovered  by French mathematician simon  denis  poisson. It is a discrete probability distribution. Meaning : In bi

Properties of standard deviation, PROPERTIES   1. The value of stand...

PROPERTIES   1. The value of standard deviation remains the same if, in a series each of the observation is increased or decreased by a constant quantity. In statistical lan

Explain survey development process, Problem: A survey usually originate...

Problem: A survey usually originates when an individual or an institution is confronted with an information need and the existing data are  insufficient. Planning the questionn

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd