Multicollinearity

As the degree of correlation between the independent variables increases, the regression coefficients become less reliable. That is, although the independent variables may together explain the dependent variable, but because of multicollinearity the coefficients of the explanatory variables may be rejected. It can happen that the model may be accepted (through ANOVA and the F test), but the individual coefficients may be rejected (through the t test). This is because the interplay among the independent variables reduces the influence of the individual variables in the model. In the extreme case, if two variables are identical, then the influence of each one in the model would be reduced. Multicollinearity does not reduce the accuracy of the model (the predictive powers), but it hurts any sensitivity analysis - if we increase one explanatory variable by one unit, what happens?

This is a subject in itself, but the reader should be aware of the importance of multicollinearity.

Multiple Correlation Coefficients

So far we have come across correlation coefficients between two variables X and Y. However in the case of a multiple regression equation like

                   Y = a + b₁ X₁ + b₂ X₂

we see that Y can be correlated to both X₁ and X₂. Hence we can have a coefficient of multiple correlation which will measure the correlation between Y and both X₁ and X₂.