Reference no: EM1349253
Q1) Suppose we have a sample of N pairs xi, yi drawn i.i.d. from distribution characterized as given:
xi ∼ h(x), the design density
yi = f(xi) + εi, f is the regression function
εi ∼ (0, σ2) (mean zero, variance σ2)
We construct an estimator for f linear in the yi,ˆ f(x0) =XNi=1 li(x0;X)yi, where the weights li(x0;X) do not depend on the yi, but do depend on the entire training sequence of xi, denoted here by X.
(a) Show that linear regression and k-nearest-neighbor regression are members of this class of estimators. Describe explicitly the weights li(x0;X) in each of these cases.
(b) Decompose the conditional mean-squared error EY|X (f(x0) - ˆ f(x0))2 into a conditional squared bias and a conditional variance component. Like X, Y represents the entire training sequence of yi.
(c) Decompose the (unconditional) mean-squared error EY,X (f(x0) - ˆ f(x0))2 into a squared bias and a variance component.
(d) Establish a relationship between the squared biases and variances in the above two cases.