Exercise 1
Redo exercise 1 from Sheet 1 using logistic regression (transform the
response label -1 to 0). Alternatively use logistic regression on these
data
[classification.data].
Although, as we will see, logistic regression can be impleemnted in R
via
glm, you are asked here to implement the method by
yourself. For the optimization you can reuse the gradient descent method
developed in previous exercises or you can use
optim.
Exercise 2
In exercise 3 of Sheet 2 use 1/2 of the data for training the models, 1/4 of the data to select the model (k-nearest neighbor or linear regression) and 1/4 to assess the performance of the best model selected.
Exercise 3 Bayesian prediction
In class we saw an example with binary variables. Often however we encounter discrete variables that can take on one of K possible mutually exclusive states. A way to handle this situation is to express such variables by a K-dimensional vector x→ in which one of the xk elements equals to 1 and all remaining elements equal 0. Consider a sample described by m multinomial random variables (X1, X2, …, Xm), where Xi ∼ Mult(θ) for each m, and where the Xi are assumed conditionally independent given θ. Let θ ∼ Dir(α). Now consider a random variable Xnew ∼ Mult(θ) that is assumed conditionally independent of (X1, X2, …, Xm) given θ. Compute the predictive distribution:
| p(xnew | x1 , x2 ,…, xN ,α) |
by integrating over θ.