DM825 - Introduction to Machine Learning
Sheet 5, Spring 2011 [pdf format]

Exercise 1 Linear discriminants.

  1. [(a)]Develop analytically the formulas of a generative algorithm with Gaussian likelihood for a k-way classification problem. In particular, estimate the model parameters.
  2. Derive the explicit formula of the decision boundaries in the case of two predictor variables.
  3. Implement the analysis in R using the data:
    Iris <- data.frame(cbind(iris[,c(2,3)], Sp = rep(c("s","c","v"), rep(50,3)))) train <- sample(1:150, 75) table(Iris$Sp[train])

    Plot the contour of the Gaussian distribution and linear discriminant

  4. Compare your results with those of the lda function from the package MASS in R.

    Deepening: read section 4.3.3 of B2 and inspect the outcome of lda when run on the full data with all 4 predictors, ie:

    Iris <- data.frame(cbind(iris, Sp = rep(c("s","c","v"), rep(50,3)))) z <- lda(Sp ~ Sepal.Length + Sepal.Width + Petal.Length + Petal.Width, Iris, prior = c(1,1,1)/3, subset = train) # predict(z, Iris[-train, ])$class plot(z,dimen=1) plot(z,type="density",dimen=1) plot(z,dimen=2)