10.6 - Quadratic Discriminant Analysis

10.6 - Quadratic Discriminant Analysis

Linear Discriminant Analysis is for homogeneous variance-covariance matrices. However not all cases come from such simplified situations. Quadratic Discriminant Analysis is used for heterogeneous variance-covariance matrices:

$\Sigma_i \ne \Sigma_j$ for some $i \ne j$

Again, this allows the variance-covariance matrices to depend on the population.

Quadratic discriminant analysis calculates a Quadratic Score Function:

$s^Q_i (\mathbf{x}) = -\frac{1}{2}\log{|\mathbf{\Sigma_i}|}-\frac{1}{2}{\mathbf{(x-\mu_i)'\Sigma^{-1}_i(x - \mu_i)}}+\log{p_i}$

This is a function of population mean vectors and the variance-covariance matrices for the ith group. Similarly we will determine a separate quadratic score function for each of the groups.

This is of course a function of the unknown population mean vector for group i and the variance-covariance matrix for group i. These will have to be estimated from the training data. As before, we replace the unknown values of $\boldsymbol{\mu_i}$,$\mathbf{\Sigma_i}$, and $p_i$ by their estimates to obtain the estimated quadratic score function as shown below:

$\hat{s}^Q_i (\mathbf{x}) = -\frac{1}{2}\log{|\mathbf{S_i}|}-\frac{1}{2}{\mathbf{(x-\bar{x}_i)'S^{-1}_i(x - \bar{x}_i)}}+\log{p_i}$

All natural logs are used in this function.

Decision Rule: Our decision rule remains the same as well. We will classify the sample unit into the population that has the largest quadratic score function.

$\hat{s}^Q_i (\mathbf{x}) = -\frac{1}{2}\log{|\mathbf{S_i}|}-\frac{1}{2}{\mathbf{(x-\bar{x})'S^{-1}_i(x -\bar{x})}}+\log{p_i}$

Let's illustrate this using the Swiss Bank Notes example...

  Link ↥ Has Tooltip/Popover Toggleable Visibility