Italian Wines and k-means
I had a couple of hours to kill, and given that I have some
high class friends who requested a piece on wine rather than single malts (ok, some
people will complain it’s Italian, but still…). I am trying to show how easily
a simple old fashioned algorithm like k-means (it is about 50 years old) can do a decent job of
classifying data. I am also trying to show that it is important to know what
you are trying to do and choose the right algorithm and the right way to apply
it.
The starting point is the wine dataset, a chemical analysis of wines grown in the same region in
Italy but derived from three different cultivars (you’ll have to taste and
decide which is which, if you do please let me know), and there are 13
different dimensions: alcohol, malic acid, ash, alkalinity of ash, magnesium,
total phenols, flavonoids, non-flavonoid phenols, proanthocyanins, colour
intensity, hue, od280/od315 of diluted wines and proline. The dataset contains
178 data points classed into the 3 cultivars which I split into a training and
validation dataset.
Basically I am trying to get
k-means to learn what makes each cultivar (k-means works on centroids, hence
the centroid can be thought as the ‘typical’ chemical composition of that
cultivar) from the training dataset, then apply it to the validation dataset.
The metric I will use for accuracy is simply the percentage of correctly
classified cultivars.
First, I blindly applied k-means.
We already know that the actual number of clusters in the distribution is 3, so
I set K to 3 and use all the variables.
This is what a simple
visualisation of actual distribution of the validation dataset looks like; each
node represents a wine type; they are colour coded as per the cultivar they
belong to: red for cultivar 1, green for cultivar 2 and blue for cultivar 3.
Keeping the original colour
scheme, we can see that k-means doesn’t do a very good job at predicting the
right cluster for each cultivar.
|
|
Predicted
|
|||
|
A
|
B
|
C
|
||
|
Actual
|
1
|
12
|
0
|
12
|
|
2
|
0
|
17
|
4
|
|
|
3
|
0
|
12
|
5
|
|
The accuracy of the kmeans is only
47%.
But if we decide to work off
normalised values, the picture becomes much clearer:
Even visually this looks like a
picture with clearer segments. Applying k-means on this normalised data:
K-means does a much better job at
distinguishing the cultivars based on the normalised data:
|
|
Predicted
|
|||
|
A
|
B
|
C
|
||
|
Actual
|
1
|
24
|
|
|
|
2
|
4
|
1
|
16
|
|
|
3
|
|
17
|
|
|
The accuracy of k-means jumps to
92%.
In conclusion, it pays to have a
clear idea of what the objective is and use the right approach. In this
example, it should have been obvious right from the outset that standardisation
is necessary; without it the results would be skewed by the fact that the
scales for the various measures are very different and this difference
distracts from the aim of the analysis. Also, we can see that there sometimes
is no need for very sophisticated techniques, and a quick piece of analysis can
deliver reasonable results with some fore-thought.
The original source of the data is: https://archive.ics.uci.edu/ml/datasets/Wine
