Support Vector Machine. This is another intimidating machine learning term (along with terms like gradient descent!) However, you can use this concept in practice without digging into the crazy maths best left to the academics.
Today, I completed a lab that runs you through a simple implementation of a support vector machine (SVM) model. This technique involves a supervised machine learning algorithm that classifies data by thrusting it into a higher dimensional space, and then finding a hyperplane that can easily group the data into separate classes.
The IBM intro to ML course I’m doing made the simple illustration below.
In this first image, we have data with just one dimension. Our data has 1 feature along the x-axis, running from -10 to 10. This dataset is “not linearly separable”. There’s no clear way of classifying the blue dots from the red dots.
However, if we can go from one dimension to a higher dimension (go from 1-D to 2-D) by finding and selecting additional features of our data, we might be able to separate the data with a line. Here’s an example of what can happen.
In the above, we have 2 features for our data that can help us predict whether a dot is red or blue. We have values -10 to 10 on the horizontal axis and we have values 0 to 100 on the vertical axis (2 dimensions). Notice that in this higher dimension, a pattern emerges that allows us to draw a straight line (of the form y = mx + c), which can help us make predictions about whether a dot is red or blue.
This thrusting of data into higher dimensions (formally known as ‘kerneling’) is key to SVM. The ‘support vectors’ are the data points that are closest to the hyperplane (a line in the example above and below.)
SVM can work even in 3 dimensions or higher. Below is a 3D example.
Note that it’s more tricky to visualise a 4 dimensions and beyond.
The code for the lab that I completed is here. Below is a preview of the data I used. This shows just two features of a cell that’s either benign or malignant.
Key takeaway:
- Once again, I’m amazed that there are tools you can use to train a machine learning model with just two key lines of code.
