Very powerful tools for classifying data by using support vectors. Can even outperform neural networks in tasks like reading handwritten digits.
How it works is imagine we have 2 classes on a 2d graph and depending on where we put an unclassified point, we want to be able to classify it
what we did for K-nearest was check for the points that had the most similar properties to the unclassified point. What we do with a support vector is we actually train with a mathematical function to split the data
alright easy. We need to make these curves/lines/planes/curvedplanes/hyperplanes/hypercurvedplanes.
Now there are usually an infinite ammount of ways to split some lines, you can have lines like this
and they all split the classes
so the line we use is going to be the most generalized line. The one that is the farthest away from all points as possible. This is how it would look like:
in order to find that line, we use support vectors. We have a line, and we draw a parallel support vector to the nearest point of a class.
the spacing between each nearest class point to the line is the margin. If the margin for each class is the same or close enough to the same, then this line is good.
Kernels
For certain data sets, it is impossible to draw lines that seperate data.
like if I wanted to seperate blue and red, I couldn’t because its not possible for a line to segment half way.
What a kernel is is turning a dataset higher one dimension, then finding the hyperplane to seperate the data. Since its a 2d dataset, we will elevate it to the 3rd dimensions and make a plane that only touches blue and another plane that only touches red.
Soft margins
Are a way to allow outliers in the divison up to a certain point.
above is the optimal way to divide the 2 classes, however some blue is in the red class and some red is in the blue class. The soft margin we can give it is 2 to tell it we can allow up to 2 misclassifications so there is a better model. By allowing some misclassifications in the moment, you make the model better in the future.
Soft margins very softly affect data scores. They very slightly move the margin to be bigger or smaller
Programming
The module we use is from sci-py and it is from sklearn.svm import SVC
firstly, get your data training and testing data.
then we will make our classifier model which is SVC classfier.
you make the kernel argument any of the following:
-
rbf. Slow but accurate
-
poly / polynomial
-
linear. Quick but choppy
-
sigmoid
-
precomputed
we can also give the classifier a soft margin for the parameter C
then lets train it. Simple fit with the testing data
and at last we can test the model using score
decent results
compared to a K-nearest algorithm, the support vector ranks higher 4-7% better.
Unrandom states
To stop random training and testing splits, then you need to give the train_set_split a seed.
random_state = any number
string formatting like C and such
is how you format python strings. You need the f’’ with some curly braces for the datatype to format