Diese Webseite nutzt nur technisch notwendige Cookies.

Support Vector Machine explained > Example + Python GUIDE

Support Vector Machine explained - - Coding Lab Steffen Lippke

What the heck is a Support Vector Machine?

Complex data science is not as complex as you think.

This is your explanation to understand Support Vector Machines without prior knowledge!

Let’s go !

What is a Support Vector Machine

00 Der Neue
00 The new one

A Support Vector Machine (SVM) is a supervised machine learning model that divides individual objects into classes or sorts new objects into the classes.

Data Scientists, researchers, mathematicians and many others use the SVM for various analysis purposes such as classification. The SVM works with features that a Principal Component Analysis (PCA) has previously determined to be relevant:

Recommended tutorial on this: Principal Component Analysis Explained

Application of the Support Vector Machine

Why do I need an SVM? Data Scientists want to replace expensive, inaccurate and slow human work with efficient mathematical models(Machine Learning). Researchers use SVMs to classify ..

The applications of SVMs are unlimited as long as you want to group many objects into two or more classes and you have enough meaningful parameters available.

If you are interested in more Data Science tutorials, you can check out the tutorials in my Data Science category. SVMs are just the beginning.

Explanation with an example

Instead of bombarding you with formulas and weird descriptions, I’ll explain the SVM with a fictional example.

The ancient grizzly vs. the normal grizzly

5300 years ago, an ancient grizzly roamed the forests of Canada. The primeval grizzly weighed 3 tons, was 4 metres high and had 34 cm long teeth (This is all true, of course). Of the skeleton, often only the femur bone remains complete. This looks very similar to the femur bone of today’s grizzly.

The goal and task

When a bone is found, a Support Vector Machine is to recognise whether it is a real original grizzly or a normal grizzly from Canada on the basis of weight and length.

Example: Archaeologist – Bone determination of an ancient grizzly vs. today’s grizzly

Available characteristics from statistics

The Data Scientist need the key characteristics from the researchers to distinguish. The researchers have identified two features that can be used to make a determination in a matter of minutes:

  • Weight: continuous interval scale in grams
  • Length: continuous interval scale in millimetres
02 Zwei Achsen
02 Two axes

The weight and length of the bone should help distinguish between the ancient grizzly and today’s North American grizzly bears:

Characteristic boneAncestral grizzlyNormal grizzly
Weight800 – 1200 g600 – 900 g
Length300 – 400 mm200 – 300 mm
Roughly correct

When researchers determine the type of grizzly, they have to do a C14 sample to determine the age of the bone. C14 is an isotope of carbon that decays radioactively and is ingested by every living creature with its food. After death, the concentration of C14 decreases year by year, allowing researchers to determine the year of death.

Such a sample is expensive and time-consuming. An SVM can assist researchers by correctly assigning bones based on 95% of the parameters.

03 Klassfikation
03 Classification

Model with two features

We look at the scatter of the C14 samples performed so far on a graph with two axes.

Weight is plotted on the y-axis and length on the x-axis. The researchers have run 100 C14 samples for 100 bones, which clearly tell us which bear type is present.

The SVM needs a starting data set, a foundation, to work.

A Support Vector Machine places a separation line between the two point clouds f(x) = mx + b,which separates the clusters.

04 Trennungsgerade
04 Separation line

Challenge with Support Vector Machines

Problem: How does the SVM distinguish a baby grizzly that died from a giant grizzly from modern times?

05 Achtung Fehler
05 Attention Error

Some points of the primal grizzly are close to the point cloud of the normal grizzly. If a new C14 sample shows that the bone of a present-day giant grizzly is present, the SVM must shift its separation line.

06 Verschieben so gehts - Was ist eine Support Vector Machine Tutorial Steffen Lippke Coding Lab
06 Shifting this is how – What is a Support Vector Machine Tutorial Steffen Lippke Coding Lab

The SVM does not like outliers.

The Miss Classifications

Watch out!

07 noch mehr Fehler
07 even more errors

More C14 samples show even more mis-classifications. Why does the SVM have to shift the degrees of separation so far?

If the SVM pulls the straight line back a few centimetres, the number of future mis-classifications drops considerably. In this position, the SVM is correct in most cases.

08 Zurück
08 Back

Types of Margins

The SVM uses the following margins:

  • The Data Scientist used the “soft margin” when they are not directly separable with a line
09 Soft Margin
09 Soft Margin
  • Hard Margin are separable with a line or a hyperplane (plane in the 3D coordinate system)
10 Hard Margin
10 Hard Margin

Practical implementation with Python

You can use Python and the machine learning library scikit-learn to implement many projects in the field of data science. These include, for example :

The function shuffel randomly distributes the classified features into a test dataset and a training dataset.

from sklearn.utils import shuffle

X, Y = shuffle(X,Y)

The function Cross_validation creates the actual support vector machine.

from sklearn.cross_validation import train_test_split

x_train, x_test, y_train, y_test = train_test_split(X, Y, train_size=0.9)

If you want to start your first steps with Machine Learning, then start your journey with the ‘Programming AI yourself’ tutorial.

#Import des Bib
from sklearn import svm

# Klassifzierung mit dem Support Vector und Linaren Kernel
model = svm.SVC(kernel='linear') 

# Beginne das Traing
model.fit(X_train, y_train)

# Klassifziere die neuen Beispiele und überprüfe diese
y_pred = model.predict(X_test)

What is the kernel trick?

SVMs don’t just support linear dividers.

When there are multiple dimensions, the SVM creates a plane that separates the clusters. Alternatively, the SVM uses radial or polynomial kernels that extend separation by linear/planar planes through curved separators. An SVM cannot separate the clusters in all cases.

The SVM is only as good as the basic model. You, as a human, have to sort many data points correctly. The SVM will reproduce any error you give the model to calculate during initial loading. Look for high quality data that is available in a high quantity.

Support Vector Machine with 3+ features

An SVM can handle multiple variables.

Use multiple features (dimensions) to model other features such as width, bone density, fracture strength, etc. in the SVM.

With three features, the SVM separates the clusters in a 3D coordinate system using a plane.

A graphical representation of 5 dimensions or more is only confusing. Mathematically, models with 3+ dimensions are feasible. The more features an SVM has, the more computing power your computer needs.

Leave a Reply

Your email address will not be published. Required fields are marked *


Jeden Monat teile ich mit Mitgliedern
4 neue praxisnahe Tutorials (je 1000+ Wörter).


Trage Deine Mail, damit Du
Deine Coding + Hacking Skills erweitern kannst!