SlideShare a Scribd company logo
Support Vector Machines
Machine Learning - SVM
Support Vector Machines
Divide dataset
into training and
test samples
Train the
model using
training dataset
Test using sample
training data
Performance
metrics (Finalize
the model)
Improve the
model using error
analysis
Remember the general flow of a machine learning problem:
There can be several models depending on the problem statement
We will discuss about one such model - SVM
Machine Learning - SVM
● Support vector machine is
○ Very powerful and versatile model
○ Capable of performing
■ Linear and
■ Nonlinear classification
■ Regression and
■ Outlier detection
● Well suited for small or medium sized datasets
Support Vector Machines
Machine Learning - SVM
Support Vector Machines
● In this session we will learn about
○ Linear SVM Classification
○ Nonlinear SVM Classification and
○ SVM Regression

Recommended for you

Naive Bayes Classifier | Naive Bayes Algorithm | Naive Bayes Classifier With ...
Naive Bayes Classifier | Naive Bayes Algorithm | Naive Bayes Classifier With ...Naive Bayes Classifier | Naive Bayes Algorithm | Naive Bayes Classifier With ...
Naive Bayes Classifier | Naive Bayes Algorithm | Naive Bayes Classifier With ...

This Naive Bayes Classifier tutorial presentation will introduce you to the basic concepts of Naive Bayes classifier, what is Naive Bayes and Bayes theorem, conditional probability concepts used in Bayes theorem, where is Naive Bayes classifier used, how Naive Bayes algorithm works with solved examples, advantages of Naive Bayes. By the end of this presentation, you will also implement Naive Bayes algorithm for text classification in Python. The topics covered in this Naive Bayes presentation are as follows: 1. What is Naive Bayes? 2. Naive Bayes and Machine Learning 3. Why do we need Naive Bayes? 4. Understanding Naive Bayes Classifier 5. Advantages of Naive Bayes Classifier 6. Demo - Text Classification using Naive Bayes - - - - - - - - Simplilearn’s Machine Learning course will make you an expert in Machine Learning, a form of Artificial Intelligence that automates data analysis to enable computers to learn and adapt through experience to do specific tasks without explicit programming. You will master Machine Learning concepts and techniques including supervised and unsupervised learning, mathematical and heuristic aspects, hands-on modeling to develop algorithms and prepare you for the role of Machine Learning Engineer Why learn Machine Learning? Machine Learning is rapidly being deployed in all kinds of industries, creating a huge demand for skilled professionals. The Machine Learning market size is expected to grow from USD 1.03 billion in 2016 to USD 8.81 billion by 2022, at a Compound Annual Growth Rate (CAGR) of 44.1% during the forecast period. You can gain in-depth knowledge of Machine Learning by taking our Machine Learning certification training course. With Simplilearn’s Machine Learning course, you will prepare for a career as a Machine Learning engineer as you master concepts and techniques including supervised and unsupervised learning, mathematical and heuristic aspects, and hands-on modeling to develop algorithms. Those who complete the course will be able to: 1. Master the concepts of supervised, unsupervised and reinforcement learning concepts and modeling. 2. Gain practical mastery over principles, algorithms, and applications of Machine Learning through a hands-on approach which includes working on 28 projects and one capstone project. 3. Acquire thorough knowledge of the mathematical and heuristic aspects of Machine Learning. 4. Understand the concepts and operation of support vector machines, kernel SVM, Naive Bayes, decision tree classifier, random forest classifier, logistic regression, K-nearest neighbors, K-means clustering and more. - - - - - - - -

naive bayes classifernaive bayes algorithmmachine learning algorithms
Random forest algorithm
Random forest algorithmRandom forest algorithm
Random forest algorithm

The document discusses the random forest algorithm. It introduces random forest as a supervised classification algorithm that builds multiple decision trees and merges them to provide a more accurate and stable prediction. It then provides an example pseudocode that randomly selects features to calculate the best split points to build decision trees, repeating the process to create a forest of trees. The document notes key advantages of random forest are that it avoids overfitting and can be used for both classification and regression tasks.

random forestalgorithm
Ensemble methods in machine learning
Ensemble methods in machine learningEnsemble methods in machine learning
Ensemble methods in machine learning

Ensemble methods combine multiple machine learning models to obtain better predictive performance than from any individual model. There are two main types of ensemble methods: sequential (e.g AdaBoost) where models are generated one after the other, and parallel (e.g Random Forest) where models are generated independently. Popular ensemble methods include bagging, boosting, and stacking. Bagging averages predictions from models trained on random samples of the data, while boosting focuses on correcting previous models' errors. Stacking trains a meta-model on predictions from other models to produce a final prediction.

#machinelearning #ensemblemethods #ensemble
Machine Learning - SVM
Support Vector Machines
Linear SVM Classification
Machine Learning
Human Supervision?
Supervised
Machine Learning
Unsupervised
Reinforcement
Classification
Regression
How they
generalize?
Learn Incrementally?
What is Classification?
Machine Learning - SVM
5
Not 5
What is Classification?
Identifying to which label something belongs to
Machine Learning - SVM
Examples of Classification
● Classifying emails as spam or not spam
Q. What type of classification is this?

Recommended for you

Support Vector Machine
Support Vector MachineSupport Vector Machine
Support Vector Machine

This document provides an overview of support vector machines (SVMs). It discusses how SVMs can be used to perform classification tasks by finding optimal separating hyperplanes that maximize the margin between different classes. The document outlines how SVMs solve an optimization problem to find these optimal hyperplanes using techniques like Lagrange duality, kernels, and soft margins. It also covers model selection methods like cross-validation and discusses extensions of SVMs to multi-class classification problems.

machine learningsvm
Classification Algorithm.
Classification Algorithm.Classification Algorithm.
Classification Algorithm.

This document discusses various classification algorithms including k-nearest neighbors, decision trees, naive Bayes classifier, and logistic regression. It provides examples of how each algorithm works. For k-nearest neighbors, it shows how an unknown data point would be classified based on its nearest neighbors. For decision trees, it illustrates how a tree is built by splitting the data into subsets at each node until pure subsets are reached. It also provides an example decision tree to predict whether Amit will play cricket. For naive Bayes, it gives an example of calculating the probability of cancer given a patient is a smoker.

k-nearest neighborsdecision treesnaive baye's classifier.
Support vector machine-SVM's
Support vector machine-SVM'sSupport vector machine-SVM's
Support vector machine-SVM's

In machine learning, support-vector machines (SVMs, also support-vector networks) are supervised learning models with associated learning algorithms that analyze data for classification and regression analysis

classificationsvmmachine learning
Machine Learning - SVM
Examples of Classification
● Classifying emails as spam or not spam
Q. What type of classification is this? Ans: Binary
Machine Learning - SVM
● Classifying flowers of a particular species like the Iris Dataset
Examples of Classification
Q. What type of classification is this?
Machine Learning - SVM
● Classifying flowers of a particular species like the Iris Dataset
Examples of Classification
Q. What type of classification is this? Ans: Multi-class classification
Machine Learning - SVM
● Classifying a credit card transaction as fraudulent or not
Examples of Classification

Recommended for you

Understanding Bagging and Boosting
Understanding Bagging and BoostingUnderstanding Bagging and Boosting
Understanding Bagging and Boosting

Slide explaining the distinction between bagging and boosting while understanding the bias variance trade-off. Followed by some lesser known scope of supervised learning. understanding the effect of tree split metric in deciding feature importance. Then understanding the effect of threshold on classification accuracy. Additionally, how to adjust model threshold for classification in supervised learning. Note: Limitation of Accuracy metric (baseline accuracy), alternative metrics, their use case and their advantage and limitations were briefly discussed.

baggingboostingbias and variance tradeoff
Machine Learning - Ensemble Methods
Machine Learning - Ensemble MethodsMachine Learning - Ensemble Methods
Machine Learning - Ensemble Methods

Abstract: This PDSG workshop introduces basic concepts of ensemble methods in machine learning. Concepts covered are Condercet Jury Theorem, Weak Learners, Decision Stumps, Bagging and Majority Voting. Level: Fundamental Requirements: No prior programming or statistics knowledge required.

aiartifical intelligencemachine learning
Naïve Bayes Classifier Algorithm.pptx
Naïve Bayes Classifier Algorithm.pptxNaïve Bayes Classifier Algorithm.pptx
Naïve Bayes Classifier Algorithm.pptx

The document provides an overview of the Naive Bayes algorithm for classification problems. It begins by explaining that Naive Bayes is a supervised learning algorithm based on Bayes' theorem. It then explains the key aspects of Naive Bayes: - It assumes independence between features (naive) and uses Bayes' theorem to calculate probabilities (Bayes). - Bayes' theorem is used to calculate the probability of a hypothesis given observed data. - An example demonstrates how Naive Bayes classifies weather data to predict whether to play or not play. The document concludes by discussing the advantages, disadvantages, applications, and types of Naive Bayes models, as well as providing Python code to implement a Naive Bayes classifier.

machine learningdatabasedata science
Machine Learning - SVM
Examples of Classification
● Face recognition
Q. What type of classification is this?
Machine Learning - SVM
Examples of Classification
● Face recognition
Q. What type of classification is this? Ans: Multi-label classification
Machine Learning - SVM
5
Not 5
Recap of 5 and Not 5 Classification Problem
Binary Classification Multiclass Classification
Q. What is the classifier we used for the Binary Classification?
Machine Learning - SVM
5
Not 5
Recap of 5 and Not 5 Classification Problem
Binary Classification Multiclass Classification
Q. What is the classifier we used for the Binary Classification?
Ans: SGDClassifier

Recommended for you

Loss Functions for Deep Learning - Javier Ruiz Hidalgo - UPC Barcelona 2018
Loss Functions for Deep Learning - Javier Ruiz Hidalgo - UPC Barcelona 2018Loss Functions for Deep Learning - Javier Ruiz Hidalgo - UPC Barcelona 2018
Loss Functions for Deep Learning - Javier Ruiz Hidalgo - UPC Barcelona 2018

https://telecombcn-dl.github.io/2018-dlai/ Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks or Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles of deep learning from both an algorithmic and computational perspectives.

deep learningneural networksloss function
Supervised and Unsupervised Learning In Machine Learning | Machine Learning T...
Supervised and Unsupervised Learning In Machine Learning | Machine Learning T...Supervised and Unsupervised Learning In Machine Learning | Machine Learning T...
Supervised and Unsupervised Learning In Machine Learning | Machine Learning T...

This document provides an overview of machine learning, including: - Machine learning allows computers to learn from data without being explicitly programmed, through processes like analyzing data, training models on past data, and making predictions. - The main types of machine learning are supervised learning, which uses labeled training data to predict outputs, and unsupervised learning, which finds patterns in unlabeled data. - Common supervised learning tasks include classification (like spam filtering) and regression (like weather prediction). Unsupervised learning includes clustering, like customer segmentation, and association, like market basket analysis. - Supervised and unsupervised learning are used in many areas like risk assessment, image classification, fraud detection, customer analytics, and more

supervised and unsupervised learningsupervised and unsupervised learning in machine lewhat is supervised learning
An Introduction to Supervised Machine Learning and Pattern Classification: Th...
An Introduction to Supervised Machine Learning and Pattern Classification: Th...An Introduction to Supervised Machine Learning and Pattern Classification: Th...
An Introduction to Supervised Machine Learning and Pattern Classification: Th...

The document provides an introduction to supervised machine learning and pattern classification. It begins with an overview of the speaker's background and research interests. Key concepts covered include definitions of machine learning, examples of machine learning applications, and the differences between supervised, unsupervised, and reinforcement learning. The rest of the document outlines the typical workflow for a supervised learning problem, including data collection and preprocessing, model training and evaluation, and model selection. Common classification algorithms like decision trees, naive Bayes, and support vector machines are briefly explained. The presentation concludes with discussions around choosing the right algorithm and avoiding overfitting.

machine learningdata miningdata science
Machine Learning - SVM
5
Not 5
Recap of 5 and Not 5 Classification Problem
Binary Classification Multiclass Classification
Q. What is the classifier we used for the Multiclass Classification?
Machine Learning - SVM
5
Not 5
Recap of 5 and Not 5 Classification Problem
Binary Classification Multiclass Classification
Q. What is the classifier we used for the Multiclass Classification?
Ans: SGDClassifier - OvO and OvA
Machine Learning - SVM
What is Linear Classification?
Machine Learning - SVM
What is Linear Classification?
● The two classes can be separated easily with a ‘straight’ line
‘Straight’ is the keyword. It means linear classification.

Recommended for you

Support Vector machine
Support Vector machineSupport Vector machine
Support Vector machine

Support Vector Machine (SVM) is a supervised machine learning algorithm that can be used for both classification and regression analysis. It works by finding a hyperplane in an N-dimensional space that distinctly classifies the data points. SVM selects the hyperplane that has the largest distance to the nearest training data points of any class, since larger the margin lower the generalization error of the classifier. SVM can efficiently perform nonlinear classification by implicitly mapping their inputs into high-dimensional feature spaces.

svm
Support Vector machine
Support Vector machineSupport Vector machine
Support Vector machine

Support vector machines (SVMs) are a type of supervised machine learning algorithm used for classification and regression. SVMs learn by finding the optimal separating hyperplane that maximizes the margin between two classes of objects. SVMs can efficiently perform nonlinear classification using kernel methods. Key features of SVMs include their ability to handle high dimensional data, use different kernel functions for nonlinear classification, and avoid overfitting.

svmdata mining
Ensemble learning
Ensemble learningEnsemble learning
Ensemble learning

ensemble methods use multiple learning algorithms to obtain better predictive performance than could be obtained from any of the constituent learning algorithms alone. An ensemble is itself a supervised learning algorithm, because it can be trained and then used to make predictions. The trained ensemble, therefore, represents a single hypothesis. This hypothesis, however, is not necessarily contained within the hypothesis space of the models from which it is built.

ensemble learningensemblelearning
Machine Learning - SVM
What is Linear Classification?
● For example: IRIS Dataset
○ Features: Sepal Length, Petal Length
○ Class: Iris Virginica OR Iris Versicolor OR Iris Setosa
Machine Learning - SVM
What is Linear Classification?
Sepal Length Petal Length Flower Type
1.212 4.1 Iris-Versicolor
0.5 1.545 Iris-Setosa
0.122 1.64 Iris-Setosa
0.2343 ... Iris-Setosa
0.1 ... Iris-Setosa
1.32 ... Iris-Versicolor
Machine Learning - SVM
What is Linear Classification?
● For the above IRIS Dataset, what is the type of Machine Learning model?
○ Classification or Regression?
■ Ans:
Machine Learning - SVM
What is Linear Classification?
● For the above IRIS Dataset, what is the type of Machine Learning model?
○ Classification or Regression?
■ Ans: Classification

Recommended for you

Logistic regression in Machine Learning
Logistic regression in Machine LearningLogistic regression in Machine Learning
Logistic regression in Machine Learning

Logistic regression is a predictive analysis algorithm that can be used for classification problems. It estimates the probabilities of different classes using the logistic function, which outputs values between 0 and 1. Logistic regression transforms its output using the sigmoid function to return a probability value. It is used for problems like email spam detection, fraud detection, and tumor classification. The independent variables should be independent of each other and the dependent variable must be categorical. Gradient descent is used to minimize the loss function and optimize the model parameters during training.

logistic regression in machine learninglogistic regressionclassification
Hyperparameter Optimization for Machine Learning
Hyperparameter Optimization for Machine LearningHyperparameter Optimization for Machine Learning
Hyperparameter Optimization for Machine Learning

▸ Machine Learning / Deep Learning models require to set the value of many hyperparameters ▸ Common examples: regularization coefficients, dropout rate, or number of neurons per layer in a Neural Network ▸ Instead of relying on some "expert advice", this presentation shows how to automatically find optimal hyperparameters ▸ Exhaustive Search, Monte Carlo Search, Bayesian Optimization, and Evolutionary Algorithms are explained with concrete examples

machine learningoptimization
Classification
ClassificationClassification
Classification

The document discusses machine learning classification using the MNIST dataset of handwritten digits. It begins by defining classification and providing examples. It then describes the MNIST dataset and how it is fetched in scikit-learn. The document outlines the steps of classification which include dividing the data into training and test sets, training a classifier on the training set, testing it on the test set, and evaluating performance. It specifically trains a stochastic gradient descent (SGD) classifier on the MNIST data. The performance is evaluated using cross validation accuracy, confusion matrix, and metrics like precision and recall.

machine learning classificationmachine learning classification algorithmsclassification
Machine Learning - SVM
What is Linear Classification?
● What is the type of Supervised Machine Learning model?
○ Classification or Regression?
■ Ans: Classification
○ What type of classification?
■ Binary Classification
■ Multi-label Classification
■ Multi-output Classification
■ Multi-class Classification
Machine Learning - SVM
What is Linear Classification?
● What is the type of Supervised Machine Learning model?
○ Classification or Regression?
■ Ans: Classification
○ What type of classification?
■ Binary Classification
■ Multi-label Classification
■ Multi-output Classification
■ Ans: Multi-class Classification
Machine Learning - SVM
What is Linear Classification?
● For the IRIS dataset above:
○ Number of features?
■ Ans:
○ Number of classes?
■ Ans:
Machine Learning - SVM
What is Linear Classification?
● For the IRIS dataset above:
○ Number of features?
■ Ans: 2
○ Number of classes?
■ Ans: 3

Recommended for you

Machine learning- key concepts
Machine learning- key conceptsMachine learning- key concepts
Machine learning- key concepts

This document outlines an introductory machine learning course, covering key concepts, applications, and types of machine learning like supervised and unsupervised learning. It discusses techniques like linear regression, classification, and handling overfitting. The course will include tutorials on sentiment analysis, spam filtering, stock prediction, image recognition and recommendation engines using Python and Scala. Later classes cover machine learning at scale using tools like Spark MLLib.

Recommender Systems from A to Z – Model Training
Recommender Systems from A to Z – Model TrainingRecommender Systems from A to Z – Model Training
Recommender Systems from A to Z – Model Training

This second meetup will be about training different models for our recommender system. We will review the simple models we can build as a baseline. After that, we will present the recommender system as an optimization problem and discuss different training losses. We will mention linear models and matrix factorization techniques. We will end the presentation with a simple introduction to non-linear models and deep learning.

recommenationmachine learningoptimization
ML Study Jams - Session 3.pptx
ML Study Jams - Session 3.pptxML Study Jams - Session 3.pptx
ML Study Jams - Session 3.pptx

This document summarizes a session on classification models. It discusses classifying data by learning classification rules from labeled input data. Specifically, it covers k-nearest neighbors classification, where the target point is assigned the majority class of its k nearest neighbors. It also discusses extending linear regression to classification by using it to generate real-valued outputs, then thresholding those values to classify instances into classes 0 or 1. However, it notes that logistic regression is a better approach, using a sigmoid function to convert real-valued outputs to a probability between 0 and 1 before thresholding for classification. The document highlights key considerations for classification like dealing with multiple classes and class imbalance. It also discusses evaluating classifiers, ensemble methods, and overfitting/under

machine learninggdscbangalore
Machine Learning - SVM
What is Linear Classification?
● When we plot the two features on the graph and label it by color
○ The classes can be divided using a straight line
○ Hence, linear classification
Straight Line (Linear Classification)
Machine Learning - SVM
Linear SVM Classification
Linear SVM
Classification
Nonlinear SVM
Classification
SVM
Regression
Bad model
versus good-
model (Large
Margin)
Classification
Soft Margin
versus Hard-
margin
Classification
Machine Learning - SVM
Linear SVM Classification - Large Margin
Pink and red decision boundaries are
very close to the instances - bad
model
Decision Boundary as far away from
training instances - good model
Large Margin Classification
Widest possible street
Machine Learning - SVM
Linear SVM Classification - Large Margin
May not perform well on new
instances
Adding training instances may not
affect the decision boundary
Large Margin Classification

Recommended for you

Text Classification with Lucene/Solr, Apache Hadoop and LibSVM
Text Classification with Lucene/Solr, Apache Hadoop and LibSVMText Classification with Lucene/Solr, Apache Hadoop and LibSVM
Text Classification with Lucene/Solr, Apache Hadoop and LibSVM

In this session we will show how to build a text classifier using the Apache Lucene/Solr with libSVM libraries. We classify our corpus of job offers into a number of predefined categories. Each indexed document (a job offer) then belongs to zero, one or more categories. Known machine learning techniques for text classification include naïve bayes model, logistic regression, neural network, support vector machine (SVM), etc. We use Lucene/Solr to construct the features vector. Then we use the libsvm library known as the reference implementation of the SVM model to classify the document. We construct as many one-vs-all svm classifiers as there are classes in our setting, then using the Hadoop MapReduce Framework we reconcile the result of our classifiers. The end result is a scalable multi-class classifier. Finally we outline how the classifier is used to enrich basic solr keyword search.

lucene revolutionlucene/solr revolutionsolr
Scalable machine learning
Scalable machine learningScalable machine learning
Scalable machine learning

This document discusses scaling machine learning algorithms to large datasets. It introduces machine learning and linear regression, explaining how to represent data with features and labels. Linear regression finds the line of best fit by minimizing a squared loss function. Computing the closed-form solution is inefficient for large datasets due to expensive matrix operations. Gradient descent is introduced as an iterative alternative that is easily parallelized. It works by taking steps in the direction of steepest descent until converging to a solution.

big datamachine learningcloud computing
OM-DS-Fall2022-Session10-Support vector machine.pdf
OM-DS-Fall2022-Session10-Support vector machine.pdfOM-DS-Fall2022-Session10-Support vector machine.pdf
OM-DS-Fall2022-Session10-Support vector machine.pdf

This document provides an overview of support vector machines (SVM) including: 1) Maximal margin classifiers which find the optimal separating hyperplane with the maximum margin between classes. Support vectors are the data points that determine the hyperplane. 2) Support vector classifiers which allow some misclassified data points by introducing slack variables. This makes the classifier more robust. 3) SVM can handle non-linear decision boundaries using kernel methods to map data into higher dimensional feature spaces where a linear separator can be found. Common kernels include linear, polynomial and radial basis function kernels. 4) Multi-class classification with SVM can be done with one-vs-one or one-vs-all approaches. 5) SVM

Machine Learning - SVM
Linear SVM Classification - Large Margin
Large Margin Classification
Widest possible street
Support Vectors
● What are Support vectors?
○ Vectors or the training set located closest to the classifier OR
○ Vectors or the training sets located at the edge of the street
Machine Learning - SVM
Switch to Notebook
Machine Learning - SVM
Linear SVM Classification - Example 1
X1 X2 Label
1 50 0
5 20 0
3 80 1
5 60 1
● Without Scaling
● Training dataset
Machine Learning - SVM
Linear SVM Classification - Example 1
● Model the classifier, plot the points and the classifier
>>> Xs = np.array([[1, 50], [5, 20], [3, 80], [5,
60]]).astype(np.float64)
>>> ys = np.array([0, 0, 1, 1])
>>> svm_clf = SVC(kernel="linear", C=100)
>>> svm_clf.fit(Xs, ys)
>>> plt.plot(Xs[:, 0][ys==1], Xs[:, 1][ys==1], "bo")
>>> plt.plot(Xs[:, 0][ys==0], Xs[:, 1][ys==0], "ms")
>>> plot_svc_decision_boundary(svm_clf, 0, 6)
>>> plt.xlabel("$x_0$", fontsize=20)
>>> plt.ylabel("$x_1$ ", fontsize=20, rotation=0)
>>> plt.title("Unscaled", fontsize=16)
>>> plt.axis([0, 6, 0, 90])

Recommended for you

Machine Learning: An introduction โดย รศ.ดร.สุรพงค์ เอื้อวัฒนามงคล
Machine Learning: An introduction โดย รศ.ดร.สุรพงค์  เอื้อวัฒนามงคลMachine Learning: An introduction โดย รศ.ดร.สุรพงค์  เอื้อวัฒนามงคล
Machine Learning: An introduction โดย รศ.ดร.สุรพงค์ เอื้อวัฒนามงคล

Machine Learning: An introduction โดย รศ.ดร.สุรพงค์ เอื้อวัฒนามงคล ในงาน THE FIRST NIDA BUSINESS ANALYTICS AND DATA SCIENCES CONTEST/CONFERENCE

data sciencesmachine learningcomputer sciences
Application of Machine Learning in Agriculture
Application of Machine  Learning in AgricultureApplication of Machine  Learning in Agriculture
Application of Machine Learning in Agriculture

With the growing trend of machine learning, it is needless to say how machine learning can help reap benefits in agriculture. It will be boon for the farmer welfare.

uasdstatisticsanalytics
Understanding computer vision with Deep Learning
Understanding computer vision with Deep LearningUnderstanding computer vision with Deep Learning
Understanding computer vision with Deep Learning

Computer vision is a branch of computer science which deals with recognising objects, people and identifying patterns in visuals. It is basically analogous to the vision of an animal. Topics covered: 1. Overview of Machine Learning 2. Basics of Deep Learning 3. What is computer vision and its use-cases? 4. Various algorithms used in Computer Vision (mostly CNN) 5. Live hands-on demo of either Auto Cameraman or Face recognition system 6. What next?

computer visionmachine learningbig data
Machine Learning - SVM
Linear SVM Classification - Example 1
● Model the classifier, plot the points and the classifier
Machine Learning - SVM
Linear SVM Classification - Example 1
● What is the problem?
Machine Learning - SVM
Linear SVM Classification - Example 1
● What is the problem?
○ X0 ranges from 0 to 6 while
○ X1 ranges from 20 to 80
● Solution: Feature Scaling
Machine Learning Project
Feature Scaling
Feature Scaling
Quick Revision
from
Preparing the data for ML Algorithms in End-to-End Project

Recommended for you

Understanding computer vision with Deep Learning
Understanding computer vision with Deep LearningUnderstanding computer vision with Deep Learning
Understanding computer vision with Deep Learning

Topics covered in the Webinar 1. Overview of Machine Learning 2. Basics of Deep Learning 3. What is computer vision and its use-cases? 4. Various algorithms used in Computer Vision (mostly CNN) 5. Live hands-on demo of either Auto Cameraman or Face recognition system 6. What next? Presented by Sandeep Giri www.cloudxlab.com

deep learningcomputer visionmachine learning
Understanding computer vision with Deep Learning
Understanding computer vision with Deep LearningUnderstanding computer vision with Deep Learning
Understanding computer vision with Deep Learning

Computer vision is a branch of computer science which deals with recognising objects, people and identifying patterns in visuals. It is basically analogous to the vision of an animal. Topics covered: 1. Overview of Machine Learning 2. Basics of Deep Learning 3. What is computer vision and its use-cases? 4. Various algorithms used in Computer Vision (mostly CNN) 5. Live hands-on demo of either Auto Cameraman or Face recognition system 6. What next?

machine learningartificial intelligencecomputer vision
3ml.pdf
3ml.pdf3ml.pdf
3ml.pdf

This document discusses machine learning algorithms for classification problems, specifically logistic regression. It explains that logistic regression predicts the probability of a binary outcome using a sigmoid function. Unlike linear regression, which is used for continuous outputs, logistic regression is used for classification problems where the output is discrete/categorical. It describes how logistic regression learns model parameters through gradient descent optimization of a likelihood function to minimize error. Regularization techniques can also be used to address overfitting issues that may arise from limited training data.

ml slides
Machine Learning Project
Feature Scaling
● ML algorithms do not perform well
○ When the input numerical attributes have very different scales
● Feature Scaling is one of the most important
○ Transformation we need to apply to our data
● Two ways to make sure all attributes have same scale
○ Min-max scaling
○ Standardization
Machine Learning Project
Feature Scaling
Min-max Scaling
● Also known as Normalization
● Normalized values are in the range of [0, 1]
Machine Learning Project
Feature Scaling
Min-max Scaling
● Also known as Normalization
● Normalized values are in the range of [0, 1]
Original Value
Normalized Value
Machine Learning Project
Feature Scaling
Min-max Scaling - Example
# Creating DataFrame first
>>> import pandas as pd
>>> s1 = pd.Series([1, 2, 3, 4, 5, 6], index=(range(6)))
>>> s2 = pd.Series([10, 9, 8, 7, 6, 5], index=(range(6)))
>>> df = pd.DataFrame(s1, columns=['s1'])
>>> df['s2'] = s2
>>> df

Recommended for you

Support Vector Machines USING MACHINE LEARNING HOW IT WORKS
Support Vector Machines USING MACHINE LEARNING HOW IT WORKSSupport Vector Machines USING MACHINE LEARNING HOW IT WORKS
Support Vector Machines USING MACHINE LEARNING HOW IT WORKS

This document discusses support vector machines (SVM), a supervised machine learning algorithm used for classification and regression. It explains that SVM finds the optimal boundary, known as a hyperplane, that separates classes with the maximum margin. When data is not linearly separable, kernel functions can transform the data into a higher-dimensional space to make it separable. The document discusses SVM for both linearly separable and non-separable data, kernel functions, hyperparameters, and approaches for multiclass classification like one-vs-one and one-vs-all.

svmsupervisory learningunsupervisory learning
background.pptx
background.pptxbackground.pptx
background.pptx

Basic machine learning background with Python scikit-learn This document provides an overview of machine learning and the Python scikit-learn library. It introduces key machine learning concepts like classification, linear models, support vector machines, decision trees, bagging, boosting, and clustering. It also demonstrates how to perform tasks like SVM classification, decision tree modeling, random forest, principal component analysis, and k-means clustering using scikit-learn. The document concludes that scikit-learn can handle large datasets and recommends Keras for deep learning.

Machine Learning : why we should know and how it works
Machine Learning : why we should know and how it worksMachine Learning : why we should know and how it works
Machine Learning : why we should know and how it works

This document provides an overview of machine learning, including: - An introduction to machine learning and why it is important. - The main types of machine learning algorithms: supervised learning, unsupervised learning, and deep neural networks. - Examples of how machine learning algorithms work, such as logistic regression, support vector machines, and k-means clustering. - How machine learning is being applied in various industries like healthcare, commerce, and more.

machine learningaidnn
Machine Learning Project
Feature Scaling
Min-max Scaling - Example
# Use Scikit-Learn minmax_scaling
>>> from mlxtend.preprocessing import minmax_scaling
>>> minmax_scaling(df, columns=['s1', 's2'])
Original Scaled (In range of
0 and 1)
Machine Learning Project
Feature Scaling
Standardization
● In Machine Learning, we handle various types of data like
○ Audio signals and
○ Pixel values for image data
○ And this data can include multiple dimensions
Machine Learning Project
Feature Scaling
Standardization
We scale the values by calculating
○ How many standard deviation is the value away from the mean
SAT scores ~ N(mean = 1500, SD = 300)
ACT scores ~ N(mean = 21, SD = 5)
Machine Learning Project
Feature Scaling
Standardization
● The general method of calculation
○ Calculate distribution mean and standard deviation for each feature
○ Subtract the mean from each feature
○ Divide the result from previous step of each feature by its standard
deviation
Standardized Value

Recommended for you

Online advertising and large scale model fitting
Online advertising and large scale model fittingOnline advertising and large scale model fitting
Online advertising and large scale model fitting

This document discusses online advertising and techniques for fitting large-scale models to advertising data. It outlines batch and online algorithms for logistic regression, including parallelizing existing batch algorithms and stochastic gradient descent. The document also discusses using alternating direction method of multipliers and follow the proximal regularized leader to fit models to large datasets across multiple machines. It provides examples of how major companies like LinkedIn and Facebook implement hybrid online-batch algorithms at large scale.

Keynote at IWLS 2017
Keynote at IWLS 2017Keynote at IWLS 2017
Keynote at IWLS 2017

Machine learning techniques can be applied in formal verification in several ways: 1) To enhance current formal verification tools by automating tasks like debugging, specification mining, and theorem proving. 2) To enable the development of new formal verification tools by applying machine learning to problems like SAT solving, model checking, and property checking. 3) Specific applications include using machine learning for debugging and root cause identification, learning specifications from runtime traces, aiding theorem proving by selecting heuristics, and tuning SAT solver parameters and selection.

machine learningformal verificationdeep learning
Hadoop France meetup Feb2016 : recommendations with spark
Hadoop France meetup  Feb2016 : recommendations with sparkHadoop France meetup  Feb2016 : recommendations with spark
Hadoop France meetup Feb2016 : recommendations with spark

Système de recommandations de produits sur un site marchand par Koby KARP, Data Scientist (Equancy) & Hervé MIGNOT, Partner at Equancy La recommandation reste un outil clé pour la personnalisation des sites marchands et le sujet est loin d’être épuisé. La prise en compte de la particularité d’un marché peut nécessité d’adapter le traitement et les algorithmes utilisés. Après une revue des techniques de recommandations, nous présenterons la démarche spécifique que nous avons adopté. Le système a été développé sous Spark pour la préparation des données et le calcul des modèles de recommandations. Une API simple et son service ont été développé pour délivrer les recommandations aux applications clientes.

recommandationhadoopmeetup
Machine Learning Project
Feature Scaling
Standardization
● In Standardization, features are rescaled
● So that output will have the properties of
● Standard normal distribution with
○ Zero mean and
○ Unit variance
Mean Standard Deviation
Machine Learning Project
Feature Scaling
Standardization
● Scikit-Learn provides
○ StandardScaler class for standardization
Machine Learning Project
Feature Scaling
Which One to Use?
● Min-max scales in the range of [0,1]
● Standardization does not bound values to a specific range
○ It may be problem for some algorithms
○ Example- Neural networks expect an input value ranging from 0 to 1
● We’ll learn more use cases as we proceed in the course
Machine Learning Project
Feature Scaling
Back to original Example 1

Recommended for you

MLEARN 210 B Autumn 2018: Lecture 1
MLEARN 210 B Autumn 2018: Lecture 1MLEARN 210 B Autumn 2018: Lecture 1
MLEARN 210 B Autumn 2018: Lecture 1

This document provides an introduction and overview for a course on machine learning. It outlines the course structure, assignments, and expectations. The course will cover topics including linear regression, classification, model selection, and dimensionality reduction. It will teach students how to analyze data, preprocess it, extract features, train models, and evaluate model performance. The goal is for students to understand core machine learning algorithms and concepts. Required materials include an introduction to statistical learning textbook.

Deep Learning Overview
Deep Learning OverviewDeep Learning Overview
Deep Learning Overview

This document provides an agenda for an introduction to deep learning presentation. It begins with an introduction to basic AI, machine learning, and deep learning terms. It then briefly discusses use cases of deep learning. The document outlines how to approach a deep learning problem, including which tools and algorithms to use. It concludes with a question and answer section.

cloudxlabdeep learningmachine learning
Recurrent Neural Networks
Recurrent Neural NetworksRecurrent Neural Networks
Recurrent Neural Networks

This document discusses recurrent neural networks (RNNs) and their applications. It begins by explaining that RNNs can process input sequences of arbitrary lengths, unlike other neural networks. It then provides examples of RNN applications, such as predicting time series data, autonomous driving, natural language processing, and music generation. The document goes on to describe the fundamental concepts of RNNs, including recurrent neurons, memory cells, and different types of RNN architectures for processing input/output sequences. It concludes by demonstrating how to implement basic RNNs using TensorFlow's static_rnn function.

recurrent neural networkrecurrent neural network tutorialrecurrent neural network tensorflow
Machine Learning - SVM
Linear SVM Classification - Example 2
x1 x2 Label x1 (Scaled) x2 (Scaled)
1 50 0 -1.5 -0.1154
5 20 0 0.9 -1.5011107
3 80 1 -0.3 1.27017
5 60 1 0.9 0.3464
Mean (m1) =
3.5
Std Dev (s1)
= 1.65
Mean (m2) =
52.5
Std Dev (s2)
= 21.65
(x-m1)/s1 (x-s2)/m2
● With Scaling
Machine Learning - SVM
Linear SVM Classification - Example 2
● Scaling of features
X_new = (x-m1)/s1
● What kind of scaling is this?
○ Normalization
○ Standardization
Machine Learning - SVM
Linear SVM Classification - Example 2
● Scaling of features
X_new = (x-m1)/s1
● What kind of scaling is this?
○ Normalization
○ Standardization
Machine Learning - SVM
Linear SVM Classification - Example 2
● Scaling of features
X_new = (x-m1)/s1
● What kind of scaling is this?
○ Normalization
○ Standardization
● What is the module available in scikit_learn to perform standardization?

Recommended for you

Natural Language Processing
Natural Language ProcessingNatural Language Processing
Natural Language Processing

Natural Language Processing (NLP) is a field of artificial intelligence that deals with interactions between computers and human languages. NLP aims to program computers to process and analyze large amounts of natural language data. Some common NLP tasks include speech recognition, text classification, machine translation, question answering, and more. Popular NLP tools include Stanford CoreNLP, NLTK, OpenNLP, and TextBlob. Vectorization is commonly used to represent text in a way that can be used for machine learning algorithms like calculating text similarity. Tf-idf is a common technique used to weigh words based on their frequency and importance.

cloudxlabdeep learningmachine learning
Naive Bayes
Naive BayesNaive Bayes
Naive Bayes

- Naive Bayes is a classification technique based on Bayes' theorem that uses "naive" independence assumptions. It is easy to build and can perform well even with large datasets. - It works by calculating the posterior probability for each class given predictor values using the Bayes theorem and independence assumptions between predictors. The class with the highest posterior probability is predicted. - It is commonly used for text classification, spam filtering, and sentiment analysis due to its fast performance and high success rates compared to other algorithms.

cloudxlabdeep learningmachine learning
Autoencoders
AutoencodersAutoencoders
Autoencoders

An autoencoder is an artificial neural network that is trained to copy its input to its output. It consists of an encoder that compresses the input into a lower-dimensional latent-space encoding, and a decoder that reconstructs the output from this encoding. Autoencoders are useful for dimensionality reduction, feature learning, and generative modeling. When constrained by limiting the latent space or adding noise, autoencoders are forced to learn efficient representations of the input data. For example, a linear autoencoder trained with mean squared error performs principal component analysis.

autoencoderdenoising autoencodervariational autoencoder
Machine Learning - SVM
Linear SVM Classification - Example 2
● Scaling of features
X_new = (x-m1)/s1
● What kind of scaling is this?
○ Normalization
○ Standardization
● What is the module available in scikit_learn to perform standardization?
○ Answer: StandardScalar
Machine Learning - SVM
Linear SVM Classification - Example 2
● Scaling the input training data
>>> from sklearn.preprocessing import StandardScaler
>>> scaler = StandardScaler()
>>> X_scaled = scaler.fit_transform(Xs)
>>> print(X_scaled)
[[-1.50755672 -0.11547005]
[ 0.90453403 -1.5011107 ]
[-0.30151134 1.27017059]
[ 0.90453403 0.34641016]]
Machine Learning - SVM
Linear SVM Classification - Example 2
● Building the model, plotting the decision boundary and the training points
>>> svm_clf.fit(X_scaled, ys)
>>> plt.plot(X_scaled[:, 0][ys==1], X_scaled[:, 1][ys==1], "bo")
>>> plt.plot(X_scaled[:, 0][ys==0], X_scaled[:, 1][ys==0], "ms")
>>> plot_svc_decision_boundary(svm_clf, -2, 2)
>>> plt.ylabel("$x_{1scaled}$", fontsize=20)
>>> plt.xlabel("$x_{0scaled}$", fontsize=20)
>>> plt.title("Scaled", fontsize=16)
>>> plt.axis([-2, 2, -2, 2])
Machine Learning - SVM
Linear SVM Classification - Example 2
● Output decision boundary for a scaled training data

Recommended for you

Training Deep Neural Nets
Training Deep Neural NetsTraining Deep Neural Nets
Training Deep Neural Nets

The document discusses challenges in training deep neural networks and solutions to those challenges. Training deep neural networks with many layers and parameters can be slow and prone to overfitting. A key challenge is the vanishing gradient problem, where the gradients shrink exponentially small as they propagate through many layers, making earlier layers very slow to train. Solutions include using initialization techniques like He initialization and activation functions like ReLU and leaky ReLU that do not saturate, preventing gradients from vanishing. Later improvements include the ELU activation function.

vanishing gradientvanishing gradient problemxavier initialization
Reinforcement Learning
Reinforcement LearningReinforcement Learning
Reinforcement Learning

( Machine Learning & Deep Learning Specialization Training: https://goo.gl/5u2RiS ) This CloudxLab Reinforcement Learning tutorial helps you to understand Reinforcement Learning in detail. Below are the topics covered in this tutorial: 1) What is Reinforcement? 2) Reinforcement Learning an Introduction 3) Reinforcement Learning Example 4) Learning to Optimize Rewards 5) Policy Search - Brute Force Approach, Genetic Algorithms and Optimization Techniques 6) OpenAI Gym 7) The Credit Assignment Problem 8) Inverse Reinforcement Learning 9) Playing Atari with Deep Reinforcement Learning 10) Policy Gradients 11) Markov Decision Processes

reinforcement learningopenai gymwhat is reinforcement
Apache Spark - Key Value RDD - Transformations | Big Data Hadoop Spark Tutori...
Apache Spark - Key Value RDD - Transformations | Big Data Hadoop Spark Tutori...Apache Spark - Key Value RDD - Transformations | Big Data Hadoop Spark Tutori...
Apache Spark - Key Value RDD - Transformations | Big Data Hadoop Spark Tutori...

The document provides information about key-value RDD transformations and actions in Spark. It defines transformations like keys(), values(), groupByKey(), combineByKey(), sortByKey(), subtractByKey(), join(), leftOuterJoin(), rightOuterJoin(), and cogroup(). It also defines actions like countByKey() and lookup() that can be performed on pair RDDs. Examples are given showing how to use these transformations and actions to manipulate key-value RDDs.

spark transformationsspark combinebykeyspark join
Machine Learning - SVM
Linear SVM Classification
● Unscaled vs Scaled comparison
X0 X1 Label X0 (Scaled) X1 (Scaled)
1 50 0 -1.5 -0.1154
5 20 0 0.9 -1.5011107
3 80 1 -0.3 1.27017
5 60 1 0.9 0.3464
Mean (m1) =
3.5
Std Dev (s1)
= 1.65
Mean (m2) =
52.5
Std Dev (s2)
= 21.65
(x-m1)/s1 (x-m2)/s2
Machine Learning - SVM
Linear SVM Classification
● Unscaled vs Scaled
Widestpossiblestreet
Machine Learning - SVM
Linear SVM Classification
● Unscaled vs Scaled
○ Linear SVM sensitive to scaling
○ Feature scaling an important part of data preparation
■ Normalization
■ Standardization
○ Scaled features produce better result for the above example
Machine Learning - SVM
Switch to Notebook

Recommended for you

Advanced Spark Programming - Part 2 | Big Data Hadoop Spark Tutorial | CloudxLab
Advanced Spark Programming - Part 2 | Big Data Hadoop Spark Tutorial | CloudxLabAdvanced Spark Programming - Part 2 | Big Data Hadoop Spark Tutorial | CloudxLab
Advanced Spark Programming - Part 2 | Big Data Hadoop Spark Tutorial | CloudxLab

Big Data with Hadoop & Spark Training: http://bit.ly/2kyRTuW This CloudxLab Advanced Spark Programming tutorial helps you to understand Advanced Spark Programming in detail. Below are the topics covered in this slide: 1) Shared Variables - Accumulators & Broadcast Variables 2) Accumulators and Fault Tolerance 3) Custom Accumulators - Version 1.x & Version 2.x 4) Examples of Broadcast Variables 5) Key Performance Considerations - Level of Parallelism 6) Serialization Format - Kryo 7) Memory Management 8) Hardware Provisioning

spark accumulatoraccumulators in sparkspark broadcast variable
Apache Spark - Dataframes & Spark SQL - Part 2 | Big Data Hadoop Spark Tutori...
Apache Spark - Dataframes & Spark SQL - Part 2 | Big Data Hadoop Spark Tutori...Apache Spark - Dataframes & Spark SQL - Part 2 | Big Data Hadoop Spark Tutori...
Apache Spark - Dataframes & Spark SQL - Part 2 | Big Data Hadoop Spark Tutori...

Big Data with Hadoop & Spark Training: http://bit.ly/2sm9c61 This CloudxLab Introduction to Spark SQL & DataFrames tutorial helps you to understand Spark SQL & DataFrames in detail. Below are the topics covered in this slide: 1) Loading XML 2) What is RPC - Remote Process Call 3) Loading AVRO 4) Data Sources - Parquet 5) Creating DataFrames From Hive Table 6) Setting up Distributed SQL Engine

spark dataframespark dataframe apidataframe spark
Apache Spark - Dataframes & Spark SQL - Part 1 | Big Data Hadoop Spark Tutori...
Apache Spark - Dataframes & Spark SQL - Part 1 | Big Data Hadoop Spark Tutori...Apache Spark - Dataframes & Spark SQL - Part 1 | Big Data Hadoop Spark Tutori...
Apache Spark - Dataframes & Spark SQL - Part 1 | Big Data Hadoop Spark Tutori...

Big Data with Hadoop & Spark Training: http://bit.ly/2sf2z6i This CloudxLab Introduction to Spark SQL & DataFrames tutorial helps you to understand Spark SQL & DataFrames in detail. Below are the topics covered in this slide: 1) Introduction to DataFrames 2) Creating DataFrames from JSON 3) DataFrame Operations 4) Running SQL Queries Programmatically 5) Datasets 6) Inferring the Schema Using Reflection 7) Programmatically Specifying the Schema

spark dataframespark dataframe apidataframe spark
Machine Learning - SVM
Linear SVM Classification
Linear SVM
Classification
Nonlinear SVM
Classification
SVM
Regression
Bad model
versus good-
model (Large
Margin-
Standardized)
Classification
Soft Margin
versus Hard-
margin
Classification
Machine Learning - SVM
Linear SVM Classification - Hard Margin
● Hard Margin Classification
○ Strictly impose that all the instances should be
■ Off the street and
■ On a particular side of the decision boundary
○ Issues:
■ Works only if the data is linearly separable
■ Quite sensitive to outliers
Machine Learning - SVM
Linear SVM Classification - Hard Margin
Question - Is it possible to classify this using SVM Hard Margin
Classification?
See the code in notebook
Machine Learning - SVM
Linear SVM Classification - Hard Margin
Question - Is it possible to classify this using SVM Hard Margin
Classification?
See the code in notebook

Recommended for you

Apache Spark - Running on a Cluster | Big Data Hadoop Spark Tutorial | CloudxLab
Apache Spark - Running on a Cluster | Big Data Hadoop Spark Tutorial | CloudxLabApache Spark - Running on a Cluster | Big Data Hadoop Spark Tutorial | CloudxLab
Apache Spark - Running on a Cluster | Big Data Hadoop Spark Tutorial | CloudxLab

(Big Data with Hadoop & Spark Training: http://bit.ly/2IUsWca This CloudxLab Running in a Cluster tutorial helps you to understand running Spark in the cluster in detail. Below are the topics covered in this tutorial: 1) Spark Runtime Architecture 2) Driver Node 3) Scheduling Tasks on Executors 4) Understanding the Architecture 5) Cluster Managers 6) Executors 7) Launching a Program using spark-submit 8) Local Mode & Cluster-Mode 9) Installing Standalone Cluster 10) Cluster Mode - YARN 11) Launching a Program on YARN 12) Cluster Mode - Mesos and AWS EC2 13) Deployment Modes - Client and Cluster 14) Which Cluster Manager to Use? 15) Common flags for spark-submit

spark executor memoryspark architecturespark executor
Introduction to SparkR | Big Data Hadoop Spark Tutorial | CloudxLab
Introduction to SparkR | Big Data Hadoop Spark Tutorial | CloudxLabIntroduction to SparkR | Big Data Hadoop Spark Tutorial | CloudxLab
Introduction to SparkR | Big Data Hadoop Spark Tutorial | CloudxLab

Big Data with Hadoop & Spark Training: http://bit.ly/2LCTufA This CloudxLab Introduction to SparkR tutorial helps you to understand SparkR in detail. Below are the topics covered in this tutorial: 1) SparkR (R on Spark) 2) SparkR DataFrames 3) Launch SparkR 4) Creating DataFrames from Local DataFrames 5) DataFrame Operation 6) Creating DataFrames - From JSON 7) Running SQL Queries from SparkR

sparkrsparkr tutorialbig data analytics
Introduction to NoSQL | Big Data Hadoop Spark Tutorial | CloudxLab
Introduction to NoSQL | Big Data Hadoop Spark Tutorial | CloudxLabIntroduction to NoSQL | Big Data Hadoop Spark Tutorial | CloudxLab
Introduction to NoSQL | Big Data Hadoop Spark Tutorial | CloudxLab

1) NoSQL databases are non-relational and schema-free, providing alternatives to SQL databases for big data and high availability applications. 2) Common NoSQL database models include key-value stores, column-oriented databases, document databases, and graph databases. 3) The CAP theorem states that a distributed data store can only provide two out of three guarantees around consistency, availability, and partition tolerance.

nosqlnosql databasemongodb
Machine Learning - SVM
Linear SVM Classification - Hard Margin
Question - Is it possible to classify this using SVM Hard Margin
Classification?
See the code in notebook
Machine Learning - SVM
Linear SVM Classification - Hard Margin
Question - Is it possible to classify this using SVM Hard Margin
Classification?
Answer - Yes, but what is the problem?
Yes
See the code in notebook
Machine Learning - SVM
Linear SVM Classification - Hard Margin
Yes
Question - Is it possible to classify this using SVM Hard Margin
Classification?
Answer - Yes, but what is the problem?
Outlier is the problem
Machine Learning - SVM
Linear SVM Classification - Soft Margin
Soft Margin Classification
Is keeping a balance between
○ Keeping the street as large as possible
○ Limiting the margin violations
○ Regulated by using ‘C’ parameter

Recommended for you

Introduction to MapReduce - Hadoop Streaming | Big Data Hadoop Spark Tutorial...
Introduction to MapReduce - Hadoop Streaming | Big Data Hadoop Spark Tutorial...Introduction to MapReduce - Hadoop Streaming | Big Data Hadoop Spark Tutorial...
Introduction to MapReduce - Hadoop Streaming | Big Data Hadoop Spark Tutorial...

Big Data with Hadoop & Spark Training: http://bit.ly/2sh5b3E This CloudxLab Hadoop Streaming tutorial helps you to understand Hadoop Streaming in detail. Below are the topics covered in this tutorial: 1) Hadoop Streaming and Why Do We Need it? 2) Writing Streaming Jobs 3) Testing Streaming jobs and Hands-on on CloudxLab

hadoop architecturewhat is mapreducemap reduce
Introduction To TensorFlow | Deep Learning Using TensorFlow | CloudxLab
Introduction To TensorFlow | Deep Learning Using TensorFlow | CloudxLabIntroduction To TensorFlow | Deep Learning Using TensorFlow | CloudxLab
Introduction To TensorFlow | Deep Learning Using TensorFlow | CloudxLab

This document provides instructions for getting started with TensorFlow using a free CloudxLab. It outlines the following steps: 1. Open CloudxLab and enroll if not already enrolled. Otherwise go to "My Lab". 2. In "My Lab", open Jupyter and run commands to clone an ML repository containing TensorFlow examples. 3. Go to the deep learning folder in Jupyter and open the TensorFlow notebook to get started with examples.

deep learningtensorflowtensorflow tutorial
Introduction to Deep Learning | CloudxLab
Introduction to Deep Learning | CloudxLabIntroduction to Deep Learning | CloudxLab
Introduction to Deep Learning | CloudxLab

( Machine Learning & Deep Learning Specialization Training: https://goo.gl/goQxnL ) This CloudxLab Deep Learning tutorial helps you to understand Deep Learning in detail. Below are the topics covered in this tutorial: 1) What is Deep Learning 2) Deep Learning Applications 3) Artificial Neural Network 4) Deep Learning Neural Networks 5) Deep Learning Frameworks 6) AI vs Machine Learning

deep learningartificial neural networkai vs machine learning
Machine Learning - SVM
Linear SVM Classification - Soft Margin
● The balance can be regulated in Scikit-Learn using ‘c’ parameter
○ Higher ‘c’:
■ Narrower street, lower margin violations
○ Smaller ‘c’:
■ Wider street, more margin violations
>>> svm_clf = SVC(kernel="linear", C=100)
SVM Linear classification ‘C’ parameter to
regulate the street
and margin violations
Machine Learning - SVM
Linear SVM Classification - Soft Margin
Example 1: SVM Classification for IRIS data using c = 1
Steps:
● Load the IRIS data
● Model the SVM Linear classifier with the training set: fitting
● Test using a sample data
For illustration:
● Plot the decision boundary and the training samples
Something missing in the steps?
Machine Learning - SVM
Linear SVM Classification - Soft Margin
Example 1: SVM Classification for IRIS data using c = 1
Steps:
● Load the IRIS data
● Feature scaling the data
● Model the SVM Linear classifier with the training set: fitting
● Test using a sample data
For illustration:
● Plot the decision boundary and the training samples
Something missing in the steps?
Machine Learning - SVM
Switch to Notebook

Recommended for you

Dimensionality Reduction | Machine Learning | CloudxLab
Dimensionality Reduction | Machine Learning | CloudxLabDimensionality Reduction | Machine Learning | CloudxLab
Dimensionality Reduction | Machine Learning | CloudxLab

In this tutorial, we will learn the the following topics - + The Curse of Dimensionality + Main Approaches for Dimensionality Reduction + PCA - Principal Component Analysis + Kernel PCA + LLE + Other Dimensionality Reduction Techniques

dimensionality reductionkernel pcadimensionality reduction techniques
Ensemble Learning and Random Forests
Ensemble Learning and Random ForestsEnsemble Learning and Random Forests
Ensemble Learning and Random Forests

In this tutorial, we will learn the the following topics - + Voting Classifiers + Bagging and Pasting + Random Patches and Random Subspaces + Random Forests + Boosting + Stacking

ensemble learningensemble machine learningrandom forest
Decision Trees
Decision TreesDecision Trees
Decision Trees

In this tutorial, we will learn the the following topics - + Training and Visualizing a Decision Tree + Making Predictions + Estimating Class Probabilities + The CART Training Algorithm + Computational Complexity + Gini Impurity or Entropy? + Regularization Hyperparameters + Regression + Instability

decision treesdecision treedecision tree analysis
Machine Learning - SVM
Linear SVM Classification - Soft Margin
Example 1: SVM Classification for IRIS data using c = 1
Steps:
● Load the IRIS data
>>> from sklearn import datasets
>>> from sklearn.pipeline import Pipeline
>>> from sklearn.preprocessing import StandardScaler
>>> from sklearn.svm import LinearSVC
>>> iris = datasets.load_iris()
>>> X = iris["data"][:, (2, 3)] # petal length, petal width
>>> y = (iris["target"] == 2).astype(np.float64) # Iris-Virginica
Machine Learning - SVM
Linear SVM Classification - Soft Margin
Example 1: SVM Classification for IRIS data using c = 1
Steps:
● Load the IRIS data
● Feature scaling the data
● Model the SVM Linear classifier with the training set: fitting
>>> scaler = StandardScaler()
>>> svm_clf2 = LinearSVC(C=1, loss="hinge")
>>> scaled_svm_clf2 = Pipeline((("scaler", scaler), ("linear_svc",
svm_clf2), ))
>>> scaled_svm_clf2.fit(X, y)
Machine Learning - SVM
Linear SVM Classification - Soft Margin
Example 1: SVM Classification for IRIS data using c = 1
Steps:
● Load the IRIS data
● Feature scaling the data
● Model the SVM Linear classifier with the training set: fitting
● Test using a sample data
>>> scaled_svm_clf1.predict([[5.5, 1.7]])
array([ 1.])
Machine Learning - SVM
Linear SVM Classification - Soft Margin
Example 1: SVM Classification for IRIS data using c = 1
Illustration:
● Plot the decision boundary along with the training data
○ Convert to unscaled parameters
■ Training data and decision boundary as calculated
○ Find support vectors
○ Plot it on the graph

Recommended for you

Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - Mydbops
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - MydbopsScaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - Mydbops
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - Mydbops

This presentation, delivered at the Postgres Bangalore (PGBLR) Meetup-2 on June 29th, 2024, dives deep into connection pooling for PostgreSQL databases. Aakash M, a PostgreSQL Tech Lead at Mydbops, explores the challenges of managing numerous connections and explains how connection pooling optimizes performance and resource utilization. Key Takeaways: * Understand why connection pooling is essential for high-traffic applications * Explore various connection poolers available for PostgreSQL, including pgbouncer * Learn the configuration options and functionalities of pgbouncer * Discover best practices for monitoring and troubleshooting connection pooling setups * Gain insights into real-world use cases and considerations for production environments This presentation is ideal for: * Database administrators (DBAs) * Developers working with PostgreSQL * DevOps engineers * Anyone interested in optimizing PostgreSQL performance Contact info@mydbops.com for PostgreSQL Managed, Consulting and Remote DBA Services

postgresqlpgsqldatabase
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf

Kief Morris rethinks the infrastructure code delivery lifecycle, advocating for a shift towards composable infrastructure systems. We should shift to designing around deployable components rather than code modules, use more useful levels of abstraction, and drive design and deployment from applications rather than bottom-up, monolithic architecture and delivery.

infrastructure as codeclouddevops
Measuring the Impact of Network Latency at Twitter
Measuring the Impact of Network Latency at TwitterMeasuring the Impact of Network Latency at Twitter
Measuring the Impact of Network Latency at Twitter

Widya Salim and Victor Ma will outline the causal impact analysis, framework, and key learnings used to quantify the impact of reducing Twitter's network latency.

Machine Learning - SVM
Linear SVM Classification - Soft Margin
Example 1: SVM Classification for IRIS data using c = 1
Illustration:
● Plot the decision boundary along with the training data
○ Convert to unscaled parameters
# Convert to unscaled parameters
>>> b2 = svm_clf1.decision_function([-scaler.mean_ / scaler.scale_])
>>> w2 = svm_clf1.coef_[0] / scaler.scale_
>>> svm_clf1.intercept_ = np.array([b2])
>>> svm_clf1.coef_ = np.array([w2])
Machine Learning - SVM
Linear SVM Classification - Soft Margin
Example 1: SVM Classification for IRIS data using c = 1
Illustration:
● Plot the decision boundary along with the training data
○ Find support vectors
# Find support vectors (LinearSVC does not do this automatically)
>>> t = y * 2 - 1
>>> support_vectors_idx2 = (t * (X.dot(w2) + b2) < 1).ravel()
>>> svm_clf1.support_vectors_ = X[support_vectors_idx2]
Machine Learning - SVM
Linear SVM Classification - Soft Margin
Example 1: SVM Classification for IRIS data using c = 1
Illustration:
● Plot the decision boundary along with the training data
○ Plot
>>> plt.plot(X[:, 0][y==1], X[:, 1][y==1], "g^")
>>> plt.plot(X[:, 0][y==0], X[:, 1][y==0], "bs")
>>> plot_svc_decision_boundary(svm_clf2, 4, 6)
>>> plt.xlabel("Petal length", fontsize=14)
>>> plt.title("$C = {}$".format(svm_clf2.C), fontsize=16)
>>> plt.axis([4, 6, 0.8, 2.8])
>>> plt.show()
Machine Learning - SVM
Linear SVM Classification - Soft Margin
Example 1: SVM Classification for IRIS data using c = 1
Illustration:
● Plot the decision boundary along with the training data

Recommended for you

Choose our Linux Web Hosting for a seamless and successful online presence
Choose our Linux Web Hosting for a seamless and successful online presenceChoose our Linux Web Hosting for a seamless and successful online presence
Choose our Linux Web Hosting for a seamless and successful online presence

Our Linux Web Hosting plans offer unbeatable performance, security, and scalability, ensuring your website runs smoothly and efficiently. Visit- https://onliveserver.com/linux-web-hosting/

cheap linux hosting
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-InTrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In

Six months into 2024, and it is clear the privacy ecosystem takes no days off!! Regulators continue to implement and enforce new regulations, businesses strive to meet requirements, and technology advances like AI have privacy professionals scratching their heads about managing risk. What can we learn about the first six months of data privacy trends and events in 2024? How should this inform your privacy program management for the rest of the year? Join TrustArc, Goodwin, and Snyk privacy experts as they discuss the changes we’ve seen in the first half of 2024 and gain insight into the concrete, actionable steps you can take to up-level your privacy program in the second half of the year. This webinar will review: - Key changes to privacy regulations in 2024 - Key themes in privacy and data governance in 2024 - How to maximize your privacy program in the second half of 2024

data privacyprivacy complianceai
20240704 QFM023 Engineering Leadership Reading List June 2024
20240704 QFM023 Engineering Leadership Reading List June 202420240704 QFM023 Engineering Leadership Reading List June 2024
20240704 QFM023 Engineering Leadership Reading List June 2024

Everything that I found interesting about engineering leadership last month

quantumfaxmachine
Machine Learning - SVM
Linear SVM Classification - Soft Margin
We repeat the same model for c = 100 and compare it with c =1
Machine Learning - SVM
Linear SVM Classification - Soft Margin
Question - What is the model we used here?
● SVC (kernel=’linear’, C=1)
● SGDClassifier(loss=’hinge’, alpha = 1/(m*c))
● LinearSVC
Machine Learning - SVM
Linear SVM Classification - Soft Margin
Question - What is the model we used here?
● SVC (kernel=’linear’, C=1)
● SGDClassifier(loss=’hinge’, alpha = 1/(m*c))
● Ans: LinearSVC
Machine Learning - SVM
Linear SVM Classification
Linear SVM
Classification
Nonlinear
SVM
Classification
SVM
Regression
Bad model
versus good-
model (Large
Margin)
Classification
Soft Margin
versus Hard-
margin
Classification

Recommended for you

Observability For You and Me with OpenTelemetry
Observability For You and Me with OpenTelemetryObservability For You and Me with OpenTelemetry
Observability For You and Me with OpenTelemetry

Are you interested in dipping your toes in the cloud native observability waters, but as an engineer you are not sure where to get started with tracing problems through your microservices and application landscapes on Kubernetes? Then this is the session for you, where we take you on your first steps in an active open-source project that offers a buffet of languages, challenges, and opportunities for getting started with telemetry data. The project is called openTelemetry, but before diving into the specifics, we’ll start with de-mystifying key concepts and terms such as observability, telemetry, instrumentation, cardinality, percentile to lay a foundation. After understanding the nuts and bolts of observability and distributed traces, we’ll explore the openTelemetry community; its Special Interest Groups (SIGs), repositories, and how to become not only an end-user, but possibly a contributor.We will wrap up with an overview of the components in this project, such as the Collector, the OpenTelemetry protocol (OTLP), its APIs, and its SDKs. Attendees will leave with an understanding of key observability concepts, become grounded in distributed tracing terminology, be aware of the components of openTelemetry, and know how to take their first steps to an open-source contribution! Key Takeaways: Open source, vendor neutral instrumentation is an exciting new reality as the industry standardizes on openTelemetry for observability. OpenTelemetry is on a mission to enable effective observability by making high-quality, portable telemetry ubiquitous. The world of observability and monitoring today has a steep learning curve and in order to achieve ubiquity, the project would benefit from growing our contributor community.

cloudcloud native observabilitycloud native
Coordinate Systems in FME 101 - Webinar Slides
Coordinate Systems in FME 101 - Webinar SlidesCoordinate Systems in FME 101 - Webinar Slides
Coordinate Systems in FME 101 - Webinar Slides

If you’ve ever had to analyze a map or GPS data, chances are you’ve encountered and even worked with coordinate systems. As historical data continually updates through GPS, understanding coordinate systems is increasingly crucial. However, not everyone knows why they exist or how to effectively use them for data-driven insights. During this webinar, you’ll learn exactly what coordinate systems are and how you can use FME to maintain and transform your data’s coordinate systems in an easy-to-digest way, accurately representing the geographical space that it exists within. During this webinar, you will have the chance to: - Enhance Your Understanding: Gain a clear overview of what coordinate systems are and their value - Learn Practical Applications: Why we need datams and projections, plus units between coordinate systems - Maximize with FME: Understand how FME handles coordinate systems, including a brief summary of the 3 main reprojectors - Custom Coordinate Systems: Learn how to work with FME and coordinate systems beyond what is natively supported - Look Ahead: Gain insights into where FME is headed with coordinate systems in the future Don’t miss the opportunity to improve the value you receive from your coordinate system data, ultimately allowing you to streamline your data analysis and maximize your time. See you there!

DealBook of Ukraine: 2024 edition
DealBook of Ukraine: 2024 editionDealBook of Ukraine: 2024 edition
DealBook of Ukraine: 2024 edition

The DealBook is our annual overview of the Ukrainian tech investment industry. This edition comprehensively covers the full year 2023 and the first deals of 2024.

Machine Learning - SVM
Linear SVM Classification
Linear SVM
Classification
Nonlinear
SVM
Classification
SVM
Regression
SVC
Polynomial
Kernel +
Standard Scaler
SVC RBF
Kernel +
Standard Scaler
Polynomial
Features +
StandardScal
er +
LinearSVC
Machine Learning - SVM
Nonlinear SVM Classification
● Many datasets cannot be linearly separable
○ Approach 1: Add more features as polynomial features
■ Can result in a linearly separable dataset
Machine Learning - SVM
Nonlinear SVM Classification
Approach 1: Add more features as polynomial features
○ Question - Is this linearly separable?
Machine Learning - SVM
Nonlinear SVM Classification
Approach 1: Add more features as polynomial features
● Question - Is this linearly separable? - No

Recommended for you

Fluttercon 2024: Showing that you care about security - OpenSSF Scorecards fo...
Fluttercon 2024: Showing that you care about security - OpenSSF Scorecards fo...Fluttercon 2024: Showing that you care about security - OpenSSF Scorecards fo...
Fluttercon 2024: Showing that you care about security - OpenSSF Scorecards fo...

Have you noticed the OpenSSF Scorecard badges on the official Dart and Flutter repos? It's Google's way of showing that they care about security. Practices such as pinning dependencies, branch protection, required reviews, continuous integration tests etc. are measured to provide a score and accompanying badge. You can do the same for your projects, and this presentation will show you how, with an emphasis on the unique challenges that come up when working with Dart and Flutter. The session will provide a walkthrough of the steps involved in securing a first repository, and then what it takes to repeat that process across an organization with multiple repos. It will also look at the ongoing maintenance involved once scorecards have been implemented, and how aspects of that maintenance can be better automated to minimize toil.

dartflutteropenssf
How Social Media Hackers Help You to See Your Wife's Message.pdf
How Social Media Hackers Help You to See Your Wife's Message.pdfHow Social Media Hackers Help You to See Your Wife's Message.pdf
How Social Media Hackers Help You to See Your Wife's Message.pdf

In the modern digital era, social media platforms have become integral to our daily lives. These platforms, including Facebook, Instagram, WhatsApp, and Snapchat, offer countless ways to connect, share, and communicate.

social media hackerfacebook hackerhire a instagram hacker
Quantum Communications Q&A with Gemini LLM
Quantum Communications Q&A with Gemini LLMQuantum Communications Q&A with Gemini LLM
Quantum Communications Q&A with Gemini LLM

Quantum Communications Q&A with Gemini LLM. These are based on Shannon's Noisy channel Theorem and offers how the classical theory applies to the quantum world.

quantum communicationsshannon's channel theoremclassical theory
Machine Learning - SVM
Nonlinear SVM Classification
Approach 1: Add more features as polynomial features
● What if we transform this data and add a new feature that is squared
of the original dataset
Original X0 (X1) Label X0_new (X2 = X1^2)
-4 1 16
-3 1 9
-2 0 4
-1 0 1
0 0 0
1 0 1
2 0 4
3 1 9
4 1 16
Machine Learning - SVM
Nonlinear SVM Classification
Approach 1: Add more features as polynomial features
● We plot the new feature along with the old feature
Machine Learning - SVM
Nonlinear SVM Classification
Approach 1: Add more features as polynomial features
● Question - Is it linearly separable?
Machine Learning - SVM
Nonlinear SVM Classification
Approach 1: Add more features as polynomial features
● Question - Is it linearly separable? YES

Recommended for you

20240705 QFM024 Irresponsible AI Reading List June 2024
20240705 QFM024 Irresponsible AI Reading List June 202420240705 QFM024 Irresponsible AI Reading List June 2024
20240705 QFM024 Irresponsible AI Reading List June 2024

Everything that I found interesting last month about the irresponsible use of machine intelligence

quantumfaxmachine
Best Programming Language for Civil Engineers
Best Programming Language for Civil EngineersBest Programming Language for Civil Engineers
Best Programming Language for Civil Engineers

The integration of programming into civil engineering is transforming the industry. We can design complex infrastructure projects and analyse large datasets. Imagine revolutionizing the way we build our cities and infrastructure, all by the power of coding. Programming skills are no longer just a bonus—they’re a game changer in this era. Technology is revolutionizing civil engineering by integrating advanced tools and techniques. Programming allows for the automation of repetitive tasks, enhancing the accuracy of designs, simulations, and analyses. With the advent of artificial intelligence and machine learning, engineers can now predict structural behaviors under various conditions, optimize material usage, and improve project planning.

programmingcodingcivil engineering
WhatsApp Image 2024-03-27 at 08.19.52_bfd93109.pdf
WhatsApp Image 2024-03-27 at 08.19.52_bfd93109.pdfWhatsApp Image 2024-03-27 at 08.19.52_bfd93109.pdf
WhatsApp Image 2024-03-27 at 08.19.52_bfd93109.pdf

Profile portofolio

Machine Learning - SVM
Switch to Notebook
Machine Learning - SVM
Nonlinear SVM Classification: Example
Approach 1: Add more features as polynomial features
● MOONS Dataset
○ Random dataset generator provided by sklearn library
○ 2d or 2 features
○ Single Label
○ Binary Classification
Machine Learning - SVM
Nonlinear SVM Classification: Example
● MOONS Dataset
>>> from sklearn.datasets import make_moons
>>> X, y = make_moons(n_samples=5, noise=0.15, random_state=42)
Result:
[[-0.92892087 0.20526752]
[ 1.86247597 0.48137792]
[-0.30164443 0.42607949]
[ 1.05888696 -0.1393777 ]
[ 1.01197477 -0.52392748]]
[0 1 1 0 1]
No. of samples
seed
Machine Learning - SVM
Nonlinear SVM Classification: Example
● MOONS Dataset
Result:
[[-0.92892087 0.20526752]
[ 1.86247597 0.48137792]
[-0.30164443 0.42607949]
[ 1.05888696 -0.1393777 ]
[ 1.01197477 -0.52392748]]
[0 1 1 0 1]

Recommended for you

Mitigating the Impact of State Management in Cloud Stream Processing Systems
Mitigating the Impact of State Management in Cloud Stream Processing SystemsMitigating the Impact of State Management in Cloud Stream Processing Systems
Mitigating the Impact of State Management in Cloud Stream Processing Systems

Stream processing is a crucial component of modern data infrastructure, but constructing an efficient and scalable stream processing system can be challenging. Decoupling compute and storage architecture has emerged as an effective solution to these challenges, but it can introduce high latency issues, especially when dealing with complex continuous queries that necessitate managing extra-large internal states. In this talk, we focus on addressing the high latency issues associated with S3 storage in stream processing systems that employ a decoupled compute and storage architecture. We delve into the root causes of latency in this context and explore various techniques to minimize the impact of S3 latency on stream processing performance. Our proposed approach is to implement a tiered storage mechanism that leverages a blend of high-performance and low-cost storage tiers to reduce data movement between the compute and storage layers while maintaining efficient processing. Throughout the talk, we will present experimental results that demonstrate the effectiveness of our approach in mitigating the impact of S3 latency on stream processing. By the end of the talk, attendees will have gained insights into how to optimize their stream processing systems for reduced latency and improved cost-efficiency.

Quality Patents: Patents That Stand the Test of Time
Quality Patents: Patents That Stand the Test of TimeQuality Patents: Patents That Stand the Test of Time
Quality Patents: Patents That Stand the Test of Time

Is your patent a vanity piece of paper for your office wall? Or is it a reliable, defendable, assertable, property right? The difference is often quality. Is your patent simply a transactional cost and a large pile of legal bills for your startup? Or is it a leverageable asset worthy of attracting precious investment dollars, worth its cost in multiples of valuation? The difference is often quality. Is your patent application only good enough to get through the examination process? Or has it been crafted to stand the tests of time and varied audiences if you later need to assert that document against an infringer, find yourself litigating with it in an Article 3 Court at the hands of a judge and jury, God forbid, end up having to defend its validity at the PTAB, or even needing to use it to block pirated imports at the International Trade Commission? The difference is often quality. Quality will be our focus for a good chunk of the remainder of this season. What goes into a quality patent, and where possible, how do you get it without breaking the bank? ** Episode Overview ** In this first episode of our quality series, Kristen Hansen and the panel discuss: ⦿ What do we mean when we say patent quality? ⦿ Why is patent quality important? ⦿ How to balance quality and budget ⦿ The importance of searching, continuations, and draftsperson domain expertise ⦿ Very practical tips, tricks, examples, and Kristen’s Musts for drafting quality applications https://www.aurorapatents.com/patently-strategic-podcast.html

patentspatent applicationpatent prosecution
Password Rotation in 2024 is still Relevant
Password Rotation in 2024 is still RelevantPassword Rotation in 2024 is still Relevant
Password Rotation in 2024 is still Relevant

Password Rotation in 2024 is still Relevant

passwordmanagementrotation
Machine Learning - SVM
Nonlinear SVM Classification: Example
● MOONS Dataset
○ Similarly generate 100 such samples
>>> from sklearn.datasets import make_moons
>>> X, y = make_moons(n_samples=100, noise=0.15, random_state=42)
Machine Learning - SVM
Nonlinear SVM Classification: Example
● MOONS Dataset
○ Similarly generate 100 such samples
○ Plotting the dataset
>>> def plot_dataset(X, y, axes):
>>> plt.plot(X[:, 0][y==0], X[:, 1][y==0], "bs")
>>> plt.plot(X[:, 0][y==1], X[:, 1][y==1], "g^")
>>> plt.axis(axes)
>>> plt.grid(True, which='both')
>>> plt.xlabel(r"$x_1$", fontsize=20)
>>> plt.ylabel(r"$x_2$", fontsize=20, rotation=0)
>>> plot_dataset(X, y, [-1.5, 2.5, -1, 1.5])
>>> plt.show()
Machine Learning - SVM
Nonlinear SVM Classification: Example
● MOONS Dataset
○ Similarly generate 100 such samples
○ Plotting the dataset
Machine Learning - SVM
Nonlinear SVM Classification: Example
● MOONS Dataset
○ Q. How to classify this using linear classifier?

Recommended for you

Comparison Table of DiskWarrior Alternatives.pdf
Comparison Table of DiskWarrior Alternatives.pdfComparison Table of DiskWarrior Alternatives.pdf
Comparison Table of DiskWarrior Alternatives.pdf

To help you choose the best DiskWarrior alternative, we've compiled a comparison table summarizing the features, pros, cons, and pricing of six alternatives.

data recoverydatadiskwarrior
Research Directions for Cross Reality Interfaces
Research Directions for Cross Reality InterfacesResearch Directions for Cross Reality Interfaces
Research Directions for Cross Reality Interfaces

An invited talk given by Mark Billinghurst on Research Directions for Cross Reality Interfaces. This was given on July 2nd 2024 as part of the 2024 Summer School on Cross Reality in Hagenberg, Austria (July 1st - 7th)

augmented realitycross realityvirtual reality
Machine Learning - SVM
Nonlinear SVM Classification: Example
● MOONS Dataset
○ Q. How to classify this using linear classifier?
○ Ans: Add more features as polynomial features
Machine Learning - SVM
Nonlinear SVM Classification: Example
● Adding polynomial features
○ What does adding polynomial features mean
○ Let us consider another example
X1 X2 Label
-0.083 0.577 1
1.071 0.205 0
1 x1 x2 x1^2 x1*x2 x2^2 Label
1 -0.083 0.577 0.007 -0.048 0.333 1
1 1.071 0.205 1.147 0.22 0.22 0
Degree = 2
Machine Learning - SVM
Nonlinear SVM Classification: Example
● Adding polynomial features
○ What does adding polynomial features mean
○ Let us consider another example
>>> from sklearn.preprocessing import PolynomialFeatures
>>> X, y = make_moons(n_samples=2, noise=0.15, random_state=42)
>>> np.set_printoptions(precision=2)
>>> print(X)
>>> print(y)
>>> poly=PolynomialFeatures(degree=3)
>>> x1=poly.fit_transform(X)*100
>>> print(x1)
Machine Learning - SVM
Nonlinear SVM Classification: Example
● Adding polynomial features
○ What does adding polynomial features mean
○ Let us consider another example
X = [[-0.08 0.58]
[ 1.07 0.21]]
y = [1 0]
X1 =
[[ 1. -0.08 0.58 0.01 -0.05 0.33 -0. 0. -0.03 0.19]
[ 1. 1.07 0.21 1.15 0.22 0.04 1.23 0.24 0.05 0.01]]

Recommended for you

Machine Learning - SVM
Nonlinear SVM Classification: Example
● MOONS Dataset
○ Q. How to classify this using linear classifier?
○ Ans: Added more features as polynomial features
Machine Learning - SVM
Nonlinear SVM Classification: Example
● MOONS Dataset
○ Add more features with degree 3
○ Scale the new features using StandardScaler()
○ Use SVM Classifier
● All the above steps can be performed in a single iteration using a
Pipeline
Machine Learning - SVM
Nonlinear SVM Classification: Example
>>> from sklearn.pipeline import Pipeline
>>> from sklearn.preprocessing import PolynomialFeatures
>>> polynomial_svm_clf = Pipeline((
("poly_features", PolynomialFeatures(degree=3)),
("scaler", StandardScaler()),
("svm_clf", LinearSVC(C=10, loss="hinge"))
))
>>> polynomial_svm_clf.fit(X, y)
Machine Learning - SVM
Nonlinear SVM Classification: Example
● MOONS Dataset
○ Plotting the dataset along with the classifier (decision boundary)
just modeled

Recommended for you

Machine Learning - SVM
Nonlinear SVM Classification: Example
def plot_predictions(clf, axes):
x0s = np.linspace(axes[0], axes[1], 100)
x1s = np.linspace(axes[2], axes[3], 100)
x0, x1 = np.meshgrid(x0s, x1s)
X = np.c_[x0.ravel(), x1.ravel()]
y_pred = clf.predict(X).reshape(x0.shape)
y_decision = clf.decision_function(X).reshape(x0.shape)
plt.contourf(x0, x1, y_pred, cmap=plt.cm.brg, alpha=0.2)
plt.contourf(x0, x1, y_decision, cmap=plt.cm.brg, alpha=0.1)
plot_predictions(polynomial_svm_clf, [-1.5, 2.5, -1, 1.5])
plot_dataset(X, y, [-1.5, 2.5, -1, 1.5])
plt.show()
Machine Learning - SVM
Nonlinear SVM Classification: Example
Machine Learning - SVM
Switch to Notebook
Machine Learning - SVM
Linear SVM Classification
Linear SVM
Classification
Nonlinear
SVM
Classification
SVM
Regression
SVC
Polynomial
Kernel +
Standard
Scaler
SVC RBF
Kernel +
Standard Scaler
Polynomial
Features +
StandardScaler
+ LinearSVC

Recommended for you

Machine Learning - SVM
Nonlinear SVM Classification
Polynomial Kernel
● Adding polynomial features works great
○ Low polynomial degree cannot deal with complex datasets
○ High polynomial degree makes the model slow due to huge
number of features
● How to overcome the slowness due to huge features?
● Ans: Polynomial Kernels or Kernel trick
Machine Learning - SVM
Nonlinear SVM Classification
Polynomial Kernel
● Adding polynomial features works great
○ Low polynomial degree cannot deal with complex datasets
○ High polynomial degree makes the model slow due to huge
number of features
● How to overcome the slowness due to huge features?
● Ans: Polynomial Kernels or Kernel trick
○ Makes it possible to get the same result as when using high
polynomial degree
○ Without having to add the features which makes the model slow
Machine Learning - SVM
Nonlinear SVM Classification
Polynomial Kernel in Scikit-Learn
● Can be implement in Scikit Learn using SVC Classifier
● Without having to use PolynomialFeatures as in LinearSVC
>>> from sklearn.svm import SVC
>>> poly_kernel_svm_clf = Pipeline((
("scaler", StandardScaler()),
("svm_clf", SVC(
kernel="poly", degree=3, coef0=1, C=5))))
kernel
controls how much the model is
influenced by high-degree polynomials
vs low-degree
coef0
Machine Learning - SVM
Nonlinear SVM Classification
Polynomial Kernel in Scikit-Learn
● Training the classifier using higher degree of polynomial features
# train SVM classifier using 10th-degree polynomial
kernel (for comparison)
>>> poly100_kernel_svm_clf = Pipeline((
("scaler", StandardScaler()),
("svm_clf", SVC(kernel="poly", degree=10,
coef0=100, C=5))
))

Recommended for you

Machine Learning - SVM
Nonlinear SVM Classification
Polynomial Kernel in Scikit-Learn
● Observing the difference in the two cases
Machine Learning - SVM
Switch to Notebook
Machine Learning - SVM
Nonlinear SVM Classification
Linear SVM
Classification
Nonlinear
SVM
Classification
SVM
Regression
SVC
Polynomial
Kernel +
Standard Scaler
SVC RBF
Kernel +
Standard
Scaler
Polynomial
Features +
StandardScaler
+ LinearSVC
Machine Learning - SVM
Nonlinear SVM Classification - SVC RBF
Adding similar features
● Another technique of solving nonlinear classifications
● Add features computed using a similarity function
● Similarity function measures how each instance resembles a particular
‘landmark’

Recommended for you

Machine Learning - SVM
Nonlinear SVM Classification - SVC RBF
● Is this linearly separable? NO
Machine Learning - SVM
Nonlinear SVM Classification - SVC RBF
● Introduce landmarks - x
Machine Learning - SVM
Nonlinear SVM Classification - SVC RBF
● Calculate distance using the formula:
Machine Learning - SVM
Nonlinear SVM Classification - SVC RBF
● New features: distances from landmarks x=-2 and x=1
Original X0 (X1) Label X2 - distance
from Landmark 1
X3 - distance
from Landmark 2
-4 1 0.3 0
-3 1 0.74 0.01
-2 0 1 0.07
-1 0 0.74 0.3
0 0 0.3 0.74
1 0 0.07 1
2 0 0.01 0.74
3 1 0 0.3
4 1 0 0.07

Recommended for you

Machine Learning - SVM
Nonlinear SVM Classification - SVC RBF
● Plot the new features and do linear classification
Machine Learning - SVM
Nonlinear SVM Classification - SVC RBF
● Similarity Function: Using SciKit Learn
# define similarity function to be Gaussian Radial Basis Function
(RBF)
# equals 0 (far away) to 1 (at landmark)
>>> def gaussian_rbf(x, landmark, gamma):
return np.exp(-gamma * np.linalg.norm(x - landmark, axis=1)**2)
>>> gamma = 0.3
>>> x1s = np.linspace(-4.5, 4.5, 200).reshape(-1, 1)
>>> x2s = gaussian_rbf(x1s, -2, gamma)
>>> x3s = gaussian_rbf(x1s, 1, gamma)
>>> XK = np.c_[gaussian_rbf(X1D, -2, gamma), gaussian_rbf(X1D, 1,
gamma)]
>>> yk = np.array([0, 0, 1, 1, 1, 1, 1, 0, 0])
>>> print(XK)
Machine Learning - SVM
Nonlinear SVM Classification - SVC RBF
● Similarity Function: Using SciKit Learn
○ Upon plotting, the difference can be observed
Machine Learning - SVM
Switch to Notebook

Recommended for you

Machine Learning - SVM
Nonlinear SVM Classification - SVC RBF
Similarity Function: How to select the landmarks?
● Create a landmark at each and every instance of the dataset
Drawback
● If training set is huge, number of new features added will be huge
Machine Learning - SVM
Nonlinear SVM Classification - SVC RBF
● Ideally how many new features should be added in this?
Original X0 (X1) Label
-4 1
-3 1
-2 0
-1 0
0 0
1 0
2 0
3 1
4 1
Machine Learning - SVM
Nonlinear SVM Classification
● Ideally how many new features should be added in this? Ans: 9
Original X0 (X1) Label
-4 1
-3 1
-2 0
-1 0
0 0
1 0
2 0
3 1
4 1
Machine Learning - SVM
Nonlinear SVM Classification - SVC RBF
● Ideally how many new features should be added in this? Ans: 9
● The training set converts into 9 instances with 9 features
● Imagine doing this with huge training datasets

Recommended for you

Machine Learning - SVM
Nonlinear SVM Classification - SVC RBF
Gaussian RBF Kernel
● Polynomial Feature addition becomes slow with higher degrees
○ Kernel trick solves it
● Similarity function becomes slow with higher number of training
dataset
○ SVM kernel trick again solves the problem
Machine Learning - SVM
Nonlinear SVM Classification - SVC RBF
Gaussian RBF Kernel
● It lets us to get similar results as if
○ We had added many similarity features
○ Without actually having to add them
Machine Learning - SVM
Nonlinear SVM Classification - SVC RBF
Gaussian RBF Kernel in ScikitLean
>>> rbf_kernel_svm_clf = Pipeline((
("scaler", StandardScaler()),
("svm_clf", SVC(kernel="rbf", gamma=5, C=0.001))
))
>>> rbf_kernel_svm_clf.fit(X, y)
Machine Learning - SVM
Nonlinear SVM Classification - SVC RBF
Gaussian RBF Kernel in ScikitLearn
● Plotting with different hyper parameters

Recommended for you

Machine Learning - SVM
Nonlinear SVM Classification - SVC RBF
Gaussian RBF Kernel in Scikit-Learn
Plotting with different hyper parameters
Increasing Gamma Small Gamma
Makes bell curve narrower Makes the bell curve wider
Reduces influence of each
instance
Instances have a larger range of
influence
Decision boundary becomes
irregular
Decision boundary becomes
smoother
Machine Learning - SVM
Switch to Notebook
Machine Learning - SVM
Computational Complexity
Which kernel to use when?
1. Linear Kernel First
a. LinearSVC faster than SVC(kernel=’linear’) for large datasets with a
lot of features
1. Gaussian RBF kernel
1. Other kernels: Cross validation and grid search
Machine Learning - SVM
Computational Complexity
Linear SVC
● Based on liblinear library
● Scales linearly with number of instances and number of features
● Does not support kernel tricks
● Time complexity is: O(m * n)

Recommended for you

Machine Learning - SVM
Computational Complexity
m = number of training sets
n = number of features
SVC Class
● Based on libsvm library
● Support kernel tricks
● Time complexity is: O(m^2 * n) and O(m^3 * n)
● Dreadfully slow when the number of training sets increases
● Perfect for complex but small or medium training sets
Machine Learning - SVM
SVM Classification - Comparison
LinearSVC SVC SGDClassifier
Fast Slow for large datasets
Perfect for small but
complex training sets
Does not converge as
fast as LinearSVC but
can be useful for
datasets that do not fit
in memory
Machine Learning - SVM
Linear SVM Classification
Linear SVM
Classification
Nonlinear SVM
Classification
SVM
Regression
SVC
Polynomial
Kernel +
Standard Scaler
SVC RBF
Kernel +
Standard Scaler
Polynomial
Features +
StandardScaler
+ LinearSVC
Machine Learning - SVM
Linear SVM Classification
Linear SVM
Classification
Nonlinear SVM
Classification
SVM
Regression
Nonlinear
SVM: SVR
Polynomial
Kernel +
degree + C +
epsilon
Linear SVM:
LinearSVR +
Epsilon

Recommended for you

Machine Learning - SVM
SVM Regression
SVM Classifier SVM Regression
Find the largest possible street
between the two classes limiting
margin violations
Fit as many instances as possible on
the street while limiting margin
violations
Widest possible street
Machine Learning - SVM
SVM Regression - Linear
● Width of the SVM Regression model is controlled by a hyperparameter 𝜺
or epsilon.
● Adding training instances within the margin does not affect the model’s
predictions,
○ Hence model is said to be 𝜺-insensitive
Machine Learning - SVM
SVM Regression - Linear
Linear Regression in Scikit-Learn:
LinearSVR can be used
>>> from sklearn.svm import LinearSVR
>>> svm_reg = LinearSVR(epsilon=1.5)
>>> svm_reg.fit(X, y)
Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn
Step 1: Generating random numbers and making a linear relationship
>>> from sklearn.svm import LinearSVR
>>> import numpy.random as rnd
>>> import matplotlib.pyplot as plt
>>> rnd.seed(42)
>>> m = 50
>>> X = 2 * rnd.rand(m,1)
>>> y = (4 + 3 * X + rnd.randn(m,1)).ravel()
>>> plt.scatter(X,y)
>>> plt.show()

Recommended for you

Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn
Step 1: Generating random numbers and making a linear relationship
Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn
Step 2: Fitting a linear Support Vector Regression model to the data
>>> from sklearn.svm import LinearSVR
>>> svm_reg1 = LinearSVR(epsilon=1.5)
>>> svm_reg1.fit(X,y)
>>> x1s = np.linspace(0,2,100)
>>> y1s = svm_reg1.coef_*x1s + svm_reg1.intercept_
>>> plt.scatter(X,y)
>>> plt.plot(x1s, y1s)
>>> plt.show()
Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn
Step 2: Fitting a linear Support Vector Regression model to the data
Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn
Step 3: Plotting the epsilon lines
>>> y1s_eps1 = y1s + 1.5
>>> y1s_eps2 = y1s - 1.5
>>> plt.scatter(X,y)
>>> plt.plot(x1s, y1s)
>>> plt.plot(x1s, y1s_eps1,'k--')
>>> plt.plot(x1s, y1s_eps2,'k--')
>>> plt.xlabel(r"$x_1$", fontsize=18)
>>> plt.ylabel(r"$y$", fontsize=18)
>>> plt.title('eps = 1.5')
>>> plt.show()

Recommended for you

Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn
Step 3: Plotting the epsilon lines
Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn
Step 4: Finding the instances off-the-street and plotting
>>> y_pred = svm_reg1.predict(X)
>>> supp_vec_X = X[np.abs(y-y_pred)>1.5]
>>> supp_vec_y = y[np.abs(y-y_pred)>1.5]
>>> plt.scatter(supp_vec_X,supp_vec_y)
>>> plt.show()
Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn
Step 4: Finding the instances off-the-street and plotting
Machine Learning - SVM
Switch to Notebook

Recommended for you

Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn with eps = 0.5
Step 1: Generating random numbers and making a linear relationship
>>> from sklearn.svm import LinearSVR
>>> import numpy.random as rnd
>>> import matplotlib.pyplot as plt
>>> rnd.seed(42)
>>> m = 50
>>> X = 2 * rnd.rand(m,1)
>>> y = (4 + 3 * X + rnd.randn(m,1)).ravel()
>>> plt.scatter(X,y)
>>> plt.show()
Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn
Step 1: Generating random numbers and making a linear relationship
Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn
Step 2: Fitting a linear Support Vector Regression model to the data
>>> from sklearn.svm import LinearSVR
>>> svm_reg1 = LinearSVR(epsilon = 0.5)
>>> svm_reg1.fit(X,y)
>>> x1s = np.linspace(0,2,100)
>>> y1s = svm_reg1.coef_*x1s + svm_reg1.intercept_
>>> plt.scatter(X,y)
>>> plt.plot(x1s, y1s)
>>> plt.show()
Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn
Step 2: Fitting a linear Support Vector Regression model to the data

Recommended for you

Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn
Step 3: Plotting the epsilon lines
>>> y1s_eps1 = y1s + 0.5
>>> y1s_eps2 = y1s - 0.5
>>> plt.scatter(X,y)
>>> plt.plot(x1s, y1s)
>>> plt.plot(x1s, y1s_eps1,'k--')
>>> plt.plot(x1s, y1s_eps2,'k--')
>>> plt.xlabel(r"$x_1$", fontsize=18)
>>> plt.ylabel(r"$y$", fontsize=18)
>>> plt.title('eps = 1.5')
>>> plt.show()
Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn
Step 3: Plotting the epsilon lines
Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn
Step 4: Finding the instances off-the-street and plotting
>>> y_pred = svm_reg1.predict(X)
>>> supp_vec_X = X[np.abs(y-y_pred)>0.5]
>>> supp_vec_y = y[np.abs(y-y_pred)>0.5]
>>> plt.scatter(supp_vec_X,supp_vec_y)
>>> plt.show()
Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn
Step 4: Finding the instances off-the-street and plotting

Recommended for you

Machine Learning - SVM
Switch to Notebook
Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn
Comparison for epsilon = 0.5 and epsilon = 1.5, observations?
Machine Learning - SVM
Switch to Notebook
Machine Learning - SVM
Linear SVM Regression - Example
● Linear SVM Regression in Scikit-Learn
○ Comparison for eps = 0.5 and eps = 1.5, observations?
■ Number of instances off-the-street are higher for eps=0.5
○ Cannot conclude on which is a better model
Remember: the goal is to maximise the number of training sets within
the epsilon line

Recommended for you

Machine Learning - SVM
Linear SVM Classification
Linear SVM
Classification
Nonlinear SVM
Classification
SVM
Regression
Nonlinear
SVM: SVR
Polynomial
Kernel +
degree + C +
epsilon
Linear SVM:
LinearSVR +
Epsilon
Machine Learning - SVM
SVM Nonlinear Regression
● A ‘kernelized’ SVM Regression model can be used
Machine Learning - SVM
>>> from sklearn.svm import SVR
>>> svm_poly_reg = SVR(kernel="poly", degree=2, C=100,
epsilon=0.1)
>>> svm_poly_reg.fit(X, y)
SVM Nonlinear Regression
● A ‘kernelized’ SVM Regression model can be used
● C - penalty for being outside the margin or error in classification
● Higher C -> Classification: lesser violations, Regression: lesser
regularization
● Lower C -> Classification: more violations, Regression: more
regularization
epsilon = margin parameter C = regularization parameters
Machine Learning - SVM
SVM Nonlinear Regression - Example 1
Nonlinear SVM Regression in Scikit-Learn for a quadratic distributed data

Recommended for you

Machine Learning - SVM
Nonlinear SVM Regression in Scikit-Learn
Step 1: Generating random numbers and making a quadratic
relationship
>>> from sklearn.svm import SVR
>>> import numpy.random as rnd
>>> import matplotlib.pyplot as plt
>>> rnd.seed(42)
>>> m = 100
>>> X = 2 * rnd.rand(m,1) -1
>>> y = (0.2 + 0.1 * X + 0.5 * X**2 + rnd.randn(m, 1)/10).ravel()
>>> plt.scatter(X,y)
>>> plt.show()
SVM Nonlinear Regression - Example 1
Machine Learning - SVM
Nonlinear SVM Regression in Scikit-Learn
Step 1: Generating random numbers and making a quadratic
relationship
SVM Nonlinear Regression - Example 1
Machine Learning - SVM
Nonlinear SVM Regression in Scikit-Learn
Step 2: Fitting a Support Vector Regression model (degree=2) to the
data
>>> from sklearn.svm import SVR
>>> svr_poly_reg1 = SVR(kernel="poly", degree=2, C =
100, epsilon = 0.1)
>>> svr_poly_reg1.fit(X,y)
>>> print(svr_poly_reg1.C)
>>> x1s = np.linspace(-1,1,200)
>>> plot_svm_regression(svr_poly_reg1, X, y, [-1, 1, 0,
1])
SVM Nonlinear Regression - Example 1
Machine Learning - SVM
Nonlinear SVM Regression in Scikit-Learn
Step 2: Fitting a Support Vector Regression model (degree=2) to the
data
SVM Nonlinear Regression - Example 1

Recommended for you

Machine Learning - SVM
Nonlinear SVM Regression in Scikit-Learn
Step 3: Plotting the epsilon lines
>>> y1s_eps1 = y1s + 0.1
>>> y1s_eps2 = y1s - 0.1
>>> plt.scatter(X,y)
>>> plt.plot(x1s, y1s)
>>> plt.plot(x1s, y1s_eps1,'k--')
>>> plt.plot(x1s, y1s_eps2,'k--')
>>> plt.xlabel(r"$x_1$", fontsize=18)
>>> plt.ylabel(r"$y$", fontsize=18)
>>> plt.title('eps = 1.5')
>>> plt.show()
SVM Nonlinear Regression - Example 1
Machine Learning - SVM
Nonlinear SVM Regression in Scikit-Learn
Step 3: Plotting the epsilon lines
SVM Nonlinear Regression - Example 1
Machine Learning - SVM
Nonlinear SVM Regression in Scikit-Learn
Step 4: Finding the instances off-the-street and plotting
>>> y1_predict = svr_poly_reg1.predict(X)
>>> supp_vectors_X = X[np.abs(y-y1_predict)>0.1]
>>> supp_vectors_y = y[np.abs(y-y1_predict)>0.1]
>>> plt.scatter(supp_vectors_X ,supp_vectors_y)
>>> plt.show()
SVM Nonlinear Regression - Example 1
Machine Learning - SVM
Nonlinear SVM Regression in Scikit-Learn
Step 4: Finding the instances off-the-street and plotting
SVM Nonlinear Regression - Example 1

Recommended for you

Machine Learning - SVM
Switch to Notebook
Machine Learning - SVM
SVM Nonlinear Regression - Comparison
Machine Learning - SVM
Switch to Notebook
Machine Learning - SVM
SVM Nonlinear Regression - Comparison
The model as calculated with different hyper-parameters can be observed
- Higher eps: Less number of instances off-the-street
- Higher C: Less number of instances off-the-street
However, higher eps and lesser number of violations does not always imply
a better model.
Similarly, higher C can lead to overfitting

Recommended for you

Machine Learning - SVM
SVM Classification Summary
Linear Classification
- Bad-model versus good-model: large-margin classification
- SVM Sensitivity to feature scaling
- Hard margin versus Soft margin classification
Nonlinear SVM Classification
- Adding polynomial features and solving using kernel trick
- Adding similarity features - Gaussian RBF function and kernel trick
Computational comparison for SVC, SVCLinear and SGDClassifier
Machine Learning - SVM
SVM Regression Summary
SVM Regression (Linear and Non Linear)
- SVM Linear Regression using LinearSVR and controlling the width of the margin using
epsilon
- Using Kernel-ized SVM Regression model to model non-linear models - SVR with kernel,
StandardScaler
Machine Learning - SVM
How do SVMs work? - Under the Hood
Machine Learning - SVM
Linear SVM - Decision Functions
● Let petal width be denoted by x1 and petal length be denoted by x2.
● Decision Function ‘h’ can be defined as w1 * x1 + w2 * x2 + b.
○ If h < 0, then class=0, else class =1.
● It can be represented by the equation below

Recommended for you

Machine Learning - SVM
Linear SVM - Decision Functions
Machine Learning - SVM
Linear SVM - Decision Functions
Training the SVM Classifier would mean:
● Finding w and b such that
● Margin is as wide as possible while
● Avoiding margin violations (hard margin) or
● Limiting them (soft margin)
Machine Learning - SVM
Linear SVM - Decision Functions
Training the SVM Classifier would mean:
● Finding w and b such that
● Margin is as wide as possible while
● Avoiding margin violations (hard margin) or
● Limiting them (soft margin)
Q. Remember hard margin and soft margin?
Machine Learning - SVM
Linear SVM - Decision Functions
Training the SVM Classifier would mean:
● Finding w and b such that
● Margin is as wide as possible while
● Avoiding margin violations (hard margin) or
● Limiting them (soft margin)
Q. How do we achieve the above?

Recommended for you

Machine Learning - SVM
Linear SVM - Decision Functions
Training the SVM Classifier would mean:
● Finding w and b such that
● Margin is as wide as possible while
● Avoiding margin violations (hard margin) or
● Limiting them (soft margin)
Q. How do we achieve the above?
● Optimization
Machine Learning - SVM
Linear SVM - Decision Functions
What do we know?
- For a 2d dataset, the slope of the margin is equal to w = [w1, w2] and
the slope of the decision boundary is equal to w1^2 + w2^2
- For a n-dimensional dataset, w = [w1, w2, w3, ... , wn] and the slope of
the decision function is denoted by || w ||
Machine Learning - SVM
Linear SVM - Decision Functions
What we also know,
- Smaller the weight vector, larger is the margin
Machine Learning - SVM
Linear SVM - Decision Functions
- So, in order to achieve the best classifier
- we can minimize || w || to maximize the margin, can we?

Recommended for you

Machine Learning - SVM
Linear SVM - Decision Functions
- So, in order to achieve the best classifier
- we can minimize || w || to maximize the margin, can we?
- No
- For hard margin, we need to ensure
- Decision function > 1 for all positive training instances
- Decision function < -1 for all negative training instances
Machine Learning - SVM
Linear SVM - Decision Functions
So, the problem basically becomes:
Where
t(i) =1 for positive instances and t(i) = -1 for negative instances
Machine Learning - SVM
Linear SVM - Decision Functions
- So, in order to achieve the best classifier
- we can minimize || w || to maximize the margin, can we?
- No
- For soft margin, we need to include a slack variable to the minimization
equation
- Two conflicting goals:
- Minimize the weights matrix to maximize the margin
- Minimize the slack variable to reduce margin violations
- C hyper-parameter allows us to define the trade-off between the two
Machine Learning - SVM
Linear SVM - Decision Functions
So, the problem for soft-margin basically becomes:
Where
t(i) =1 for positive instances and t(i) = -1 for negative instances

Recommended for you

Machine Learning - SVM
Linear SVM - Decision Functions
Both hard-margin and soft-margin problems are
- Convex quadratic problems with
- Linear constraints
Such problems are known as Quadratic Programming (QP) problems
- Can be solved using off-the-shelf solvers
- Using variety of techniques
- We will not discuss this in the session
Machine Learning - SVM
Linear SVM - Decision Functions
So now we know that the hard-margin and soft-margin classifiers
- Is an optimization problem to
- minimize the cost
- given certain constraints
- The optimization is a quadratic programming (QP) problem
- Which is solved using off-the-shelf solver
- Basically, the classifier function is calling a QP solver in the backend to
calculate the weights of the decision boundary
Machine Learning - SVM
Dual Problem
The original constrained optimization problem , known as the primal
problem, can be expressed as another closely related problem known as
dual problem
Machine Learning - SVM
Dual Problem
Dual problem gives a lower bound to the solution of the primal problem,
but under some circumstances gives the same result.
- SVM problems meet these conditions, hence have same solution for both
primal and dual problems.

Recommended for you

Machine Learning - SVM
Dual Problem
Can be expressed as
Primal problem
Dual problem
Machine Learning - SVM
Dual Problem
Solution from the above dual problem can be transformed to the solution
of the original primal problem using:
Machine Learning - SVM
Dual Problem
Primal Problem Dual Problem
Slow to solve Faster to solve than the primal when
the number of training instances are
smaller than the number of features
Kernel trick is not possible Hence, making the kernel trick
possible
Machine Learning - SVM
Kernelized SVM
- When did we use SVM Kernel?
- Review (Slides 91 to 125)

Recommended for you

Machine Learning - SVM
Kernelized SVM
When do we use kernelized SVMs?
- We applied a 2nd degree polynomial transformation
- And then train a linear SVM classifier on the transformed training set
Machine Learning - SVM
Kernelized SVM
The 2nd-degree polynomial transformed set is 3-dimensional instead of
two-dimensional. (dropping the initial features)
Machine Learning - SVM
Kernelized SVM
If there are two sets of 2-dimensional feature sets, a and b.
We apply 2nd degree polynomial mapping and then compute the dot
product of the transformed vectors?
- Why do we do this?
- The dual problem requires dot product of feature sets
Machine Learning - SVM
Kernelized SVM
- The dot product of transformed vectors
- Is equal to the square of the dot product of the original vectors

Recommended for you

Machine Learning - SVM
Kernelized SVM
- Each degree transformation requires a lot of computation
- Dual problem shall contain dot product of the transformed features
matrix
- Instead, the original feature can be dot-multiplied and squared
- Transformation of the original matrix is not required
- The above trick makes the whole process much more computationally
efficient
Machine Learning - SVM
Kernelized SVM
- Kernel function represented by K
- Capable of computing transformed based only on the original vectors
without having to compute the transformation.
Machine Learning - SVM
Online SVMs
What is Online Learning?
Recap: incremental learning of the model as data gathers more
datasets on the go
Machine Learning
Machine Learning - Online Learning

Recommended for you

Machine Learning
Machine Learning - Online Learning
● Train system incrementally
○ By feeding new data sequentially
○ Or in batches
● System can learn from new data on the fly
● Good for systems where data is a continuous flow
○ Stock prices
Machine Learning
Machine Learning - Online Learning
Using online learning to handle huge datasets
Machine Learning
Machine Learning - Online Learning
Using online learning to handle huge datasets
● Can be used to train huge datasets
○ That can not be fit in one machine
○ The training data gets divided into batches and
○ System gets trained on each batch incrementally
Machine Learning
Machine Learning - Online Learning
Challenges in online learning
● System’s performance gradually declines
○ If bad data is fed to the system
○ Bad data can come from
○ Malfunctioning sensor or robot
○ Someone spamming your system

Recommended for you

Machine Learning
Machine Learning - Online Learning
Challenges in online learning
● Closely monitor the system
○ Turn off the learning if there is a performance drop
○ Or monitor the input data and remove anomalies
Machine Learning
Machine Learning - Online Learning
Challenges in online learning
● Closely monitor the system
○ Turn off the learning if there is a performance drop
○ Or monitor the input data and remove anomalies
Machine Learning
Machine Learning - Online Learning
● Can be implemented using Linear SVM classifiers
○ One method is Gradient Descent, e.g SGDClassifier
○ Covered previously in Chapter 3 and earlier in SVM Classification
● Cost function for SGD Classification can be written as
Maximizes the margin Penalty function for wrong classification
Hinge loss
Machine Learning
Machine Learning - Online Learning
● Online Learning can also be implemented using Kernelized SVMs
○ Currently implemented in Matlab and CPP
○ For large scale nonlinear problems, we should also consider using neural
networks which will be covered in ANN course.

Recommended for you

Archives
Machine Learning - SVM
Kernelized SVM
Machine Learning - SVM
Linear SVM Classification
Linear SVM
Classification
Nonlinear SVM
Classification
SVM
Regression

More Related Content

What's hot

Deep Dive into Hyperparameter Tuning
Deep Dive into Hyperparameter TuningDeep Dive into Hyperparameter Tuning
Deep Dive into Hyperparameter Tuning
Shubhmay Potdar
 
Support Vector Machine ppt presentation
Support Vector Machine ppt presentationSupport Vector Machine ppt presentation
Support Vector Machine ppt presentation
AyanaRukasar
 
Classification techniques in data mining
Classification techniques in data miningClassification techniques in data mining
Classification techniques in data mining
Kamal Acharya
 
Naive Bayes Classifier | Naive Bayes Algorithm | Naive Bayes Classifier With ...
Naive Bayes Classifier | Naive Bayes Algorithm | Naive Bayes Classifier With ...Naive Bayes Classifier | Naive Bayes Algorithm | Naive Bayes Classifier With ...
Naive Bayes Classifier | Naive Bayes Algorithm | Naive Bayes Classifier With ...
Simplilearn
 
Random forest algorithm
Random forest algorithmRandom forest algorithm
Random forest algorithm
Rashid Ansari
 
Ensemble methods in machine learning
Ensemble methods in machine learningEnsemble methods in machine learning
Ensemble methods in machine learning
SANTHOSH RAJA M G
 
Support Vector Machine
Support Vector MachineSupport Vector Machine
Support Vector Machine
Shao-Chuan Wang
 
Classification Algorithm.
Classification Algorithm.Classification Algorithm.
Classification Algorithm.
Megha Sharma
 
Support vector machine-SVM's
Support vector machine-SVM'sSupport vector machine-SVM's
Support vector machine-SVM's
Anudeep Chowdary Kamepalli
 
Understanding Bagging and Boosting
Understanding Bagging and BoostingUnderstanding Bagging and Boosting
Understanding Bagging and Boosting
Mohit Rajput
 
Machine Learning - Ensemble Methods
Machine Learning - Ensemble MethodsMachine Learning - Ensemble Methods
Machine Learning - Ensemble Methods
Andrew Ferlitsch
 
Naïve Bayes Classifier Algorithm.pptx
Naïve Bayes Classifier Algorithm.pptxNaïve Bayes Classifier Algorithm.pptx
Naïve Bayes Classifier Algorithm.pptx
Shubham Jaybhaye
 
Loss Functions for Deep Learning - Javier Ruiz Hidalgo - UPC Barcelona 2018
Loss Functions for Deep Learning - Javier Ruiz Hidalgo - UPC Barcelona 2018Loss Functions for Deep Learning - Javier Ruiz Hidalgo - UPC Barcelona 2018
Loss Functions for Deep Learning - Javier Ruiz Hidalgo - UPC Barcelona 2018
Universitat Politècnica de Catalunya
 
Supervised and Unsupervised Learning In Machine Learning | Machine Learning T...
Supervised and Unsupervised Learning In Machine Learning | Machine Learning T...Supervised and Unsupervised Learning In Machine Learning | Machine Learning T...
Supervised and Unsupervised Learning In Machine Learning | Machine Learning T...
Simplilearn
 
An Introduction to Supervised Machine Learning and Pattern Classification: Th...
An Introduction to Supervised Machine Learning and Pattern Classification: Th...An Introduction to Supervised Machine Learning and Pattern Classification: Th...
An Introduction to Supervised Machine Learning and Pattern Classification: Th...
Sebastian Raschka
 
Support Vector machine
Support Vector machineSupport Vector machine
Support Vector machine
Anandha L Ranganathan
 
Support Vector machine
Support Vector machineSupport Vector machine
Support Vector machine
Anandha L Ranganathan
 
Ensemble learning
Ensemble learningEnsemble learning
Ensemble learning
Mustafa Sherazi
 
Logistic regression in Machine Learning
Logistic regression in Machine LearningLogistic regression in Machine Learning
Logistic regression in Machine Learning
Kuppusamy P
 
Hyperparameter Optimization for Machine Learning
Hyperparameter Optimization for Machine LearningHyperparameter Optimization for Machine Learning
Hyperparameter Optimization for Machine Learning
Francesco Casalegno
 

What's hot (20)

Deep Dive into Hyperparameter Tuning
Deep Dive into Hyperparameter TuningDeep Dive into Hyperparameter Tuning
Deep Dive into Hyperparameter Tuning
 
Support Vector Machine ppt presentation
Support Vector Machine ppt presentationSupport Vector Machine ppt presentation
Support Vector Machine ppt presentation
 
Classification techniques in data mining
Classification techniques in data miningClassification techniques in data mining
Classification techniques in data mining
 
Naive Bayes Classifier | Naive Bayes Algorithm | Naive Bayes Classifier With ...
Naive Bayes Classifier | Naive Bayes Algorithm | Naive Bayes Classifier With ...Naive Bayes Classifier | Naive Bayes Algorithm | Naive Bayes Classifier With ...
Naive Bayes Classifier | Naive Bayes Algorithm | Naive Bayes Classifier With ...
 
Random forest algorithm
Random forest algorithmRandom forest algorithm
Random forest algorithm
 
Ensemble methods in machine learning
Ensemble methods in machine learningEnsemble methods in machine learning
Ensemble methods in machine learning
 
Support Vector Machine
Support Vector MachineSupport Vector Machine
Support Vector Machine
 
Classification Algorithm.
Classification Algorithm.Classification Algorithm.
Classification Algorithm.
 
Support vector machine-SVM's
Support vector machine-SVM'sSupport vector machine-SVM's
Support vector machine-SVM's
 
Understanding Bagging and Boosting
Understanding Bagging and BoostingUnderstanding Bagging and Boosting
Understanding Bagging and Boosting
 
Machine Learning - Ensemble Methods
Machine Learning - Ensemble MethodsMachine Learning - Ensemble Methods
Machine Learning - Ensemble Methods
 
Naïve Bayes Classifier Algorithm.pptx
Naïve Bayes Classifier Algorithm.pptxNaïve Bayes Classifier Algorithm.pptx
Naïve Bayes Classifier Algorithm.pptx
 
Loss Functions for Deep Learning - Javier Ruiz Hidalgo - UPC Barcelona 2018
Loss Functions for Deep Learning - Javier Ruiz Hidalgo - UPC Barcelona 2018Loss Functions for Deep Learning - Javier Ruiz Hidalgo - UPC Barcelona 2018
Loss Functions for Deep Learning - Javier Ruiz Hidalgo - UPC Barcelona 2018
 
Supervised and Unsupervised Learning In Machine Learning | Machine Learning T...
Supervised and Unsupervised Learning In Machine Learning | Machine Learning T...Supervised and Unsupervised Learning In Machine Learning | Machine Learning T...
Supervised and Unsupervised Learning In Machine Learning | Machine Learning T...
 
An Introduction to Supervised Machine Learning and Pattern Classification: Th...
An Introduction to Supervised Machine Learning and Pattern Classification: Th...An Introduction to Supervised Machine Learning and Pattern Classification: Th...
An Introduction to Supervised Machine Learning and Pattern Classification: Th...
 
Support Vector machine
Support Vector machineSupport Vector machine
Support Vector machine
 
Support Vector machine
Support Vector machineSupport Vector machine
Support Vector machine
 
Ensemble learning
Ensemble learningEnsemble learning
Ensemble learning
 
Logistic regression in Machine Learning
Logistic regression in Machine LearningLogistic regression in Machine Learning
Logistic regression in Machine Learning
 
Hyperparameter Optimization for Machine Learning
Hyperparameter Optimization for Machine LearningHyperparameter Optimization for Machine Learning
Hyperparameter Optimization for Machine Learning
 

Similar to Support Vector Machines

Classification
ClassificationClassification
Classification
CloudxLab
 
Machine learning- key concepts
Machine learning- key conceptsMachine learning- key concepts
Machine learning- key concepts
Amir Ziai
 
Recommender Systems from A to Z – Model Training
Recommender Systems from A to Z – Model TrainingRecommender Systems from A to Z – Model Training
Recommender Systems from A to Z – Model Training
Crossing Minds
 
ML Study Jams - Session 3.pptx
ML Study Jams - Session 3.pptxML Study Jams - Session 3.pptx
ML Study Jams - Session 3.pptx
MayankChadha14
 
Text Classification with Lucene/Solr, Apache Hadoop and LibSVM
Text Classification with Lucene/Solr, Apache Hadoop and LibSVMText Classification with Lucene/Solr, Apache Hadoop and LibSVM
Text Classification with Lucene/Solr, Apache Hadoop and LibSVM
lucenerevolution
 
Scalable machine learning
Scalable machine learningScalable machine learning
Scalable machine learning
Tien-Yang (Aiden) Wu
 
OM-DS-Fall2022-Session10-Support vector machine.pdf
OM-DS-Fall2022-Session10-Support vector machine.pdfOM-DS-Fall2022-Session10-Support vector machine.pdf
OM-DS-Fall2022-Session10-Support vector machine.pdf
ssuserb016ab
 
Machine Learning: An introduction โดย รศ.ดร.สุรพงค์ เอื้อวัฒนามงคล
Machine Learning: An introduction โดย รศ.ดร.สุรพงค์  เอื้อวัฒนามงคลMachine Learning: An introduction โดย รศ.ดร.สุรพงค์  เอื้อวัฒนามงคล
Machine Learning: An introduction โดย รศ.ดร.สุรพงค์ เอื้อวัฒนามงคล
BAINIDA
 
Application of Machine Learning in Agriculture
Application of Machine  Learning in AgricultureApplication of Machine  Learning in Agriculture
Application of Machine Learning in Agriculture
Aman Vasisht
 
Understanding computer vision with Deep Learning
Understanding computer vision with Deep LearningUnderstanding computer vision with Deep Learning
Understanding computer vision with Deep Learning
CloudxLab
 
Understanding computer vision with Deep Learning
Understanding computer vision with Deep LearningUnderstanding computer vision with Deep Learning
Understanding computer vision with Deep Learning
ShubhWadekar
 
Understanding computer vision with Deep Learning
Understanding computer vision with Deep LearningUnderstanding computer vision with Deep Learning
Understanding computer vision with Deep Learning
knowbigdata
 
3ml.pdf
3ml.pdf3ml.pdf
3ml.pdf
MianAdnan27
 
Support Vector Machines USING MACHINE LEARNING HOW IT WORKS
Support Vector Machines USING MACHINE LEARNING HOW IT WORKSSupport Vector Machines USING MACHINE LEARNING HOW IT WORKS
Support Vector Machines USING MACHINE LEARNING HOW IT WORKS
rajalakshmi5921
 
background.pptx
background.pptxbackground.pptx
background.pptx
KabileshCm
 
Machine Learning : why we should know and how it works
Machine Learning : why we should know and how it worksMachine Learning : why we should know and how it works
Machine Learning : why we should know and how it works
Kevin Lee
 
Online advertising and large scale model fitting
Online advertising and large scale model fittingOnline advertising and large scale model fitting
Online advertising and large scale model fitting
Wush Wu
 
Keynote at IWLS 2017
Keynote at IWLS 2017Keynote at IWLS 2017
Keynote at IWLS 2017
Manish Pandey
 
Hadoop France meetup Feb2016 : recommendations with spark
Hadoop France meetup  Feb2016 : recommendations with sparkHadoop France meetup  Feb2016 : recommendations with spark
Hadoop France meetup Feb2016 : recommendations with spark
Modern Data Stack France
 
MLEARN 210 B Autumn 2018: Lecture 1
MLEARN 210 B Autumn 2018: Lecture 1MLEARN 210 B Autumn 2018: Lecture 1
MLEARN 210 B Autumn 2018: Lecture 1
heinestien
 

Similar to Support Vector Machines (20)

Classification
ClassificationClassification
Classification
 
Machine learning- key concepts
Machine learning- key conceptsMachine learning- key concepts
Machine learning- key concepts
 
Recommender Systems from A to Z – Model Training
Recommender Systems from A to Z – Model TrainingRecommender Systems from A to Z – Model Training
Recommender Systems from A to Z – Model Training
 
ML Study Jams - Session 3.pptx
ML Study Jams - Session 3.pptxML Study Jams - Session 3.pptx
ML Study Jams - Session 3.pptx
 
Text Classification with Lucene/Solr, Apache Hadoop and LibSVM
Text Classification with Lucene/Solr, Apache Hadoop and LibSVMText Classification with Lucene/Solr, Apache Hadoop and LibSVM
Text Classification with Lucene/Solr, Apache Hadoop and LibSVM
 
Scalable machine learning
Scalable machine learningScalable machine learning
Scalable machine learning
 
OM-DS-Fall2022-Session10-Support vector machine.pdf
OM-DS-Fall2022-Session10-Support vector machine.pdfOM-DS-Fall2022-Session10-Support vector machine.pdf
OM-DS-Fall2022-Session10-Support vector machine.pdf
 
Machine Learning: An introduction โดย รศ.ดร.สุรพงค์ เอื้อวัฒนามงคล
Machine Learning: An introduction โดย รศ.ดร.สุรพงค์  เอื้อวัฒนามงคลMachine Learning: An introduction โดย รศ.ดร.สุรพงค์  เอื้อวัฒนามงคล
Machine Learning: An introduction โดย รศ.ดร.สุรพงค์ เอื้อวัฒนามงคล
 
Application of Machine Learning in Agriculture
Application of Machine  Learning in AgricultureApplication of Machine  Learning in Agriculture
Application of Machine Learning in Agriculture
 
Understanding computer vision with Deep Learning
Understanding computer vision with Deep LearningUnderstanding computer vision with Deep Learning
Understanding computer vision with Deep Learning
 
Understanding computer vision with Deep Learning
Understanding computer vision with Deep LearningUnderstanding computer vision with Deep Learning
Understanding computer vision with Deep Learning
 
Understanding computer vision with Deep Learning
Understanding computer vision with Deep LearningUnderstanding computer vision with Deep Learning
Understanding computer vision with Deep Learning
 
3ml.pdf
3ml.pdf3ml.pdf
3ml.pdf
 
Support Vector Machines USING MACHINE LEARNING HOW IT WORKS
Support Vector Machines USING MACHINE LEARNING HOW IT WORKSSupport Vector Machines USING MACHINE LEARNING HOW IT WORKS
Support Vector Machines USING MACHINE LEARNING HOW IT WORKS
 
background.pptx
background.pptxbackground.pptx
background.pptx
 
Machine Learning : why we should know and how it works
Machine Learning : why we should know and how it worksMachine Learning : why we should know and how it works
Machine Learning : why we should know and how it works
 
Online advertising and large scale model fitting
Online advertising and large scale model fittingOnline advertising and large scale model fitting
Online advertising and large scale model fitting
 
Keynote at IWLS 2017
Keynote at IWLS 2017Keynote at IWLS 2017
Keynote at IWLS 2017
 
Hadoop France meetup Feb2016 : recommendations with spark
Hadoop France meetup  Feb2016 : recommendations with sparkHadoop France meetup  Feb2016 : recommendations with spark
Hadoop France meetup Feb2016 : recommendations with spark
 
MLEARN 210 B Autumn 2018: Lecture 1
MLEARN 210 B Autumn 2018: Lecture 1MLEARN 210 B Autumn 2018: Lecture 1
MLEARN 210 B Autumn 2018: Lecture 1
 

More from CloudxLab

Deep Learning Overview
Deep Learning OverviewDeep Learning Overview
Deep Learning Overview
CloudxLab
 
Recurrent Neural Networks
Recurrent Neural NetworksRecurrent Neural Networks
Recurrent Neural Networks
CloudxLab
 
Natural Language Processing
Natural Language ProcessingNatural Language Processing
Natural Language Processing
CloudxLab
 
Naive Bayes
Naive BayesNaive Bayes
Naive Bayes
CloudxLab
 
Autoencoders
AutoencodersAutoencoders
Autoencoders
CloudxLab
 
Training Deep Neural Nets
Training Deep Neural NetsTraining Deep Neural Nets
Training Deep Neural Nets
CloudxLab
 
Reinforcement Learning
Reinforcement LearningReinforcement Learning
Reinforcement Learning
CloudxLab
 
Apache Spark - Key Value RDD - Transformations | Big Data Hadoop Spark Tutori...
Apache Spark - Key Value RDD - Transformations | Big Data Hadoop Spark Tutori...Apache Spark - Key Value RDD - Transformations | Big Data Hadoop Spark Tutori...
Apache Spark - Key Value RDD - Transformations | Big Data Hadoop Spark Tutori...
CloudxLab
 
Advanced Spark Programming - Part 2 | Big Data Hadoop Spark Tutorial | CloudxLab
Advanced Spark Programming - Part 2 | Big Data Hadoop Spark Tutorial | CloudxLabAdvanced Spark Programming - Part 2 | Big Data Hadoop Spark Tutorial | CloudxLab
Advanced Spark Programming - Part 2 | Big Data Hadoop Spark Tutorial | CloudxLab
CloudxLab
 
Apache Spark - Dataframes & Spark SQL - Part 2 | Big Data Hadoop Spark Tutori...
Apache Spark - Dataframes & Spark SQL - Part 2 | Big Data Hadoop Spark Tutori...Apache Spark - Dataframes & Spark SQL - Part 2 | Big Data Hadoop Spark Tutori...
Apache Spark - Dataframes & Spark SQL - Part 2 | Big Data Hadoop Spark Tutori...
CloudxLab
 
Apache Spark - Dataframes & Spark SQL - Part 1 | Big Data Hadoop Spark Tutori...
Apache Spark - Dataframes & Spark SQL - Part 1 | Big Data Hadoop Spark Tutori...Apache Spark - Dataframes & Spark SQL - Part 1 | Big Data Hadoop Spark Tutori...
Apache Spark - Dataframes & Spark SQL - Part 1 | Big Data Hadoop Spark Tutori...
CloudxLab
 
Apache Spark - Running on a Cluster | Big Data Hadoop Spark Tutorial | CloudxLab
Apache Spark - Running on a Cluster | Big Data Hadoop Spark Tutorial | CloudxLabApache Spark - Running on a Cluster | Big Data Hadoop Spark Tutorial | CloudxLab
Apache Spark - Running on a Cluster | Big Data Hadoop Spark Tutorial | CloudxLab
CloudxLab
 
Introduction to SparkR | Big Data Hadoop Spark Tutorial | CloudxLab
Introduction to SparkR | Big Data Hadoop Spark Tutorial | CloudxLabIntroduction to SparkR | Big Data Hadoop Spark Tutorial | CloudxLab
Introduction to SparkR | Big Data Hadoop Spark Tutorial | CloudxLab
CloudxLab
 
Introduction to NoSQL | Big Data Hadoop Spark Tutorial | CloudxLab
Introduction to NoSQL | Big Data Hadoop Spark Tutorial | CloudxLabIntroduction to NoSQL | Big Data Hadoop Spark Tutorial | CloudxLab
Introduction to NoSQL | Big Data Hadoop Spark Tutorial | CloudxLab
CloudxLab
 
Introduction to MapReduce - Hadoop Streaming | Big Data Hadoop Spark Tutorial...
Introduction to MapReduce - Hadoop Streaming | Big Data Hadoop Spark Tutorial...Introduction to MapReduce - Hadoop Streaming | Big Data Hadoop Spark Tutorial...
Introduction to MapReduce - Hadoop Streaming | Big Data Hadoop Spark Tutorial...
CloudxLab
 
Introduction To TensorFlow | Deep Learning Using TensorFlow | CloudxLab
Introduction To TensorFlow | Deep Learning Using TensorFlow | CloudxLabIntroduction To TensorFlow | Deep Learning Using TensorFlow | CloudxLab
Introduction To TensorFlow | Deep Learning Using TensorFlow | CloudxLab
CloudxLab
 
Introduction to Deep Learning | CloudxLab
Introduction to Deep Learning | CloudxLabIntroduction to Deep Learning | CloudxLab
Introduction to Deep Learning | CloudxLab
CloudxLab
 
Dimensionality Reduction | Machine Learning | CloudxLab
Dimensionality Reduction | Machine Learning | CloudxLabDimensionality Reduction | Machine Learning | CloudxLab
Dimensionality Reduction | Machine Learning | CloudxLab
CloudxLab
 
Ensemble Learning and Random Forests
Ensemble Learning and Random ForestsEnsemble Learning and Random Forests
Ensemble Learning and Random Forests
CloudxLab
 
Decision Trees
Decision TreesDecision Trees
Decision Trees
CloudxLab
 

More from CloudxLab (20)

Deep Learning Overview
Deep Learning OverviewDeep Learning Overview
Deep Learning Overview
 
Recurrent Neural Networks
Recurrent Neural NetworksRecurrent Neural Networks
Recurrent Neural Networks
 
Natural Language Processing
Natural Language ProcessingNatural Language Processing
Natural Language Processing
 
Naive Bayes
Naive BayesNaive Bayes
Naive Bayes
 
Autoencoders
AutoencodersAutoencoders
Autoencoders
 
Training Deep Neural Nets
Training Deep Neural NetsTraining Deep Neural Nets
Training Deep Neural Nets
 
Reinforcement Learning
Reinforcement LearningReinforcement Learning
Reinforcement Learning
 
Apache Spark - Key Value RDD - Transformations | Big Data Hadoop Spark Tutori...
Apache Spark - Key Value RDD - Transformations | Big Data Hadoop Spark Tutori...Apache Spark - Key Value RDD - Transformations | Big Data Hadoop Spark Tutori...
Apache Spark - Key Value RDD - Transformations | Big Data Hadoop Spark Tutori...
 
Advanced Spark Programming - Part 2 | Big Data Hadoop Spark Tutorial | CloudxLab
Advanced Spark Programming - Part 2 | Big Data Hadoop Spark Tutorial | CloudxLabAdvanced Spark Programming - Part 2 | Big Data Hadoop Spark Tutorial | CloudxLab
Advanced Spark Programming - Part 2 | Big Data Hadoop Spark Tutorial | CloudxLab
 
Apache Spark - Dataframes & Spark SQL - Part 2 | Big Data Hadoop Spark Tutori...
Apache Spark - Dataframes & Spark SQL - Part 2 | Big Data Hadoop Spark Tutori...Apache Spark - Dataframes & Spark SQL - Part 2 | Big Data Hadoop Spark Tutori...
Apache Spark - Dataframes & Spark SQL - Part 2 | Big Data Hadoop Spark Tutori...
 
Apache Spark - Dataframes & Spark SQL - Part 1 | Big Data Hadoop Spark Tutori...
Apache Spark - Dataframes & Spark SQL - Part 1 | Big Data Hadoop Spark Tutori...Apache Spark - Dataframes & Spark SQL - Part 1 | Big Data Hadoop Spark Tutori...
Apache Spark - Dataframes & Spark SQL - Part 1 | Big Data Hadoop Spark Tutori...
 
Apache Spark - Running on a Cluster | Big Data Hadoop Spark Tutorial | CloudxLab
Apache Spark - Running on a Cluster | Big Data Hadoop Spark Tutorial | CloudxLabApache Spark - Running on a Cluster | Big Data Hadoop Spark Tutorial | CloudxLab
Apache Spark - Running on a Cluster | Big Data Hadoop Spark Tutorial | CloudxLab
 
Introduction to SparkR | Big Data Hadoop Spark Tutorial | CloudxLab
Introduction to SparkR | Big Data Hadoop Spark Tutorial | CloudxLabIntroduction to SparkR | Big Data Hadoop Spark Tutorial | CloudxLab
Introduction to SparkR | Big Data Hadoop Spark Tutorial | CloudxLab
 
Introduction to NoSQL | Big Data Hadoop Spark Tutorial | CloudxLab
Introduction to NoSQL | Big Data Hadoop Spark Tutorial | CloudxLabIntroduction to NoSQL | Big Data Hadoop Spark Tutorial | CloudxLab
Introduction to NoSQL | Big Data Hadoop Spark Tutorial | CloudxLab
 
Introduction to MapReduce - Hadoop Streaming | Big Data Hadoop Spark Tutorial...
Introduction to MapReduce - Hadoop Streaming | Big Data Hadoop Spark Tutorial...Introduction to MapReduce - Hadoop Streaming | Big Data Hadoop Spark Tutorial...
Introduction to MapReduce - Hadoop Streaming | Big Data Hadoop Spark Tutorial...
 
Introduction To TensorFlow | Deep Learning Using TensorFlow | CloudxLab
Introduction To TensorFlow | Deep Learning Using TensorFlow | CloudxLabIntroduction To TensorFlow | Deep Learning Using TensorFlow | CloudxLab
Introduction To TensorFlow | Deep Learning Using TensorFlow | CloudxLab
 
Introduction to Deep Learning | CloudxLab
Introduction to Deep Learning | CloudxLabIntroduction to Deep Learning | CloudxLab
Introduction to Deep Learning | CloudxLab
 
Dimensionality Reduction | Machine Learning | CloudxLab
Dimensionality Reduction | Machine Learning | CloudxLabDimensionality Reduction | Machine Learning | CloudxLab
Dimensionality Reduction | Machine Learning | CloudxLab
 
Ensemble Learning and Random Forests
Ensemble Learning and Random ForestsEnsemble Learning and Random Forests
Ensemble Learning and Random Forests
 
Decision Trees
Decision TreesDecision Trees
Decision Trees
 

Recently uploaded

Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - Mydbops
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - MydbopsScaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - Mydbops
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - Mydbops
Mydbops
 
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
Kief Morris
 
Measuring the Impact of Network Latency at Twitter
Measuring the Impact of Network Latency at TwitterMeasuring the Impact of Network Latency at Twitter
Measuring the Impact of Network Latency at Twitter
ScyllaDB
 
Choose our Linux Web Hosting for a seamless and successful online presence
Choose our Linux Web Hosting for a seamless and successful online presenceChoose our Linux Web Hosting for a seamless and successful online presence
Choose our Linux Web Hosting for a seamless and successful online presence
rajancomputerfbd
 
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-InTrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc
 
20240704 QFM023 Engineering Leadership Reading List June 2024
20240704 QFM023 Engineering Leadership Reading List June 202420240704 QFM023 Engineering Leadership Reading List June 2024
20240704 QFM023 Engineering Leadership Reading List June 2024
Matthew Sinclair
 
Observability For You and Me with OpenTelemetry
Observability For You and Me with OpenTelemetryObservability For You and Me with OpenTelemetry
Observability For You and Me with OpenTelemetry
Eric D. Schabell
 
Coordinate Systems in FME 101 - Webinar Slides
Coordinate Systems in FME 101 - Webinar SlidesCoordinate Systems in FME 101 - Webinar Slides
Coordinate Systems in FME 101 - Webinar Slides
Safe Software
 
DealBook of Ukraine: 2024 edition
DealBook of Ukraine: 2024 editionDealBook of Ukraine: 2024 edition
DealBook of Ukraine: 2024 edition
Yevgen Sysoyev
 
Fluttercon 2024: Showing that you care about security - OpenSSF Scorecards fo...
Fluttercon 2024: Showing that you care about security - OpenSSF Scorecards fo...Fluttercon 2024: Showing that you care about security - OpenSSF Scorecards fo...
Fluttercon 2024: Showing that you care about security - OpenSSF Scorecards fo...
Chris Swan
 
How Social Media Hackers Help You to See Your Wife's Message.pdf
How Social Media Hackers Help You to See Your Wife's Message.pdfHow Social Media Hackers Help You to See Your Wife's Message.pdf
How Social Media Hackers Help You to See Your Wife's Message.pdf
HackersList
 
Quantum Communications Q&A with Gemini LLM
Quantum Communications Q&A with Gemini LLMQuantum Communications Q&A with Gemini LLM
Quantum Communications Q&A with Gemini LLM
Vijayananda Mohire
 
20240705 QFM024 Irresponsible AI Reading List June 2024
20240705 QFM024 Irresponsible AI Reading List June 202420240705 QFM024 Irresponsible AI Reading List June 2024
20240705 QFM024 Irresponsible AI Reading List June 2024
Matthew Sinclair
 
Best Programming Language for Civil Engineers
Best Programming Language for Civil EngineersBest Programming Language for Civil Engineers
Best Programming Language for Civil Engineers
Awais Yaseen
 
WhatsApp Image 2024-03-27 at 08.19.52_bfd93109.pdf
WhatsApp Image 2024-03-27 at 08.19.52_bfd93109.pdfWhatsApp Image 2024-03-27 at 08.19.52_bfd93109.pdf
WhatsApp Image 2024-03-27 at 08.19.52_bfd93109.pdf
ArgaBisma
 
Mitigating the Impact of State Management in Cloud Stream Processing Systems
Mitigating the Impact of State Management in Cloud Stream Processing SystemsMitigating the Impact of State Management in Cloud Stream Processing Systems
Mitigating the Impact of State Management in Cloud Stream Processing Systems
ScyllaDB
 
Quality Patents: Patents That Stand the Test of Time
Quality Patents: Patents That Stand the Test of TimeQuality Patents: Patents That Stand the Test of Time
Quality Patents: Patents That Stand the Test of Time
Aurora Consulting
 
Password Rotation in 2024 is still Relevant
Password Rotation in 2024 is still RelevantPassword Rotation in 2024 is still Relevant
Password Rotation in 2024 is still Relevant
Bert Blevins
 
Comparison Table of DiskWarrior Alternatives.pdf
Comparison Table of DiskWarrior Alternatives.pdfComparison Table of DiskWarrior Alternatives.pdf
Comparison Table of DiskWarrior Alternatives.pdf
Andrey Yasko
 
Research Directions for Cross Reality Interfaces
Research Directions for Cross Reality InterfacesResearch Directions for Cross Reality Interfaces
Research Directions for Cross Reality Interfaces
Mark Billinghurst
 

Recently uploaded (20)

Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - Mydbops
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - MydbopsScaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - Mydbops
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - Mydbops
 
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
 
Measuring the Impact of Network Latency at Twitter
Measuring the Impact of Network Latency at TwitterMeasuring the Impact of Network Latency at Twitter
Measuring the Impact of Network Latency at Twitter
 
Choose our Linux Web Hosting for a seamless and successful online presence
Choose our Linux Web Hosting for a seamless and successful online presenceChoose our Linux Web Hosting for a seamless and successful online presence
Choose our Linux Web Hosting for a seamless and successful online presence
 
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-InTrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
 
20240704 QFM023 Engineering Leadership Reading List June 2024
20240704 QFM023 Engineering Leadership Reading List June 202420240704 QFM023 Engineering Leadership Reading List June 2024
20240704 QFM023 Engineering Leadership Reading List June 2024
 
Observability For You and Me with OpenTelemetry
Observability For You and Me with OpenTelemetryObservability For You and Me with OpenTelemetry
Observability For You and Me with OpenTelemetry
 
Coordinate Systems in FME 101 - Webinar Slides
Coordinate Systems in FME 101 - Webinar SlidesCoordinate Systems in FME 101 - Webinar Slides
Coordinate Systems in FME 101 - Webinar Slides
 
DealBook of Ukraine: 2024 edition
DealBook of Ukraine: 2024 editionDealBook of Ukraine: 2024 edition
DealBook of Ukraine: 2024 edition
 
Fluttercon 2024: Showing that you care about security - OpenSSF Scorecards fo...
Fluttercon 2024: Showing that you care about security - OpenSSF Scorecards fo...Fluttercon 2024: Showing that you care about security - OpenSSF Scorecards fo...
Fluttercon 2024: Showing that you care about security - OpenSSF Scorecards fo...
 
How Social Media Hackers Help You to See Your Wife's Message.pdf
How Social Media Hackers Help You to See Your Wife's Message.pdfHow Social Media Hackers Help You to See Your Wife's Message.pdf
How Social Media Hackers Help You to See Your Wife's Message.pdf
 
Quantum Communications Q&A with Gemini LLM
Quantum Communications Q&A with Gemini LLMQuantum Communications Q&A with Gemini LLM
Quantum Communications Q&A with Gemini LLM
 
20240705 QFM024 Irresponsible AI Reading List June 2024
20240705 QFM024 Irresponsible AI Reading List June 202420240705 QFM024 Irresponsible AI Reading List June 2024
20240705 QFM024 Irresponsible AI Reading List June 2024
 
Best Programming Language for Civil Engineers
Best Programming Language for Civil EngineersBest Programming Language for Civil Engineers
Best Programming Language for Civil Engineers
 
WhatsApp Image 2024-03-27 at 08.19.52_bfd93109.pdf
WhatsApp Image 2024-03-27 at 08.19.52_bfd93109.pdfWhatsApp Image 2024-03-27 at 08.19.52_bfd93109.pdf
WhatsApp Image 2024-03-27 at 08.19.52_bfd93109.pdf
 
Mitigating the Impact of State Management in Cloud Stream Processing Systems
Mitigating the Impact of State Management in Cloud Stream Processing SystemsMitigating the Impact of State Management in Cloud Stream Processing Systems
Mitigating the Impact of State Management in Cloud Stream Processing Systems
 
Quality Patents: Patents That Stand the Test of Time
Quality Patents: Patents That Stand the Test of TimeQuality Patents: Patents That Stand the Test of Time
Quality Patents: Patents That Stand the Test of Time
 
Password Rotation in 2024 is still Relevant
Password Rotation in 2024 is still RelevantPassword Rotation in 2024 is still Relevant
Password Rotation in 2024 is still Relevant
 
Comparison Table of DiskWarrior Alternatives.pdf
Comparison Table of DiskWarrior Alternatives.pdfComparison Table of DiskWarrior Alternatives.pdf
Comparison Table of DiskWarrior Alternatives.pdf
 
Research Directions for Cross Reality Interfaces
Research Directions for Cross Reality InterfacesResearch Directions for Cross Reality Interfaces
Research Directions for Cross Reality Interfaces
 

Support Vector Machines

  • 2. Machine Learning - SVM Support Vector Machines Divide dataset into training and test samples Train the model using training dataset Test using sample training data Performance metrics (Finalize the model) Improve the model using error analysis Remember the general flow of a machine learning problem: There can be several models depending on the problem statement We will discuss about one such model - SVM
  • 3. Machine Learning - SVM ● Support vector machine is ○ Very powerful and versatile model ○ Capable of performing ■ Linear and ■ Nonlinear classification ■ Regression and ■ Outlier detection ● Well suited for small or medium sized datasets Support Vector Machines
  • 4. Machine Learning - SVM Support Vector Machines ● In this session we will learn about ○ Linear SVM Classification ○ Nonlinear SVM Classification and ○ SVM Regression
  • 5. Machine Learning - SVM Support Vector Machines Linear SVM Classification
  • 6. Machine Learning Human Supervision? Supervised Machine Learning Unsupervised Reinforcement Classification Regression How they generalize? Learn Incrementally? What is Classification?
  • 7. Machine Learning - SVM 5 Not 5 What is Classification? Identifying to which label something belongs to
  • 8. Machine Learning - SVM Examples of Classification ● Classifying emails as spam or not spam Q. What type of classification is this?
  • 9. Machine Learning - SVM Examples of Classification ● Classifying emails as spam or not spam Q. What type of classification is this? Ans: Binary
  • 10. Machine Learning - SVM ● Classifying flowers of a particular species like the Iris Dataset Examples of Classification Q. What type of classification is this?
  • 11. Machine Learning - SVM ● Classifying flowers of a particular species like the Iris Dataset Examples of Classification Q. What type of classification is this? Ans: Multi-class classification
  • 12. Machine Learning - SVM ● Classifying a credit card transaction as fraudulent or not Examples of Classification
  • 13. Machine Learning - SVM Examples of Classification ● Face recognition Q. What type of classification is this?
  • 14. Machine Learning - SVM Examples of Classification ● Face recognition Q. What type of classification is this? Ans: Multi-label classification
  • 15. Machine Learning - SVM 5 Not 5 Recap of 5 and Not 5 Classification Problem Binary Classification Multiclass Classification Q. What is the classifier we used for the Binary Classification?
  • 16. Machine Learning - SVM 5 Not 5 Recap of 5 and Not 5 Classification Problem Binary Classification Multiclass Classification Q. What is the classifier we used for the Binary Classification? Ans: SGDClassifier
  • 17. Machine Learning - SVM 5 Not 5 Recap of 5 and Not 5 Classification Problem Binary Classification Multiclass Classification Q. What is the classifier we used for the Multiclass Classification?
  • 18. Machine Learning - SVM 5 Not 5 Recap of 5 and Not 5 Classification Problem Binary Classification Multiclass Classification Q. What is the classifier we used for the Multiclass Classification? Ans: SGDClassifier - OvO and OvA
  • 19. Machine Learning - SVM What is Linear Classification?
  • 20. Machine Learning - SVM What is Linear Classification? ● The two classes can be separated easily with a ‘straight’ line ‘Straight’ is the keyword. It means linear classification.
  • 21. Machine Learning - SVM What is Linear Classification? ● For example: IRIS Dataset ○ Features: Sepal Length, Petal Length ○ Class: Iris Virginica OR Iris Versicolor OR Iris Setosa
  • 22. Machine Learning - SVM What is Linear Classification? Sepal Length Petal Length Flower Type 1.212 4.1 Iris-Versicolor 0.5 1.545 Iris-Setosa 0.122 1.64 Iris-Setosa 0.2343 ... Iris-Setosa 0.1 ... Iris-Setosa 1.32 ... Iris-Versicolor
  • 23. Machine Learning - SVM What is Linear Classification? ● For the above IRIS Dataset, what is the type of Machine Learning model? ○ Classification or Regression? ■ Ans:
  • 24. Machine Learning - SVM What is Linear Classification? ● For the above IRIS Dataset, what is the type of Machine Learning model? ○ Classification or Regression? ■ Ans: Classification
  • 25. Machine Learning - SVM What is Linear Classification? ● What is the type of Supervised Machine Learning model? ○ Classification or Regression? ■ Ans: Classification ○ What type of classification? ■ Binary Classification ■ Multi-label Classification ■ Multi-output Classification ■ Multi-class Classification
  • 26. Machine Learning - SVM What is Linear Classification? ● What is the type of Supervised Machine Learning model? ○ Classification or Regression? ■ Ans: Classification ○ What type of classification? ■ Binary Classification ■ Multi-label Classification ■ Multi-output Classification ■ Ans: Multi-class Classification
  • 27. Machine Learning - SVM What is Linear Classification? ● For the IRIS dataset above: ○ Number of features? ■ Ans: ○ Number of classes? ■ Ans:
  • 28. Machine Learning - SVM What is Linear Classification? ● For the IRIS dataset above: ○ Number of features? ■ Ans: 2 ○ Number of classes? ■ Ans: 3
  • 29. Machine Learning - SVM What is Linear Classification? ● When we plot the two features on the graph and label it by color ○ The classes can be divided using a straight line ○ Hence, linear classification Straight Line (Linear Classification)
  • 30. Machine Learning - SVM Linear SVM Classification Linear SVM Classification Nonlinear SVM Classification SVM Regression Bad model versus good- model (Large Margin) Classification Soft Margin versus Hard- margin Classification
  • 31. Machine Learning - SVM Linear SVM Classification - Large Margin Pink and red decision boundaries are very close to the instances - bad model Decision Boundary as far away from training instances - good model Large Margin Classification Widest possible street
  • 32. Machine Learning - SVM Linear SVM Classification - Large Margin May not perform well on new instances Adding training instances may not affect the decision boundary Large Margin Classification
  • 33. Machine Learning - SVM Linear SVM Classification - Large Margin Large Margin Classification Widest possible street Support Vectors ● What are Support vectors? ○ Vectors or the training set located closest to the classifier OR ○ Vectors or the training sets located at the edge of the street
  • 34. Machine Learning - SVM Switch to Notebook
  • 35. Machine Learning - SVM Linear SVM Classification - Example 1 X1 X2 Label 1 50 0 5 20 0 3 80 1 5 60 1 ● Without Scaling ● Training dataset
  • 36. Machine Learning - SVM Linear SVM Classification - Example 1 ● Model the classifier, plot the points and the classifier >>> Xs = np.array([[1, 50], [5, 20], [3, 80], [5, 60]]).astype(np.float64) >>> ys = np.array([0, 0, 1, 1]) >>> svm_clf = SVC(kernel="linear", C=100) >>> svm_clf.fit(Xs, ys) >>> plt.plot(Xs[:, 0][ys==1], Xs[:, 1][ys==1], "bo") >>> plt.plot(Xs[:, 0][ys==0], Xs[:, 1][ys==0], "ms") >>> plot_svc_decision_boundary(svm_clf, 0, 6) >>> plt.xlabel("$x_0$", fontsize=20) >>> plt.ylabel("$x_1$ ", fontsize=20, rotation=0) >>> plt.title("Unscaled", fontsize=16) >>> plt.axis([0, 6, 0, 90])
  • 37. Machine Learning - SVM Linear SVM Classification - Example 1 ● Model the classifier, plot the points and the classifier
  • 38. Machine Learning - SVM Linear SVM Classification - Example 1 ● What is the problem?
  • 39. Machine Learning - SVM Linear SVM Classification - Example 1 ● What is the problem? ○ X0 ranges from 0 to 6 while ○ X1 ranges from 20 to 80 ● Solution: Feature Scaling
  • 40. Machine Learning Project Feature Scaling Feature Scaling Quick Revision from Preparing the data for ML Algorithms in End-to-End Project
  • 41. Machine Learning Project Feature Scaling ● ML algorithms do not perform well ○ When the input numerical attributes have very different scales ● Feature Scaling is one of the most important ○ Transformation we need to apply to our data ● Two ways to make sure all attributes have same scale ○ Min-max scaling ○ Standardization
  • 42. Machine Learning Project Feature Scaling Min-max Scaling ● Also known as Normalization ● Normalized values are in the range of [0, 1]
  • 43. Machine Learning Project Feature Scaling Min-max Scaling ● Also known as Normalization ● Normalized values are in the range of [0, 1] Original Value Normalized Value
  • 44. Machine Learning Project Feature Scaling Min-max Scaling - Example # Creating DataFrame first >>> import pandas as pd >>> s1 = pd.Series([1, 2, 3, 4, 5, 6], index=(range(6))) >>> s2 = pd.Series([10, 9, 8, 7, 6, 5], index=(range(6))) >>> df = pd.DataFrame(s1, columns=['s1']) >>> df['s2'] = s2 >>> df
  • 45. Machine Learning Project Feature Scaling Min-max Scaling - Example # Use Scikit-Learn minmax_scaling >>> from mlxtend.preprocessing import minmax_scaling >>> minmax_scaling(df, columns=['s1', 's2']) Original Scaled (In range of 0 and 1)
  • 46. Machine Learning Project Feature Scaling Standardization ● In Machine Learning, we handle various types of data like ○ Audio signals and ○ Pixel values for image data ○ And this data can include multiple dimensions
  • 47. Machine Learning Project Feature Scaling Standardization We scale the values by calculating ○ How many standard deviation is the value away from the mean SAT scores ~ N(mean = 1500, SD = 300) ACT scores ~ N(mean = 21, SD = 5)
  • 48. Machine Learning Project Feature Scaling Standardization ● The general method of calculation ○ Calculate distribution mean and standard deviation for each feature ○ Subtract the mean from each feature ○ Divide the result from previous step of each feature by its standard deviation Standardized Value
  • 49. Machine Learning Project Feature Scaling Standardization ● In Standardization, features are rescaled ● So that output will have the properties of ● Standard normal distribution with ○ Zero mean and ○ Unit variance Mean Standard Deviation
  • 50. Machine Learning Project Feature Scaling Standardization ● Scikit-Learn provides ○ StandardScaler class for standardization
  • 51. Machine Learning Project Feature Scaling Which One to Use? ● Min-max scales in the range of [0,1] ● Standardization does not bound values to a specific range ○ It may be problem for some algorithms ○ Example- Neural networks expect an input value ranging from 0 to 1 ● We’ll learn more use cases as we proceed in the course
  • 52. Machine Learning Project Feature Scaling Back to original Example 1
  • 53. Machine Learning - SVM Linear SVM Classification - Example 2 x1 x2 Label x1 (Scaled) x2 (Scaled) 1 50 0 -1.5 -0.1154 5 20 0 0.9 -1.5011107 3 80 1 -0.3 1.27017 5 60 1 0.9 0.3464 Mean (m1) = 3.5 Std Dev (s1) = 1.65 Mean (m2) = 52.5 Std Dev (s2) = 21.65 (x-m1)/s1 (x-s2)/m2 ● With Scaling
  • 54. Machine Learning - SVM Linear SVM Classification - Example 2 ● Scaling of features X_new = (x-m1)/s1 ● What kind of scaling is this? ○ Normalization ○ Standardization
  • 55. Machine Learning - SVM Linear SVM Classification - Example 2 ● Scaling of features X_new = (x-m1)/s1 ● What kind of scaling is this? ○ Normalization ○ Standardization
  • 56. Machine Learning - SVM Linear SVM Classification - Example 2 ● Scaling of features X_new = (x-m1)/s1 ● What kind of scaling is this? ○ Normalization ○ Standardization ● What is the module available in scikit_learn to perform standardization?
  • 57. Machine Learning - SVM Linear SVM Classification - Example 2 ● Scaling of features X_new = (x-m1)/s1 ● What kind of scaling is this? ○ Normalization ○ Standardization ● What is the module available in scikit_learn to perform standardization? ○ Answer: StandardScalar
  • 58. Machine Learning - SVM Linear SVM Classification - Example 2 ● Scaling the input training data >>> from sklearn.preprocessing import StandardScaler >>> scaler = StandardScaler() >>> X_scaled = scaler.fit_transform(Xs) >>> print(X_scaled) [[-1.50755672 -0.11547005] [ 0.90453403 -1.5011107 ] [-0.30151134 1.27017059] [ 0.90453403 0.34641016]]
  • 59. Machine Learning - SVM Linear SVM Classification - Example 2 ● Building the model, plotting the decision boundary and the training points >>> svm_clf.fit(X_scaled, ys) >>> plt.plot(X_scaled[:, 0][ys==1], X_scaled[:, 1][ys==1], "bo") >>> plt.plot(X_scaled[:, 0][ys==0], X_scaled[:, 1][ys==0], "ms") >>> plot_svc_decision_boundary(svm_clf, -2, 2) >>> plt.ylabel("$x_{1scaled}$", fontsize=20) >>> plt.xlabel("$x_{0scaled}$", fontsize=20) >>> plt.title("Scaled", fontsize=16) >>> plt.axis([-2, 2, -2, 2])
  • 60. Machine Learning - SVM Linear SVM Classification - Example 2 ● Output decision boundary for a scaled training data
  • 61. Machine Learning - SVM Linear SVM Classification ● Unscaled vs Scaled comparison X0 X1 Label X0 (Scaled) X1 (Scaled) 1 50 0 -1.5 -0.1154 5 20 0 0.9 -1.5011107 3 80 1 -0.3 1.27017 5 60 1 0.9 0.3464 Mean (m1) = 3.5 Std Dev (s1) = 1.65 Mean (m2) = 52.5 Std Dev (s2) = 21.65 (x-m1)/s1 (x-m2)/s2
  • 62. Machine Learning - SVM Linear SVM Classification ● Unscaled vs Scaled Widestpossiblestreet
  • 63. Machine Learning - SVM Linear SVM Classification ● Unscaled vs Scaled ○ Linear SVM sensitive to scaling ○ Feature scaling an important part of data preparation ■ Normalization ■ Standardization ○ Scaled features produce better result for the above example
  • 64. Machine Learning - SVM Switch to Notebook
  • 65. Machine Learning - SVM Linear SVM Classification Linear SVM Classification Nonlinear SVM Classification SVM Regression Bad model versus good- model (Large Margin- Standardized) Classification Soft Margin versus Hard- margin Classification
  • 66. Machine Learning - SVM Linear SVM Classification - Hard Margin ● Hard Margin Classification ○ Strictly impose that all the instances should be ■ Off the street and ■ On a particular side of the decision boundary ○ Issues: ■ Works only if the data is linearly separable ■ Quite sensitive to outliers
  • 67. Machine Learning - SVM Linear SVM Classification - Hard Margin Question - Is it possible to classify this using SVM Hard Margin Classification? See the code in notebook
  • 68. Machine Learning - SVM Linear SVM Classification - Hard Margin Question - Is it possible to classify this using SVM Hard Margin Classification? See the code in notebook
  • 69. Machine Learning - SVM Linear SVM Classification - Hard Margin Question - Is it possible to classify this using SVM Hard Margin Classification? See the code in notebook
  • 70. Machine Learning - SVM Linear SVM Classification - Hard Margin Question - Is it possible to classify this using SVM Hard Margin Classification? Answer - Yes, but what is the problem? Yes See the code in notebook
  • 71. Machine Learning - SVM Linear SVM Classification - Hard Margin Yes Question - Is it possible to classify this using SVM Hard Margin Classification? Answer - Yes, but what is the problem? Outlier is the problem
  • 72. Machine Learning - SVM Linear SVM Classification - Soft Margin Soft Margin Classification Is keeping a balance between ○ Keeping the street as large as possible ○ Limiting the margin violations ○ Regulated by using ‘C’ parameter
  • 73. Machine Learning - SVM Linear SVM Classification - Soft Margin ● The balance can be regulated in Scikit-Learn using ‘c’ parameter ○ Higher ‘c’: ■ Narrower street, lower margin violations ○ Smaller ‘c’: ■ Wider street, more margin violations >>> svm_clf = SVC(kernel="linear", C=100) SVM Linear classification ‘C’ parameter to regulate the street and margin violations
  • 74. Machine Learning - SVM Linear SVM Classification - Soft Margin Example 1: SVM Classification for IRIS data using c = 1 Steps: ● Load the IRIS data ● Model the SVM Linear classifier with the training set: fitting ● Test using a sample data For illustration: ● Plot the decision boundary and the training samples Something missing in the steps?
  • 75. Machine Learning - SVM Linear SVM Classification - Soft Margin Example 1: SVM Classification for IRIS data using c = 1 Steps: ● Load the IRIS data ● Feature scaling the data ● Model the SVM Linear classifier with the training set: fitting ● Test using a sample data For illustration: ● Plot the decision boundary and the training samples Something missing in the steps?
  • 76. Machine Learning - SVM Switch to Notebook
  • 77. Machine Learning - SVM Linear SVM Classification - Soft Margin Example 1: SVM Classification for IRIS data using c = 1 Steps: ● Load the IRIS data >>> from sklearn import datasets >>> from sklearn.pipeline import Pipeline >>> from sklearn.preprocessing import StandardScaler >>> from sklearn.svm import LinearSVC >>> iris = datasets.load_iris() >>> X = iris["data"][:, (2, 3)] # petal length, petal width >>> y = (iris["target"] == 2).astype(np.float64) # Iris-Virginica
  • 78. Machine Learning - SVM Linear SVM Classification - Soft Margin Example 1: SVM Classification for IRIS data using c = 1 Steps: ● Load the IRIS data ● Feature scaling the data ● Model the SVM Linear classifier with the training set: fitting >>> scaler = StandardScaler() >>> svm_clf2 = LinearSVC(C=1, loss="hinge") >>> scaled_svm_clf2 = Pipeline((("scaler", scaler), ("linear_svc", svm_clf2), )) >>> scaled_svm_clf2.fit(X, y)
  • 79. Machine Learning - SVM Linear SVM Classification - Soft Margin Example 1: SVM Classification for IRIS data using c = 1 Steps: ● Load the IRIS data ● Feature scaling the data ● Model the SVM Linear classifier with the training set: fitting ● Test using a sample data >>> scaled_svm_clf1.predict([[5.5, 1.7]]) array([ 1.])
  • 80. Machine Learning - SVM Linear SVM Classification - Soft Margin Example 1: SVM Classification for IRIS data using c = 1 Illustration: ● Plot the decision boundary along with the training data ○ Convert to unscaled parameters ■ Training data and decision boundary as calculated ○ Find support vectors ○ Plot it on the graph
  • 81. Machine Learning - SVM Linear SVM Classification - Soft Margin Example 1: SVM Classification for IRIS data using c = 1 Illustration: ● Plot the decision boundary along with the training data ○ Convert to unscaled parameters # Convert to unscaled parameters >>> b2 = svm_clf1.decision_function([-scaler.mean_ / scaler.scale_]) >>> w2 = svm_clf1.coef_[0] / scaler.scale_ >>> svm_clf1.intercept_ = np.array([b2]) >>> svm_clf1.coef_ = np.array([w2])
  • 82. Machine Learning - SVM Linear SVM Classification - Soft Margin Example 1: SVM Classification for IRIS data using c = 1 Illustration: ● Plot the decision boundary along with the training data ○ Find support vectors # Find support vectors (LinearSVC does not do this automatically) >>> t = y * 2 - 1 >>> support_vectors_idx2 = (t * (X.dot(w2) + b2) < 1).ravel() >>> svm_clf1.support_vectors_ = X[support_vectors_idx2]
  • 83. Machine Learning - SVM Linear SVM Classification - Soft Margin Example 1: SVM Classification for IRIS data using c = 1 Illustration: ● Plot the decision boundary along with the training data ○ Plot >>> plt.plot(X[:, 0][y==1], X[:, 1][y==1], "g^") >>> plt.plot(X[:, 0][y==0], X[:, 1][y==0], "bs") >>> plot_svc_decision_boundary(svm_clf2, 4, 6) >>> plt.xlabel("Petal length", fontsize=14) >>> plt.title("$C = {}$".format(svm_clf2.C), fontsize=16) >>> plt.axis([4, 6, 0.8, 2.8]) >>> plt.show()
  • 84. Machine Learning - SVM Linear SVM Classification - Soft Margin Example 1: SVM Classification for IRIS data using c = 1 Illustration: ● Plot the decision boundary along with the training data
  • 85. Machine Learning - SVM Linear SVM Classification - Soft Margin We repeat the same model for c = 100 and compare it with c =1
  • 86. Machine Learning - SVM Linear SVM Classification - Soft Margin Question - What is the model we used here? ● SVC (kernel=’linear’, C=1) ● SGDClassifier(loss=’hinge’, alpha = 1/(m*c)) ● LinearSVC
  • 87. Machine Learning - SVM Linear SVM Classification - Soft Margin Question - What is the model we used here? ● SVC (kernel=’linear’, C=1) ● SGDClassifier(loss=’hinge’, alpha = 1/(m*c)) ● Ans: LinearSVC
  • 88. Machine Learning - SVM Linear SVM Classification Linear SVM Classification Nonlinear SVM Classification SVM Regression Bad model versus good- model (Large Margin) Classification Soft Margin versus Hard- margin Classification
  • 89. Machine Learning - SVM Linear SVM Classification Linear SVM Classification Nonlinear SVM Classification SVM Regression SVC Polynomial Kernel + Standard Scaler SVC RBF Kernel + Standard Scaler Polynomial Features + StandardScal er + LinearSVC
  • 90. Machine Learning - SVM Nonlinear SVM Classification ● Many datasets cannot be linearly separable ○ Approach 1: Add more features as polynomial features ■ Can result in a linearly separable dataset
  • 91. Machine Learning - SVM Nonlinear SVM Classification Approach 1: Add more features as polynomial features ○ Question - Is this linearly separable?
  • 92. Machine Learning - SVM Nonlinear SVM Classification Approach 1: Add more features as polynomial features ● Question - Is this linearly separable? - No
  • 93. Machine Learning - SVM Nonlinear SVM Classification Approach 1: Add more features as polynomial features ● What if we transform this data and add a new feature that is squared of the original dataset Original X0 (X1) Label X0_new (X2 = X1^2) -4 1 16 -3 1 9 -2 0 4 -1 0 1 0 0 0 1 0 1 2 0 4 3 1 9 4 1 16
  • 94. Machine Learning - SVM Nonlinear SVM Classification Approach 1: Add more features as polynomial features ● We plot the new feature along with the old feature
  • 95. Machine Learning - SVM Nonlinear SVM Classification Approach 1: Add more features as polynomial features ● Question - Is it linearly separable?
  • 96. Machine Learning - SVM Nonlinear SVM Classification Approach 1: Add more features as polynomial features ● Question - Is it linearly separable? YES
  • 97. Machine Learning - SVM Switch to Notebook
  • 98. Machine Learning - SVM Nonlinear SVM Classification: Example Approach 1: Add more features as polynomial features ● MOONS Dataset ○ Random dataset generator provided by sklearn library ○ 2d or 2 features ○ Single Label ○ Binary Classification
  • 99. Machine Learning - SVM Nonlinear SVM Classification: Example ● MOONS Dataset >>> from sklearn.datasets import make_moons >>> X, y = make_moons(n_samples=5, noise=0.15, random_state=42) Result: [[-0.92892087 0.20526752] [ 1.86247597 0.48137792] [-0.30164443 0.42607949] [ 1.05888696 -0.1393777 ] [ 1.01197477 -0.52392748]] [0 1 1 0 1] No. of samples seed
  • 100. Machine Learning - SVM Nonlinear SVM Classification: Example ● MOONS Dataset Result: [[-0.92892087 0.20526752] [ 1.86247597 0.48137792] [-0.30164443 0.42607949] [ 1.05888696 -0.1393777 ] [ 1.01197477 -0.52392748]] [0 1 1 0 1]
  • 101. Machine Learning - SVM Nonlinear SVM Classification: Example ● MOONS Dataset ○ Similarly generate 100 such samples >>> from sklearn.datasets import make_moons >>> X, y = make_moons(n_samples=100, noise=0.15, random_state=42)
  • 102. Machine Learning - SVM Nonlinear SVM Classification: Example ● MOONS Dataset ○ Similarly generate 100 such samples ○ Plotting the dataset >>> def plot_dataset(X, y, axes): >>> plt.plot(X[:, 0][y==0], X[:, 1][y==0], "bs") >>> plt.plot(X[:, 0][y==1], X[:, 1][y==1], "g^") >>> plt.axis(axes) >>> plt.grid(True, which='both') >>> plt.xlabel(r"$x_1$", fontsize=20) >>> plt.ylabel(r"$x_2$", fontsize=20, rotation=0) >>> plot_dataset(X, y, [-1.5, 2.5, -1, 1.5]) >>> plt.show()
  • 103. Machine Learning - SVM Nonlinear SVM Classification: Example ● MOONS Dataset ○ Similarly generate 100 such samples ○ Plotting the dataset
  • 104. Machine Learning - SVM Nonlinear SVM Classification: Example ● MOONS Dataset ○ Q. How to classify this using linear classifier?
  • 105. Machine Learning - SVM Nonlinear SVM Classification: Example ● MOONS Dataset ○ Q. How to classify this using linear classifier? ○ Ans: Add more features as polynomial features
  • 106. Machine Learning - SVM Nonlinear SVM Classification: Example ● Adding polynomial features ○ What does adding polynomial features mean ○ Let us consider another example X1 X2 Label -0.083 0.577 1 1.071 0.205 0 1 x1 x2 x1^2 x1*x2 x2^2 Label 1 -0.083 0.577 0.007 -0.048 0.333 1 1 1.071 0.205 1.147 0.22 0.22 0 Degree = 2
  • 107. Machine Learning - SVM Nonlinear SVM Classification: Example ● Adding polynomial features ○ What does adding polynomial features mean ○ Let us consider another example >>> from sklearn.preprocessing import PolynomialFeatures >>> X, y = make_moons(n_samples=2, noise=0.15, random_state=42) >>> np.set_printoptions(precision=2) >>> print(X) >>> print(y) >>> poly=PolynomialFeatures(degree=3) >>> x1=poly.fit_transform(X)*100 >>> print(x1)
  • 108. Machine Learning - SVM Nonlinear SVM Classification: Example ● Adding polynomial features ○ What does adding polynomial features mean ○ Let us consider another example X = [[-0.08 0.58] [ 1.07 0.21]] y = [1 0] X1 = [[ 1. -0.08 0.58 0.01 -0.05 0.33 -0. 0. -0.03 0.19] [ 1. 1.07 0.21 1.15 0.22 0.04 1.23 0.24 0.05 0.01]]
  • 109. Machine Learning - SVM Nonlinear SVM Classification: Example ● MOONS Dataset ○ Q. How to classify this using linear classifier? ○ Ans: Added more features as polynomial features
  • 110. Machine Learning - SVM Nonlinear SVM Classification: Example ● MOONS Dataset ○ Add more features with degree 3 ○ Scale the new features using StandardScaler() ○ Use SVM Classifier ● All the above steps can be performed in a single iteration using a Pipeline
  • 111. Machine Learning - SVM Nonlinear SVM Classification: Example >>> from sklearn.pipeline import Pipeline >>> from sklearn.preprocessing import PolynomialFeatures >>> polynomial_svm_clf = Pipeline(( ("poly_features", PolynomialFeatures(degree=3)), ("scaler", StandardScaler()), ("svm_clf", LinearSVC(C=10, loss="hinge")) )) >>> polynomial_svm_clf.fit(X, y)
  • 112. Machine Learning - SVM Nonlinear SVM Classification: Example ● MOONS Dataset ○ Plotting the dataset along with the classifier (decision boundary) just modeled
  • 113. Machine Learning - SVM Nonlinear SVM Classification: Example def plot_predictions(clf, axes): x0s = np.linspace(axes[0], axes[1], 100) x1s = np.linspace(axes[2], axes[3], 100) x0, x1 = np.meshgrid(x0s, x1s) X = np.c_[x0.ravel(), x1.ravel()] y_pred = clf.predict(X).reshape(x0.shape) y_decision = clf.decision_function(X).reshape(x0.shape) plt.contourf(x0, x1, y_pred, cmap=plt.cm.brg, alpha=0.2) plt.contourf(x0, x1, y_decision, cmap=plt.cm.brg, alpha=0.1) plot_predictions(polynomial_svm_clf, [-1.5, 2.5, -1, 1.5]) plot_dataset(X, y, [-1.5, 2.5, -1, 1.5]) plt.show()
  • 114. Machine Learning - SVM Nonlinear SVM Classification: Example
  • 115. Machine Learning - SVM Switch to Notebook
  • 116. Machine Learning - SVM Linear SVM Classification Linear SVM Classification Nonlinear SVM Classification SVM Regression SVC Polynomial Kernel + Standard Scaler SVC RBF Kernel + Standard Scaler Polynomial Features + StandardScaler + LinearSVC
  • 117. Machine Learning - SVM Nonlinear SVM Classification Polynomial Kernel ● Adding polynomial features works great ○ Low polynomial degree cannot deal with complex datasets ○ High polynomial degree makes the model slow due to huge number of features ● How to overcome the slowness due to huge features? ● Ans: Polynomial Kernels or Kernel trick
  • 118. Machine Learning - SVM Nonlinear SVM Classification Polynomial Kernel ● Adding polynomial features works great ○ Low polynomial degree cannot deal with complex datasets ○ High polynomial degree makes the model slow due to huge number of features ● How to overcome the slowness due to huge features? ● Ans: Polynomial Kernels or Kernel trick ○ Makes it possible to get the same result as when using high polynomial degree ○ Without having to add the features which makes the model slow
  • 119. Machine Learning - SVM Nonlinear SVM Classification Polynomial Kernel in Scikit-Learn ● Can be implement in Scikit Learn using SVC Classifier ● Without having to use PolynomialFeatures as in LinearSVC >>> from sklearn.svm import SVC >>> poly_kernel_svm_clf = Pipeline(( ("scaler", StandardScaler()), ("svm_clf", SVC( kernel="poly", degree=3, coef0=1, C=5)))) kernel controls how much the model is influenced by high-degree polynomials vs low-degree coef0
  • 120. Machine Learning - SVM Nonlinear SVM Classification Polynomial Kernel in Scikit-Learn ● Training the classifier using higher degree of polynomial features # train SVM classifier using 10th-degree polynomial kernel (for comparison) >>> poly100_kernel_svm_clf = Pipeline(( ("scaler", StandardScaler()), ("svm_clf", SVC(kernel="poly", degree=10, coef0=100, C=5)) ))
  • 121. Machine Learning - SVM Nonlinear SVM Classification Polynomial Kernel in Scikit-Learn ● Observing the difference in the two cases
  • 122. Machine Learning - SVM Switch to Notebook
  • 123. Machine Learning - SVM Nonlinear SVM Classification Linear SVM Classification Nonlinear SVM Classification SVM Regression SVC Polynomial Kernel + Standard Scaler SVC RBF Kernel + Standard Scaler Polynomial Features + StandardScaler + LinearSVC
  • 124. Machine Learning - SVM Nonlinear SVM Classification - SVC RBF Adding similar features ● Another technique of solving nonlinear classifications ● Add features computed using a similarity function ● Similarity function measures how each instance resembles a particular ‘landmark’
  • 125. Machine Learning - SVM Nonlinear SVM Classification - SVC RBF ● Is this linearly separable? NO
  • 126. Machine Learning - SVM Nonlinear SVM Classification - SVC RBF ● Introduce landmarks - x
  • 127. Machine Learning - SVM Nonlinear SVM Classification - SVC RBF ● Calculate distance using the formula:
  • 128. Machine Learning - SVM Nonlinear SVM Classification - SVC RBF ● New features: distances from landmarks x=-2 and x=1 Original X0 (X1) Label X2 - distance from Landmark 1 X3 - distance from Landmark 2 -4 1 0.3 0 -3 1 0.74 0.01 -2 0 1 0.07 -1 0 0.74 0.3 0 0 0.3 0.74 1 0 0.07 1 2 0 0.01 0.74 3 1 0 0.3 4 1 0 0.07
  • 129. Machine Learning - SVM Nonlinear SVM Classification - SVC RBF ● Plot the new features and do linear classification
  • 130. Machine Learning - SVM Nonlinear SVM Classification - SVC RBF ● Similarity Function: Using SciKit Learn # define similarity function to be Gaussian Radial Basis Function (RBF) # equals 0 (far away) to 1 (at landmark) >>> def gaussian_rbf(x, landmark, gamma): return np.exp(-gamma * np.linalg.norm(x - landmark, axis=1)**2) >>> gamma = 0.3 >>> x1s = np.linspace(-4.5, 4.5, 200).reshape(-1, 1) >>> x2s = gaussian_rbf(x1s, -2, gamma) >>> x3s = gaussian_rbf(x1s, 1, gamma) >>> XK = np.c_[gaussian_rbf(X1D, -2, gamma), gaussian_rbf(X1D, 1, gamma)] >>> yk = np.array([0, 0, 1, 1, 1, 1, 1, 0, 0]) >>> print(XK)
  • 131. Machine Learning - SVM Nonlinear SVM Classification - SVC RBF ● Similarity Function: Using SciKit Learn ○ Upon plotting, the difference can be observed
  • 132. Machine Learning - SVM Switch to Notebook
  • 133. Machine Learning - SVM Nonlinear SVM Classification - SVC RBF Similarity Function: How to select the landmarks? ● Create a landmark at each and every instance of the dataset Drawback ● If training set is huge, number of new features added will be huge
  • 134. Machine Learning - SVM Nonlinear SVM Classification - SVC RBF ● Ideally how many new features should be added in this? Original X0 (X1) Label -4 1 -3 1 -2 0 -1 0 0 0 1 0 2 0 3 1 4 1
  • 135. Machine Learning - SVM Nonlinear SVM Classification ● Ideally how many new features should be added in this? Ans: 9 Original X0 (X1) Label -4 1 -3 1 -2 0 -1 0 0 0 1 0 2 0 3 1 4 1
  • 136. Machine Learning - SVM Nonlinear SVM Classification - SVC RBF ● Ideally how many new features should be added in this? Ans: 9 ● The training set converts into 9 instances with 9 features ● Imagine doing this with huge training datasets
  • 137. Machine Learning - SVM Nonlinear SVM Classification - SVC RBF Gaussian RBF Kernel ● Polynomial Feature addition becomes slow with higher degrees ○ Kernel trick solves it ● Similarity function becomes slow with higher number of training dataset ○ SVM kernel trick again solves the problem
  • 138. Machine Learning - SVM Nonlinear SVM Classification - SVC RBF Gaussian RBF Kernel ● It lets us to get similar results as if ○ We had added many similarity features ○ Without actually having to add them
  • 139. Machine Learning - SVM Nonlinear SVM Classification - SVC RBF Gaussian RBF Kernel in ScikitLean >>> rbf_kernel_svm_clf = Pipeline(( ("scaler", StandardScaler()), ("svm_clf", SVC(kernel="rbf", gamma=5, C=0.001)) )) >>> rbf_kernel_svm_clf.fit(X, y)
  • 140. Machine Learning - SVM Nonlinear SVM Classification - SVC RBF Gaussian RBF Kernel in ScikitLearn ● Plotting with different hyper parameters
  • 141. Machine Learning - SVM Nonlinear SVM Classification - SVC RBF Gaussian RBF Kernel in Scikit-Learn Plotting with different hyper parameters Increasing Gamma Small Gamma Makes bell curve narrower Makes the bell curve wider Reduces influence of each instance Instances have a larger range of influence Decision boundary becomes irregular Decision boundary becomes smoother
  • 142. Machine Learning - SVM Switch to Notebook
  • 143. Machine Learning - SVM Computational Complexity Which kernel to use when? 1. Linear Kernel First a. LinearSVC faster than SVC(kernel=’linear’) for large datasets with a lot of features 1. Gaussian RBF kernel 1. Other kernels: Cross validation and grid search
  • 144. Machine Learning - SVM Computational Complexity Linear SVC ● Based on liblinear library ● Scales linearly with number of instances and number of features ● Does not support kernel tricks ● Time complexity is: O(m * n)
  • 145. Machine Learning - SVM Computational Complexity m = number of training sets n = number of features SVC Class ● Based on libsvm library ● Support kernel tricks ● Time complexity is: O(m^2 * n) and O(m^3 * n) ● Dreadfully slow when the number of training sets increases ● Perfect for complex but small or medium training sets
  • 146. Machine Learning - SVM SVM Classification - Comparison LinearSVC SVC SGDClassifier Fast Slow for large datasets Perfect for small but complex training sets Does not converge as fast as LinearSVC but can be useful for datasets that do not fit in memory
  • 147. Machine Learning - SVM Linear SVM Classification Linear SVM Classification Nonlinear SVM Classification SVM Regression SVC Polynomial Kernel + Standard Scaler SVC RBF Kernel + Standard Scaler Polynomial Features + StandardScaler + LinearSVC
  • 148. Machine Learning - SVM Linear SVM Classification Linear SVM Classification Nonlinear SVM Classification SVM Regression Nonlinear SVM: SVR Polynomial Kernel + degree + C + epsilon Linear SVM: LinearSVR + Epsilon
  • 149. Machine Learning - SVM SVM Regression SVM Classifier SVM Regression Find the largest possible street between the two classes limiting margin violations Fit as many instances as possible on the street while limiting margin violations Widest possible street
  • 150. Machine Learning - SVM SVM Regression - Linear ● Width of the SVM Regression model is controlled by a hyperparameter 𝜺 or epsilon. ● Adding training instances within the margin does not affect the model’s predictions, ○ Hence model is said to be 𝜺-insensitive
  • 151. Machine Learning - SVM SVM Regression - Linear Linear Regression in Scikit-Learn: LinearSVR can be used >>> from sklearn.svm import LinearSVR >>> svm_reg = LinearSVR(epsilon=1.5) >>> svm_reg.fit(X, y)
  • 152. Machine Learning - SVM Linear SVM Regression - Example Linear SVM Regression in Scikit-Learn Step 1: Generating random numbers and making a linear relationship >>> from sklearn.svm import LinearSVR >>> import numpy.random as rnd >>> import matplotlib.pyplot as plt >>> rnd.seed(42) >>> m = 50 >>> X = 2 * rnd.rand(m,1) >>> y = (4 + 3 * X + rnd.randn(m,1)).ravel() >>> plt.scatter(X,y) >>> plt.show()
  • 153. Machine Learning - SVM Linear SVM Regression - Example Linear SVM Regression in Scikit-Learn Step 1: Generating random numbers and making a linear relationship
  • 154. Machine Learning - SVM Linear SVM Regression - Example Linear SVM Regression in Scikit-Learn Step 2: Fitting a linear Support Vector Regression model to the data >>> from sklearn.svm import LinearSVR >>> svm_reg1 = LinearSVR(epsilon=1.5) >>> svm_reg1.fit(X,y) >>> x1s = np.linspace(0,2,100) >>> y1s = svm_reg1.coef_*x1s + svm_reg1.intercept_ >>> plt.scatter(X,y) >>> plt.plot(x1s, y1s) >>> plt.show()
  • 155. Machine Learning - SVM Linear SVM Regression - Example Linear SVM Regression in Scikit-Learn Step 2: Fitting a linear Support Vector Regression model to the data
  • 156. Machine Learning - SVM Linear SVM Regression - Example Linear SVM Regression in Scikit-Learn Step 3: Plotting the epsilon lines >>> y1s_eps1 = y1s + 1.5 >>> y1s_eps2 = y1s - 1.5 >>> plt.scatter(X,y) >>> plt.plot(x1s, y1s) >>> plt.plot(x1s, y1s_eps1,'k--') >>> plt.plot(x1s, y1s_eps2,'k--') >>> plt.xlabel(r"$x_1$", fontsize=18) >>> plt.ylabel(r"$y$", fontsize=18) >>> plt.title('eps = 1.5') >>> plt.show()
  • 157. Machine Learning - SVM Linear SVM Regression - Example Linear SVM Regression in Scikit-Learn Step 3: Plotting the epsilon lines
  • 158. Machine Learning - SVM Linear SVM Regression - Example Linear SVM Regression in Scikit-Learn Step 4: Finding the instances off-the-street and plotting >>> y_pred = svm_reg1.predict(X) >>> supp_vec_X = X[np.abs(y-y_pred)>1.5] >>> supp_vec_y = y[np.abs(y-y_pred)>1.5] >>> plt.scatter(supp_vec_X,supp_vec_y) >>> plt.show()
  • 159. Machine Learning - SVM Linear SVM Regression - Example Linear SVM Regression in Scikit-Learn Step 4: Finding the instances off-the-street and plotting
  • 160. Machine Learning - SVM Switch to Notebook
  • 161. Machine Learning - SVM Linear SVM Regression - Example Linear SVM Regression in Scikit-Learn with eps = 0.5 Step 1: Generating random numbers and making a linear relationship >>> from sklearn.svm import LinearSVR >>> import numpy.random as rnd >>> import matplotlib.pyplot as plt >>> rnd.seed(42) >>> m = 50 >>> X = 2 * rnd.rand(m,1) >>> y = (4 + 3 * X + rnd.randn(m,1)).ravel() >>> plt.scatter(X,y) >>> plt.show()
  • 162. Machine Learning - SVM Linear SVM Regression - Example Linear SVM Regression in Scikit-Learn Step 1: Generating random numbers and making a linear relationship
  • 163. Machine Learning - SVM Linear SVM Regression - Example Linear SVM Regression in Scikit-Learn Step 2: Fitting a linear Support Vector Regression model to the data >>> from sklearn.svm import LinearSVR >>> svm_reg1 = LinearSVR(epsilon = 0.5) >>> svm_reg1.fit(X,y) >>> x1s = np.linspace(0,2,100) >>> y1s = svm_reg1.coef_*x1s + svm_reg1.intercept_ >>> plt.scatter(X,y) >>> plt.plot(x1s, y1s) >>> plt.show()
  • 164. Machine Learning - SVM Linear SVM Regression - Example Linear SVM Regression in Scikit-Learn Step 2: Fitting a linear Support Vector Regression model to the data
  • 165. Machine Learning - SVM Linear SVM Regression - Example Linear SVM Regression in Scikit-Learn Step 3: Plotting the epsilon lines >>> y1s_eps1 = y1s + 0.5 >>> y1s_eps2 = y1s - 0.5 >>> plt.scatter(X,y) >>> plt.plot(x1s, y1s) >>> plt.plot(x1s, y1s_eps1,'k--') >>> plt.plot(x1s, y1s_eps2,'k--') >>> plt.xlabel(r"$x_1$", fontsize=18) >>> plt.ylabel(r"$y$", fontsize=18) >>> plt.title('eps = 1.5') >>> plt.show()
  • 166. Machine Learning - SVM Linear SVM Regression - Example Linear SVM Regression in Scikit-Learn Step 3: Plotting the epsilon lines
  • 167. Machine Learning - SVM Linear SVM Regression - Example Linear SVM Regression in Scikit-Learn Step 4: Finding the instances off-the-street and plotting >>> y_pred = svm_reg1.predict(X) >>> supp_vec_X = X[np.abs(y-y_pred)>0.5] >>> supp_vec_y = y[np.abs(y-y_pred)>0.5] >>> plt.scatter(supp_vec_X,supp_vec_y) >>> plt.show()
  • 168. Machine Learning - SVM Linear SVM Regression - Example Linear SVM Regression in Scikit-Learn Step 4: Finding the instances off-the-street and plotting
  • 169. Machine Learning - SVM Switch to Notebook
  • 170. Machine Learning - SVM Linear SVM Regression - Example Linear SVM Regression in Scikit-Learn Comparison for epsilon = 0.5 and epsilon = 1.5, observations?
  • 171. Machine Learning - SVM Switch to Notebook
  • 172. Machine Learning - SVM Linear SVM Regression - Example ● Linear SVM Regression in Scikit-Learn ○ Comparison for eps = 0.5 and eps = 1.5, observations? ■ Number of instances off-the-street are higher for eps=0.5 ○ Cannot conclude on which is a better model Remember: the goal is to maximise the number of training sets within the epsilon line
  • 173. Machine Learning - SVM Linear SVM Classification Linear SVM Classification Nonlinear SVM Classification SVM Regression Nonlinear SVM: SVR Polynomial Kernel + degree + C + epsilon Linear SVM: LinearSVR + Epsilon
  • 174. Machine Learning - SVM SVM Nonlinear Regression ● A ‘kernelized’ SVM Regression model can be used
  • 175. Machine Learning - SVM >>> from sklearn.svm import SVR >>> svm_poly_reg = SVR(kernel="poly", degree=2, C=100, epsilon=0.1) >>> svm_poly_reg.fit(X, y) SVM Nonlinear Regression ● A ‘kernelized’ SVM Regression model can be used ● C - penalty for being outside the margin or error in classification ● Higher C -> Classification: lesser violations, Regression: lesser regularization ● Lower C -> Classification: more violations, Regression: more regularization epsilon = margin parameter C = regularization parameters
  • 176. Machine Learning - SVM SVM Nonlinear Regression - Example 1 Nonlinear SVM Regression in Scikit-Learn for a quadratic distributed data
  • 177. Machine Learning - SVM Nonlinear SVM Regression in Scikit-Learn Step 1: Generating random numbers and making a quadratic relationship >>> from sklearn.svm import SVR >>> import numpy.random as rnd >>> import matplotlib.pyplot as plt >>> rnd.seed(42) >>> m = 100 >>> X = 2 * rnd.rand(m,1) -1 >>> y = (0.2 + 0.1 * X + 0.5 * X**2 + rnd.randn(m, 1)/10).ravel() >>> plt.scatter(X,y) >>> plt.show() SVM Nonlinear Regression - Example 1
  • 178. Machine Learning - SVM Nonlinear SVM Regression in Scikit-Learn Step 1: Generating random numbers and making a quadratic relationship SVM Nonlinear Regression - Example 1
  • 179. Machine Learning - SVM Nonlinear SVM Regression in Scikit-Learn Step 2: Fitting a Support Vector Regression model (degree=2) to the data >>> from sklearn.svm import SVR >>> svr_poly_reg1 = SVR(kernel="poly", degree=2, C = 100, epsilon = 0.1) >>> svr_poly_reg1.fit(X,y) >>> print(svr_poly_reg1.C) >>> x1s = np.linspace(-1,1,200) >>> plot_svm_regression(svr_poly_reg1, X, y, [-1, 1, 0, 1]) SVM Nonlinear Regression - Example 1
  • 180. Machine Learning - SVM Nonlinear SVM Regression in Scikit-Learn Step 2: Fitting a Support Vector Regression model (degree=2) to the data SVM Nonlinear Regression - Example 1
  • 181. Machine Learning - SVM Nonlinear SVM Regression in Scikit-Learn Step 3: Plotting the epsilon lines >>> y1s_eps1 = y1s + 0.1 >>> y1s_eps2 = y1s - 0.1 >>> plt.scatter(X,y) >>> plt.plot(x1s, y1s) >>> plt.plot(x1s, y1s_eps1,'k--') >>> plt.plot(x1s, y1s_eps2,'k--') >>> plt.xlabel(r"$x_1$", fontsize=18) >>> plt.ylabel(r"$y$", fontsize=18) >>> plt.title('eps = 1.5') >>> plt.show() SVM Nonlinear Regression - Example 1
  • 182. Machine Learning - SVM Nonlinear SVM Regression in Scikit-Learn Step 3: Plotting the epsilon lines SVM Nonlinear Regression - Example 1
  • 183. Machine Learning - SVM Nonlinear SVM Regression in Scikit-Learn Step 4: Finding the instances off-the-street and plotting >>> y1_predict = svr_poly_reg1.predict(X) >>> supp_vectors_X = X[np.abs(y-y1_predict)>0.1] >>> supp_vectors_y = y[np.abs(y-y1_predict)>0.1] >>> plt.scatter(supp_vectors_X ,supp_vectors_y) >>> plt.show() SVM Nonlinear Regression - Example 1
  • 184. Machine Learning - SVM Nonlinear SVM Regression in Scikit-Learn Step 4: Finding the instances off-the-street and plotting SVM Nonlinear Regression - Example 1
  • 185. Machine Learning - SVM Switch to Notebook
  • 186. Machine Learning - SVM SVM Nonlinear Regression - Comparison
  • 187. Machine Learning - SVM Switch to Notebook
  • 188. Machine Learning - SVM SVM Nonlinear Regression - Comparison The model as calculated with different hyper-parameters can be observed - Higher eps: Less number of instances off-the-street - Higher C: Less number of instances off-the-street However, higher eps and lesser number of violations does not always imply a better model. Similarly, higher C can lead to overfitting
  • 189. Machine Learning - SVM SVM Classification Summary Linear Classification - Bad-model versus good-model: large-margin classification - SVM Sensitivity to feature scaling - Hard margin versus Soft margin classification Nonlinear SVM Classification - Adding polynomial features and solving using kernel trick - Adding similarity features - Gaussian RBF function and kernel trick Computational comparison for SVC, SVCLinear and SGDClassifier
  • 190. Machine Learning - SVM SVM Regression Summary SVM Regression (Linear and Non Linear) - SVM Linear Regression using LinearSVR and controlling the width of the margin using epsilon - Using Kernel-ized SVM Regression model to model non-linear models - SVR with kernel, StandardScaler
  • 191. Machine Learning - SVM How do SVMs work? - Under the Hood
  • 192. Machine Learning - SVM Linear SVM - Decision Functions ● Let petal width be denoted by x1 and petal length be denoted by x2. ● Decision Function ‘h’ can be defined as w1 * x1 + w2 * x2 + b. ○ If h < 0, then class=0, else class =1. ● It can be represented by the equation below
  • 193. Machine Learning - SVM Linear SVM - Decision Functions
  • 194. Machine Learning - SVM Linear SVM - Decision Functions Training the SVM Classifier would mean: ● Finding w and b such that ● Margin is as wide as possible while ● Avoiding margin violations (hard margin) or ● Limiting them (soft margin)
  • 195. Machine Learning - SVM Linear SVM - Decision Functions Training the SVM Classifier would mean: ● Finding w and b such that ● Margin is as wide as possible while ● Avoiding margin violations (hard margin) or ● Limiting them (soft margin) Q. Remember hard margin and soft margin?
  • 196. Machine Learning - SVM Linear SVM - Decision Functions Training the SVM Classifier would mean: ● Finding w and b such that ● Margin is as wide as possible while ● Avoiding margin violations (hard margin) or ● Limiting them (soft margin) Q. How do we achieve the above?
  • 197. Machine Learning - SVM Linear SVM - Decision Functions Training the SVM Classifier would mean: ● Finding w and b such that ● Margin is as wide as possible while ● Avoiding margin violations (hard margin) or ● Limiting them (soft margin) Q. How do we achieve the above? ● Optimization
  • 198. Machine Learning - SVM Linear SVM - Decision Functions What do we know? - For a 2d dataset, the slope of the margin is equal to w = [w1, w2] and the slope of the decision boundary is equal to w1^2 + w2^2 - For a n-dimensional dataset, w = [w1, w2, w3, ... , wn] and the slope of the decision function is denoted by || w ||
  • 199. Machine Learning - SVM Linear SVM - Decision Functions What we also know, - Smaller the weight vector, larger is the margin
  • 200. Machine Learning - SVM Linear SVM - Decision Functions - So, in order to achieve the best classifier - we can minimize || w || to maximize the margin, can we?
  • 201. Machine Learning - SVM Linear SVM - Decision Functions - So, in order to achieve the best classifier - we can minimize || w || to maximize the margin, can we? - No - For hard margin, we need to ensure - Decision function > 1 for all positive training instances - Decision function < -1 for all negative training instances
  • 202. Machine Learning - SVM Linear SVM - Decision Functions So, the problem basically becomes: Where t(i) =1 for positive instances and t(i) = -1 for negative instances
  • 203. Machine Learning - SVM Linear SVM - Decision Functions - So, in order to achieve the best classifier - we can minimize || w || to maximize the margin, can we? - No - For soft margin, we need to include a slack variable to the minimization equation - Two conflicting goals: - Minimize the weights matrix to maximize the margin - Minimize the slack variable to reduce margin violations - C hyper-parameter allows us to define the trade-off between the two
  • 204. Machine Learning - SVM Linear SVM - Decision Functions So, the problem for soft-margin basically becomes: Where t(i) =1 for positive instances and t(i) = -1 for negative instances
  • 205. Machine Learning - SVM Linear SVM - Decision Functions Both hard-margin and soft-margin problems are - Convex quadratic problems with - Linear constraints Such problems are known as Quadratic Programming (QP) problems - Can be solved using off-the-shelf solvers - Using variety of techniques - We will not discuss this in the session
  • 206. Machine Learning - SVM Linear SVM - Decision Functions So now we know that the hard-margin and soft-margin classifiers - Is an optimization problem to - minimize the cost - given certain constraints - The optimization is a quadratic programming (QP) problem - Which is solved using off-the-shelf solver - Basically, the classifier function is calling a QP solver in the backend to calculate the weights of the decision boundary
  • 207. Machine Learning - SVM Dual Problem The original constrained optimization problem , known as the primal problem, can be expressed as another closely related problem known as dual problem
  • 208. Machine Learning - SVM Dual Problem Dual problem gives a lower bound to the solution of the primal problem, but under some circumstances gives the same result. - SVM problems meet these conditions, hence have same solution for both primal and dual problems.
  • 209. Machine Learning - SVM Dual Problem Can be expressed as Primal problem Dual problem
  • 210. Machine Learning - SVM Dual Problem Solution from the above dual problem can be transformed to the solution of the original primal problem using:
  • 211. Machine Learning - SVM Dual Problem Primal Problem Dual Problem Slow to solve Faster to solve than the primal when the number of training instances are smaller than the number of features Kernel trick is not possible Hence, making the kernel trick possible
  • 212. Machine Learning - SVM Kernelized SVM - When did we use SVM Kernel? - Review (Slides 91 to 125)
  • 213. Machine Learning - SVM Kernelized SVM When do we use kernelized SVMs? - We applied a 2nd degree polynomial transformation - And then train a linear SVM classifier on the transformed training set
  • 214. Machine Learning - SVM Kernelized SVM The 2nd-degree polynomial transformed set is 3-dimensional instead of two-dimensional. (dropping the initial features)
  • 215. Machine Learning - SVM Kernelized SVM If there are two sets of 2-dimensional feature sets, a and b. We apply 2nd degree polynomial mapping and then compute the dot product of the transformed vectors? - Why do we do this? - The dual problem requires dot product of feature sets
  • 216. Machine Learning - SVM Kernelized SVM - The dot product of transformed vectors - Is equal to the square of the dot product of the original vectors
  • 217. Machine Learning - SVM Kernelized SVM - Each degree transformation requires a lot of computation - Dual problem shall contain dot product of the transformed features matrix - Instead, the original feature can be dot-multiplied and squared - Transformation of the original matrix is not required - The above trick makes the whole process much more computationally efficient
  • 218. Machine Learning - SVM Kernelized SVM - Kernel function represented by K - Capable of computing transformed based only on the original vectors without having to compute the transformation.
  • 219. Machine Learning - SVM Online SVMs What is Online Learning? Recap: incremental learning of the model as data gathers more datasets on the go
  • 220. Machine Learning Machine Learning - Online Learning
  • 221. Machine Learning Machine Learning - Online Learning ● Train system incrementally ○ By feeding new data sequentially ○ Or in batches ● System can learn from new data on the fly ● Good for systems where data is a continuous flow ○ Stock prices
  • 222. Machine Learning Machine Learning - Online Learning Using online learning to handle huge datasets
  • 223. Machine Learning Machine Learning - Online Learning Using online learning to handle huge datasets ● Can be used to train huge datasets ○ That can not be fit in one machine ○ The training data gets divided into batches and ○ System gets trained on each batch incrementally
  • 224. Machine Learning Machine Learning - Online Learning Challenges in online learning ● System’s performance gradually declines ○ If bad data is fed to the system ○ Bad data can come from ○ Malfunctioning sensor or robot ○ Someone spamming your system
  • 225. Machine Learning Machine Learning - Online Learning Challenges in online learning ● Closely monitor the system ○ Turn off the learning if there is a performance drop ○ Or monitor the input data and remove anomalies
  • 226. Machine Learning Machine Learning - Online Learning Challenges in online learning ● Closely monitor the system ○ Turn off the learning if there is a performance drop ○ Or monitor the input data and remove anomalies
  • 227. Machine Learning Machine Learning - Online Learning ● Can be implemented using Linear SVM classifiers ○ One method is Gradient Descent, e.g SGDClassifier ○ Covered previously in Chapter 3 and earlier in SVM Classification ● Cost function for SGD Classification can be written as Maximizes the margin Penalty function for wrong classification Hinge loss
  • 228. Machine Learning Machine Learning - Online Learning ● Online Learning can also be implemented using Kernelized SVMs ○ Currently implemented in Matlab and CPP ○ For large scale nonlinear problems, we should also consider using neural networks which will be covered in ANN course.
  • 230. Machine Learning - SVM Kernelized SVM
  • 231. Machine Learning - SVM Linear SVM Classification Linear SVM Classification Nonlinear SVM Classification SVM Regression