In this tutorial, we will learn the the following topics -
+ Linear SVM Classification
+ Soft Margin Classification
+ Nonlinear SVM Classification
+ Polynomial Kernel
+ Adding Similarity Features
+ Gaussian RBF Kernel
+ Computational Complexity
+ SVM Regression
This Naive Bayes Classifier tutorial presentation will introduce you to the basic concepts of Naive Bayes classifier, what is Naive Bayes and Bayes theorem, conditional probability concepts used in Bayes theorem, where is Naive Bayes classifier used, how Naive Bayes algorithm works with solved examples, advantages of Naive Bayes. By the end of this presentation, you will also implement Naive Bayes algorithm for text classification in Python.
The topics covered in this Naive Bayes presentation are as follows:
1. What is Naive Bayes?
2. Naive Bayes and Machine Learning
3. Why do we need Naive Bayes?
4. Understanding Naive Bayes Classifier
5. Advantages of Naive Bayes Classifier
6. Demo - Text Classification using Naive Bayes
- - - - - - - -
Simplilearn’s Machine Learning course will make you an expert in Machine Learning, a form of Artificial Intelligence that automates data analysis to enable computers to learn and adapt through experience to do specific tasks without explicit programming. You will master Machine Learning concepts and techniques including supervised and unsupervised learning, mathematical and heuristic aspects, hands-on modeling to develop algorithms and prepare you for the role of Machine Learning Engineer
Why learn Machine Learning?
Machine Learning is rapidly being deployed in all kinds of industries, creating a huge demand for skilled professionals. The Machine Learning market size is expected to grow from USD 1.03 billion in 2016 to USD 8.81 billion by 2022, at a Compound Annual Growth Rate (CAGR) of 44.1% during the forecast period.
You can gain in-depth knowledge of Machine Learning by taking our Machine Learning certification training course. With Simplilearn’s Machine Learning course, you will prepare for a career as a Machine Learning engineer as you master concepts and techniques including supervised and unsupervised learning, mathematical and heuristic aspects, and hands-on modeling to develop algorithms. Those who complete the course will be able to:
1. Master the concepts of supervised, unsupervised and reinforcement learning concepts and modeling.
2. Gain practical mastery over principles, algorithms, and applications of Machine Learning through a hands-on approach which includes working on 28 projects and one capstone project.
3. Acquire thorough knowledge of the mathematical and heuristic aspects of Machine Learning.
4. Understand the concepts and operation of support vector machines, kernel SVM, Naive Bayes, decision tree classifier, random forest classifier, logistic regression, K-nearest neighbors, K-means clustering and more.
- - - - - - - -
The document discusses the random forest algorithm. It introduces random forest as a supervised classification algorithm that builds multiple decision trees and merges them to provide a more accurate and stable prediction. It then provides an example pseudocode that randomly selects features to calculate the best split points to build decision trees, repeating the process to create a forest of trees. The document notes key advantages of random forest are that it avoids overfitting and can be used for both classification and regression tasks.
Ensemble methods combine multiple machine learning models to obtain better predictive performance than from any individual model. There are two main types of ensemble methods: sequential (e.g AdaBoost) where models are generated one after the other, and parallel (e.g Random Forest) where models are generated independently. Popular ensemble methods include bagging, boosting, and stacking. Bagging averages predictions from models trained on random samples of the data, while boosting focuses on correcting previous models' errors. Stacking trains a meta-model on predictions from other models to produce a final prediction.
This document provides an overview of support vector machines (SVMs). It discusses how SVMs can be used to perform classification tasks by finding optimal separating hyperplanes that maximize the margin between different classes. The document outlines how SVMs solve an optimization problem to find these optimal hyperplanes using techniques like Lagrange duality, kernels, and soft margins. It also covers model selection methods like cross-validation and discusses extensions of SVMs to multi-class classification problems.
This document discusses various classification algorithms including k-nearest neighbors, decision trees, naive Bayes classifier, and logistic regression. It provides examples of how each algorithm works. For k-nearest neighbors, it shows how an unknown data point would be classified based on its nearest neighbors. For decision trees, it illustrates how a tree is built by splitting the data into subsets at each node until pure subsets are reached. It also provides an example decision tree to predict whether Amit will play cricket. For naive Bayes, it gives an example of calculating the probability of cancer given a patient is a smoker.
In machine learning, support-vector machines (SVMs, also support-vector networks) are supervised learning models with associated learning algorithms that analyze data for classification and regression analysis
Slide explaining the distinction between bagging and boosting while understanding the bias variance trade-off. Followed by some lesser known scope of supervised learning. understanding the effect of tree split metric in deciding feature importance. Then understanding the effect of threshold on classification accuracy. Additionally, how to adjust model threshold for classification in supervised learning.
Note: Limitation of Accuracy metric (baseline accuracy), alternative metrics, their use case and their advantage and limitations were briefly discussed.
Abstract: This PDSG workshop introduces basic concepts of ensemble methods in machine learning. Concepts covered are Condercet Jury Theorem, Weak Learners, Decision Stumps, Bagging and Majority Voting.
Level: Fundamental
Requirements: No prior programming or statistics knowledge required.
The document provides an overview of the Naive Bayes algorithm for classification problems. It begins by explaining that Naive Bayes is a supervised learning algorithm based on Bayes' theorem. It then explains the key aspects of Naive Bayes:
- It assumes independence between features (naive) and uses Bayes' theorem to calculate probabilities (Bayes).
- Bayes' theorem is used to calculate the probability of a hypothesis given observed data.
- An example demonstrates how Naive Bayes classifies weather data to predict whether to play or not play.
The document concludes by discussing the advantages, disadvantages, applications, and types of Naive Bayes models, as well as providing Python code to implement a Naive Bayes classifier.
Loss Functions for Deep Learning - Javier Ruiz Hidalgo - UPC Barcelona 2018
https://telecombcn-dl.github.io/2018-dlai/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks or Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles of deep learning from both an algorithmic and computational perspectives.
Supervised and Unsupervised Learning In Machine Learning | Machine Learning T...
This document provides an overview of machine learning, including:
- Machine learning allows computers to learn from data without being explicitly programmed, through processes like analyzing data, training models on past data, and making predictions.
- The main types of machine learning are supervised learning, which uses labeled training data to predict outputs, and unsupervised learning, which finds patterns in unlabeled data.
- Common supervised learning tasks include classification (like spam filtering) and regression (like weather prediction). Unsupervised learning includes clustering, like customer segmentation, and association, like market basket analysis.
- Supervised and unsupervised learning are used in many areas like risk assessment, image classification, fraud detection, customer analytics, and more
An Introduction to Supervised Machine Learning and Pattern Classification: Th...
The document provides an introduction to supervised machine learning and pattern classification. It begins with an overview of the speaker's background and research interests. Key concepts covered include definitions of machine learning, examples of machine learning applications, and the differences between supervised, unsupervised, and reinforcement learning. The rest of the document outlines the typical workflow for a supervised learning problem, including data collection and preprocessing, model training and evaluation, and model selection. Common classification algorithms like decision trees, naive Bayes, and support vector machines are briefly explained. The presentation concludes with discussions around choosing the right algorithm and avoiding overfitting.
Support Vector Machine (SVM) is a supervised machine learning algorithm that can be used for both classification and regression analysis. It works by finding a hyperplane in an N-dimensional space that distinctly classifies the data points. SVM selects the hyperplane that has the largest distance to the nearest training data points of any class, since larger the margin lower the generalization error of the classifier. SVM can efficiently perform nonlinear classification by implicitly mapping their inputs into high-dimensional feature spaces.
Support vector machines (SVMs) are a type of supervised machine learning algorithm used for classification and regression. SVMs learn by finding the optimal separating hyperplane that maximizes the margin between two classes of objects. SVMs can efficiently perform nonlinear classification using kernel methods. Key features of SVMs include their ability to handle high dimensional data, use different kernel functions for nonlinear classification, and avoid overfitting.
ensemble methods use multiple learning algorithms to obtain better predictive performance than could be obtained from any of the constituent learning algorithms alone.
An ensemble is itself a supervised learning algorithm, because it can be trained and then used to make predictions. The trained ensemble, therefore, represents a single hypothesis. This hypothesis, however, is not necessarily contained within the hypothesis space of the models from which it is built.
Logistic regression is a predictive analysis algorithm that can be used for classification problems. It estimates the probabilities of different classes using the logistic function, which outputs values between 0 and 1. Logistic regression transforms its output using the sigmoid function to return a probability value. It is used for problems like email spam detection, fraud detection, and tumor classification. The independent variables should be independent of each other and the dependent variable must be categorical. Gradient descent is used to minimize the loss function and optimize the model parameters during training.
▸ Machine Learning / Deep Learning models require to set the value of many hyperparameters
▸ Common examples: regularization coefficients, dropout rate, or number of neurons per layer in a Neural Network
▸ Instead of relying on some "expert advice", this presentation shows how to automatically find optimal hyperparameters
▸ Exhaustive Search, Monte Carlo Search, Bayesian Optimization, and Evolutionary Algorithms are explained with concrete examples
The document discusses machine learning classification using the MNIST dataset of handwritten digits. It begins by defining classification and providing examples. It then describes the MNIST dataset and how it is fetched in scikit-learn. The document outlines the steps of classification which include dividing the data into training and test sets, training a classifier on the training set, testing it on the test set, and evaluating performance. It specifically trains a stochastic gradient descent (SGD) classifier on the MNIST data. The performance is evaluated using cross validation accuracy, confusion matrix, and metrics like precision and recall.
This document outlines an introductory machine learning course, covering key concepts, applications, and types of machine learning like supervised and unsupervised learning. It discusses techniques like linear regression, classification, and handling overfitting. The course will include tutorials on sentiment analysis, spam filtering, stock prediction, image recognition and recommendation engines using Python and Scala. Later classes cover machine learning at scale using tools like Spark MLLib.
This second meetup will be about training different models for our recommender system. We will review the simple models we can build as a baseline. After that, we will present the recommender system as an optimization problem and discuss different training losses. We will mention linear models and matrix factorization techniques. We will end the presentation with a simple introduction to non-linear models and deep learning.
This document summarizes a session on classification models. It discusses classifying data by learning classification rules from labeled input data. Specifically, it covers k-nearest neighbors classification, where the target point is assigned the majority class of its k nearest neighbors. It also discusses extending linear regression to classification by using it to generate real-valued outputs, then thresholding those values to classify instances into classes 0 or 1. However, it notes that logistic regression is a better approach, using a sigmoid function to convert real-valued outputs to a probability between 0 and 1 before thresholding for classification. The document highlights key considerations for classification like dealing with multiple classes and class imbalance. It also discusses evaluating classifiers, ensemble methods, and overfitting/under
Text Classification with Lucene/Solr, Apache Hadoop and LibSVM
In this session we will show how to build a text classifier using the Apache Lucene/Solr with libSVM libraries. We classify our corpus of job offers into a number of predefined categories. Each indexed document (a job offer) then belongs to zero, one or more categories. Known machine learning techniques for text classification include naïve bayes model, logistic regression, neural network, support vector machine (SVM), etc. We use Lucene/Solr to construct the features vector. Then we use the libsvm library known as the reference implementation of the SVM model to classify the document. We construct as many one-vs-all svm classifiers as there are classes in our setting, then using the Hadoop MapReduce Framework we reconcile the result of our classifiers. The end result is a scalable multi-class classifier. Finally we outline how the classifier is used to enrich basic solr keyword search.
This document discusses scaling machine learning algorithms to large datasets. It introduces machine learning and linear regression, explaining how to represent data with features and labels. Linear regression finds the line of best fit by minimizing a squared loss function. Computing the closed-form solution is inefficient for large datasets due to expensive matrix operations. Gradient descent is introduced as an iterative alternative that is easily parallelized. It works by taking steps in the direction of steepest descent until converging to a solution.
This document provides an overview of support vector machines (SVM) including:
1) Maximal margin classifiers which find the optimal separating hyperplane with the maximum margin between classes. Support vectors are the data points that determine the hyperplane.
2) Support vector classifiers which allow some misclassified data points by introducing slack variables. This makes the classifier more robust.
3) SVM can handle non-linear decision boundaries using kernel methods to map data into higher dimensional feature spaces where a linear separator can be found. Common kernels include linear, polynomial and radial basis function kernels.
4) Multi-class classification with SVM can be done with one-vs-one or one-vs-all approaches.
5) SVM
With the growing trend of machine learning, it is needless to say how machine learning can help reap benefits in agriculture. It will be boon for the farmer welfare.
Computer vision is a branch of computer science which deals with recognising objects, people and identifying patterns in visuals. It is basically analogous to the vision of an animal.
Topics covered:
1. Overview of Machine Learning
2. Basics of Deep Learning
3. What is computer vision and its use-cases?
4. Various algorithms used in Computer Vision (mostly CNN)
5. Live hands-on demo of either Auto Cameraman or Face recognition system
6. What next?
Topics covered in the Webinar
1. Overview of Machine Learning
2. Basics of Deep Learning
3. What is computer vision and its use-cases?
4. Various algorithms used in Computer Vision (mostly CNN)
5. Live hands-on demo of either Auto Cameraman or Face recognition system
6. What next?
Presented by Sandeep Giri
www.cloudxlab.com
Computer vision is a branch of computer science which deals with recognising objects, people and identifying patterns in visuals. It is basically analogous to the vision of an animal.
Topics covered:
1. Overview of Machine Learning
2. Basics of Deep Learning
3. What is computer vision and its use-cases?
4. Various algorithms used in Computer Vision (mostly CNN)
5. Live hands-on demo of either Auto Cameraman or Face recognition system
6. What next?
This document discusses machine learning algorithms for classification problems, specifically logistic regression. It explains that logistic regression predicts the probability of a binary outcome using a sigmoid function. Unlike linear regression, which is used for continuous outputs, logistic regression is used for classification problems where the output is discrete/categorical. It describes how logistic regression learns model parameters through gradient descent optimization of a likelihood function to minimize error. Regularization techniques can also be used to address overfitting issues that may arise from limited training data.
Support Vector Machines USING MACHINE LEARNING HOW IT WORKS
This document discusses support vector machines (SVM), a supervised machine learning algorithm used for classification and regression. It explains that SVM finds the optimal boundary, known as a hyperplane, that separates classes with the maximum margin. When data is not linearly separable, kernel functions can transform the data into a higher-dimensional space to make it separable. The document discusses SVM for both linearly separable and non-separable data, kernel functions, hyperparameters, and approaches for multiclass classification like one-vs-one and one-vs-all.
Basic machine learning background with Python scikit-learn
This document provides an overview of machine learning and the Python scikit-learn library. It introduces key machine learning concepts like classification, linear models, support vector machines, decision trees, bagging, boosting, and clustering. It also demonstrates how to perform tasks like SVM classification, decision tree modeling, random forest, principal component analysis, and k-means clustering using scikit-learn. The document concludes that scikit-learn can handle large datasets and recommends Keras for deep learning.
Machine Learning : why we should know and how it works
This document provides an overview of machine learning, including:
- An introduction to machine learning and why it is important.
- The main types of machine learning algorithms: supervised learning, unsupervised learning, and deep neural networks.
- Examples of how machine learning algorithms work, such as logistic regression, support vector machines, and k-means clustering.
- How machine learning is being applied in various industries like healthcare, commerce, and more.
This document discusses online advertising and techniques for fitting large-scale models to advertising data. It outlines batch and online algorithms for logistic regression, including parallelizing existing batch algorithms and stochastic gradient descent. The document also discusses using alternating direction method of multipliers and follow the proximal regularized leader to fit models to large datasets across multiple machines. It provides examples of how major companies like LinkedIn and Facebook implement hybrid online-batch algorithms at large scale.
Machine learning techniques can be applied in formal verification in several ways:
1) To enhance current formal verification tools by automating tasks like debugging, specification mining, and theorem proving.
2) To enable the development of new formal verification tools by applying machine learning to problems like SAT solving, model checking, and property checking.
3) Specific applications include using machine learning for debugging and root cause identification, learning specifications from runtime traces, aiding theorem proving by selecting heuristics, and tuning SAT solver parameters and selection.
Hadoop France meetup Feb2016 : recommendations with spark
Système de recommandations de produits sur un site marchand par Koby KARP, Data Scientist (Equancy) & Hervé MIGNOT, Partner at Equancy
La recommandation reste un outil clé pour la personnalisation des sites marchands et le sujet est loin d’être épuisé. La prise en compte de la particularité d’un marché peut nécessité d’adapter le traitement et les algorithmes utilisés. Après une revue des techniques de recommandations, nous présenterons la démarche spécifique que nous avons adopté. Le système a été développé sous Spark pour la préparation des données et le calcul des modèles de recommandations. Une API simple et son service ont été développé pour délivrer les recommandations aux applications clientes.
This document provides an introduction and overview for a course on machine learning. It outlines the course structure, assignments, and expectations. The course will cover topics including linear regression, classification, model selection, and dimensionality reduction. It will teach students how to analyze data, preprocess it, extract features, train models, and evaluate model performance. The goal is for students to understand core machine learning algorithms and concepts. Required materials include an introduction to statistical learning textbook.
This document provides an agenda for an introduction to deep learning presentation. It begins with an introduction to basic AI, machine learning, and deep learning terms. It then briefly discusses use cases of deep learning. The document outlines how to approach a deep learning problem, including which tools and algorithms to use. It concludes with a question and answer section.
This document discusses recurrent neural networks (RNNs) and their applications. It begins by explaining that RNNs can process input sequences of arbitrary lengths, unlike other neural networks. It then provides examples of RNN applications, such as predicting time series data, autonomous driving, natural language processing, and music generation. The document goes on to describe the fundamental concepts of RNNs, including recurrent neurons, memory cells, and different types of RNN architectures for processing input/output sequences. It concludes by demonstrating how to implement basic RNNs using TensorFlow's static_rnn function.
Natural Language Processing (NLP) is a field of artificial intelligence that deals with interactions between computers and human languages. NLP aims to program computers to process and analyze large amounts of natural language data. Some common NLP tasks include speech recognition, text classification, machine translation, question answering, and more. Popular NLP tools include Stanford CoreNLP, NLTK, OpenNLP, and TextBlob. Vectorization is commonly used to represent text in a way that can be used for machine learning algorithms like calculating text similarity. Tf-idf is a common technique used to weigh words based on their frequency and importance.
- Naive Bayes is a classification technique based on Bayes' theorem that uses "naive" independence assumptions. It is easy to build and can perform well even with large datasets.
- It works by calculating the posterior probability for each class given predictor values using the Bayes theorem and independence assumptions between predictors. The class with the highest posterior probability is predicted.
- It is commonly used for text classification, spam filtering, and sentiment analysis due to its fast performance and high success rates compared to other algorithms.
An autoencoder is an artificial neural network that is trained to copy its input to its output. It consists of an encoder that compresses the input into a lower-dimensional latent-space encoding, and a decoder that reconstructs the output from this encoding. Autoencoders are useful for dimensionality reduction, feature learning, and generative modeling. When constrained by limiting the latent space or adding noise, autoencoders are forced to learn efficient representations of the input data. For example, a linear autoencoder trained with mean squared error performs principal component analysis.
The document discusses challenges in training deep neural networks and solutions to those challenges. Training deep neural networks with many layers and parameters can be slow and prone to overfitting. A key challenge is the vanishing gradient problem, where the gradients shrink exponentially small as they propagate through many layers, making earlier layers very slow to train. Solutions include using initialization techniques like He initialization and activation functions like ReLU and leaky ReLU that do not saturate, preventing gradients from vanishing. Later improvements include the ELU activation function.
( Machine Learning & Deep Learning Specialization Training: https://goo.gl/5u2RiS )
This CloudxLab Reinforcement Learning tutorial helps you to understand Reinforcement Learning in detail. Below are the topics covered in this tutorial:
1) What is Reinforcement?
2) Reinforcement Learning an Introduction
3) Reinforcement Learning Example
4) Learning to Optimize Rewards
5) Policy Search - Brute Force Approach, Genetic Algorithms and Optimization Techniques
6) OpenAI Gym
7) The Credit Assignment Problem
8) Inverse Reinforcement Learning
9) Playing Atari with Deep Reinforcement Learning
10) Policy Gradients
11) Markov Decision Processes
Apache Spark - Key Value RDD - Transformations | Big Data Hadoop Spark Tutori...
The document provides information about key-value RDD transformations and actions in Spark. It defines transformations like keys(), values(), groupByKey(), combineByKey(), sortByKey(), subtractByKey(), join(), leftOuterJoin(), rightOuterJoin(), and cogroup(). It also defines actions like countByKey() and lookup() that can be performed on pair RDDs. Examples are given showing how to use these transformations and actions to manipulate key-value RDDs.
Advanced Spark Programming - Part 2 | Big Data Hadoop Spark Tutorial | CloudxLab
Big Data with Hadoop & Spark Training: http://bit.ly/2kyRTuW
This CloudxLab Advanced Spark Programming tutorial helps you to understand Advanced Spark Programming in detail. Below are the topics covered in this slide:
1) Shared Variables - Accumulators & Broadcast Variables
2) Accumulators and Fault Tolerance
3) Custom Accumulators - Version 1.x & Version 2.x
4) Examples of Broadcast Variables
5) Key Performance Considerations - Level of Parallelism
6) Serialization Format - Kryo
7) Memory Management
8) Hardware Provisioning
Apache Spark - Dataframes & Spark SQL - Part 2 | Big Data Hadoop Spark Tutori...
Big Data with Hadoop & Spark Training: http://bit.ly/2sm9c61
This CloudxLab Introduction to Spark SQL & DataFrames tutorial helps you to understand Spark SQL & DataFrames in detail. Below are the topics covered in this slide:
1) Loading XML
2) What is RPC - Remote Process Call
3) Loading AVRO
4) Data Sources - Parquet
5) Creating DataFrames From Hive Table
6) Setting up Distributed SQL Engine
Apache Spark - Dataframes & Spark SQL - Part 1 | Big Data Hadoop Spark Tutori...
Big Data with Hadoop & Spark Training: http://bit.ly/2sf2z6i
This CloudxLab Introduction to Spark SQL & DataFrames tutorial helps you to understand Spark SQL & DataFrames in detail. Below are the topics covered in this slide:
1) Introduction to DataFrames
2) Creating DataFrames from JSON
3) DataFrame Operations
4) Running SQL Queries Programmatically
5) Datasets
6) Inferring the Schema Using Reflection
7) Programmatically Specifying the Schema
Apache Spark - Running on a Cluster | Big Data Hadoop Spark Tutorial | CloudxLab
(Big Data with Hadoop & Spark Training: http://bit.ly/2IUsWca
This CloudxLab Running in a Cluster tutorial helps you to understand running Spark in the cluster in detail. Below are the topics covered in this tutorial:
1) Spark Runtime Architecture
2) Driver Node
3) Scheduling Tasks on Executors
4) Understanding the Architecture
5) Cluster Managers
6) Executors
7) Launching a Program using spark-submit
8) Local Mode & Cluster-Mode
9) Installing Standalone Cluster
10) Cluster Mode - YARN
11) Launching a Program on YARN
12) Cluster Mode - Mesos and AWS EC2
13) Deployment Modes - Client and Cluster
14) Which Cluster Manager to Use?
15) Common flags for spark-submit
Introduction to SparkR | Big Data Hadoop Spark Tutorial | CloudxLab
Big Data with Hadoop & Spark Training: http://bit.ly/2LCTufA
This CloudxLab Introduction to SparkR tutorial helps you to understand SparkR in detail. Below are the topics covered in this tutorial:
1) SparkR (R on Spark)
2) SparkR DataFrames
3) Launch SparkR
4) Creating DataFrames from Local DataFrames
5) DataFrame Operation
6) Creating DataFrames - From JSON
7) Running SQL Queries from SparkR
Introduction to NoSQL | Big Data Hadoop Spark Tutorial | CloudxLab
1) NoSQL databases are non-relational and schema-free, providing alternatives to SQL databases for big data and high availability applications.
2) Common NoSQL database models include key-value stores, column-oriented databases, document databases, and graph databases.
3) The CAP theorem states that a distributed data store can only provide two out of three guarantees around consistency, availability, and partition tolerance.
Introduction to MapReduce - Hadoop Streaming | Big Data Hadoop Spark Tutorial...
Big Data with Hadoop & Spark Training: http://bit.ly/2sh5b3E
This CloudxLab Hadoop Streaming tutorial helps you to understand Hadoop Streaming in detail. Below are the topics covered in this tutorial:
1) Hadoop Streaming and Why Do We Need it?
2) Writing Streaming Jobs
3) Testing Streaming jobs and Hands-on on CloudxLab
Introduction To TensorFlow | Deep Learning Using TensorFlow | CloudxLab
This document provides instructions for getting started with TensorFlow using a free CloudxLab. It outlines the following steps:
1. Open CloudxLab and enroll if not already enrolled. Otherwise go to "My Lab".
2. In "My Lab", open Jupyter and run commands to clone an ML repository containing TensorFlow examples.
3. Go to the deep learning folder in Jupyter and open the TensorFlow notebook to get started with examples.
( Machine Learning & Deep Learning Specialization Training: https://goo.gl/goQxnL )
This CloudxLab Deep Learning tutorial helps you to understand Deep Learning in detail. Below are the topics covered in this tutorial:
1) What is Deep Learning
2) Deep Learning Applications
3) Artificial Neural Network
4) Deep Learning Neural Networks
5) Deep Learning Frameworks
6) AI vs Machine Learning
In this tutorial, we will learn the the following topics -
+ The Curse of Dimensionality
+ Main Approaches for Dimensionality Reduction
+ PCA - Principal Component Analysis
+ Kernel PCA
+ LLE
+ Other Dimensionality Reduction Techniques
In this tutorial, we will learn the the following topics -
+ Voting Classifiers
+ Bagging and Pasting
+ Random Patches and Random Subspaces
+ Random Forests
+ Boosting
+ Stacking
In this tutorial, we will learn the the following topics -
+ Training and Visualizing a Decision Tree
+ Making Predictions
+ Estimating Class Probabilities
+ The CART Training Algorithm
+ Computational Complexity
+ Gini Impurity or Entropy?
+ Regularization Hyperparameters
+ Regression
+ Instability
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - Mydbops
This presentation, delivered at the Postgres Bangalore (PGBLR) Meetup-2 on June 29th, 2024, dives deep into connection pooling for PostgreSQL databases. Aakash M, a PostgreSQL Tech Lead at Mydbops, explores the challenges of managing numerous connections and explains how connection pooling optimizes performance and resource utilization.
Key Takeaways:
* Understand why connection pooling is essential for high-traffic applications
* Explore various connection poolers available for PostgreSQL, including pgbouncer
* Learn the configuration options and functionalities of pgbouncer
* Discover best practices for monitoring and troubleshooting connection pooling setups
* Gain insights into real-world use cases and considerations for production environments
This presentation is ideal for:
* Database administrators (DBAs)
* Developers working with PostgreSQL
* DevOps engineers
* Anyone interested in optimizing PostgreSQL performance
Contact info@mydbops.com for PostgreSQL Managed, Consulting and Remote DBA Services
Kief Morris rethinks the infrastructure code delivery lifecycle, advocating for a shift towards composable infrastructure systems. We should shift to designing around deployable components rather than code modules, use more useful levels of abstraction, and drive design and deployment from applications rather than bottom-up, monolithic architecture and delivery.
Measuring the Impact of Network Latency at Twitter
Widya Salim and Victor Ma will outline the causal impact analysis, framework, and key learnings used to quantify the impact of reducing Twitter's network latency.
Choose our Linux Web Hosting for a seamless and successful online presence
Our Linux Web Hosting plans offer unbeatable performance, security, and scalability, ensuring your website runs smoothly and efficiently.
Visit- https://onliveserver.com/linux-web-hosting/
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
Six months into 2024, and it is clear the privacy ecosystem takes no days off!! Regulators continue to implement and enforce new regulations, businesses strive to meet requirements, and technology advances like AI have privacy professionals scratching their heads about managing risk.
What can we learn about the first six months of data privacy trends and events in 2024? How should this inform your privacy program management for the rest of the year?
Join TrustArc, Goodwin, and Snyk privacy experts as they discuss the changes we’ve seen in the first half of 2024 and gain insight into the concrete, actionable steps you can take to up-level your privacy program in the second half of the year.
This webinar will review:
- Key changes to privacy regulations in 2024
- Key themes in privacy and data governance in 2024
- How to maximize your privacy program in the second half of 2024
Are you interested in dipping your toes in the cloud native observability waters, but as an engineer you are not sure where to get started with tracing problems through your microservices and application landscapes on Kubernetes? Then this is the session for you, where we take you on your first steps in an active open-source project that offers a buffet of languages, challenges, and opportunities for getting started with telemetry data.
The project is called openTelemetry, but before diving into the specifics, we’ll start with de-mystifying key concepts and terms such as observability, telemetry, instrumentation, cardinality, percentile to lay a foundation. After understanding the nuts and bolts of observability and distributed traces, we’ll explore the openTelemetry community; its Special Interest Groups (SIGs), repositories, and how to become not only an end-user, but possibly a contributor.We will wrap up with an overview of the components in this project, such as the Collector, the OpenTelemetry protocol (OTLP), its APIs, and its SDKs.
Attendees will leave with an understanding of key observability concepts, become grounded in distributed tracing terminology, be aware of the components of openTelemetry, and know how to take their first steps to an open-source contribution!
Key Takeaways: Open source, vendor neutral instrumentation is an exciting new reality as the industry standardizes on openTelemetry for observability. OpenTelemetry is on a mission to enable effective observability by making high-quality, portable telemetry ubiquitous. The world of observability and monitoring today has a steep learning curve and in order to achieve ubiquity, the project would benefit from growing our contributor community.
If you’ve ever had to analyze a map or GPS data, chances are you’ve encountered and even worked with coordinate systems. As historical data continually updates through GPS, understanding coordinate systems is increasingly crucial. However, not everyone knows why they exist or how to effectively use them for data-driven insights.
During this webinar, you’ll learn exactly what coordinate systems are and how you can use FME to maintain and transform your data’s coordinate systems in an easy-to-digest way, accurately representing the geographical space that it exists within. During this webinar, you will have the chance to:
- Enhance Your Understanding: Gain a clear overview of what coordinate systems are and their value
- Learn Practical Applications: Why we need datams and projections, plus units between coordinate systems
- Maximize with FME: Understand how FME handles coordinate systems, including a brief summary of the 3 main reprojectors
- Custom Coordinate Systems: Learn how to work with FME and coordinate systems beyond what is natively supported
- Look Ahead: Gain insights into where FME is headed with coordinate systems in the future
Don’t miss the opportunity to improve the value you receive from your coordinate system data, ultimately allowing you to streamline your data analysis and maximize your time. See you there!
The DealBook is our annual overview of the Ukrainian tech investment industry. This edition comprehensively covers the full year 2023 and the first deals of 2024.
Fluttercon 2024: Showing that you care about security - OpenSSF Scorecards fo...
Have you noticed the OpenSSF Scorecard badges on the official Dart and Flutter repos? It's Google's way of showing that they care about security. Practices such as pinning dependencies, branch protection, required reviews, continuous integration tests etc. are measured to provide a score and accompanying badge.
You can do the same for your projects, and this presentation will show you how, with an emphasis on the unique challenges that come up when working with Dart and Flutter.
The session will provide a walkthrough of the steps involved in securing a first repository, and then what it takes to repeat that process across an organization with multiple repos. It will also look at the ongoing maintenance involved once scorecards have been implemented, and how aspects of that maintenance can be better automated to minimize toil.
How Social Media Hackers Help You to See Your Wife's Message.pdf
In the modern digital era, social media platforms have become integral to our daily lives. These platforms, including Facebook, Instagram, WhatsApp, and Snapchat, offer countless ways to connect, share, and communicate.
Quantum Communications Q&A with Gemini LLM. These are based on Shannon's Noisy channel Theorem and offers how the classical theory applies to the quantum world.
The integration of programming into civil engineering is transforming the industry. We can design complex infrastructure projects and analyse large datasets. Imagine revolutionizing the way we build our cities and infrastructure, all by the power of coding. Programming skills are no longer just a bonus—they’re a game changer in this era.
Technology is revolutionizing civil engineering by integrating advanced tools and techniques. Programming allows for the automation of repetitive tasks, enhancing the accuracy of designs, simulations, and analyses. With the advent of artificial intelligence and machine learning, engineers can now predict structural behaviors under various conditions, optimize material usage, and improve project planning.
Mitigating the Impact of State Management in Cloud Stream Processing Systems
Stream processing is a crucial component of modern data infrastructure, but constructing an efficient and scalable stream processing system can be challenging. Decoupling compute and storage architecture has emerged as an effective solution to these challenges, but it can introduce high latency issues, especially when dealing with complex continuous queries that necessitate managing extra-large internal states.
In this talk, we focus on addressing the high latency issues associated with S3 storage in stream processing systems that employ a decoupled compute and storage architecture. We delve into the root causes of latency in this context and explore various techniques to minimize the impact of S3 latency on stream processing performance. Our proposed approach is to implement a tiered storage mechanism that leverages a blend of high-performance and low-cost storage tiers to reduce data movement between the compute and storage layers while maintaining efficient processing.
Throughout the talk, we will present experimental results that demonstrate the effectiveness of our approach in mitigating the impact of S3 latency on stream processing. By the end of the talk, attendees will have gained insights into how to optimize their stream processing systems for reduced latency and improved cost-efficiency.
Quality Patents: Patents That Stand the Test of Time
Is your patent a vanity piece of paper for your office wall? Or is it a reliable, defendable, assertable, property right? The difference is often quality.
Is your patent simply a transactional cost and a large pile of legal bills for your startup? Or is it a leverageable asset worthy of attracting precious investment dollars, worth its cost in multiples of valuation? The difference is often quality.
Is your patent application only good enough to get through the examination process? Or has it been crafted to stand the tests of time and varied audiences if you later need to assert that document against an infringer, find yourself litigating with it in an Article 3 Court at the hands of a judge and jury, God forbid, end up having to defend its validity at the PTAB, or even needing to use it to block pirated imports at the International Trade Commission? The difference is often quality.
Quality will be our focus for a good chunk of the remainder of this season. What goes into a quality patent, and where possible, how do you get it without breaking the bank?
** Episode Overview **
In this first episode of our quality series, Kristen Hansen and the panel discuss:
⦿ What do we mean when we say patent quality?
⦿ Why is patent quality important?
⦿ How to balance quality and budget
⦿ The importance of searching, continuations, and draftsperson domain expertise
⦿ Very practical tips, tricks, examples, and Kristen’s Musts for drafting quality applications
https://www.aurorapatents.com/patently-strategic-podcast.html
To help you choose the best DiskWarrior alternative, we've compiled a comparison table summarizing the features, pros, cons, and pricing of six alternatives.
An invited talk given by Mark Billinghurst on Research Directions for Cross Reality Interfaces. This was given on July 2nd 2024 as part of the 2024 Summer School on Cross Reality in Hagenberg, Austria (July 1st - 7th)
This document provides an overview of different techniques for hyperparameter tuning in machine learning models. It begins with introductions to grid search and random search, then discusses sequential model-based optimization techniques like Bayesian optimization and Tree-of-Parzen Estimators. Evolutionary algorithms like CMA-ES and particle-based methods like particle swarm optimization are also covered. Multi-fidelity methods like successive halving and Hyperband are described, along with recommendations on when to use different techniques. The document concludes by listing several popular libraries for hyperparameter tuning.
Support Vector Machine ppt presentationAyanaRukasar
Support vector machines (SVM) is a supervised machine learning algorithm used for both classification and regression problems. However, it is primarily used for classification. The goal of SVM is to create the best decision boundary, known as a hyperplane, that separates clusters of data points. It chooses extreme data points as support vectors to define the hyperplane. SVM is effective for problems that are not linearly separable by transforming them into higher dimensional spaces. It works well when there is a clear margin of separation between classes and is effective for high dimensional data. An example use case in Python is presented.
Classification techniques in data miningKamal Acharya
The document discusses classification algorithms in machine learning. It provides an overview of various classification algorithms including decision tree classifiers, rule-based classifiers, nearest neighbor classifiers, Bayesian classifiers, and artificial neural network classifiers. It then describes the supervised learning process for classification, which involves using a training set to construct a classification model and then applying the model to a test set to classify new data. Finally, it provides a detailed example of how a decision tree classifier is constructed from a training dataset and how it can be used to classify data in the test set.
This Naive Bayes Classifier tutorial presentation will introduce you to the basic concepts of Naive Bayes classifier, what is Naive Bayes and Bayes theorem, conditional probability concepts used in Bayes theorem, where is Naive Bayes classifier used, how Naive Bayes algorithm works with solved examples, advantages of Naive Bayes. By the end of this presentation, you will also implement Naive Bayes algorithm for text classification in Python.
The topics covered in this Naive Bayes presentation are as follows:
1. What is Naive Bayes?
2. Naive Bayes and Machine Learning
3. Why do we need Naive Bayes?
4. Understanding Naive Bayes Classifier
5. Advantages of Naive Bayes Classifier
6. Demo - Text Classification using Naive Bayes
- - - - - - - -
Simplilearn’s Machine Learning course will make you an expert in Machine Learning, a form of Artificial Intelligence that automates data analysis to enable computers to learn and adapt through experience to do specific tasks without explicit programming. You will master Machine Learning concepts and techniques including supervised and unsupervised learning, mathematical and heuristic aspects, hands-on modeling to develop algorithms and prepare you for the role of Machine Learning Engineer
Why learn Machine Learning?
Machine Learning is rapidly being deployed in all kinds of industries, creating a huge demand for skilled professionals. The Machine Learning market size is expected to grow from USD 1.03 billion in 2016 to USD 8.81 billion by 2022, at a Compound Annual Growth Rate (CAGR) of 44.1% during the forecast period.
You can gain in-depth knowledge of Machine Learning by taking our Machine Learning certification training course. With Simplilearn’s Machine Learning course, you will prepare for a career as a Machine Learning engineer as you master concepts and techniques including supervised and unsupervised learning, mathematical and heuristic aspects, and hands-on modeling to develop algorithms. Those who complete the course will be able to:
1. Master the concepts of supervised, unsupervised and reinforcement learning concepts and modeling.
2. Gain practical mastery over principles, algorithms, and applications of Machine Learning through a hands-on approach which includes working on 28 projects and one capstone project.
3. Acquire thorough knowledge of the mathematical and heuristic aspects of Machine Learning.
4. Understand the concepts and operation of support vector machines, kernel SVM, Naive Bayes, decision tree classifier, random forest classifier, logistic regression, K-nearest neighbors, K-means clustering and more.
- - - - - - - -
The document discusses the random forest algorithm. It introduces random forest as a supervised classification algorithm that builds multiple decision trees and merges them to provide a more accurate and stable prediction. It then provides an example pseudocode that randomly selects features to calculate the best split points to build decision trees, repeating the process to create a forest of trees. The document notes key advantages of random forest are that it avoids overfitting and can be used for both classification and regression tasks.
Ensemble methods combine multiple machine learning models to obtain better predictive performance than from any individual model. There are two main types of ensemble methods: sequential (e.g AdaBoost) where models are generated one after the other, and parallel (e.g Random Forest) where models are generated independently. Popular ensemble methods include bagging, boosting, and stacking. Bagging averages predictions from models trained on random samples of the data, while boosting focuses on correcting previous models' errors. Stacking trains a meta-model on predictions from other models to produce a final prediction.
This document provides an overview of support vector machines (SVMs). It discusses how SVMs can be used to perform classification tasks by finding optimal separating hyperplanes that maximize the margin between different classes. The document outlines how SVMs solve an optimization problem to find these optimal hyperplanes using techniques like Lagrange duality, kernels, and soft margins. It also covers model selection methods like cross-validation and discusses extensions of SVMs to multi-class classification problems.
This document discusses various classification algorithms including k-nearest neighbors, decision trees, naive Bayes classifier, and logistic regression. It provides examples of how each algorithm works. For k-nearest neighbors, it shows how an unknown data point would be classified based on its nearest neighbors. For decision trees, it illustrates how a tree is built by splitting the data into subsets at each node until pure subsets are reached. It also provides an example decision tree to predict whether Amit will play cricket. For naive Bayes, it gives an example of calculating the probability of cancer given a patient is a smoker.
In machine learning, support-vector machines (SVMs, also support-vector networks) are supervised learning models with associated learning algorithms that analyze data for classification and regression analysis
Slide explaining the distinction between bagging and boosting while understanding the bias variance trade-off. Followed by some lesser known scope of supervised learning. understanding the effect of tree split metric in deciding feature importance. Then understanding the effect of threshold on classification accuracy. Additionally, how to adjust model threshold for classification in supervised learning.
Note: Limitation of Accuracy metric (baseline accuracy), alternative metrics, their use case and their advantage and limitations were briefly discussed.
Abstract: This PDSG workshop introduces basic concepts of ensemble methods in machine learning. Concepts covered are Condercet Jury Theorem, Weak Learners, Decision Stumps, Bagging and Majority Voting.
Level: Fundamental
Requirements: No prior programming or statistics knowledge required.
The document provides an overview of the Naive Bayes algorithm for classification problems. It begins by explaining that Naive Bayes is a supervised learning algorithm based on Bayes' theorem. It then explains the key aspects of Naive Bayes:
- It assumes independence between features (naive) and uses Bayes' theorem to calculate probabilities (Bayes).
- Bayes' theorem is used to calculate the probability of a hypothesis given observed data.
- An example demonstrates how Naive Bayes classifies weather data to predict whether to play or not play.
The document concludes by discussing the advantages, disadvantages, applications, and types of Naive Bayes models, as well as providing Python code to implement a Naive Bayes classifier.
https://telecombcn-dl.github.io/2018-dlai/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks or Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles of deep learning from both an algorithmic and computational perspectives.
Supervised and Unsupervised Learning In Machine Learning | Machine Learning T...Simplilearn
This document provides an overview of machine learning, including:
- Machine learning allows computers to learn from data without being explicitly programmed, through processes like analyzing data, training models on past data, and making predictions.
- The main types of machine learning are supervised learning, which uses labeled training data to predict outputs, and unsupervised learning, which finds patterns in unlabeled data.
- Common supervised learning tasks include classification (like spam filtering) and regression (like weather prediction). Unsupervised learning includes clustering, like customer segmentation, and association, like market basket analysis.
- Supervised and unsupervised learning are used in many areas like risk assessment, image classification, fraud detection, customer analytics, and more
An Introduction to Supervised Machine Learning and Pattern Classification: Th...Sebastian Raschka
The document provides an introduction to supervised machine learning and pattern classification. It begins with an overview of the speaker's background and research interests. Key concepts covered include definitions of machine learning, examples of machine learning applications, and the differences between supervised, unsupervised, and reinforcement learning. The rest of the document outlines the typical workflow for a supervised learning problem, including data collection and preprocessing, model training and evaluation, and model selection. Common classification algorithms like decision trees, naive Bayes, and support vector machines are briefly explained. The presentation concludes with discussions around choosing the right algorithm and avoiding overfitting.
Support Vector Machine (SVM) is a supervised machine learning algorithm that can be used for both classification and regression analysis. It works by finding a hyperplane in an N-dimensional space that distinctly classifies the data points. SVM selects the hyperplane that has the largest distance to the nearest training data points of any class, since larger the margin lower the generalization error of the classifier. SVM can efficiently perform nonlinear classification by implicitly mapping their inputs into high-dimensional feature spaces.
Support vector machines (SVMs) are a type of supervised machine learning algorithm used for classification and regression. SVMs learn by finding the optimal separating hyperplane that maximizes the margin between two classes of objects. SVMs can efficiently perform nonlinear classification using kernel methods. Key features of SVMs include their ability to handle high dimensional data, use different kernel functions for nonlinear classification, and avoid overfitting.
ensemble methods use multiple learning algorithms to obtain better predictive performance than could be obtained from any of the constituent learning algorithms alone.
An ensemble is itself a supervised learning algorithm, because it can be trained and then used to make predictions. The trained ensemble, therefore, represents a single hypothesis. This hypothesis, however, is not necessarily contained within the hypothesis space of the models from which it is built.
Logistic regression in Machine LearningKuppusamy P
Logistic regression is a predictive analysis algorithm that can be used for classification problems. It estimates the probabilities of different classes using the logistic function, which outputs values between 0 and 1. Logistic regression transforms its output using the sigmoid function to return a probability value. It is used for problems like email spam detection, fraud detection, and tumor classification. The independent variables should be independent of each other and the dependent variable must be categorical. Gradient descent is used to minimize the loss function and optimize the model parameters during training.
▸ Machine Learning / Deep Learning models require to set the value of many hyperparameters
▸ Common examples: regularization coefficients, dropout rate, or number of neurons per layer in a Neural Network
▸ Instead of relying on some "expert advice", this presentation shows how to automatically find optimal hyperparameters
▸ Exhaustive Search, Monte Carlo Search, Bayesian Optimization, and Evolutionary Algorithms are explained with concrete examples
The document discusses machine learning classification using the MNIST dataset of handwritten digits. It begins by defining classification and providing examples. It then describes the MNIST dataset and how it is fetched in scikit-learn. The document outlines the steps of classification which include dividing the data into training and test sets, training a classifier on the training set, testing it on the test set, and evaluating performance. It specifically trains a stochastic gradient descent (SGD) classifier on the MNIST data. The performance is evaluated using cross validation accuracy, confusion matrix, and metrics like precision and recall.
This document outlines an introductory machine learning course, covering key concepts, applications, and types of machine learning like supervised and unsupervised learning. It discusses techniques like linear regression, classification, and handling overfitting. The course will include tutorials on sentiment analysis, spam filtering, stock prediction, image recognition and recommendation engines using Python and Scala. Later classes cover machine learning at scale using tools like Spark MLLib.
Recommender Systems from A to Z – Model TrainingCrossing Minds
This second meetup will be about training different models for our recommender system. We will review the simple models we can build as a baseline. After that, we will present the recommender system as an optimization problem and discuss different training losses. We will mention linear models and matrix factorization techniques. We will end the presentation with a simple introduction to non-linear models and deep learning.
This document summarizes a session on classification models. It discusses classifying data by learning classification rules from labeled input data. Specifically, it covers k-nearest neighbors classification, where the target point is assigned the majority class of its k nearest neighbors. It also discusses extending linear regression to classification by using it to generate real-valued outputs, then thresholding those values to classify instances into classes 0 or 1. However, it notes that logistic regression is a better approach, using a sigmoid function to convert real-valued outputs to a probability between 0 and 1 before thresholding for classification. The document highlights key considerations for classification like dealing with multiple classes and class imbalance. It also discusses evaluating classifiers, ensemble methods, and overfitting/under
Text Classification with Lucene/Solr, Apache Hadoop and LibSVMlucenerevolution
In this session we will show how to build a text classifier using the Apache Lucene/Solr with libSVM libraries. We classify our corpus of job offers into a number of predefined categories. Each indexed document (a job offer) then belongs to zero, one or more categories. Known machine learning techniques for text classification include naïve bayes model, logistic regression, neural network, support vector machine (SVM), etc. We use Lucene/Solr to construct the features vector. Then we use the libsvm library known as the reference implementation of the SVM model to classify the document. We construct as many one-vs-all svm classifiers as there are classes in our setting, then using the Hadoop MapReduce Framework we reconcile the result of our classifiers. The end result is a scalable multi-class classifier. Finally we outline how the classifier is used to enrich basic solr keyword search.
This document discusses scaling machine learning algorithms to large datasets. It introduces machine learning and linear regression, explaining how to represent data with features and labels. Linear regression finds the line of best fit by minimizing a squared loss function. Computing the closed-form solution is inefficient for large datasets due to expensive matrix operations. Gradient descent is introduced as an iterative alternative that is easily parallelized. It works by taking steps in the direction of steepest descent until converging to a solution.
This document provides an overview of support vector machines (SVM) including:
1) Maximal margin classifiers which find the optimal separating hyperplane with the maximum margin between classes. Support vectors are the data points that determine the hyperplane.
2) Support vector classifiers which allow some misclassified data points by introducing slack variables. This makes the classifier more robust.
3) SVM can handle non-linear decision boundaries using kernel methods to map data into higher dimensional feature spaces where a linear separator can be found. Common kernels include linear, polynomial and radial basis function kernels.
4) Multi-class classification with SVM can be done with one-vs-one or one-vs-all approaches.
5) SVM
Application of Machine Learning in AgricultureAman Vasisht
With the growing trend of machine learning, it is needless to say how machine learning can help reap benefits in agriculture. It will be boon for the farmer welfare.
Understanding computer vision with Deep LearningCloudxLab
Computer vision is a branch of computer science which deals with recognising objects, people and identifying patterns in visuals. It is basically analogous to the vision of an animal.
Topics covered:
1. Overview of Machine Learning
2. Basics of Deep Learning
3. What is computer vision and its use-cases?
4. Various algorithms used in Computer Vision (mostly CNN)
5. Live hands-on demo of either Auto Cameraman or Face recognition system
6. What next?
Understanding computer vision with Deep LearningShubhWadekar
Topics covered in the Webinar
1. Overview of Machine Learning
2. Basics of Deep Learning
3. What is computer vision and its use-cases?
4. Various algorithms used in Computer Vision (mostly CNN)
5. Live hands-on demo of either Auto Cameraman or Face recognition system
6. What next?
Presented by Sandeep Giri
www.cloudxlab.com
Understanding computer vision with Deep Learningknowbigdata
Computer vision is a branch of computer science which deals with recognising objects, people and identifying patterns in visuals. It is basically analogous to the vision of an animal.
Topics covered:
1. Overview of Machine Learning
2. Basics of Deep Learning
3. What is computer vision and its use-cases?
4. Various algorithms used in Computer Vision (mostly CNN)
5. Live hands-on demo of either Auto Cameraman or Face recognition system
6. What next?
This document discusses machine learning algorithms for classification problems, specifically logistic regression. It explains that logistic regression predicts the probability of a binary outcome using a sigmoid function. Unlike linear regression, which is used for continuous outputs, logistic regression is used for classification problems where the output is discrete/categorical. It describes how logistic regression learns model parameters through gradient descent optimization of a likelihood function to minimize error. Regularization techniques can also be used to address overfitting issues that may arise from limited training data.
Support Vector Machines USING MACHINE LEARNING HOW IT WORKSrajalakshmi5921
This document discusses support vector machines (SVM), a supervised machine learning algorithm used for classification and regression. It explains that SVM finds the optimal boundary, known as a hyperplane, that separates classes with the maximum margin. When data is not linearly separable, kernel functions can transform the data into a higher-dimensional space to make it separable. The document discusses SVM for both linearly separable and non-separable data, kernel functions, hyperparameters, and approaches for multiclass classification like one-vs-one and one-vs-all.
Basic machine learning background with Python scikit-learn
This document provides an overview of machine learning and the Python scikit-learn library. It introduces key machine learning concepts like classification, linear models, support vector machines, decision trees, bagging, boosting, and clustering. It also demonstrates how to perform tasks like SVM classification, decision tree modeling, random forest, principal component analysis, and k-means clustering using scikit-learn. The document concludes that scikit-learn can handle large datasets and recommends Keras for deep learning.
Machine Learning : why we should know and how it worksKevin Lee
This document provides an overview of machine learning, including:
- An introduction to machine learning and why it is important.
- The main types of machine learning algorithms: supervised learning, unsupervised learning, and deep neural networks.
- Examples of how machine learning algorithms work, such as logistic regression, support vector machines, and k-means clustering.
- How machine learning is being applied in various industries like healthcare, commerce, and more.
Online advertising and large scale model fittingWush Wu
This document discusses online advertising and techniques for fitting large-scale models to advertising data. It outlines batch and online algorithms for logistic regression, including parallelizing existing batch algorithms and stochastic gradient descent. The document also discusses using alternating direction method of multipliers and follow the proximal regularized leader to fit models to large datasets across multiple machines. It provides examples of how major companies like LinkedIn and Facebook implement hybrid online-batch algorithms at large scale.
Machine learning techniques can be applied in formal verification in several ways:
1) To enhance current formal verification tools by automating tasks like debugging, specification mining, and theorem proving.
2) To enable the development of new formal verification tools by applying machine learning to problems like SAT solving, model checking, and property checking.
3) Specific applications include using machine learning for debugging and root cause identification, learning specifications from runtime traces, aiding theorem proving by selecting heuristics, and tuning SAT solver parameters and selection.
Système de recommandations de produits sur un site marchand par Koby KARP, Data Scientist (Equancy) & Hervé MIGNOT, Partner at Equancy
La recommandation reste un outil clé pour la personnalisation des sites marchands et le sujet est loin d’être épuisé. La prise en compte de la particularité d’un marché peut nécessité d’adapter le traitement et les algorithmes utilisés. Après une revue des techniques de recommandations, nous présenterons la démarche spécifique que nous avons adopté. Le système a été développé sous Spark pour la préparation des données et le calcul des modèles de recommandations. Une API simple et son service ont été développé pour délivrer les recommandations aux applications clientes.
This document provides an introduction and overview for a course on machine learning. It outlines the course structure, assignments, and expectations. The course will cover topics including linear regression, classification, model selection, and dimensionality reduction. It will teach students how to analyze data, preprocess it, extract features, train models, and evaluate model performance. The goal is for students to understand core machine learning algorithms and concepts. Required materials include an introduction to statistical learning textbook.
This document provides an agenda for an introduction to deep learning presentation. It begins with an introduction to basic AI, machine learning, and deep learning terms. It then briefly discusses use cases of deep learning. The document outlines how to approach a deep learning problem, including which tools and algorithms to use. It concludes with a question and answer section.
This document discusses recurrent neural networks (RNNs) and their applications. It begins by explaining that RNNs can process input sequences of arbitrary lengths, unlike other neural networks. It then provides examples of RNN applications, such as predicting time series data, autonomous driving, natural language processing, and music generation. The document goes on to describe the fundamental concepts of RNNs, including recurrent neurons, memory cells, and different types of RNN architectures for processing input/output sequences. It concludes by demonstrating how to implement basic RNNs using TensorFlow's static_rnn function.
Natural Language Processing (NLP) is a field of artificial intelligence that deals with interactions between computers and human languages. NLP aims to program computers to process and analyze large amounts of natural language data. Some common NLP tasks include speech recognition, text classification, machine translation, question answering, and more. Popular NLP tools include Stanford CoreNLP, NLTK, OpenNLP, and TextBlob. Vectorization is commonly used to represent text in a way that can be used for machine learning algorithms like calculating text similarity. Tf-idf is a common technique used to weigh words based on their frequency and importance.
- Naive Bayes is a classification technique based on Bayes' theorem that uses "naive" independence assumptions. It is easy to build and can perform well even with large datasets.
- It works by calculating the posterior probability for each class given predictor values using the Bayes theorem and independence assumptions between predictors. The class with the highest posterior probability is predicted.
- It is commonly used for text classification, spam filtering, and sentiment analysis due to its fast performance and high success rates compared to other algorithms.
An autoencoder is an artificial neural network that is trained to copy its input to its output. It consists of an encoder that compresses the input into a lower-dimensional latent-space encoding, and a decoder that reconstructs the output from this encoding. Autoencoders are useful for dimensionality reduction, feature learning, and generative modeling. When constrained by limiting the latent space or adding noise, autoencoders are forced to learn efficient representations of the input data. For example, a linear autoencoder trained with mean squared error performs principal component analysis.
The document discusses challenges in training deep neural networks and solutions to those challenges. Training deep neural networks with many layers and parameters can be slow and prone to overfitting. A key challenge is the vanishing gradient problem, where the gradients shrink exponentially small as they propagate through many layers, making earlier layers very slow to train. Solutions include using initialization techniques like He initialization and activation functions like ReLU and leaky ReLU that do not saturate, preventing gradients from vanishing. Later improvements include the ELU activation function.
( Machine Learning & Deep Learning Specialization Training: https://goo.gl/5u2RiS )
This CloudxLab Reinforcement Learning tutorial helps you to understand Reinforcement Learning in detail. Below are the topics covered in this tutorial:
1) What is Reinforcement?
2) Reinforcement Learning an Introduction
3) Reinforcement Learning Example
4) Learning to Optimize Rewards
5) Policy Search - Brute Force Approach, Genetic Algorithms and Optimization Techniques
6) OpenAI Gym
7) The Credit Assignment Problem
8) Inverse Reinforcement Learning
9) Playing Atari with Deep Reinforcement Learning
10) Policy Gradients
11) Markov Decision Processes
Apache Spark - Key Value RDD - Transformations | Big Data Hadoop Spark Tutori...CloudxLab
The document provides information about key-value RDD transformations and actions in Spark. It defines transformations like keys(), values(), groupByKey(), combineByKey(), sortByKey(), subtractByKey(), join(), leftOuterJoin(), rightOuterJoin(), and cogroup(). It also defines actions like countByKey() and lookup() that can be performed on pair RDDs. Examples are given showing how to use these transformations and actions to manipulate key-value RDDs.
Advanced Spark Programming - Part 2 | Big Data Hadoop Spark Tutorial | CloudxLabCloudxLab
Big Data with Hadoop & Spark Training: http://bit.ly/2kyRTuW
This CloudxLab Advanced Spark Programming tutorial helps you to understand Advanced Spark Programming in detail. Below are the topics covered in this slide:
1) Shared Variables - Accumulators & Broadcast Variables
2) Accumulators and Fault Tolerance
3) Custom Accumulators - Version 1.x & Version 2.x
4) Examples of Broadcast Variables
5) Key Performance Considerations - Level of Parallelism
6) Serialization Format - Kryo
7) Memory Management
8) Hardware Provisioning
Apache Spark - Dataframes & Spark SQL - Part 2 | Big Data Hadoop Spark Tutori...CloudxLab
Big Data with Hadoop & Spark Training: http://bit.ly/2sm9c61
This CloudxLab Introduction to Spark SQL & DataFrames tutorial helps you to understand Spark SQL & DataFrames in detail. Below are the topics covered in this slide:
1) Loading XML
2) What is RPC - Remote Process Call
3) Loading AVRO
4) Data Sources - Parquet
5) Creating DataFrames From Hive Table
6) Setting up Distributed SQL Engine
Apache Spark - Dataframes & Spark SQL - Part 1 | Big Data Hadoop Spark Tutori...CloudxLab
Big Data with Hadoop & Spark Training: http://bit.ly/2sf2z6i
This CloudxLab Introduction to Spark SQL & DataFrames tutorial helps you to understand Spark SQL & DataFrames in detail. Below are the topics covered in this slide:
1) Introduction to DataFrames
2) Creating DataFrames from JSON
3) DataFrame Operations
4) Running SQL Queries Programmatically
5) Datasets
6) Inferring the Schema Using Reflection
7) Programmatically Specifying the Schema
Apache Spark - Running on a Cluster | Big Data Hadoop Spark Tutorial | CloudxLabCloudxLab
(Big Data with Hadoop & Spark Training: http://bit.ly/2IUsWca
This CloudxLab Running in a Cluster tutorial helps you to understand running Spark in the cluster in detail. Below are the topics covered in this tutorial:
1) Spark Runtime Architecture
2) Driver Node
3) Scheduling Tasks on Executors
4) Understanding the Architecture
5) Cluster Managers
6) Executors
7) Launching a Program using spark-submit
8) Local Mode & Cluster-Mode
9) Installing Standalone Cluster
10) Cluster Mode - YARN
11) Launching a Program on YARN
12) Cluster Mode - Mesos and AWS EC2
13) Deployment Modes - Client and Cluster
14) Which Cluster Manager to Use?
15) Common flags for spark-submit
Introduction to SparkR | Big Data Hadoop Spark Tutorial | CloudxLabCloudxLab
Big Data with Hadoop & Spark Training: http://bit.ly/2LCTufA
This CloudxLab Introduction to SparkR tutorial helps you to understand SparkR in detail. Below are the topics covered in this tutorial:
1) SparkR (R on Spark)
2) SparkR DataFrames
3) Launch SparkR
4) Creating DataFrames from Local DataFrames
5) DataFrame Operation
6) Creating DataFrames - From JSON
7) Running SQL Queries from SparkR
Introduction to NoSQL | Big Data Hadoop Spark Tutorial | CloudxLabCloudxLab
1) NoSQL databases are non-relational and schema-free, providing alternatives to SQL databases for big data and high availability applications.
2) Common NoSQL database models include key-value stores, column-oriented databases, document databases, and graph databases.
3) The CAP theorem states that a distributed data store can only provide two out of three guarantees around consistency, availability, and partition tolerance.
Introduction to MapReduce - Hadoop Streaming | Big Data Hadoop Spark Tutorial...CloudxLab
Big Data with Hadoop & Spark Training: http://bit.ly/2sh5b3E
This CloudxLab Hadoop Streaming tutorial helps you to understand Hadoop Streaming in detail. Below are the topics covered in this tutorial:
1) Hadoop Streaming and Why Do We Need it?
2) Writing Streaming Jobs
3) Testing Streaming jobs and Hands-on on CloudxLab
Introduction To TensorFlow | Deep Learning Using TensorFlow | CloudxLabCloudxLab
This document provides instructions for getting started with TensorFlow using a free CloudxLab. It outlines the following steps:
1. Open CloudxLab and enroll if not already enrolled. Otherwise go to "My Lab".
2. In "My Lab", open Jupyter and run commands to clone an ML repository containing TensorFlow examples.
3. Go to the deep learning folder in Jupyter and open the TensorFlow notebook to get started with examples.
Introduction to Deep Learning | CloudxLabCloudxLab
( Machine Learning & Deep Learning Specialization Training: https://goo.gl/goQxnL )
This CloudxLab Deep Learning tutorial helps you to understand Deep Learning in detail. Below are the topics covered in this tutorial:
1) What is Deep Learning
2) Deep Learning Applications
3) Artificial Neural Network
4) Deep Learning Neural Networks
5) Deep Learning Frameworks
6) AI vs Machine Learning
In this tutorial, we will learn the the following topics -
+ The Curse of Dimensionality
+ Main Approaches for Dimensionality Reduction
+ PCA - Principal Component Analysis
+ Kernel PCA
+ LLE
+ Other Dimensionality Reduction Techniques
In this tutorial, we will learn the the following topics -
+ Voting Classifiers
+ Bagging and Pasting
+ Random Patches and Random Subspaces
+ Random Forests
+ Boosting
+ Stacking
In this tutorial, we will learn the the following topics -
+ Training and Visualizing a Decision Tree
+ Making Predictions
+ Estimating Class Probabilities
+ The CART Training Algorithm
+ Computational Complexity
+ Gini Impurity or Entropy?
+ Regularization Hyperparameters
+ Regression
+ Instability
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - MydbopsMydbops
This presentation, delivered at the Postgres Bangalore (PGBLR) Meetup-2 on June 29th, 2024, dives deep into connection pooling for PostgreSQL databases. Aakash M, a PostgreSQL Tech Lead at Mydbops, explores the challenges of managing numerous connections and explains how connection pooling optimizes performance and resource utilization.
Key Takeaways:
* Understand why connection pooling is essential for high-traffic applications
* Explore various connection poolers available for PostgreSQL, including pgbouncer
* Learn the configuration options and functionalities of pgbouncer
* Discover best practices for monitoring and troubleshooting connection pooling setups
* Gain insights into real-world use cases and considerations for production environments
This presentation is ideal for:
* Database administrators (DBAs)
* Developers working with PostgreSQL
* DevOps engineers
* Anyone interested in optimizing PostgreSQL performance
Contact info@mydbops.com for PostgreSQL Managed, Consulting and Remote DBA Services
Kief Morris rethinks the infrastructure code delivery lifecycle, advocating for a shift towards composable infrastructure systems. We should shift to designing around deployable components rather than code modules, use more useful levels of abstraction, and drive design and deployment from applications rather than bottom-up, monolithic architecture and delivery.
Measuring the Impact of Network Latency at TwitterScyllaDB
Widya Salim and Victor Ma will outline the causal impact analysis, framework, and key learnings used to quantify the impact of reducing Twitter's network latency.
Choose our Linux Web Hosting for a seamless and successful online presencerajancomputerfbd
Our Linux Web Hosting plans offer unbeatable performance, security, and scalability, ensuring your website runs smoothly and efficiently.
Visit- https://onliveserver.com/linux-web-hosting/
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-InTrustArc
Six months into 2024, and it is clear the privacy ecosystem takes no days off!! Regulators continue to implement and enforce new regulations, businesses strive to meet requirements, and technology advances like AI have privacy professionals scratching their heads about managing risk.
What can we learn about the first six months of data privacy trends and events in 2024? How should this inform your privacy program management for the rest of the year?
Join TrustArc, Goodwin, and Snyk privacy experts as they discuss the changes we’ve seen in the first half of 2024 and gain insight into the concrete, actionable steps you can take to up-level your privacy program in the second half of the year.
This webinar will review:
- Key changes to privacy regulations in 2024
- Key themes in privacy and data governance in 2024
- How to maximize your privacy program in the second half of 2024
Are you interested in dipping your toes in the cloud native observability waters, but as an engineer you are not sure where to get started with tracing problems through your microservices and application landscapes on Kubernetes? Then this is the session for you, where we take you on your first steps in an active open-source project that offers a buffet of languages, challenges, and opportunities for getting started with telemetry data.
The project is called openTelemetry, but before diving into the specifics, we’ll start with de-mystifying key concepts and terms such as observability, telemetry, instrumentation, cardinality, percentile to lay a foundation. After understanding the nuts and bolts of observability and distributed traces, we’ll explore the openTelemetry community; its Special Interest Groups (SIGs), repositories, and how to become not only an end-user, but possibly a contributor.We will wrap up with an overview of the components in this project, such as the Collector, the OpenTelemetry protocol (OTLP), its APIs, and its SDKs.
Attendees will leave with an understanding of key observability concepts, become grounded in distributed tracing terminology, be aware of the components of openTelemetry, and know how to take their first steps to an open-source contribution!
Key Takeaways: Open source, vendor neutral instrumentation is an exciting new reality as the industry standardizes on openTelemetry for observability. OpenTelemetry is on a mission to enable effective observability by making high-quality, portable telemetry ubiquitous. The world of observability and monitoring today has a steep learning curve and in order to achieve ubiquity, the project would benefit from growing our contributor community.
Coordinate Systems in FME 101 - Webinar SlidesSafe Software
If you’ve ever had to analyze a map or GPS data, chances are you’ve encountered and even worked with coordinate systems. As historical data continually updates through GPS, understanding coordinate systems is increasingly crucial. However, not everyone knows why they exist or how to effectively use them for data-driven insights.
During this webinar, you’ll learn exactly what coordinate systems are and how you can use FME to maintain and transform your data’s coordinate systems in an easy-to-digest way, accurately representing the geographical space that it exists within. During this webinar, you will have the chance to:
- Enhance Your Understanding: Gain a clear overview of what coordinate systems are and their value
- Learn Practical Applications: Why we need datams and projections, plus units between coordinate systems
- Maximize with FME: Understand how FME handles coordinate systems, including a brief summary of the 3 main reprojectors
- Custom Coordinate Systems: Learn how to work with FME and coordinate systems beyond what is natively supported
- Look Ahead: Gain insights into where FME is headed with coordinate systems in the future
Don’t miss the opportunity to improve the value you receive from your coordinate system data, ultimately allowing you to streamline your data analysis and maximize your time. See you there!
The DealBook is our annual overview of the Ukrainian tech investment industry. This edition comprehensively covers the full year 2023 and the first deals of 2024.
Fluttercon 2024: Showing that you care about security - OpenSSF Scorecards fo...Chris Swan
Have you noticed the OpenSSF Scorecard badges on the official Dart and Flutter repos? It's Google's way of showing that they care about security. Practices such as pinning dependencies, branch protection, required reviews, continuous integration tests etc. are measured to provide a score and accompanying badge.
You can do the same for your projects, and this presentation will show you how, with an emphasis on the unique challenges that come up when working with Dart and Flutter.
The session will provide a walkthrough of the steps involved in securing a first repository, and then what it takes to repeat that process across an organization with multiple repos. It will also look at the ongoing maintenance involved once scorecards have been implemented, and how aspects of that maintenance can be better automated to minimize toil.
How Social Media Hackers Help You to See Your Wife's Message.pdfHackersList
In the modern digital era, social media platforms have become integral to our daily lives. These platforms, including Facebook, Instagram, WhatsApp, and Snapchat, offer countless ways to connect, share, and communicate.
Quantum Communications Q&A with Gemini LLM. These are based on Shannon's Noisy channel Theorem and offers how the classical theory applies to the quantum world.
Best Programming Language for Civil EngineersAwais Yaseen
The integration of programming into civil engineering is transforming the industry. We can design complex infrastructure projects and analyse large datasets. Imagine revolutionizing the way we build our cities and infrastructure, all by the power of coding. Programming skills are no longer just a bonus—they’re a game changer in this era.
Technology is revolutionizing civil engineering by integrating advanced tools and techniques. Programming allows for the automation of repetitive tasks, enhancing the accuracy of designs, simulations, and analyses. With the advent of artificial intelligence and machine learning, engineers can now predict structural behaviors under various conditions, optimize material usage, and improve project planning.
Mitigating the Impact of State Management in Cloud Stream Processing SystemsScyllaDB
Stream processing is a crucial component of modern data infrastructure, but constructing an efficient and scalable stream processing system can be challenging. Decoupling compute and storage architecture has emerged as an effective solution to these challenges, but it can introduce high latency issues, especially when dealing with complex continuous queries that necessitate managing extra-large internal states.
In this talk, we focus on addressing the high latency issues associated with S3 storage in stream processing systems that employ a decoupled compute and storage architecture. We delve into the root causes of latency in this context and explore various techniques to minimize the impact of S3 latency on stream processing performance. Our proposed approach is to implement a tiered storage mechanism that leverages a blend of high-performance and low-cost storage tiers to reduce data movement between the compute and storage layers while maintaining efficient processing.
Throughout the talk, we will present experimental results that demonstrate the effectiveness of our approach in mitigating the impact of S3 latency on stream processing. By the end of the talk, attendees will have gained insights into how to optimize their stream processing systems for reduced latency and improved cost-efficiency.
Quality Patents: Patents That Stand the Test of TimeAurora Consulting
Is your patent a vanity piece of paper for your office wall? Or is it a reliable, defendable, assertable, property right? The difference is often quality.
Is your patent simply a transactional cost and a large pile of legal bills for your startup? Or is it a leverageable asset worthy of attracting precious investment dollars, worth its cost in multiples of valuation? The difference is often quality.
Is your patent application only good enough to get through the examination process? Or has it been crafted to stand the tests of time and varied audiences if you later need to assert that document against an infringer, find yourself litigating with it in an Article 3 Court at the hands of a judge and jury, God forbid, end up having to defend its validity at the PTAB, or even needing to use it to block pirated imports at the International Trade Commission? The difference is often quality.
Quality will be our focus for a good chunk of the remainder of this season. What goes into a quality patent, and where possible, how do you get it without breaking the bank?
** Episode Overview **
In this first episode of our quality series, Kristen Hansen and the panel discuss:
⦿ What do we mean when we say patent quality?
⦿ Why is patent quality important?
⦿ How to balance quality and budget
⦿ The importance of searching, continuations, and draftsperson domain expertise
⦿ Very practical tips, tricks, examples, and Kristen’s Musts for drafting quality applications
https://www.aurorapatents.com/patently-strategic-podcast.html
Comparison Table of DiskWarrior Alternatives.pdfAndrey Yasko
To help you choose the best DiskWarrior alternative, we've compiled a comparison table summarizing the features, pros, cons, and pricing of six alternatives.
An invited talk given by Mark Billinghurst on Research Directions for Cross Reality Interfaces. This was given on July 2nd 2024 as part of the 2024 Summer School on Cross Reality in Hagenberg, Austria (July 1st - 7th)
2. Machine Learning - SVM
Support Vector Machines
Divide dataset
into training and
test samples
Train the
model using
training dataset
Test using sample
training data
Performance
metrics (Finalize
the model)
Improve the
model using error
analysis
Remember the general flow of a machine learning problem:
There can be several models depending on the problem statement
We will discuss about one such model - SVM
3. Machine Learning - SVM
● Support vector machine is
○ Very powerful and versatile model
○ Capable of performing
■ Linear and
■ Nonlinear classification
■ Regression and
■ Outlier detection
● Well suited for small or medium sized datasets
Support Vector Machines
4. Machine Learning - SVM
Support Vector Machines
● In this session we will learn about
○ Linear SVM Classification
○ Nonlinear SVM Classification and
○ SVM Regression
7. Machine Learning - SVM
5
Not 5
What is Classification?
Identifying to which label something belongs to
8. Machine Learning - SVM
Examples of Classification
● Classifying emails as spam or not spam
Q. What type of classification is this?
9. Machine Learning - SVM
Examples of Classification
● Classifying emails as spam or not spam
Q. What type of classification is this? Ans: Binary
10. Machine Learning - SVM
● Classifying flowers of a particular species like the Iris Dataset
Examples of Classification
Q. What type of classification is this?
11. Machine Learning - SVM
● Classifying flowers of a particular species like the Iris Dataset
Examples of Classification
Q. What type of classification is this? Ans: Multi-class classification
12. Machine Learning - SVM
● Classifying a credit card transaction as fraudulent or not
Examples of Classification
13. Machine Learning - SVM
Examples of Classification
● Face recognition
Q. What type of classification is this?
14. Machine Learning - SVM
Examples of Classification
● Face recognition
Q. What type of classification is this? Ans: Multi-label classification
15. Machine Learning - SVM
5
Not 5
Recap of 5 and Not 5 Classification Problem
Binary Classification Multiclass Classification
Q. What is the classifier we used for the Binary Classification?
16. Machine Learning - SVM
5
Not 5
Recap of 5 and Not 5 Classification Problem
Binary Classification Multiclass Classification
Q. What is the classifier we used for the Binary Classification?
Ans: SGDClassifier
17. Machine Learning - SVM
5
Not 5
Recap of 5 and Not 5 Classification Problem
Binary Classification Multiclass Classification
Q. What is the classifier we used for the Multiclass Classification?
18. Machine Learning - SVM
5
Not 5
Recap of 5 and Not 5 Classification Problem
Binary Classification Multiclass Classification
Q. What is the classifier we used for the Multiclass Classification?
Ans: SGDClassifier - OvO and OvA
20. Machine Learning - SVM
What is Linear Classification?
● The two classes can be separated easily with a ‘straight’ line
‘Straight’ is the keyword. It means linear classification.
21. Machine Learning - SVM
What is Linear Classification?
● For example: IRIS Dataset
○ Features: Sepal Length, Petal Length
○ Class: Iris Virginica OR Iris Versicolor OR Iris Setosa
22. Machine Learning - SVM
What is Linear Classification?
Sepal Length Petal Length Flower Type
1.212 4.1 Iris-Versicolor
0.5 1.545 Iris-Setosa
0.122 1.64 Iris-Setosa
0.2343 ... Iris-Setosa
0.1 ... Iris-Setosa
1.32 ... Iris-Versicolor
23. Machine Learning - SVM
What is Linear Classification?
● For the above IRIS Dataset, what is the type of Machine Learning model?
○ Classification or Regression?
■ Ans:
24. Machine Learning - SVM
What is Linear Classification?
● For the above IRIS Dataset, what is the type of Machine Learning model?
○ Classification or Regression?
■ Ans: Classification
25. Machine Learning - SVM
What is Linear Classification?
● What is the type of Supervised Machine Learning model?
○ Classification or Regression?
■ Ans: Classification
○ What type of classification?
■ Binary Classification
■ Multi-label Classification
■ Multi-output Classification
■ Multi-class Classification
26. Machine Learning - SVM
What is Linear Classification?
● What is the type of Supervised Machine Learning model?
○ Classification or Regression?
■ Ans: Classification
○ What type of classification?
■ Binary Classification
■ Multi-label Classification
■ Multi-output Classification
■ Ans: Multi-class Classification
27. Machine Learning - SVM
What is Linear Classification?
● For the IRIS dataset above:
○ Number of features?
■ Ans:
○ Number of classes?
■ Ans:
28. Machine Learning - SVM
What is Linear Classification?
● For the IRIS dataset above:
○ Number of features?
■ Ans: 2
○ Number of classes?
■ Ans: 3
29. Machine Learning - SVM
What is Linear Classification?
● When we plot the two features on the graph and label it by color
○ The classes can be divided using a straight line
○ Hence, linear classification
Straight Line (Linear Classification)
30. Machine Learning - SVM
Linear SVM Classification
Linear SVM
Classification
Nonlinear SVM
Classification
SVM
Regression
Bad model
versus good-
model (Large
Margin)
Classification
Soft Margin
versus Hard-
margin
Classification
31. Machine Learning - SVM
Linear SVM Classification - Large Margin
Pink and red decision boundaries are
very close to the instances - bad
model
Decision Boundary as far away from
training instances - good model
Large Margin Classification
Widest possible street
32. Machine Learning - SVM
Linear SVM Classification - Large Margin
May not perform well on new
instances
Adding training instances may not
affect the decision boundary
Large Margin Classification
33. Machine Learning - SVM
Linear SVM Classification - Large Margin
Large Margin Classification
Widest possible street
Support Vectors
● What are Support vectors?
○ Vectors or the training set located closest to the classifier OR
○ Vectors or the training sets located at the edge of the street
37. Machine Learning - SVM
Linear SVM Classification - Example 1
● Model the classifier, plot the points and the classifier
38. Machine Learning - SVM
Linear SVM Classification - Example 1
● What is the problem?
39. Machine Learning - SVM
Linear SVM Classification - Example 1
● What is the problem?
○ X0 ranges from 0 to 6 while
○ X1 ranges from 20 to 80
● Solution: Feature Scaling
40. Machine Learning Project
Feature Scaling
Feature Scaling
Quick Revision
from
Preparing the data for ML Algorithms in End-to-End Project
41. Machine Learning Project
Feature Scaling
● ML algorithms do not perform well
○ When the input numerical attributes have very different scales
● Feature Scaling is one of the most important
○ Transformation we need to apply to our data
● Two ways to make sure all attributes have same scale
○ Min-max scaling
○ Standardization
42. Machine Learning Project
Feature Scaling
Min-max Scaling
● Also known as Normalization
● Normalized values are in the range of [0, 1]
43. Machine Learning Project
Feature Scaling
Min-max Scaling
● Also known as Normalization
● Normalized values are in the range of [0, 1]
Original Value
Normalized Value
45. Machine Learning Project
Feature Scaling
Min-max Scaling - Example
# Use Scikit-Learn minmax_scaling
>>> from mlxtend.preprocessing import minmax_scaling
>>> minmax_scaling(df, columns=['s1', 's2'])
Original Scaled (In range of
0 and 1)
46. Machine Learning Project
Feature Scaling
Standardization
● In Machine Learning, we handle various types of data like
○ Audio signals and
○ Pixel values for image data
○ And this data can include multiple dimensions
47. Machine Learning Project
Feature Scaling
Standardization
We scale the values by calculating
○ How many standard deviation is the value away from the mean
SAT scores ~ N(mean = 1500, SD = 300)
ACT scores ~ N(mean = 21, SD = 5)
48. Machine Learning Project
Feature Scaling
Standardization
● The general method of calculation
○ Calculate distribution mean and standard deviation for each feature
○ Subtract the mean from each feature
○ Divide the result from previous step of each feature by its standard
deviation
Standardized Value
49. Machine Learning Project
Feature Scaling
Standardization
● In Standardization, features are rescaled
● So that output will have the properties of
● Standard normal distribution with
○ Zero mean and
○ Unit variance
Mean Standard Deviation
51. Machine Learning Project
Feature Scaling
Which One to Use?
● Min-max scales in the range of [0,1]
● Standardization does not bound values to a specific range
○ It may be problem for some algorithms
○ Example- Neural networks expect an input value ranging from 0 to 1
● We’ll learn more use cases as we proceed in the course
53. Machine Learning - SVM
Linear SVM Classification - Example 2
x1 x2 Label x1 (Scaled) x2 (Scaled)
1 50 0 -1.5 -0.1154
5 20 0 0.9 -1.5011107
3 80 1 -0.3 1.27017
5 60 1 0.9 0.3464
Mean (m1) =
3.5
Std Dev (s1)
= 1.65
Mean (m2) =
52.5
Std Dev (s2)
= 21.65
(x-m1)/s1 (x-s2)/m2
● With Scaling
54. Machine Learning - SVM
Linear SVM Classification - Example 2
● Scaling of features
X_new = (x-m1)/s1
● What kind of scaling is this?
○ Normalization
○ Standardization
55. Machine Learning - SVM
Linear SVM Classification - Example 2
● Scaling of features
X_new = (x-m1)/s1
● What kind of scaling is this?
○ Normalization
○ Standardization
56. Machine Learning - SVM
Linear SVM Classification - Example 2
● Scaling of features
X_new = (x-m1)/s1
● What kind of scaling is this?
○ Normalization
○ Standardization
● What is the module available in scikit_learn to perform standardization?
57. Machine Learning - SVM
Linear SVM Classification - Example 2
● Scaling of features
X_new = (x-m1)/s1
● What kind of scaling is this?
○ Normalization
○ Standardization
● What is the module available in scikit_learn to perform standardization?
○ Answer: StandardScalar
58. Machine Learning - SVM
Linear SVM Classification - Example 2
● Scaling the input training data
>>> from sklearn.preprocessing import StandardScaler
>>> scaler = StandardScaler()
>>> X_scaled = scaler.fit_transform(Xs)
>>> print(X_scaled)
[[-1.50755672 -0.11547005]
[ 0.90453403 -1.5011107 ]
[-0.30151134 1.27017059]
[ 0.90453403 0.34641016]]
59. Machine Learning - SVM
Linear SVM Classification - Example 2
● Building the model, plotting the decision boundary and the training points
>>> svm_clf.fit(X_scaled, ys)
>>> plt.plot(X_scaled[:, 0][ys==1], X_scaled[:, 1][ys==1], "bo")
>>> plt.plot(X_scaled[:, 0][ys==0], X_scaled[:, 1][ys==0], "ms")
>>> plot_svc_decision_boundary(svm_clf, -2, 2)
>>> plt.ylabel("$x_{1scaled}$", fontsize=20)
>>> plt.xlabel("$x_{0scaled}$", fontsize=20)
>>> plt.title("Scaled", fontsize=16)
>>> plt.axis([-2, 2, -2, 2])
60. Machine Learning - SVM
Linear SVM Classification - Example 2
● Output decision boundary for a scaled training data
61. Machine Learning - SVM
Linear SVM Classification
● Unscaled vs Scaled comparison
X0 X1 Label X0 (Scaled) X1 (Scaled)
1 50 0 -1.5 -0.1154
5 20 0 0.9 -1.5011107
3 80 1 -0.3 1.27017
5 60 1 0.9 0.3464
Mean (m1) =
3.5
Std Dev (s1)
= 1.65
Mean (m2) =
52.5
Std Dev (s2)
= 21.65
(x-m1)/s1 (x-m2)/s2
62. Machine Learning - SVM
Linear SVM Classification
● Unscaled vs Scaled
Widestpossiblestreet
63. Machine Learning - SVM
Linear SVM Classification
● Unscaled vs Scaled
○ Linear SVM sensitive to scaling
○ Feature scaling an important part of data preparation
■ Normalization
■ Standardization
○ Scaled features produce better result for the above example
65. Machine Learning - SVM
Linear SVM Classification
Linear SVM
Classification
Nonlinear SVM
Classification
SVM
Regression
Bad model
versus good-
model (Large
Margin-
Standardized)
Classification
Soft Margin
versus Hard-
margin
Classification
66. Machine Learning - SVM
Linear SVM Classification - Hard Margin
● Hard Margin Classification
○ Strictly impose that all the instances should be
■ Off the street and
■ On a particular side of the decision boundary
○ Issues:
■ Works only if the data is linearly separable
■ Quite sensitive to outliers
67. Machine Learning - SVM
Linear SVM Classification - Hard Margin
Question - Is it possible to classify this using SVM Hard Margin
Classification?
See the code in notebook
68. Machine Learning - SVM
Linear SVM Classification - Hard Margin
Question - Is it possible to classify this using SVM Hard Margin
Classification?
See the code in notebook
69. Machine Learning - SVM
Linear SVM Classification - Hard Margin
Question - Is it possible to classify this using SVM Hard Margin
Classification?
See the code in notebook
70. Machine Learning - SVM
Linear SVM Classification - Hard Margin
Question - Is it possible to classify this using SVM Hard Margin
Classification?
Answer - Yes, but what is the problem?
Yes
See the code in notebook
71. Machine Learning - SVM
Linear SVM Classification - Hard Margin
Yes
Question - Is it possible to classify this using SVM Hard Margin
Classification?
Answer - Yes, but what is the problem?
Outlier is the problem
72. Machine Learning - SVM
Linear SVM Classification - Soft Margin
Soft Margin Classification
Is keeping a balance between
○ Keeping the street as large as possible
○ Limiting the margin violations
○ Regulated by using ‘C’ parameter
73. Machine Learning - SVM
Linear SVM Classification - Soft Margin
● The balance can be regulated in Scikit-Learn using ‘c’ parameter
○ Higher ‘c’:
■ Narrower street, lower margin violations
○ Smaller ‘c’:
■ Wider street, more margin violations
>>> svm_clf = SVC(kernel="linear", C=100)
SVM Linear classification ‘C’ parameter to
regulate the street
and margin violations
74. Machine Learning - SVM
Linear SVM Classification - Soft Margin
Example 1: SVM Classification for IRIS data using c = 1
Steps:
● Load the IRIS data
● Model the SVM Linear classifier with the training set: fitting
● Test using a sample data
For illustration:
● Plot the decision boundary and the training samples
Something missing in the steps?
75. Machine Learning - SVM
Linear SVM Classification - Soft Margin
Example 1: SVM Classification for IRIS data using c = 1
Steps:
● Load the IRIS data
● Feature scaling the data
● Model the SVM Linear classifier with the training set: fitting
● Test using a sample data
For illustration:
● Plot the decision boundary and the training samples
Something missing in the steps?
77. Machine Learning - SVM
Linear SVM Classification - Soft Margin
Example 1: SVM Classification for IRIS data using c = 1
Steps:
● Load the IRIS data
>>> from sklearn import datasets
>>> from sklearn.pipeline import Pipeline
>>> from sklearn.preprocessing import StandardScaler
>>> from sklearn.svm import LinearSVC
>>> iris = datasets.load_iris()
>>> X = iris["data"][:, (2, 3)] # petal length, petal width
>>> y = (iris["target"] == 2).astype(np.float64) # Iris-Virginica
78. Machine Learning - SVM
Linear SVM Classification - Soft Margin
Example 1: SVM Classification for IRIS data using c = 1
Steps:
● Load the IRIS data
● Feature scaling the data
● Model the SVM Linear classifier with the training set: fitting
>>> scaler = StandardScaler()
>>> svm_clf2 = LinearSVC(C=1, loss="hinge")
>>> scaled_svm_clf2 = Pipeline((("scaler", scaler), ("linear_svc",
svm_clf2), ))
>>> scaled_svm_clf2.fit(X, y)
79. Machine Learning - SVM
Linear SVM Classification - Soft Margin
Example 1: SVM Classification for IRIS data using c = 1
Steps:
● Load the IRIS data
● Feature scaling the data
● Model the SVM Linear classifier with the training set: fitting
● Test using a sample data
>>> scaled_svm_clf1.predict([[5.5, 1.7]])
array([ 1.])
80. Machine Learning - SVM
Linear SVM Classification - Soft Margin
Example 1: SVM Classification for IRIS data using c = 1
Illustration:
● Plot the decision boundary along with the training data
○ Convert to unscaled parameters
■ Training data and decision boundary as calculated
○ Find support vectors
○ Plot it on the graph
81. Machine Learning - SVM
Linear SVM Classification - Soft Margin
Example 1: SVM Classification for IRIS data using c = 1
Illustration:
● Plot the decision boundary along with the training data
○ Convert to unscaled parameters
# Convert to unscaled parameters
>>> b2 = svm_clf1.decision_function([-scaler.mean_ / scaler.scale_])
>>> w2 = svm_clf1.coef_[0] / scaler.scale_
>>> svm_clf1.intercept_ = np.array([b2])
>>> svm_clf1.coef_ = np.array([w2])
82. Machine Learning - SVM
Linear SVM Classification - Soft Margin
Example 1: SVM Classification for IRIS data using c = 1
Illustration:
● Plot the decision boundary along with the training data
○ Find support vectors
# Find support vectors (LinearSVC does not do this automatically)
>>> t = y * 2 - 1
>>> support_vectors_idx2 = (t * (X.dot(w2) + b2) < 1).ravel()
>>> svm_clf1.support_vectors_ = X[support_vectors_idx2]
83. Machine Learning - SVM
Linear SVM Classification - Soft Margin
Example 1: SVM Classification for IRIS data using c = 1
Illustration:
● Plot the decision boundary along with the training data
○ Plot
>>> plt.plot(X[:, 0][y==1], X[:, 1][y==1], "g^")
>>> plt.plot(X[:, 0][y==0], X[:, 1][y==0], "bs")
>>> plot_svc_decision_boundary(svm_clf2, 4, 6)
>>> plt.xlabel("Petal length", fontsize=14)
>>> plt.title("$C = {}$".format(svm_clf2.C), fontsize=16)
>>> plt.axis([4, 6, 0.8, 2.8])
>>> plt.show()
84. Machine Learning - SVM
Linear SVM Classification - Soft Margin
Example 1: SVM Classification for IRIS data using c = 1
Illustration:
● Plot the decision boundary along with the training data
85. Machine Learning - SVM
Linear SVM Classification - Soft Margin
We repeat the same model for c = 100 and compare it with c =1
86. Machine Learning - SVM
Linear SVM Classification - Soft Margin
Question - What is the model we used here?
● SVC (kernel=’linear’, C=1)
● SGDClassifier(loss=’hinge’, alpha = 1/(m*c))
● LinearSVC
87. Machine Learning - SVM
Linear SVM Classification - Soft Margin
Question - What is the model we used here?
● SVC (kernel=’linear’, C=1)
● SGDClassifier(loss=’hinge’, alpha = 1/(m*c))
● Ans: LinearSVC
88. Machine Learning - SVM
Linear SVM Classification
Linear SVM
Classification
Nonlinear
SVM
Classification
SVM
Regression
Bad model
versus good-
model (Large
Margin)
Classification
Soft Margin
versus Hard-
margin
Classification
89. Machine Learning - SVM
Linear SVM Classification
Linear SVM
Classification
Nonlinear
SVM
Classification
SVM
Regression
SVC
Polynomial
Kernel +
Standard Scaler
SVC RBF
Kernel +
Standard Scaler
Polynomial
Features +
StandardScal
er +
LinearSVC
90. Machine Learning - SVM
Nonlinear SVM Classification
● Many datasets cannot be linearly separable
○ Approach 1: Add more features as polynomial features
■ Can result in a linearly separable dataset
91. Machine Learning - SVM
Nonlinear SVM Classification
Approach 1: Add more features as polynomial features
○ Question - Is this linearly separable?
92. Machine Learning - SVM
Nonlinear SVM Classification
Approach 1: Add more features as polynomial features
● Question - Is this linearly separable? - No
93. Machine Learning - SVM
Nonlinear SVM Classification
Approach 1: Add more features as polynomial features
● What if we transform this data and add a new feature that is squared
of the original dataset
Original X0 (X1) Label X0_new (X2 = X1^2)
-4 1 16
-3 1 9
-2 0 4
-1 0 1
0 0 0
1 0 1
2 0 4
3 1 9
4 1 16
94. Machine Learning - SVM
Nonlinear SVM Classification
Approach 1: Add more features as polynomial features
● We plot the new feature along with the old feature
95. Machine Learning - SVM
Nonlinear SVM Classification
Approach 1: Add more features as polynomial features
● Question - Is it linearly separable?
96. Machine Learning - SVM
Nonlinear SVM Classification
Approach 1: Add more features as polynomial features
● Question - Is it linearly separable? YES
98. Machine Learning - SVM
Nonlinear SVM Classification: Example
Approach 1: Add more features as polynomial features
● MOONS Dataset
○ Random dataset generator provided by sklearn library
○ 2d or 2 features
○ Single Label
○ Binary Classification
103. Machine Learning - SVM
Nonlinear SVM Classification: Example
● MOONS Dataset
○ Similarly generate 100 such samples
○ Plotting the dataset
104. Machine Learning - SVM
Nonlinear SVM Classification: Example
● MOONS Dataset
○ Q. How to classify this using linear classifier?
105. Machine Learning - SVM
Nonlinear SVM Classification: Example
● MOONS Dataset
○ Q. How to classify this using linear classifier?
○ Ans: Add more features as polynomial features
106. Machine Learning - SVM
Nonlinear SVM Classification: Example
● Adding polynomial features
○ What does adding polynomial features mean
○ Let us consider another example
X1 X2 Label
-0.083 0.577 1
1.071 0.205 0
1 x1 x2 x1^2 x1*x2 x2^2 Label
1 -0.083 0.577 0.007 -0.048 0.333 1
1 1.071 0.205 1.147 0.22 0.22 0
Degree = 2
107. Machine Learning - SVM
Nonlinear SVM Classification: Example
● Adding polynomial features
○ What does adding polynomial features mean
○ Let us consider another example
>>> from sklearn.preprocessing import PolynomialFeatures
>>> X, y = make_moons(n_samples=2, noise=0.15, random_state=42)
>>> np.set_printoptions(precision=2)
>>> print(X)
>>> print(y)
>>> poly=PolynomialFeatures(degree=3)
>>> x1=poly.fit_transform(X)*100
>>> print(x1)
108. Machine Learning - SVM
Nonlinear SVM Classification: Example
● Adding polynomial features
○ What does adding polynomial features mean
○ Let us consider another example
X = [[-0.08 0.58]
[ 1.07 0.21]]
y = [1 0]
X1 =
[[ 1. -0.08 0.58 0.01 -0.05 0.33 -0. 0. -0.03 0.19]
[ 1. 1.07 0.21 1.15 0.22 0.04 1.23 0.24 0.05 0.01]]
109. Machine Learning - SVM
Nonlinear SVM Classification: Example
● MOONS Dataset
○ Q. How to classify this using linear classifier?
○ Ans: Added more features as polynomial features
110. Machine Learning - SVM
Nonlinear SVM Classification: Example
● MOONS Dataset
○ Add more features with degree 3
○ Scale the new features using StandardScaler()
○ Use SVM Classifier
● All the above steps can be performed in a single iteration using a
Pipeline
112. Machine Learning - SVM
Nonlinear SVM Classification: Example
● MOONS Dataset
○ Plotting the dataset along with the classifier (decision boundary)
just modeled
116. Machine Learning - SVM
Linear SVM Classification
Linear SVM
Classification
Nonlinear
SVM
Classification
SVM
Regression
SVC
Polynomial
Kernel +
Standard
Scaler
SVC RBF
Kernel +
Standard Scaler
Polynomial
Features +
StandardScaler
+ LinearSVC
117. Machine Learning - SVM
Nonlinear SVM Classification
Polynomial Kernel
● Adding polynomial features works great
○ Low polynomial degree cannot deal with complex datasets
○ High polynomial degree makes the model slow due to huge
number of features
● How to overcome the slowness due to huge features?
● Ans: Polynomial Kernels or Kernel trick
118. Machine Learning - SVM
Nonlinear SVM Classification
Polynomial Kernel
● Adding polynomial features works great
○ Low polynomial degree cannot deal with complex datasets
○ High polynomial degree makes the model slow due to huge
number of features
● How to overcome the slowness due to huge features?
● Ans: Polynomial Kernels or Kernel trick
○ Makes it possible to get the same result as when using high
polynomial degree
○ Without having to add the features which makes the model slow
119. Machine Learning - SVM
Nonlinear SVM Classification
Polynomial Kernel in Scikit-Learn
● Can be implement in Scikit Learn using SVC Classifier
● Without having to use PolynomialFeatures as in LinearSVC
>>> from sklearn.svm import SVC
>>> poly_kernel_svm_clf = Pipeline((
("scaler", StandardScaler()),
("svm_clf", SVC(
kernel="poly", degree=3, coef0=1, C=5))))
kernel
controls how much the model is
influenced by high-degree polynomials
vs low-degree
coef0
120. Machine Learning - SVM
Nonlinear SVM Classification
Polynomial Kernel in Scikit-Learn
● Training the classifier using higher degree of polynomial features
# train SVM classifier using 10th-degree polynomial
kernel (for comparison)
>>> poly100_kernel_svm_clf = Pipeline((
("scaler", StandardScaler()),
("svm_clf", SVC(kernel="poly", degree=10,
coef0=100, C=5))
))
121. Machine Learning - SVM
Nonlinear SVM Classification
Polynomial Kernel in Scikit-Learn
● Observing the difference in the two cases
123. Machine Learning - SVM
Nonlinear SVM Classification
Linear SVM
Classification
Nonlinear
SVM
Classification
SVM
Regression
SVC
Polynomial
Kernel +
Standard Scaler
SVC RBF
Kernel +
Standard
Scaler
Polynomial
Features +
StandardScaler
+ LinearSVC
124. Machine Learning - SVM
Nonlinear SVM Classification - SVC RBF
Adding similar features
● Another technique of solving nonlinear classifications
● Add features computed using a similarity function
● Similarity function measures how each instance resembles a particular
‘landmark’
125. Machine Learning - SVM
Nonlinear SVM Classification - SVC RBF
● Is this linearly separable? NO
131. Machine Learning - SVM
Nonlinear SVM Classification - SVC RBF
● Similarity Function: Using SciKit Learn
○ Upon plotting, the difference can be observed
133. Machine Learning - SVM
Nonlinear SVM Classification - SVC RBF
Similarity Function: How to select the landmarks?
● Create a landmark at each and every instance of the dataset
Drawback
● If training set is huge, number of new features added will be huge
134. Machine Learning - SVM
Nonlinear SVM Classification - SVC RBF
● Ideally how many new features should be added in this?
Original X0 (X1) Label
-4 1
-3 1
-2 0
-1 0
0 0
1 0
2 0
3 1
4 1
135. Machine Learning - SVM
Nonlinear SVM Classification
● Ideally how many new features should be added in this? Ans: 9
Original X0 (X1) Label
-4 1
-3 1
-2 0
-1 0
0 0
1 0
2 0
3 1
4 1
136. Machine Learning - SVM
Nonlinear SVM Classification - SVC RBF
● Ideally how many new features should be added in this? Ans: 9
● The training set converts into 9 instances with 9 features
● Imagine doing this with huge training datasets
137. Machine Learning - SVM
Nonlinear SVM Classification - SVC RBF
Gaussian RBF Kernel
● Polynomial Feature addition becomes slow with higher degrees
○ Kernel trick solves it
● Similarity function becomes slow with higher number of training
dataset
○ SVM kernel trick again solves the problem
138. Machine Learning - SVM
Nonlinear SVM Classification - SVC RBF
Gaussian RBF Kernel
● It lets us to get similar results as if
○ We had added many similarity features
○ Without actually having to add them
140. Machine Learning - SVM
Nonlinear SVM Classification - SVC RBF
Gaussian RBF Kernel in ScikitLearn
● Plotting with different hyper parameters
141. Machine Learning - SVM
Nonlinear SVM Classification - SVC RBF
Gaussian RBF Kernel in Scikit-Learn
Plotting with different hyper parameters
Increasing Gamma Small Gamma
Makes bell curve narrower Makes the bell curve wider
Reduces influence of each
instance
Instances have a larger range of
influence
Decision boundary becomes
irregular
Decision boundary becomes
smoother
143. Machine Learning - SVM
Computational Complexity
Which kernel to use when?
1. Linear Kernel First
a. LinearSVC faster than SVC(kernel=’linear’) for large datasets with a
lot of features
1. Gaussian RBF kernel
1. Other kernels: Cross validation and grid search
144. Machine Learning - SVM
Computational Complexity
Linear SVC
● Based on liblinear library
● Scales linearly with number of instances and number of features
● Does not support kernel tricks
● Time complexity is: O(m * n)
145. Machine Learning - SVM
Computational Complexity
m = number of training sets
n = number of features
SVC Class
● Based on libsvm library
● Support kernel tricks
● Time complexity is: O(m^2 * n) and O(m^3 * n)
● Dreadfully slow when the number of training sets increases
● Perfect for complex but small or medium training sets
146. Machine Learning - SVM
SVM Classification - Comparison
LinearSVC SVC SGDClassifier
Fast Slow for large datasets
Perfect for small but
complex training sets
Does not converge as
fast as LinearSVC but
can be useful for
datasets that do not fit
in memory
147. Machine Learning - SVM
Linear SVM Classification
Linear SVM
Classification
Nonlinear SVM
Classification
SVM
Regression
SVC
Polynomial
Kernel +
Standard Scaler
SVC RBF
Kernel +
Standard Scaler
Polynomial
Features +
StandardScaler
+ LinearSVC
148. Machine Learning - SVM
Linear SVM Classification
Linear SVM
Classification
Nonlinear SVM
Classification
SVM
Regression
Nonlinear
SVM: SVR
Polynomial
Kernel +
degree + C +
epsilon
Linear SVM:
LinearSVR +
Epsilon
149. Machine Learning - SVM
SVM Regression
SVM Classifier SVM Regression
Find the largest possible street
between the two classes limiting
margin violations
Fit as many instances as possible on
the street while limiting margin
violations
Widest possible street
150. Machine Learning - SVM
SVM Regression - Linear
● Width of the SVM Regression model is controlled by a hyperparameter 𝜺
or epsilon.
● Adding training instances within the margin does not affect the model’s
predictions,
○ Hence model is said to be 𝜺-insensitive
151. Machine Learning - SVM
SVM Regression - Linear
Linear Regression in Scikit-Learn:
LinearSVR can be used
>>> from sklearn.svm import LinearSVR
>>> svm_reg = LinearSVR(epsilon=1.5)
>>> svm_reg.fit(X, y)
152. Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn
Step 1: Generating random numbers and making a linear relationship
>>> from sklearn.svm import LinearSVR
>>> import numpy.random as rnd
>>> import matplotlib.pyplot as plt
>>> rnd.seed(42)
>>> m = 50
>>> X = 2 * rnd.rand(m,1)
>>> y = (4 + 3 * X + rnd.randn(m,1)).ravel()
>>> plt.scatter(X,y)
>>> plt.show()
153. Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn
Step 1: Generating random numbers and making a linear relationship
154. Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn
Step 2: Fitting a linear Support Vector Regression model to the data
>>> from sklearn.svm import LinearSVR
>>> svm_reg1 = LinearSVR(epsilon=1.5)
>>> svm_reg1.fit(X,y)
>>> x1s = np.linspace(0,2,100)
>>> y1s = svm_reg1.coef_*x1s + svm_reg1.intercept_
>>> plt.scatter(X,y)
>>> plt.plot(x1s, y1s)
>>> plt.show()
155. Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn
Step 2: Fitting a linear Support Vector Regression model to the data
157. Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn
Step 3: Plotting the epsilon lines
158. Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn
Step 4: Finding the instances off-the-street and plotting
>>> y_pred = svm_reg1.predict(X)
>>> supp_vec_X = X[np.abs(y-y_pred)>1.5]
>>> supp_vec_y = y[np.abs(y-y_pred)>1.5]
>>> plt.scatter(supp_vec_X,supp_vec_y)
>>> plt.show()
159. Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn
Step 4: Finding the instances off-the-street and plotting
161. Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn with eps = 0.5
Step 1: Generating random numbers and making a linear relationship
>>> from sklearn.svm import LinearSVR
>>> import numpy.random as rnd
>>> import matplotlib.pyplot as plt
>>> rnd.seed(42)
>>> m = 50
>>> X = 2 * rnd.rand(m,1)
>>> y = (4 + 3 * X + rnd.randn(m,1)).ravel()
>>> plt.scatter(X,y)
>>> plt.show()
162. Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn
Step 1: Generating random numbers and making a linear relationship
163. Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn
Step 2: Fitting a linear Support Vector Regression model to the data
>>> from sklearn.svm import LinearSVR
>>> svm_reg1 = LinearSVR(epsilon = 0.5)
>>> svm_reg1.fit(X,y)
>>> x1s = np.linspace(0,2,100)
>>> y1s = svm_reg1.coef_*x1s + svm_reg1.intercept_
>>> plt.scatter(X,y)
>>> plt.plot(x1s, y1s)
>>> plt.show()
164. Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn
Step 2: Fitting a linear Support Vector Regression model to the data
166. Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn
Step 3: Plotting the epsilon lines
167. Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn
Step 4: Finding the instances off-the-street and plotting
>>> y_pred = svm_reg1.predict(X)
>>> supp_vec_X = X[np.abs(y-y_pred)>0.5]
>>> supp_vec_y = y[np.abs(y-y_pred)>0.5]
>>> plt.scatter(supp_vec_X,supp_vec_y)
>>> plt.show()
168. Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn
Step 4: Finding the instances off-the-street and plotting
170. Machine Learning - SVM
Linear SVM Regression - Example
Linear SVM Regression in Scikit-Learn
Comparison for epsilon = 0.5 and epsilon = 1.5, observations?
172. Machine Learning - SVM
Linear SVM Regression - Example
● Linear SVM Regression in Scikit-Learn
○ Comparison for eps = 0.5 and eps = 1.5, observations?
■ Number of instances off-the-street are higher for eps=0.5
○ Cannot conclude on which is a better model
Remember: the goal is to maximise the number of training sets within
the epsilon line
173. Machine Learning - SVM
Linear SVM Classification
Linear SVM
Classification
Nonlinear SVM
Classification
SVM
Regression
Nonlinear
SVM: SVR
Polynomial
Kernel +
degree + C +
epsilon
Linear SVM:
LinearSVR +
Epsilon
174. Machine Learning - SVM
SVM Nonlinear Regression
● A ‘kernelized’ SVM Regression model can be used
175. Machine Learning - SVM
>>> from sklearn.svm import SVR
>>> svm_poly_reg = SVR(kernel="poly", degree=2, C=100,
epsilon=0.1)
>>> svm_poly_reg.fit(X, y)
SVM Nonlinear Regression
● A ‘kernelized’ SVM Regression model can be used
● C - penalty for being outside the margin or error in classification
● Higher C -> Classification: lesser violations, Regression: lesser
regularization
● Lower C -> Classification: more violations, Regression: more
regularization
epsilon = margin parameter C = regularization parameters
176. Machine Learning - SVM
SVM Nonlinear Regression - Example 1
Nonlinear SVM Regression in Scikit-Learn for a quadratic distributed data
177. Machine Learning - SVM
Nonlinear SVM Regression in Scikit-Learn
Step 1: Generating random numbers and making a quadratic
relationship
>>> from sklearn.svm import SVR
>>> import numpy.random as rnd
>>> import matplotlib.pyplot as plt
>>> rnd.seed(42)
>>> m = 100
>>> X = 2 * rnd.rand(m,1) -1
>>> y = (0.2 + 0.1 * X + 0.5 * X**2 + rnd.randn(m, 1)/10).ravel()
>>> plt.scatter(X,y)
>>> plt.show()
SVM Nonlinear Regression - Example 1
178. Machine Learning - SVM
Nonlinear SVM Regression in Scikit-Learn
Step 1: Generating random numbers and making a quadratic
relationship
SVM Nonlinear Regression - Example 1
179. Machine Learning - SVM
Nonlinear SVM Regression in Scikit-Learn
Step 2: Fitting a Support Vector Regression model (degree=2) to the
data
>>> from sklearn.svm import SVR
>>> svr_poly_reg1 = SVR(kernel="poly", degree=2, C =
100, epsilon = 0.1)
>>> svr_poly_reg1.fit(X,y)
>>> print(svr_poly_reg1.C)
>>> x1s = np.linspace(-1,1,200)
>>> plot_svm_regression(svr_poly_reg1, X, y, [-1, 1, 0,
1])
SVM Nonlinear Regression - Example 1
180. Machine Learning - SVM
Nonlinear SVM Regression in Scikit-Learn
Step 2: Fitting a Support Vector Regression model (degree=2) to the
data
SVM Nonlinear Regression - Example 1
188. Machine Learning - SVM
SVM Nonlinear Regression - Comparison
The model as calculated with different hyper-parameters can be observed
- Higher eps: Less number of instances off-the-street
- Higher C: Less number of instances off-the-street
However, higher eps and lesser number of violations does not always imply
a better model.
Similarly, higher C can lead to overfitting
189. Machine Learning - SVM
SVM Classification Summary
Linear Classification
- Bad-model versus good-model: large-margin classification
- SVM Sensitivity to feature scaling
- Hard margin versus Soft margin classification
Nonlinear SVM Classification
- Adding polynomial features and solving using kernel trick
- Adding similarity features - Gaussian RBF function and kernel trick
Computational comparison for SVC, SVCLinear and SGDClassifier
190. Machine Learning - SVM
SVM Regression Summary
SVM Regression (Linear and Non Linear)
- SVM Linear Regression using LinearSVR and controlling the width of the margin using
epsilon
- Using Kernel-ized SVM Regression model to model non-linear models - SVR with kernel,
StandardScaler
192. Machine Learning - SVM
Linear SVM - Decision Functions
● Let petal width be denoted by x1 and petal length be denoted by x2.
● Decision Function ‘h’ can be defined as w1 * x1 + w2 * x2 + b.
○ If h < 0, then class=0, else class =1.
● It can be represented by the equation below
194. Machine Learning - SVM
Linear SVM - Decision Functions
Training the SVM Classifier would mean:
● Finding w and b such that
● Margin is as wide as possible while
● Avoiding margin violations (hard margin) or
● Limiting them (soft margin)
195. Machine Learning - SVM
Linear SVM - Decision Functions
Training the SVM Classifier would mean:
● Finding w and b such that
● Margin is as wide as possible while
● Avoiding margin violations (hard margin) or
● Limiting them (soft margin)
Q. Remember hard margin and soft margin?
196. Machine Learning - SVM
Linear SVM - Decision Functions
Training the SVM Classifier would mean:
● Finding w and b such that
● Margin is as wide as possible while
● Avoiding margin violations (hard margin) or
● Limiting them (soft margin)
Q. How do we achieve the above?
197. Machine Learning - SVM
Linear SVM - Decision Functions
Training the SVM Classifier would mean:
● Finding w and b such that
● Margin is as wide as possible while
● Avoiding margin violations (hard margin) or
● Limiting them (soft margin)
Q. How do we achieve the above?
● Optimization
198. Machine Learning - SVM
Linear SVM - Decision Functions
What do we know?
- For a 2d dataset, the slope of the margin is equal to w = [w1, w2] and
the slope of the decision boundary is equal to w1^2 + w2^2
- For a n-dimensional dataset, w = [w1, w2, w3, ... , wn] and the slope of
the decision function is denoted by || w ||
199. Machine Learning - SVM
Linear SVM - Decision Functions
What we also know,
- Smaller the weight vector, larger is the margin
200. Machine Learning - SVM
Linear SVM - Decision Functions
- So, in order to achieve the best classifier
- we can minimize || w || to maximize the margin, can we?
201. Machine Learning - SVM
Linear SVM - Decision Functions
- So, in order to achieve the best classifier
- we can minimize || w || to maximize the margin, can we?
- No
- For hard margin, we need to ensure
- Decision function > 1 for all positive training instances
- Decision function < -1 for all negative training instances
202. Machine Learning - SVM
Linear SVM - Decision Functions
So, the problem basically becomes:
Where
t(i) =1 for positive instances and t(i) = -1 for negative instances
203. Machine Learning - SVM
Linear SVM - Decision Functions
- So, in order to achieve the best classifier
- we can minimize || w || to maximize the margin, can we?
- No
- For soft margin, we need to include a slack variable to the minimization
equation
- Two conflicting goals:
- Minimize the weights matrix to maximize the margin
- Minimize the slack variable to reduce margin violations
- C hyper-parameter allows us to define the trade-off between the two
204. Machine Learning - SVM
Linear SVM - Decision Functions
So, the problem for soft-margin basically becomes:
Where
t(i) =1 for positive instances and t(i) = -1 for negative instances
205. Machine Learning - SVM
Linear SVM - Decision Functions
Both hard-margin and soft-margin problems are
- Convex quadratic problems with
- Linear constraints
Such problems are known as Quadratic Programming (QP) problems
- Can be solved using off-the-shelf solvers
- Using variety of techniques
- We will not discuss this in the session
206. Machine Learning - SVM
Linear SVM - Decision Functions
So now we know that the hard-margin and soft-margin classifiers
- Is an optimization problem to
- minimize the cost
- given certain constraints
- The optimization is a quadratic programming (QP) problem
- Which is solved using off-the-shelf solver
- Basically, the classifier function is calling a QP solver in the backend to
calculate the weights of the decision boundary
207. Machine Learning - SVM
Dual Problem
The original constrained optimization problem , known as the primal
problem, can be expressed as another closely related problem known as
dual problem
208. Machine Learning - SVM
Dual Problem
Dual problem gives a lower bound to the solution of the primal problem,
but under some circumstances gives the same result.
- SVM problems meet these conditions, hence have same solution for both
primal and dual problems.
209. Machine Learning - SVM
Dual Problem
Can be expressed as
Primal problem
Dual problem
210. Machine Learning - SVM
Dual Problem
Solution from the above dual problem can be transformed to the solution
of the original primal problem using:
211. Machine Learning - SVM
Dual Problem
Primal Problem Dual Problem
Slow to solve Faster to solve than the primal when
the number of training instances are
smaller than the number of features
Kernel trick is not possible Hence, making the kernel trick
possible
212. Machine Learning - SVM
Kernelized SVM
- When did we use SVM Kernel?
- Review (Slides 91 to 125)
213. Machine Learning - SVM
Kernelized SVM
When do we use kernelized SVMs?
- We applied a 2nd degree polynomial transformation
- And then train a linear SVM classifier on the transformed training set
214. Machine Learning - SVM
Kernelized SVM
The 2nd-degree polynomial transformed set is 3-dimensional instead of
two-dimensional. (dropping the initial features)
215. Machine Learning - SVM
Kernelized SVM
If there are two sets of 2-dimensional feature sets, a and b.
We apply 2nd degree polynomial mapping and then compute the dot
product of the transformed vectors?
- Why do we do this?
- The dual problem requires dot product of feature sets
216. Machine Learning - SVM
Kernelized SVM
- The dot product of transformed vectors
- Is equal to the square of the dot product of the original vectors
217. Machine Learning - SVM
Kernelized SVM
- Each degree transformation requires a lot of computation
- Dual problem shall contain dot product of the transformed features
matrix
- Instead, the original feature can be dot-multiplied and squared
- Transformation of the original matrix is not required
- The above trick makes the whole process much more computationally
efficient
218. Machine Learning - SVM
Kernelized SVM
- Kernel function represented by K
- Capable of computing transformed based only on the original vectors
without having to compute the transformation.
219. Machine Learning - SVM
Online SVMs
What is Online Learning?
Recap: incremental learning of the model as data gathers more
datasets on the go
221. Machine Learning
Machine Learning - Online Learning
● Train system incrementally
○ By feeding new data sequentially
○ Or in batches
● System can learn from new data on the fly
● Good for systems where data is a continuous flow
○ Stock prices
223. Machine Learning
Machine Learning - Online Learning
Using online learning to handle huge datasets
● Can be used to train huge datasets
○ That can not be fit in one machine
○ The training data gets divided into batches and
○ System gets trained on each batch incrementally
224. Machine Learning
Machine Learning - Online Learning
Challenges in online learning
● System’s performance gradually declines
○ If bad data is fed to the system
○ Bad data can come from
○ Malfunctioning sensor or robot
○ Someone spamming your system
225. Machine Learning
Machine Learning - Online Learning
Challenges in online learning
● Closely monitor the system
○ Turn off the learning if there is a performance drop
○ Or monitor the input data and remove anomalies
226. Machine Learning
Machine Learning - Online Learning
Challenges in online learning
● Closely monitor the system
○ Turn off the learning if there is a performance drop
○ Or monitor the input data and remove anomalies
227. Machine Learning
Machine Learning - Online Learning
● Can be implemented using Linear SVM classifiers
○ One method is Gradient Descent, e.g SGDClassifier
○ Covered previously in Chapter 3 and earlier in SVM Classification
● Cost function for SGD Classification can be written as
Maximizes the margin Penalty function for wrong classification
Hinge loss
228. Machine Learning
Machine Learning - Online Learning
● Online Learning can also be implemented using Kernelized SVMs
○ Currently implemented in Matlab and CPP
○ For large scale nonlinear problems, we should also consider using neural
networks which will be covered in ANN course.