Deep Learning in your Browser: powered by WebGL
- 1. Deep Learning and WebGL
WebGL Meetup 03/27/2018
GTC SJ Convention Center
Oswald Campesato
oswald@perceptrons.io
ocampesato@yahoo.com
- 7. AI/ML/DL: How They Differ
Traditional AI (20th century):
based on collections of rules
Led to expert systems in the 1980s
The era of LISP and Prolog
- 8. AI/ML/DL: How They Differ
Machine Learning:
Started in the 1950s (approximate)
Alan Turing and “learning machines”
Data-driven (not rule-based)
Many types of algorithms
Involves optimization
- 9. AI/ML/DL: How They Differ
Deep Learning:
Started in the 1950s (approximate)
The “perceptron” (basis of NNs)
Data-driven (not rule-based)
large (even massive) data sets
Involves neural networks (CNNs: ~1970s)
Lots of heuristics
Heavily based on empirical results
- 10. AI/ML/DL: Commonality
All of them involve a model
A model represents a system
Goal: a good predictive model
The model is based on:
Many rules (for AI)
data and algorithms (for ML)
large sets of data (for DL)
- 11. A Basic Model in Machine Learning
Let’s perform the following steps:
1) Start with a simple model (2 variables)
2) Generalize that model (n variables)
3) See how it might apply to a NN
- 12. Linear Regression
One of the simplest models in ML
Fits a line (y = m*x + b) to data in 2D
Finds best line by minimizing MSE:
m = average of x values (“mean”)
b also has a closed form solution
- 15. Linear Regression: example #1
One feature (independent variable):
X = number of square feet
Predicted value (dependent variable):
Y = cost of a house
A very “coarse grained” model
We can devise a much better model
- 16. Linear Regression: example #2
Multiple features:
X1 = # of square feet
X2 = # of bedrooms
X3 = # of bathrooms (dependency?)
X4 = age of house
X5 = cost of nearby houses
X6 = corner lot (or not): Boolean
a much better model (6 features)
- 17. Linear Multivariate Analysis
General form of multivariate equation:
Y = w1*x1 + w2*x2 + . . . + wn*xn + b
w1, w2, . . . , wn are numeric values
x1, x2, . . . , xn are variables (features)
Properties of variables:
Can be independent (Naïve Bayes)
weak/strong dependencies can exist
- 19. Neural Networks: equations
Node “values” in first hidden layer:
N1 = w11*x1+w21*x2+…+wn1*xn
N2 = w12*x1+w22*x2+…+wn2*xn
N3 = w13*x1+w23*x2+…+wn3*xn
. . .
Nn = w1n*x1+w2n*x2+…+wnn*xn
Similar equations for other pairs of layers
- 20. Neural Networks: Matrices
From inputs to first hidden layer:
Y1 = W1*X + B1 (X/Y1/B1: vectors; W1: matrix)
From first to second hidden layers:
Y2 = W2*X + B2 (X/Y2/B2: vectors; W2: matrix)
From second to third hidden layers:
Y3 = W3*X + B3 (X/Y3/B3: vectors; W3: matrix)
Apply an “activation function” to y values
- 21. Neural Networks (general)
Multiple hidden layers:
Layer composition is your decision
Activation functions: sigmoid, tanh, RELU
https://en.wikipedia.org/wiki/Activation_function
Back propagation (1980s)
https://en.wikipedia.org/wiki/Backpropagation
=> Initial weights: small random numbers
- 26. What’s the “Best” Activation Function?
Initially: sigmoid was popular
Then: tanh became popular
Now: RELU is preferred (better results)
Softmax: for FC (fully connected) layers
NB: sigmoid and tanh are used in LSTMs
- 30. CNNs versus RNNs
CNNs (Convolutional NNs):
Good for image processing
2000: CNNs processed 10-20% of all checks
=> Approximately 60% of all NNs
RNNs (Recurrent NNs):
Good for NLP and audio
- 35. How do Filters Work?
compute “inner product” with an image
The “padding” parameter
Filter size: 3x3, 5x5, or 1x1 are common
Stride values: 1 or 2 (possibly greater)
“parallelize” multiple filters then merge
- 37. CNN in Python/Keras (fragment)
from keras.models import Sequential
from keras.layers.core import Dense, Dropout, Flatten, Activation
from keras.layers.convolutional import Conv2D, MaxPooling2D
from keras.optimizers import Adadelta
input_shape = (3, 32, 32)
nb_classes = 10
model = Sequential()
model.add(Conv2D(32, (3, 3), padding='same’,
input_shape=input_shape))
model.add(Activation('relu'))
model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
- 38. What is TensorFlow?
An open source framework for ML and DL
A “computation” graph
Created by Google (released 11/2015)
Evolved from Google Brain
Linux and Mac OS X support (VM for Windows)
TF home page: https://www.tensorflow.org/
- 39. What is TensorFlow?
Support for Python, Java, C++
Desktop, server, mobile device (TensorFlow Lite)
CPU/GPU/TPU support
Visualization via TensorBoard
Can be embedded in Python scripts
Installation: pip install tensorflow
TensorFlow cluster:
https://www.tensorflow.org/deploy/distributed
- 40. TensorFlow Use Cases (Generic)
Image recognition
Computer vision
Voice/sound recognition
Time series analysis
Language detection
Language translation
Text-based processing
Handwriting Recognition
- 41. What is TensorFlow?
Graph: graph of operations (DAG)
Sessions: contains Graph(s)
lazy execution (default)
operations in parallel (default)
Nodes: operators/variables/constants
Edges: tensors
=> graphs are split into subgraphs and
executed in parallel (or multiple CPUs)
- 42. TensorFlow Graph Execution
Execute statements in a tf.Session() object
Invoke the “run” method of that object
“eager” execution is now possible
Not part of the mainline yet
Installation: pip install tf-nightly
- 43. What is a Tensor?
TF tensors are n-dimensional arrays
TF tensors are very similar to numpy ndarrays
scalar number: a zeroth-order tensor
vector: a first-order tensor
matrix: a second-order tensor
3-dimensional array: a 3rd order tensor
https://dzone.com/articles/tensorflow-simplified-
examples
- 44. TensorFlow “primitive types”
tf.constant: initialized immediately
tf.placeholder (a function):
+ initial value is not required
+ assigned value via feed_dict at run time
+ are not modified during training
tf.Variable (a class):
+ initial value is required
+ updated during training
+ in-memory buffer (saved/restored from disk)
+ can be shared between works (distributed env)
- 45. TensorFlow: constants (immutable)
import tensorflow as tf # tf-const.py
aconst = tf.constant(3.0)
print(aconst)
# output: Tensor("Const:0", shape=(), dtype=float32)
sess = tf.Session()
print(sess.run(aconst))
# output: 3.0
sess.close()
# => there's a better way…
- 47. TensorFlow Arithmetic
import tensorflow as tf # basic1.py
a = tf.add(4, 2)
b = tf.subtract(8, 6)
c = tf.multiply(a, 3)
d = tf.div(a, 6)
with tf.Session() as sess:
print(sess.run(a)) # 6
print(sess.run(b)) # 2
print(sess.run(c)) # 18
print(sess.run(d)) # 1
- 48. TensorFlow Arithmetic Methods
import tensorflow as tf #tf-math-ops.py
PI = 3.141592
sess = tf.Session()
print(sess.run(tf.div(12,8)))
print(sess.run(tf.floordiv(20.0,8.0)))
print(sess.run(tf.sin(PI)))
print(sess.run(tf.cos(PI)))
print(sess.run(tf.div(tf.sin(PI/4.), tf.cos(PI/4.))))
- 50. TF placeholders and feed_dict
import tensorflow as tf # tf-var-multiply.py
a = tf.placeholder("float")
b = tf.placeholder("float")
c = tf.multiply(a,b)
# initialize a and b:
feed_dict = {a:2, b:3}
# multiply a and b:
with tf.Session() as sess:
print(sess.run(c, feed_dict))
- 51. Deep Learning JavaScript Toolkits
Tensorfire ( https://tenso.rs )
deeplearn.js ( https://deeplearnjs.org )
keras-js ( https://github.com/transcranial/keras-js )
MXNetJS ( https://github.com/dmlc/mxnet.js )
ML.js ( https://github.com/mljs/ml )
Jeeliz ( https://jeeliz.com )
- 52. What is TensorFire?
uses WebGL to run NNs in browsers
TensorFire has two parts:
1) a GLSL-based language to write massively parallel
WebGL shaders that operate on 4D tensors
2) a high-level library for importing models trained with
Keras or TensorFlow
- 53. What is TensorFire?
imitates the structure of TensorFlow & numpy
uses WebGL JavaScript API for 2D/3D graphics
can also perform computations on the GPU
hence it offers significant performance
- 54. What is TensorFire?
it supports low-precision quantized tensors
supports browsers that don't fully support the
OES_texture_float extension
https://petewarden.com/2015/05/23/why-are-eight-
bits-enough-for-deep-neural-networks
- 55. What is TensorFire?
works on any GPU (with/without CUDA)
on computers with AMD graphics
the latest 2016 Retina Macbook Pro
https://github.com/tensorfire
- 56. What is TensorFire?
the Tensorfire demo app:
1) runs the style-transfer neural network
2) renders in your browser
3) as fast as CPU TensorFlow on a desktop
https://tenso.rs/demos/fast-neural-style/
https://tenso.rs/demos/rock-paper-scissors/
- 57. TensorFire Code Sample
Go to ‘demos’ directory
Go to backprop subdirectory
Open backprop.html
https://github.com/tensorfire/tensorfire/blob/master/demos/bac
kprop/backprop.html
- 58. What is deeplearn.js?
an open-source library
developed by the Google Brain PAIR team
API closely mirrors TensorFlow eager API
can be used with TypeScript (preferred)
can be used with plain JavaScript
- 59. What is deeplearn.js?
train NNs in a browser OR
run pre-trained models (inference mode)
A hardware-accelerated machine intelligence
library for the web
- 60. What is deeplearn.js?
install via yarn/npm:
yarn add deeplearn or npm install deeplearn
available as a global variable named dl
Use a version (e.g. 0.5.0) instead of @latest:
<script src="https://cdn.jsdelivr.net/npm/deeplearn@0.5.0">
</script>
OR
<script src="https://unpkg.com/deeplearn@latest"></script>
- 61. What is deeplearn.js?
import * as dl from 'deeplearn';
const a = dl.tensor1d([1, 2, 3]);
const b = dl.scalar(2);
const result = a.add(b);
//Float(32Array([3,4,5])
result.data().then(data => console.log(data));
// Alternatively can use a blocking call:
//Float(32Array([3,4,5])
console.log(result.dataSync());
- 62. deeplearn.js Code Sample
Go to ‘demos’ directory
Go to mnist_eager subdirectory
Open model.ts
https://github.com/PAIR-
code/deeplearnjs/blob/master/demos/mnist_eager/model.ts
- 63. deeplearn.js Useful Links
port TensorFlow models to deeplearn.js:
https://deeplearnjs.org/demos/mnist/mnist.html
API reference:
https://deeplearnjs.org/docs/api/index.html
Other samples:
https://deeplearnjs.org/index.html#demos
- 64. Deep Learning, JS, and GPU Support
Matrix comparison of JS toolkits:
https://towardsdatascience.com/gpu-accelerated-neural-
networks-in-javascript-195d6f8e69ef
- 65. About Me: Recent Books
1) HTML5 Canvas and CSS3 Graphics (2013)
2) jQuery, CSS3, and HTML5 for Mobile (2013)
3) HTML5 Pocket Primer (2013)
4) jQuery Pocket Primer (2013)
5) HTML5 Mobile Pocket Primer (2014)
6) D3 Pocket Primer (2015)
7) Python Pocket Primer (2015)
8) SVG Pocket Primer (2016)
9) CSS3 Pocket Primer (2016)
10) Android Pocket Primer (2017)
11) Angular Pocket Primer (2017)
12) Data Cleaning Pocket Primer (2018)
13) RegEx Pocket Primer (2018)
- 66. About Me: Training
=> Deep Learning. Keras, and TensorFlow:
http://codeavision.io/training/deep-learning-workshop
=> Instructor at UCSC (May/2018):
Deep Learning with TensorFlow
=> Mobile and TensorFlow Lite (WIP)
=> R and Deep Learning (WIP)
=> Android for Beginners