TensorFlow meetup: Keras - Pytorch - TensorFlow.js
- 1. Tensorflow meetup 09 Oct 2018, Ghent
- 2. - First week of February (to be announced) - Topics: TF 2.0 - TF probability - ? - Speakers: ? Next time
- 4. Today Keras Stijn Decubber @sdcubber PyTorch Xander Steenbrugge @xsteenbrugge TensorFlow.js Simon Plovyt
- 6. Deep Learning for Humans @fchollet, Keras author “High-level neural networks API, capable of running on multiple backends … focus on fast experimentation and being able to go from idea to result with the least possible delay”
- 7. https://youtu.be/dfQ8lZ9dTjs “A deep learning frontend”
- 8. Guiding principles User friendly! Simple & consistent APIs Modular Easily extensible - Sequential api - Functional api - Source code ‘Power scores’ = aggregation of job listings, online articles, github activity, arxiv mentions. Source: https://towardsdatascience.com/deep-learning-framework-power-scores-2018-23607ddf297a
- 9. Sequential API Models are linear stacks of Layers First layer requires an input shape Layers are the core building blocks of keras models
- 10. Sequential API Visualizing models The batch dimension doesn’t matter
- 11. Training the model In-memory data: numpy arrays
- 12. Training the model In-memory data: numpy arrays
- 13. Training the model Using validation data Running inference Returns a np array of predictions
- 14. Training the model Out-of-memory data: with generators Best practice is to subclass the Sequence object, which allows for safe multiprocessing __getitem__ method yields data in batches __len__ required to calculate no of batches/epoch
- 15. Training the model Out-of-memory data: with generators See stackoverflow and github for code Common use case: out-of-memory image data Read images from disk on the fly
- 16. Callbacks Enhancing the training loop Callback = an object that implements specific logic and is called by the model at various points during training Keras provides callbacks for early stopping, learning rate scheduling, tensorboard, model saving...
- 17. Saving and Restoring Models
- 18. Saving and Restoring Models
- 19. Functional API For complex architectures Sequential API: linear stacks of layers Functional API: DAG wrapped in a Keras model Layers are called on previous layers
- 20. Functional API For complex architectures Specify inputs Multiple outputs
- 21. Functional API For complex architectures
- 22. Functional API For complex architectures Define a layer once Call it twice Wrap arbitrary TF operations in a Lambda layer Call it twice
- 23. Functional API For complex architectures Define a layer once Call it twice Wrap arbitrary TF operations in a Lambda layer Call it twice
- 24. Full flexibility Activations, Losses and Optimizers
- 25. Full flexibility Activations, Losses and Optimizers
- 26. Full flexibility Activations, Losses and Optimizers
- 27. Full flexibility Ultimate flexibility: subclass source code See https://stackoverflow.com/questions/51874695/early-stop-when-validation-loss-satisfies-certain-criteria/51875492#51875492
- 28. Full flexibility Ultimate flexibility: subclass source code See https://stackoverflow.com/questions/51874695/early-stop-when-validation-loss-satisfies-certain-criteria/51875492#51875492 Solution: subclass Keras’ EarlyStopping Callback and implement custom logic
- 29. Full flexibility Ultimate flexibility: subclass source code See https://stackoverflow.com/questions/51874695/early-stop-when-validation-loss-satisfies-certain-criteria/51875492#51875492 Solution: subclass Keras’ EarlyStopping Callback and implement custom logic callbacks = [MyCallBack(threshold=0.001, min_epochs=10, verbose=1)] model.fit(features, labels, validation_data=(validation_feat, validation_labels), callbacks=callbacks, epochs=100)
- 30. Pretrained models Transfer learning & model finetuning
- 31. Pretrained models Transfer learning & model finetuning model = ResNet50(weights= 'imagenet') preds = model.predict(x) model = VGG16(weights= 'imagenet', include_top=False) features = model.predict(x) Directly use a model Extract convolutional features base_model = VGG19(weights= 'imagenet') model = Model(inputs=base_model.input, outputs=base_model.get_layer('block4_pool').output) block4_pool_features = model.predict(x) Extract features at any level of the model Functional API: DAG wrapped in a Keras model
- 32. # create the base pre-trained model base_model = InceptionV3(weights= 'imagenet', include_top= False) # add a global spatial average pooling layer x = base_model.output x = GlobalAveragePooling2D()(x) # let's add a fully-connected layer x = Dense(1024, activation='relu')(x) # and a logistic layer -- let's say we have 200 classes predictions = Dense( 200, activation='softmax')(x) # this is the model we will train model = Model(inputs=base_model.input, outputs=predictions) Pretrained models Transfer learning & model finetuning Reuse model base, put custom classifier on top
- 33. Tensorflow integration: tf.keras TensorFlow 2.0 will rely heavily on its Keras integration as thé high level API to use ● comprises full Keras API ● Better integration with TF features: estimators, dataset API… ● Fully compatible with TF serving: build & train with Keras, productionize with TF serving ● current best practice See https://www.youtube.com/watch?v=dfQ8lZ9dTjs
- 43. Getting started ● Guides and examples at https://keras.io ● Stackoverflow ● End-to-end example blogpost with tf.Keras: https://medium.com/tensorflow/training-and-servin g-ml-models-with-tf-keras-fd975cc0fa27
- 46. 46 ● Came out of the “Torch” framework, which was used at DeepMind before TensorFlow came along ● → support for Python instead of Lua led to “PyTorch” ● Shared support from: Pytorch History
- 47. 47 The Computation Graph Deep Learning libraries usually have 2 interpreters: 1. Host language (eg Python) 2. The computation graph (eg C++ backbone) Can then be optimized and run in parallel on a GPU
- 48. 48 The Static Computation Graph
- 49. 49 Dynamic Computation Graph (~ Chainer and Dynet)
- 52. 52 → Automatically runs the operation in the graph and fetches the current values & type Pytorch Printing
- 55. 55 Dynamic Computation Graph “In TensorFlow you define graph statically before a model can run. Communication with the outer world is performed via tf.Session object and tf.Placeholder which will be substituted by external data at runtime. When you write in TensorFlow sometimes you feel that your model is behind a brick wall with several tiny holes to communicate over. In PyTorch things are more imperative and dynamic: you can define, change and execute nodes as you go, no special session interfaces or placeholders. Overall, the framework is more tightly integrated with Python language and feels more native most of the times.“
- 56. 56
- 60. Flexible gradient computation across multiple subgraphs 60
- 61. Explicit control over the backpropagation process: 61
- 62. 62 Fine-tuning is God Damn Easy!
- 63. 63 Easy data augmentation at runtime:
- 64. 64 Easily customize model pipelines: VAE encoder VAE decoder Latent Encoding Input Data
- 65. 65 Easily customize model pipelines: VAE encoder VAE decoder Latent Encoding Some other Model pipeline Labels Input Data
- 66. 66 Easily customize model pipelines:
- 68. 68 Running your code on GPU requires ZERO changes
- 69. 69 Saving and restoring models Saving and Loading models is waaaay less of a hassle compared to TF ● No meta_graph, index, graph_def, session, ... ● Load a model in a single line:
- 70. 70 Saving and restoring models Saving and Loading models is waaaay less of a hassle compared to TF ● No meta_graph, index, graph_def, session, ... ● Load a model in a single line:
- 71. 71 But I need everything...
- 72. 72 Let’s talk speed Training Time (lower = better) ResNet50 VGG16 VGG19
- 73. 73 Let’s talk user base (Feb 2018)
- 74. 74 Let’s talk Cloud Integration
- 75. 75 Varia ● PyTorch easily integrates with TensorBoard! TensorBoard is just great… ● Since computation graphs in PyTorch are defined at runtime you can use your favorite Python debugging tools such as pdb, ipdb, PyCharm debugger or old trusty print statements :)
- 76. 76 Some of the big guys are already on board...
- 77. 77 Some of the big guys are already on board...
- 78. 78 One of PyTorch’s biggest strengths is its first-class Python integration, imperative style, simplicity of the API and options. These are aspects that make PyTorch good for research and hackability. One of its biggest downsides has been production-support. What we mean by production-support is the countless things one has to do to models to run them efficiently at massive scale: ● exporting to C++-only runtimes for use in larger projects ● optimizing mobile systems on iPhone, Android, Qualcomm and other systems ● using more efficient data layouts and performing kernel fusion to do faster inference (saving 10% of speed or memory at scale is a big win) ● quantized inference (such as 8-bit inference) 1.0
- 79. 79 Growing Cloud Service support: 1.0
- 80. 80 “OMG I just read this cool new paper and they have an implementation in …” “Great! I’ll be able to run this code easily in a distributed training setup and serve it to thousands of customers through an API with tf.serving” “Great! I’ll be able to dive into the code, figure out how it works and easily tweak the entire codebase into something I can use!”
- 83. Any application that can be written in JavaScript will eventually be written in JavaScript. Jeff Atwood (co-founder StackOverflow.com) 11 years ago
- 84. Just another language? [ python meetup ] [ javascript meetup ]
- 85. Just another language? The complete picture → popularity amongst developers on Stackoverflow → amount of code written (GitHub)
- 86. Just another language? In terms of Machine Learning source: State of the Developer (Q1 2017) – Machine Learning Languages Shootout
- 87. Why TensorFlow.js? TF.js is more than just TF in JavaScript
- 88. Why TensorFlow.js? Machine Learning in the browser Server Browser
- 89. Why TensorFlow.js? No drivers or installs required It just works! No ‘install’ difference with server-side ML. GPU acceleration is available through WebGL Highly interactive playground.tensorflow.com
- 90. Why TensorFlow.js? Perfect for transfer learning Hard to train entire models, but tuning is very feasible. MobileNetPoseNet
- 92. Why TensorFlow.js? Easily generalizable to edge devices No major adjustments required to provide functionality. Direct access to sensor data New applications involving camera, microphone and accelerometer. Accelerometer Camera Microphone Location
- 93. Why TensorFlow.js? Eliminating server-side processing Model has to be downloaded once, but is cached afterwards. Feedback is still possible, e.g. by exchanging model weights (“memory”).
- 94. Why TensorFlow.js? Eliminating server-side processing Eliminate data flow. No input data needs to be sent back and forth Low latency, near instant results. Huge savings in server costs.
- 95. Why TensorFlow.js? Enables the usage of private data Enables tuning the model with private data. The resulting weights can still be stored. No concerns as the data is not exchanged. Models can always be used in offline mode. No need for central data storage. Performance Data Medical Data Conversation Data Financial Data Contact Data Media
- 96. Why TensorFlow.js? Future applications Possibilities for Browser Extensions & Plug-ins Encryption Website accessibility Text autocompletion Applications for edge devices ML frameworks for Web Developers (with low latency) Recommender systems Website-specific context generation Voice controlled search engines or assistants
- 97. Why TensorFlow.js? Easy model conversion: Easy model importing: source: Nikhil Thorat and Daniel Smilkov @ TensorFlow Dev Summit 2018
- 98. Why TensorFlow.js? Good Great Summary A tiny bit of JavaScript versus Low latency predictions Privacy guarantees Avoiding data flow and elimination of server costs New applications