SlideShare a Scribd company logo

PipelineAI Real-Time Machine Learning - Global Artificial Intelligence Conference - Santa Clara - Jan 23 2019

•
•
Chris Fregly
Chris Fregly

Perform Online Predictions using Slack A/B and multi-armed bandit model compare Train Online Models with Kafka Streams Create new models quickly Deploy to production safely Mirror traffic to validate online performance Any Framework, Any Hardware, Any Cloud Dashboard to manage the lifecycle of models from local development to live production Generates optimized runtimes for the models Custom targeting rules, shadow mode, and percentage-based rollouts to safely test features in live production Continuous model training, model validation, and pipeline optimization https://youtu.be/zpkH9oiIovU https://www.meetup.com/Advanced-Spark-and-TensorFlow-Meetup/events/258276286/ Related Links PipelineAI Home: https://pipeline.ai PipelineAI Community Edition: https://community.pipeline.ai PipelineAI GitHub: https://github.com/PipelineAI/pipeline PipelineAI Quick Start: https://quickstart.pipeline.ai Advanced Spark and TensorFlow Meetup (SF-based, Global Reach): https://www.meetup.com/Advanced-Spark-and-TensorFlow-Meetup YouTube Videos: https://youtube.pipeline.ai SlideShare Presentations: https://slideshare.pipeline.ai Slack Support: https://joinslack.pipeline.ai Web Support and Knowledge Base: https://support.pipeline.ai Email Support: help@pipeline.ai

1 of 17
Download to read offline
“Halliburton uses PipelineAI to power its Oil & Gas Vertical Cloud” (LIFE Conference Keynote 2018) “PipelineAI is… Uber Michelangelo for AI-First Enterprises.” “PipelineAI is… AWS SageMaker for Industry Vertical Clouds.” Chris Fregly Founder @ PipelineAI chris@pipeline.ai Global AI Conference Santa Clara, CA Jan 23, 2019
Problem 2 It’s Hard to Balance the 3 “Cy’s” of AI Privacy Accuracy Latency Solution: Experiment in Live Production to Find the Right Balance
Current Solution: Cloud Lock-In 3 https://aws.amazon.com/blogs/machine-learning/automated-and-continuous-deployment-of-amazon-sagemaker-models-with-aws-step-functions/ (Dec 2018)
PipelineAI Solution: 1-Click & Multi-Cloud x11Generated Models1Original Model x3Clouds 4 Arbitrage cost savings across all public cloud providers Find best performing model among all generated models
Mission & Value Proposition 5x smaller and 3x faster models Easy integration with Enterprise systems Auto-tune accuracy vs. latency vs. privacy vs. cost Safely explore new models in seconds vs. months Unified runtime across language, framework & cloud 5 The Premium Enterprise AI Runtime
Market Validation 6 Existing AI Industry Vertical Clouds GE Edison Salesforce Einstein PipelineAI-based Vertical Clouds (2018) Halliburton Open Earth Cloud, Huawei Cloud, Expedia Cloud (2019) Honeywell, ARM, Nielsen Analytics
DEMO
Perform Online Predictions using Slack A/B and multi-armed bandit model compare Train Online Models with Kafka Streams Create new models quickly Deploy to production safely Mirror traffic to validate online performance PipelineAI: Real-Time Machine Learning
Advantages of PipelineAI ● Any Framework, Any Hardware, Any Cloud ● Dashboard to manage the lifecycle of models from local development to live production ● Generates optimized runtimes for the models ● Custom targeting rules, shadow mode, and percentage-based rollouts to safely test features in live production ● Continuous model training, model validation, and pipeline optimization
Let’s start with a simple prediction... dog or cat? https://joinslack.pipeline.ai
Slack - Run Prediction with image Cat? Dog? /predict https://images.ctfassets.net/kvimhx6nhg7h/5WclEHFxUksuS2IwsUECE6/a29fa9692 0666f9d4eb7c456403e4f9d/Tan-cat-in-a-cone.png Model Variant Confidence of Each Prediction Possible Predictions
COMPOSE/ ENSEMBLE Architecture for Online Prediction /predict <img> Archive Model 3 (Canary) Model 1 Model 2 INPUT ARCHIVE RESPONSE REQUEST Select prediction with highest confidence (via customizable Objective Function) Replay for future use Compare Canary to live Model 1 and Model 2 Mirrored Traffic Live Traffic Traffic Routing /predict: Pass an image URL to classify (cat or dog) via model prediction REST API /predict_archive
Validate new model performance
Online Model Training with Streams /label <img> <label> Training Stream Distributed Filesystem Deploy model Model 3 (Canary) Train model Model 1 Model 2 /label: Add new training data (human feedback loop) to improve the model /train: Create a new model with the latest training data /deploy: Deploy the model as a Canary alongside live models /route: Mirror the live traffic to Canary to validate model performance /label_data
Slack - Train Model /label https://images.ctfassets.net/kvimhx6nhg7h/5WclEHFxUksuS2IwsUECE6/a29fa96920 666f9d4eb7c456403e4f9d/Tan-cat-in-a-cone.png cat
Slack API: Outbound Webhook to PipelineAI REST API
Thank You! 17 Privacy Accuracy Latency Contact me: chris@pipeline.ai

More Related Content

PipelineAI Real-Time Machine Learning - Global Artificial Intelligence Conference - Santa Clara - Jan 23 2019

  • 1. ��Halliburton uses PipelineAI to power its Oil & Gas Vertical Cloud” (LIFE Conference Keynote 2018) “PipelineAI is… Uber Michelangelo for AI-First Enterprises.” “PipelineAI is… AWS SageMaker for Industry Vertical Clouds.” Chris Fregly Founder @ PipelineAI chris@pipeline.ai Global AI Conference Santa Clara, CA Jan 23, 2019
  • 2. Problem 2 It’s Hard to Balance the 3 “Cy’s” of AI Privacy Accuracy Latency Solution: Experiment in Live Production to Find the Right Balance
  • 3. Current Solution: Cloud Lock-In 3 https://aws.amazon.com/blogs/machine-learning/automated-and-continuous-deployment-of-amazon-sagemaker-models-with-aws-step-functions/ (Dec 2018)
  • 4. PipelineAI Solution: 1-Click & Multi-Cloud x11Generated Models1Original Model x3Clouds 4 Arbitrage cost savings across all public cloud providers Find best performing model among all generated models
  • 5. Mission & Value Proposition 5x smaller and 3x faster models Easy integration with Enterprise systems Auto-tune accuracy vs. latency vs. privacy vs. cost Safely explore new models in seconds vs. months Unified runtime across language, framework & cloud 5 The Premium Enterprise AI Runtime
  • 6. Market Validation 6 Existing AI Industry Vertical Clouds GE Edison Salesforce Einstein PipelineAI-based Vertical Clouds (2018) Halliburton Open Earth Cloud, Huawei Cloud, Expedia Cloud (2019) Honeywell, ARM, Nielsen Analytics
  • 8. Perform Online Predictions using Slack A/B and multi-armed bandit model compare Train Online Models with Kafka Streams Create new models quickly Deploy to production safely Mirror traffic to validate online performance PipelineAI: Real-Time Machine Learning
  • 9. Advantages of PipelineAI ● Any Framework, Any Hardware, Any Cloud ● Dashboard to manage the lifecycle of models from local development to live production ● Generates optimized runtimes for the models ● Custom targeting rules, shadow mode, and percentage-based rollouts to safely test features in live production ● Continuous model training, model validation, and pipeline optimization
  • 10. Let’s start with a simple prediction... dog or cat? https://joinslack.pipeline.ai
  • 11. Slack - Run Prediction with image Cat? Dog? /predict https://images.ctfassets.net/kvimhx6nhg7h/5WclEHFxUksuS2IwsUECE6/a29fa9692 0666f9d4eb7c456403e4f9d/Tan-cat-in-a-cone.png Model Variant Confidence of Each Prediction Possible Predictions
  • 12. COMPOSE/ ENSEMBLE Architecture for Online Prediction /predict <img> Archive Model 3 (Canary) Model 1 Model 2 INPUT ARCHIVE RESPONSE REQUEST Select prediction with highest confidence (via customizable Objective Function) Replay for future use Compare Canary to live Model 1 and Model 2 Mirrored Traffic Live Traffic Traffic Routing /predict: Pass an image URL to classify (cat or dog) via model prediction REST API /predict_archive
  • 13. Validate new model performance
  • 14. Online Model Training with Streams /label <img> <label> Training Stream Distributed Filesystem Deploy model Model 3 (Canary) Train model Model 1 Model 2 /label: Add new training data (human feedback loop) to improve the model /train: Create a new model with the latest training data /deploy: Deploy the model as a Canary alongside live models /route: Mirror the live traffic to Canary to validate model performance /label_data
  • 15. Slack - Train Model /label https://images.ctfassets.net/kvimhx6nhg7h/5WclEHFxUksuS2IwsUECE6/a29fa96920 666f9d4eb7c456403e4f9d/Tan-cat-in-a-cone.png cat
  • 16. Slack API: Outbound Webhook to PipelineAI REST API
  • 17. Thank You! 17 Privacy Accuracy Latency Contact me: chris@pipeline.ai