SlideShare a Scribd company logo

Operationalizing Machine Learning Using GPU-accelerated, In-database Analytics

•
•
Kinetica
Kinetica

Mate Radalj's presentation on how to operationalize machine learning using GPU-accelerated, in-database analytics, given at the Bay Area GPU-Accelerated Computing Meetup on October 19, 2017. Presentation includes use cases and links to demos.

1 of 27
Download to read offline
OPERATIONALIZING MACHINE LEARNING USING GPU ACCELERATED, IN-DATABASE ANALYTICS 1
Why a GPU Database? • Leverage Innovations in CPU and GPU technology • Big Data • Traditional Analytics • Emerging AI/ML/Deep Learning Computing • Real-Time Ingestion • Geospatial and Temporal • Single Hardware Platform • Simplified Software Stack 3
AI/ML/Deep Learning Lifecycle •Create, extract, transform, and process big data: batch and streams • Apply ML to data. •Model pre-processing •Model execution •Model post-processing •Within an ecosystem of general analytics •Supporting a range of human and machine consumers 9
4 AI/ML/Deep Learning Lifecycle
5 Typical AI Process: High Latency, Rigid, Complex HW and SW Stack SPECIALIZED AI/ DATA SCIENCE TOOLS SUBSET DATA SCIENTISTSBUSINESS USERS EXTRACT EXTRACTING DATA FOR AI IS EXPENSIVE AND SLOW ENTERPRISES STRUGGLE TO MAKE AI MODELS AVAILABLE TO BUSINESS ??? • MapReduce • Spark • Cassandra • SQL Databases • DFS • CPU Compute Nodes • GPU Compute Nodes Proliferation of Hardware & Software Components
Kinetica: A More Ideal AI Process 6 Monte Carlo Risk Custom Function 2 Custom Function 3 API EXPOSES CUSTOM FUNCTIONS WHICH CAN BE MADE AVAILABLE TO BUSINESS USERS BUSINESS USERS DATA SCIENTISTS UDFs Single Hardware Platform • Analytics • AI/ML/Deep Learning • Power of in-memory SQL • Integrated CPU/GPU • Bomb with Streams
Current Inefficient Use of Python 7 python • Interpreted • Single threaded • Clean, transform • Flow: for each member • Pre-process • Model execute • Post-process =
Optimized SQL and Python UDF with Kinetica 8 = SQL UDF python SQL • Pre-process • Binary executable code • Superior optimization • declarative SQL • Model execute • Only essential imperative model code • Not relational set processing • Post-process • Binary executable code • Superior optimization • Declarative SQL
Various ETL/ELT Head Node Worker 1 KINETICA: 10 Node Cluster Worker 9 Fact and dimensions tables for various Use Cases Billions of rows Massive Stream Ingestion Massive Fast Analytics Apache Tomcat Applications Servers • Spring Endpoint oriented architecture • Horizontal elastic scaling Full Model Pipeline 1 Various ETL/ELT Full Model Pipeline N Prompts Project Overall technology Architecture 9 Fast Streaming Projects Fast Analytics Projects
Use Case Examples
MNIST: Simple Image Processing Use Case 11 A Parametric Model: Python Using TensorFlow Model Training • Set of image files stored in Kinetica Database Table • Grey: 1 channel, 2D • Color: 3 channel, 3D • Python UDF in Kinetica using TensorFlow • Convert each image in Kinetica table to flattened 1D Array and insert into data frame • Or use raw image format and insert into data frame. • Call TensorFlow: convert data frame to tensors • N layer Neural Network • Runs on GPU • Output = table TFModel • coefficients Model Serving • Python UDF in Kinetica using TensorFlow • Input = table TFModel table. • Output = table mnist_inference_out Model Analytics • SQL!!! Predict images of numbers: 0, 1, 2, 4, 4, 5, 6, 7, 8, 9
UDF train_nd_udf.py Machine 0 Rank 0 Tom 0 Table mnist_training Shard 0 Table TFModel Shard 0 Table mnist_inference Shard 0 Table mnist_inference_out Shard 0 Tom 1 Table mnist_training Shard 1 Table TFModel Shard 1 Table mnist_inference Shard 1 Table mnist_inference_out Shard 1 Tom 2 Table mnist_training Shard 2 Table TFModel Shard 2 Table mnist_inference Shard 2 Table mnist_inference_out Shard 2 Tom 3 Table mnist_training Shard 3 Table TFModel Shard 3 Table mnist_inference Shard 3 Table mnist_inference_out Shard 3 Machine 0 Rank 0 Tom 4 Table mnist_training Shard 4 Table TFModel Shard 4 Table mnist_inference Shard 4 Table mnist_inference_out Shard 4 Tom 5 Table mnist_training Shard 5 Table TFModel Shard 5 Table mnist_inference Shard 5 Table mnist_inference_out Shard 5 Tom 6 Table mnist_training Shard 6 Table TFModel Shard 6 Table mnist_inference Shard 6 Table mnist_inference_out Shard 6 Tom 7 Table mnist_training Shard 7 Table TFModel Shard 7 Table mnist_inference Shard 7 Table mnist_inference_out Shard 7 UDF UDF UDF UDF UDF UDF UDF UDF Model Training & Inference Data Model: MPP Sharding 12
Demo’s 13 UDF in Kinetica 1. Write the UDF: pythjon, java, c/c++, and javascript. 2. Register the UDF 3. Invoke the UDF https://bitbucket.org/gisfederal/gsk-imagerecognition/src https://bitbucket.org/gisfederal/gsk-imagerecognition/src
Amit Vij | CEO | Kinetica 14 Bringing it All Together with Geospatial
15 IoT Data Challenges and Geospatial EXPLOSION OF DATA Structured and unstructured Devices, Sensors Industrial IoT REAL-TIME DEMANDS Current Technology: I/O Bound Compute Bound EXISTING SOLUTIONS NOT WORKING Too Complex Batch Processing Duct taping 5-10 technologies
16 Accelerated Geospatial with Kinetica | Fast, Scalable, Flexible Solution • Full data provisioning • Scale and speed • Flexibility • Simplicity Bonus • Converge AI and BI • Streaming Analytics
Kinetica Database | Geospatial 101 17 Geospatial Objects z Points Lines Polygons Tracks Labels Spatial Operations Accelerated Spatial Operations SQL Expression & API Support Spatial Queries, Filters & Joins Geospatial Event Triggers Geospatial Visualization Server-side Rendering Vector data via WMS Complex Symbology Support Several Built-in Geospatial Renderers 1 2 3
Use Cases 18
Kinetica Machine Learning Use Cases . 19 OLAP Performance, Scalability, Stability Geospatial Processing & Visualization API for GPU Powered Data & Compute Orchestration • Activity Based Intelligence • Oil & Gas • Drilling optimization • Logistics • Last Mile • Fleet Management Full Data Science Model Pipeline ML / AI Augmentation Geospatial Fast Ingest, Fast Streaming, and Fast Analytics • Supply Chain Management • Replenishment: Real- time mass streaming ingest and analytics • Integrated planning: Massively concurrent, high throughput analytics
INTELLIGENCE | US Army - INSCOM Oracle Spatial (92 Minutes) 42x Lower Space 28x Lower Cost 38x Lower Power Cost U.S Army INSCOM Shift from Oracle to GPUdb GPUdb (20ms) 1 GPUdb server vs 42 servers with Oracle 10gR2 (2011) NEW CAPABILITIES DELIVERED • Intel analysts can do real-time geospatial analytics on over 200 streaming data feeds • Military analysts are able to query and visualize billions to trillions of near real-time objects SOLUTION OVERVIEW • US Army’s in-memory computational engine for geospatial and temporal data. • Queries down from 92 minutes to less than 1 second • Replaced 42 Oracle 10gR2 servers with a single Kinetica server – 42x lower space, 28x lower cost, 38X lower power cost 20
LOGISTICS | Workforce optimization NEW CAPABILITIES DELIVERED • Real-time delivery and pickup notifications, shipment routing, just-in-time supplies • Real-time route optimization - route planning, rerouting • Geospatial analytics to uncover overlapping coverage areas, uncovered areas, and distribution bottlenecks SOLUTION OVERVIEW • Collect, process, and analyze 200,000 messages per minute for real-time streaming analytics. 15,000 daily sessions with 5 9’s uptime 21
PIPE LINE & WELL RESEARCH | Location-based analytics 22 NEW CAPABILITIES DELIVERED • Geospatial visualization and analytics of massive number of wells, pipelines by land ownership, region etc. • Custom visualizations and charts for data-driven insights • Embedded solution with seamless Node.js integration, GPU acceleration SOLUTION OVERVIEW • Kinetica running in RSEG’s Amazon Web Services VPC deployment
Automotive | Connected Car Analytics with Machine Learning 23 Activity-Based Intelligence • Behavioral analytics • Ex: Lost driver alerts • GPS data + real-time route tracking • Rule-based analytics • Ex: Dangerous driving • Rainy weather + condition of road + MPH • Complicated pattern recognition • Ex: Tailgating • Collecting speed and brake data • Model trained to classify driving patterns Intelligence Streaming Data Actuation loop
24 Demos
On-the-fly Time Referenced Server-side Video Generation
On-the-fly Time Referenced Heatmap Video Generation
Contact: / kinetica.com / Email: info@kinetica.com Thank You! /www.nvidia.com/analytics /www.nvidia.com/dgx1 /Email: dgxanalytics@nvidia.com Save the Date December 7th, Kinetica HQ, San Francisco Holiday Networking and ”Housewarming” Event for new Kinetica SF HQ

More Related Content

Operationalizing Machine Learning Using GPU-accelerated, In-database Analytics

  • 1. OPERATIONALIZING MACHINE LEARNING USING GPU ACCELERATED, IN-DATABASE ANALYTICS 1
  • 2. Why a GPU Database? • Leverage Innovations in CPU and GPU technology • Big Data • Traditional Analytics • Emerging AI/ML/Deep Learning Computing • Real-Time Ingestion • Geospatial and Temporal • Single Hardware Platform • Simplified Software Stack 3
  • 3. AI/ML/Deep Learning Lifecycle •Create, extract, transform, and process big data: batch and streams • Apply ML to data. •Model pre-processing •Model execution •Model post-processing •Within an ecosystem of general analytics •Supporting a range of human and machine consumers 9
  • 5. 5 Typical AI Process: High Latency, Rigid, Complex HW and SW Stack SPECIALIZED AI/ DATA SCIENCE TOOLS SUBSET DATA SCIENTISTSBUSINESS USERS EXTRACT EXTRACTING DATA FOR AI IS EXPENSIVE AND SLOW ENTERPRISES STRUGGLE TO MAKE AI MODELS AVAILABLE TO BUSINESS ??? • MapReduce • Spark • Cassandra • SQL Databases • DFS • CPU Compute Nodes • GPU Compute Nodes Proliferation of Hardware & Software Components
  • 6. Kinetica: A More Ideal AI Process 6 Monte Carlo Risk Custom Function 2 Custom Function 3 API EXPOSES CUSTOM FUNCTIONS WHICH CAN BE MADE AVAILABLE TO BUSINESS USERS BUSINESS USERS DATA SCIENTISTS UDFs Single Hardware Platform • Analytics • AI/ML/Deep Learning • Power of in-memory SQL • Integrated CPU/GPU • Bomb with Streams
  • 7. Current Inefficient Use of Python 7 python • Interpreted • Single threaded • Clean, transform • Flow: for each member • Pre-process • Model execute • Post-process =
  • 8. Optimized SQL and Python UDF with Kinetica 8 = SQL UDF python SQL • Pre-process • Binary executable code • Superior optimization • declarative SQL • Model execute • Only essential imperative model code • Not relational set processing • Post-process • Binary executable code • Superior optimization • Declarative SQL
  • 9. Various ETL/ELT Head Node Worker 1 KINETICA: 10 Node Cluster Worker 9 Fact and dimensions tables for various Use Cases Billions of rows Massive Stream Ingestion Massive Fast Analytics Apache Tomcat Applications Servers • Spring Endpoint oriented architecture • Horizontal elastic scaling Full Model Pipeline 1 Various ETL/ELT Full Model Pipeline N Prompts Project Overall technology Architecture 9 Fast Streaming Projects Fast Analytics Projects
  • 11. MNIST: Simple Image Processing Use Case 11 A Parametric Model: Python Using TensorFlow Model Training • Set of image files stored in Kinetica Database Table • Grey: 1 channel, 2D • Color: 3 channel, 3D • Python UDF in Kinetica using TensorFlow • Convert each image in Kinetica table to flattened 1D Array and insert into data frame • Or use raw image format and insert into data frame. • Call TensorFlow: convert data frame to tensors • N layer Neural Network • Runs on GPU • Output = table TFModel • coefficients Model Serving • Python UDF in Kinetica using TensorFlow • Input = table TFModel table. • Output = table mnist_inference_out Model Analytics • SQL!!! Predict images of numbers: 0, 1, 2, 4, 4, 5, 6, 7, 8, 9
  • 12. UDF train_nd_udf.py Machine 0 Rank 0 Tom 0 Table mnist_training Shard 0 Table TFModel Shard 0 Table mnist_inference Shard 0 Table mnist_inference_out Shard 0 Tom 1 Table mnist_training Shard 1 Table TFModel Shard 1 Table mnist_inference Shard 1 Table mnist_inference_out Shard 1 Tom 2 Table mnist_training Shard 2 Table TFModel Shard 2 Table mnist_inference Shard 2 Table mnist_inference_out Shard 2 Tom 3 Table mnist_training Shard 3 Table TFModel Shard 3 Table mnist_inference Shard 3 Table mnist_inference_out Shard 3 Machine 0 Rank 0 Tom 4 Table mnist_training Shard 4 Table TFModel Shard 4 Table mnist_inference Shard 4 Table mnist_inference_out Shard 4 Tom 5 Table mnist_training Shard 5 Table TFModel Shard 5 Table mnist_inference Shard 5 Table mnist_inference_out Shard 5 Tom 6 Table mnist_training Shard 6 Table TFModel Shard 6 Table mnist_inference Shard 6 Table mnist_inference_out Shard 6 Tom 7 Table mnist_training Shard 7 Table TFModel Shard 7 Table mnist_inference Shard 7 Table mnist_inference_out Shard 7 UDF UDF UDF UDF UDF UDF UDF UDF Model Training & Inference Data Model: MPP Sharding 12
  • 13. Demo’s 13 UDF in Kinetica 1. Write the UDF: pythjon, java, c/c++, and javascript. 2. Register the UDF 3. Invoke the UDF https://bitbucket.org/gisfederal/gsk-imagerecognition/src https://bitbucket.org/gisfederal/gsk-imagerecognition/src
  • 14. Amit Vij | CEO | Kinetica 14 Bringing it All Together with Geospatial
  • 15. 15 IoT Data Challenges and Geospatial EXPLOSION OF DATA Structured and unstructured Devices, Sensors Industrial IoT REAL-TIME DEMANDS Current Technology: I/O Bound Compute Bound EXISTING SOLUTIONS NOT WORKING Too Complex Batch Processing Duct taping 5-10 technologies
  • 16. 16 Accelerated Geospatial with Kinetica | Fast, Scalable, Flexible Solution • Full data provisioning • Scale and speed • Flexibility • Simplicity Bonus • Converge AI and BI • Streaming Analytics
  • 17. Kinetica Database | Geospatial 101 17 Geospatial Objects z Points Lines Polygons Tracks Labels Spatial Operations Accelerated Spatial Operations SQL Expression & API Support Spatial Queries, Filters & Joins Geospatial Event Triggers Geospatial Visualization Server-side Rendering Vector data via WMS Complex Symbology Support Several Built-in Geospatial Renderers 1 2 3
  • 19. Kinetica Machine Learning Use Cases . 19 OLAP Performance, Scalability, Stability Geospatial Processing & Visualization API for GPU Powered Data & Compute Orchestration • Activity Based Intelligence • Oil & Gas • Drilling optimization • Logistics • Last Mile • Fleet Management Full Data Science Model Pipeline ML / AI Augmentation Geospatial Fast Ingest, Fast Streaming, and Fast Analytics • Supply Chain Management • Replenishment: Real- time mass streaming ingest and analytics • Integrated planning: Massively concurrent, high throughput analytics
  • 20. INTELLIGENCE | US Army - INSCOM Oracle Spatial (92 Minutes) 42x Lower Space 28x Lower Cost 38x Lower Power Cost U.S Army INSCOM Shift from Oracle to GPUdb GPUdb (20ms) 1 GPUdb server vs 42 servers with Oracle 10gR2 (2011) NEW CAPABILITIES DELIVERED • Intel analysts can do real-time geospatial analytics on over 200 streaming data feeds • Military analysts are able to query and visualize billions to trillions of near real-time objects SOLUTION OVERVIEW • US Army’s in-memory computational engine for geospatial and temporal data. • Queries down from 92 minutes to less than 1 second • Replaced 42 Oracle 10gR2 servers with a single Kinetica server – 42x lower space, 28x lower cost, 38X lower power cost 20
  • 21. LOGISTICS | Workforce optimization NEW CAPABILITIES DELIVERED • Real-time delivery and pickup notifications, shipment routing, just-in-time supplies • Real-time route optimization - route planning, rerouting • Geospatial analytics to uncover overlapping coverage areas, uncovered areas, and distribution bottlenecks SOLUTION OVERVIEW • Collect, process, and analyze 200,000 messages per minute for real-time streaming analytics. 15,000 daily sessions with 5 9’s uptime 21
  • 22. PIPE LINE & WELL RESEARCH | Location-based analytics 22 NEW CAPABILITIES DELIVERED • Geospatial visualization and analytics of massive number of wells, pipelines by land ownership, region etc. • Custom visualizations and charts for data-driven insights • Embedded solution with seamless Node.js integration, GPU acceleration SOLUTION OVERVIEW • Kinetica running in RSEG’s Amazon Web Services VPC deployment
  • 23. Automotive | Connected Car Analytics with Machine Learning 23 Activity-Based Intelligence • Behavioral analytics • Ex: Lost driver alerts • GPS data + real-time route tracking • Rule-based analytics • Ex: Dangerous driving • Rainy weather + condition of road + MPH • Complicated pattern recognition • Ex: Tailgating • Collecting speed and brake data • Model trained to classify driving patterns Intelligence Streaming Data Actuation loop
  • 25. On-the-fly Time Referenced Server-side Video Generation
  • 26. On-the-fly Time Referenced Heatmap Video Generation
  • 27. Contact: / kinetica.com / Email: info@kinetica.com Thank You! /www.nvidia.com/analytics /www.nvidia.com/dgx1 /Email: dgxanalytics@nvidia.com Save the Date December 7th, Kinetica HQ, San Francisco Holiday Networking and ”Housewarming” Event for new Kinetica SF HQ