AMF305_Autonomous Driving Algorithm Development on Amazon AI
•
1 like•798 views
Over the next decade, accelerating autonomous driving technology—including advances in artificial intelligence, sensors, cameras, radar and data analytics—are set to transform how we commute. In this session, you learn how to use Amazon AI for a highly productive, on demand, and scalable autonomous driving development environment. We compare the most popular AI frameworks including TensorFlow and MXNet for use in autonomous driving workloads. You learn about the AWS optimizations on MXNet that yield near linear scalability for training deep neural networks and convolutional neural networks. We demonstrate the ease of getting started on AWS AI by using a sample training dataset for building an object detection model on AWS. This session is intended for audiences who have some exposure to the underlying concepts for AI-based autonomous driving development. After attending the session, you can get started with AI development on AWS by using a sample dataset for building an object detection model.
Report
Share
AMF305_Autonomous Driving Algorithm Development on Amazon AI
- 1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS re:INVENT Autonomous Driving Algorithm Development on Amazon AI S u n i l M a l l y a , S r . A I S o l u t i o n s A r c h i t e c t , A W S D e e p L e a r n i n g A M F 3 0 5 N o v e m b e r 2 7 , 2 0 1 7
- 2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Agenda • Introduction to autonomous vehicles: Development and deployment • Components of autonomous vehicle platforms • Application of deep learning/AI in autonomous vehicles • AWS AI overview • Datasets to get started with deep learning for autonomous vehicles • Live demo—train a simple neural network for autonomous driving • AWS connected vehicles: Architecture • Customer examples: Autonomous technology development on AWS
- 3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Introduction—what does AD involve? What’s involved Development, testing, and validation of AI/ML algorithms with GPUs, AI Frameworks, training data sets Computer vision and detection algorithms, e.g., Lane detection, signals… Localization Path planning HD maps: Data collection, annotation, establishing ground truth 1. Autonomous driving development What’s involved Sensors and managing sensor data HD maps and other inputs Edge computing for sensor fusion Edge computing for execution of AI/ML algorithms 2. Autonomous driving runtime
- 4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Introduction Requirements: Terabyte-scale data acquisition, petabyte-scale data storage and management HD map creation Image tagging Algorithm training—ML and CNN Regression testing 1. Autonomous driving development Requirements: HD map updates and V2V Localization and event processing Caching, in-vehicle edge computing 2. Autonomous driving runtime
- 5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. One platform for autonomous driving needs Requirements: Terabyte-scale data acquisition, petabyte-scale data storage and management HD map creation Image tagging Algorithm training—ML and CNN Regression testing 1. Autonomous Driving Development Requirements: HD map updates and V2V Localization and event processing Caching, in-vehicle edge computing 2. Autonomous Driving Runtime Petabyte-scale, low cost, secure data transfer and storage Cloud-native machine learning, deep learning, and AI High performance compute including GPU and FGPA instances, on-demand AWS Capabilities: Serverless architecture Content delivery with smart compression Secure device integration with cloud edge compute AWS Capabilities: Snowball Amazon S3Amazon Glacier Amazon ML AWS Deep Learning Amazon Rekognition HPC FPGA Instances GPU Instances P2F1 CloudFront Acceleration Greengrass Lambda IoT IAM API Gateway
- 6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AI and deep learning for autonomous vehicles
- 7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AI, machine learning, and deep learning
- 8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS AI stack
- 9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. But where’s the data for deep learning?
- 10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Public datasets • Object detection • KITTI Vision benchmark: www.cvlibs.net/datasets/kitti/ • Street Scenes: https://www.cityscapes-dataset.com/ • UCSD LISA: http://cvrr.ucsd.edu/LISA/datasets.html • Umich/ Ford: http://robots.engin.umich.edu/SoftwareData/Ford • Self-driving • Udacity self-driving: https://github.com/udacity/self-driving-car/tree/master/datasets • Oxford robocar: http://ori.ox.ac.uk/the-oxford-robotcar-dataset/ • MIT self-driving: http://selfdrivingcars.mit.edu/ • Autonomous car dataset: http://cvssp.org/data/diplecs/ • Comma.ai: https://archive.org/details/comma-dataset • Thunderhill self-racing cars: http://data.selfracingcars.com/
- 11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Data collection and organization Direct Connect Amazon S3 Kinesis Snowball Snowmobile IoT Database Migration DynamoDB Elasticsearch CloudSearch Amazon Glacier Amazon Aurora Streams, Firehose, Analytics Upload RDS MYSQL, Postgres
- 12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Data collection and organization Direct Connect Amazon S3 Kinesis Snowball Snowmobile IoT Database Migration DynamoDB Elasticsearch CloudSearch Amazon Glacier Amazon Aurora Streams, Firehose, Analytics Upload Amazon EC2 Amazon EMR Amazon Redshift Athena Amazon QuickSight Amazon AI Lambda RDS MYSQL, Postgres
- 13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Data collection and organization Direct Connect Amazon S3 Kinesis Snowball Snowmobile IoT Database Migration DynamoDB Elasticsearch CloudSearch Amazon Glacier RDS MYSQL, Postgres Amazon Aurora Streams, Firehose, Analytics Upload Amazon EC2 Amazon EMR Amazon Redshift Athena Amazon QuickSight Amazon AI Lambda AWS GLUE MTurk
- 14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Transfer data in to AWS AWS Snowball Edge 100 TB data transport device with on-board compute
- 15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Transfer data in to AWS: 1 PB
- 16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Let’s build a simple self-driving car neural network
- 17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Recap 1.Use the Deep Learning AMI to setup the environment 2.Set up SSH tunnel for your Jupyter Notebook 3.Download the dataset from https://github.com/awslabs/RobocarRally 4.Clone the GitHub repo and use it as reference to build your first autonomous driving model
- 18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. data = mx.symbol.Variable('data') # first conv layer conv1 = mx.sym.Convolution(data=data, kernel=(5,5), num_filter=20) tanh1 = mx.sym.Activation(data=conv1, act_type=”relu") pool1 = mx.sym.Pooling(data=tanh1, pool_type="max", kernel=(2,2), stride=(2,2)) # second conv layer conv2 = mx.sym.Convolution(data=pool1, kernel=(5,5), num_filter=50) tanh2 = mx.sym.Activation(data=conv2, act_type=”relu") pool2 = mx.sym.Pooling(data=tanh2, pool_type="max", kernel=(2,2), stride=(2,2)) # first fullc layer flatten = mx.sym.Flatten(data=pool2) fc1 = mx.symbol.FullyConnected(data=flatten, num_hidden=500) tanh3 = mx.sym.Activation(data=fc1, act_type=”relu") # second fullc fc2 = mx.sym.FullyConnected(data=tanh3, num_hidden=10) # softmax loss lenet = mx.sym.SoftmaxOutput(data=fc2, name='softmax') Simple convolutional neural net in MXNet
- 19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Connected vehicles on AWS
- 20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS-connected vehicle: Act locally Respond to local events quickly Operate offline Simplified device programming Reduce the cost of running IoT applications Local Lambda Local Device Shadows Local Security Greengrass AWS Local Broker
- 21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS-connected vehicle architecture
- 22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Customer stories: Autonomous vehicles on AWS
- 23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. TuSimple, a leader in self-driving technology, uses Apache MXNet to build sophisticated deep learning algorithms for computer vision and driving simulation. Relies on MXNet to teach computers how to recognize and track objects and to make decisions to avoid collisions and prioritize safety. Simulated a billion miles of road driving with a wide range of variables and driving conditions—the largest simulation of its kind in history. Used NVIDIA GPUs, NVIDIA DRIVE PX 2, Jetson TX2, CUDA, TensorRT, and cuDNN to develop its autonomous driving solution.
- 24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Using deep learning to map a virtual world Mapillary uses deep learning to create street-level virtual environments by stitching together crowd-sourced photos. Applied fine-grain computer vision algorithms to combine 142 million user-submitted images, creating nearly 2 million miles of mapped roads. Accelerated training and inference of deep neural networks for graphic-intensive workloads using AWS EC2 P2 and G2 instances. Use Caffe and TensorFlow to gain insight from large volumes of unstructured public data to improve global mobility and transportation.
- 25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Assisting drone navigation with deep learning Iris Automation uses computer vision and deep learning to help unmanned aerial vehicles (UAVs) detect objects and avoid collisions. Analyzes and draws insights from videos captured by drone’s cameras in real time—the system has a detection range of over 500 meters, significantly farther than what other systems can currently do. Uses NVIDIA GPUs on AWS to train deep learning models and a Jetson TX1 onboard the UAV to analyze video capture in real time.
- 26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Robocar Rally 2017 Don’t miss it !! Pinyon Ballroom, ARIA Hotel Race 9 p.m.–12 midnight 11/27 https://reinvent.awsevents.com/learn/robocar-rally/
- 28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Thank you! s m a l l y a @ a m a z o n . c o m