SlideShare a Scribd company logo

AWS re:Invent 2016: Serverless Computing Patterns at Expedia (SVR306) )

•
•
Amazon Web Services
Amazon Web Services

In the middle of 2015, Expedia started using AWS Lambda for serverless computing. We built boilerplate templates in Node.js, Java, and Python so development teams could build and deploy serverless applications into AWS. Currently, we have 300 AWS Lambda functions processing 40 million invocations per day. In this session, we will discuss how development teams use boilerplate templates to create serverless applications with Amazon API Gateway and AWS Lambda and how they deploy them to AWS. We will cover patterns, architectural design choices, and the benefits --- like cost, scale, availability, and operations --- of running serverless applications.

1 of 73
Download to read offline
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Kuldeep Chowhan 12/01/2016 SVR306 Serverless Computing Patterns @ Expedia, Inc.
Kuldeep Chowhan Principal Engineer @ Expedia, Inc. @this_is_kuldeep
Expedia, Inc., at a glance Notes: Expedia, Inc. results shown for 2015. 1Gross bookings is an operating and statistical metric that captures the total dollar value, generally inclusive of taxes and fees, of travel services booked by our customers; gross bookings excludes HomeAway. 2Monthly visits based on data for Brand Expedia (including Travelocity and Wotif), Hotels.com, Orbitz Worldwide, HomeAway, trivago and Hotwire combined during 2015. Sources: Overall travel industry growth rate based on Phocuswright data for 2015 Y/Y. Largest travel company $60B gross bookings1 $6.6B revenue growing >2X faster vs. industry 18K+ employees globally 450M+ monthly site visits2 75+ countries served 1.5M+ lodging options for travelers
What to Expect from the Session • AWS Lambda Usage at Expedia • Serverless Apps Creation at Expedia • Different Serverless App architectures • Demo
AWS Usage @ Expedia
AWS re:Invent 2016: Serverless Computing Patterns at Expedia (SVR306) )
What is AWS Lambda? • Run code for virtually any type of application • No Servers to Manage • Zero administration • Continuous Scaling • Sub second Metering
How does AWS Lambda works?
Primer – AWS Lambda Creation
AWS re:Invent 2016: Serverless Computing Patterns at Expedia (SVR306) )
AWS re:Invent 2016: Serverless Computing Patterns at Expedia (SVR306) )
AWS re:Invent 2016: Serverless Computing Patterns at Expedia (SVR306) )
Demo
Serverless Architectures @ Expedia
CI/CD Platform utilizing Serverless Computing
CI/CD System Platform utilizing Serverless Computing GitHub
CI/CD System Platform utilizing Serverless Computing GitHub Webhooks (Push Events)
CI/CD System Platform utilizing Serverless Computing GitHub Webhooks (Push Events) Amazon API Gateway
CI/CD System Platform utilizing Serverless Computing GitHub Webhooks (Push Events) Amazon API Gateway Lambda Decider/Decorator
CI/CD System Platform utilizing Serverless Computing GitHub Webhooks (Push Events) Amazon API Gateway Amazon SNS Lambda Decider/Decorator
CI/CD System Platform utilizing Serverless Computing GitHub Webhooks (Push Events) Amazon API Gateway Amazon SNS Lambda Deployment Config Changes Lambda Decider/Decorator
CI/CD System Platform utilizing Serverless Computing GitHub Webhooks (Push Events) Amazon API Gateway Amazon SNS Lambda Deployment Config Changes Amazon Elasticsearch Service Lambda Decider/Decorator
CI/CD System Platform utilizing Serverless Computing GitHub Webhooks (Push Events) Amazon API Gateway Amazon SNS Lambda Deployment Config Changes Lambda Developers pushing to master directly Amazon Elasticsearch Service Lambda Decider/Decorator
CI/CD System Platform utilizing Serverless Computing GitHub Webhooks (Push Events) Amazon API Gateway Amazon SNS Lambda Deployment Config Changes Lambda Developers pushing to master directly Amazon Elasticsearch Service Amazon SES Lambda Decider/Decorator
CI/CD System Platform utilizing Serverless Computing GitHub Webhooks (Push Events) Amazon API Gateway Amazon SNS Lambda Deployment Config Changes Lambda App Details Changes Lambda Developers pushing to master directly Amazon Elasticsearch Service Amazon SES Lambda Decider/Decorator
CI/CD System Platform utilizing Serverless Computing GitHub Webhooks (Push Events) Amazon API Gateway Amazon SNS Lambda Deployment Config Changes Lambda App Details Changes Lambda Developers pushing to master directly Amazon DynamoDB Amazon Elasticsearch Service Amazon SES Lambda Decider/Decorator
AWS re:Invent 2016: Serverless Computing Patterns at Expedia (SVR306) )
Demo
Control System for Expedia’s AWS Infrastructure Governance
Control System for our AWS Infrastructure Governance AWS CloudTrail
Control System for our AWS Infrastructure Governance AWS CloudTrail S3 Bucket
Control System for our AWS Infrastructure Governance AWS CloudTrail S3 Bucket Collector Lambda
Control System for our AWS Infrastructure Governance AWS CloudTrail S3 Bucket Collector Lambda DynamoDB Events Table
Control System for our AWS Infrastructure Governance AWS CloudTrail S3 Bucket Collector Lambda DynamoDB Events Table CloudWatch Events - Schedule
Control System for our AWS Infrastructure Governance AWS CloudTrail S3 Bucket Collector Lambda DynamoDB Events Table CloudWatch Events - Schedule Dispatcher Lambda
Control System for our AWS Infrastructure Governance AWS CloudTrail S3 Bucket Collector Lambda DynamoDB Events Table CloudWatch Events - Schedule Dispatcher Lambda Amazon SNS
Control System for our AWS Infrastructure Governance AWS CloudTrail S3 Bucket Collector Lambda DynamoDB Events Table CloudWatch Events - Schedule Dispatcher Lambda Amazon SNS Validator Lambda
Control System for our AWS Infrastructure Governance AWS CloudTrail S3 Bucket Collector Lambda DynamoDB Events Table CloudWatch Events - Schedule Dispatcher Lambda Amazon SNS Validator Lambda Amazon SNS
Control System for our AWS Infrastructure Governance AWS CloudTrail S3 Bucket Collector Lambda DynamoDB Events Table CloudWatch Events - Schedule Dispatcher Lambda Amazon SNS Validator Lambda Destroyer Lambda Amazon SNS
Control System for our AWS Infrastructure Governance AWS CloudTrail S3 Bucket Collector Lambda DynamoDB Events Table CloudWatch Events - Schedule Dispatcher Lambda Amazon SNS Validator Lambda Destroyer Lambda Amazon SNS Amazon SNS
Control System for our AWS Infrastructure Governance AWS CloudTrail S3 Bucket Collector Lambda DynamoDB Events Table CloudWatch Events - Schedule Dispatcher Lambda Amazon SNS Validator Lambda Destroyer Lambda ACK Lambda Amazon SNS Amazon SNS
Control System for our AWS Infrastructure Governance AWS CloudTrail S3 Bucket Collector Lambda DynamoDB Events Table CloudWatch Events - Schedule Dispatcher Lambda Amazon SNS Validator Lambda Destroyer Lambda ACK Lambda DynamoDB Sim Table DynamoDB History Table Amazon SNS Amazon SNS
Demo
Test & Learn Dashboard Aggregations
Test & Learn Dashboard Aggregations Amazon DynamoDB Stream
Test & Learn Dashboard Aggregations Amazon DynamoDB Stream Amazon Lambda
Test & Learn Dashboard Aggregations Amazon DynamoDB Stream Amazon Lambda Amazon SQS
Test & Learn Dashboard Aggregations Amazon DynamoDB Stream Amazon Lambda Amazon SQS
Using Lambda to trigger Typeahead Memory Refresh
Typeahead (ESS – Expedia Suggest Service) • Typeahead is the first feature that users interact with while searching for what they are looking for, so providing best possible suggestions to them is very essential • ElasticCache Redis + IN-MEMORY data for cache misses • 25 GB Heap Size per EC2 Instance • Daily refresh of this data and it takes 15 mins to load the data
AWS re:Invent 2016: Serverless Computing Patterns at Expedia (SVR306) )
The Problem • Currently each instance that runs ESS in prod has to have enough memory to support this data refresh process while still serving requests • When the occupied heap reaches its maximum limit, the application: • often freezes resulting in not able to recommend right suggestions to the customers • pauses until garbage collection completes discarding the large, outdated BLOB from memory
Solution • Eliminate data refresh process via crontab • Launch N new instances by modifying the Auto Scaling group every day during refresh time • Modify Auto Scaling group to terminate N instances using OldestFirst EC2 instance termination policy when 2N instances are InService • Schedule AWS Lambda to own this responsibility everyday to scale up and down
Using Lambda to scale up the Auto Scaling group
Using Lambda to scale up the Auto Scaling group CloudWatch Events - Schedule
Using Lambda to scale up the Auto Scaling group CloudWatch Events - Schedule Amazon Lambda
Using Lambda to scale up the Auto Scaling group CloudWatch Events - Schedule Amazon Lambda Validate ASG for Scale UP
Using Lambda to scale up the Auto Scaling group CloudWatch Events - Schedule Amazon Lambda Validate ASG for Scale UP Update Reference Tag on ASG Abort & Exit Failure
Using Lambda to scale up the Auto Scaling group CloudWatch Events - Schedule Amazon Lambda Validate ASG for Scale UP Update Reference Tag on ASG Scale up ASG Abort & Exit Failure Failure
Using Lambda to scale up the Auto Scaling group CloudWatch Events - Schedule Amazon Lambda Validate ASG for Scale UP Update Reference Tag on ASG Scale up ASG Notify & Exit Abort & Exit Failure Failure Failure
Using Lambda to scale down the Auto Scaling group
Using Lambda to scale down the Auto Scaling group CloudWatch Events - Schedule
Using Lambda to scale down the Auto Scaling group CloudWatch Events - Schedule Amazon Lambda
Using Lambda to scale down the Auto Scaling group CloudWatch Events - Schedule Amazon Lambda Validate ASG for Scale DOWN
Using Lambda to scale down the Auto Scaling group CloudWatch Events - Schedule Amazon Lambda Validate ASG for Scale DOWN Scale DOWN ASG Abort & Exit Failure
Using Lambda to scale down the Auto Scaling group CloudWatch Events - Schedule Amazon Lambda Validate ASG for Scale DOWN Scale DOWN ASG Remove the tag from ASG Abort & Exit Failure Failure
Using Lambda to scale down the Auto Scaling group CloudWatch Events - Schedule Amazon Lambda Validate ASG for Scale DOWN Scale DOWN ASG Remove the tag from ASG Notify & Exit Abort & Exit Failure Failure Failure
Traveler Profile Generation Using Homegrown Lambda
AWS re:Invent 2016: Serverless Computing Patterns at Expedia (SVR306) )
Benefits of Serverless Computing • Cost • Operations • Scale • Opportunity • Time to market
What’s next? • Improve API Gateway integration in Lambda creation process • Add support for logging to different sources other than CloudWatch • Process hotel images using Lambda
Thank you! Kuldeep Chowhan @this_is_kuldeep
Remember to complete your evaluations!

More Related Content

AWS re:Invent 2016: Serverless Computing Patterns at Expedia (SVR306) )

  • 1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Kuldeep Chowhan 12/01/2016 SVR306 Serverless Computing Patterns @ Expedia, Inc.
  • 2. Kuldeep Chowhan Principal Engineer @ Expedia, Inc. @this_is_kuldeep
  • 3. Expedia, Inc., at a glance Notes: Expedia, Inc. results shown for 2015. 1Gross bookings is an operating and statistical metric that captures the total dollar value, generally inclusive of taxes and fees, of travel services booked by our customers; gross bookings excludes HomeAway. 2Monthly visits based on data for Brand Expedia (including Travelocity and Wotif), Hotels.com, Orbitz Worldwide, HomeAway, trivago and Hotwire combined during 2015. Sources: Overall travel industry growth rate based on Phocuswright data for 2015 Y/Y. Largest travel company $60B gross bookings1 $6.6B revenue growing >2X faster vs. industry 18K+ employees globally 450M+ monthly site visits2 75+ countries served 1.5M+ lodging options for travelers
  • 4. What to Expect from the Session • AWS Lambda Usage at Expedia • Serverless Apps Creation at Expedia • Different Serverless App architectures • Demo
  • 5. AWS Usage @ Expedia
  • 7. What is AWS Lambda? • Run code for virtually any type of application • No Servers to Manage • Zero administration • Continuous Scaling • Sub second Metering
  • 8. How does AWS Lambda works?
  • 9. Primer – AWS Lambda Creation
  • 13. Demo
  • 16. CI/CD System Platform utilizing Serverless Computing GitHub
  • 17. CI/CD System Platform utilizing Serverless Computing GitHub Webhooks (Push Events)
  • 18. CI/CD System Platform utilizing Serverless Computing GitHub Webhooks (Push Events) Amazon API Gateway
  • 19. CI/CD System Platform utilizing Serverless Computing GitHub Webhooks (Push Events) Amazon API Gateway Lambda Decider/Decorator
  • 20. CI/CD System Platform utilizing Serverless Computing GitHub Webhooks (Push Events) Amazon API Gateway Amazon SNS Lambda Decider/Decorator
  • 21. CI/CD System Platform utilizing Serverless Computing GitHub Webhooks (Push Events) Amazon API Gateway Amazon SNS Lambda Deployment Config Changes Lambda Decider/Decorator
  • 22. CI/CD System Platform utilizing Serverless Computing GitHub Webhooks (Push Events) Amazon API Gateway Amazon SNS Lambda Deployment Config Changes Amazon Elasticsearch Service Lambda Decider/Decorator
  • 23. CI/CD System Platform utilizing Serverless Computing GitHub Webhooks (Push Events) Amazon API Gateway Amazon SNS Lambda Deployment Config Changes Lambda Developers pushing to master directly Amazon Elasticsearch Service Lambda Decider/Decorator
  • 24. CI/CD System Platform utilizing Serverless Computing GitHub Webhooks (Push Events) Amazon API Gateway Amazon SNS Lambda Deployment Config Changes Lambda Developers pushing to master directly Amazon Elasticsearch Service Amazon SES Lambda Decider/Decorator
  • 25. CI/CD System Platform utilizing Serverless Computing GitHub Webhooks (Push Events) Amazon API Gateway Amazon SNS Lambda Deployment Config Changes Lambda App Details Changes Lambda Developers pushing to master directly Amazon Elasticsearch Service Amazon SES Lambda Decider/Decorator
  • 26. CI/CD System Platform utilizing Serverless Computing GitHub Webhooks (Push Events) Amazon API Gateway Amazon SNS Lambda Deployment Config Changes Lambda App Details Changes Lambda Developers pushing to master directly Amazon DynamoDB Amazon Elasticsearch Service Amazon SES Lambda Decider/Decorator
  • 28. Demo
  • 29. Control System for Expedia’s AWS Infrastructure Governance
  • 30. Control System for our AWS Infrastructure Governance AWS CloudTrail
  • 31. Control System for our AWS Infrastructure Governance AWS CloudTrail S3 Bucket
  • 32. Control System for our AWS Infrastructure Governance AWS CloudTrail S3 Bucket Collector Lambda
  • 33. Control System for our AWS Infrastructure Governance AWS CloudTrail S3 Bucket Collector Lambda DynamoDB Events Table
  • 34. Control System for our AWS Infrastructure Governance AWS CloudTrail S3 Bucket Collector Lambda DynamoDB Events Table CloudWatch Events - Schedule
  • 35. Control System for our AWS Infrastructure Governance AWS CloudTrail S3 Bucket Collector Lambda DynamoDB Events Table CloudWatch Events - Schedule Dispatcher Lambda
  • 36. Control System for our AWS Infrastructure Governance AWS CloudTrail S3 Bucket Collector Lambda DynamoDB Events Table CloudWatch Events - Schedule Dispatcher Lambda Amazon SNS
  • 37. Control System for our AWS Infrastructure Governance AWS CloudTrail S3 Bucket Collector Lambda DynamoDB Events Table CloudWatch Events - Schedule Dispatcher Lambda Amazon SNS Validator Lambda
  • 38. Control System for our AWS Infrastructure Governance AWS CloudTrail S3 Bucket Collector Lambda DynamoDB Events Table CloudWatch Events - Schedule Dispatcher Lambda Amazon SNS Validator Lambda Amazon SNS
  • 39. Control System for our AWS Infrastructure Governance AWS CloudTrail S3 Bucket Collector Lambda DynamoDB Events Table CloudWatch Events - Schedule Dispatcher Lambda Amazon SNS Validator Lambda Destroyer Lambda Amazon SNS
  • 40. Control System for our AWS Infrastructure Governance AWS CloudTrail S3 Bucket Collector Lambda DynamoDB Events Table CloudWatch Events - Schedule Dispatcher Lambda Amazon SNS Validator Lambda Destroyer Lambda Amazon SNS Amazon SNS
  • 41. Control System for our AWS Infrastructure Governance AWS CloudTrail S3 Bucket Collector Lambda DynamoDB Events Table CloudWatch Events - Schedule Dispatcher Lambda Amazon SNS Validator Lambda Destroyer Lambda ACK Lambda Amazon SNS Amazon SNS
  • 42. Control System for our AWS Infrastructure Governance AWS CloudTrail S3 Bucket Collector Lambda DynamoDB Events Table CloudWatch Events - Schedule Dispatcher Lambda Amazon SNS Validator Lambda Destroyer Lambda ACK Lambda DynamoDB Sim Table DynamoDB History Table Amazon SNS Amazon SNS
  • 43. Demo
  • 44. Test & Learn Dashboard Aggregations
  • 45. Test & Learn Dashboard Aggregations Amazon DynamoDB Stream
  • 46. Test & Learn Dashboard Aggregations Amazon DynamoDB Stream Amazon Lambda
  • 47. Test & Learn Dashboard Aggregations Amazon DynamoDB Stream Amazon Lambda Amazon SQS
  • 48. Test & Learn Dashboard Aggregations Amazon DynamoDB Stream Amazon Lambda Amazon SQS
  • 49. Using Lambda to trigger Typeahead Memory Refresh
  • 50. Typeahead (ESS – Expedia Suggest Service) • Typeahead is the first feature that users interact with while searching for what they are looking for, so providing best possible suggestions to them is very essential • ElasticCache Redis + IN-MEMORY data for cache misses • 25 GB Heap Size per EC2 Instance • Daily refresh of this data and it takes 15 mins to load the data
  • 52. The Problem • Currently each instance that runs ESS in prod has to have enough memory to support this data refresh process while still serving requests • When the occupied heap reaches its maximum limit, the application: • often freezes resulting in not able to recommend right suggestions to the customers • pauses until garbage collection completes discarding the large, outdated BLOB from memory
  • 53. Solution • Eliminate data refresh process via crontab • Launch N new instances by modifying the Auto Scaling group every day during refresh time • Modify Auto Scaling group to terminate N instances using OldestFirst EC2 instance termination policy when 2N instances are InService • Schedule AWS Lambda to own this responsibility everyday to scale up and down
  • 54. Using Lambda to scale up the Auto Scaling group
  • 55. Using Lambda to scale up the Auto Scaling group CloudWatch Events - Schedule
  • 56. Using Lambda to scale up the Auto Scaling group CloudWatch Events - Schedule Amazon Lambda
  • 57. Using Lambda to scale up the Auto Scaling group CloudWatch Events - Schedule Amazon Lambda Validate ASG for Scale UP
  • 58. Using Lambda to scale up the Auto Scaling group CloudWatch Events - Schedule Amazon Lambda Validate ASG for Scale UP Update Reference Tag on ASG Abort & Exit Failure
  • 59. Using Lambda to scale up the Auto Scaling group CloudWatch Events - Schedule Amazon Lambda Validate ASG for Scale UP Update Reference Tag on ASG Scale up ASG Abort & Exit Failure Failure
  • 60. Using Lambda to scale up the Auto Scaling group CloudWatch Events - Schedule Amazon Lambda Validate ASG for Scale UP Update Reference Tag on ASG Scale up ASG Notify & Exit Abort & Exit Failure Failure Failure
  • 61. Using Lambda to scale down the Auto Scaling group
  • 62. Using Lambda to scale down the Auto Scaling group CloudWatch Events - Schedule
  • 63. Using Lambda to scale down the Auto Scaling group CloudWatch Events - Schedule Amazon Lambda
  • 64. Using Lambda to scale down the Auto Scaling group CloudWatch Events - Schedule Amazon Lambda Validate ASG for Scale DOWN
  • 65. Using Lambda to scale down the Auto Scaling group CloudWatch Events - Schedule Amazon Lambda Validate ASG for Scale DOWN Scale DOWN ASG Abort & Exit Failure
  • 66. Using Lambda to scale down the Auto Scaling group CloudWatch Events - Schedule Amazon Lambda Validate ASG for Scale DOWN Scale DOWN ASG Remove the tag from ASG Abort & Exit Failure Failure
  • 67. Using Lambda to scale down the Auto Scaling group CloudWatch Events - Schedule Amazon Lambda Validate ASG for Scale DOWN Scale DOWN ASG Remove the tag from ASG Notify & Exit Abort & Exit Failure Failure Failure
  • 70. Benefits of Serverless Computing • Cost • Operations • Scale • Opportunity • Time to market
  • 71. What’s next? • Improve API Gateway integration in Lambda creation process • Add support for logging to different sources other than CloudWatch • Process hotel images using Lambda