SlideShare a Scribd company logo

AWS Lambda at JUST EAT

•
•
A
Andrew Brown

- Just Eat is a leading digital marketplace for takeaway food delivery founded in 2001 operating in 13 markets globally. It has processed up to 2,500 orders per minute at peak times. - Just Eat migrated to AWS 5 years ago and runs hundreds of EC2 instances at peak dinner times using scheduled scaling, CloudFormation, and other AWS services. - AWS Lambda was introduced in 2014 and Just Eat started using it for micro tasks like resetting delivery times, publishing SNS messages, and provisioning instance access to reduce infrastructure costs and management compared to running EC2 fleets.

Related slideshows

Packer + Ansible을 이용한 AMI 생성 및 AutoScaling Group 이미지 교체 이야기
Packer + Ansible을 이용한 AMI 생성 및 AutoScaling Group 이미지 교체 이야기Packer + Ansible을 이용한 AMI 생성 및 AutoScaling Group 이미지 교체 이야기
Packer + Ansible을 이용한 AMI 생성 및 AutoScaling Group 이미지 교체 이야기
 
Amazon EC2 Container Service: Manage Docker-Enabled Apps in EC2
Amazon EC2 Container Service: Manage Docker-Enabled Apps in EC2Amazon EC2 Container Service: Manage Docker-Enabled Apps in EC2
Amazon EC2 Container Service: Manage Docker-Enabled Apps in EC2
 
Data backup and Disaster recovery with AWS
Data backup and Disaster recovery with AWSData backup and Disaster recovery with AWS
Data backup and Disaster recovery with AWS
 
1 of 28
Download to read offline
AWS Lambda at JUST EAT ANDREW BROWN - Senior Platform Engineer
Overview Just Eat Early Adoption Use Cases Serverless
• The world’s leading digital marketplace for takeaway food delivery • Founded in 2001 • Operate in 13 markets around the globe JUST EAT • Platform peak has processed 2,500 orders per minute • Thats a lot of Food!
TECHNOLOGY • Run hundreds of EC2 Instances at peak / dinner time • Scheduled Scaling of Resources - EC2, Dynamo Capacity • Heavily utilise CloudFormation • Just Eat Migrated to AWS 5 years ago - All In!
AWS - Setup • Multiple AWS Accounts • QA Daily Launch and Teardown • On average an instance less than 3 days old • • Approx 150 “features” - Each Feature has a Stack
What’s good about our Setup? Developers can concentrate on writing code • Packer, CodeDeploy, Consul, Cloudformation • Developers create package and deploy! • PaaS, IaaS
What’s good about our Setup? Consistency • Many accounts, environments, countries, features • All get the same AWS resources - e.g. Security Groups • Templated Cloudformations
AWS - Lambda • Re:Invent 2014 announcement • AWS Lambda runs your code in response to events • • “we could use that for x, y, and z!”
So Why Use Lambda? • Lightweight • Cheaper • Less to manage • Quicker to Deploy
But Why Should JE Use Lambda? Let’s review One Feature • CloudFormation • EC2 Instances x 3 - Multi AZ • EBS Volumes • ELB Endpoint and Health Check • IAM Roles • DNS • Security Groups, Subnets, VPC • ASG settings, LaunchConfiguration ~ 1600 lines of JSON $0.246 x 3 x 24 x 365 (m4.large) $0.10 x 30 x 3 x 12 (gp2) $0.028 x 24 x 365 (classic) ~= $7k
So Why Use Lambda? What does this Feature do? • Resets Delivery Time every morning • Publishes a SNS message about an order • Provision access for an instance • Tidy up resources
Early Lambdas ● New Technology! ● Multiple languages and frameworks ● Seen as additional parts of a feature ● Using the Console….
Early Lambdas Limitations / Problems ● Consistency (that’s Just Eat not AWS!) ● No VPC Support (Added Feb 2016) ● No Cron in Cloudformation ● Cloudformation Network Still Attached
Use Cases - PaaSBot • CloudFormation Stack per Environment • Windows EC2 Instance • Ran a bunch of Scheduled Tasks
Use Cases - PaaSBot Clean up EBS Volumes • Cron build in Teamcity - one per env • Scheduled Windows Task (one per account, assumed roles) • Lambda? Report Metrics • T2 Credit Balances • AWS Limits • Lambda?
Use Cases - PaaSBot Security Audit • Check Security Group rules • Check User access
Use Cases - PaaSLambda Lambdas • No Windows Instance(s) • Micro-Scheduled Tasks • Easier to update • Better visibility
Use Cases - Instance Access • Recently started using HashiCorp Consul and Vault • Concerns : “What can my instance access?” • Consul ACLS and Vault Tokens - limit what key/values you can access • ACLs Initially provisioned via Deployment Tool
Use Cases - Instance Launch • Lambdas Created with Consul and Vault Cloudformations - nodejs • ASG Event -> CloudWatch Events -> Trigger Lambda • Provisions ACL and AppId login • Removes Access when Instance terminated
Use Cases - Instance Launch Launch Stack i-12345 i-12345 i-12345 ASG CloudWatch Event { “acl” : a1a1a1-b2b2b2 } { “token” : secrettoken }
Use Cases - Instance Terminate Delete Stack / Scale Down i-12345 i-12345 i-12345 ASG CloudWatch Event
Different Approach • Starting to create Lambda Features • Needed a consistent approach • Recently Implemented Serverless
Serverless • Still in “Beta”
Serverless Good Points • Brings Back Consistency • Ease to deploy • Defined Dependencies • Being Quickly Adopted • Track signal strength • Calculate cost • Check capacity • Start Ad-hoc backup / Prune Log files
Serverless Not so Good Points • Currently each Feature includes their AWS resources - previously templated • Code repeated • Monitoring & Logging Immature
Lessons Learnt & Future Plans • Difficult to integrate into our Platform - Versatility • Still a few Bugs - • Serverless - Based on Initial Uptake • AWS Lambda & API Gateway • Will it replace our APIs? - Maybe
Any questions?
Thank you

More Related Content

AWS Lambda at JUST EAT

  • 1. AWS Lambda at JUST EAT ANDREW BROWN - Senior Platform Engineer
  • 3. • The world’s leading digital marketplace for takeaway food delivery • Founded in 2001 • Operate in 13 markets around the globe JUST EAT • Platform peak has processed 2,500 orders per minute • Thats a lot of Food!
  • 4. TECHNOLOGY • Run hundreds of EC2 Instances at peak / dinner time • Scheduled Scaling of Resources - EC2, Dynamo Capacity • Heavily utilise CloudFormation • Just Eat Migrated to AWS 5 years ago - All In!
  • 5. AWS - Setup • Multiple AWS Accounts • QA Daily Launch and Teardown • On average an instance less than 3 days old • • Approx 150 “features” - Each Feature has a Stack
  • 6. What’s good about our Setup? Developers can concentrate on writing code • Packer, CodeDeploy, Consul, Cloudformation • Developers create package and deploy! • PaaS, IaaS
  • 7. What’s good about our Setup? Consistency • Many accounts, environments, countries, features • All get the same AWS resources - e.g. Security Groups • Templated Cloudformations
  • 8. AWS - Lambda • Re:Invent 2014 announcement • AWS Lambda runs your code in response to events • • “we could use that for x, y, and z!”
  • 9. So Why Use Lambda? • Lightweight • Cheaper • Less to manage • Quicker to Deploy
  • 10. But Why Should JE Use Lambda? Let’s review One Feature • CloudFormation • EC2 Instances x 3 - Multi AZ • EBS Volumes • ELB Endpoint and Health Check • IAM Roles • DNS • Security Groups, Subnets, VPC • ASG settings, LaunchConfiguration ~ 1600 lines of JSON $0.246 x 3 x 24 x 365 (m4.large) $0.10 x 30 x 3 x 12 (gp2) $0.028 x 24 x 365 (classic) ~= $7k
  • 11. So Why Use Lambda? What does this Feature do? • Resets Delivery Time every morning • Publishes a SNS message about an order • Provision access for an instance • Tidy up resources
  • 12. Early Lambdas ● New Technology! ● Multiple languages and frameworks ● Seen as additional parts of a feature ● Using the Console….
  • 13. Early Lambdas Limitations / Problems ● Consistency (that’s Just Eat not AWS!) ● No VPC Support (Added Feb 2016) ● No Cron in Cloudformation ● Cloudformation Network Still Attached
  • 14. Use Cases - PaaSBot • CloudFormation Stack per Environment • Windows EC2 Instance • Ran a bunch of Scheduled Tasks
  • 15. Use Cases - PaaSBot Clean up EBS Volumes • Cron build in Teamcity - one per env • Scheduled Windows Task (one per account, assumed roles) • Lambda? Report Metrics • T2 Credit Balances • AWS Limits • Lambda?
  • 16. Use Cases - PaaSBot Security Audit • Check Security Group rules • Check User access
  • 17. Use Cases - PaaSLambda Lambdas • No Windows Instance(s) • Micro-Scheduled Tasks • Easier to update • Better visibility
  • 18. Use Cases - Instance Access • Recently started using HashiCorp Consul and Vault • Concerns : “What can my instance access?” • Consul ACLS and Vault Tokens - limit what key/values you can access • ACLs Initially provisioned via Deployment Tool
  • 19. Use Cases - Instance Launch • Lambdas Created with Consul and Vault Cloudformations - nodejs • ASG Event -> CloudWatch Events -> Trigger Lambda • Provisions ACL and AppId login • Removes Access when Instance terminated
  • 20. Use Cases - Instance Launch Launch Stack i-12345 i-12345 i-12345 ASG CloudWatch Event { “acl” : a1a1a1-b2b2b2 } { “token” : secrettoken }
  • 21. Use Cases - Instance Terminate Delete Stack / Scale Down i-12345 i-12345 i-12345 ASG CloudWatch Event
  • 22. Different Approach • Starting to create Lambda Features • Needed a consistent approach • Recently Implemented Serverless
  • 24. Serverless Good Points • Brings Back Consistency • Ease to deploy • Defined Dependencies • Being Quickly Adopted • Track signal strength • Calculate cost • Check capacity • Start Ad-hoc backup / Prune Log files
  • 25. Serverless Not so Good Points • Currently each Feature includes their AWS resources - previously templated • Code repeated • Monitoring & Logging Immature
  • 26. Lessons Learnt & Future Plans • Difficult to integrate into our Platform - Versatility • Still a few Bugs - • Serverless - Based on Initial Uptake • AWS Lambda & API Gateway • Will it replace our APIs? - Maybe