SlideShare a Scribd company logo
© 2010 TubeMogul Incorporated.  All Rights Reserved. TubeMogul and AWS Cloudy Scaling
What's a TubeMogul? © 2010 TubeMogul Incorporated.  All Rights Reserved. OneLoad - The nexus of online video Hyper-syndication Upload video to 25 video sharing sites in one shot In-player video analytics Provides in-depth reporting back to site owner and content creators
Challenges - Why AWS? Challenges AWS Solution Small Engineering / Ops teams No hardware to maintain Limited budget Linear scale cost Requirements constantly in flux Easy to change configurations System size unknown at outset Godsend here 24/7 SLAs Uptime perception was a concern (notorious S3 outages, etc.) ->  Can you do better???
Evaluation Process Already successful with our syndication platform Disk I/O Bonnie i/o tests MADM and LVM striping EBS first-use penalty! Network throughput Long-running tests to ensure consistent performance Elastic Load-Balancer Had packet drops with traffic bursts Cost TCO analysis is complex – weight lower CapEx against additional OpEx and opportunity cost http://aws.amazon.com/economics AWS has regular cost reductions
InPlay Architecture August 24, 2009 Metacafe Dailymotion Brightcove App-01 App-02 App-nn LB-01 LB-02 Had-01 Had-02 Had-nn MySQL-01 MySQL-02 MySQL-nn Load Balancers App Servers Hadoop Cluster MySQL Cluster REST API EC2 WWW Video Players RAID0 striped EBS HAProxy Elastic IPs Needed web scale-out and resiliency!!
Lessons Learned LDAP and DNS needed for large systems Multi-EBS nodes require serious management SSH auth only!  Make sure this is bullet-proof!! May have dependencies on DNS, LDAP, etc. Ops guys need to be coders Need deployment scripts on top of AMIs EC2 instances can and do fail Make sure your architect accordingly!  Make backups! Use elastic IPs for well-known addresses, even on private interfaces Consider monitoring requirements – more is better!
Lessons Learned ELB didn't work well for our application at the time Haven't re-evaluated this AWS does not provide free support Posting instance IDs on public forums! Network and CPU performance has been stable and consistent.  :) Overall as a startup, it's an amazing thing...
Thanks! Adam Rose CTO, TubeMogul [email_address]

More Related Content

AWS Customer Presentation - How TubeMogul uses AWS

  • 1. © 2010 TubeMogul Incorporated. All Rights Reserved. TubeMogul and AWS Cloudy Scaling
  • 2. What's a TubeMogul? © 2010 TubeMogul Incorporated. All Rights Reserved. OneLoad - The nexus of online video Hyper-syndication Upload video to 25 video sharing sites in one shot In-player video analytics Provides in-depth reporting back to site owner and content creators
  • 3. Challenges - Why AWS? Challenges AWS Solution Small Engineering / Ops teams No hardware to maintain Limited budget Linear scale cost Requirements constantly in flux Easy to change configurations System size unknown at outset Godsend here 24/7 SLAs Uptime perception was a concern (notorious S3 outages, etc.) -> Can you do better???
  • 4. Evaluation Process Already successful with our syndication platform Disk I/O Bonnie i/o tests MADM and LVM striping EBS first-use penalty! Network throughput Long-running tests to ensure consistent performance Elastic Load-Balancer Had packet drops with traffic bursts Cost TCO analysis is complex – weight lower CapEx against additional OpEx and opportunity cost http://aws.amazon.com/economics AWS has regular cost reductions
  • 5. InPlay Architecture August 24, 2009 Metacafe Dailymotion Brightcove App-01 App-02 App-nn LB-01 LB-02 Had-01 Had-02 Had-nn MySQL-01 MySQL-02 MySQL-nn Load Balancers App Servers Hadoop Cluster MySQL Cluster REST API EC2 WWW Video Players RAID0 striped EBS HAProxy Elastic IPs Needed web scale-out and resiliency!!
  • 6. Lessons Learned LDAP and DNS needed for large systems Multi-EBS nodes require serious management SSH auth only! Make sure this is bullet-proof!! May have dependencies on DNS, LDAP, etc. Ops guys need to be coders Need deployment scripts on top of AMIs EC2 instances can and do fail Make sure your architect accordingly! Make backups! Use elastic IPs for well-known addresses, even on private interfaces Consider monitoring requirements – more is better!
  • 7. Lessons Learned ELB didn't work well for our application at the time Haven't re-evaluated this AWS does not provide free support Posting instance IDs on public forums! Network and CPU performance has been stable and consistent. :) Overall as a startup, it's an amazing thing...
  • 8. Thanks! Adam Rose CTO, TubeMogul [email_address]