SlideShare a Scribd company logo

How to Create a Scalable System using Force.com, Heroku and Iron.io

•
•
Travis Reeder
Travis Reeder

This presentation shows how and why to use queues and workers to decouple and make your application more scalable.

1 of 24
Create a Massively Scalable System with Force.com and Heroku Travis Reeder, Iron.io, CTO and Co-founder
Safe harbor Safe harbor statement under the Private Securities Litigation Reform Act of 1995: This presentation may contain forward-looking statements that involve risks, uncertainties, and assumptions. If any such uncertainties materialize or if any of the assumptions proves incorrect, the results of salesforce.com, inc. could differ materially from the results expressed or implied by the forward-looking statements we make. All statements other than statements of historical fact could be deemed forward-looking, including any projections of product or service availability, subscriber growth, earnings, revenues, or other financial items and any statements regarding strategies or plans of management for future operations, statements of belief, any statements concerning new, planned, or upgraded services or technology developments and customer contracts or use of our services. The risks and uncertainties referred to above include – but are not limited to – risks associated with developing and delivering new functionality for our service, new products and services, our new business model, our past operating losses, possible fluctuations in our operating results and rate of growth, interruptions or delays in our Web hosting, breach of our security measures, the outcome of intellectual property and other litigation, risks associated with possible mergers and acquisitions, the immature market in which we operate, our relatively limited operating history, our ability to expand, retain, and motivate our employees and manage our growth, new releases of our service and successful customer deployment, our limited history reselling non-salesforce.com products, and utilization and selling to larger enterprise customers. Further information on potential factors that could affect the financial results of salesforce.com, inc. is included in our annual report on Form 10-Q for the most recent fiscal quarter ended July 31, 2012. This documents and others containing important disclosures are available on the SEC Filings section of the Investor Information section of our Web site. Any unreleased services or features referenced in this or other presentations, press releases or public statements are not currently available and may not be delivered on time or at all. Customers who purchase our services should make the purchase decisions based upon features that are currently available. Salesforce.com, inc. assumes no obligation and does not intend to update these forward- looking statements.
I am Travis Reeder CTO and Co-Founder of Iron.io Iron.io provides massively scalable and elastic cloud infrastructure services: IronWorker, IronMQ and IronCache. Building things to scale is our business.
Scalability Scalability is the ability of a system, network, or process, to handle a growing amount of work in a capable manner or its ability to be enlarged to accommodate that growth. Source: Wikipedia
Systems integration Systems integration is the process of linking together different computing systems and software applications physically or functionally, to act as a coordinated whole. Source: Wikipedia
The example application Lead gen form. User fills out form, stores the info, posts to Salesforce as a lead and sends user an email. Easy right?
Easy? Not at scale  How do you keep your application responding fast at high load?  How do you deal with spikes? • Daily deal Site at 8am • You get TechCrunched  How do you deal with Rate Limits? • Rate limits at Salesforce – 5000 requests per day (208 per hour) – 5 concurrent requests
There is a simple solution Queues  Message Queues  Task Queues
Queues An amazing developer tool, great for:  Spikability  Keeping database load consistent  Decoupling parts of your application  Improving application performance • If it doesn’t need to be done while user is waiting, then don’t do it, put it on a queue  Integrations / communication between systems
Spikability If you do all your work in your application, you need servers... lots of servers.
Spikability without queues Need enough servers to handle the maximum traffic you might see.
Spikability with queues Queues soak up spikes with no performance or data loss.
Keeping database load consistent You decide how fast you want to process, not your traffic.
Decoupling / integrations Using queues to communicate between parts of your system is a good idea. Splitting your application into smaller parts can also be a good idea.  Loosely coupled  If one part fails, the others don’t  You can upgrade different parts separately  Maintain and find issues much easier  Oftentimes you have no control over it if it’s a third party
Architecture overview Heroku application Salesforce Boomi Email
The Heroku app Simple form. The only thing the controller does is queue up a worker task.
IronWorker task queue Create a worker and upload it once: ironworker.tasks.create(“lead_worker", config.merge(name: params[:name], email: params[:email], company: params[:company)) iron_worker upload lead_worker Then queue up tasks for it:
IronMQ message queue Very simple to use. One side pushes messages onto a queue (in our example, this is our lead gen app): The other side pulls them off (Boomi in this case): A message can be any arbitrary data that both sides understand. ironmq.queue(“lead”).post(msg.to_json) ironmq.queue(“lead”).get()
Boomi Boomi enables drag and drop integrations between SaaS and On- Premise applications via its out of the box connectors.
Scaling • On the front end, we’ve offloaded most of the work and there is no database to worry about, but if you need to scale it, Heroku makes it easy: • IronWorker and IronMQ are fully elastic so no scaling required. • Boomi and Salesforce, the main things to consider are rate limits and other limits. heroku ps:scale web=20
How to Create a Scalable System using Force.com, Heroku and Iron.io
Let’s Try It! Grab your phone and go to: http://ironforce.herokuapp.com This demo is running on a single Heroku Dyno and it's ready to handle this room and then some. I promise I won’t spam you. But feel free to reply and contact me.
Travis Reeder CTO and co-founder Code can be found at: https://github.com/treeder/ironforce
How to Create a Scalable System using Force.com, Heroku and Iron.io

More Related Content

How to Create a Scalable System using Force.com, Heroku and Iron.io

  • 1. Create a Massively Scalable System with Force.com and Heroku Travis Reeder, Iron.io, CTO and Co-founder
  • 2. Safe harbor Safe harbor statement under the Private Securities Litigation Reform Act of 1995: This presentation may contain forward-looking statements that involve risks, uncertainties, and assumptions. If any such uncertainties materialize or if any of the assumptions proves incorrect, the results of salesforce.com, inc. could differ materially from the results expressed or implied by the forward-looking statements we make. All statements other than statements of historical fact could be deemed forward-looking, including any projections of product or service availability, subscriber growth, earnings, revenues, or other financial items and any statements regarding strategies or plans of management for future operations, statements of belief, any statements concerning new, planned, or upgraded services or technology developments and customer contracts or use of our services. The risks and uncertainties referred to above include – but are not limited to – risks associated with developing and delivering new functionality for our service, new products and services, our new business model, our past operating losses, possible fluctuations in our operating results and rate of growth, interruptions or delays in our Web hosting, breach of our security measures, the outcome of intellectual property and other litigation, risks associated with possible mergers and acquisitions, the immature market in which we operate, our relatively limited operating history, our ability to expand, retain, and motivate our employees and manage our growth, new releases of our service and successful customer deployment, our limited history reselling non-salesforce.com products, and utilization and selling to larger enterprise customers. Further information on potential factors that could affect the financial results of salesforce.com, inc. is included in our annual report on Form 10-Q for the most recent fiscal quarter ended July 31, 2012. This documents and others containing important disclosures are available on the SEC Filings section of the Investor Information section of our Web site. Any unreleased services or features referenced in this or other presentations, press releases or public statements are not currently available and may not be delivered on time or at all. Customers who purchase our services should make the purchase decisions based upon features that are currently available. Salesforce.com, inc. assumes no obligation and does not intend to update these forward- looking statements.
  • 3. I am Travis Reeder CTO and Co-Founder of Iron.io Iron.io provides massively scalable and elastic cloud infrastructure services: IronWorker, IronMQ and IronCache. Building things to scale is our business.
  • 4. Scalability Scalability is the ability of a system, network, or process, to handle a growing amount of work in a capable manner or its ability to be enlarged to accommodate that growth. Source: Wikipedia
  • 5. Systems integration Systems integration is the process of linking together different computing systems and software applications physically or functionally, to act as a coordinated whole. Source: Wikipedia
  • 6. The example application Lead gen form. User fills out form, stores the info, posts to Salesforce as a lead and sends user an email. Easy right?
  • 7. Easy? Not at scale  How do you keep your application responding fast at high load?  How do you deal with spikes? • Daily deal Site at 8am • You get TechCrunched  How do you deal with Rate Limits? • Rate limits at Salesforce – 5000 requests per day (208 per hour) – 5 concurrent requests
  • 8. There is a simple solution Queues  Message Queues  Task Queues
  • 9. Queues An amazing developer tool, great for:  Spikability  Keeping database load consistent  Decoupling parts of your application  Improving application performance • If it doesn’t need to be done while user is waiting, then don’t do it, put it on a queue  Integrations / communication between systems
  • 10. Spikability If you do all your work in your application, you need servers... lots of servers.
  • 11. Spikability without queues Need enough servers to handle the maximum traffic you might see.
  • 12. Spikability with queues Queues soak up spikes with no performance or data loss.
  • 13. Keeping database load consistent You decide how fast you want to process, not your traffic.
  • 14. Decoupling / integrations Using queues to communicate between parts of your system is a good idea. Splitting your application into smaller parts can also be a good idea.  Loosely coupled  If one part fails, the others don’t  You can upgrade different parts separately  Maintain and find issues much easier  Oftentimes you have no control over it if it’s a third party
  • 16. The Heroku app Simple form. The only thing the controller does is queue up a worker task.
  • 17. IronWorker task queue Create a worker and upload it once: ironworker.tasks.create(“lead_worker", config.merge(name: params[:name], email: params[:email], company: params[:company)) iron_worker upload lead_worker Then queue up tasks for it:
  • 18. IronMQ message queue Very simple to use. One side pushes messages onto a queue (in our example, this is our lead gen app): The other side pulls them off (Boomi in this case): A message can be any arbitrary data that both sides understand. ironmq.queue(“lead”).post(msg.to_json) ironmq.queue(“lead”).get()
  • 19. Boomi Boomi enables drag and drop integrations between SaaS and On- Premise applications via its out of the box connectors.
  • 20. Scaling • On the front end, we’ve offloaded most of the work and there is no database to worry about, but if you need to scale it, Heroku makes it easy: • IronWorker and IronMQ are fully elastic so no scaling required. • Boomi and Salesforce, the main things to consider are rate limits and other limits. heroku ps:scale web=20
  • 22. Let’s Try It! Grab your phone and go to: http://ironforce.herokuapp.com This demo is running on a single Heroku Dyno and it's ready to handle this room and then some. I promise I won’t spam you. But feel free to reply and contact me.
  • 23. Travis Reeder CTO and co-founder Code can be found at: https://github.com/treeder/ironforce

Editor's Notes

  1. In other words, a scalable application is built such that it can grow grow as needed.
  2. We’ll be covering both of these topics. Building a scalable app and integrating the application with Salesforce (and email?)
  3. Daily deal site like one kings lane get vast majority oftraffic at 8am to noon.Most api’s have rate limits. If our lead gen app got 100,000 people one day, the system breaks.
  4. There is a simple solution that can help with all of these problems. CLICKCheck out this Apple fan, he’s pumped, he got his iphone. Sure he had to wait a bit in the lineup around the block, but he got what he wanted without getting trampled.
  5. Let’s dig a bit deeper into the first two.
  6. This is a scene from the Matrix where they are standing in a huge white data center, they were about to launch their application, and Neo says “I need servers, lots of servers.” I think that’s how it happened anyways. Let me show you why.
  7. The red part is the servers you’ll need to handle the spikes. As you can see, a lot of wasted capacity to handle a single spike.
  8. Traffic spikes don’t matter. Queue soaks it up (orange) and with a bit of extra time, all the work is processed and we’re all good.
  9. Maintain and find issues: code base is much smaller and isolated.Eg: your front end is taking orders, but your backend merchant system is down, it doesn’t matter.
  10. Show lead_app.rb
  11. Show workers/lead_worker.worker and lead_worker.rb. Can run it once or a million times with same worker. Show that it’s uploaded in hud. https://hud.iron.io/tq/projects/4f10ae4fb21c531b30001448/jobs
  12. Show main.rb
  13. If you need to scale after all this…
  14. And did you notice? NO SERVERS!