Windy cityrails performance_tuning

Rails Performance John McCaffrey Railsperformance.blogspot.com

John McCaffrey: Its all about me Doing Rails since 2007 3rd time presenting at WindyCityRails 2008: Advanced Firebug and JS unit testing 2009: PDF Generation in Rails 2010: Rails Performance Tuning Addicted to Performance Tuning railsperformance.blogspot.com Feedback: spkr8.com/t/4413 Or bit.ly/PerfR8 @J_McCaffrey [email_address] (the slides will be available, with tons of extra references)

What are we gonna cover? Performance touches a lot of layers We have a very diverse audience

Measuring Page Load Fixing page load problems Analyzing Server Performance Database issues Ruby issues Q&A Starting off on the right foot

It all started... def organization_branches get_organizations .flatten.uniq.select{|org| org.children.size > 0} end def organization_leaves get_organizations .flatten.uniq.select{|org| org.children.size == 0} end def load_organizations branches = current_user .organization_branches leaves = current_user .organization_leaves @branches = branches.to_json(: only => [:id, :name]) @leaves = leaves.to_json(: only => [:id, :name]) @organizations = @criteria .branch ? leaves.collect {|o| [ o.name.titleize, o.id ] }.sort : branches.collect {|o| [ o.name.titleize, o.id ] }.sort end

def organization_branches get_organizations .flatten.uniq.select{|org| org.children.size > 0} end def organization_leaves get_organizations .flatten.uniq.select{|org| org.children.size == 0} end def load_organizations branches = current_user .organization_branches leaves = current_user .organization_leaves @branches = branches.to_json(: only => [:id, :name]) @leaves = leaves.to_json(: only => [:id, :name]) @organizations = @criteria .branch ? leaves.collect {|o| [ o.name.titleize, o.id ] }.sort : branches.collect {|o| [ o.name.titleize, o.id ] }.sort end It all started... 2811 queries! Down to 17!!

And then... Invoicing issue: If a customer resubmits an order with 2 years, they should be charged $9 but were only being charged $4.50 Do people ever calculate the ROI on software projects? 53626 * $4.50 = $241,317 (just in the last year)

Our theme of the day: Leverage Multiplying the power of your efforts, to achieve the greatest value

Performance and Business metrics: Bing.com Slower site = revenue drop 1 sec = 2.8% 2 sec = 4.3%

Performance and Business metrics: Shopzilla.com http://www.phpied.com/the-performance-business-pitch/ Went from 6 sec down to 1.2sec

Fred Wilson: Speed is the most important feature 10 Golden Principles of Successful Web Apps Fred Wilson, Venture Capitalist www.avc.com Speed Instant Utility Software is media Less is More Make it programmable Make it personal Make it RESTful Discoverability Clean Playful

Give yourself a fighting chance, make a plan Alois Reitbauer, dynatrace Anti-patterns Thinking Scalability something you just sprinkle on later Guessing, not testing Ad-hoc, unstructured Waiting until it hurts, not proactively planning Thinking that any early planning is Premature optimization Video on wiki.parleys.com

Premature optimization &quot;Premature optimization is the root of all evil&quot; -Donald Knuth

Premature optimization Some people really like Donald Knuth

Premature optimization, with DJ DK like, really, really like him

Premature optimization &quot;Premature optimization is the root of all evil' -Donald Knuth &quot;Quoting Knuth, as a way to avoid planning for performance issues, is a cop-out&quot; @J_McCaffrey &quot;Don't waste time on making optimization changes until you KNOW they are necessary&quot;

If you can not measure it, you can not improve it. — Lord Kelvin

Let's do this the right way Measure: Understand the tools to use, at each layer, to measure and monitor performance Create repeatable tests: Be able to invoke the system and easily test Isolate your changes: One thing at a time That's it: Measure, Test, Isolated changes

Terminology Response time (avg) == Latency Requests per second == Throughput Load : How much work is there? Utilization : How much of the resources are in use? Scalability : How well can we handle the load? Throughput : How many tasks can be done per unit of time? Concurrency : How many tasks can we do at once? Capacity : How big can our throughput go before things fall apart?

Terminology Performance: The resources and Time for a single request Measured in Response Time Throughput: The number requests that can be handled per/time Measured in Requests per second Scalability: Ability to grow and handle more requests, without significantly degrading the experience Performance != Scalability

Terminology Let's say: 1 worker that can compute a job in .5 sec Throughput = 2 jobs per sec Latency = .5 sec per job Performance != Scalability Adding more workers will not improve the latency But not having enough workers can increase the response time, due to Queuing and delay

Resources: Things that we want to measure System Business Time Cpu Memory Network Disk Money Development time QA and testing time User time

There are performance opportunities at each layer

Waterfall: Twitter.com Waterfall Total time: 3815 ms Dynamic generation: 250 ms (~7%) Static: 3369 ms (~89%) * Will not add to 100%

Waterfall When the backend is actually doing work

Walmart: Waterfall Waterfall Total time: 5520 ms Dynamic generation: 388 ms (~7%) Static: 4580 ms (~82%)

Firebug Netpanel Simple and repeatable way to measure page load Usually, less than 20% of the time is spent in fetching the HTML So how do we optimize the other 80%?

YSlow Created by Steve Souders Observes the netpanel data, applies rules and gives you a score. Deep understanding of how the browser works Its the best way to have a repeatable, measurement of your site's load performance http://developer.yahoo.com/yslow/

YSlow Minimize HTTP Requests Use a Content Delivery Network Add an Expires or a Cache-Control Header Gzip Components Put StyleSheets at the Top Put Scripts at the Bottom Avoid CSS Expressions Make JavaScript and CSS External Reduce DNS Lookups Minify JavaScript and CSS Avoid Redirects Remove Duplicate Scripts Configure ETags Make AJAX Cacheable Use GET for AJAX Requests Reduce the Number of DOM Elements No 404s Reduce Cookie Size Use Cookie-Free Domains for Components Avoid Filters Do Not Scale Images in HTML Make favicon.ico Small and Cacheable

Yslow: Created by Steve Souders http://stevesouders.com/cuzillion/

Google Page speed Similar to YSlow includes paint events separates out ads, trackers from your content better recommendations and optimizations 1 click to see optimized version of your files http://code.google.com/speed/page-speed

Make it repeatable http://www.showslow.com/ track your site, or your competitors will check every 24hrs for you webpagetest.org gtmetrix.org zoompf.com loadimpact.com gomez.com

Summary Page load is the place to start Read everything in the Yslow and PageSpeed rules Encourage other devs to play with YSlow, PageSpeed Show Webpagtest.org, and zoompf.com Now that we have a way to measure page performance, we can think about making changes

Isn't everyone already doing this? Analysis of the Alexa top 1000 sites found: 42% don't gzip 44% have more than 2 css files 56% serve css from a cookied domain 62% don't minify 31% have more than 100k size css

Compress with Gzip Can save you 50% to 80% of your bandwidth Simplest performance improvement you can make Beware differences in browsers File types: html, js, css, xml, rss, txt, font, json Probably the simplest and smartest thing you can do for your app!

Combine and minify javascript and css Jsmin or YUICompressor, or Sprockets Asset_packager javascript_include_tag :cache => &quot;cache/all&quot; Load common js libraries from google http://github.com/rpheath/google_ajax_libraries_api Sprites and image optimzation spriteme.org JqueryUI already gives you sprited images Image optimization Smush.it

Expires and Browser Caching Setting a far future expires tells the browser not to ask for the file Using the query string lets us break the caching by having the browser ask for a new file There is a problem with using a URL query string this way http://blog.eliotsykes.com/2010/05/06/why-rails-asset-caching-is-broken/

Rails Caching Take your dynamic content, and make it static Page Action Fragment Full length tutorials http://railslab.newrelic.com/ http://railscasts.com/episodes?search=caching Greg Pollack Aloha on Rails http://vimeo.com/10860860

Request-log-analyzer • http://github.com/wvanbergen/request-log-analyzer • Request distribution per hour • Most requested • HTTP methods • HTTP statuses returned • Rails action cache hits • Request duration • View rendering time • Database time • Process blockers • Failed requests

Commercial tools New Relic Developer plugin is a good start Free version doesn’t get you much Bronze version with heroku Scoutapp.com Similar to new Relic Large selection of community plugins

Testing tools Apache bench ab -n 10 -c 2 http://www.somewhere.com/ Httperf httperf --server localhost --port 3000 --uri / --num-conns 10000 Jmeter yes, its ugly, but its powerful

Database issues Common database issues Bad queries Not utilizing explain Inadequate indexes N+1 queries Selecting more data than is needed Inconsistent queries for the same data

Query_reviewer Common database issues Bad queries Not utilizing explain http:// github.com/dsboulder/query_reviewer Runs explain on all of your queries, outputs to div in page

Rails_indexes http://github.com/eladmeidar/rails_indexes Foreign key indexs • Columns that need to be sorted • Lookup fields • Columns that are used in a GROUP BY • Rake tasks to find missing indexes. New one I haven’t tried yet http:// github.com/samdanavia/ambitious_query_indexer

Bullet plugin Bullet plugin • http://github.com/flyerhzm/bullet Help you reduce the number of queries with alerts (and growl).

Slim_scrooge Be more specific in your select http://github.com/sdsykes/slim_scrooge Instruments your code Observes the usage pattern Suggests/changes your select statement When invoked within the exact same context

Ruby Issues Most are due to memory problems Slow GC Not release references Capturing scope Profiling will reveal what’s going on http:// guides.rubyonrails.org/performance_testing.html If you want to learn everything there is to know about profiling Aman Gupta Joe Damato http:// timetobleed.com / http:// memprof.com / They’ve already identified and fixed several memory leaks

Ruby issues: Faster Ruby Libraries C Extension++ • XML parser http://nokogiri.org/ • JSON parser http://github.com/brianmario/yajl-ruby/ • CSV parser http://www.toastyapps.com/excelsior/ • HTTP client http://github.com/pauldix/typhoeus Date http:// github.com/jeremyevans/home_run

Take home points Performance Performance affects the bottom line The biggest performance win is usually in improving the load time Continue to monitor and test your apps performance over time Gzip, combine and minify to get a big boost Lack of indexes is likely to be one of your biggest backend issues Try to stay up to date with Gems, Plugins, and latest Rails Upgrading to ruby 1.9 will give you a huge performance boost

Links: great info on performance Performance Scaling Rails series http:// railslab.newrelic.com / RailsCasts.com Velocity Conference http://en.oreilly.com/velocity2010 google io conference http://code.google.com/events/io/2010/ dynatrace site http:// ajax.dynatrace.com /pages/ http:// www.igvita.com / http:// www.mysqlperformanceblog.com /

Links Questions? @j_mccaffrey [email_address]

Beyond ySlow: improving page performance outside the initial load MySql and Postgres Database tuning Heroku: Building Scalable apps from the start Tuning performance for Mobile devices Performance Improvements coming in Rails 3, and 3.1 Speeding up your tests

Wait, I made it through that? Architectural musings Background long running tasks Email, image processing, reports, feeds Leverage Rack, Sinatra or Padrino Parallel processing, Event Machine Learn from heroku, scalable from the start Revaluate your cloudy-ness Great RailsConf presentation by James Golick

Wait, I made it through that? Designing for performance Watch how they really use it Multiple tabs? Back and forth between list and detail view Great presentation at WindyCityRails2009 by David Eisinger Optimizing perceived performance Make it feel fast

Questions for you Architectural musings Moving to or already on Ruby 1.9? Serving your content from a separate domain? Cookieless?

Windy cityrails performance_tuning

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Windy cityrails performance_tuning

Similar to Windy cityrails performance_tuning (20)

More from John McCaffrey

More from John McCaffrey (8)

Recently uploaded

Recently uploaded (20)

Windy cityrails performance_tuning