SlideShare a Scribd company logo
PyGotham 2014
Introduction to
Profiling
Perrin Harkins
“We should forget about small efficiencies, say
about 97% of the time: premature optimization is
the root of all evil. Yet we should not pass up our
opportunities in that critical 3%. A good
programmer will not be lulled into complacency
by such reasoning, he will be wise to look carefully
at the critical code; but only after that code has
been identified.”
–Donald Knuth
“Bottlenecks occur in surprising places, so don't
try to second guess and put in a speed hack until
you have proven that's where the bottleneck is.”
–Rob Pike
What will a profiler tell us?
❖ Function execution time!
❖ Memory usage, etc. are possible, but for another day!
❖ More about line profiling later!
❖ Real (wall clock) time!
❖ Inclusive vs exclusive time!
❖ Number of calls, primitive and recursive

Recommended for you

Caching the Uncacheable: Leveraging Your CDN to Cache Dynamic Content
Caching the Uncacheable: Leveraging Your CDN to Cache Dynamic ContentCaching the Uncacheable: Leveraging Your CDN to Cache Dynamic Content
Caching the Uncacheable: Leveraging Your CDN to Cache Dynamic Content

June 25, 2014. Hooman Beheshti, VP Technology at Fastly, discusses how using a real-time, modern CDN that provides instant cache invalidation and real-time analytics allows for instantaneous control over dynamic content caching. In this session, he looks at the challenges CDNs face with dynamic content and how you can use programmatic means to fully integrate your applications with your CDN.

content delivery networkcdndynamic content
Event Driven Architecture Concepts in Web Technologies - Part 1
Event Driven Architecture Concepts in Web Technologies - Part 1Event Driven Architecture Concepts in Web Technologies - Part 1
Event Driven Architecture Concepts in Web Technologies - Part 1

The document discusses event-driven architecture and how it has evolved from processes to threads to events. It provides examples to illustrate synchronous vs asynchronous processing and event-driven vs process-driven approaches. It describes how Node.js uses a single thread and event loop architecture to handle asynchronous I/O calls via callbacks. Various real-time applications that can benefit from Node.js' event-driven approach are listed.

event-drivennodejsweb
opentsdb in a real enviroment
opentsdb in a real enviromentopentsdb in a real enviroment
opentsdb in a real enviroment

This document discusses the architecture of OpenTSDB and provides instructions for setting up and using OpenTSDB to collect CPU metrics from a host. It describes: 1) The main components of OpenTSDB including the TSD, HBase, and Hadoop. 2) How to start and stop OpenTSDB by starting/stopping TSD, HBase, and Hadoop. 3) A sample scenario of collecting CPU load average metrics from a host and inserting them into OpenTSDB using a script to post metrics to the TSD socket.

hbasemetrics collectionopentsdb
cProfile
❖ Generates profile data that can be read in shell or GUI
tools!
❖ 30% or more speed penalty
cProfile
From command line:!
$ python -m cProfile -o myscript.prof myscript.py
cProfile
Or, in your program:!
import cProfile	
cProfile.run('slow_function', 'myscript.prof')
cProfile
Or, even more flexible:!
pr = cProfile.Profile()	
pr.enable()	
… thing you want to profile …!
pr.disable()

Recommended for you

LCA2014 - Introduction to Go
LCA2014 - Introduction to GoLCA2014 - Introduction to Go
LCA2014 - Introduction to Go

Google's Go is a relatively new systems programming language that has recently gained a lot of traction with developers. It brings together the ease and efficiency of development in modern interpreted languages like Python, Perl, and Ruby with the efficiency and safety of a statically typed, compiled language like C/C++ and Java. On top of that, Go is a language built for modern hardware and problems. With built-in support for concurrency, programmers can easily build software to scale up to today's many-core beasts. Programming in Go is really nice, and in this tutorial, you will learn why. We will cover an introduction to the Go programming language, and together we will build a multi-user network service demonstrating all of the major principles of programming in Go.

golang
Developing cacheable PHP applications - PHPLimburgBE 2018
Developing cacheable PHP applications - PHPLimburgBE 2018Developing cacheable PHP applications - PHPLimburgBE 2018
Developing cacheable PHP applications - PHPLimburgBE 2018

In this presentation, I show the audience how to implement HTTP caching best practices in a non-intrusive way in PHP Symfony 4 code base. This presentation focuses on topics like: - Caching using cache-control headers - Cache variations using the Vary header - Conditional requests using headers like ETag & If-None-Match - ESI discovery & parsing using headers like Surrogate-Capability & Surrogate-Control - Caching stateful content using JSON Web Token Validation in Varnish More information about this presentation is available at https://feryn.eu/speaking/developing-cacheable-php-applications-php-limburg-be/

cachingcachehttp
Top Node.js Metrics to Watch
Top Node.js Metrics to WatchTop Node.js Metrics to Watch
Top Node.js Metrics to Watch

This document discusses key metrics to monitor for Node.js applications, including event loop latency, garbage collection cycles and time, process memory usage, HTTP request and error rates, and correlating metrics across worker processes. It provides examples of metric thresholds and issues that could be detected, such as high garbage collection times indicating a problem or an event loop blocking issue leading to high latency.

monitoringdevopsperformance
pstats
import pstats	
profile = pstats.Stats('myscript.prof')	
profile.add('myscript.prof2')	
profile.strip_dirs()	
profile.sort_stats('cumulative')	
profile.print_stats(20)
12192418 function calls (11990470 primitive calls) in 84.268 seconds	
!
Ordered by: cumulative time	
List reduced from 1211 to 20 due to restriction <20>	
!
ncalls tottime percall cumtime percall filename:lineno(function)	
1 0.000 0.000 84.402 84.402 <string>:1(<module>)	
1 0.021 0.021 84.402 84.402 act_bench.py:243(_do_act)	
500 0.096 0.000 84.381 0.169 __init__.py:170(act)	
500 0.007 0.000 35.874 0.072 petition_actions.py:460(save)	
500 0.066 0.000 33.431 0.067 action_processor.py:1303(save)	
500 0.160 0.000 22.684 0.045 users.py:1002(save)	
10501 0.175 0.000 21.963 0.002 query.py:852(_fetch_all)	
14001 0.286 0.000 21.472 0.002 compiler.py:758(execute_sql)	
6501 0.047 0.000 14.200 0.002 query.py:76(__len__)
profile.print_callees('full_clean', 10)	
!
List reduced from 1211 to 2 due to restriction <'full_clean'>	
!
Function called...	
ncalls tottime cumtime	
forms.py:260(full_clean) -> 500 0.177 2.855 forms.py:
277(_clean_fields)	
500 0.003 0.030 forms.py:298(_clean_form)	
500 0.031 2.784 models.py:
393(_post_clean)	
base.py:918(full_clean) -> 500 0.001 0.001 base.py:738(clean)	
500 0.096 2.399 base.py:952(clean_fields)
profile.print_callers('full_clean')	
!
List reduced from 1211 to 2 due to restriction <'full_clean'>	
!
Function was called by...	
ncalls tottime cumtime	
forms.py:260(full_clean) <- 500 0.009 5.678 forms.py:117(errors)	
base.py:918(full_clean) <- 500 0.005 2.405 models.py:
393(_post_clean)

Recommended for you

Odoo Online platform: architecture and challenges
Odoo Online platform: architecture and challengesOdoo Online platform: architecture and challenges
Odoo Online platform: architecture and challenges

A short introduction to the technical architecture of the Odoo Online platform, including the advanced integrated features (instant DNS, email gateways, etc.), and the technical aspect of the SLA. By Olivier Dony - Lead Developer & Community Manager, OpenERP

 
by Odoo
odoo online platform saas architecture challenges
Altitude SF 2017: Debugging Fastly VCL 101
Altitude SF 2017: Debugging Fastly VCL 101Altitude SF 2017: Debugging Fastly VCL 101
Altitude SF 2017: Debugging Fastly VCL 101

Learn from Fastly veteran Cassandra Dixon on some of the most common customer issues we see — such as why things aren’t caching, misconfigured origins, issues with intermediary proxies, and VCL snafus — and the best ways to resolve them. We’ll also discuss our unique approach to debugging — using seemingly mundane tools to diagnose issues in creative ways — and how you can apply these methods to your own organization to get the most out of Fastly’s offerings.

varnishtechnologysoftware
Performance and stability testing \w Gatling
Performance and stability testing \w GatlingPerformance and stability testing \w Gatling
Performance and stability testing \w Gatling

Explore how Gatling can help with performance & stability testing in TDD fashion.

gatlingjavaperformance
KCacheGrind
!
❖ GUI for viewing profile data!
❖ Run your profile output through pyprof2calltree!
❖ On a Mac, qcachegrind is easier to install
PyGotham 2014 Introduction to Profiling
PyGotham 2014 Introduction to Profiling
RunSnakeRun
❖ Squaremap of call tree!
❖ Maybe useful for spotting large exclusive time functions

Recommended for you

Full Stack Load Testing
Full Stack Load Testing Full Stack Load Testing
Full Stack Load Testing

A talk I gave at the Boston Web Performance Meetup in August 2014. Performance is one of the most challenging issues in modern web app design, in large part because modeling, testing, and validating performance before deploying to production is so challenging. While many ops teams have nailed down the problem of re-creating pre-production environments that closely mimic production, those environments frequently rely on known-good components beyond the application code itself: AWS ELB, F5 load balancers, CDNs, Varnish, and more. Testing plug-in components like that can be challenging, because their performance characteristics don't directly align with application metrics. - How many simultaneous users can my load balancer support? - What sort of network load will I put on my CDN (i.e., how much will it cost?) - How do different user behavior patterns affect performance? In this meetup, we'll introduce a novel tool in this toolbox: tcpreplay, an open-source tool for replaying packet capture files back at an application. By replaying user traffic to a staging environment, you can test the effects of - Network saturation to the load balancer - High numbers of users / IPs - Lots of traffic to your other monitoring tools!

Tips on how to improve the performance of your custom modules for high volume...
Tips on how to improve the performance of your custom modules for high volume...Tips on how to improve the performance of your custom modules for high volume...
Tips on how to improve the performance of your custom modules for high volume...

The document discusses performance optimization for OpenERP deployments handling high volumes of transactions and data. It provides recommendations around hardware sizing, PostgreSQL and OpenERP architecture, monitoring tools, and analyzing PostgreSQL logs and statistics. Key recommendations include proper sizing based on load testing, optimizing PostgreSQL configuration and storage, monitoring response times and locks, and analyzing logs to identify performance bottlenecks like long-running queries or full table scans.

 
by Odoo
openerpopenerpdays
Creating Reusable Puppet Profiles
Creating Reusable Puppet ProfilesCreating Reusable Puppet Profiles
Creating Reusable Puppet Profiles

"Roles and Profiles" is now the ubiquitous design pattern to create your puppet code tree. In this talk we will discuss writing reusable and maintainable profiles. We ll start by introducing creating module structures and will move on to type hinting and setting appropriate defaults. Finally we ll discuss the importance and the enforcing of code style conventions that allows multiple teams or projects to inner-source profiless

puppetpdk
PyGotham 2014 Introduction to Profiling
PyGotham 2014 Introduction to Profiling
Using your results
❖ Bottom up approach!
❖ Start with a large exclusive time sub!
❖ Climb up call graph to find something you can affect!
❖ "We're spending a lot of time in deepcopy(). What's
calling that so much?"!
❖ Might miss higher-level fixes
Using your results
❖ Top down approach!
❖ Start with a large inclusive time sub!
❖ Walk down call graph to find something you can
affect!
❖ "We're spending a lot of time in this validate() method.
What's it doing that takes so long?"!
❖ Look for structural changes

Recommended for you

Engage 2013 - Multi Channel Data Collection
Engage 2013 - Multi Channel Data CollectionEngage 2013 - Multi Channel Data Collection
Engage 2013 - Multi Channel Data Collection

The document discusses methods for collecting multi-channel data across different domains and platforms. It describes how cross-domain tracking works using shared cookies to pass visitor IDs between sites. It also covers using iframes to collect data across domains by adding tracking code or using postMessages. Additionally, it discusses collecting event-based data using selectors and JavaScript events, and capturing video playback events with HTML5 event listeners.

engage 2013
Stupid Boot Tricks: using ipxe and chef to get to boot management bliss
Stupid Boot Tricks: using ipxe and chef to get to boot management blissStupid Boot Tricks: using ipxe and chef to get to boot management bliss
Stupid Boot Tricks: using ipxe and chef to get to boot management bliss

Jason Cook discusses his experience setting up boot infrastructure for Fastly's caching clusters. He outlines how they moved from using existing tools like Cobbler and Razor to building their own solution called Donner using iPXE to boot machines over HTTP. Donner uses Chef to store machine metadata and configuration which allows the boot process to install operating systems, configure networking, and run Chef on first boot to provision machines.

netbootchefchefconf
Run Node Run
Run Node RunRun Node Run
Run Node Run

This document discusses optimizing JavaScript performance in Node.js. It covers benchmarking Node applications, tips for writing efficient JavaScript that avoids hidden classes and dictionary mode in the V8 engine, profiling Node to find hot spots, and how the V8 optimizing compiler works. The presenter emphasizes the importance of speed and provides resources for further optimizing Node applications.

node.jsnodev8
Line profiling
❖ line_profiler does exist!
❖ Results are not very actionable!
❖ If you get this far, you probably should stop (or refactor
your methods!)
Good profiling technique
❖ Create a repeatable benchmark test!
❖ Allows you to measure progress!
❖ Iterations/second!
❖ Time for n iterations
What usually helps
❖ Removing unnecessary work!
❖ “We load that config data every time, even when we don’t
use it.”!
❖ Using a more efficient algorithm
What usually helps
❖ Batching I/O (disk or net) operations!
❖ Database stuff!
❖ SQL tuning!
❖ Indexes!
❖ Transactions

Recommended for you

On Centralizing Logs
On Centralizing LogsOn Centralizing Logs
On Centralizing Logs

On Centralizing Logs with Syslog, LogStash, Elasticsearch, Kibana. Presentation from Radu Gheorghe from Sematext at Monitorama EU 2013.

sysloglogstashlogging
Mobile App Performance: Getting the Most from APIs (MBL203) | AWS re:Invent ...
Mobile App Performance:  Getting the Most from APIs (MBL203) | AWS re:Invent ...Mobile App Performance:  Getting the Most from APIs (MBL203) | AWS re:Invent ...
Mobile App Performance: Getting the Most from APIs (MBL203) | AWS re:Invent ...

(Presented by New Relic) Too often, developers think of a mobile app as simply code running on the device. A mobile app is much more than that. Every web API used by an app becomes as much a part of the app as the code running on the device. But while mobile developers have control over their code, they don't always have control over the APIs they use. Web APIs and their infrastructure impact app performance and ultimately the user experience. This presentation covers some of the essential aspects of app performance management when web APIs are present, including: -HTTP headers are your friend--stop ignoring all they have to tell you -Control your network connections on the device—don't just leave things to the OS -Configure all your caches and use them -Whatever you do, measure early and often The session includes a customer story from the CTO of Mirego, and demos of New Relic mobile app performance monitoring, where you see how to drill down into specific requests to see performance by response time, throughput, and data transfer size.

mobilestartupsapp
Building source code level profiler for C++.pdf
Building source code level profiler for C++.pdfBuilding source code level profiler for C++.pdf
Building source code level profiler for C++.pdf

1. The document describes building a source code level profiler for C++ applications. It outlines 4 milestones: logging execution time, reducing macros, tracking function hit counts, and call path profiling using a radix tree. 2. Key aspects discussed include using timers to log function durations, storing profiling data in a timed entry class, and maintaining a call tree using a radix tree with nodes representing functions and profiling data. 3. The goal is to develop a customizable profiler to identify performance bottlenecks by profiling execution times and call paths at the source code level.

What usually helps
❖ Caching!
❖ Easy to add, hard to live with!
❖ Code complexity!
❖ Invalidation calls!
❖ Dependency tracking!
❖ Business customers care about data freshness
Thank you!

More Related Content

What's hot

Zend Con 2008 Slides
Zend Con 2008 SlidesZend Con 2008 Slides
Zend Con 2008 Slides
mkherlakian
 
5 things MySql
5 things MySql5 things MySql
5 things MySql
sarahnovotny
 
WordPress Need For Speed
WordPress Need For SpeedWordPress Need For Speed
WordPress Need For Speed
pdeschen
 
Caching the Uncacheable: Leveraging Your CDN to Cache Dynamic Content
Caching the Uncacheable: Leveraging Your CDN to Cache Dynamic ContentCaching the Uncacheable: Leveraging Your CDN to Cache Dynamic Content
Caching the Uncacheable: Leveraging Your CDN to Cache Dynamic Content
Fastly
 
Event Driven Architecture Concepts in Web Technologies - Part 1
Event Driven Architecture Concepts in Web Technologies - Part 1Event Driven Architecture Concepts in Web Technologies - Part 1
Event Driven Architecture Concepts in Web Technologies - Part 1
Hamidreza Soleimani
 
opentsdb in a real enviroment
opentsdb in a real enviromentopentsdb in a real enviroment
opentsdb in a real enviroment
Chen Robert
 
LCA2014 - Introduction to Go
LCA2014 - Introduction to GoLCA2014 - Introduction to Go
LCA2014 - Introduction to Go
dreamwidth
 
Developing cacheable PHP applications - PHPLimburgBE 2018
Developing cacheable PHP applications - PHPLimburgBE 2018Developing cacheable PHP applications - PHPLimburgBE 2018
Developing cacheable PHP applications - PHPLimburgBE 2018
Thijs Feryn
 
Top Node.js Metrics to Watch
Top Node.js Metrics to WatchTop Node.js Metrics to Watch
Top Node.js Metrics to Watch
Sematext Group, Inc.
 
Odoo Online platform: architecture and challenges
Odoo Online platform: architecture and challengesOdoo Online platform: architecture and challenges
Odoo Online platform: architecture and challenges
Odoo
 
Altitude SF 2017: Debugging Fastly VCL 101
Altitude SF 2017: Debugging Fastly VCL 101Altitude SF 2017: Debugging Fastly VCL 101
Altitude SF 2017: Debugging Fastly VCL 101
Fastly
 
Performance and stability testing \w Gatling
Performance and stability testing \w GatlingPerformance and stability testing \w Gatling
Performance and stability testing \w Gatling
Dmitry Vrublevsky
 
Full Stack Load Testing
Full Stack Load Testing Full Stack Load Testing
Full Stack Load Testing
Terral R Jordan
 
Tips on how to improve the performance of your custom modules for high volume...
Tips on how to improve the performance of your custom modules for high volume...Tips on how to improve the performance of your custom modules for high volume...
Tips on how to improve the performance of your custom modules for high volume...
Odoo
 
Creating Reusable Puppet Profiles
Creating Reusable Puppet ProfilesCreating Reusable Puppet Profiles
Creating Reusable Puppet Profiles
Bram Vogelaar
 
Engage 2013 - Multi Channel Data Collection
Engage 2013 - Multi Channel Data CollectionEngage 2013 - Multi Channel Data Collection
Engage 2013 - Multi Channel Data Collection
Webtrends
 
Stupid Boot Tricks: using ipxe and chef to get to boot management bliss
Stupid Boot Tricks: using ipxe and chef to get to boot management blissStupid Boot Tricks: using ipxe and chef to get to boot management bliss
Stupid Boot Tricks: using ipxe and chef to get to boot management bliss
macslide
 
Run Node Run
Run Node RunRun Node Run
Run Node Run
Kevin Swiber
 
On Centralizing Logs
On Centralizing LogsOn Centralizing Logs
On Centralizing Logs
Sematext Group, Inc.
 
Mobile App Performance: Getting the Most from APIs (MBL203) | AWS re:Invent ...
Mobile App Performance:  Getting the Most from APIs (MBL203) | AWS re:Invent ...Mobile App Performance:  Getting the Most from APIs (MBL203) | AWS re:Invent ...
Mobile App Performance: Getting the Most from APIs (MBL203) | AWS re:Invent ...
Amazon Web Services
 

What's hot (20)

Zend Con 2008 Slides
Zend Con 2008 SlidesZend Con 2008 Slides
Zend Con 2008 Slides
 
5 things MySql
5 things MySql5 things MySql
5 things MySql
 
WordPress Need For Speed
WordPress Need For SpeedWordPress Need For Speed
WordPress Need For Speed
 
Caching the Uncacheable: Leveraging Your CDN to Cache Dynamic Content
Caching the Uncacheable: Leveraging Your CDN to Cache Dynamic ContentCaching the Uncacheable: Leveraging Your CDN to Cache Dynamic Content
Caching the Uncacheable: Leveraging Your CDN to Cache Dynamic Content
 
Event Driven Architecture Concepts in Web Technologies - Part 1
Event Driven Architecture Concepts in Web Technologies - Part 1Event Driven Architecture Concepts in Web Technologies - Part 1
Event Driven Architecture Concepts in Web Technologies - Part 1
 
opentsdb in a real enviroment
opentsdb in a real enviromentopentsdb in a real enviroment
opentsdb in a real enviroment
 
LCA2014 - Introduction to Go
LCA2014 - Introduction to GoLCA2014 - Introduction to Go
LCA2014 - Introduction to Go
 
Developing cacheable PHP applications - PHPLimburgBE 2018
Developing cacheable PHP applications - PHPLimburgBE 2018Developing cacheable PHP applications - PHPLimburgBE 2018
Developing cacheable PHP applications - PHPLimburgBE 2018
 
Top Node.js Metrics to Watch
Top Node.js Metrics to WatchTop Node.js Metrics to Watch
Top Node.js Metrics to Watch
 
Odoo Online platform: architecture and challenges
Odoo Online platform: architecture and challengesOdoo Online platform: architecture and challenges
Odoo Online platform: architecture and challenges
 
Altitude SF 2017: Debugging Fastly VCL 101
Altitude SF 2017: Debugging Fastly VCL 101Altitude SF 2017: Debugging Fastly VCL 101
Altitude SF 2017: Debugging Fastly VCL 101
 
Performance and stability testing \w Gatling
Performance and stability testing \w GatlingPerformance and stability testing \w Gatling
Performance and stability testing \w Gatling
 
Full Stack Load Testing
Full Stack Load Testing Full Stack Load Testing
Full Stack Load Testing
 
Tips on how to improve the performance of your custom modules for high volume...
Tips on how to improve the performance of your custom modules for high volume...Tips on how to improve the performance of your custom modules for high volume...
Tips on how to improve the performance of your custom modules for high volume...
 
Creating Reusable Puppet Profiles
Creating Reusable Puppet ProfilesCreating Reusable Puppet Profiles
Creating Reusable Puppet Profiles
 
Engage 2013 - Multi Channel Data Collection
Engage 2013 - Multi Channel Data CollectionEngage 2013 - Multi Channel Data Collection
Engage 2013 - Multi Channel Data Collection
 
Stupid Boot Tricks: using ipxe and chef to get to boot management bliss
Stupid Boot Tricks: using ipxe and chef to get to boot management blissStupid Boot Tricks: using ipxe and chef to get to boot management bliss
Stupid Boot Tricks: using ipxe and chef to get to boot management bliss
 
Run Node Run
Run Node RunRun Node Run
Run Node Run
 
On Centralizing Logs
On Centralizing LogsOn Centralizing Logs
On Centralizing Logs
 
Mobile App Performance: Getting the Most from APIs (MBL203) | AWS re:Invent ...
Mobile App Performance:  Getting the Most from APIs (MBL203) | AWS re:Invent ...Mobile App Performance:  Getting the Most from APIs (MBL203) | AWS re:Invent ...
Mobile App Performance: Getting the Most from APIs (MBL203) | AWS re:Invent ...
 

Similar to PyGotham 2014 Introduction to Profiling

Building source code level profiler for C++.pdf
Building source code level profiler for C++.pdfBuilding source code level profiler for C++.pdf
Building source code level profiler for C++.pdf
ssuser28de9e
 
Advertising Fraud Detection at Scale at T-Mobile
Advertising Fraud Detection at Scale at T-MobileAdvertising Fraud Detection at Scale at T-Mobile
Advertising Fraud Detection at Scale at T-Mobile
Databricks
 
php & performance
 php & performance php & performance
php & performance
simon8410
 
Fantastic performance and where to find it
Fantastic performance and where to find itFantastic performance and where to find it
Fantastic performance and where to find it
RichardWarburton
 
PHP & Performance
PHP & PerformancePHP & Performance
PHP & Performance
毅 吕
 
Testing Wi-Fi with OSS Tools
Testing Wi-Fi with OSS ToolsTesting Wi-Fi with OSS Tools
Testing Wi-Fi with OSS Tools
All Things Open
 
Introduction to Industrial Control Systems : Pentesting PLCs 101 (BlackHat Eu...
Introduction to Industrial Control Systems : Pentesting PLCs 101 (BlackHat Eu...Introduction to Industrial Control Systems : Pentesting PLCs 101 (BlackHat Eu...
Introduction to Industrial Control Systems : Pentesting PLCs 101 (BlackHat Eu...
arnaudsoullie
 
PerfUG 3 - perfs système
PerfUG 3 - perfs systèmePerfUG 3 - perfs système
PerfUG 3 - perfs système
Ludovic Piot
 
Solve the colocation conundrum: Performance and density at scale with Kubernetes
Solve the colocation conundrum: Performance and density at scale with KubernetesSolve the colocation conundrum: Performance and density at scale with Kubernetes
Solve the colocation conundrum: Performance and density at scale with Kubernetes
Niklas Quarfot Nielsen
 
Puppet Camp DC 2015: Distributed OpenSCAP Compliance Validation with MCollective
Puppet Camp DC 2015: Distributed OpenSCAP Compliance Validation with MCollectivePuppet Camp DC 2015: Distributed OpenSCAP Compliance Validation with MCollective
Puppet Camp DC 2015: Distributed OpenSCAP Compliance Validation with MCollective
Puppet
 
PipelineAI Optimizes Your Enterprise AI Pipeline from Distributed Training to...
PipelineAI Optimizes Your Enterprise AI Pipeline from Distributed Training to...PipelineAI Optimizes Your Enterprise AI Pipeline from Distributed Training to...
PipelineAI Optimizes Your Enterprise AI Pipeline from Distributed Training to...
Chris Fregly
 
Being HAPI! Reverse Proxying on Purpose
Being HAPI! Reverse Proxying on PurposeBeing HAPI! Reverse Proxying on Purpose
Being HAPI! Reverse Proxying on Purpose
Aman Kohli
 
[PASS Summit 2016] Blazing Fast, Planet-Scale Customer Scenarios with Azure D...
[PASS Summit 2016] Blazing Fast, Planet-Scale Customer Scenarios with Azure D...[PASS Summit 2016] Blazing Fast, Planet-Scale Customer Scenarios with Azure D...
[PASS Summit 2016] Blazing Fast, Planet-Scale Customer Scenarios with Azure D...
Andrew Liu
 
Optimizing, Profiling, and Deploying TensorFlow AI Models with GPUs - San Fra...
Optimizing, Profiling, and Deploying TensorFlow AI Models with GPUs - San Fra...Optimizing, Profiling, and Deploying TensorFlow AI Models with GPUs - San Fra...
Optimizing, Profiling, and Deploying TensorFlow AI Models with GPUs - San Fra...
Chris Fregly
 
What’s eating python performance
What’s eating python performanceWhat’s eating python performance
What’s eating python performance
Piotr Przymus
 
Building Google's ML Engine from Scratch on AWS with GPUs, Kubernetes, Istio,...
Building Google's ML Engine from Scratch on AWS with GPUs, Kubernetes, Istio,...Building Google's ML Engine from Scratch on AWS with GPUs, Kubernetes, Istio,...
Building Google's ML Engine from Scratch on AWS with GPUs, Kubernetes, Istio,...
Chris Fregly
 
Designing High Performance RTC Signaling Servers
Designing High Performance RTC Signaling ServersDesigning High Performance RTC Signaling Servers
Designing High Performance RTC Signaling Servers
Daniel-Constantin Mierla
 
Hadoop Query Performance Smackdown
Hadoop Query Performance SmackdownHadoop Query Performance Smackdown
Hadoop Query Performance Smackdown
DataWorks Summit
 
The Real World - Plugging the Enterprise Into It (nodejs)
The Real World - Plugging  the Enterprise Into It (nodejs)The Real World - Plugging  the Enterprise Into It (nodejs)
The Real World - Plugging the Enterprise Into It (nodejs)
Aman Kohli
 
Engineering Challenges Doing Intrusion Detection in the Cloud
Engineering Challenges Doing Intrusion Detection in the CloudEngineering Challenges Doing Intrusion Detection in the Cloud
Engineering Challenges Doing Intrusion Detection in the Cloud
randomuserid
 

Similar to PyGotham 2014 Introduction to Profiling (20)

Building source code level profiler for C++.pdf
Building source code level profiler for C++.pdfBuilding source code level profiler for C++.pdf
Building source code level profiler for C++.pdf
 
Advertising Fraud Detection at Scale at T-Mobile
Advertising Fraud Detection at Scale at T-MobileAdvertising Fraud Detection at Scale at T-Mobile
Advertising Fraud Detection at Scale at T-Mobile
 
php & performance
 php & performance php & performance
php & performance
 
Fantastic performance and where to find it
Fantastic performance and where to find itFantastic performance and where to find it
Fantastic performance and where to find it
 
PHP & Performance
PHP & PerformancePHP & Performance
PHP & Performance
 
Testing Wi-Fi with OSS Tools
Testing Wi-Fi with OSS ToolsTesting Wi-Fi with OSS Tools
Testing Wi-Fi with OSS Tools
 
Introduction to Industrial Control Systems : Pentesting PLCs 101 (BlackHat Eu...
Introduction to Industrial Control Systems : Pentesting PLCs 101 (BlackHat Eu...Introduction to Industrial Control Systems : Pentesting PLCs 101 (BlackHat Eu...
Introduction to Industrial Control Systems : Pentesting PLCs 101 (BlackHat Eu...
 
PerfUG 3 - perfs système
PerfUG 3 - perfs systèmePerfUG 3 - perfs système
PerfUG 3 - perfs système
 
Solve the colocation conundrum: Performance and density at scale with Kubernetes
Solve the colocation conundrum: Performance and density at scale with KubernetesSolve the colocation conundrum: Performance and density at scale with Kubernetes
Solve the colocation conundrum: Performance and density at scale with Kubernetes
 
Puppet Camp DC 2015: Distributed OpenSCAP Compliance Validation with MCollective
Puppet Camp DC 2015: Distributed OpenSCAP Compliance Validation with MCollectivePuppet Camp DC 2015: Distributed OpenSCAP Compliance Validation with MCollective
Puppet Camp DC 2015: Distributed OpenSCAP Compliance Validation with MCollective
 
PipelineAI Optimizes Your Enterprise AI Pipeline from Distributed Training to...
PipelineAI Optimizes Your Enterprise AI Pipeline from Distributed Training to...PipelineAI Optimizes Your Enterprise AI Pipeline from Distributed Training to...
PipelineAI Optimizes Your Enterprise AI Pipeline from Distributed Training to...
 
Being HAPI! Reverse Proxying on Purpose
Being HAPI! Reverse Proxying on PurposeBeing HAPI! Reverse Proxying on Purpose
Being HAPI! Reverse Proxying on Purpose
 
[PASS Summit 2016] Blazing Fast, Planet-Scale Customer Scenarios with Azure D...
[PASS Summit 2016] Blazing Fast, Planet-Scale Customer Scenarios with Azure D...[PASS Summit 2016] Blazing Fast, Planet-Scale Customer Scenarios with Azure D...
[PASS Summit 2016] Blazing Fast, Planet-Scale Customer Scenarios with Azure D...
 
Optimizing, Profiling, and Deploying TensorFlow AI Models with GPUs - San Fra...
Optimizing, Profiling, and Deploying TensorFlow AI Models with GPUs - San Fra...Optimizing, Profiling, and Deploying TensorFlow AI Models with GPUs - San Fra...
Optimizing, Profiling, and Deploying TensorFlow AI Models with GPUs - San Fra...
 
What’s eating python performance
What’s eating python performanceWhat’s eating python performance
What’s eating python performance
 
Building Google's ML Engine from Scratch on AWS with GPUs, Kubernetes, Istio,...
Building Google's ML Engine from Scratch on AWS with GPUs, Kubernetes, Istio,...Building Google's ML Engine from Scratch on AWS with GPUs, Kubernetes, Istio,...
Building Google's ML Engine from Scratch on AWS with GPUs, Kubernetes, Istio,...
 
Designing High Performance RTC Signaling Servers
Designing High Performance RTC Signaling ServersDesigning High Performance RTC Signaling Servers
Designing High Performance RTC Signaling Servers
 
Hadoop Query Performance Smackdown
Hadoop Query Performance SmackdownHadoop Query Performance Smackdown
Hadoop Query Performance Smackdown
 
The Real World - Plugging the Enterprise Into It (nodejs)
The Real World - Plugging  the Enterprise Into It (nodejs)The Real World - Plugging  the Enterprise Into It (nodejs)
The Real World - Plugging the Enterprise Into It (nodejs)
 
Engineering Challenges Doing Intrusion Detection in the Cloud
Engineering Challenges Doing Intrusion Detection in the CloudEngineering Challenges Doing Intrusion Detection in the Cloud
Engineering Challenges Doing Intrusion Detection in the Cloud
 

More from Perrin Harkins

Care and feeding notes
Care and feeding notesCare and feeding notes
Care and feeding notes
Perrin Harkins
 
Low maintenance perl notes
Low maintenance perl notesLow maintenance perl notes
Low maintenance perl notes
Perrin Harkins
 
Choosing a Web Architecture for Perl
Choosing a Web Architecture for PerlChoosing a Web Architecture for Perl
Choosing a Web Architecture for Perl
Perrin Harkins
 
Efficient Shared Data in Perl
Efficient Shared Data in PerlEfficient Shared Data in Perl
Efficient Shared Data in Perl
Perrin Harkins
 
Choosing a Templating System
Choosing a Templating SystemChoosing a Templating System
Choosing a Templating System
Perrin Harkins
 
Scaling Databases with DBIx::Router
Scaling Databases with DBIx::RouterScaling Databases with DBIx::Router
Scaling Databases with DBIx::Router
Perrin Harkins
 
Low-Maintenance Perl
Low-Maintenance PerlLow-Maintenance Perl
Low-Maintenance Perl
Perrin Harkins
 
Care and Feeding of Large Web Applications
Care and Feeding of Large Web ApplicationsCare and Feeding of Large Web Applications
Care and Feeding of Large Web Applications
Perrin Harkins
 
Top 10 Perl Performance Tips
Top 10 Perl Performance TipsTop 10 Perl Performance Tips
Top 10 Perl Performance Tips
Perrin Harkins
 
The Most Common Template Toolkit Mistake
The Most Common Template Toolkit MistakeThe Most Common Template Toolkit Mistake
The Most Common Template Toolkit Mistake
Perrin Harkins
 

More from Perrin Harkins (10)

Care and feeding notes
Care and feeding notesCare and feeding notes
Care and feeding notes
 
Low maintenance perl notes
Low maintenance perl notesLow maintenance perl notes
Low maintenance perl notes
 
Choosing a Web Architecture for Perl
Choosing a Web Architecture for PerlChoosing a Web Architecture for Perl
Choosing a Web Architecture for Perl
 
Efficient Shared Data in Perl
Efficient Shared Data in PerlEfficient Shared Data in Perl
Efficient Shared Data in Perl
 
Choosing a Templating System
Choosing a Templating SystemChoosing a Templating System
Choosing a Templating System
 
Scaling Databases with DBIx::Router
Scaling Databases with DBIx::RouterScaling Databases with DBIx::Router
Scaling Databases with DBIx::Router
 
Low-Maintenance Perl
Low-Maintenance PerlLow-Maintenance Perl
Low-Maintenance Perl
 
Care and Feeding of Large Web Applications
Care and Feeding of Large Web ApplicationsCare and Feeding of Large Web Applications
Care and Feeding of Large Web Applications
 
Top 10 Perl Performance Tips
Top 10 Perl Performance TipsTop 10 Perl Performance Tips
Top 10 Perl Performance Tips
 
The Most Common Template Toolkit Mistake
The Most Common Template Toolkit MistakeThe Most Common Template Toolkit Mistake
The Most Common Template Toolkit Mistake
 

Recently uploaded

BLOCKCHAIN FOR DUMMIES: GUIDEBOOK FOR ALL
BLOCKCHAIN FOR DUMMIES: GUIDEBOOK FOR ALLBLOCKCHAIN FOR DUMMIES: GUIDEBOOK FOR ALL
BLOCKCHAIN FOR DUMMIES: GUIDEBOOK FOR ALL
Liveplex
 
WPRiders Company Presentation Slide Deck
WPRiders Company Presentation Slide DeckWPRiders Company Presentation Slide Deck
WPRiders Company Presentation Slide Deck
Lidia A.
 
Choose our Linux Web Hosting for a seamless and successful online presence
Choose our Linux Web Hosting for a seamless and successful online presenceChoose our Linux Web Hosting for a seamless and successful online presence
Choose our Linux Web Hosting for a seamless and successful online presence
rajancomputerfbd
 
What's New in Copilot for Microsoft365 May 2024.pptx
What's New in Copilot for Microsoft365 May 2024.pptxWhat's New in Copilot for Microsoft365 May 2024.pptx
What's New in Copilot for Microsoft365 May 2024.pptx
Stephanie Beckett
 
20240702 QFM021 Machine Intelligence Reading List June 2024
20240702 QFM021 Machine Intelligence Reading List June 202420240702 QFM021 Machine Intelligence Reading List June 2024
20240702 QFM021 Machine Intelligence Reading List June 2024
Matthew Sinclair
 
7 Most Powerful Solar Storms in the History of Earth.pdf
7 Most Powerful Solar Storms in the History of Earth.pdf7 Most Powerful Solar Storms in the History of Earth.pdf
7 Most Powerful Solar Storms in the History of Earth.pdf
Enterprise Wired
 
DealBook of Ukraine: 2024 edition
DealBook of Ukraine: 2024 editionDealBook of Ukraine: 2024 edition
DealBook of Ukraine: 2024 edition
Yevgen Sysoyev
 
Quantum Communications Q&A with Gemini LLM
Quantum Communications Q&A with Gemini LLMQuantum Communications Q&A with Gemini LLM
Quantum Communications Q&A with Gemini LLM
Vijayananda Mohire
 
Best Programming Language for Civil Engineers
Best Programming Language for Civil EngineersBest Programming Language for Civil Engineers
Best Programming Language for Civil Engineers
Awais Yaseen
 
INDIAN AIR FORCE FIGHTER PLANES LIST.pdf
INDIAN AIR FORCE FIGHTER PLANES LIST.pdfINDIAN AIR FORCE FIGHTER PLANES LIST.pdf
INDIAN AIR FORCE FIGHTER PLANES LIST.pdf
jackson110191
 
find out more about the role of autonomous vehicles in facing global challenges
find out more about the role of autonomous vehicles in facing global challengesfind out more about the role of autonomous vehicles in facing global challenges
find out more about the role of autonomous vehicles in facing global challenges
huseindihon
 
What’s New in Teams Calling, Meetings and Devices May 2024
What’s New in Teams Calling, Meetings and Devices May 2024What’s New in Teams Calling, Meetings and Devices May 2024
What’s New in Teams Calling, Meetings and Devices May 2024
Stephanie Beckett
 
20240705 QFM024 Irresponsible AI Reading List June 2024
20240705 QFM024 Irresponsible AI Reading List June 202420240705 QFM024 Irresponsible AI Reading List June 2024
20240705 QFM024 Irresponsible AI Reading List June 2024
Matthew Sinclair
 
Understanding Insider Security Threats: Types, Examples, Effects, and Mitigat...
Understanding Insider Security Threats: Types, Examples, Effects, and Mitigat...Understanding Insider Security Threats: Types, Examples, Effects, and Mitigat...
Understanding Insider Security Threats: Types, Examples, Effects, and Mitigat...
Bert Blevins
 
Manual | Product | Research Presentation
Manual | Product | Research PresentationManual | Product | Research Presentation
Manual | Product | Research Presentation
welrejdoall
 
Research Directions for Cross Reality Interfaces
Research Directions for Cross Reality InterfacesResearch Directions for Cross Reality Interfaces
Research Directions for Cross Reality Interfaces
Mark Billinghurst
 
Comparison Table of DiskWarrior Alternatives.pdf
Comparison Table of DiskWarrior Alternatives.pdfComparison Table of DiskWarrior Alternatives.pdf
Comparison Table of DiskWarrior Alternatives.pdf
Andrey Yasko
 
How to Build a Profitable IoT Product.pptx
How to Build a Profitable IoT Product.pptxHow to Build a Profitable IoT Product.pptx
How to Build a Profitable IoT Product.pptx
Adam Dunkels
 
Active Inference is a veryyyyyyyyyyyyyyyyyyyyyyyy
Active Inference is a veryyyyyyyyyyyyyyyyyyyyyyyyActive Inference is a veryyyyyyyyyyyyyyyyyyyyyyyy
Active Inference is a veryyyyyyyyyyyyyyyyyyyyyyyy
RaminGhanbari2
 
Paradigm Shifts in User Modeling: A Journey from Historical Foundations to Em...
Paradigm Shifts in User Modeling: A Journey from Historical Foundations to Em...Paradigm Shifts in User Modeling: A Journey from Historical Foundations to Em...
Paradigm Shifts in User Modeling: A Journey from Historical Foundations to Em...
Erasmo Purificato
 

Recently uploaded (20)

BLOCKCHAIN FOR DUMMIES: GUIDEBOOK FOR ALL
BLOCKCHAIN FOR DUMMIES: GUIDEBOOK FOR ALLBLOCKCHAIN FOR DUMMIES: GUIDEBOOK FOR ALL
BLOCKCHAIN FOR DUMMIES: GUIDEBOOK FOR ALL
 
WPRiders Company Presentation Slide Deck
WPRiders Company Presentation Slide DeckWPRiders Company Presentation Slide Deck
WPRiders Company Presentation Slide Deck
 
Choose our Linux Web Hosting for a seamless and successful online presence
Choose our Linux Web Hosting for a seamless and successful online presenceChoose our Linux Web Hosting for a seamless and successful online presence
Choose our Linux Web Hosting for a seamless and successful online presence
 
What's New in Copilot for Microsoft365 May 2024.pptx
What's New in Copilot for Microsoft365 May 2024.pptxWhat's New in Copilot for Microsoft365 May 2024.pptx
What's New in Copilot for Microsoft365 May 2024.pptx
 
20240702 QFM021 Machine Intelligence Reading List June 2024
20240702 QFM021 Machine Intelligence Reading List June 202420240702 QFM021 Machine Intelligence Reading List June 2024
20240702 QFM021 Machine Intelligence Reading List June 2024
 
7 Most Powerful Solar Storms in the History of Earth.pdf
7 Most Powerful Solar Storms in the History of Earth.pdf7 Most Powerful Solar Storms in the History of Earth.pdf
7 Most Powerful Solar Storms in the History of Earth.pdf
 
DealBook of Ukraine: 2024 edition
DealBook of Ukraine: 2024 editionDealBook of Ukraine: 2024 edition
DealBook of Ukraine: 2024 edition
 
Quantum Communications Q&A with Gemini LLM
Quantum Communications Q&A with Gemini LLMQuantum Communications Q&A with Gemini LLM
Quantum Communications Q&A with Gemini LLM
 
Best Programming Language for Civil Engineers
Best Programming Language for Civil EngineersBest Programming Language for Civil Engineers
Best Programming Language for Civil Engineers
 
INDIAN AIR FORCE FIGHTER PLANES LIST.pdf
INDIAN AIR FORCE FIGHTER PLANES LIST.pdfINDIAN AIR FORCE FIGHTER PLANES LIST.pdf
INDIAN AIR FORCE FIGHTER PLANES LIST.pdf
 
find out more about the role of autonomous vehicles in facing global challenges
find out more about the role of autonomous vehicles in facing global challengesfind out more about the role of autonomous vehicles in facing global challenges
find out more about the role of autonomous vehicles in facing global challenges
 
What’s New in Teams Calling, Meetings and Devices May 2024
What’s New in Teams Calling, Meetings and Devices May 2024What’s New in Teams Calling, Meetings and Devices May 2024
What’s New in Teams Calling, Meetings and Devices May 2024
 
20240705 QFM024 Irresponsible AI Reading List June 2024
20240705 QFM024 Irresponsible AI Reading List June 202420240705 QFM024 Irresponsible AI Reading List June 2024
20240705 QFM024 Irresponsible AI Reading List June 2024
 
Understanding Insider Security Threats: Types, Examples, Effects, and Mitigat...
Understanding Insider Security Threats: Types, Examples, Effects, and Mitigat...Understanding Insider Security Threats: Types, Examples, Effects, and Mitigat...
Understanding Insider Security Threats: Types, Examples, Effects, and Mitigat...
 
Manual | Product | Research Presentation
Manual | Product | Research PresentationManual | Product | Research Presentation
Manual | Product | Research Presentation
 
Research Directions for Cross Reality Interfaces
Research Directions for Cross Reality InterfacesResearch Directions for Cross Reality Interfaces
Research Directions for Cross Reality Interfaces
 
Comparison Table of DiskWarrior Alternatives.pdf
Comparison Table of DiskWarrior Alternatives.pdfComparison Table of DiskWarrior Alternatives.pdf
Comparison Table of DiskWarrior Alternatives.pdf
 
How to Build a Profitable IoT Product.pptx
How to Build a Profitable IoT Product.pptxHow to Build a Profitable IoT Product.pptx
How to Build a Profitable IoT Product.pptx
 
Active Inference is a veryyyyyyyyyyyyyyyyyyyyyyyy
Active Inference is a veryyyyyyyyyyyyyyyyyyyyyyyyActive Inference is a veryyyyyyyyyyyyyyyyyyyyyyyy
Active Inference is a veryyyyyyyyyyyyyyyyyyyyyyyy
 
Paradigm Shifts in User Modeling: A Journey from Historical Foundations to Em...
Paradigm Shifts in User Modeling: A Journey from Historical Foundations to Em...Paradigm Shifts in User Modeling: A Journey from Historical Foundations to Em...
Paradigm Shifts in User Modeling: A Journey from Historical Foundations to Em...
 

PyGotham 2014 Introduction to Profiling

  • 2. “We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%. A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code; but only after that code has been identified.” –Donald Knuth
  • 3. “Bottlenecks occur in surprising places, so don't try to second guess and put in a speed hack until you have proven that's where the bottleneck is.” –Rob Pike
  • 4. What will a profiler tell us? ❖ Function execution time! ❖ Memory usage, etc. are possible, but for another day! ❖ More about line profiling later! ❖ Real (wall clock) time! ❖ Inclusive vs exclusive time! ❖ Number of calls, primitive and recursive
  • 5. cProfile ❖ Generates profile data that can be read in shell or GUI tools! ❖ 30% or more speed penalty
  • 6. cProfile From command line:! $ python -m cProfile -o myscript.prof myscript.py
  • 7. cProfile Or, in your program:! import cProfile cProfile.run('slow_function', 'myscript.prof')
  • 8. cProfile Or, even more flexible:! pr = cProfile.Profile() pr.enable() … thing you want to profile …! pr.disable()
  • 9. pstats import pstats profile = pstats.Stats('myscript.prof') profile.add('myscript.prof2') profile.strip_dirs() profile.sort_stats('cumulative') profile.print_stats(20)
  • 10. 12192418 function calls (11990470 primitive calls) in 84.268 seconds ! Ordered by: cumulative time List reduced from 1211 to 20 due to restriction <20> ! ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 84.402 84.402 <string>:1(<module>) 1 0.021 0.021 84.402 84.402 act_bench.py:243(_do_act) 500 0.096 0.000 84.381 0.169 __init__.py:170(act) 500 0.007 0.000 35.874 0.072 petition_actions.py:460(save) 500 0.066 0.000 33.431 0.067 action_processor.py:1303(save) 500 0.160 0.000 22.684 0.045 users.py:1002(save) 10501 0.175 0.000 21.963 0.002 query.py:852(_fetch_all) 14001 0.286 0.000 21.472 0.002 compiler.py:758(execute_sql) 6501 0.047 0.000 14.200 0.002 query.py:76(__len__)
  • 11. profile.print_callees('full_clean', 10) ! List reduced from 1211 to 2 due to restriction <'full_clean'> ! Function called... ncalls tottime cumtime forms.py:260(full_clean) -> 500 0.177 2.855 forms.py: 277(_clean_fields) 500 0.003 0.030 forms.py:298(_clean_form) 500 0.031 2.784 models.py: 393(_post_clean) base.py:918(full_clean) -> 500 0.001 0.001 base.py:738(clean) 500 0.096 2.399 base.py:952(clean_fields)
  • 12. profile.print_callers('full_clean') ! List reduced from 1211 to 2 due to restriction <'full_clean'> ! Function was called by... ncalls tottime cumtime forms.py:260(full_clean) <- 500 0.009 5.678 forms.py:117(errors) base.py:918(full_clean) <- 500 0.005 2.405 models.py: 393(_post_clean)
  • 13. KCacheGrind ! ❖ GUI for viewing profile data! ❖ Run your profile output through pyprof2calltree! ❖ On a Mac, qcachegrind is easier to install
  • 16. RunSnakeRun ❖ Squaremap of call tree! ❖ Maybe useful for spotting large exclusive time functions
  • 19. Using your results ❖ Bottom up approach! ❖ Start with a large exclusive time sub! ❖ Climb up call graph to find something you can affect! ❖ "We're spending a lot of time in deepcopy(). What's calling that so much?"! ❖ Might miss higher-level fixes
  • 20. Using your results ❖ Top down approach! ❖ Start with a large inclusive time sub! ❖ Walk down call graph to find something you can affect! ❖ "We're spending a lot of time in this validate() method. What's it doing that takes so long?"! ❖ Look for structural changes
  • 21. Line profiling ❖ line_profiler does exist! ❖ Results are not very actionable! ❖ If you get this far, you probably should stop (or refactor your methods!)
  • 22. Good profiling technique ❖ Create a repeatable benchmark test! ❖ Allows you to measure progress! ❖ Iterations/second! ❖ Time for n iterations
  • 23. What usually helps ❖ Removing unnecessary work! ❖ “We load that config data every time, even when we don’t use it.”! ❖ Using a more efficient algorithm
  • 24. What usually helps ❖ Batching I/O (disk or net) operations! ❖ Database stuff! ❖ SQL tuning! ❖ Indexes! ❖ Transactions
  • 25. What usually helps ❖ Caching! ❖ Easy to add, hard to live with! ❖ Code complexity! ❖ Invalidation calls! ❖ Dependency tracking! ❖ Business customers care about data freshness