Performance checking tools

Question

Performance checkers: What tools are available for attaining the metrics necessary to perform a performance review? Bench-marking is often not easily done by hand after all.

This is a follow-up to this question about formatting and error checking tools.

Tools listed here should help getting metrics for questions with performance (info). Speed and time-complexity, memory usage, hang-ups, etc.

Preferably people should run these tools before posting their code on Code Review. However, since not everyone will follow this advice, feel free to use these tools in the reviewing process if required.

Post at least the name of the tool, the official website and the languages targeted by the tool. Free tools are preferred, make sure to add a note if your tool is paid/commercial.

Answer 1

Java

• jmh

  JMH is a Java harness for building, running, and analysing nano/micro/milli/macro benchmarks written in Java and other languages targetting the JVM.

• MicroBench

  Simpler microbenchmarks of Java8 code

• caliper

  Caliper is Google's open-source framework for writing, running and viewing the results of JavaMicrobenchmarks.

Answer 2

Linux/Unix command line

The time command will give you a rough idea of execution time. See this SO answer for details and caveats.

Answer 3

Python

timeit — built-in module for timing the execution of small bits of Python code. Start here! Many performance problems can be tackled using timeit and a willingness to experiment.

profile — built-in module for function-based deterministic profiling. Use this when your code is organized into functions and you're not sure which one is taking all the time.

line_profiler — package for line-by-line profiling. Try this if your code is not organized into functions, or if you tried profile and there was no clear culprit function.

benchmark — framework for writing benchmarks: like unittest but for performance tests rather than functionality tests.

Answer 4

Any running process on OS X can be tested using the Instruments application which comes with Xcode.

This is not limited to code that is written (or copy & pasted into) Xcode and compiled, built, run, etc., with Xcode. If it is a running process in OS X, Instruments can attach to it an analyze a ton of different metrics.

Answer 5

Browsers

All the major, modern browsers come with a suite of developer tools, including profilers:

• Chrome DevTools
• Safari Developer Tools
• Internet Explorer Developer Tools (aka F12 Tools)
• Firefox Developer Tools
• Opera Developer Tools (aka Dragonfly)

Answer 6

Ruby

Included in Ruby's stdlib:

• The Benchmark module let's you time the execution of any block of code.

• The Profiler module which gives a more detailed report for scripts, with execution time broken down by method calls.

Gems:

• ruby-prof is a pretty feature-rich profiler that can also benchmark memory usage and garbage collection.