Information about how much time in spent in a function, based on the input of this function [closed]

Question

Is there a (quantitative) tool to measure performance of functions based on its input?

So far, the tools I used to measure performance of my code, tells me how much time I spent in functions (like Jetbrain Dottrace for .Net), but I'd like to have more information about the parameters passed to the function in order to know which parameters impact the most the performance.

Let's say that I have function like that:

int myFunction(int myParam1, int myParam 2) {
  // Do and return something based on the value of myParam1 and myParam2. 
  // The code is likely to use if, for, while, switch, etc....
}

If would like a tool that would allow me to tell me how much time is spent in myFunction based on the value of myParam1 and myParam2.

For example, the tool would give me a result looking like this:

For "myFunction" :
 value   |  value   | Number of | Average
myParam1 | myParam2 |   call    | time
---------|----------|-----------|--------
   1     |    5     |   500     | 301 ms
   2     |    5     |   250     | 1253 ms
   3     |    7     |   1268    | 538 ms
...

That would mean that myFunction has been call 500 times with myParam1=1 and myParam2=5, and that with those parameters, it took on average 301ms to return a value.

The idea behind that is to do some statistical optimization by organizing my code such that, the blocs of codes that are the most likely to be executed are tested before the one that are less likely to be executed.

To put it bluntly, if I know which values are used the most, I can reorganize the if/while/for etc.. structure of the function (and the whole program) to optimize it.

I'd like to find such tools for C++, Java or.Net.

Note: I am not looking for technical tips to optimize the code (like passing parameters as const, inlining functions, initializing the capacity of vectors and the like).

Answer 1

Your unit tests should cover this. Wrap the calls in a stopwatch and you're good to go.

I'll assume that you don't have any then, so here's one way to get quickly up to speed and get some.

Write a parameterized unit test around your function using Pex and then run that/those tests. Have ANTS running when you do - or just set a stopwatch before/after the call to the function.

Essentially, if I remember correctly, Pex uses a theorem prover to reach every branch and potential exception/boundary condition using the parameters to the function. Check the parameters it comes up with, critically think about some of your own and you should have what you're looking for.

Answer 2

In C++, if this is for just a particular method with particular inputs, it's pretty easy to write a helper class that will time how long it takes to execute a particular scope. You grab the time in the constructor and in the destructor you calculate the current time and save it in a static data structure.

This is a technique I've used with embedded devices for which I had no profiler.

Beware: I haven't compiled/tested this:

typedef std::map<std::pair<int,int>, std::pair<int,int> > timeMap;

class TimeIt {
private:
    static timeMap times;
    clock_t timeCalled;
    int myParm1;
    int myParm2;

public:
    TimeIt(int parm1, int parm2) : timeCalled(clock()), myParm1(parm1), myParm2(parm2) {}

    ~TimeIt() {
        clock_t runtime = clock() - timeCalled;
        if(timeMap::iterator it = times.find(std::pair<int,int>(myParm1,myParm2))) {
            it->second->first++;
            it->second->second+=runtime;
        }
        else {
            times[std::pair<int,int>(myParm1,myParm2)] = std::pair<int,int>(1,runtime);
        }
    }

    static void report() {
        for(timeMap::iterator it=times.begin();it!=times.end();it++) {
             cout << it->first->first << " | ";
             cout << it->first->second << " | ";
             cout << it->second->first << " | ";
             cout << (it->second->second/it->second->first) << endl;
        }
    }
}

Then in your method, do:

int myFunction(int myParam1, int myParam 2) {

    TimeIt timeIt;

    // Do and return something based on the value of myParam1 and myParam2. 
    // The code is likely to use if, for, while, switch, etc.... 
}

And when you want the data:

TimeIt::report();

I'm sure you can do something similar in .NET or Java. The most important bit is to take care when you grab the clock() time. You don't want to do anything of significance in the destructor before grabbing the clock() time.

Obviously if you want a more general solution, you want a real profiler.

Answer 3

.NET-only answer

To the best of my knowledge, there's no way you can retrieve the actual value of the parameters passed to a method.

Sure, you can get an array of ParameterInfo objects, through which you can get to know a lot about the parameters... but sadly you can't access their values. Granted, there are some ways to investigate the stack trace, but they are not guaranteed to be robust enough under all conditions.

So, no, it appears that such a tool can't even be developed for .NET.

I don't know about other platforms, though.

Answer 4

Although such a tool could be very helpful, I'm not aware of any that offer line-based hotspot analysis. Since profilers are tricky to hack and easy to get wrong, you are probably certainly better off temporarily splitting your code into different functions just so a normal profiler can tell you what you need to know.

Answer 5

I doubt you'll find such tool, therefore most likely you'll have to implement your own.

However, there are code profilers (for example callgrind to profile c and c++ code), which provides you with some value which represents a relative time spent in a function.

Answer 6

If you want to profile just one function, and you know the small number of input combinations you are interested in, you can add some kind of "switch mechanism" which will make the calls visible in any typical profiler:

int myFunction_internal(int myParam1, int myParam 2) {
  // the main body
}

int myFunction_p1_p5() {
    return myFunction_internal(1,5);
}

int myFunction_p2_p5() {
    return myFunction_internal(2,5);
}

int myFunction_other(int myParam1, int myParam2) {
    return myFunction_internal(myParam1,myParam2);
}

int myFunction(int myParam1, int myParam2) {
   if(myParam1==1 && myParam2==5)
        return myFunction_p1_p5();
   if(myParam1==2 && myParam2==5)
        return myFunction_p2_p5();
   return myFunction_other(myParam1,myParam2);
}